How this repo was built

How this repo was built#

A reproducible record of how the repository, the environment and these docs were set up, from an empty folder to a published private GitHub repository. Commands are for Windows PowerShell.

Prerequisites#

Naming conventions#

  • Repository names use kebab-case, for example trade-engine, not trade_engine. This matches the distribution name in pyproject.toml.

  • reStructuredText file names use snake_case, for example multi_asset.rst, not multi-asset.rst. The name in a toctree or a :doc: reference is the same name without the extension.

  • Python packages and modules stay snake_case (src/trade_engine/) because a hyphen is not valid in an import name.

Create the GitHub repository with the CLI#

Install the GitHub CLI and sign in:

winget install GitHub.cli
gh auth login

Restart the terminal after the install so gh is on PATH, and choose GitHub.com, HTTPS and a browser login when gh auth login asks. Check it with gh auth status.

Create a new, empty private repository on GitHub:

gh repo create my-new-repo --private

To publish an existing local folder (this project) instead, run it from the project root. --source . uses the current folder, --remote origin names the remote, and --push pushes the current branch:

git init -b main
git add .
git commit -m "Initial commit"
gh repo create trade-engine --private --source . --remote origin --push

Check what is staged before the first commit. .gitignore already excludes .venv/, .idea/, .pytest_cache/, __pycache__/, *.egg-info/, .env, db/*.db, db/*.duckdb, data/generated/ and docs/_build/, so virtual environments, IDE settings, database files, generated data and built docs stay out of the repository. Never commit passwords or connection strings; they come from environment variables (see Setup: configuration, PostgreSQL and Redis).

Afterwards, the usual loop is:

git status
git add -p
git commit -m "Describe the change"
git push

Published repository#

This project is published, privately, at samkhalilian/trade-engine. The repository was created empty on GitHub first, so the local folder was connected to it instead of using gh repo create --source. From the project root:

git init -b main
git remote add origin https://github.com/samkhalilian/trade-engine.git
git add .
git status                       # review what is staged
git commit -m "Initial commit"
git push -u origin main

Notes:

  • Check that the remote repository is empty before the first push. A first push to a repository that already has commits is rejected, and should be resolved by reviewing those commits, not by forcing the push.

  • gh was installed with winget after the terminal was opened, so it was not yet on PATH there. Open a new terminal, or call it by its full path ("$env:ProgramFiles\GitHub CLI\gh.exe").

  • Git uses the HTTPS credentials stored by gh auth login, so no token is typed or stored in the project.

  • To publish the built documentation, see How to build these docs (Publishing the docs to GitHub Pages).

Project scaffold#

  1. Create the layout described in Trade Engine — v1 Spec (Project layout): installable code in src/trade_engine/, batch jobs in scripts/, tests in tests/, raw input data in data/raw/, and database files in db/.

  2. Declare the package and its dependencies in pyproject.toml. Runtime dependencies are flask, sqlalchemy, pandas, numpy, scipy, redis and duckdb. Optional groups keep the rest out of a minimal install:

    Extra

    Adds

    dev

    pytest and fakeredis (tests run without a Redis server)

    postgres

    psycopg driver for running the clean layer on PostgreSQL

    docs

    sphinx and sphinx-book-theme

  3. Create a virtual environment and install the package in editable mode with the extras:

    python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -e ".[dev,docs,postgres]"

    If PowerShell blocks the activation script, run Set-ExecutionPolicy -Scope Process -ExecutionPolicy RemoteSigned first; it applies to that terminal only.

Build order#

The code was built in layers, each tested before the next relied on it:

  1. domain/: pure P&L, risk and market-data validation logic, unit-tested alone.

  2. db/ and models/: one engine and declarative base per logical database.

  3. repositories/ and services/: data access and use cases.

  4. staging/ and analytics/: the Redis staging queue and the DuckDB export.

  5. web/ and scripts/: the Flask API and the batch jobs, with no business rules of their own.

Run the tests at each step:

python -m pytest

Run the application#

docker compose -f docker-compose.redis.yml up -d
python scripts/init_db.py
python scripts/load_market_data.py --origin SIMULATED --source gbm-seed-42
python scripts/seed_sample_trades.py
python scripts/run_eod.py
python scripts/run_app.py

See the README quick start and Setup: configuration, PostgreSQL and Redis for PostgreSQL, REDIS_URL and the other settings.

Build these docs#

The documentation is a Sphinx project in docs/; see How to build these docs for how it is built, viewed and edited.

Contents