How this repo was built
=======================

A reproducible record of how the repository, the environment and these docs were set
up, from an empty folder to a published private GitHub repository. Commands are for
Windows PowerShell.

Prerequisites
-------------

- Python 3.10 or newer and Git.
- `Docker Desktop <https://www.docker.com/products/docker-desktop/>`_ with the WSL 2
  backend, for Redis and optionally PostgreSQL (see :doc:`/database/setup`).
- The GitHub CLI, installed and authenticated as below.

Naming conventions
------------------

- **Repository names use kebab-case**, for example ``trade-engine``, not
  ``trade_engine``. This matches the distribution name in ``pyproject.toml``.
- **reStructuredText file names use snake_case**, for example ``multi_asset.rst``, not
  ``multi-asset.rst``. The name in a ``toctree`` or a ``:doc:`` reference is the same
  name without the extension.
- Python packages and modules stay snake_case (``src/trade_engine/``) because a hyphen
  is not valid in an import name.

Create the GitHub repository with the CLI
-----------------------------------------

Install the GitHub CLI and sign in:

.. code:: powershell

   winget install GitHub.cli
   gh auth login

Restart the terminal after the install so ``gh`` is on ``PATH``, and choose
GitHub.com, HTTPS and a browser login when ``gh auth login`` asks. Check it with
``gh auth status``.

Create a new, empty private repository on GitHub:

.. code:: powershell

   gh repo create my-new-repo --private

To publish an existing local folder (this project) instead, run it from the project
root. ``--source .`` uses the current folder, ``--remote origin`` names the remote,
and ``--push`` pushes the current branch:

.. code:: powershell

   git init -b main
   git add .
   git commit -m "Initial commit"
   gh repo create trade-engine --private --source . --remote origin --push

Check what is staged before the first commit. ``.gitignore`` already excludes
``.venv/``, ``.idea/``, ``.pytest_cache/``, ``__pycache__/``, ``*.egg-info/``, ``.env``,
``db/*.db``, ``db/*.duckdb``, ``data/generated/`` and ``docs/_build/``, so virtual
environments, IDE settings, database files, generated data and built docs stay out of
the repository. Never commit passwords or connection strings; they
come from environment variables (see :doc:`/database/setup`).

Afterwards, the usual loop is:

.. code:: powershell

   git status
   git add -p
   git commit -m "Describe the change"
   git push

Published repository
--------------------

This project is published, privately, at
https://github.com/samkhalilian/trade-engine. The repository was created empty on
GitHub first, so the local folder was connected to it instead of using
``gh repo create --source``. From the project root:

.. code:: powershell

   git init -b main
   git remote add origin https://github.com/samkhalilian/trade-engine.git
   git add .
   git status                       # review what is staged
   git commit -m "Initial commit"
   git push -u origin main

Notes:

- Check that the remote repository is empty before the first push. A first push to a
  repository that already has commits is rejected, and should be resolved by
  reviewing those commits, not by forcing the push.
- ``gh`` was installed with ``winget`` after the terminal was opened, so it was not
  yet on ``PATH`` there. Open a new terminal, or call it by its full path
  (``"$env:ProgramFiles\GitHub CLI\gh.exe"``).
- Git uses the HTTPS credentials stored by ``gh auth login``, so no token is typed or
  stored in the project.
- To publish the built documentation, see :doc:`/appendix/build_docs` (Publishing the
  docs to GitHub Pages).

Project scaffold
----------------

1. Create the layout described in :doc:`/SPEC` (Project layout): installable code in
   ``src/trade_engine/``, batch jobs in ``scripts/``, tests in ``tests/``, raw input
   data in ``data/raw/``, and database files in ``db/``.
2. Declare the package and its dependencies in ``pyproject.toml``. Runtime
   dependencies are ``flask``, ``sqlalchemy``, ``pandas``, ``numpy``, ``scipy``,
   ``redis`` and ``duckdb``. Optional groups keep the rest out of a minimal install:

   .. list-table::
      :header-rows: 1

      * - Extra
        - Adds
      * - ``dev``
        - ``pytest`` and ``fakeredis`` (tests run without a Redis server)
      * - ``postgres``
        - ``psycopg`` driver for running the clean layer on PostgreSQL
      * - ``docs``
        - ``sphinx`` and ``sphinx-book-theme``

3. Create a virtual environment and install the package in editable mode with the
   extras:

   .. code:: powershell

      python -m venv .venv
      .\.venv\Scripts\Activate.ps1
      python -m pip install -e ".[dev,docs,postgres]"

   If PowerShell blocks the activation script, run
   ``Set-ExecutionPolicy -Scope Process -ExecutionPolicy RemoteSigned`` first; it
   applies to that terminal only.

Build order
-----------

The code was built in layers, each tested before the next relied on it:

1. ``domain/``: pure P&L, risk and market-data validation logic, unit-tested alone.
2. ``db/`` and ``models/``: one engine and declarative base per logical database.
3. ``repositories/`` and ``services/``: data access and use cases.
4. ``staging/`` and ``analytics/``: the Redis staging queue and the DuckDB export.
5. ``web/`` and ``scripts/``: the Flask API and the batch jobs, with no business
   rules of their own.

Run the tests at each step:

.. code:: powershell

   python -m pytest

Run the application
-------------------

.. code:: powershell

   docker compose -f docker-compose.redis.yml up -d
   python scripts/init_db.py
   python scripts/load_market_data.py --origin SIMULATED --source gbm-seed-42
   python scripts/seed_sample_trades.py
   python scripts/run_eod.py
   python scripts/run_app.py

See the README quick start and :doc:`/database/setup` for PostgreSQL, ``REDIS_URL``
and the other settings.

Build these docs
----------------

The documentation is a Sphinx project in ``docs/``; see
:doc:`/appendix/build_docs` for how it is built, viewed and edited.