Setup: configuration, PostgreSQL and Redis
==========================================

Connection settings, running the clean layer on PostgreSQL, and running Redis
(for example in Docker on Windows). For why the system is split this way see
:doc:`/database/architecture`.

Running on PostgreSQL and Redis
-------------------------------

``src/trade_engine/config.py`` resolves one connection string per logical database
(``Settings.from_env()``), each overridable via an environment variable:

.. list-table::
   :header-rows: 1

   * - Database
     - Environment variable
     - Default
   * - reference
     - ``REFERENCE_DB_URL``
     - ``sqlite:///<root>/db/reference.db``
   * - market
     - ``MARKET_DB_URL``
     - ``sqlite:///<root>/db/market.db``
   * - trade
     - ``TRADE_DB_URL``
     - ``sqlite:///<root>/db/trade.db``
   * - risk
     - ``RISK_DB_URL``
     - ``sqlite:///<root>/db/risk.db``

``TRADE_ENGINE_ROOT`` overrides ``<root>`` (used by tests to run in a temporary
folder). Two more settings cover the other tiers:

.. list-table::
   :header-rows: 1

   * - Setting
     - Environment variable
     - Default
   * - Staging Redis
     - ``REDIS_URL``
     - ``redis://localhost:6379/0``
   * - DuckDB analytics file
     - ``ANALYTICS_DB_PATH``
     - ``<root>/db/analytics.duckdb``

To run the clean layer on PostgreSQL:

1. Install the driver: ``pip install -e .[postgres]`` (``psycopg`` 3).
2. Start PostgreSQL. ``docker-compose.yml`` runs one server and creates the four
   databases (``reference_db``, ``market_db``, ``trade_db``, ``risk_db``) from
   ``docker/postgres/init-databases.sql``; set ``POSTGRES_PASSWORD`` in your shell
   first, then ``docker compose up -d``.
3. Point each database at it, using the ``postgresql+psycopg`` scheme, for example
   ``TRADE_DB_URL=postgresql+psycopg://trade_engine:<password>@localhost:5432/trade_db``.
   Set all four variables, or the unset ones stay on SQLite.
4. Run ``scripts/init_db.py``.

Redis runs separately from PostgreSQL; see `Running Redis in Docker on Windows`_.
When Redis is unreachable, staging fails with an error and nothing is written to
the clean databases.

No other code changes are needed, because all queries go through SQLAlchemy
sessions, not SQLite-specific SQL. After that point, schema changes must go
through migrations, not ``create_all``.

Running Redis in Docker on Windows
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Redis has no native Windows build, so run it in a container (or inside WSL; see
below). ``docker-compose.redis.yml`` starts one Redis 7 server with append-only
persistence.

**Prerequisites**

- Docker Desktop for Windows with the **WSL 2 backend** enabled (the default on
  current installs), and Docker Desktop running. ``docker --version`` and
  ``docker compose version`` must both work in PowerShell.

**Start, check, stop** (from the project root in PowerShell):

.. code:: powershell

   docker compose -f docker-compose.redis.yml up -d
   docker compose -f docker-compose.redis.yml exec redis redis-cli ping   # PONG
   docker compose -f docker-compose.redis.yml down                        # stop, keep data
   docker compose -f docker-compose.redis.yml down -v                     # stop and delete data

The container publishes port 6379 on ``127.0.0.1`` only, so the default
``REDIS_URL=redis://localhost:6379/0`` works from Windows with no configuration.
The server has no password and must not be exposed beyond localhost; before using
it on a shared host, add ``--requirepass`` (and put the password in ``REDIS_URL``,
not in source).

**Persistence.** Data lives in the ``redis_data`` Docker volume with append-only
logging, so staged batches survive a container restart. ``down -v`` deletes them.

**Inspecting the staging area:**

.. code:: powershell

   docker compose -f docker-compose.redis.yml exec redis redis-cli XLEN trade_engine:staging:market
   docker compose -f docker-compose.redis.yml exec redis redis-cli XLEN trade_engine:staging:market:rejected

The first is the number of batches waiting to be processed; the second holds
batches that failed validation, with the reason.

**Troubleshooting**

.. list-table::
   :header-rows: 1

   * - Symptom
     - Cause and fix
   * - ``Redis staging error: Error 10061 connecting to localhost:6379``
     - Redis is not running. Start Docker Desktop, then run the ``up -d`` command above.
   * - ``port is already allocated`` or ``address already in use`` on 6379
     - Another Redis (for example one running inside WSL) already uses the port. Stop it, or change the mapping to ``127.0.0.1:6380:6379`` in the compose file and set ``REDIS_URL=redis://localhost:6380/0``.
   * - ``docker`` is not recognized
     - Docker Desktop is not installed, or PowerShell was opened before it was added to ``PATH``; reopen the terminal.

**Alternative: Redis inside WSL.** Install Redis in a WSL distribution and start it
there (``sudo service redis-server start``). WSL 2 forwards ``localhost:6379`` to
Windows, so the same default ``REDIS_URL`` works. Use one or the other, not both,
because they share the port.