trade-next/trade-docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(plan): record sol dlob hot-path rollout

Browse Source
This commit is contained in:
mpabi
2026-04-12 18:10:52 +02:00
parent a1921b213d
commit 4fdde9e86f
1 changed files with 12 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
doc/migration-todo.md
View File

@@ -350,14 +350,22 @@ Date: `2026-04-12`
- In progress.
- Substep A is now in place on `sol`: deploy `trade-ingestor` first in a schema-compatible canary mode and verify its secret wiring, rollout behavior, and Hasura connectivity.
- Substep B remains pending: reconstruct the live `R001` DLOB derived writer chain and the `dlob_*_derived_latest` tables so the canary ingestor can switch from `dlob_stats_latest` to the exact current live query path.
- Substep B is partially complete on `sol`: reconstruct the live `R001` DLOB hot-path writer chain and the `dlob_hot_*` tables first, then switch the canary ingestor from `dlob_stats_latest` to the exact live derived query path only after the hot-path is stable.
- Done on `2026-04-12`: extended `trade-next/trade-gitops` canary environment with `trade-ingestor`, synced `trade-ingestor-tokens`, and added `trade-ingestor` rollout checks to the `deploy-trade-r001-canary` workflow.
- Done on `2026-04-12`: manually validated `trade-ingestor` on `sol`; deployment reached `Available`, restart count stayed at `0`, and startup logs confirmed `HASURA_GRAPHQL_URL=http://hasura:8080/v1/graphql`, markets `SOL-PERP,PUMP-PERP`, and mode `dlob_stats_ticks`.
- Done on `2026-04-12`: after push of commit `b8f1303`, the GitOps path on `sol` rolled a fresh `trade-ingestor` pod and it remained healthy with restart count `0`.
- Done on `2026-04-12`: after push of commit `948c37c`, the GitOps path on `sol` rolled a fresh `trade-ingestor` pod and it remained healthy with restart count `0`.
- Done on `2026-04-12`: added `environments/sol/trade-infra` to `trade-next/trade-gitops` with host-backed `postgres-host`, `redis-host`, `agave-rpc-host`, `agave-ws-host`, and `agave-grpc-host` services so canary workloads can consume the shared host services through Kubernetes DNS.
- Done on `2026-04-12`: installed host-side `systemd` socket proxies and `ufw` rules on `sol` so pods in `k3s` can reach Agave RPC on `8899`, WebSocket on `8900`, and Yellowstone gRPC on `10000`.
- Done on `2026-04-12`: created the `trade-dlob-rpc` secret on `sol` from the live host Geyser token and wired the canary namespace to the host-backed Agave endpoints without hardcoding the token in Git.
- Done on `2026-04-12`: added `dlob-redis`, `dlob-publisher-hot`, `dlob-hot-redis-to-postgres-raw-writer`, and `dlob-hot-postgres-to-postgres-derived-writer` to `trade-r001-canary` in GitOps.
- Done on `2026-04-12`: reran `postgres-migrate` and `hasura-bootstrap` with additive `dlob_hot_*` schema and Hasura metadata on `sol`.
- Done on `2026-04-12`: manually validated the DLOB hot-path on `sol`; the raw writer persisted `dlob_hot_snapshot_*`, the derived writer persisted `dlob_hot_derived_*`, and `dlob_hot_derived_latest` contained current rows for markets `0`, `20`, and `72`.
- Remaining on `2026-04-12`: bring in the live `dlob-publisher-all` and `dlob_all_*` chain under GitOps, then switch `trade-ingestor` from `dlob_stats_ticks` to the exact live derived query path and validate API and UI behavior against the new tables.
### 23. Validate host health, agave health, trade health, and end-to-end flow after rollout
- Pending.
- Initial validation on `2026-04-12`: after the DLOB hot-path rollout, sampled Agave slot lag on `sol` at `0` versus `https://api.mainnet-beta.solana.com`, with local health still reporting `ok`.
### 24. Keep `mevnode_bot` as a rollback target until the new stack is stable
@@ -365,9 +373,9 @@ Date: `2026-04-12`
## Chapter 3
- [ ] 01. Define hard database and cache constraints.
- [x] 01. Define hard database and cache constraints.
- [ ] 02. Reconstruct the current live `mevnode_bot` and `trade-staging` state into canonical Git/IaC in a new Gitea organization.
- [ ] 03. Fix `dlob-publisher-hot` startup, readiness, and liveness.
- [x] 03. Fix `dlob-publisher-hot` startup, readiness, and liveness.
- [ ] 04. Fix `dlob-publisher-all` startup, readiness, and liveness.
- [ ] 05. Confirm whether trade Redis is meant to run as a single instance or as Redis Cluster.
- [ ] 06. Align publisher Redis client mode with the actual deployed Redis topology.