trade-next/trade-docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(plan): record canary dlob all-path rollout

Browse Source
This commit is contained in:
mpabi
2026-04-12 18:19:59 +02:00
parent 4fdde9e86f
commit 138a4bd953
1 changed files with 6 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
doc/migration-todo.md
View File

@@ -350,7 +350,7 @@ Date: `2026-04-12`
- In progress. - In progress.
- Substep A is now in place on `sol`: deploy `trade-ingestor` first in a schema-compatible canary mode and verify its secret wiring, rollout behavior, and Hasura connectivity. - Substep A is now in place on `sol`: deploy `trade-ingestor` first in a schema-compatible canary mode and verify its secret wiring, rollout behavior, and Hasura connectivity.
- Substep B is partially complete on `sol`: reconstruct the live `R001` DLOB hot-path writer chain and the `dlob_hot_*` tables first, then switch the canary ingestor from `dlob_stats_latest` to the exact live derived query path only after the hot-path is stable. - Substep B is now functionally in place on `sol`: the live `R001` DLOB hot-path and all-path writer chains and the `dlob_hot_*` plus `dlob_all_*` tables are reconstructed first, and the remaining step is to switch the canary ingestor from `dlob_stats_latest` to the exact live derived query path.
- Done on `2026-04-12`: extended `trade-next/trade-gitops` canary environment with `trade-ingestor`, synced `trade-ingestor-tokens`, and added `trade-ingestor` rollout checks to the `deploy-trade-r001-canary` workflow. - Done on `2026-04-12`: extended `trade-next/trade-gitops` canary environment with `trade-ingestor`, synced `trade-ingestor-tokens`, and added `trade-ingestor` rollout checks to the `deploy-trade-r001-canary` workflow.
- Done on `2026-04-12`: manually validated `trade-ingestor` on `sol`; deployment reached `Available`, restart count stayed at `0`, and startup logs confirmed `HASURA_GRAPHQL_URL=http://hasura:8080/v1/graphql`, markets `SOL-PERP,PUMP-PERP`, and mode `dlob_stats_ticks`. - Done on `2026-04-12`: manually validated `trade-ingestor` on `sol`; deployment reached `Available`, restart count stayed at `0`, and startup logs confirmed `HASURA_GRAPHQL_URL=http://hasura:8080/v1/graphql`, markets `SOL-PERP,PUMP-PERP`, and mode `dlob_stats_ticks`.
- Done on `2026-04-12`: after push of commit `948c37c`, the GitOps path on `sol` rolled a fresh `trade-ingestor` pod and it remained healthy with restart count `0`. - Done on `2026-04-12`: after push of commit `948c37c`, the GitOps path on `sol` rolled a fresh `trade-ingestor` pod and it remained healthy with restart count `0`.
@@ -360,7 +360,10 @@ Date: `2026-04-12`
- Done on `2026-04-12`: added `dlob-redis`, `dlob-publisher-hot`, `dlob-hot-redis-to-postgres-raw-writer`, and `dlob-hot-postgres-to-postgres-derived-writer` to `trade-r001-canary` in GitOps. - Done on `2026-04-12`: added `dlob-redis`, `dlob-publisher-hot`, `dlob-hot-redis-to-postgres-raw-writer`, and `dlob-hot-postgres-to-postgres-derived-writer` to `trade-r001-canary` in GitOps.
- Done on `2026-04-12`: reran `postgres-migrate` and `hasura-bootstrap` with additive `dlob_hot_*` schema and Hasura metadata on `sol`. - Done on `2026-04-12`: reran `postgres-migrate` and `hasura-bootstrap` with additive `dlob_hot_*` schema and Hasura metadata on `sol`.
- Done on `2026-04-12`: manually validated the DLOB hot-path on `sol`; the raw writer persisted `dlob_hot_snapshot_*`, the derived writer persisted `dlob_hot_derived_*`, and `dlob_hot_derived_latest` contained current rows for markets `0`, `20`, and `72`. - Done on `2026-04-12`: manually validated the DLOB hot-path on `sol`; the raw writer persisted `dlob_hot_snapshot_*`, the derived writer persisted `dlob_hot_derived_*`, and `dlob_hot_derived_latest` contained current rows for markets `0`, `20`, and `72`.
- Remaining on `2026-04-12`: bring in the live `dlob-publisher-all` and `dlob_all_*` chain under GitOps, then switch `trade-ingestor` from `dlob_stats_ticks` to the exact live derived query path and validate API and UI behavior against the new tables. - Done on `2026-04-12`: added `dlob-publisher-all` and `dlob-all-redis-to-postgres-derived-writer` to `trade-r001-canary` in GitOps and validated that both deployments reached `Available` on `sol`.
- Done on `2026-04-12`: tuned the new `dlob-publisher-all` and `dlob-all-redis-to-postgres-derived-writer` manifests to fit the namespace guardrails by setting `limits.cpu=500m` and `RollingUpdate` with `maxSurge=0`, `maxUnavailable=1`, which removed the `ResourceQuota` rollout deadlock at `limits.cpu=6`.
- Done on `2026-04-12`: manually validated the DLOB all-path on `sol`; `dlob_all_derived_latest` and `dlob_all_derived_ts` were populated and the new publisher and writer stayed healthy after rollout.
- Remaining on `2026-04-12`: switch `trade-ingestor` from `dlob_stats_ticks` to the exact live derived query path and validate API and UI behavior against `dlob_hot_derived_latest` plus `dlob_all_derived_latest`.
### 23. Validate host health, agave health, trade health, and end-to-end flow after rollout ### 23. Validate host health, agave health, trade health, and end-to-end flow after rollout
@@ -376,7 +379,7 @@ Date: `2026-04-12`
- [x] 01. Define hard database and cache constraints. - [x] 01. Define hard database and cache constraints.
- [ ] 02. Reconstruct the current live `mevnode_bot` and `trade-staging` state into canonical Git/IaC in a new Gitea organization. - [ ] 02. Reconstruct the current live `mevnode_bot` and `trade-staging` state into canonical Git/IaC in a new Gitea organization.
- [x] 03. Fix `dlob-publisher-hot` startup, readiness, and liveness. - [x] 03. Fix `dlob-publisher-hot` startup, readiness, and liveness.
- [ ] 04. Fix `dlob-publisher-all` startup, readiness, and liveness. - [x] 04. Fix `dlob-publisher-all` startup, readiness, and liveness.
- [ ] 05. Confirm whether trade Redis is meant to run as a single instance or as Redis Cluster. - [ ] 05. Confirm whether trade Redis is meant to run as a single instance or as Redis Cluster.
- [ ] 06. Align publisher Redis client mode with the actual deployed Redis topology. - [ ] 06. Align publisher Redis client mode with the actual deployed Redis topology.
- [ ] 07. Restore or recreate the missing `gitea-registry` image pull secret. - [ ] 07. Restore or recreate the missing `gitea-registry` image pull secret.