feat: add playbook cohort remediation queue
This commit is contained in:
+11
-1
@@ -25,6 +25,13 @@ Automation execution-layer PXX remains closed and its known scope stays frozen.
|
||||
|
||||
- state: automation_execution_layer_pxx_known_scope_closed
|
||||
- current_phase: Long-running Analysis Decision Monitor With Ops Rollup
|
||||
- latest_analysis_decision_monitor_rollup_slice: automation_execution_layer_p7_monitor_maturity_rollup
|
||||
- latest_analysis_decision_monitor_rollup_status: blocked_attention_required_refreshed_2026-05-22T13:27:14Z
|
||||
- latest_analysis_decision_monitor_rollup_path: `.claude/analysis_decision_monitor_maturity_rollup.json`
|
||||
- latest_analysis_decision_monitor_rollup_next_slice: repair_analysis_decision_monitor_blockers
|
||||
- latest_analysis_decision_monitor_rollup_failure_reasons: latest_monitor_not_passed, stable_monitoring_window_not_reached, latest_monitor_has_blockers, runtime_health_not_ok, p6_governance_tick_not_noop
|
||||
- latest_analysis_decision_monitor_rollup_drift_finding: fresh_remote_drift_check_not_run_local_deploy_password_missing; stored_2026-05-11_drift_artifact_was_clear_but_is_now_historical_only
|
||||
- latest_analysis_decision_monitor_rollup_alert_quality_finding: required_review_fields_remain_present_but_quality_status_is_needs_attention_because_the_monitor_is_not_passed_and_has_blockers
|
||||
- active_product_phase: Decision Assistant Phase H/F/D/G/E Dense Recall + Templates + Announcements + Peer Baselines
|
||||
- latest_decision_assistant_slice: unified_cockpit_summary
|
||||
- latest_decision_assistant_change_slice: review_packet_delta_summary
|
||||
@@ -725,6 +732,7 @@ Automation execution-layer PXX remains closed and its known scope stays frozen.
|
||||
- P23 unattended harness remains a local/remote smoke verifier only and must not use external channels, require real trades, or alter execution permission, live readiness, trust bucket, risk governor, or operator confirmation.
|
||||
- P24 remote ops smoke remains verification only: its visual-watch step uses dry-run cleanup and it must not use external channels, require real trades, or alter execution permission, live readiness, trust bucket, risk governor, or operator confirmation.
|
||||
- PXX closure gate is verification/state-freeze only and must not create a Research Council phase, alter execution permission, enable real-live readiness, change risk governor/trust bucket behavior, or weaken operator confirmation.
|
||||
- The latest P7 monitor maturity rollup refresh is a `System Ops` status artifact only; it must not be promoted back into trader-facing doctrine work while monitor blockers remain active.
|
||||
- A final rerun of `scripts/run_alert_dispatcher.py` and `scripts/run_trading_runtime.py` hit local PostgreSQL connection timeout because nothing was listening on `127.0.0.1:5432`; the P14 code path is still covered by full unit tests and earlier successful smoke during implementation.
|
||||
- P15 dispatcher smoke succeeded with `suppressed_noise_policy`, `workbench_only`, and `policy_external_allowed` counters present. P15 runtime smoke reached `alert_dispatch` successfully, but the overall runtime returned external environment blockers for market stream disconnected and OKX read transport connection refused.
|
||||
- P16 direct browser smoke succeeded locally with `browser_capture_status=captured`. Script-level capture without a watch run still returns safe no-op `watch_run_missing`.
|
||||
@@ -852,10 +860,12 @@ Automation execution-layer PXX remains closed and its known scope stays frozen.
|
||||
- D5 Time / Session Kernel is complete, deployed, and remote-verified: added `DOCTRINE_SESSION_PHASE=D5_TIME_SESSION_KERNEL`, `TimeSessionRule`, `GET /trading/time-session-kernel`, Strategy Backtest `time_session_backtest_features` / `time_session_kernel_status` / `time_session_kernel_summary`, and `docs/AI_ICT_DoctrineKernel_TimeSessionKernel_D5_2026-05-22_v1.md`. Asia / London / NY AM / NY PM, killzone, Judas/manipulation window, opening range, weekly profile, and news/no-trade time are now measurable session-quality gates; `missing_session_window`, `session_window_unverified`, `outside_session_window`, `session_not_allowed`, `missing_killzone`, `killzone_not_active`, `judas_window_wait`, `opening_range_unresolved`, `weekly_profile_missing`, `news_time_no_trade`, and `session_invalidation_missing` block promotion until timing is explicit. Validation passed focused D5/API/backtest/docs tests, full unit discovery (`846` tests), script static checks (`79` scripts), secret static checks, JS syntax checks, deploy smoke over 11 endpoints, remote ops smoke, read-only live analysis smoke, `/trading/time-session-kernel` field smoke, `/trading/backtests` D5 field smoke, and Playwright screenshot evidence at `output/playwright/d5-time-session-strategy-backtest-1366x768.png`. Remote latest backtest `#513` is correctly blocked by D5 as `study_only / session_window_unverified` instead of treating `ASIA` plus `mark_outside` as a verified session window. No execution/readiness/risk/trust/operator-confirmation gate was changed.
|
||||
- D6 Model Playbook Kernel is complete, deployed, and remote-verified: added `DOCTRINE_MODEL_PLAYBOOK_PHASE=D6_MODEL_PLAYBOOK_KERNEL`, `ModelPlaybookRule`, `GET /trading/model-playbook-kernel`, Strategy Backtest `model_playbook_backtest_features` / `model_playbook_kernel_status` / `model_playbook_kernel_summary`, and `docs/AI_ICT_DoctrineKernel_ModelPlaybookKernel_D6_2026-05-22_v1.md`. Raw `model_code` values such as `SWEEP_MSS_FVG` now stay `study_only` until they resolve to a registered setup playbook with explicit Range / Draw / Liquidity / PD Array / Session / Confirmation / Invalidation / Target, batch cohort, quality-gate, journal-feedback, and operator-runbook contracts. Validation passed focused D6/API/backtest/docs tests, full unit discovery (`853` tests), script static checks (`79` scripts), secret static checks, JS syntax checks, deploy smoke over 11 endpoints, remote ops smoke, read-only live analysis smoke, `/trading/model-playbook-kernel` field smoke, `/trading/backtests` D6 field smoke, and Playwright screenshot evidence at `output/playwright/d6-model-playbook-strategy-backtest-fullpage-zh.png`. Remote latest backtest `#513` is correctly blocked by D6 as `study_only / missing_model_playbook` instead of promoting a surface-shape model as a tradeable setup. No execution/readiness/risk/trust/operator-confirmation gate was changed.
|
||||
- D7 Trader Product Rebuild / Playbook Promotion Gate is complete, deployed, and remote-verified: added `build_trader_product_promotion_gate`, `candidate_watchlist_tiers.*[].trader_product_promotion_gate`, `candidate_watchlist_summary.trader_product_gate_summary`, `trader_mode_summary.trader_product_gate_summary`, and `docs/AI_ICT_DoctrineKernel_TraderProductGate_D7_2026-05-22_v1.md`. Trader Cockpit `Review Now` now requires playbook-mapped + D3/D4/D5/D6 Doctrine-complete + Strategy Backtest-supported + active Trading Plan-approved candidates; otherwise the candidate is routed to study-only / `archive_only`, the focus matrix becomes `study_only`, and Trader Mode refuses to fall back to a decision-packet focus. Remote latest Trader Mode payload correctly shows `allowed_focus_count=0`, `study_only_count=5`, status `Observe Only`, and no primary opportunity, while the latest raw backtest remains D6-blocked. Validation passed focused operator/doctrine/static/docs tests, full unit discovery (`853` tests), script static checks (`79` scripts), secret static checks, JS syntax checks, deploy smoke over 11 endpoints, remote ops smoke, read-only live analysis smoke, direct Trader Mode D7 field smoke, and Playwright screenshot evidence at `output/playwright/d7-trader-product-gate-trader-mode-zh.png`. No execution/readiness/risk/trust/operator-confirmation gate was changed.
|
||||
- D8 Playbook-Specific Historical Cohort Gate is complete locally: added `DOCTRINE_PLAYBOOK_HISTORICAL_COHORT_PHASE=D8_PLAYBOOK_SPECIFIC_HISTORICAL_COHORT_GATE`, `build_playbook_historical_cohort_descriptor`, `summarize_playbook_historical_cohorts`, Strategy Backtest `playbook_historical_cohort_summary`, `scripts/run_backtest_batch.py` JSON/text cohort rollup output, the `Playbook Historical Gate` / `Playbook Historical Cohorts` Strategy Backtest UI readout, and `docs/AI_ICT_DoctrineKernel_PlaybookHistoricalCohortGate_D8_2026-05-22_v1.md`. D8 now groups historical evidence by `setup_playbook_id`, `session_group`, `timeframe_key`, and `model_family`, and distinguishes `study_only`, `unmapped_study_only`, and `manual_review_candidate` without promoting anything into live focus. Validation passed focused D8/API/batch/docs tests (`46` tests), full unit discovery (`856` tests), script static checks (`79` scripts), secret static checks, `node --check src/web/app.js`, `node --check src/web/i18n.js`, and `git diff --check` with only LF/CRLF warnings. No execution/readiness/risk/trust/operator-confirmation gate was changed.
|
||||
- D9 Playbook Remediation Queue is complete locally: added `DOCTRINE_PLAYBOOK_REMEDIATION_PHASE=D9_PLAYBOOK_REMEDIATION_QUEUE`, `summarize_playbook_remediation_queue`, `docs/AI_ICT_DoctrineKernel_PlaybookRemediationQueue_D9_2026-05-22_v1.md`, Strategy Backtest `playbook_remediation_queue_summary`, and batch-output remediation routing in `scripts/run_backtest_batch.py`. D9 now converts failed D8 cohorts into remediation items with a dominant rule family, dominant blocker reason, Study Notes prompt, Strategy Backtest review prompt, and a primary route of either `strategy_backtest` or `study_notes`, without promoting anything into Trader Cockpit focus. Validation passed focused D9/API/batch/docs tests (`47` tests), full unit discovery (`857` tests), script static checks (`79` scripts), secret static checks, `node --check src/web/app.js`, `node --check src/web/i18n.js`, and `git diff --check` with only LF/CRLF warnings. No execution/readiness/risk/trust/operator-confirmation gate was changed.
|
||||
|
||||
## Exact Next Step
|
||||
|
||||
Use `docs/AI_ICT_DoctrineKernel全书规则内核重建_D0_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel市场语言规则内核_D1_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel_PlaybookBacktestCohort_D2_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel_LiquidityKernel_D3_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel_PDArrayReactionKernel_D4_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_TimeSessionKernel_D5_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_ModelPlaybookKernel_D6_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_TraderProductGate_D7_2026-05-22_v1.md`, `docs/ICT交易全书_出版终校版.md`, `docs/AI_ICT_硬性主线约束_必须遵循ICT交易全书_2026-05-21_v1.md`, `docs/AI_ICT_实盘辅助分析决策助手ICT主线开发方案_2026-05-21_v1.md`, `docs/AI_ICT_批量历史回测验证_S16_2026-05-21_v1.md`, `SESSION.md`, and `TASKS.md` as the active anchors. Start D8 Playbook-Specific Historical Cohort Gate: run and summarize batch backtests by registered `setup_playbook_id`, session, timeframe set, and model family so Strategy Backtest can distinguish which book-derived playbooks remain study-only versus eligible for manual review. Do not restart external-context/archive/digest long-tail work, do not add order-submission features, do not physically delete S1 deprecated surfaces yet, and do not fabricate R2 paper samples from blocked candidates. Keep `execution_allowed`, `decision_support_ready`, `real_live_ready`, risk governor, trust bucket, and operator confirmation protocol unchanged.
|
||||
Use `docs/AI_ICT_DoctrineKernel全书规则内核重建_D0_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel市场语言规则内核_D1_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel_PlaybookBacktestCohort_D2_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel_LiquidityKernel_D3_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel_PDArrayReactionKernel_D4_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_TimeSessionKernel_D5_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_ModelPlaybookKernel_D6_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_TraderProductGate_D7_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_PlaybookHistoricalCohortGate_D8_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_PlaybookRemediationQueue_D9_2026-05-22_v1.md`, `docs/ICT交易全书_出版终校版.md`, `docs/AI_ICT_硬性主线约束_必须遵循ICT交易全书_2026-05-21_v1.md`, `docs/AI_ICT_实盘辅助分析决策助手ICT主线开发方案_2026-05-21_v1.md`, `docs/AI_ICT_批量历史回测验证_S16_2026-05-21_v1.md`, `SESSION.md`, and `TASKS.md` as the active anchors. Start D10 Remediation Artifacts: generate per-cohort remediation artifacts from D9 so Strategy Backtest gets saved review prompts and Study Notes gets queue-ready draft language, without adding any new trader-facing signal or promotion path. Do not restart external-context/archive/digest long-tail work, do not add order-submission features, do not physically delete S1 deprecated surfaces yet, and do not fabricate R2 paper samples from blocked candidates. Keep `execution_allowed`, `decision_support_ready`, `real_live_ready`, risk governor, trust bucket, and operator confirmation protocol unchanged.
|
||||
|
||||
## Validation Commands Used Recently
|
||||
|
||||
|
||||
@@ -290,6 +290,11 @@
|
||||
|
||||
- [x] state: automation_execution_layer_pxx_known_scope_closed
|
||||
- [x] current_phase: Long-running Analysis Decision Monitor With Ops Rollup
|
||||
- [x] latest_analysis_decision_monitor_rollup_refresh: `2026-05-22T13:27:14Z` -> `.claude/analysis_decision_monitor_maturity_rollup.json`
|
||||
- [x] latest_analysis_decision_monitor_rollup_status: blocked / attention_required / next_slice=`repair_analysis_decision_monitor_blockers`
|
||||
- [x] latest_analysis_decision_monitor_rollup_failure_reasons: latest_monitor_not_passed, stable_monitoring_window_not_reached, latest_monitor_has_blockers, runtime_health_not_ok, and p6_governance_tick_not_noop
|
||||
- [x] latest_analysis_decision_monitor_rollup_alert_quality: required review fields are still present, but quality is `needs_attention` because the monitor is not passed and has blockers
|
||||
- [x] latest_analysis_decision_monitor_rollup_drift_note: fresh remote drift check was not rerun because `AI_ICT_DEPLOY_PASSWORD` is absent locally; the stored 2026-05-11 drift artifact was `clear` and is now historical only
|
||||
- [x] active_product_phase: Decision Assistant Phase H/F/D/G/E Dense Recall + Templates + Announcements + Peer Baselines
|
||||
- [x] latest_decision_assistant_slice: unified_cockpit_summary
|
||||
- [x] latest_decision_assistant_change_slice: review_packet_delta_summary
|
||||
@@ -839,6 +844,7 @@
|
||||
## Blocked
|
||||
|
||||
- [x] External chart adapter is optional in the current analysis-only phase and no external chart URL is required for the main loop.
|
||||
- [ ] `repair_analysis_decision_monitor_blockers`: clear `launch_no_blockers`, `launch_analysis_only_ready`, `launch_restricted_decision_support_ready`, `runtime_health_ok`, `market_stream_connected`, and `p6_governance_tick_noop` before re-entering P7-2/P7-3
|
||||
|
||||
## Standing Guardrails
|
||||
|
||||
@@ -959,7 +965,9 @@
|
||||
- [x] D5 Time / Session Kernel: rebuilt Asia / London / NY AM / NY PM / killzone / Judas window / opening range / weekly profile / news time into measurable session-quality gates instead of loose time labels; Strategy Backtest now exposes `time_session_backtest_features`, `time_session_kernel_status`, `time_session_kernel_summary`, and `/trading/time-session-kernel` without changing execution/readiness gates
|
||||
- [x] D6 Model Playbook Kernel: raw model codes now stay study-only until they resolve to registered setup playbooks with Range / Draw / Liquidity / PD Array / Session / Confirmation / Invalidation / Target, batch-cohort, quality-gate, journal-feedback, and operator-runbook contracts; Strategy Backtest exposes D6 features/status/summary and `/trading/model-playbook-kernel` exposes the D6 registry without changing execution/readiness gates
|
||||
- [x] D7 Trader Product Rebuild / Playbook Promotion Gate: only playbook-mapped, backtest-supported, plan-approved setups may re-enter Trader Cockpit focus; study-only candidates are routed out of `Review Now` and remain Strategy Backtest / Study Notes / System Ops evidence
|
||||
- [ ] D8 Playbook-Specific Historical Cohort Gate: run and summarize batch backtests by registered `setup_playbook_id`, session, timeframe set, and model family so Strategy Backtest can distinguish study-only playbooks from manual-review-eligible playbooks
|
||||
- [x] D8 Playbook-Specific Historical Cohort Gate: grouped historical evidence by `setup_playbook_id`, session, timeframe set, and model family; added `playbook_historical_cohort_summary` to Strategy Backtest and batch output; surfaced `study_only`, `unmapped_study_only`, and `manual_review_candidate` without changing execution/readiness gates
|
||||
- [x] D9 Playbook Remediation Queue: summarize the dominant D8 cohort blocker family, blocker reason, and remediation route; add `playbook_remediation_queue_summary` to Strategy Backtest and batch output; route failed cohorts to `strategy_backtest` or `study_notes` without changing execution/readiness gates
|
||||
- [ ] D10 Remediation Artifacts: turn D9 queue items into saved Strategy Backtest review prompts and Study Notes draft artifacts before any new trader-facing expansion
|
||||
- [x] R3 OKX sandbox isolated smoke tooling: guarded script and service can call OKX demo submit/query/cancel/query/fills only when `OKX_SANDBOX_SMOKE_EXECUTE=1`; local validation, deploy smoke, and remote ops smoke passed
|
||||
- [ ] R3 sandbox credential permission gate deferred: current phase is assisted analysis / decision support only, so do not request trade-permission credentials or make submit/cancel smoke the next gate
|
||||
- [ ] Long-term target: complete `Real-time Assisted Trading V1` without enabling default live auto-trading
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
# AI_ICT Doctrine Kernel Playbook Historical Cohort Gate D8 2026-05-22 v1
|
||||
|
||||
## Scope
|
||||
|
||||
D8 turns historical backtest evidence into playbook-specific cohorts. The goal is not to add new trader-facing signals. The goal is to answer one Strategy Backtest question:
|
||||
|
||||
`For each registered book-derived playbook, which session / timeframe / model-family cohort is still study-only, and which cohort may enter manual review?`
|
||||
|
||||
## ICT Chain Served
|
||||
|
||||
D8 serves the hard mainline chain:
|
||||
|
||||
`Range / Draw / Location -> Liquidity / Sweep / Inducement / Path -> Displacement / MSS / FVG / Retest / Session -> backtest evidence -> pre-market plan -> intraday unique focus or no-trade reason -> journal / study-notes feedback`.
|
||||
|
||||
It sits at `backtest evidence`. A D8 pass does not create a live-trading signal and does not bypass Trading Plan, Trader Cockpit, Journal, or R2 evidence gates.
|
||||
|
||||
## Product Rule
|
||||
|
||||
Historical evidence is grouped by:
|
||||
|
||||
- `setup_playbook_id`
|
||||
- `session_group`
|
||||
- `timeframe_key`
|
||||
- `model_family`
|
||||
|
||||
A cohort is `manual_review_candidate` only when:
|
||||
|
||||
- it maps to a registered `setup_playbook_id`,
|
||||
- at least one historical run in that cohort passes the Strategy Backtest quality gate,
|
||||
- at least one historical run in that cohort passes the Doctrine promotion gate.
|
||||
|
||||
All other cohorts remain `study_only` or `unmapped_study_only`.
|
||||
|
||||
## Implemented Surface
|
||||
|
||||
Backend:
|
||||
|
||||
- `DOCTRINE_PLAYBOOK_HISTORICAL_COHORT_PHASE = D8_PLAYBOOK_SPECIFIC_HISTORICAL_COHORT_GATE`
|
||||
- `build_playbook_historical_cohort_descriptor`
|
||||
- `summarize_playbook_historical_cohorts`
|
||||
- `/trading/backtests` now returns `playbook_historical_cohort_summary`
|
||||
|
||||
Batch script:
|
||||
|
||||
- `scripts/run_backtest_batch.py` now includes `playbook_historical_cohort_summary` in JSON output.
|
||||
- CLI text output prints manual-review candidate count versus study-only cohort count.
|
||||
|
||||
Frontend:
|
||||
|
||||
- Strategy Backtest shows `Playbook Historical Gate`.
|
||||
- Strategy Backtest lists top `Playbook Historical Cohorts` by playbook / session / timeframe / model family.
|
||||
|
||||
## Read-Only Boundary
|
||||
|
||||
D8 is read-only historical analysis. It does not add order submission, does not change `execution_allowed`, does not change `decision_support_ready`, does not change `real_live_ready`, and does not loosen risk governor / trust bucket / operator confirmation protocol.
|
||||
|
||||
## Validation Meaning
|
||||
|
||||
If D8 marks a cohort `study_only`, it must not enter Trader Cockpit `Review Now`.
|
||||
|
||||
If D8 marks a cohort `manual_review_candidate`, it still only means the historical cohort may be reviewed by a human. It does not become a live focus until the Trading Plan, Trader Product Gate, R2 paper/operator evidence, and current market context all pass.
|
||||
|
||||
## Next Slice
|
||||
|
||||
D9 should turn failed D8 cohort blockers into a playbook remediation queue: identify which ICT rule family failed most often for each playbook cohort, then route remediation to Study Notes / Strategy Backtest before adding any new trader-facing feature.
|
||||
@@ -0,0 +1,81 @@
|
||||
# AI_ICT Doctrine Kernel Playbook Remediation Queue D9 2026-05-22 v1
|
||||
|
||||
## Scope
|
||||
|
||||
D9 turns failed D8 cohorts into a remediation queue. The goal is not to add any new trader-facing signal. The goal is to answer one post-backtest question:
|
||||
|
||||
`For each failed playbook cohort, which rule family fails most often, and should the next fix start in Strategy Backtest or Study Notes?`
|
||||
|
||||
## ICT Chain Served
|
||||
|
||||
D9 still serves the hard mainline chain:
|
||||
|
||||
`Range / Draw / Location -> Liquidity / Sweep / Inducement / Path -> Displacement / MSS / FVG / Retest / Session -> backtest evidence -> pre-market plan -> intraday unique focus or no-trade reason -> journal / study-notes feedback`.
|
||||
|
||||
It sits between `backtest evidence` and `journal / study-notes feedback`. A D9 queue item is remediation evidence only. It is not a promotion gate bypass and not a live focus signal.
|
||||
|
||||
## Product Rule
|
||||
|
||||
A failed D8 cohort must be reduced to:
|
||||
|
||||
- one dominant remediation family,
|
||||
- one dominant blocker reason code,
|
||||
- one primary remediation route,
|
||||
- one Study Notes prompt,
|
||||
- one Strategy Backtest review prompt.
|
||||
|
||||
The queue may route a cohort to:
|
||||
|
||||
- `strategy_backtest` first when the dominant blocker is historical quality evidence,
|
||||
- `study_notes` first when the dominant blocker is model definition, liquidity/confirmation chain, session discipline, plan alignment, or risk boundary.
|
||||
|
||||
No D9 output may promote a cohort into Trader Cockpit `Review Now`.
|
||||
|
||||
## Implemented Surface
|
||||
|
||||
Backend:
|
||||
|
||||
- `DOCTRINE_PLAYBOOK_REMEDIATION_PHASE = D9_PLAYBOOK_REMEDIATION_QUEUE`
|
||||
- `summarize_playbook_remediation_queue`
|
||||
- `/trading/backtests` now returns `playbook_remediation_queue_summary`
|
||||
|
||||
Batch script:
|
||||
|
||||
- `scripts/run_backtest_batch.py` now includes `playbook_remediation_queue_summary` in JSON output.
|
||||
- CLI text output prints queued cohort count plus `Strategy Backtest first` versus `Study Notes first`.
|
||||
|
||||
Frontend:
|
||||
|
||||
- Strategy Backtest shows `Playbook Remediation Queue`.
|
||||
- Strategy Backtest lists the top queued cohorts with the dominant remediation family and route.
|
||||
|
||||
## Remediation Families
|
||||
|
||||
D9 groups dominant blockers into these families:
|
||||
|
||||
- `backtest_quality_gate`
|
||||
- `model_playbook_contract`
|
||||
- `liquidity_path`
|
||||
- `pd_array_reaction`
|
||||
- `time_session`
|
||||
- `trading_plan_alignment`
|
||||
- `risk_boundary`
|
||||
- `general_study_only`
|
||||
|
||||
These families are then mapped into the existing Chapter 46-47 feedback classes so the queue can point back to Study Notes instead of inventing a new feedback taxonomy.
|
||||
|
||||
## Read-Only Boundary
|
||||
|
||||
D9 is read-only historical analysis and remediation routing. It does not add order submission, does not change `execution_allowed`, does not change `decision_support_ready`, does not change `real_live_ready`, and does not loosen risk governor / trust bucket / operator confirmation protocol.
|
||||
|
||||
## Validation Meaning
|
||||
|
||||
If D9 routes a cohort to `strategy_backtest`, the operator should inspect sample size, expectancy, profit factor, drawdown, executed-signal sufficiency, and related cohort evidence before revisiting the playbook.
|
||||
|
||||
If D9 routes a cohort to `study_notes`, the operator should tighten the model definition, confirmation chain, session rule, plan rule, or no-trade boundary before any trader-facing reconsideration.
|
||||
|
||||
In both cases the cohort remains `study_only` until D8 and D7 gates pass again.
|
||||
|
||||
## Next Slice
|
||||
|
||||
D10 should turn D9 queue items into saved remediation artifacts: generate per-cohort Study Notes drafts and Strategy Backtest review prompts so the operator can close the loop without scanning raw reason codes.
|
||||
@@ -18,7 +18,11 @@ from scripts.run_backtest import (
|
||||
build_quality_gate_from_env,
|
||||
build_replay_config_from_env,
|
||||
)
|
||||
from src.services.ict_mainline import build_doctrine_backtest_cohort
|
||||
from src.services.ict_mainline import (
|
||||
build_doctrine_backtest_cohort,
|
||||
summarize_playbook_historical_cohorts,
|
||||
summarize_playbook_remediation_queue,
|
||||
)
|
||||
from src.services.backtest import (
|
||||
BACKTEST_MODE_REPLAY_NO_LOOKAHEAD,
|
||||
BACKTEST_SUBJECT_PLAN_REPLAY,
|
||||
@@ -78,6 +82,19 @@ def main(env=None) -> int:
|
||||
print(f"Total results: {payload['total_result_count']}; executed: {payload['total_executed_signals']}")
|
||||
if payload["quality_gate_reason_counts"]:
|
||||
print(f"Top quality gate reasons: {_format_counts(payload['quality_gate_reason_counts'])}")
|
||||
d8_summary = payload.get("playbook_historical_cohort_summary") or {}
|
||||
print(
|
||||
"Playbook cohorts: "
|
||||
f"{d8_summary.get('manual_review_eligible_count', 0)} manual-review candidates / "
|
||||
f"{d8_summary.get('study_only_count', 0)} study-only"
|
||||
)
|
||||
d9_summary = payload.get("playbook_remediation_queue_summary") or {}
|
||||
print(
|
||||
"Remediation queue: "
|
||||
f"{d9_summary.get('queue_count', 0)} queued / "
|
||||
f"{d9_summary.get('strategy_backtest_route_count', 0)} Strategy Backtest first / "
|
||||
f"{d9_summary.get('study_notes_route_count', 0)} Study Notes first"
|
||||
)
|
||||
print(payload["operator_summary"])
|
||||
return 2 if require_quality_gate and payload["blocked_quality_gate_count"] else 0
|
||||
|
||||
@@ -181,6 +198,7 @@ def run_backtest_batch(
|
||||
"doctrine_completeness": doctrine_cohort.get("doctrine_completeness"),
|
||||
"result_count": len(outcomes),
|
||||
"executed_signals": int(summary.get("executed_signals") or 0),
|
||||
"total_r": float(summary.get("total_r") or 0.0),
|
||||
"expectancy_r": float(summary.get("expectancy_r") or 0.0),
|
||||
"executed_expectancy_r": float(summary.get("executed_expectancy_r") or 0.0),
|
||||
"profit_factor_r": summary.get("profit_factor_r"),
|
||||
@@ -193,6 +211,8 @@ def run_backtest_batch(
|
||||
|
||||
passed_count = sum(1 for item in results if item["quality_gate"].get("passed"))
|
||||
blocked_count = len(results) - passed_count
|
||||
playbook_historical_cohort_summary = summarize_playbook_historical_cohorts(results)
|
||||
playbook_remediation_queue_summary = summarize_playbook_remediation_queue(results)
|
||||
payload = {
|
||||
"status": "blocked" if require_quality_gate and blocked_count else "completed",
|
||||
"mode": "batch_historical_backtest",
|
||||
@@ -209,6 +229,8 @@ def run_backtest_batch(
|
||||
"aggregate_by_subject": aggregate_by_subject,
|
||||
"aggregate_by_session_group": aggregate_by_session_group,
|
||||
"aggregate_by_doctrine_cohort": aggregate_by_doctrine_cohort,
|
||||
"playbook_historical_cohort_summary": playbook_historical_cohort_summary,
|
||||
"playbook_remediation_queue_summary": playbook_remediation_queue_summary,
|
||||
"top_runs_by_expectancy": sorted(
|
||||
results,
|
||||
key=lambda item: (float(item.get("executed_expectancy_r") or 0.0), int(item.get("executed_signals") or 0)),
|
||||
|
||||
@@ -23,6 +23,8 @@ from src.services.ict_mainline import (
|
||||
summarize_liquidity_kernel_statuses,
|
||||
summarize_model_playbook_kernel_statuses,
|
||||
summarize_pd_array_reaction_statuses,
|
||||
summarize_playbook_historical_cohorts,
|
||||
summarize_playbook_remediation_queue,
|
||||
summarize_time_session_kernel_statuses,
|
||||
)
|
||||
|
||||
@@ -49,6 +51,8 @@ def get_strategy_backtest_surface(instrument_id: Optional[int] = None, limit: in
|
||||
pd_array_reaction_summary = summarize_pd_array_reaction_statuses(items)
|
||||
time_session_kernel_summary = summarize_time_session_kernel_statuses(items)
|
||||
model_playbook_kernel_summary = summarize_model_playbook_kernel_statuses(items)
|
||||
playbook_historical_cohort_summary = summarize_playbook_historical_cohorts(items)
|
||||
playbook_remediation_queue_summary = summarize_playbook_remediation_queue(items)
|
||||
return {
|
||||
"instrument_id": instrument_id,
|
||||
"item_count": len(items),
|
||||
@@ -62,6 +66,8 @@ def get_strategy_backtest_surface(instrument_id: Optional[int] = None, limit: in
|
||||
"pd_array_reaction_summary": pd_array_reaction_summary,
|
||||
"time_session_kernel_summary": time_session_kernel_summary,
|
||||
"model_playbook_kernel_summary": model_playbook_kernel_summary,
|
||||
"playbook_historical_cohort_summary": playbook_historical_cohort_summary,
|
||||
"playbook_remediation_queue_summary": playbook_remediation_queue_summary,
|
||||
"ict_rulebook_mapping": ict_rulebook_mapping,
|
||||
"items": items,
|
||||
}
|
||||
|
||||
@@ -22,12 +22,15 @@ from src.services.ict_mainline.doctrine import (
|
||||
DOCTRINE_KERNEL_PHASE,
|
||||
DOCTRINE_LIQUIDITY_PHASE,
|
||||
DOCTRINE_MODEL_PLAYBOOK_PHASE,
|
||||
DOCTRINE_PLAYBOOK_HISTORICAL_COHORT_PHASE,
|
||||
DOCTRINE_PLAYBOOK_REMEDIATION_PHASE,
|
||||
DOCTRINE_PLAYBOOK_PHASE,
|
||||
DOCTRINE_REACTION_PHASE,
|
||||
DOCTRINE_SESSION_PHASE,
|
||||
build_doctrine_backtest_cohort,
|
||||
build_liquidity_backtest_features,
|
||||
build_model_playbook_backtest_features,
|
||||
build_playbook_historical_cohort_descriptor,
|
||||
build_pd_array_backtest_features,
|
||||
build_time_session_backtest_features,
|
||||
build_doctrine_rule_matrix,
|
||||
@@ -54,6 +57,8 @@ from src.services.ict_mainline.doctrine import (
|
||||
summarize_doctrine_rule_gaps,
|
||||
summarize_liquidity_kernel_statuses,
|
||||
summarize_model_playbook_kernel_statuses,
|
||||
summarize_playbook_historical_cohorts,
|
||||
summarize_playbook_remediation_queue,
|
||||
summarize_pd_array_reaction_statuses,
|
||||
summarize_time_session_kernel_statuses,
|
||||
)
|
||||
@@ -62,11 +67,14 @@ __all__ = [
|
||||
"DOCTRINE_KERNEL_PHASE",
|
||||
"DOCTRINE_LIQUIDITY_PHASE",
|
||||
"DOCTRINE_MODEL_PLAYBOOK_PHASE",
|
||||
"DOCTRINE_PLAYBOOK_HISTORICAL_COHORT_PHASE",
|
||||
"DOCTRINE_PLAYBOOK_REMEDIATION_PHASE",
|
||||
"DOCTRINE_PLAYBOOK_PHASE",
|
||||
"DOCTRINE_REACTION_PHASE",
|
||||
"DOCTRINE_SESSION_PHASE",
|
||||
"ICT_MAINLINE_CHAIN",
|
||||
"build_doctrine_backtest_cohort",
|
||||
"build_playbook_historical_cohort_descriptor",
|
||||
"build_liquidity_backtest_features",
|
||||
"build_model_playbook_backtest_features",
|
||||
"build_pd_array_backtest_features",
|
||||
@@ -107,6 +115,8 @@ __all__ = [
|
||||
"summarize_doctrine_rule_gaps",
|
||||
"summarize_liquidity_kernel_statuses",
|
||||
"summarize_model_playbook_kernel_statuses",
|
||||
"summarize_playbook_historical_cohorts",
|
||||
"summarize_playbook_remediation_queue",
|
||||
"summarize_pd_array_reaction_statuses",
|
||||
"summarize_time_session_kernel_statuses",
|
||||
"summarize_book_rule_errors",
|
||||
|
||||
@@ -5,6 +5,8 @@ import json
|
||||
import re
|
||||
from typing import Iterable
|
||||
|
||||
from src.services.ict_mainline.feedback import BOOK_RULE_ERROR_CLASSES
|
||||
|
||||
|
||||
DOCTRINE_KERNEL_PHASE = "D1_MARKET_LANGUAGE_KERNEL"
|
||||
DOCTRINE_PLAYBOOK_PHASE = "D2_PLAYBOOK_BACKTEST_COHORT"
|
||||
@@ -12,6 +14,20 @@ DOCTRINE_LIQUIDITY_PHASE = "D3_LIQUIDITY_KERNEL"
|
||||
DOCTRINE_REACTION_PHASE = "D4_PD_ARRAY_REACTION_KERNEL"
|
||||
DOCTRINE_SESSION_PHASE = "D5_TIME_SESSION_KERNEL"
|
||||
DOCTRINE_MODEL_PLAYBOOK_PHASE = "D6_MODEL_PLAYBOOK_KERNEL"
|
||||
DOCTRINE_PLAYBOOK_HISTORICAL_COHORT_PHASE = "D8_PLAYBOOK_SPECIFIC_HISTORICAL_COHORT_GATE"
|
||||
DOCTRINE_PLAYBOOK_REMEDIATION_PHASE = "D9_PLAYBOOK_REMEDIATION_QUEUE"
|
||||
|
||||
|
||||
PLAYBOOK_REMEDIATION_FAMILY_LABELS = {
|
||||
"backtest_quality_gate": "Backtest Quality Gate",
|
||||
"model_playbook_contract": "Model Playbook Contract",
|
||||
"liquidity_path": "Liquidity / Sweep / Path",
|
||||
"pd_array_reaction": "PD Array / Reaction",
|
||||
"time_session": "Time / Session",
|
||||
"trading_plan_alignment": "Trading Plan Alignment",
|
||||
"risk_boundary": "Risk Boundary",
|
||||
"general_study_only": "General Study-Only",
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -1282,6 +1298,281 @@ def summarize_doctrine_backtest_cohorts(items: Iterable[dict]) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def build_playbook_historical_cohort_descriptor(payload: object) -> dict:
|
||||
"""Build the D8 cohort dimensions without promoting anything to live focus."""
|
||||
quality = dict(_extract_quality_gate(payload) or {})
|
||||
cohort = {}
|
||||
if isinstance(payload, dict):
|
||||
cohort = dict(payload.get("doctrine_backtest_cohort") or {})
|
||||
if not cohort:
|
||||
cohort = build_doctrine_backtest_cohort(payload, quality_gate=quality)
|
||||
|
||||
setup_playbook_id = _string_or_empty(cohort.get("setup_playbook_id") or extract_setup_playbook_id(payload)) or "missing_playbook"
|
||||
session_codes = _extract_session_codes(payload, cohort)
|
||||
session_group = "ALL" if not session_codes else ",".join(session_codes)
|
||||
timeframe_set = _extract_timeframe_set(payload)
|
||||
timeframe_key = _timeframe_set_key(timeframe_set)
|
||||
model_family = _extract_model_family(payload, cohort)
|
||||
cohort_key = f"{setup_playbook_id}::{session_group}::{timeframe_key}::{model_family}"
|
||||
return {
|
||||
"phase": DOCTRINE_PLAYBOOK_HISTORICAL_COHORT_PHASE,
|
||||
"cohort_key": cohort_key,
|
||||
"cohort_key_fields": ["setup_playbook_id", "session_group", "timeframe_key", "model_family"],
|
||||
"setup_playbook_id": setup_playbook_id,
|
||||
"session_group": session_group,
|
||||
"session_codes": session_codes,
|
||||
"timeframe_set": timeframe_set,
|
||||
"timeframe_key": timeframe_key,
|
||||
"model_family": model_family,
|
||||
"doctrine_cohort_key": cohort.get("cohort_key") or "",
|
||||
"doctrine_completeness": cohort.get("doctrine_completeness") or "incomplete",
|
||||
"quality_gate_passed": bool(quality.get("passed") or cohort.get("quality_gate_passed")),
|
||||
"promotion_gate_passed": bool((cohort.get("promotion_gate") or {}).get("passed")),
|
||||
"promotion_blockers": _dedupe([str(item) for item in list((cohort.get("promotion_gate") or {}).get("reasons") or [])]),
|
||||
"quality_gate_reasons": _dedupe([str(item) for item in list(quality.get("reasons") or [])]),
|
||||
"execution_boundary": "read_only_backtest_no_order_submission",
|
||||
}
|
||||
|
||||
|
||||
def summarize_playbook_historical_cohorts(items: Iterable[dict]) -> dict:
|
||||
cohorts: dict[str, dict] = {}
|
||||
total_runs = 0
|
||||
total_results = 0
|
||||
total_executed = 0
|
||||
for item in items:
|
||||
total_runs += 1
|
||||
descriptor = build_playbook_historical_cohort_descriptor(item)
|
||||
key = str(descriptor.get("cohort_key") or "missing_playbook::ALL::unknown_tf::missing_model_family")
|
||||
bucket = cohorts.setdefault(
|
||||
key,
|
||||
{
|
||||
"cohort_key": key,
|
||||
"setup_playbook_id": descriptor.get("setup_playbook_id"),
|
||||
"session_group": descriptor.get("session_group"),
|
||||
"timeframe_key": descriptor.get("timeframe_key"),
|
||||
"timeframe_set": dict(descriptor.get("timeframe_set") or {}),
|
||||
"model_family": descriptor.get("model_family"),
|
||||
"run_count": 0,
|
||||
"quality_gate_passed_count": 0,
|
||||
"promotion_gate_passed_count": 0,
|
||||
"manual_review_eligible_count": 0,
|
||||
"study_only_count": 0,
|
||||
"total_result_count": 0,
|
||||
"total_executed_signals": 0,
|
||||
"total_r": 0.0,
|
||||
"promotion_blocker_counts": {},
|
||||
"quality_gate_reason_counts": {},
|
||||
"sample_run_ids": [],
|
||||
"execution_boundary": "read_only_backtest_no_order_submission",
|
||||
},
|
||||
)
|
||||
bucket["run_count"] += 1
|
||||
if descriptor.get("quality_gate_passed"):
|
||||
bucket["quality_gate_passed_count"] += 1
|
||||
if descriptor.get("promotion_gate_passed"):
|
||||
bucket["promotion_gate_passed_count"] += 1
|
||||
|
||||
summary = _extract_backtest_summary(item)
|
||||
result_count = _extract_int_first(
|
||||
item,
|
||||
summary,
|
||||
keys=("result_count", "total_signals", "candidate_count"),
|
||||
)
|
||||
executed_signals = _extract_int_first(
|
||||
item,
|
||||
summary,
|
||||
keys=("executed_signals", "total_executed_signals"),
|
||||
)
|
||||
total_r = _extract_float_first(item, summary, keys=("total_r",))
|
||||
bucket["total_result_count"] += result_count
|
||||
bucket["total_executed_signals"] += executed_signals
|
||||
bucket["total_r"] += total_r
|
||||
total_results += result_count
|
||||
total_executed += executed_signals
|
||||
|
||||
run_id = _extract_first_present_value(item, keys=("backtest_run_id", "id", "run_id"))
|
||||
if run_id is not None and len(bucket["sample_run_ids"]) < 5:
|
||||
bucket["sample_run_ids"].append(run_id)
|
||||
for reason in list(descriptor.get("promotion_blockers") or []):
|
||||
bucket["promotion_blocker_counts"][reason] = int(bucket["promotion_blocker_counts"].get(reason) or 0) + 1
|
||||
for reason in list(descriptor.get("quality_gate_reasons") or []):
|
||||
bucket["quality_gate_reason_counts"][reason] = int(bucket["quality_gate_reason_counts"].get(reason) or 0) + 1
|
||||
|
||||
manual_review_eligible_count = 0
|
||||
study_only_count = 0
|
||||
for bucket in cohorts.values():
|
||||
setup_playbook_id = str(bucket.get("setup_playbook_id") or "")
|
||||
eligible = (
|
||||
setup_playbook_id not in {"", "missing_playbook"}
|
||||
and int(bucket.get("quality_gate_passed_count") or 0) > 0
|
||||
and int(bucket.get("promotion_gate_passed_count") or 0) > 0
|
||||
)
|
||||
if eligible:
|
||||
bucket["status"] = "manual_review_candidate"
|
||||
bucket["manual_review_eligible"] = True
|
||||
bucket["manual_review_eligible_count"] = 1
|
||||
bucket["study_only_count"] = 0
|
||||
bucket["eligibility_reason"] = "playbook_historical_cohort_passed"
|
||||
manual_review_eligible_count += 1
|
||||
else:
|
||||
bucket["status"] = "unmapped_study_only" if setup_playbook_id in {"", "missing_playbook"} else "study_only"
|
||||
bucket["manual_review_eligible"] = False
|
||||
bucket["manual_review_eligible_count"] = 0
|
||||
bucket["study_only_count"] = 1
|
||||
bucket["eligibility_reason"] = _first_count_key(bucket["promotion_blocker_counts"]) or _first_count_key(bucket["quality_gate_reason_counts"]) or "historical_cohort_not_passed"
|
||||
study_only_count += 1
|
||||
bucket["must_not_trade"] = True
|
||||
bucket["trader_cockpit_promotion_requires_active_plan"] = True
|
||||
|
||||
sorted_cohorts = sorted(
|
||||
cohorts.values(),
|
||||
key=lambda item: (
|
||||
0 if item.get("manual_review_eligible") else 1,
|
||||
-int(item.get("run_count") or 0),
|
||||
str(item.get("cohort_key") or ""),
|
||||
),
|
||||
)
|
||||
return {
|
||||
"phase": DOCTRINE_PLAYBOOK_HISTORICAL_COHORT_PHASE,
|
||||
"cohort_count": len(sorted_cohorts),
|
||||
"run_count": total_runs,
|
||||
"manual_review_eligible_count": manual_review_eligible_count,
|
||||
"study_only_count": study_only_count,
|
||||
"total_result_count": total_results,
|
||||
"total_executed_signals": total_executed,
|
||||
"cohorts": sorted_cohorts,
|
||||
"operator_summary": (
|
||||
"No playbook-specific historical cohorts yet."
|
||||
if not sorted_cohorts
|
||||
else f"{manual_review_eligible_count} cohorts may enter manual review; {study_only_count} cohorts remain study-only."
|
||||
),
|
||||
"execution_boundary": "read_only_backtest_no_order_submission",
|
||||
"must_not_trade": True,
|
||||
}
|
||||
|
||||
|
||||
def summarize_playbook_remediation_queue(items: Iterable[dict]) -> dict:
|
||||
d8_summary = summarize_playbook_historical_cohorts(items)
|
||||
bucket_map: dict[str, dict] = {}
|
||||
family_counts: dict[str, int] = {}
|
||||
for item in items:
|
||||
descriptor = build_playbook_historical_cohort_descriptor(item)
|
||||
key = str(descriptor.get("cohort_key") or "missing_playbook::ALL::unknown_tf::missing_model_family")
|
||||
bucket = bucket_map.setdefault(
|
||||
key,
|
||||
{
|
||||
"reason_counts": {},
|
||||
"family_counts": {},
|
||||
"promotion_blocker_counts": {},
|
||||
"quality_gate_reason_counts": {},
|
||||
"historical_failure_reason_counts": {},
|
||||
},
|
||||
)
|
||||
for reason in list(descriptor.get("promotion_blockers") or []):
|
||||
_increment_reason_count(bucket["promotion_blocker_counts"], reason, 1)
|
||||
_increment_reason_count(bucket["reason_counts"], reason, 1)
|
||||
for reason in list(descriptor.get("quality_gate_reasons") or []):
|
||||
_increment_reason_count(bucket["quality_gate_reason_counts"], reason, 1)
|
||||
_increment_reason_count(bucket["reason_counts"], reason, 1)
|
||||
for reason, count in _extract_failure_reason_counts(item).items():
|
||||
_increment_reason_count(bucket["historical_failure_reason_counts"], reason, count)
|
||||
_increment_reason_count(bucket["reason_counts"], reason, count)
|
||||
|
||||
queue_items = []
|
||||
strategy_backtest_route_count = 0
|
||||
study_notes_route_count = 0
|
||||
for cohort in list(d8_summary.get("cohorts") or []):
|
||||
if cohort.get("manual_review_eligible"):
|
||||
continue
|
||||
key = str(cohort.get("cohort_key") or "")
|
||||
bucket = bucket_map.get(
|
||||
key,
|
||||
{
|
||||
"reason_counts": {},
|
||||
"family_counts": {},
|
||||
"promotion_blocker_counts": {},
|
||||
"quality_gate_reason_counts": {},
|
||||
"historical_failure_reason_counts": {},
|
||||
},
|
||||
)
|
||||
reason_counts = dict(bucket.get("reason_counts") or {})
|
||||
family_counts_for_bucket = _build_remediation_family_counts(reason_counts)
|
||||
dominant_reason = _first_count_key(reason_counts) or str(cohort.get("eligibility_reason") or "historical_cohort_not_passed")
|
||||
dominant_family = _first_count_key(family_counts_for_bucket) or _resolve_playbook_remediation_family(dominant_reason)
|
||||
dominant_reason_count = int(reason_counts.get(dominant_reason) or 0)
|
||||
route = _build_playbook_remediation_route(dominant_family=dominant_family, dominant_reason=dominant_reason)
|
||||
definition = BOOK_RULE_ERROR_CLASSES[route["error_class"]]
|
||||
_increment_reason_count(family_counts, dominant_family, 1)
|
||||
if route["primary_route"] == "strategy_backtest":
|
||||
strategy_backtest_route_count += 1
|
||||
else:
|
||||
study_notes_route_count += 1
|
||||
queue_items.append(
|
||||
{
|
||||
"cohort_key": key,
|
||||
"setup_playbook_id": cohort.get("setup_playbook_id"),
|
||||
"session_group": cohort.get("session_group"),
|
||||
"timeframe_key": cohort.get("timeframe_key"),
|
||||
"timeframe_set": dict(cohort.get("timeframe_set") or {}),
|
||||
"model_family": cohort.get("model_family"),
|
||||
"status": "queued",
|
||||
"cohort_status": cohort.get("status"),
|
||||
"dominant_remediation_family": dominant_family,
|
||||
"dominant_remediation_family_label": PLAYBOOK_REMEDIATION_FAMILY_LABELS.get(dominant_family, "General Study-Only"),
|
||||
"dominant_error_class": route["error_class"],
|
||||
"dominant_error_label": definition["label"],
|
||||
"dominant_reason_code": dominant_reason,
|
||||
"dominant_reason_count": dominant_reason_count,
|
||||
"route_targets": list(route["route_targets"]),
|
||||
"primary_route": route["primary_route"],
|
||||
"secondary_route": route["secondary_route"],
|
||||
"study_note_prompt": definition["study_note_prompt"],
|
||||
"strategy_backtest_prompt": route["strategy_backtest_prompt"],
|
||||
"next_action": definition["next_action"],
|
||||
"book_anchor": definition["book_anchor"],
|
||||
"promotion_blocker_counts": dict(bucket.get("promotion_blocker_counts") or {}),
|
||||
"quality_gate_reason_counts": dict(bucket.get("quality_gate_reason_counts") or {}),
|
||||
"historical_failure_reason_counts": dict(bucket.get("historical_failure_reason_counts") or {}),
|
||||
"remediation_reason_counts": reason_counts,
|
||||
"remediation_family_counts": family_counts_for_bucket,
|
||||
"top_reason_codes": _top_count_entries(reason_counts, limit=5),
|
||||
"run_count": int(cohort.get("run_count") or 0),
|
||||
"total_result_count": int(cohort.get("total_result_count") or 0),
|
||||
"total_executed_signals": int(cohort.get("total_executed_signals") or 0),
|
||||
"total_r": float(cohort.get("total_r") or 0.0),
|
||||
"must_not_trade": True,
|
||||
"operator_summary": (
|
||||
f"Route this {cohort.get('cohort_status') or 'study_only'} cohort to "
|
||||
f"{route['primary_route'].replace('_', ' ')} first; keep it out of Trader Cockpit focus."
|
||||
),
|
||||
}
|
||||
)
|
||||
|
||||
queue_items = sorted(
|
||||
queue_items,
|
||||
key=lambda item: (
|
||||
-int(item.get("dominant_reason_count") or 0),
|
||||
-int(item.get("run_count") or 0),
|
||||
str(item.get("cohort_key") or ""),
|
||||
),
|
||||
)
|
||||
return {
|
||||
"phase": DOCTRINE_PLAYBOOK_REMEDIATION_PHASE,
|
||||
"queue_count": len(queue_items),
|
||||
"strategy_backtest_route_count": strategy_backtest_route_count,
|
||||
"study_notes_route_count": study_notes_route_count,
|
||||
"dominant_family_counts": family_counts,
|
||||
"items": queue_items,
|
||||
"operator_summary": (
|
||||
"No failed playbook cohorts need remediation right now."
|
||||
if not queue_items
|
||||
else f"{len(queue_items)} failed cohorts queued: {strategy_backtest_route_count} route to Strategy Backtest first, {study_notes_route_count} route to Study Notes first."
|
||||
),
|
||||
"execution_boundary": "read_only_backtest_no_order_submission",
|
||||
"must_not_trade": True,
|
||||
}
|
||||
|
||||
|
||||
def build_liquidity_backtest_features(payload: object) -> dict:
|
||||
features = {
|
||||
"liquidity_target_side": _string_or_empty(
|
||||
@@ -2149,6 +2440,272 @@ def summarize_doctrine_rule_gaps(items: Iterable[dict]) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def _extract_session_codes(payload: object, cohort: dict | None = None) -> list[str]:
|
||||
for candidate in (
|
||||
_extract_first_present_value(payload, keys=("session_codes", "sessions")),
|
||||
_extract_first_present_value(payload, keys=("allowed_session_codes", "session_group")),
|
||||
(cohort or {}).get("session_codes"),
|
||||
):
|
||||
values = _as_string_list(candidate)
|
||||
if values:
|
||||
return values
|
||||
session_code = _string_or_empty(
|
||||
_extract_first_present_value(payload, keys=("session_code", "session"))
|
||||
or ((cohort or {}).get("time_session_backtest_features") or {}).get("session_code")
|
||||
)
|
||||
return [] if not session_code or session_code.upper() == "ALL" else [session_code]
|
||||
|
||||
|
||||
def _extract_timeframe_set(payload: object) -> dict:
|
||||
raw = _extract_first_present_value(payload, keys=("timeframe_set",))
|
||||
if isinstance(raw, dict):
|
||||
return {
|
||||
"bias_tf": _string_or_empty(raw.get("bias_tf") or raw.get("bias_timeframe") or raw.get("bias")),
|
||||
"setup_tf": _string_or_empty(raw.get("setup_tf") or raw.get("setup_timeframe") or raw.get("setup")),
|
||||
"trigger_tf": _string_or_empty(raw.get("trigger_tf") or raw.get("trigger_timeframe") or raw.get("trigger")),
|
||||
}
|
||||
return {
|
||||
"bias_tf": _string_or_empty(_extract_first_present_value(payload, keys=("bias_tf", "bias_timeframe"))),
|
||||
"setup_tf": _string_or_empty(_extract_first_present_value(payload, keys=("setup_tf", "setup_timeframe"))),
|
||||
"trigger_tf": _string_or_empty(_extract_first_present_value(payload, keys=("trigger_tf", "trigger_timeframe"))),
|
||||
}
|
||||
|
||||
|
||||
def _timeframe_set_key(timeframe_set: dict) -> str:
|
||||
bias_tf = _string_or_empty(timeframe_set.get("bias_tf")) or "missing_bias_tf"
|
||||
setup_tf = _string_or_empty(timeframe_set.get("setup_tf")) or "missing_setup_tf"
|
||||
trigger_tf = _string_or_empty(timeframe_set.get("trigger_tf")) or "missing_trigger_tf"
|
||||
return f"{bias_tf}/{setup_tf}/{trigger_tf}"
|
||||
|
||||
|
||||
def _extract_model_family(payload: object, cohort: dict | None = None) -> str:
|
||||
for candidate in (
|
||||
_extract_first_present_value(payload, keys=("model_family", "playbook_model_family", "setup_family", "cohort_family")),
|
||||
((cohort or {}).get("model_playbook_backtest_features") or {}).get("model_family"),
|
||||
_extract_first_present_value(payload, keys=("candidate_model_code", "model_code")),
|
||||
((cohort or {}).get("playbook_gate") or {}).get("setup_name"),
|
||||
):
|
||||
value = _string_or_empty(candidate)
|
||||
if value:
|
||||
return value
|
||||
return "missing_model_family"
|
||||
|
||||
|
||||
def _extract_backtest_summary(payload: object) -> dict:
|
||||
if isinstance(payload, dict):
|
||||
summary = payload.get("summary")
|
||||
if isinstance(summary, dict):
|
||||
return dict(summary)
|
||||
return {}
|
||||
|
||||
|
||||
def _extract_failure_reason_counts(payload: object) -> dict[str, int]:
|
||||
if not isinstance(payload, dict):
|
||||
return {}
|
||||
summary = _extract_backtest_summary(payload)
|
||||
counts: dict[str, int] = {}
|
||||
raw_failure_counts = summary.get("failure_reasons")
|
||||
if isinstance(raw_failure_counts, dict):
|
||||
for reason, count in raw_failure_counts.items():
|
||||
_increment_reason_count(counts, reason, count)
|
||||
top_failure_reasons = payload.get("top_failure_reasons")
|
||||
if isinstance(top_failure_reasons, list):
|
||||
for item in top_failure_reasons:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
_increment_reason_count(counts, item.get("code"), item.get("count"))
|
||||
return counts
|
||||
|
||||
|
||||
def _build_remediation_family_counts(reason_counts: dict[str, int]) -> dict[str, int]:
|
||||
counts: dict[str, int] = {}
|
||||
for reason, count in reason_counts.items():
|
||||
family = _resolve_playbook_remediation_family(reason)
|
||||
_increment_reason_count(counts, family, count)
|
||||
return counts
|
||||
|
||||
|
||||
def _resolve_playbook_remediation_family(reason_code: object) -> str:
|
||||
token = _normalize_reason_token(reason_code)
|
||||
if any(
|
||||
marker in token
|
||||
for marker in (
|
||||
"sample_size",
|
||||
"executed_sample",
|
||||
"expectancy",
|
||||
"profit_factor",
|
||||
"drawdown",
|
||||
"consecutive_loss",
|
||||
"same_candle_entry_exit_ambiguous",
|
||||
"no_future_candles",
|
||||
"stopped_out",
|
||||
"quality_gate",
|
||||
)
|
||||
):
|
||||
return "backtest_quality_gate"
|
||||
if any(
|
||||
marker in token
|
||||
for marker in (
|
||||
"missing_model",
|
||||
"missing_playbook",
|
||||
"cohort",
|
||||
"operator_runbook",
|
||||
"journal_feedback",
|
||||
"study_note_contract",
|
||||
"doctrine",
|
||||
)
|
||||
):
|
||||
return "model_playbook_contract"
|
||||
if any(
|
||||
marker in token
|
||||
for marker in (
|
||||
"liquidity",
|
||||
"sweep",
|
||||
"raid",
|
||||
"manipulation",
|
||||
"working_draw",
|
||||
"pool",
|
||||
"reorder_trigger",
|
||||
)
|
||||
):
|
||||
return "liquidity_path"
|
||||
if any(
|
||||
marker in token
|
||||
for marker in (
|
||||
"pd_array",
|
||||
"array_",
|
||||
"fvg",
|
||||
"order_block",
|
||||
"breaker",
|
||||
"mitigation",
|
||||
"rebalanc",
|
||||
"reaction",
|
||||
"retest",
|
||||
"displacement",
|
||||
"mss",
|
||||
)
|
||||
):
|
||||
return "pd_array_reaction"
|
||||
if any(
|
||||
marker in token
|
||||
for marker in (
|
||||
"session",
|
||||
"killzone",
|
||||
"judas",
|
||||
"opening_range",
|
||||
"weekly_profile",
|
||||
"news_time",
|
||||
)
|
||||
):
|
||||
return "time_session"
|
||||
if any(marker in token for marker in ("trading_plan", "plan", "draw", "location", "bias", "invalidation")):
|
||||
return "trading_plan_alignment"
|
||||
if any(marker in token for marker in ("blocked", "risk", "not_allowed", "do_not_use", "stale", "observe_only")):
|
||||
return "risk_boundary"
|
||||
return "general_study_only"
|
||||
|
||||
|
||||
def _build_playbook_remediation_route(*, dominant_family: str, dominant_reason: str) -> dict:
|
||||
if dominant_family == "backtest_quality_gate":
|
||||
return {
|
||||
"error_class": "backtest_evidence_gap",
|
||||
"primary_route": "strategy_backtest",
|
||||
"secondary_route": "study_notes",
|
||||
"route_targets": ["strategy_backtest", "study_notes"],
|
||||
"strategy_backtest_prompt": "Review cohort sample size, expectancy, profit factor, drawdown, and executed-signal sufficiency before any promotion.",
|
||||
}
|
||||
if dominant_family in {"time_session", "trading_plan_alignment"}:
|
||||
error_class = (
|
||||
"risk_boundary_gap"
|
||||
if any(marker in _normalize_reason_token(dominant_reason) for marker in ("news_time", "outside_session", "not_allowed"))
|
||||
else "plan_alignment_gap"
|
||||
)
|
||||
return {
|
||||
"error_class": error_class,
|
||||
"primary_route": "study_notes",
|
||||
"secondary_route": "strategy_backtest",
|
||||
"route_targets": ["study_notes", "strategy_backtest"],
|
||||
"strategy_backtest_prompt": "Filter the cohort by session and plan context to confirm whether the failure is structural or just a bad timeframe/session mix.",
|
||||
}
|
||||
if dominant_family == "risk_boundary":
|
||||
return {
|
||||
"error_class": "risk_boundary_gap",
|
||||
"primary_route": "study_notes",
|
||||
"secondary_route": "strategy_backtest",
|
||||
"route_targets": ["study_notes", "strategy_backtest"],
|
||||
"strategy_backtest_prompt": "Review the blocked cohort in Strategy Backtest only after the no-trade boundary and data-quality reasons are explicit.",
|
||||
}
|
||||
if dominant_family in {"liquidity_path", "pd_array_reaction"}:
|
||||
return {
|
||||
"error_class": "confirmation_gap",
|
||||
"primary_route": "study_notes",
|
||||
"secondary_route": "strategy_backtest",
|
||||
"route_targets": ["study_notes", "strategy_backtest"],
|
||||
"strategy_backtest_prompt": "Use Strategy Backtest to inspect which confirmation link failed most often for this playbook cohort.",
|
||||
}
|
||||
return {
|
||||
"error_class": "model_definition_gap",
|
||||
"primary_route": "study_notes",
|
||||
"secondary_route": "strategy_backtest",
|
||||
"route_targets": ["study_notes", "strategy_backtest"],
|
||||
"strategy_backtest_prompt": "Keep the cohort in Strategy Backtest while the playbook definition, contracts, and failure boundaries are clarified.",
|
||||
}
|
||||
|
||||
|
||||
def _top_count_entries(values: dict[str, int], *, limit: int) -> list[dict]:
|
||||
return [
|
||||
{"code": key, "count": int(value)}
|
||||
for key, value in sorted(values.items(), key=lambda item: (-int(item[1]), str(item[0])))[:limit]
|
||||
]
|
||||
|
||||
|
||||
def _increment_reason_count(container: dict[str, int], reason: object, count: object) -> None:
|
||||
key = str(reason or "").strip()
|
||||
if not key:
|
||||
return
|
||||
try:
|
||||
numeric = int(count)
|
||||
except (TypeError, ValueError):
|
||||
numeric = 0
|
||||
if numeric <= 0:
|
||||
return
|
||||
container[key] = int(container.get(key) or 0) + numeric
|
||||
|
||||
|
||||
def _normalize_reason_token(value: object) -> str:
|
||||
return str(value or "").strip().lower().replace("-", "_").replace(" ", "_")
|
||||
|
||||
|
||||
def _extract_int_first(*payloads: object, keys: tuple[str, ...]) -> int:
|
||||
for payload in payloads:
|
||||
value = _extract_first_present_value(payload, keys=keys)
|
||||
if value is None or value == "":
|
||||
continue
|
||||
try:
|
||||
return int(value)
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
return 0
|
||||
|
||||
|
||||
def _extract_float_first(*payloads: object, keys: tuple[str, ...]) -> float:
|
||||
for payload in payloads:
|
||||
value = _extract_first_present_value(payload, keys=keys)
|
||||
if value is None or value == "":
|
||||
continue
|
||||
try:
|
||||
return float(value)
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
return 0.0
|
||||
|
||||
|
||||
def _first_count_key(values: dict) -> str:
|
||||
if not values:
|
||||
return ""
|
||||
return str(sorted(values.items(), key=lambda item: (-int(item[1]), str(item[0])))[0][0])
|
||||
|
||||
|
||||
def _payload_to_search_text(payload: object) -> str:
|
||||
try:
|
||||
raw = json.dumps(payload, ensure_ascii=False, default=str)
|
||||
|
||||
@@ -5532,6 +5532,8 @@ function renderStrategyBacktests(payload) {
|
||||
const operatorSummary = payload?.operator_summary || {};
|
||||
const doctrineCohort = latest.doctrine_backtest_cohort || summary.doctrine_backtest_cohort || {};
|
||||
const doctrinePromotionGate = doctrineCohort.promotion_gate || {};
|
||||
const playbookHistorical = payload?.playbook_historical_cohort_summary || {};
|
||||
const playbookRemediation = payload?.playbook_remediation_queue_summary || {};
|
||||
const liquidityStatus =
|
||||
latest.liquidity_kernel_status ||
|
||||
summary.liquidity_kernel_status ||
|
||||
@@ -5602,6 +5604,14 @@ function renderStrategyBacktests(payload) {
|
||||
['Model Playbook Reason', formatBacktestDecision(modelPlaybookStatus.reason || 'missing_model_playbook')],
|
||||
['Model Playbook', modelPlaybookFeatures.model_playbook_id || 'missing_model_playbook'],
|
||||
['Model Family', modelPlaybookFeatures.model_family || 'missing_model_family'],
|
||||
[
|
||||
'Playbook Historical Gate',
|
||||
`${playbookHistorical.manual_review_eligible_count ?? 0} manual-review / ${playbookHistorical.study_only_count ?? 0} study-only`,
|
||||
],
|
||||
[
|
||||
'Playbook Remediation Queue',
|
||||
`${playbookRemediation.queue_count ?? 0} queued / ${playbookRemediation.strategy_backtest_route_count ?? 0} backtest-first / ${playbookRemediation.study_notes_route_count ?? 0} notes-first`,
|
||||
],
|
||||
['Promotion Gate', doctrinePromotionGate.passed ? 'Passed' : 'Blocked'],
|
||||
['Instrument ID', latest.instrument_id ?? 'n/a'],
|
||||
['Timeframe', formatJsonInline(latest.timeframe_set || {})],
|
||||
@@ -5690,6 +5700,26 @@ function renderStrategyBacktests(payload) {
|
||||
])
|
||||
)
|
||||
);
|
||||
detailGrid.appendChild(
|
||||
buildItemCard(
|
||||
'Playbook Historical Cohorts',
|
||||
`${playbookHistorical.manual_review_eligible_count ?? 0} manual-review / ${playbookHistorical.study_only_count ?? 0} study-only`,
|
||||
(playbookHistorical.cohorts || []).slice(0, 5).map((item) => [
|
||||
`${item.setup_playbook_id || 'missing_playbook'} / ${item.session_group || 'ALL'} / ${item.timeframe_key || 'missing_tf'} / ${item.model_family || 'missing_model_family'}`,
|
||||
`${formatBacktestDecision(item.status || 'study_only')} / ${item.run_count ?? 0} runs / ${item.quality_gate_passed_count ?? 0} passed / ${item.total_executed_signals ?? 0} executed`,
|
||||
])
|
||||
)
|
||||
);
|
||||
detailGrid.appendChild(
|
||||
buildItemCard(
|
||||
'Playbook Remediation Queue',
|
||||
`${playbookRemediation.queue_count ?? 0} queued / ${playbookRemediation.strategy_backtest_route_count ?? 0} backtest-first / ${playbookRemediation.study_notes_route_count ?? 0} notes-first`,
|
||||
(playbookRemediation.items || []).slice(0, 5).map((item) => [
|
||||
`${item.setup_playbook_id || 'missing_playbook'} / ${item.session_group || 'ALL'} / ${item.timeframe_key || 'missing_tf'} / ${item.model_family || 'missing_model_family'}`,
|
||||
`${formatBacktestDecision(item.primary_route || 'study_notes')} / ${item.dominant_remediation_family_label || item.dominant_remediation_family || 'general'} / ${item.dominant_reason_code || 'historical_cohort_not_passed'} × ${item.dominant_reason_count ?? 0}`,
|
||||
])
|
||||
)
|
||||
);
|
||||
detailGrid.appendChild(
|
||||
buildItemCard(
|
||||
'Liquidity Kernel',
|
||||
|
||||
@@ -95,6 +95,26 @@
|
||||
'Model Playbook': '模型 Playbook',
|
||||
'Model Family': '模型族',
|
||||
'Model Playbook chain mapped': '模型 Playbook 链路已映射',
|
||||
'Playbook Historical Gate': 'Playbook 历史分组门',
|
||||
'Playbook Historical Cohorts': 'Playbook 历史分组',
|
||||
'Playbook Remediation Queue': 'Playbook 修复队列',
|
||||
'backtest-first': '先看回测',
|
||||
'notes-first': '先写笔记',
|
||||
'strategy_backtest': '策略回测',
|
||||
'study_notes': '学习笔记',
|
||||
'Backtest Quality Gate': '回测质量门',
|
||||
'Model Playbook Contract': '模型 Playbook 契约',
|
||||
'Liquidity / Sweep / Path': '流动性 / Sweep / Path',
|
||||
'PD Array / Reaction': 'PD Array / 反应',
|
||||
'Time / Session': '时间 / Session',
|
||||
'Trading Plan Alignment': '交易计划对齐',
|
||||
'Risk Boundary': '风险边界',
|
||||
'General Study-Only': '通用仅研究',
|
||||
'manual-review': '人工复核',
|
||||
'manual_review_candidate': '可进入人工复核',
|
||||
'unmapped_study_only': '未映射,仅研究',
|
||||
'playbook_historical_cohort_passed': 'Playbook 历史分组已通过',
|
||||
'historical_cohort_not_passed': '历史分组未通过',
|
||||
'Product Gate': '产品门',
|
||||
'product gate': '产品门',
|
||||
'trader_product_promotion_gate_blocked': '交易员产品晋级门阻塞',
|
||||
@@ -865,6 +885,26 @@
|
||||
['Model Playbook', '模型 Playbook'],
|
||||
['Model Family', '模型族'],
|
||||
['Model Playbook chain mapped', '模型 Playbook 链路已映射'],
|
||||
['Playbook Historical Gate', 'Playbook 历史分组门'],
|
||||
['Playbook Historical Cohorts', 'Playbook 历史分组'],
|
||||
['Playbook Remediation Queue', 'Playbook 修复队列'],
|
||||
['backtest-first', '先看回测'],
|
||||
['notes-first', '先写笔记'],
|
||||
['strategy_backtest', '策略回测'],
|
||||
['study_notes', '学习笔记'],
|
||||
['Backtest Quality Gate', '回测质量门'],
|
||||
['Model Playbook Contract', '模型 Playbook 契约'],
|
||||
['Liquidity / Sweep / Path', '流动性 / Sweep / Path'],
|
||||
['PD Array / Reaction', 'PD Array / 反应'],
|
||||
['Time / Session', '时间 / Session'],
|
||||
['Trading Plan Alignment', '交易计划对齐'],
|
||||
['Risk Boundary', '风险边界'],
|
||||
['General Study-Only', '通用仅研究'],
|
||||
['manual-review', '人工复核'],
|
||||
['manual_review_candidate', '可进入人工复核'],
|
||||
['unmapped_study_only', '未映射,仅研究'],
|
||||
['playbook_historical_cohort_passed', 'Playbook 历史分组已通过'],
|
||||
['historical_cohort_not_passed', '历史分组未通过'],
|
||||
['model_playbook_mapped', '模型 Playbook 已映射'],
|
||||
['model_playbook_kernel_blocked', '模型 Playbook 内核阻塞'],
|
||||
['missing_model_playbook', '缺少模型 Playbook'],
|
||||
|
||||
@@ -11,6 +11,7 @@ from src.api.server import route_request
|
||||
from src.api.signals import get_recent_signals
|
||||
from src.services.backtest import BacktestCostConfig, BacktestQualityGate, BacktestReplayConfig
|
||||
from src.services.execution import AccountRiskState
|
||||
from src.services.ict_mainline import summarize_playbook_historical_cohorts, summarize_playbook_remediation_queue
|
||||
from src.main import run_closed_loop, select_closed_loop_backtest_subject_type
|
||||
|
||||
|
||||
@@ -138,6 +139,147 @@ class ApiTests(unittest.TestCase):
|
||||
self.assertIn("missing_session_window", result["time_session_kernel_summary"]["reason_counts"])
|
||||
self.assertEqual(result["model_playbook_kernel_summary"]["item_count"], 1)
|
||||
self.assertIn("missing_model_playbook", result["model_playbook_kernel_summary"]["reason_counts"])
|
||||
self.assertEqual(result["playbook_historical_cohort_summary"]["phase"], "D8_PLAYBOOK_SPECIFIC_HISTORICAL_COHORT_GATE")
|
||||
self.assertEqual(result["playbook_historical_cohort_summary"]["cohort_count"], 1)
|
||||
self.assertEqual(result["playbook_historical_cohort_summary"]["study_only_count"], 1)
|
||||
self.assertEqual(result["playbook_historical_cohort_summary"]["cohorts"][0]["status"], "unmapped_study_only")
|
||||
d9_summary = result["playbook_remediation_queue_summary"]
|
||||
self.assertEqual(d9_summary["phase"], "D9_PLAYBOOK_REMEDIATION_QUEUE")
|
||||
self.assertEqual(d9_summary["queue_count"], 1)
|
||||
self.assertEqual(d9_summary["items"][0]["primary_route"], "study_notes")
|
||||
self.assertEqual(d9_summary["items"][0]["dominant_remediation_family"], "model_playbook_contract")
|
||||
|
||||
def test_strategy_backtest_surface_groups_playbook_historical_cohorts(self) -> None:
|
||||
rows = [
|
||||
{
|
||||
"id": 11,
|
||||
"subject_type": "plan_replay",
|
||||
"model_code": "OSOK_DAILY_RANGE_SWEEP_MSS_FVG_RETRACE",
|
||||
"candidate_model_code": "OSOK_DAILY_RANGE_SWEEP_MSS_FVG_RETRACE",
|
||||
"timeframe_set": {"bias_tf": "1h", "setup_tf": "15m", "trigger_tf": "5m"},
|
||||
"config": {
|
||||
"run_metadata": {
|
||||
"setup_playbook_id": "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL",
|
||||
},
|
||||
"session_codes": ["LONDON"],
|
||||
},
|
||||
"summary": {
|
||||
"total_signals": 12,
|
||||
"executed_signals": 5,
|
||||
"total_r": -2.5,
|
||||
"quality_gate": {"passed": False, "reasons": ["profit_factor_below_threshold"]},
|
||||
},
|
||||
}
|
||||
]
|
||||
with patch("src.api.backtest.SessionLocal", return_value=FakeSession(rows)):
|
||||
result = get_strategy_backtest_surface(instrument_id=1, limit=1)
|
||||
|
||||
d8_summary = result["playbook_historical_cohort_summary"]
|
||||
self.assertEqual(d8_summary["cohort_count"], 1)
|
||||
cohort = d8_summary["cohorts"][0]
|
||||
self.assertEqual(cohort["setup_playbook_id"], "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL")
|
||||
self.assertEqual(cohort["session_group"], "LONDON")
|
||||
self.assertEqual(cohort["timeframe_key"], "1h/15m/5m")
|
||||
self.assertEqual(cohort["model_family"], "london_discount_reversal")
|
||||
self.assertEqual(cohort["status"], "study_only")
|
||||
self.assertFalse(cohort["manual_review_eligible"])
|
||||
self.assertEqual(cohort["total_result_count"], 12)
|
||||
self.assertEqual(cohort["total_executed_signals"], 5)
|
||||
self.assertIn("profit_factor_below_threshold", cohort["quality_gate_reason_counts"])
|
||||
d9_summary = result["playbook_remediation_queue_summary"]
|
||||
self.assertEqual(d9_summary["queue_count"], 1)
|
||||
self.assertEqual(d9_summary["strategy_backtest_route_count"], 0)
|
||||
self.assertEqual(d9_summary["study_notes_route_count"], 1)
|
||||
self.assertEqual(d9_summary["items"][0]["primary_route"], "study_notes")
|
||||
self.assertIn("profit_factor_below_threshold", d9_summary["items"][0]["quality_gate_reason_counts"])
|
||||
|
||||
def test_playbook_historical_cohort_summary_marks_manual_review_candidates(self) -> None:
|
||||
result = summarize_playbook_historical_cohorts(
|
||||
[
|
||||
{
|
||||
"backtest_run_id": 77,
|
||||
"timeframe_set": {"bias_tf": "4h", "setup_tf": "15m", "trigger_tf": "5m"},
|
||||
"session_codes": ["LONDON"],
|
||||
"summary": {
|
||||
"total_signals": 21,
|
||||
"executed_signals": 9,
|
||||
"total_r": 6.5,
|
||||
"quality_gate": {"passed": True, "reasons": []},
|
||||
},
|
||||
"doctrine_backtest_cohort": {
|
||||
"setup_playbook_id": "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL",
|
||||
"cohort_key": "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL::complete",
|
||||
"doctrine_completeness": "complete",
|
||||
"promotion_gate": {"passed": True, "reasons": []},
|
||||
},
|
||||
}
|
||||
]
|
||||
)
|
||||
|
||||
self.assertEqual(result["manual_review_eligible_count"], 1)
|
||||
self.assertEqual(result["study_only_count"], 0)
|
||||
cohort = result["cohorts"][0]
|
||||
self.assertEqual(cohort["status"], "manual_review_candidate")
|
||||
self.assertTrue(cohort["manual_review_eligible"])
|
||||
self.assertEqual(cohort["eligibility_reason"], "playbook_historical_cohort_passed")
|
||||
self.assertEqual(cohort["session_group"], "LONDON")
|
||||
self.assertEqual(cohort["timeframe_key"], "4h/15m/5m")
|
||||
self.assertEqual(cohort["total_result_count"], 21)
|
||||
self.assertEqual(cohort["total_executed_signals"], 9)
|
||||
self.assertAlmostEqual(cohort["total_r"], 6.5)
|
||||
d9_summary = summarize_playbook_remediation_queue(
|
||||
[
|
||||
{
|
||||
"backtest_run_id": 77,
|
||||
"timeframe_set": {"bias_tf": "4h", "setup_tf": "15m", "trigger_tf": "5m"},
|
||||
"session_codes": ["LONDON"],
|
||||
"summary": {
|
||||
"total_signals": 21,
|
||||
"executed_signals": 9,
|
||||
"total_r": 6.5,
|
||||
"quality_gate": {"passed": True, "reasons": []},
|
||||
},
|
||||
"doctrine_backtest_cohort": {
|
||||
"setup_playbook_id": "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL",
|
||||
"cohort_key": "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL::complete",
|
||||
"doctrine_completeness": "complete",
|
||||
"promotion_gate": {"passed": True, "reasons": []},
|
||||
},
|
||||
}
|
||||
]
|
||||
)
|
||||
self.assertEqual(d9_summary["queue_count"], 0)
|
||||
|
||||
def test_playbook_remediation_queue_routes_quality_gate_failures_to_strategy_backtest(self) -> None:
|
||||
result = summarize_playbook_remediation_queue(
|
||||
[
|
||||
{
|
||||
"backtest_run_id": 99,
|
||||
"timeframe_set": {"bias_tf": "1h", "setup_tf": "15m", "trigger_tf": "5m"},
|
||||
"session_codes": ["NY_AM"],
|
||||
"summary": {
|
||||
"total_signals": 18,
|
||||
"executed_signals": 8,
|
||||
"total_r": -1.0,
|
||||
"failure_reasons": {"stopped_out": 4},
|
||||
"quality_gate": {"passed": False, "reasons": ["profit_factor_below_threshold", "expectancy_below_threshold"]},
|
||||
},
|
||||
"doctrine_backtest_cohort": {
|
||||
"setup_playbook_id": "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL",
|
||||
"cohort_key": "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL::complete",
|
||||
"doctrine_completeness": "complete",
|
||||
"promotion_gate": {"passed": True, "reasons": []},
|
||||
},
|
||||
}
|
||||
]
|
||||
)
|
||||
|
||||
self.assertEqual(result["queue_count"], 1)
|
||||
self.assertEqual(result["strategy_backtest_route_count"], 1)
|
||||
self.assertEqual(result["study_notes_route_count"], 0)
|
||||
self.assertEqual(result["items"][0]["primary_route"], "strategy_backtest")
|
||||
self.assertEqual(result["items"][0]["dominant_remediation_family"], "backtest_quality_gate")
|
||||
self.assertEqual(result["items"][0]["dominant_error_class"], "backtest_evidence_gap")
|
||||
|
||||
def test_route_request_trading_doctrine_rules_returns_d1_matrix(self) -> None:
|
||||
status, payload = route_request(
|
||||
|
||||
@@ -70,6 +70,25 @@ class RunBacktestBatchScriptTests(unittest.TestCase):
|
||||
self.assertEqual(payload["aggregate_by_subject"]["plan_replay"]["run_count"], 1)
|
||||
self.assertEqual(payload["aggregate_by_subject"]["signal"]["run_count"], 2)
|
||||
self.assertIn("ICT-CH1-6-LONDON-DISCOUNT-REVERSAL::complete", payload["aggregate_by_doctrine_cohort"])
|
||||
d8_summary = payload["playbook_historical_cohort_summary"]
|
||||
self.assertEqual(d8_summary["phase"], "D8_PLAYBOOK_SPECIFIC_HISTORICAL_COHORT_GATE")
|
||||
self.assertEqual(d8_summary["run_count"], 3)
|
||||
self.assertEqual(d8_summary["cohort_count"], 2)
|
||||
self.assertEqual(d8_summary["manual_review_eligible_count"], 0)
|
||||
self.assertEqual(d8_summary["study_only_count"], 2)
|
||||
self.assertTrue(all(item["status"] == "study_only" for item in d8_summary["cohorts"]))
|
||||
self.assertTrue(
|
||||
any(
|
||||
item["cohort_key"] == "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL::LONDON::1h/15m/5m::london_discount_reversal"
|
||||
for item in d8_summary["cohorts"]
|
||||
)
|
||||
)
|
||||
d9_summary = payload["playbook_remediation_queue_summary"]
|
||||
self.assertEqual(d9_summary["phase"], "D9_PLAYBOOK_REMEDIATION_QUEUE")
|
||||
self.assertEqual(d9_summary["queue_count"], 2)
|
||||
self.assertEqual(d9_summary["strategy_backtest_route_count"] + d9_summary["study_notes_route_count"], 2)
|
||||
self.assertTrue(all(item["primary_route"] in {"strategy_backtest", "study_notes"} for item in d9_summary["items"]))
|
||||
self.assertTrue(d9_summary["operator_summary"].startswith("2 failed cohorts queued"))
|
||||
self.assertEqual(service.calls[0]["run_metadata"]["batch_id"], "batch-test")
|
||||
self.assertEqual(service.calls[0]["run_metadata"]["setup_playbook_id"], "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL")
|
||||
self.assertEqual(service.calls[0]["run_metadata"]["execution_boundary"], "read_only_backtest_no_order_submission")
|
||||
|
||||
@@ -25,6 +25,8 @@ ACTIVE_TOP_LEVEL_DOCS = {
|
||||
"AI_ICT_DoctrineKernel_TimeSessionKernel_D5_2026-05-22_v1.md",
|
||||
"AI_ICT_DoctrineKernel_ModelPlaybookKernel_D6_2026-05-22_v1.md",
|
||||
"AI_ICT_DoctrineKernel_TraderProductGate_D7_2026-05-22_v1.md",
|
||||
"AI_ICT_DoctrineKernel_PlaybookHistoricalCohortGate_D8_2026-05-22_v1.md",
|
||||
"AI_ICT_DoctrineKernel_PlaybookRemediationQueue_D9_2026-05-22_v1.md",
|
||||
"AI_ICT_硬性主线约束_必须遵循ICT交易全书_2026-05-21_v1.md",
|
||||
"AI_ICT_实盘辅助分析决策助手ICT主线开发方案_2026-05-21_v1.md",
|
||||
"AI_ICT_主线裁剪审计_S1_2026-05-21_v1.md",
|
||||
|
||||
Reference in New Issue
Block a user