feat: add playbook cohort remediation queue

This commit is contained in:
Codex
2026-05-22 22:48:57 +08:00
parent 3a37036b56
commit 9ca85d8dd8
13 changed files with 995 additions and 3 deletions
+11 -1
View File
@@ -25,6 +25,13 @@ Automation execution-layer PXX remains closed and its known scope stays frozen.
- state: automation_execution_layer_pxx_known_scope_closed
- current_phase: Long-running Analysis Decision Monitor With Ops Rollup
- latest_analysis_decision_monitor_rollup_slice: automation_execution_layer_p7_monitor_maturity_rollup
- latest_analysis_decision_monitor_rollup_status: blocked_attention_required_refreshed_2026-05-22T13:27:14Z
- latest_analysis_decision_monitor_rollup_path: `.claude/analysis_decision_monitor_maturity_rollup.json`
- latest_analysis_decision_monitor_rollup_next_slice: repair_analysis_decision_monitor_blockers
- latest_analysis_decision_monitor_rollup_failure_reasons: latest_monitor_not_passed, stable_monitoring_window_not_reached, latest_monitor_has_blockers, runtime_health_not_ok, p6_governance_tick_not_noop
- latest_analysis_decision_monitor_rollup_drift_finding: fresh_remote_drift_check_not_run_local_deploy_password_missing; stored_2026-05-11_drift_artifact_was_clear_but_is_now_historical_only
- latest_analysis_decision_monitor_rollup_alert_quality_finding: required_review_fields_remain_present_but_quality_status_is_needs_attention_because_the_monitor_is_not_passed_and_has_blockers
- active_product_phase: Decision Assistant Phase H/F/D/G/E Dense Recall + Templates + Announcements + Peer Baselines
- latest_decision_assistant_slice: unified_cockpit_summary
- latest_decision_assistant_change_slice: review_packet_delta_summary
@@ -725,6 +732,7 @@ Automation execution-layer PXX remains closed and its known scope stays frozen.
- P23 unattended harness remains a local/remote smoke verifier only and must not use external channels, require real trades, or alter execution permission, live readiness, trust bucket, risk governor, or operator confirmation.
- P24 remote ops smoke remains verification only: its visual-watch step uses dry-run cleanup and it must not use external channels, require real trades, or alter execution permission, live readiness, trust bucket, risk governor, or operator confirmation.
- PXX closure gate is verification/state-freeze only and must not create a Research Council phase, alter execution permission, enable real-live readiness, change risk governor/trust bucket behavior, or weaken operator confirmation.
- The latest P7 monitor maturity rollup refresh is a `System Ops` status artifact only; it must not be promoted back into trader-facing doctrine work while monitor blockers remain active.
- A final rerun of `scripts/run_alert_dispatcher.py` and `scripts/run_trading_runtime.py` hit local PostgreSQL connection timeout because nothing was listening on `127.0.0.1:5432`; the P14 code path is still covered by full unit tests and earlier successful smoke during implementation.
- P15 dispatcher smoke succeeded with `suppressed_noise_policy`, `workbench_only`, and `policy_external_allowed` counters present. P15 runtime smoke reached `alert_dispatch` successfully, but the overall runtime returned external environment blockers for market stream disconnected and OKX read transport connection refused.
- P16 direct browser smoke succeeded locally with `browser_capture_status=captured`. Script-level capture without a watch run still returns safe no-op `watch_run_missing`.
@@ -852,10 +860,12 @@ Automation execution-layer PXX remains closed and its known scope stays frozen.
- D5 Time / Session Kernel is complete, deployed, and remote-verified: added `DOCTRINE_SESSION_PHASE=D5_TIME_SESSION_KERNEL`, `TimeSessionRule`, `GET /trading/time-session-kernel`, Strategy Backtest `time_session_backtest_features` / `time_session_kernel_status` / `time_session_kernel_summary`, and `docs/AI_ICT_DoctrineKernel_TimeSessionKernel_D5_2026-05-22_v1.md`. Asia / London / NY AM / NY PM, killzone, Judas/manipulation window, opening range, weekly profile, and news/no-trade time are now measurable session-quality gates; `missing_session_window`, `session_window_unverified`, `outside_session_window`, `session_not_allowed`, `missing_killzone`, `killzone_not_active`, `judas_window_wait`, `opening_range_unresolved`, `weekly_profile_missing`, `news_time_no_trade`, and `session_invalidation_missing` block promotion until timing is explicit. Validation passed focused D5/API/backtest/docs tests, full unit discovery (`846` tests), script static checks (`79` scripts), secret static checks, JS syntax checks, deploy smoke over 11 endpoints, remote ops smoke, read-only live analysis smoke, `/trading/time-session-kernel` field smoke, `/trading/backtests` D5 field smoke, and Playwright screenshot evidence at `output/playwright/d5-time-session-strategy-backtest-1366x768.png`. Remote latest backtest `#513` is correctly blocked by D5 as `study_only / session_window_unverified` instead of treating `ASIA` plus `mark_outside` as a verified session window. No execution/readiness/risk/trust/operator-confirmation gate was changed.
- D6 Model Playbook Kernel is complete, deployed, and remote-verified: added `DOCTRINE_MODEL_PLAYBOOK_PHASE=D6_MODEL_PLAYBOOK_KERNEL`, `ModelPlaybookRule`, `GET /trading/model-playbook-kernel`, Strategy Backtest `model_playbook_backtest_features` / `model_playbook_kernel_status` / `model_playbook_kernel_summary`, and `docs/AI_ICT_DoctrineKernel_ModelPlaybookKernel_D6_2026-05-22_v1.md`. Raw `model_code` values such as `SWEEP_MSS_FVG` now stay `study_only` until they resolve to a registered setup playbook with explicit Range / Draw / Liquidity / PD Array / Session / Confirmation / Invalidation / Target, batch cohort, quality-gate, journal-feedback, and operator-runbook contracts. Validation passed focused D6/API/backtest/docs tests, full unit discovery (`853` tests), script static checks (`79` scripts), secret static checks, JS syntax checks, deploy smoke over 11 endpoints, remote ops smoke, read-only live analysis smoke, `/trading/model-playbook-kernel` field smoke, `/trading/backtests` D6 field smoke, and Playwright screenshot evidence at `output/playwright/d6-model-playbook-strategy-backtest-fullpage-zh.png`. Remote latest backtest `#513` is correctly blocked by D6 as `study_only / missing_model_playbook` instead of promoting a surface-shape model as a tradeable setup. No execution/readiness/risk/trust/operator-confirmation gate was changed.
- D7 Trader Product Rebuild / Playbook Promotion Gate is complete, deployed, and remote-verified: added `build_trader_product_promotion_gate`, `candidate_watchlist_tiers.*[].trader_product_promotion_gate`, `candidate_watchlist_summary.trader_product_gate_summary`, `trader_mode_summary.trader_product_gate_summary`, and `docs/AI_ICT_DoctrineKernel_TraderProductGate_D7_2026-05-22_v1.md`. Trader Cockpit `Review Now` now requires playbook-mapped + D3/D4/D5/D6 Doctrine-complete + Strategy Backtest-supported + active Trading Plan-approved candidates; otherwise the candidate is routed to study-only / `archive_only`, the focus matrix becomes `study_only`, and Trader Mode refuses to fall back to a decision-packet focus. Remote latest Trader Mode payload correctly shows `allowed_focus_count=0`, `study_only_count=5`, status `Observe Only`, and no primary opportunity, while the latest raw backtest remains D6-blocked. Validation passed focused operator/doctrine/static/docs tests, full unit discovery (`853` tests), script static checks (`79` scripts), secret static checks, JS syntax checks, deploy smoke over 11 endpoints, remote ops smoke, read-only live analysis smoke, direct Trader Mode D7 field smoke, and Playwright screenshot evidence at `output/playwright/d7-trader-product-gate-trader-mode-zh.png`. No execution/readiness/risk/trust/operator-confirmation gate was changed.
- D8 Playbook-Specific Historical Cohort Gate is complete locally: added `DOCTRINE_PLAYBOOK_HISTORICAL_COHORT_PHASE=D8_PLAYBOOK_SPECIFIC_HISTORICAL_COHORT_GATE`, `build_playbook_historical_cohort_descriptor`, `summarize_playbook_historical_cohorts`, Strategy Backtest `playbook_historical_cohort_summary`, `scripts/run_backtest_batch.py` JSON/text cohort rollup output, the `Playbook Historical Gate` / `Playbook Historical Cohorts` Strategy Backtest UI readout, and `docs/AI_ICT_DoctrineKernel_PlaybookHistoricalCohortGate_D8_2026-05-22_v1.md`. D8 now groups historical evidence by `setup_playbook_id`, `session_group`, `timeframe_key`, and `model_family`, and distinguishes `study_only`, `unmapped_study_only`, and `manual_review_candidate` without promoting anything into live focus. Validation passed focused D8/API/batch/docs tests (`46` tests), full unit discovery (`856` tests), script static checks (`79` scripts), secret static checks, `node --check src/web/app.js`, `node --check src/web/i18n.js`, and `git diff --check` with only LF/CRLF warnings. No execution/readiness/risk/trust/operator-confirmation gate was changed.
- D9 Playbook Remediation Queue is complete locally: added `DOCTRINE_PLAYBOOK_REMEDIATION_PHASE=D9_PLAYBOOK_REMEDIATION_QUEUE`, `summarize_playbook_remediation_queue`, `docs/AI_ICT_DoctrineKernel_PlaybookRemediationQueue_D9_2026-05-22_v1.md`, Strategy Backtest `playbook_remediation_queue_summary`, and batch-output remediation routing in `scripts/run_backtest_batch.py`. D9 now converts failed D8 cohorts into remediation items with a dominant rule family, dominant blocker reason, Study Notes prompt, Strategy Backtest review prompt, and a primary route of either `strategy_backtest` or `study_notes`, without promoting anything into Trader Cockpit focus. Validation passed focused D9/API/batch/docs tests (`47` tests), full unit discovery (`857` tests), script static checks (`79` scripts), secret static checks, `node --check src/web/app.js`, `node --check src/web/i18n.js`, and `git diff --check` with only LF/CRLF warnings. No execution/readiness/risk/trust/operator-confirmation gate was changed.
## Exact Next Step
Use `docs/AI_ICT_DoctrineKernel全书规则内核重建_D0_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel市场语言规则内核_D1_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel_PlaybookBacktestCohort_D2_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel_LiquidityKernel_D3_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel_PDArrayReactionKernel_D4_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_TimeSessionKernel_D5_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_ModelPlaybookKernel_D6_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_TraderProductGate_D7_2026-05-22_v1.md`, `docs/ICT交易全书_出版终校版.md`, `docs/AI_ICT_硬性主线约束_必须遵循ICT交易全书_2026-05-21_v1.md`, `docs/AI_ICT_实盘辅助分析决策助手ICT主线开发方案_2026-05-21_v1.md`, `docs/AI_ICT_批量历史回测验证_S16_2026-05-21_v1.md`, `SESSION.md`, and `TASKS.md` as the active anchors. Start D8 Playbook-Specific Historical Cohort Gate: run and summarize batch backtests by registered `setup_playbook_id`, session, timeframe set, and model family so Strategy Backtest can distinguish which book-derived playbooks remain study-only versus eligible for manual review. Do not restart external-context/archive/digest long-tail work, do not add order-submission features, do not physically delete S1 deprecated surfaces yet, and do not fabricate R2 paper samples from blocked candidates. Keep `execution_allowed`, `decision_support_ready`, `real_live_ready`, risk governor, trust bucket, and operator confirmation protocol unchanged.
Use `docs/AI_ICT_DoctrineKernel全书规则内核重建_D0_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel市场语言规则内核_D1_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel_PlaybookBacktestCohort_D2_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel_LiquidityKernel_D3_2026-05-21_v1.md`, `docs/AI_ICT_DoctrineKernel_PDArrayReactionKernel_D4_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_TimeSessionKernel_D5_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_ModelPlaybookKernel_D6_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_TraderProductGate_D7_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_PlaybookHistoricalCohortGate_D8_2026-05-22_v1.md`, `docs/AI_ICT_DoctrineKernel_PlaybookRemediationQueue_D9_2026-05-22_v1.md`, `docs/ICT交易全书_出版终校版.md`, `docs/AI_ICT_硬性主线约束_必须遵循ICT交易全书_2026-05-21_v1.md`, `docs/AI_ICT_实盘辅助分析决策助手ICT主线开发方案_2026-05-21_v1.md`, `docs/AI_ICT_批量历史回测验证_S16_2026-05-21_v1.md`, `SESSION.md`, and `TASKS.md` as the active anchors. Start D10 Remediation Artifacts: generate per-cohort remediation artifacts from D9 so Strategy Backtest gets saved review prompts and Study Notes gets queue-ready draft language, without adding any new trader-facing signal or promotion path. Do not restart external-context/archive/digest long-tail work, do not add order-submission features, do not physically delete S1 deprecated surfaces yet, and do not fabricate R2 paper samples from blocked candidates. Keep `execution_allowed`, `decision_support_ready`, `real_live_ready`, risk governor, trust bucket, and operator confirmation protocol unchanged.
## Validation Commands Used Recently
+9 -1
View File
@@ -290,6 +290,11 @@
- [x] state: automation_execution_layer_pxx_known_scope_closed
- [x] current_phase: Long-running Analysis Decision Monitor With Ops Rollup
- [x] latest_analysis_decision_monitor_rollup_refresh: `2026-05-22T13:27:14Z` -> `.claude/analysis_decision_monitor_maturity_rollup.json`
- [x] latest_analysis_decision_monitor_rollup_status: blocked / attention_required / next_slice=`repair_analysis_decision_monitor_blockers`
- [x] latest_analysis_decision_monitor_rollup_failure_reasons: latest_monitor_not_passed, stable_monitoring_window_not_reached, latest_monitor_has_blockers, runtime_health_not_ok, and p6_governance_tick_not_noop
- [x] latest_analysis_decision_monitor_rollup_alert_quality: required review fields are still present, but quality is `needs_attention` because the monitor is not passed and has blockers
- [x] latest_analysis_decision_monitor_rollup_drift_note: fresh remote drift check was not rerun because `AI_ICT_DEPLOY_PASSWORD` is absent locally; the stored 2026-05-11 drift artifact was `clear` and is now historical only
- [x] active_product_phase: Decision Assistant Phase H/F/D/G/E Dense Recall + Templates + Announcements + Peer Baselines
- [x] latest_decision_assistant_slice: unified_cockpit_summary
- [x] latest_decision_assistant_change_slice: review_packet_delta_summary
@@ -839,6 +844,7 @@
## Blocked
- [x] External chart adapter is optional in the current analysis-only phase and no external chart URL is required for the main loop.
- [ ] `repair_analysis_decision_monitor_blockers`: clear `launch_no_blockers`, `launch_analysis_only_ready`, `launch_restricted_decision_support_ready`, `runtime_health_ok`, `market_stream_connected`, and `p6_governance_tick_noop` before re-entering P7-2/P7-3
## Standing Guardrails
@@ -959,7 +965,9 @@
- [x] D5 Time / Session Kernel: rebuilt Asia / London / NY AM / NY PM / killzone / Judas window / opening range / weekly profile / news time into measurable session-quality gates instead of loose time labels; Strategy Backtest now exposes `time_session_backtest_features`, `time_session_kernel_status`, `time_session_kernel_summary`, and `/trading/time-session-kernel` without changing execution/readiness gates
- [x] D6 Model Playbook Kernel: raw model codes now stay study-only until they resolve to registered setup playbooks with Range / Draw / Liquidity / PD Array / Session / Confirmation / Invalidation / Target, batch-cohort, quality-gate, journal-feedback, and operator-runbook contracts; Strategy Backtest exposes D6 features/status/summary and `/trading/model-playbook-kernel` exposes the D6 registry without changing execution/readiness gates
- [x] D7 Trader Product Rebuild / Playbook Promotion Gate: only playbook-mapped, backtest-supported, plan-approved setups may re-enter Trader Cockpit focus; study-only candidates are routed out of `Review Now` and remain Strategy Backtest / Study Notes / System Ops evidence
- [ ] D8 Playbook-Specific Historical Cohort Gate: run and summarize batch backtests by registered `setup_playbook_id`, session, timeframe set, and model family so Strategy Backtest can distinguish study-only playbooks from manual-review-eligible playbooks
- [x] D8 Playbook-Specific Historical Cohort Gate: grouped historical evidence by `setup_playbook_id`, session, timeframe set, and model family; added `playbook_historical_cohort_summary` to Strategy Backtest and batch output; surfaced `study_only`, `unmapped_study_only`, and `manual_review_candidate` without changing execution/readiness gates
- [x] D9 Playbook Remediation Queue: summarize the dominant D8 cohort blocker family, blocker reason, and remediation route; add `playbook_remediation_queue_summary` to Strategy Backtest and batch output; route failed cohorts to `strategy_backtest` or `study_notes` without changing execution/readiness gates
- [ ] D10 Remediation Artifacts: turn D9 queue items into saved Strategy Backtest review prompts and Study Notes draft artifacts before any new trader-facing expansion
- [x] R3 OKX sandbox isolated smoke tooling: guarded script and service can call OKX demo submit/query/cancel/query/fills only when `OKX_SANDBOX_SMOKE_EXECUTE=1`; local validation, deploy smoke, and remote ops smoke passed
- [ ] R3 sandbox credential permission gate deferred: current phase is assisted analysis / decision support only, so do not request trade-permission credentials or make submit/cancel smoke the next gate
- [ ] Long-term target: complete `Real-time Assisted Trading V1` without enabling default live auto-trading
@@ -0,0 +1,65 @@
# AI_ICT Doctrine Kernel Playbook Historical Cohort Gate D8 2026-05-22 v1
## Scope
D8 turns historical backtest evidence into playbook-specific cohorts. The goal is not to add new trader-facing signals. The goal is to answer one Strategy Backtest question:
`For each registered book-derived playbook, which session / timeframe / model-family cohort is still study-only, and which cohort may enter manual review?`
## ICT Chain Served
D8 serves the hard mainline chain:
`Range / Draw / Location -> Liquidity / Sweep / Inducement / Path -> Displacement / MSS / FVG / Retest / Session -> backtest evidence -> pre-market plan -> intraday unique focus or no-trade reason -> journal / study-notes feedback`.
It sits at `backtest evidence`. A D8 pass does not create a live-trading signal and does not bypass Trading Plan, Trader Cockpit, Journal, or R2 evidence gates.
## Product Rule
Historical evidence is grouped by:
- `setup_playbook_id`
- `session_group`
- `timeframe_key`
- `model_family`
A cohort is `manual_review_candidate` only when:
- it maps to a registered `setup_playbook_id`,
- at least one historical run in that cohort passes the Strategy Backtest quality gate,
- at least one historical run in that cohort passes the Doctrine promotion gate.
All other cohorts remain `study_only` or `unmapped_study_only`.
## Implemented Surface
Backend:
- `DOCTRINE_PLAYBOOK_HISTORICAL_COHORT_PHASE = D8_PLAYBOOK_SPECIFIC_HISTORICAL_COHORT_GATE`
- `build_playbook_historical_cohort_descriptor`
- `summarize_playbook_historical_cohorts`
- `/trading/backtests` now returns `playbook_historical_cohort_summary`
Batch script:
- `scripts/run_backtest_batch.py` now includes `playbook_historical_cohort_summary` in JSON output.
- CLI text output prints manual-review candidate count versus study-only cohort count.
Frontend:
- Strategy Backtest shows `Playbook Historical Gate`.
- Strategy Backtest lists top `Playbook Historical Cohorts` by playbook / session / timeframe / model family.
## Read-Only Boundary
D8 is read-only historical analysis. It does not add order submission, does not change `execution_allowed`, does not change `decision_support_ready`, does not change `real_live_ready`, and does not loosen risk governor / trust bucket / operator confirmation protocol.
## Validation Meaning
If D8 marks a cohort `study_only`, it must not enter Trader Cockpit `Review Now`.
If D8 marks a cohort `manual_review_candidate`, it still only means the historical cohort may be reviewed by a human. It does not become a live focus until the Trading Plan, Trader Product Gate, R2 paper/operator evidence, and current market context all pass.
## Next Slice
D9 should turn failed D8 cohort blockers into a playbook remediation queue: identify which ICT rule family failed most often for each playbook cohort, then route remediation to Study Notes / Strategy Backtest before adding any new trader-facing feature.
@@ -0,0 +1,81 @@
# AI_ICT Doctrine Kernel Playbook Remediation Queue D9 2026-05-22 v1
## Scope
D9 turns failed D8 cohorts into a remediation queue. The goal is not to add any new trader-facing signal. The goal is to answer one post-backtest question:
`For each failed playbook cohort, which rule family fails most often, and should the next fix start in Strategy Backtest or Study Notes?`
## ICT Chain Served
D9 still serves the hard mainline chain:
`Range / Draw / Location -> Liquidity / Sweep / Inducement / Path -> Displacement / MSS / FVG / Retest / Session -> backtest evidence -> pre-market plan -> intraday unique focus or no-trade reason -> journal / study-notes feedback`.
It sits between `backtest evidence` and `journal / study-notes feedback`. A D9 queue item is remediation evidence only. It is not a promotion gate bypass and not a live focus signal.
## Product Rule
A failed D8 cohort must be reduced to:
- one dominant remediation family,
- one dominant blocker reason code,
- one primary remediation route,
- one Study Notes prompt,
- one Strategy Backtest review prompt.
The queue may route a cohort to:
- `strategy_backtest` first when the dominant blocker is historical quality evidence,
- `study_notes` first when the dominant blocker is model definition, liquidity/confirmation chain, session discipline, plan alignment, or risk boundary.
No D9 output may promote a cohort into Trader Cockpit `Review Now`.
## Implemented Surface
Backend:
- `DOCTRINE_PLAYBOOK_REMEDIATION_PHASE = D9_PLAYBOOK_REMEDIATION_QUEUE`
- `summarize_playbook_remediation_queue`
- `/trading/backtests` now returns `playbook_remediation_queue_summary`
Batch script:
- `scripts/run_backtest_batch.py` now includes `playbook_remediation_queue_summary` in JSON output.
- CLI text output prints queued cohort count plus `Strategy Backtest first` versus `Study Notes first`.
Frontend:
- Strategy Backtest shows `Playbook Remediation Queue`.
- Strategy Backtest lists the top queued cohorts with the dominant remediation family and route.
## Remediation Families
D9 groups dominant blockers into these families:
- `backtest_quality_gate`
- `model_playbook_contract`
- `liquidity_path`
- `pd_array_reaction`
- `time_session`
- `trading_plan_alignment`
- `risk_boundary`
- `general_study_only`
These families are then mapped into the existing Chapter 46-47 feedback classes so the queue can point back to Study Notes instead of inventing a new feedback taxonomy.
## Read-Only Boundary
D9 is read-only historical analysis and remediation routing. It does not add order submission, does not change `execution_allowed`, does not change `decision_support_ready`, does not change `real_live_ready`, and does not loosen risk governor / trust bucket / operator confirmation protocol.
## Validation Meaning
If D9 routes a cohort to `strategy_backtest`, the operator should inspect sample size, expectancy, profit factor, drawdown, executed-signal sufficiency, and related cohort evidence before revisiting the playbook.
If D9 routes a cohort to `study_notes`, the operator should tighten the model definition, confirmation chain, session rule, plan rule, or no-trade boundary before any trader-facing reconsideration.
In both cases the cohort remains `study_only` until D8 and D7 gates pass again.
## Next Slice
D10 should turn D9 queue items into saved remediation artifacts: generate per-cohort Study Notes drafts and Strategy Backtest review prompts so the operator can close the loop without scanning raw reason codes.
+23 -1
View File
@@ -18,7 +18,11 @@ from scripts.run_backtest import (
build_quality_gate_from_env,
build_replay_config_from_env,
)
from src.services.ict_mainline import build_doctrine_backtest_cohort
from src.services.ict_mainline import (
build_doctrine_backtest_cohort,
summarize_playbook_historical_cohorts,
summarize_playbook_remediation_queue,
)
from src.services.backtest import (
BACKTEST_MODE_REPLAY_NO_LOOKAHEAD,
BACKTEST_SUBJECT_PLAN_REPLAY,
@@ -78,6 +82,19 @@ def main(env=None) -> int:
print(f"Total results: {payload['total_result_count']}; executed: {payload['total_executed_signals']}")
if payload["quality_gate_reason_counts"]:
print(f"Top quality gate reasons: {_format_counts(payload['quality_gate_reason_counts'])}")
d8_summary = payload.get("playbook_historical_cohort_summary") or {}
print(
"Playbook cohorts: "
f"{d8_summary.get('manual_review_eligible_count', 0)} manual-review candidates / "
f"{d8_summary.get('study_only_count', 0)} study-only"
)
d9_summary = payload.get("playbook_remediation_queue_summary") or {}
print(
"Remediation queue: "
f"{d9_summary.get('queue_count', 0)} queued / "
f"{d9_summary.get('strategy_backtest_route_count', 0)} Strategy Backtest first / "
f"{d9_summary.get('study_notes_route_count', 0)} Study Notes first"
)
print(payload["operator_summary"])
return 2 if require_quality_gate and payload["blocked_quality_gate_count"] else 0
@@ -181,6 +198,7 @@ def run_backtest_batch(
"doctrine_completeness": doctrine_cohort.get("doctrine_completeness"),
"result_count": len(outcomes),
"executed_signals": int(summary.get("executed_signals") or 0),
"total_r": float(summary.get("total_r") or 0.0),
"expectancy_r": float(summary.get("expectancy_r") or 0.0),
"executed_expectancy_r": float(summary.get("executed_expectancy_r") or 0.0),
"profit_factor_r": summary.get("profit_factor_r"),
@@ -193,6 +211,8 @@ def run_backtest_batch(
passed_count = sum(1 for item in results if item["quality_gate"].get("passed"))
blocked_count = len(results) - passed_count
playbook_historical_cohort_summary = summarize_playbook_historical_cohorts(results)
playbook_remediation_queue_summary = summarize_playbook_remediation_queue(results)
payload = {
"status": "blocked" if require_quality_gate and blocked_count else "completed",
"mode": "batch_historical_backtest",
@@ -209,6 +229,8 @@ def run_backtest_batch(
"aggregate_by_subject": aggregate_by_subject,
"aggregate_by_session_group": aggregate_by_session_group,
"aggregate_by_doctrine_cohort": aggregate_by_doctrine_cohort,
"playbook_historical_cohort_summary": playbook_historical_cohort_summary,
"playbook_remediation_queue_summary": playbook_remediation_queue_summary,
"top_runs_by_expectancy": sorted(
results,
key=lambda item: (float(item.get("executed_expectancy_r") or 0.0), int(item.get("executed_signals") or 0)),
+6
View File
@@ -23,6 +23,8 @@ from src.services.ict_mainline import (
summarize_liquidity_kernel_statuses,
summarize_model_playbook_kernel_statuses,
summarize_pd_array_reaction_statuses,
summarize_playbook_historical_cohorts,
summarize_playbook_remediation_queue,
summarize_time_session_kernel_statuses,
)
@@ -49,6 +51,8 @@ def get_strategy_backtest_surface(instrument_id: Optional[int] = None, limit: in
pd_array_reaction_summary = summarize_pd_array_reaction_statuses(items)
time_session_kernel_summary = summarize_time_session_kernel_statuses(items)
model_playbook_kernel_summary = summarize_model_playbook_kernel_statuses(items)
playbook_historical_cohort_summary = summarize_playbook_historical_cohorts(items)
playbook_remediation_queue_summary = summarize_playbook_remediation_queue(items)
return {
"instrument_id": instrument_id,
"item_count": len(items),
@@ -62,6 +66,8 @@ def get_strategy_backtest_surface(instrument_id: Optional[int] = None, limit: in
"pd_array_reaction_summary": pd_array_reaction_summary,
"time_session_kernel_summary": time_session_kernel_summary,
"model_playbook_kernel_summary": model_playbook_kernel_summary,
"playbook_historical_cohort_summary": playbook_historical_cohort_summary,
"playbook_remediation_queue_summary": playbook_remediation_queue_summary,
"ict_rulebook_mapping": ict_rulebook_mapping,
"items": items,
}
+10
View File
@@ -22,12 +22,15 @@ from src.services.ict_mainline.doctrine import (
DOCTRINE_KERNEL_PHASE,
DOCTRINE_LIQUIDITY_PHASE,
DOCTRINE_MODEL_PLAYBOOK_PHASE,
DOCTRINE_PLAYBOOK_HISTORICAL_COHORT_PHASE,
DOCTRINE_PLAYBOOK_REMEDIATION_PHASE,
DOCTRINE_PLAYBOOK_PHASE,
DOCTRINE_REACTION_PHASE,
DOCTRINE_SESSION_PHASE,
build_doctrine_backtest_cohort,
build_liquidity_backtest_features,
build_model_playbook_backtest_features,
build_playbook_historical_cohort_descriptor,
build_pd_array_backtest_features,
build_time_session_backtest_features,
build_doctrine_rule_matrix,
@@ -54,6 +57,8 @@ from src.services.ict_mainline.doctrine import (
summarize_doctrine_rule_gaps,
summarize_liquidity_kernel_statuses,
summarize_model_playbook_kernel_statuses,
summarize_playbook_historical_cohorts,
summarize_playbook_remediation_queue,
summarize_pd_array_reaction_statuses,
summarize_time_session_kernel_statuses,
)
@@ -62,11 +67,14 @@ __all__ = [
"DOCTRINE_KERNEL_PHASE",
"DOCTRINE_LIQUIDITY_PHASE",
"DOCTRINE_MODEL_PLAYBOOK_PHASE",
"DOCTRINE_PLAYBOOK_HISTORICAL_COHORT_PHASE",
"DOCTRINE_PLAYBOOK_REMEDIATION_PHASE",
"DOCTRINE_PLAYBOOK_PHASE",
"DOCTRINE_REACTION_PHASE",
"DOCTRINE_SESSION_PHASE",
"ICT_MAINLINE_CHAIN",
"build_doctrine_backtest_cohort",
"build_playbook_historical_cohort_descriptor",
"build_liquidity_backtest_features",
"build_model_playbook_backtest_features",
"build_pd_array_backtest_features",
@@ -107,6 +115,8 @@ __all__ = [
"summarize_doctrine_rule_gaps",
"summarize_liquidity_kernel_statuses",
"summarize_model_playbook_kernel_statuses",
"summarize_playbook_historical_cohorts",
"summarize_playbook_remediation_queue",
"summarize_pd_array_reaction_statuses",
"summarize_time_session_kernel_statuses",
"summarize_book_rule_errors",
+557
View File
@@ -5,6 +5,8 @@ import json
import re
from typing import Iterable
from src.services.ict_mainline.feedback import BOOK_RULE_ERROR_CLASSES
DOCTRINE_KERNEL_PHASE = "D1_MARKET_LANGUAGE_KERNEL"
DOCTRINE_PLAYBOOK_PHASE = "D2_PLAYBOOK_BACKTEST_COHORT"
@@ -12,6 +14,20 @@ DOCTRINE_LIQUIDITY_PHASE = "D3_LIQUIDITY_KERNEL"
DOCTRINE_REACTION_PHASE = "D4_PD_ARRAY_REACTION_KERNEL"
DOCTRINE_SESSION_PHASE = "D5_TIME_SESSION_KERNEL"
DOCTRINE_MODEL_PLAYBOOK_PHASE = "D6_MODEL_PLAYBOOK_KERNEL"
DOCTRINE_PLAYBOOK_HISTORICAL_COHORT_PHASE = "D8_PLAYBOOK_SPECIFIC_HISTORICAL_COHORT_GATE"
DOCTRINE_PLAYBOOK_REMEDIATION_PHASE = "D9_PLAYBOOK_REMEDIATION_QUEUE"
PLAYBOOK_REMEDIATION_FAMILY_LABELS = {
"backtest_quality_gate": "Backtest Quality Gate",
"model_playbook_contract": "Model Playbook Contract",
"liquidity_path": "Liquidity / Sweep / Path",
"pd_array_reaction": "PD Array / Reaction",
"time_session": "Time / Session",
"trading_plan_alignment": "Trading Plan Alignment",
"risk_boundary": "Risk Boundary",
"general_study_only": "General Study-Only",
}
@dataclass(frozen=True)
@@ -1282,6 +1298,281 @@ def summarize_doctrine_backtest_cohorts(items: Iterable[dict]) -> dict:
}
def build_playbook_historical_cohort_descriptor(payload: object) -> dict:
"""Build the D8 cohort dimensions without promoting anything to live focus."""
quality = dict(_extract_quality_gate(payload) or {})
cohort = {}
if isinstance(payload, dict):
cohort = dict(payload.get("doctrine_backtest_cohort") or {})
if not cohort:
cohort = build_doctrine_backtest_cohort(payload, quality_gate=quality)
setup_playbook_id = _string_or_empty(cohort.get("setup_playbook_id") or extract_setup_playbook_id(payload)) or "missing_playbook"
session_codes = _extract_session_codes(payload, cohort)
session_group = "ALL" if not session_codes else ",".join(session_codes)
timeframe_set = _extract_timeframe_set(payload)
timeframe_key = _timeframe_set_key(timeframe_set)
model_family = _extract_model_family(payload, cohort)
cohort_key = f"{setup_playbook_id}::{session_group}::{timeframe_key}::{model_family}"
return {
"phase": DOCTRINE_PLAYBOOK_HISTORICAL_COHORT_PHASE,
"cohort_key": cohort_key,
"cohort_key_fields": ["setup_playbook_id", "session_group", "timeframe_key", "model_family"],
"setup_playbook_id": setup_playbook_id,
"session_group": session_group,
"session_codes": session_codes,
"timeframe_set": timeframe_set,
"timeframe_key": timeframe_key,
"model_family": model_family,
"doctrine_cohort_key": cohort.get("cohort_key") or "",
"doctrine_completeness": cohort.get("doctrine_completeness") or "incomplete",
"quality_gate_passed": bool(quality.get("passed") or cohort.get("quality_gate_passed")),
"promotion_gate_passed": bool((cohort.get("promotion_gate") or {}).get("passed")),
"promotion_blockers": _dedupe([str(item) for item in list((cohort.get("promotion_gate") or {}).get("reasons") or [])]),
"quality_gate_reasons": _dedupe([str(item) for item in list(quality.get("reasons") or [])]),
"execution_boundary": "read_only_backtest_no_order_submission",
}
def summarize_playbook_historical_cohorts(items: Iterable[dict]) -> dict:
cohorts: dict[str, dict] = {}
total_runs = 0
total_results = 0
total_executed = 0
for item in items:
total_runs += 1
descriptor = build_playbook_historical_cohort_descriptor(item)
key = str(descriptor.get("cohort_key") or "missing_playbook::ALL::unknown_tf::missing_model_family")
bucket = cohorts.setdefault(
key,
{
"cohort_key": key,
"setup_playbook_id": descriptor.get("setup_playbook_id"),
"session_group": descriptor.get("session_group"),
"timeframe_key": descriptor.get("timeframe_key"),
"timeframe_set": dict(descriptor.get("timeframe_set") or {}),
"model_family": descriptor.get("model_family"),
"run_count": 0,
"quality_gate_passed_count": 0,
"promotion_gate_passed_count": 0,
"manual_review_eligible_count": 0,
"study_only_count": 0,
"total_result_count": 0,
"total_executed_signals": 0,
"total_r": 0.0,
"promotion_blocker_counts": {},
"quality_gate_reason_counts": {},
"sample_run_ids": [],
"execution_boundary": "read_only_backtest_no_order_submission",
},
)
bucket["run_count"] += 1
if descriptor.get("quality_gate_passed"):
bucket["quality_gate_passed_count"] += 1
if descriptor.get("promotion_gate_passed"):
bucket["promotion_gate_passed_count"] += 1
summary = _extract_backtest_summary(item)
result_count = _extract_int_first(
item,
summary,
keys=("result_count", "total_signals", "candidate_count"),
)
executed_signals = _extract_int_first(
item,
summary,
keys=("executed_signals", "total_executed_signals"),
)
total_r = _extract_float_first(item, summary, keys=("total_r",))
bucket["total_result_count"] += result_count
bucket["total_executed_signals"] += executed_signals
bucket["total_r"] += total_r
total_results += result_count
total_executed += executed_signals
run_id = _extract_first_present_value(item, keys=("backtest_run_id", "id", "run_id"))
if run_id is not None and len(bucket["sample_run_ids"]) < 5:
bucket["sample_run_ids"].append(run_id)
for reason in list(descriptor.get("promotion_blockers") or []):
bucket["promotion_blocker_counts"][reason] = int(bucket["promotion_blocker_counts"].get(reason) or 0) + 1
for reason in list(descriptor.get("quality_gate_reasons") or []):
bucket["quality_gate_reason_counts"][reason] = int(bucket["quality_gate_reason_counts"].get(reason) or 0) + 1
manual_review_eligible_count = 0
study_only_count = 0
for bucket in cohorts.values():
setup_playbook_id = str(bucket.get("setup_playbook_id") or "")
eligible = (
setup_playbook_id not in {"", "missing_playbook"}
and int(bucket.get("quality_gate_passed_count") or 0) > 0
and int(bucket.get("promotion_gate_passed_count") or 0) > 0
)
if eligible:
bucket["status"] = "manual_review_candidate"
bucket["manual_review_eligible"] = True
bucket["manual_review_eligible_count"] = 1
bucket["study_only_count"] = 0
bucket["eligibility_reason"] = "playbook_historical_cohort_passed"
manual_review_eligible_count += 1
else:
bucket["status"] = "unmapped_study_only" if setup_playbook_id in {"", "missing_playbook"} else "study_only"
bucket["manual_review_eligible"] = False
bucket["manual_review_eligible_count"] = 0
bucket["study_only_count"] = 1
bucket["eligibility_reason"] = _first_count_key(bucket["promotion_blocker_counts"]) or _first_count_key(bucket["quality_gate_reason_counts"]) or "historical_cohort_not_passed"
study_only_count += 1
bucket["must_not_trade"] = True
bucket["trader_cockpit_promotion_requires_active_plan"] = True
sorted_cohorts = sorted(
cohorts.values(),
key=lambda item: (
0 if item.get("manual_review_eligible") else 1,
-int(item.get("run_count") or 0),
str(item.get("cohort_key") or ""),
),
)
return {
"phase": DOCTRINE_PLAYBOOK_HISTORICAL_COHORT_PHASE,
"cohort_count": len(sorted_cohorts),
"run_count": total_runs,
"manual_review_eligible_count": manual_review_eligible_count,
"study_only_count": study_only_count,
"total_result_count": total_results,
"total_executed_signals": total_executed,
"cohorts": sorted_cohorts,
"operator_summary": (
"No playbook-specific historical cohorts yet."
if not sorted_cohorts
else f"{manual_review_eligible_count} cohorts may enter manual review; {study_only_count} cohorts remain study-only."
),
"execution_boundary": "read_only_backtest_no_order_submission",
"must_not_trade": True,
}
def summarize_playbook_remediation_queue(items: Iterable[dict]) -> dict:
d8_summary = summarize_playbook_historical_cohorts(items)
bucket_map: dict[str, dict] = {}
family_counts: dict[str, int] = {}
for item in items:
descriptor = build_playbook_historical_cohort_descriptor(item)
key = str(descriptor.get("cohort_key") or "missing_playbook::ALL::unknown_tf::missing_model_family")
bucket = bucket_map.setdefault(
key,
{
"reason_counts": {},
"family_counts": {},
"promotion_blocker_counts": {},
"quality_gate_reason_counts": {},
"historical_failure_reason_counts": {},
},
)
for reason in list(descriptor.get("promotion_blockers") or []):
_increment_reason_count(bucket["promotion_blocker_counts"], reason, 1)
_increment_reason_count(bucket["reason_counts"], reason, 1)
for reason in list(descriptor.get("quality_gate_reasons") or []):
_increment_reason_count(bucket["quality_gate_reason_counts"], reason, 1)
_increment_reason_count(bucket["reason_counts"], reason, 1)
for reason, count in _extract_failure_reason_counts(item).items():
_increment_reason_count(bucket["historical_failure_reason_counts"], reason, count)
_increment_reason_count(bucket["reason_counts"], reason, count)
queue_items = []
strategy_backtest_route_count = 0
study_notes_route_count = 0
for cohort in list(d8_summary.get("cohorts") or []):
if cohort.get("manual_review_eligible"):
continue
key = str(cohort.get("cohort_key") or "")
bucket = bucket_map.get(
key,
{
"reason_counts": {},
"family_counts": {},
"promotion_blocker_counts": {},
"quality_gate_reason_counts": {},
"historical_failure_reason_counts": {},
},
)
reason_counts = dict(bucket.get("reason_counts") or {})
family_counts_for_bucket = _build_remediation_family_counts(reason_counts)
dominant_reason = _first_count_key(reason_counts) or str(cohort.get("eligibility_reason") or "historical_cohort_not_passed")
dominant_family = _first_count_key(family_counts_for_bucket) or _resolve_playbook_remediation_family(dominant_reason)
dominant_reason_count = int(reason_counts.get(dominant_reason) or 0)
route = _build_playbook_remediation_route(dominant_family=dominant_family, dominant_reason=dominant_reason)
definition = BOOK_RULE_ERROR_CLASSES[route["error_class"]]
_increment_reason_count(family_counts, dominant_family, 1)
if route["primary_route"] == "strategy_backtest":
strategy_backtest_route_count += 1
else:
study_notes_route_count += 1
queue_items.append(
{
"cohort_key": key,
"setup_playbook_id": cohort.get("setup_playbook_id"),
"session_group": cohort.get("session_group"),
"timeframe_key": cohort.get("timeframe_key"),
"timeframe_set": dict(cohort.get("timeframe_set") or {}),
"model_family": cohort.get("model_family"),
"status": "queued",
"cohort_status": cohort.get("status"),
"dominant_remediation_family": dominant_family,
"dominant_remediation_family_label": PLAYBOOK_REMEDIATION_FAMILY_LABELS.get(dominant_family, "General Study-Only"),
"dominant_error_class": route["error_class"],
"dominant_error_label": definition["label"],
"dominant_reason_code": dominant_reason,
"dominant_reason_count": dominant_reason_count,
"route_targets": list(route["route_targets"]),
"primary_route": route["primary_route"],
"secondary_route": route["secondary_route"],
"study_note_prompt": definition["study_note_prompt"],
"strategy_backtest_prompt": route["strategy_backtest_prompt"],
"next_action": definition["next_action"],
"book_anchor": definition["book_anchor"],
"promotion_blocker_counts": dict(bucket.get("promotion_blocker_counts") or {}),
"quality_gate_reason_counts": dict(bucket.get("quality_gate_reason_counts") or {}),
"historical_failure_reason_counts": dict(bucket.get("historical_failure_reason_counts") or {}),
"remediation_reason_counts": reason_counts,
"remediation_family_counts": family_counts_for_bucket,
"top_reason_codes": _top_count_entries(reason_counts, limit=5),
"run_count": int(cohort.get("run_count") or 0),
"total_result_count": int(cohort.get("total_result_count") or 0),
"total_executed_signals": int(cohort.get("total_executed_signals") or 0),
"total_r": float(cohort.get("total_r") or 0.0),
"must_not_trade": True,
"operator_summary": (
f"Route this {cohort.get('cohort_status') or 'study_only'} cohort to "
f"{route['primary_route'].replace('_', ' ')} first; keep it out of Trader Cockpit focus."
),
}
)
queue_items = sorted(
queue_items,
key=lambda item: (
-int(item.get("dominant_reason_count") or 0),
-int(item.get("run_count") or 0),
str(item.get("cohort_key") or ""),
),
)
return {
"phase": DOCTRINE_PLAYBOOK_REMEDIATION_PHASE,
"queue_count": len(queue_items),
"strategy_backtest_route_count": strategy_backtest_route_count,
"study_notes_route_count": study_notes_route_count,
"dominant_family_counts": family_counts,
"items": queue_items,
"operator_summary": (
"No failed playbook cohorts need remediation right now."
if not queue_items
else f"{len(queue_items)} failed cohorts queued: {strategy_backtest_route_count} route to Strategy Backtest first, {study_notes_route_count} route to Study Notes first."
),
"execution_boundary": "read_only_backtest_no_order_submission",
"must_not_trade": True,
}
def build_liquidity_backtest_features(payload: object) -> dict:
features = {
"liquidity_target_side": _string_or_empty(
@@ -2149,6 +2440,272 @@ def summarize_doctrine_rule_gaps(items: Iterable[dict]) -> dict:
}
def _extract_session_codes(payload: object, cohort: dict | None = None) -> list[str]:
for candidate in (
_extract_first_present_value(payload, keys=("session_codes", "sessions")),
_extract_first_present_value(payload, keys=("allowed_session_codes", "session_group")),
(cohort or {}).get("session_codes"),
):
values = _as_string_list(candidate)
if values:
return values
session_code = _string_or_empty(
_extract_first_present_value(payload, keys=("session_code", "session"))
or ((cohort or {}).get("time_session_backtest_features") or {}).get("session_code")
)
return [] if not session_code or session_code.upper() == "ALL" else [session_code]
def _extract_timeframe_set(payload: object) -> dict:
raw = _extract_first_present_value(payload, keys=("timeframe_set",))
if isinstance(raw, dict):
return {
"bias_tf": _string_or_empty(raw.get("bias_tf") or raw.get("bias_timeframe") or raw.get("bias")),
"setup_tf": _string_or_empty(raw.get("setup_tf") or raw.get("setup_timeframe") or raw.get("setup")),
"trigger_tf": _string_or_empty(raw.get("trigger_tf") or raw.get("trigger_timeframe") or raw.get("trigger")),
}
return {
"bias_tf": _string_or_empty(_extract_first_present_value(payload, keys=("bias_tf", "bias_timeframe"))),
"setup_tf": _string_or_empty(_extract_first_present_value(payload, keys=("setup_tf", "setup_timeframe"))),
"trigger_tf": _string_or_empty(_extract_first_present_value(payload, keys=("trigger_tf", "trigger_timeframe"))),
}
def _timeframe_set_key(timeframe_set: dict) -> str:
bias_tf = _string_or_empty(timeframe_set.get("bias_tf")) or "missing_bias_tf"
setup_tf = _string_or_empty(timeframe_set.get("setup_tf")) or "missing_setup_tf"
trigger_tf = _string_or_empty(timeframe_set.get("trigger_tf")) or "missing_trigger_tf"
return f"{bias_tf}/{setup_tf}/{trigger_tf}"
def _extract_model_family(payload: object, cohort: dict | None = None) -> str:
for candidate in (
_extract_first_present_value(payload, keys=("model_family", "playbook_model_family", "setup_family", "cohort_family")),
((cohort or {}).get("model_playbook_backtest_features") or {}).get("model_family"),
_extract_first_present_value(payload, keys=("candidate_model_code", "model_code")),
((cohort or {}).get("playbook_gate") or {}).get("setup_name"),
):
value = _string_or_empty(candidate)
if value:
return value
return "missing_model_family"
def _extract_backtest_summary(payload: object) -> dict:
if isinstance(payload, dict):
summary = payload.get("summary")
if isinstance(summary, dict):
return dict(summary)
return {}
def _extract_failure_reason_counts(payload: object) -> dict[str, int]:
if not isinstance(payload, dict):
return {}
summary = _extract_backtest_summary(payload)
counts: dict[str, int] = {}
raw_failure_counts = summary.get("failure_reasons")
if isinstance(raw_failure_counts, dict):
for reason, count in raw_failure_counts.items():
_increment_reason_count(counts, reason, count)
top_failure_reasons = payload.get("top_failure_reasons")
if isinstance(top_failure_reasons, list):
for item in top_failure_reasons:
if not isinstance(item, dict):
continue
_increment_reason_count(counts, item.get("code"), item.get("count"))
return counts
def _build_remediation_family_counts(reason_counts: dict[str, int]) -> dict[str, int]:
counts: dict[str, int] = {}
for reason, count in reason_counts.items():
family = _resolve_playbook_remediation_family(reason)
_increment_reason_count(counts, family, count)
return counts
def _resolve_playbook_remediation_family(reason_code: object) -> str:
token = _normalize_reason_token(reason_code)
if any(
marker in token
for marker in (
"sample_size",
"executed_sample",
"expectancy",
"profit_factor",
"drawdown",
"consecutive_loss",
"same_candle_entry_exit_ambiguous",
"no_future_candles",
"stopped_out",
"quality_gate",
)
):
return "backtest_quality_gate"
if any(
marker in token
for marker in (
"missing_model",
"missing_playbook",
"cohort",
"operator_runbook",
"journal_feedback",
"study_note_contract",
"doctrine",
)
):
return "model_playbook_contract"
if any(
marker in token
for marker in (
"liquidity",
"sweep",
"raid",
"manipulation",
"working_draw",
"pool",
"reorder_trigger",
)
):
return "liquidity_path"
if any(
marker in token
for marker in (
"pd_array",
"array_",
"fvg",
"order_block",
"breaker",
"mitigation",
"rebalanc",
"reaction",
"retest",
"displacement",
"mss",
)
):
return "pd_array_reaction"
if any(
marker in token
for marker in (
"session",
"killzone",
"judas",
"opening_range",
"weekly_profile",
"news_time",
)
):
return "time_session"
if any(marker in token for marker in ("trading_plan", "plan", "draw", "location", "bias", "invalidation")):
return "trading_plan_alignment"
if any(marker in token for marker in ("blocked", "risk", "not_allowed", "do_not_use", "stale", "observe_only")):
return "risk_boundary"
return "general_study_only"
def _build_playbook_remediation_route(*, dominant_family: str, dominant_reason: str) -> dict:
if dominant_family == "backtest_quality_gate":
return {
"error_class": "backtest_evidence_gap",
"primary_route": "strategy_backtest",
"secondary_route": "study_notes",
"route_targets": ["strategy_backtest", "study_notes"],
"strategy_backtest_prompt": "Review cohort sample size, expectancy, profit factor, drawdown, and executed-signal sufficiency before any promotion.",
}
if dominant_family in {"time_session", "trading_plan_alignment"}:
error_class = (
"risk_boundary_gap"
if any(marker in _normalize_reason_token(dominant_reason) for marker in ("news_time", "outside_session", "not_allowed"))
else "plan_alignment_gap"
)
return {
"error_class": error_class,
"primary_route": "study_notes",
"secondary_route": "strategy_backtest",
"route_targets": ["study_notes", "strategy_backtest"],
"strategy_backtest_prompt": "Filter the cohort by session and plan context to confirm whether the failure is structural or just a bad timeframe/session mix.",
}
if dominant_family == "risk_boundary":
return {
"error_class": "risk_boundary_gap",
"primary_route": "study_notes",
"secondary_route": "strategy_backtest",
"route_targets": ["study_notes", "strategy_backtest"],
"strategy_backtest_prompt": "Review the blocked cohort in Strategy Backtest only after the no-trade boundary and data-quality reasons are explicit.",
}
if dominant_family in {"liquidity_path", "pd_array_reaction"}:
return {
"error_class": "confirmation_gap",
"primary_route": "study_notes",
"secondary_route": "strategy_backtest",
"route_targets": ["study_notes", "strategy_backtest"],
"strategy_backtest_prompt": "Use Strategy Backtest to inspect which confirmation link failed most often for this playbook cohort.",
}
return {
"error_class": "model_definition_gap",
"primary_route": "study_notes",
"secondary_route": "strategy_backtest",
"route_targets": ["study_notes", "strategy_backtest"],
"strategy_backtest_prompt": "Keep the cohort in Strategy Backtest while the playbook definition, contracts, and failure boundaries are clarified.",
}
def _top_count_entries(values: dict[str, int], *, limit: int) -> list[dict]:
return [
{"code": key, "count": int(value)}
for key, value in sorted(values.items(), key=lambda item: (-int(item[1]), str(item[0])))[:limit]
]
def _increment_reason_count(container: dict[str, int], reason: object, count: object) -> None:
key = str(reason or "").strip()
if not key:
return
try:
numeric = int(count)
except (TypeError, ValueError):
numeric = 0
if numeric <= 0:
return
container[key] = int(container.get(key) or 0) + numeric
def _normalize_reason_token(value: object) -> str:
return str(value or "").strip().lower().replace("-", "_").replace(" ", "_")
def _extract_int_first(*payloads: object, keys: tuple[str, ...]) -> int:
for payload in payloads:
value = _extract_first_present_value(payload, keys=keys)
if value is None or value == "":
continue
try:
return int(value)
except (TypeError, ValueError):
continue
return 0
def _extract_float_first(*payloads: object, keys: tuple[str, ...]) -> float:
for payload in payloads:
value = _extract_first_present_value(payload, keys=keys)
if value is None or value == "":
continue
try:
return float(value)
except (TypeError, ValueError):
continue
return 0.0
def _first_count_key(values: dict) -> str:
if not values:
return ""
return str(sorted(values.items(), key=lambda item: (-int(item[1]), str(item[0])))[0][0])
def _payload_to_search_text(payload: object) -> str:
try:
raw = json.dumps(payload, ensure_ascii=False, default=str)
+30
View File
@@ -5532,6 +5532,8 @@ function renderStrategyBacktests(payload) {
const operatorSummary = payload?.operator_summary || {};
const doctrineCohort = latest.doctrine_backtest_cohort || summary.doctrine_backtest_cohort || {};
const doctrinePromotionGate = doctrineCohort.promotion_gate || {};
const playbookHistorical = payload?.playbook_historical_cohort_summary || {};
const playbookRemediation = payload?.playbook_remediation_queue_summary || {};
const liquidityStatus =
latest.liquidity_kernel_status ||
summary.liquidity_kernel_status ||
@@ -5602,6 +5604,14 @@ function renderStrategyBacktests(payload) {
['Model Playbook Reason', formatBacktestDecision(modelPlaybookStatus.reason || 'missing_model_playbook')],
['Model Playbook', modelPlaybookFeatures.model_playbook_id || 'missing_model_playbook'],
['Model Family', modelPlaybookFeatures.model_family || 'missing_model_family'],
[
'Playbook Historical Gate',
`${playbookHistorical.manual_review_eligible_count ?? 0} manual-review / ${playbookHistorical.study_only_count ?? 0} study-only`,
],
[
'Playbook Remediation Queue',
`${playbookRemediation.queue_count ?? 0} queued / ${playbookRemediation.strategy_backtest_route_count ?? 0} backtest-first / ${playbookRemediation.study_notes_route_count ?? 0} notes-first`,
],
['Promotion Gate', doctrinePromotionGate.passed ? 'Passed' : 'Blocked'],
['Instrument ID', latest.instrument_id ?? 'n/a'],
['Timeframe', formatJsonInline(latest.timeframe_set || {})],
@@ -5690,6 +5700,26 @@ function renderStrategyBacktests(payload) {
])
)
);
detailGrid.appendChild(
buildItemCard(
'Playbook Historical Cohorts',
`${playbookHistorical.manual_review_eligible_count ?? 0} manual-review / ${playbookHistorical.study_only_count ?? 0} study-only`,
(playbookHistorical.cohorts || []).slice(0, 5).map((item) => [
`${item.setup_playbook_id || 'missing_playbook'} / ${item.session_group || 'ALL'} / ${item.timeframe_key || 'missing_tf'} / ${item.model_family || 'missing_model_family'}`,
`${formatBacktestDecision(item.status || 'study_only')} / ${item.run_count ?? 0} runs / ${item.quality_gate_passed_count ?? 0} passed / ${item.total_executed_signals ?? 0} executed`,
])
)
);
detailGrid.appendChild(
buildItemCard(
'Playbook Remediation Queue',
`${playbookRemediation.queue_count ?? 0} queued / ${playbookRemediation.strategy_backtest_route_count ?? 0} backtest-first / ${playbookRemediation.study_notes_route_count ?? 0} notes-first`,
(playbookRemediation.items || []).slice(0, 5).map((item) => [
`${item.setup_playbook_id || 'missing_playbook'} / ${item.session_group || 'ALL'} / ${item.timeframe_key || 'missing_tf'} / ${item.model_family || 'missing_model_family'}`,
`${formatBacktestDecision(item.primary_route || 'study_notes')} / ${item.dominant_remediation_family_label || item.dominant_remediation_family || 'general'} / ${item.dominant_reason_code || 'historical_cohort_not_passed'} × ${item.dominant_reason_count ?? 0}`,
])
)
);
detailGrid.appendChild(
buildItemCard(
'Liquidity Kernel',
+40
View File
@@ -95,6 +95,26 @@
'Model Playbook': '模型 Playbook',
'Model Family': '模型族',
'Model Playbook chain mapped': '模型 Playbook 链路已映射',
'Playbook Historical Gate': 'Playbook 历史分组门',
'Playbook Historical Cohorts': 'Playbook 历史分组',
'Playbook Remediation Queue': 'Playbook 修复队列',
'backtest-first': '先看回测',
'notes-first': '先写笔记',
'strategy_backtest': '策略回测',
'study_notes': '学习笔记',
'Backtest Quality Gate': '回测质量门',
'Model Playbook Contract': '模型 Playbook 契约',
'Liquidity / Sweep / Path': '流动性 / Sweep / Path',
'PD Array / Reaction': 'PD Array / 反应',
'Time / Session': '时间 / Session',
'Trading Plan Alignment': '交易计划对齐',
'Risk Boundary': '风险边界',
'General Study-Only': '通用仅研究',
'manual-review': '人工复核',
'manual_review_candidate': '可进入人工复核',
'unmapped_study_only': '未映射,仅研究',
'playbook_historical_cohort_passed': 'Playbook 历史分组已通过',
'historical_cohort_not_passed': '历史分组未通过',
'Product Gate': '产品门',
'product gate': '产品门',
'trader_product_promotion_gate_blocked': '交易员产品晋级门阻塞',
@@ -865,6 +885,26 @@
['Model Playbook', '模型 Playbook'],
['Model Family', '模型族'],
['Model Playbook chain mapped', '模型 Playbook 链路已映射'],
['Playbook Historical Gate', 'Playbook 历史分组门'],
['Playbook Historical Cohorts', 'Playbook 历史分组'],
['Playbook Remediation Queue', 'Playbook 修复队列'],
['backtest-first', '先看回测'],
['notes-first', '先写笔记'],
['strategy_backtest', '策略回测'],
['study_notes', '学习笔记'],
['Backtest Quality Gate', '回测质量门'],
['Model Playbook Contract', '模型 Playbook 契约'],
['Liquidity / Sweep / Path', '流动性 / Sweep / Path'],
['PD Array / Reaction', 'PD Array / 反应'],
['Time / Session', '时间 / Session'],
['Trading Plan Alignment', '交易计划对齐'],
['Risk Boundary', '风险边界'],
['General Study-Only', '通用仅研究'],
['manual-review', '人工复核'],
['manual_review_candidate', '可进入人工复核'],
['unmapped_study_only', '未映射,仅研究'],
['playbook_historical_cohort_passed', 'Playbook 历史分组已通过'],
['historical_cohort_not_passed', '历史分组未通过'],
['model_playbook_mapped', '模型 Playbook 已映射'],
['model_playbook_kernel_blocked', '模型 Playbook 内核阻塞'],
['missing_model_playbook', '缺少模型 Playbook'],
+142
View File
@@ -11,6 +11,7 @@ from src.api.server import route_request
from src.api.signals import get_recent_signals
from src.services.backtest import BacktestCostConfig, BacktestQualityGate, BacktestReplayConfig
from src.services.execution import AccountRiskState
from src.services.ict_mainline import summarize_playbook_historical_cohorts, summarize_playbook_remediation_queue
from src.main import run_closed_loop, select_closed_loop_backtest_subject_type
@@ -138,6 +139,147 @@ class ApiTests(unittest.TestCase):
self.assertIn("missing_session_window", result["time_session_kernel_summary"]["reason_counts"])
self.assertEqual(result["model_playbook_kernel_summary"]["item_count"], 1)
self.assertIn("missing_model_playbook", result["model_playbook_kernel_summary"]["reason_counts"])
self.assertEqual(result["playbook_historical_cohort_summary"]["phase"], "D8_PLAYBOOK_SPECIFIC_HISTORICAL_COHORT_GATE")
self.assertEqual(result["playbook_historical_cohort_summary"]["cohort_count"], 1)
self.assertEqual(result["playbook_historical_cohort_summary"]["study_only_count"], 1)
self.assertEqual(result["playbook_historical_cohort_summary"]["cohorts"][0]["status"], "unmapped_study_only")
d9_summary = result["playbook_remediation_queue_summary"]
self.assertEqual(d9_summary["phase"], "D9_PLAYBOOK_REMEDIATION_QUEUE")
self.assertEqual(d9_summary["queue_count"], 1)
self.assertEqual(d9_summary["items"][0]["primary_route"], "study_notes")
self.assertEqual(d9_summary["items"][0]["dominant_remediation_family"], "model_playbook_contract")
def test_strategy_backtest_surface_groups_playbook_historical_cohorts(self) -> None:
rows = [
{
"id": 11,
"subject_type": "plan_replay",
"model_code": "OSOK_DAILY_RANGE_SWEEP_MSS_FVG_RETRACE",
"candidate_model_code": "OSOK_DAILY_RANGE_SWEEP_MSS_FVG_RETRACE",
"timeframe_set": {"bias_tf": "1h", "setup_tf": "15m", "trigger_tf": "5m"},
"config": {
"run_metadata": {
"setup_playbook_id": "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL",
},
"session_codes": ["LONDON"],
},
"summary": {
"total_signals": 12,
"executed_signals": 5,
"total_r": -2.5,
"quality_gate": {"passed": False, "reasons": ["profit_factor_below_threshold"]},
},
}
]
with patch("src.api.backtest.SessionLocal", return_value=FakeSession(rows)):
result = get_strategy_backtest_surface(instrument_id=1, limit=1)
d8_summary = result["playbook_historical_cohort_summary"]
self.assertEqual(d8_summary["cohort_count"], 1)
cohort = d8_summary["cohorts"][0]
self.assertEqual(cohort["setup_playbook_id"], "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL")
self.assertEqual(cohort["session_group"], "LONDON")
self.assertEqual(cohort["timeframe_key"], "1h/15m/5m")
self.assertEqual(cohort["model_family"], "london_discount_reversal")
self.assertEqual(cohort["status"], "study_only")
self.assertFalse(cohort["manual_review_eligible"])
self.assertEqual(cohort["total_result_count"], 12)
self.assertEqual(cohort["total_executed_signals"], 5)
self.assertIn("profit_factor_below_threshold", cohort["quality_gate_reason_counts"])
d9_summary = result["playbook_remediation_queue_summary"]
self.assertEqual(d9_summary["queue_count"], 1)
self.assertEqual(d9_summary["strategy_backtest_route_count"], 0)
self.assertEqual(d9_summary["study_notes_route_count"], 1)
self.assertEqual(d9_summary["items"][0]["primary_route"], "study_notes")
self.assertIn("profit_factor_below_threshold", d9_summary["items"][0]["quality_gate_reason_counts"])
def test_playbook_historical_cohort_summary_marks_manual_review_candidates(self) -> None:
result = summarize_playbook_historical_cohorts(
[
{
"backtest_run_id": 77,
"timeframe_set": {"bias_tf": "4h", "setup_tf": "15m", "trigger_tf": "5m"},
"session_codes": ["LONDON"],
"summary": {
"total_signals": 21,
"executed_signals": 9,
"total_r": 6.5,
"quality_gate": {"passed": True, "reasons": []},
},
"doctrine_backtest_cohort": {
"setup_playbook_id": "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL",
"cohort_key": "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL::complete",
"doctrine_completeness": "complete",
"promotion_gate": {"passed": True, "reasons": []},
},
}
]
)
self.assertEqual(result["manual_review_eligible_count"], 1)
self.assertEqual(result["study_only_count"], 0)
cohort = result["cohorts"][0]
self.assertEqual(cohort["status"], "manual_review_candidate")
self.assertTrue(cohort["manual_review_eligible"])
self.assertEqual(cohort["eligibility_reason"], "playbook_historical_cohort_passed")
self.assertEqual(cohort["session_group"], "LONDON")
self.assertEqual(cohort["timeframe_key"], "4h/15m/5m")
self.assertEqual(cohort["total_result_count"], 21)
self.assertEqual(cohort["total_executed_signals"], 9)
self.assertAlmostEqual(cohort["total_r"], 6.5)
d9_summary = summarize_playbook_remediation_queue(
[
{
"backtest_run_id": 77,
"timeframe_set": {"bias_tf": "4h", "setup_tf": "15m", "trigger_tf": "5m"},
"session_codes": ["LONDON"],
"summary": {
"total_signals": 21,
"executed_signals": 9,
"total_r": 6.5,
"quality_gate": {"passed": True, "reasons": []},
},
"doctrine_backtest_cohort": {
"setup_playbook_id": "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL",
"cohort_key": "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL::complete",
"doctrine_completeness": "complete",
"promotion_gate": {"passed": True, "reasons": []},
},
}
]
)
self.assertEqual(d9_summary["queue_count"], 0)
def test_playbook_remediation_queue_routes_quality_gate_failures_to_strategy_backtest(self) -> None:
result = summarize_playbook_remediation_queue(
[
{
"backtest_run_id": 99,
"timeframe_set": {"bias_tf": "1h", "setup_tf": "15m", "trigger_tf": "5m"},
"session_codes": ["NY_AM"],
"summary": {
"total_signals": 18,
"executed_signals": 8,
"total_r": -1.0,
"failure_reasons": {"stopped_out": 4},
"quality_gate": {"passed": False, "reasons": ["profit_factor_below_threshold", "expectancy_below_threshold"]},
},
"doctrine_backtest_cohort": {
"setup_playbook_id": "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL",
"cohort_key": "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL::complete",
"doctrine_completeness": "complete",
"promotion_gate": {"passed": True, "reasons": []},
},
}
]
)
self.assertEqual(result["queue_count"], 1)
self.assertEqual(result["strategy_backtest_route_count"], 1)
self.assertEqual(result["study_notes_route_count"], 0)
self.assertEqual(result["items"][0]["primary_route"], "strategy_backtest")
self.assertEqual(result["items"][0]["dominant_remediation_family"], "backtest_quality_gate")
self.assertEqual(result["items"][0]["dominant_error_class"], "backtest_evidence_gap")
def test_route_request_trading_doctrine_rules_returns_d1_matrix(self) -> None:
status, payload = route_request(
+19
View File
@@ -70,6 +70,25 @@ class RunBacktestBatchScriptTests(unittest.TestCase):
self.assertEqual(payload["aggregate_by_subject"]["plan_replay"]["run_count"], 1)
self.assertEqual(payload["aggregate_by_subject"]["signal"]["run_count"], 2)
self.assertIn("ICT-CH1-6-LONDON-DISCOUNT-REVERSAL::complete", payload["aggregate_by_doctrine_cohort"])
d8_summary = payload["playbook_historical_cohort_summary"]
self.assertEqual(d8_summary["phase"], "D8_PLAYBOOK_SPECIFIC_HISTORICAL_COHORT_GATE")
self.assertEqual(d8_summary["run_count"], 3)
self.assertEqual(d8_summary["cohort_count"], 2)
self.assertEqual(d8_summary["manual_review_eligible_count"], 0)
self.assertEqual(d8_summary["study_only_count"], 2)
self.assertTrue(all(item["status"] == "study_only" for item in d8_summary["cohorts"]))
self.assertTrue(
any(
item["cohort_key"] == "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL::LONDON::1h/15m/5m::london_discount_reversal"
for item in d8_summary["cohorts"]
)
)
d9_summary = payload["playbook_remediation_queue_summary"]
self.assertEqual(d9_summary["phase"], "D9_PLAYBOOK_REMEDIATION_QUEUE")
self.assertEqual(d9_summary["queue_count"], 2)
self.assertEqual(d9_summary["strategy_backtest_route_count"] + d9_summary["study_notes_route_count"], 2)
self.assertTrue(all(item["primary_route"] in {"strategy_backtest", "study_notes"} for item in d9_summary["items"]))
self.assertTrue(d9_summary["operator_summary"].startswith("2 failed cohorts queued"))
self.assertEqual(service.calls[0]["run_metadata"]["batch_id"], "batch-test")
self.assertEqual(service.calls[0]["run_metadata"]["setup_playbook_id"], "ICT-CH1-6-LONDON-DISCOUNT-REVERSAL")
self.assertEqual(service.calls[0]["run_metadata"]["execution_boundary"], "read_only_backtest_no_order_submission")
@@ -25,6 +25,8 @@ ACTIVE_TOP_LEVEL_DOCS = {
"AI_ICT_DoctrineKernel_TimeSessionKernel_D5_2026-05-22_v1.md",
"AI_ICT_DoctrineKernel_ModelPlaybookKernel_D6_2026-05-22_v1.md",
"AI_ICT_DoctrineKernel_TraderProductGate_D7_2026-05-22_v1.md",
"AI_ICT_DoctrineKernel_PlaybookHistoricalCohortGate_D8_2026-05-22_v1.md",
"AI_ICT_DoctrineKernel_PlaybookRemediationQueue_D9_2026-05-22_v1.md",
"AI_ICT_硬性主线约束_必须遵循ICT交易全书_2026-05-21_v1.md",
"AI_ICT_实盘辅助分析决策助手ICT主线开发方案_2026-05-21_v1.md",
"AI_ICT_主线裁剪审计_S1_2026-05-21_v1.md",