Close P19 multi-channel alert routing
This commit is contained in:
Codex
+21
-6
@@ -2,7 +2,7 @@
|
||||
|
||||
## Current Goal
|
||||
|
||||
Automation execution-layer P18 is closed locally. P18 adds a stable visual media evidence block so alert deliveries and workbench visual reviews can consume screenshot/media metadata through `media_attached`, `media_linked`, and `media_unavailable` states without breaking text-only channels. The default next action is P19 `Multi-Channel Alert Routing`.
|
||||
Automation execution-layer P19 is closed locally. P19 adds structured channel routing and channel health observability so alert dispatch can distinguish enabled workbench/webhook paths from disabled WeCom/Email adapters with credential-missing reasons. The default next action is P20 `Operator Digest And Review Queue`.
|
||||
|
||||
## Current Branch
|
||||
|
||||
@@ -14,8 +14,8 @@ Automation execution-layer P18 is closed locally. P18 adds a stable visual media
|
||||
|
||||
## Automation Status
|
||||
|
||||
- state: automation_execution_layer_p18_visual_evidence_media_dispatch_closed
|
||||
- current_phase: P18 Visual Evidence Media Dispatch
|
||||
- state: automation_execution_layer_p19_multi_channel_alert_routing_closed
|
||||
- current_phase: P19 Multi-Channel Alert Routing
|
||||
- latest_p7_selector_choice: continue_periodic_analysis_decision_monitoring_with_maturity_rollup
|
||||
- latest_p14_slice: webhook_transport_retry
|
||||
- latest_p15_pxx_tasklist_status: ready
|
||||
@@ -37,7 +37,12 @@ Automation execution-layer P18 is closed locally. P18 adds a stable visual media
|
||||
- latest_visual_media_evidence_status: implemented
|
||||
- latest_visual_media_evidence_dispatch_status: visible
|
||||
- latest_visual_media_evidence_smoke_status: no_pending_alerts_safe_noop
|
||||
- next_execution_layer_phase: P19 Multi-Channel Alert Routing
|
||||
- latest_p19_slice: multi_channel_alert_routing
|
||||
- latest_channel_router_status: implemented
|
||||
- latest_channel_health_status: visible
|
||||
- latest_disabled_adapter_status: visible_credential_missing
|
||||
- latest_channel_routing_smoke_status: no_pending_alerts_safe_noop
|
||||
- next_execution_layer_phase: P20 Operator Digest And Review Queue
|
||||
- latest_visual_review_script_status: idle
|
||||
- latest_runtime_visual_watch_task_status: ok
|
||||
- latest_visual_capture_cleanup_task_status: ok
|
||||
@@ -110,6 +115,9 @@ Automation execution-layer P18 is closed locally. P18 adds a stable visual media
|
||||
- Added `build_visual_media_evidence()` and `summarize_visual_media_evidence()` so visual reviews produce a stable media evidence block.
|
||||
- Extended alert dispatcher delivery payloads, dispatcher results, alert messages, alert quality summary, runtime `alert_dispatch` details, and `run_alert_dispatcher.py` with media evidence status.
|
||||
- Extended visual review API enrichment and workbench Visual Reviews with media evidence status and primary media path.
|
||||
- Added structured channel route/capability helpers with workbench, webhook, WeCom, and email channel descriptions.
|
||||
- Added channel health summary and disabled adapter metadata for WeCom/Email credential-missing states.
|
||||
- Extended alert dispatcher payloads/results, delivery summary, runtime `alert_dispatch` details, dispatcher script output, and workbench delivery cards with per-channel observability.
|
||||
|
||||
## Current Working State
|
||||
|
||||
@@ -131,6 +139,8 @@ Automation execution-layer P18 is closed locally. P18 adds a stable visual media
|
||||
- P17 visibility is implemented: decision alert payloads carry `visual_trigger_policy`/`visual_trigger_roi`, runtime exposes trigger/skip counts, visual review summaries count trigger types and review results, and the workbench displays trigger counters.
|
||||
- P18 visual media evidence dispatch is implemented: visual review context now becomes a stable `media_evidence` block with channel-aware `media_attached`, `media_linked`, or `media_unavailable` status.
|
||||
- P18 visibility is implemented: dispatcher payloads/results, delivery summary, runtime `alert_dispatch` details, script output, visual review API, and workbench panels expose media evidence status.
|
||||
- P19 multi-channel alert routing is implemented: routing now returns `channel_route`, `channel_health`, channel capability metadata, and explicit disabled-adapter states for WeCom/Email when credentials/adapters are unavailable.
|
||||
- P19 visibility is implemented: dispatcher result exposes `channel_summary` / `channel_health` / `channel_disabled`, delivery summary exposes `channel_status_counts`, runtime `alert_dispatch` details expose channel health, and the workbench `Alert Quality` panel shows channel counts.
|
||||
- Latest `scripts/run_visual_review.py` smoke found no pending reviews: attempted=0, reviewed=0, status=idle.
|
||||
- Latest capture smoke returned `status=ignored` when no watch run was supplied, which is the expected safe no-op.
|
||||
- Latest cleanup smoke deleted one expired temporary artifact through `scripts/run_visual_capture_cleanup.py`.
|
||||
@@ -140,7 +150,7 @@ Automation execution-layer P18 is closed locally. P18 adds a stable visual media
|
||||
- Latest `scripts/run_trading_runtime.py` smoke returned blockers=none and includes a healthy `visual_watch` runtime task.
|
||||
- Latest local runtime smoke confirmed `runtime_status.tasks.alert_dispatch.details` now exposes retry and silent-window counters with blockers=none.
|
||||
|
||||
## P18 Blockers
|
||||
## P19 Blockers
|
||||
|
||||
- none
|
||||
|
||||
@@ -159,6 +169,7 @@ Automation execution-layer P18 is closed locally. P18 adds a stable visual media
|
||||
- P16 browser screenshot capture remains advisory evidence only and must not change execution permission, live readiness, trust bucket, risk governor, or operator confirmation.
|
||||
- P17 visual trigger routing remains advisory evidence routing only and must not change execution permission, live readiness, trust bucket, risk governor, or operator confirmation.
|
||||
- P18 media evidence remains delivery metadata/fallback routing only and must not change execution permission, live readiness, trust bucket, risk governor, or operator confirmation.
|
||||
- P19 channel routing remains notification routing/observability only and must not change execution permission, live readiness, trust bucket, risk governor, or operator confirmation.
|
||||
- A final rerun of `scripts/run_alert_dispatcher.py` and `scripts/run_trading_runtime.py` hit local PostgreSQL connection timeout because nothing was listening on `127.0.0.1:5432`; the P14 code path is still covered by full unit tests and earlier successful smoke during implementation.
|
||||
- P15 dispatcher smoke succeeded with `suppressed_noise_policy`, `workbench_only`, and `policy_external_allowed` counters present. P15 runtime smoke reached `alert_dispatch` successfully, but the overall runtime returned external environment blockers for market stream disconnected and OKX read transport connection refused.
|
||||
- P16 direct browser smoke succeeded locally with `browser_capture_status=captured`. Script-level capture without a watch run still returns safe no-op `watch_run_missing`.
|
||||
@@ -168,11 +179,14 @@ Automation execution-layer P18 is closed locally. P18 adds a stable visual media
|
||||
- P18 dispatcher smoke returned attempted=0 and media counters present as safe no-op: `media_attached=0`, `media_linked=0`, `media_unavailable=0`.
|
||||
- P18 quality-summary API smoke returned 200 and exposed `delivery_summary.media_attached_count`, `media_linked_count`, and `media_unavailable_count`.
|
||||
- Latest P18 runtime smoke returned blockers=none and exposes `alert_dispatch.details.media_attached/media_linked/media_unavailable`.
|
||||
- P19 dispatcher smoke returned attempted=0 and exposed `channel_health.enabled_channels=["workbench"]` plus disabled `webhook`, `wecom`, and `email` credential-missing states.
|
||||
- P19 quality-summary API smoke returned 200 and exposed `delivery_summary.channel_status_counts`.
|
||||
- Latest P19 runtime smoke returned blockers=none and exposes `alert_dispatch.details.channel_health` and `channel_summary`.
|
||||
- Real-live order submission remains intentionally disabled because the OKX order adapter is still not a real-live submitter.
|
||||
|
||||
## Exact Next Step
|
||||
|
||||
P18 is locally closed. Default next action is P19 `Multi-Channel Alert Routing`: extend channel capability/routing observability with disabled adapters and per-channel health/status counts, without changing trading gates.
|
||||
P19 is locally closed. Default next action is P20 `Operator Digest And Review Queue`: aggregate recent alerts, visual reviews, feedback, suppression, retries, media, and channel health into an operator-facing digest/review queue.
|
||||
|
||||
## Validation Commands Used Recently
|
||||
|
||||
@@ -192,4 +206,5 @@ D:\ai-exchange\.venv\Scripts\python.exe -m unittest discover -s tests/unit
|
||||
D:\ai-exchange\.venv\Scripts\python.exe scripts\run_visual_review.py
|
||||
D:\ai-exchange\.venv\Scripts\python.exe -m unittest tests.unit.test_visual_media_evidence_service tests.unit.test_alert_feedback_service tests.unit.test_alert_quality_summary_service tests.unit.test_trading_runtime_service tests.unit.test_api_and_main
|
||||
D:\ai-exchange\.venv\Scripts\python.exe scripts\run_alert_dispatcher.py
|
||||
D:\ai-exchange\.venv\Scripts\python.exe -m unittest tests.unit.test_channel_router_service tests.unit.test_alert_feedback_service tests.unit.test_alert_quality_summary_service tests.unit.test_trading_runtime_service
|
||||
```
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
|
||||
## In Progress
|
||||
|
||||
- [x] P18 Visual Evidence Media Dispatch local closure complete
|
||||
- [x] P19 Multi-Channel Alert Routing local closure complete
|
||||
|
||||
## Next Up
|
||||
|
||||
@@ -29,14 +29,15 @@
|
||||
- [x] P16 local closure complete
|
||||
- [x] P17 local closure complete
|
||||
- [x] P18 local closure complete
|
||||
- [ ] Start P19 Multi-Channel Alert Routing
|
||||
- [x] P19 local closure complete
|
||||
- [ ] Start P20 Operator Digest And Review Queue
|
||||
- [ ] Continue periodic analysis-decision monitoring with maturity rollup in parallel
|
||||
- [ ] Treat browser screenshot capture and additional external media adapters as future enhancements
|
||||
|
||||
## Automation Status
|
||||
|
||||
- [x] state: automation_execution_layer_p18_visual_evidence_media_dispatch_closed
|
||||
- [x] current_phase: P18 Visual Evidence Media Dispatch
|
||||
- [x] state: automation_execution_layer_p19_multi_channel_alert_routing_closed
|
||||
- [x] current_phase: P19 Multi-Channel Alert Routing
|
||||
- [x] latest_p7_selector_choice: continue_periodic_analysis_decision_monitoring_with_maturity_rollup
|
||||
- [x] latest_p14_slice: webhook_transport_retry
|
||||
- [x] latest_p15_pxx_tasklist_status: ready
|
||||
@@ -58,7 +59,12 @@
|
||||
- [x] latest_visual_media_evidence_status: implemented
|
||||
- [x] latest_visual_media_evidence_dispatch_status: visible
|
||||
- [x] latest_visual_media_evidence_smoke_status: no_pending_alerts_safe_noop
|
||||
- [x] next_execution_layer_phase: P19 Multi-Channel Alert Routing
|
||||
- [x] latest_p19_slice: multi_channel_alert_routing
|
||||
- [x] latest_channel_router_status: implemented
|
||||
- [x] latest_channel_health_status: visible
|
||||
- [x] latest_disabled_adapter_status: visible_credential_missing
|
||||
- [x] latest_channel_routing_smoke_status: no_pending_alerts_safe_noop
|
||||
- [x] next_execution_layer_phase: P20 Operator Digest And Review Queue
|
||||
- [x] latest_visual_review_script_status: idle
|
||||
- [x] latest_runtime_visual_watch_task_status: ok
|
||||
- [x] latest_visual_capture_cleanup_task_status: ok
|
||||
@@ -144,6 +150,11 @@
|
||||
- [x] Added `media_attached`, `media_linked`, and `media_unavailable` counters to dispatcher, delivery summary, runtime `alert_dispatch`, and dispatcher script output
|
||||
- [x] Added media evidence status and primary path visibility to Visual Reviews API/workbench
|
||||
- [x] Added focused P18 media evidence, dispatcher, quality summary, runtime, and API tests
|
||||
- [x] Added structured channel route/capability helpers for workbench, webhook, WeCom, and email
|
||||
- [x] Added channel health summary with enabled/disabled/credential-missing channel visibility
|
||||
- [x] Added disabled adapter states for WeCom and email when credentials or adapter support are unavailable
|
||||
- [x] Added per-channel summary to dispatcher results, delivery summary, runtime `alert_dispatch`, dispatcher script output, and workbench Alert Quality
|
||||
- [x] Added focused P19 channel router, disabled channel, summary, and runtime tests
|
||||
- [x] Added runtime `visual_watch` task
|
||||
- [x] Added focused service, script, API, and runtime tests
|
||||
- [x] Smoked the visual review script with no pending reviews
|
||||
@@ -163,6 +174,9 @@
|
||||
- [x] Smoked P18 dispatcher safe no-op with media counters visible
|
||||
- [x] Smoked P18 quality-summary API with media counters visible
|
||||
- [x] Smoked P18 trading runtime with blockers=none and alert dispatch media counters visible
|
||||
- [x] Smoked P19 dispatcher safe no-op with channel health visible
|
||||
- [x] Smoked P19 quality-summary API with channel status counts visible
|
||||
- [x] Smoked P19 trading runtime with blockers=none and alert dispatch channel health visible
|
||||
|
||||
## Blocked
|
||||
|
||||
@@ -177,7 +191,7 @@
|
||||
|
||||
## Future Enhancements Not Blocking P8
|
||||
|
||||
- [ ] P19 Multi-Channel Alert Routing
|
||||
- [x] none currently tracked outside the finite P15-PXX queue
|
||||
|
||||
## Deferred Until Real-Trading Phase
|
||||
|
||||
@@ -217,7 +231,7 @@
|
||||
- [x] `P16` Browser Screenshot Capture Adapter
|
||||
- [x] `P17` Visual Trigger ROI Router
|
||||
- [x] `P18` Visual Evidence Media Dispatch
|
||||
- [ ] `P19` Multi-Channel Alert Routing
|
||||
- [x] `P19` Multi-Channel Alert Routing
|
||||
- [ ] `P20` Operator Digest And Review Queue
|
||||
- [ ] `P21` Runtime Incident Journal
|
||||
- [ ] `P22` Maintenance And Retention Rollup
|
||||
|
||||
@@ -408,6 +408,7 @@ candidate -> decision alert -> dispatcher -> 外部消息 -> delivery record
|
||||
- 2026-05-12 P16 已完成 `browser screenshot capture adapter` 本地闭环：visual snapshot capture 现在在保留 HTML/JSON artifact 的同时，可通过本机 Edge/Chrome headless 生成 PNG screenshot，并把 browser capture status、HTML path、screenshot path 接入脚本、API 与 workbench Visual Reviews。
|
||||
- 2026-05-12 P17 已完成 `visual trigger ROI router` 本地闭环：视觉触发策略现在区分 `triggered` / `skipped`，普通 observe/monitoring 状态不触发视觉复核，actionable、blocked、market degraded、broker/reconciliation degraded 与 council watch item 会输出可审计 trigger type / reason codes，并在 runtime、alert quality summary、脚本和 workbench 中暴露 trigger/skip/ROI 计数。
|
||||
- 2026-05-12 P18 已完成 `visual evidence media dispatch` 本地闭环：visual review 现在会生成稳定 `media_evidence` block，外部文本通道不支持文件时降级为 `media_linked`，workbench 可消费本地截图路径时标记 `media_attached`，无视觉证据时标记 `media_unavailable`；dispatcher、delivery summary、runtime、脚本和 workbench 均可见媒体证据状态。
|
||||
- 2026-05-12 P19 已完成 `multi-channel alert routing` 本地闭环：channel router 现在输出 workbench/webhook/WeCom/Email capability 与 health summary，WeCom/Email 在凭据缺失或 adapter 未实现时会以 disabled adapter 形式返回 `credential_missing` / `adapter_not_implemented`，dispatcher、delivery summary、runtime、脚本和 workbench 均可见 per-channel 状态。
|
||||
|
||||
### 验收
|
||||
- 非重大错误不打断主循环
|
||||
|
||||
@@ -107,6 +107,8 @@
|
||||
- `candidate_blocked` -> dashboard only
|
||||
- `runtime_degraded` -> Telegram + dashboard
|
||||
|
||||
2026-05-12 P19 已落地：当前 channel router 保持现有 workbench/webhook 行为，同时输出结构化 `channel_route`、channel capability 与 channel health summary。WeCom / Email 已预留为 disabled adapter；凭据缺失时暴露 `credential_missing`，凭据存在但 adapter 未实现时暴露 `adapter_not_implemented`，不会把未配置通道伪装为成功发送。dispatcher、delivery summary、runtime 与 workbench 均可查看 per-channel 状态。
|
||||
|
||||
#### `src/services/alerts/delivery_policy_service.py`
|
||||
职责：
|
||||
|
||||
|
||||
@@ -189,6 +189,13 @@
|
||||
- 凭据缺失时显示 `channel_disabled` 或 `credential_missing`，不伪装成功。
|
||||
- 现有 webhook/workbench 行为不回归。
|
||||
|
||||
### 当前进展
|
||||
|
||||
- 2026-05-12 P19 已完成 `Multi-Channel Alert Routing` 本地闭环：`ChannelRouterService` 现在输出结构化 `channel_route`、channel capability 与 channel health summary，覆盖 workbench、webhook、WeCom、Email。
|
||||
- 2026-05-12 P19 已将 WeCom/Email 预留为 disabled adapter：凭据缺失时显示 `credential_missing`，即使未来填入凭据但 adapter 未实现也会保持 `adapter_not_implemented`，不会伪装成成功发送。
|
||||
- 2026-05-12 P19 已在 dispatcher result、delivery payload、delivery summary、runtime `alert_dispatch` details、`run_alert_dispatcher.py` 与 workbench `Alert Quality` 中暴露 per-channel status、channel disabled count 与 channel health。
|
||||
- 2026-05-12 P19 本地闭环已验证：focused P19 tests 通过，`tests/unit` 全量 679 个测试通过，script static checks 与 `node --check src/web/app.js` 通过；dispatcher smoke 在无 pending alert 时安全 no-op 且 channel health 可见，quality-summary API 与 runtime smoke 均可见 channel counts/health 且 runtime blockers=none。
|
||||
|
||||
---
|
||||
|
||||
## 10. P20 Operator Digest And Review Queue
|
||||
|
||||
@@ -36,6 +36,9 @@ def main(env=None) -> int:
|
||||
"Media attached / linked / unavailable: "
|
||||
f"{payload.get('media_attached', 0)} / {payload.get('media_linked', 0)} / {payload.get('media_unavailable', 0)}"
|
||||
)
|
||||
print(f"Channel disabled: {payload.get('channel_disabled', 0)}")
|
||||
if payload.get("channel_summary"):
|
||||
print(f"Channel summary: {json.dumps(payload.get('channel_summary'), ensure_ascii=False, sort_keys=True)}")
|
||||
items = payload.get("items") or []
|
||||
return 0 if all(item.get("status") != "failed" for item in items) else 2
|
||||
|
||||
|
||||
@@ -121,6 +121,9 @@ class Settings(BaseSettings):
|
||||
alert_noise_suppressed_operator_modes: str = "silent,workbench_only"
|
||||
alert_default_channel: str = "workbench"
|
||||
alert_webhook_url: str = ""
|
||||
alert_wecom_webhook_url: str = ""
|
||||
alert_email_smtp_url: str = ""
|
||||
alert_email_to: str = ""
|
||||
default_timezone: str = "UTC"
|
||||
session_timezone: str = "America/New_York"
|
||||
log_level: str = "INFO"
|
||||
|
||||
@@ -9,7 +9,7 @@ from src.config.settings import settings
|
||||
from src.repositories.alert_delivery_repository import AlertDeliveryRepository
|
||||
from src.repositories.decision_packet_repository import DecisionPacketRepository
|
||||
from src.services.feedback import AlertFeedbackService
|
||||
from src.services.alerts.channel_router_service import select_alert_channel
|
||||
from src.services.alerts.channel_router_service import build_channel_health_summary, select_alert_route
|
||||
from src.services.alerts.delivery_policy_service import evaluate_alert_delivery_policy
|
||||
from src.services.alerts.alert_silent_window_service import AlertSilentWindowService
|
||||
from src.services.alerts.message_template_service import build_alert_message
|
||||
@@ -74,6 +74,9 @@ class AlertDispatcherService:
|
||||
"media_attached": sum(1 for item in items if item.get("media_status") == "media_attached"),
|
||||
"media_linked": sum(1 for item in items if item.get("media_status") == "media_linked"),
|
||||
"media_unavailable": sum(1 for item in items if item.get("media_status") == "media_unavailable"),
|
||||
"channel_summary": _build_dispatch_channel_summary(items),
|
||||
"channel_health": build_channel_health_summary(),
|
||||
"channel_disabled": sum(1 for item in items if item.get("reason") == "channel_disabled"),
|
||||
"items": items,
|
||||
"silent_window": silent_window,
|
||||
"suppressed_silent_window": sum(1 for item in items if item.get("reason") == "alert_silent_window_active"),
|
||||
@@ -84,7 +87,9 @@ class AlertDispatcherService:
|
||||
if not alert:
|
||||
return {"alert_id": alert_id, "status": "missing"}
|
||||
|
||||
channel = select_alert_channel(alert)
|
||||
channel_route = select_alert_route(alert)
|
||||
channel = str(channel_route.get("selected_channel") or "workbench")
|
||||
channel_capability = dict(channel_route.get("capability") or {})
|
||||
packet = None
|
||||
packet_id = alert.get("decision_packet_id")
|
||||
if packet_id:
|
||||
@@ -113,6 +118,38 @@ class AlertDispatcherService:
|
||||
reason = str(delivery_policy.get("reason") or "")
|
||||
attempted_ts = datetime.now(timezone.utc)
|
||||
|
||||
if not bool(channel_capability.get("enabled", True)):
|
||||
reason = "channel_disabled"
|
||||
self._safe_update_alert_status(alert_id=alert_id, status="skipped", channel=channel)
|
||||
self._safe_insert_delivery(
|
||||
alert_id=alert_id,
|
||||
channel=channel,
|
||||
status="skipped",
|
||||
attempted_ts=attempted_ts,
|
||||
error=reason,
|
||||
payload={
|
||||
"reason": reason,
|
||||
"message": message,
|
||||
"dedupe_scope": dedupe_scope,
|
||||
"visual": visual,
|
||||
"media_evidence": media_evidence,
|
||||
"delivery_policy": delivery_policy,
|
||||
"channel_route": channel_route,
|
||||
},
|
||||
)
|
||||
return {
|
||||
"alert_id": alert_id,
|
||||
"status": "skipped",
|
||||
"dispatch_mode": "failed_retry" if str(alert.get("status") or "") == "failed" else "queued",
|
||||
"channel": channel,
|
||||
"reason": reason,
|
||||
"channel_route": channel_route,
|
||||
"delivery_policy": delivery_policy,
|
||||
"media_status": media_evidence.get("status"),
|
||||
"media_evidence": media_summary,
|
||||
"visual": self._visual_result_summary(visual, media_evidence=media_evidence),
|
||||
}
|
||||
|
||||
if str(alert.get("status") or "") == "failed" and not allow_failed_retry:
|
||||
reason = str(retry_state.get("reason") or "retry_not_allowed")
|
||||
if reason == "retry_backoff_active":
|
||||
@@ -124,6 +161,7 @@ class AlertDispatcherService:
|
||||
"reason": reason,
|
||||
"retry_attempt_count": retry_state.get("retry_attempt_count"),
|
||||
"next_retry_ts": retry_state.get("next_retry_ts"),
|
||||
"channel_route": channel_route,
|
||||
"delivery_policy": delivery_policy,
|
||||
"media_status": media_evidence.get("status"),
|
||||
"media_evidence": media_summary,
|
||||
@@ -137,6 +175,7 @@ class AlertDispatcherService:
|
||||
"channel": channel,
|
||||
"reason": reason,
|
||||
"next_retry_ts": retry_state.get("next_retry_ts"),
|
||||
"channel_route": channel_route,
|
||||
"delivery_policy": delivery_policy,
|
||||
"media_status": media_evidence.get("status"),
|
||||
"media_evidence": media_summary,
|
||||
@@ -156,6 +195,7 @@ class AlertDispatcherService:
|
||||
"visual": visual,
|
||||
"media_evidence": media_evidence,
|
||||
"delivery_policy": delivery_policy,
|
||||
"channel_route": channel_route,
|
||||
"dispatch_mode": "failed_retry",
|
||||
"retry_attempt_count": retry_state.get("retry_attempt_count"),
|
||||
},
|
||||
@@ -167,6 +207,7 @@ class AlertDispatcherService:
|
||||
"channel": channel,
|
||||
"reason": reason,
|
||||
"retry_attempt_count": retry_state.get("retry_attempt_count"),
|
||||
"channel_route": channel_route,
|
||||
"delivery_policy": delivery_policy,
|
||||
"media_status": media_evidence.get("status"),
|
||||
"media_evidence": media_summary,
|
||||
@@ -181,7 +222,7 @@ class AlertDispatcherService:
|
||||
status="skipped",
|
||||
attempted_ts=attempted_ts,
|
||||
error=reason,
|
||||
payload={"reason": reason, "message": message, "dedupe_scope": dedupe_scope, "visual": visual, "media_evidence": media_evidence, "delivery_policy": delivery_policy},
|
||||
payload={"reason": reason, "message": message, "dedupe_scope": dedupe_scope, "visual": visual, "media_evidence": media_evidence, "delivery_policy": delivery_policy, "channel_route": channel_route},
|
||||
)
|
||||
return {
|
||||
"alert_id": alert_id,
|
||||
@@ -189,6 +230,7 @@ class AlertDispatcherService:
|
||||
"dispatch_mode": "failed_retry" if str(alert.get("status") or "") == "failed" else "queued",
|
||||
"channel": channel,
|
||||
"reason": reason,
|
||||
"channel_route": channel_route,
|
||||
"delivery_policy": delivery_policy,
|
||||
"media_status": media_evidence.get("status"),
|
||||
"media_evidence": media_summary,
|
||||
@@ -202,7 +244,7 @@ class AlertDispatcherService:
|
||||
channel=channel,
|
||||
status="skipped",
|
||||
attempted_ts=attempted_ts,
|
||||
payload={"message": message, "reason": "workbench_only", "dedupe_scope": dedupe_scope, "visual": visual, "media_evidence": media_evidence, "delivery_policy": delivery_policy},
|
||||
payload={"message": message, "reason": "workbench_only", "dedupe_scope": dedupe_scope, "visual": visual, "media_evidence": media_evidence, "delivery_policy": delivery_policy, "channel_route": channel_route},
|
||||
)
|
||||
return {
|
||||
"alert_id": alert_id,
|
||||
@@ -210,6 +252,7 @@ class AlertDispatcherService:
|
||||
"dispatch_mode": "failed_retry" if str(alert.get("status") or "") == "failed" else "queued",
|
||||
"channel": channel,
|
||||
"reason": "workbench_only",
|
||||
"channel_route": channel_route,
|
||||
"delivery_policy": delivery_policy,
|
||||
"media_status": media_evidence.get("status"),
|
||||
"media_evidence": media_summary,
|
||||
@@ -232,7 +275,7 @@ class AlertDispatcherService:
|
||||
attempted_ts=attempted_ts,
|
||||
delivered_ts=delivered_ts,
|
||||
external_message_id=result.get("external_message_id"),
|
||||
payload={"message": message, "result": result, "dedupe_scope": dedupe_scope, "visual": visual, "media_evidence": media_evidence, "delivery_policy": delivery_policy},
|
||||
payload={"message": message, "result": result, "dedupe_scope": dedupe_scope, "visual": visual, "media_evidence": media_evidence, "delivery_policy": delivery_policy, "channel_route": channel_route},
|
||||
)
|
||||
return {
|
||||
"alert_id": alert_id,
|
||||
@@ -243,6 +286,7 @@ class AlertDispatcherService:
|
||||
"delivery_write_ok": delivery_write_ok,
|
||||
"retry_attempt_count": retry_state.get("retry_attempt_count"),
|
||||
"transport_attempt_count": int(result.get("attempt_count") or 1),
|
||||
"channel_route": channel_route,
|
||||
"delivery_policy": delivery_policy,
|
||||
"media_status": media_evidence.get("status"),
|
||||
"media_evidence": media_summary,
|
||||
@@ -257,7 +301,7 @@ class AlertDispatcherService:
|
||||
status="failed",
|
||||
attempted_ts=attempted_ts,
|
||||
error=str(exc),
|
||||
payload={"message": message, "dedupe_scope": dedupe_scope, "visual": visual, "media_evidence": media_evidence, "transport_attempt_count": transport_attempt_count, "delivery_policy": delivery_policy},
|
||||
payload={"message": message, "dedupe_scope": dedupe_scope, "visual": visual, "media_evidence": media_evidence, "transport_attempt_count": transport_attempt_count, "delivery_policy": delivery_policy, "channel_route": channel_route},
|
||||
)
|
||||
return {
|
||||
"alert_id": alert_id,
|
||||
@@ -269,6 +313,7 @@ class AlertDispatcherService:
|
||||
"delivery_write_ok": delivery_write_ok,
|
||||
"retry_attempt_count": retry_state.get("retry_attempt_count"),
|
||||
"transport_attempt_count": transport_attempt_count,
|
||||
"channel_route": channel_route,
|
||||
"delivery_policy": delivery_policy,
|
||||
"media_status": media_evidence.get("status"),
|
||||
"media_evidence": media_summary,
|
||||
@@ -451,6 +496,30 @@ def _snapshot_artifact_path(visual: dict | None) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def _build_dispatch_channel_summary(items: list[dict]) -> dict:
|
||||
summary: dict[str, dict] = {}
|
||||
for item in list(items or []):
|
||||
channel = str(item.get("channel") or "unknown")
|
||||
channel_summary = summary.setdefault(
|
||||
channel,
|
||||
{
|
||||
"attempted": 0,
|
||||
"delivered": 0,
|
||||
"failed": 0,
|
||||
"skipped": 0,
|
||||
"deferred": 0,
|
||||
"channel_disabled": 0,
|
||||
},
|
||||
)
|
||||
channel_summary["attempted"] += 1
|
||||
status = str(item.get("status") or "unknown")
|
||||
if status in channel_summary:
|
||||
channel_summary[status] += 1
|
||||
if item.get("reason") == "channel_disabled":
|
||||
channel_summary["channel_disabled"] += 1
|
||||
return summary
|
||||
|
||||
|
||||
def _parse_ts(value) -> datetime | None:
|
||||
if value in (None, ""):
|
||||
return None
|
||||
|
||||
@@ -136,6 +136,8 @@ def _build_delivery_summary(deliveries: list[dict]) -> dict:
|
||||
"media_attached_count": sum(1 for item in items if _media_status(item) == "media_attached"),
|
||||
"media_linked_count": sum(1 for item in items if _media_status(item) == "media_linked"),
|
||||
"media_unavailable_count": sum(1 for item in items if _media_status(item) == "media_unavailable"),
|
||||
"channel_status_counts": _build_channel_status_counts(items),
|
||||
"channel_disabled_count": sum(1 for item in items if str((item.get("payload") or {}).get("reason") or item.get("error") or "") == "channel_disabled"),
|
||||
"external_attempt_count": external_attempts,
|
||||
"success_rate": _rate(delivered, external_attempts),
|
||||
"latest_status": latest.get("status"),
|
||||
@@ -177,6 +179,16 @@ def _media_status(delivery: dict) -> str:
|
||||
return str(evidence.get("status") or "")
|
||||
|
||||
|
||||
def _build_channel_status_counts(deliveries: list[dict]) -> dict:
|
||||
counts: dict[str, dict[str, int]] = {}
|
||||
for delivery in deliveries:
|
||||
channel = str(delivery.get("channel") or "unknown")
|
||||
status = str(delivery.get("status") or "unknown")
|
||||
channel_counts = counts.setdefault(channel, {})
|
||||
channel_counts[status] = channel_counts.get(status, 0) + 1
|
||||
return counts
|
||||
|
||||
|
||||
def _build_visual_review_summary(rows: list[dict]) -> dict:
|
||||
items = list(rows or [])
|
||||
state_counts: dict[str, int] = {}
|
||||
|
||||
@@ -3,14 +3,128 @@ from __future__ import annotations
|
||||
from src.config.settings import settings
|
||||
|
||||
|
||||
CHANNEL_WORKBENCH = "workbench"
|
||||
CHANNEL_WEBHOOK = "webhook"
|
||||
CHANNEL_WECOM = "wecom"
|
||||
CHANNEL_EMAIL = "email"
|
||||
|
||||
WEBHOOK_ALERT_TYPES = {"candidate_actionable", "ticket_ready", "runtime_degraded"}
|
||||
KNOWN_CHANNELS = [CHANNEL_WORKBENCH, CHANNEL_WEBHOOK, CHANNEL_WECOM, CHANNEL_EMAIL]
|
||||
|
||||
|
||||
def select_alert_channel(alert: dict) -> str:
|
||||
return select_alert_route(alert)["selected_channel"]
|
||||
|
||||
|
||||
def select_alert_route(alert: dict) -> dict:
|
||||
alert_type = str(alert.get("alert_type") or "")
|
||||
action = str((alert.get("payload") or {}).get("action_recommendation") or "")
|
||||
if alert_type in WEBHOOK_ALERT_TYPES and settings.alert_webhook_url:
|
||||
return "webhook"
|
||||
if alert_type == "candidate_blocked" and action == "do_not_execute" and settings.alert_webhook_url:
|
||||
return "webhook"
|
||||
return settings.alert_default_channel
|
||||
selected = CHANNEL_WEBHOOK
|
||||
rule = "webhook_high_priority_alert"
|
||||
elif alert_type == "candidate_blocked" and action == "do_not_execute" and settings.alert_webhook_url:
|
||||
selected = CHANNEL_WEBHOOK
|
||||
rule = "webhook_do_not_execute_block"
|
||||
else:
|
||||
selected = _normalize_channel(settings.alert_default_channel)
|
||||
rule = "default_channel"
|
||||
capability = get_channel_capability(selected)
|
||||
return {
|
||||
"selected_channel": selected,
|
||||
"route_rule": rule,
|
||||
"capability": capability,
|
||||
"channel_health": build_channel_health_summary(),
|
||||
}
|
||||
|
||||
|
||||
def get_channel_capability(channel: str) -> dict:
|
||||
normalized = _normalize_channel(channel)
|
||||
if normalized == CHANNEL_WORKBENCH:
|
||||
return {
|
||||
"channel": CHANNEL_WORKBENCH,
|
||||
"enabled": True,
|
||||
"status": "enabled",
|
||||
"reason": None,
|
||||
"supports_text": True,
|
||||
"supports_media_attachment": True,
|
||||
"supports_media_link": True,
|
||||
"adapter": "workbench",
|
||||
}
|
||||
if normalized == CHANNEL_WEBHOOK:
|
||||
enabled = bool(settings.alert_webhook_url)
|
||||
return _external_capability(
|
||||
channel=CHANNEL_WEBHOOK,
|
||||
enabled=enabled,
|
||||
adapter="webhook",
|
||||
credential_key="alert_webhook_url",
|
||||
supports_media_attachment=False,
|
||||
)
|
||||
if normalized == CHANNEL_WECOM:
|
||||
has_credential = bool(settings.alert_wecom_webhook_url)
|
||||
return _external_capability(
|
||||
channel=CHANNEL_WECOM,
|
||||
enabled=False,
|
||||
adapter="wecom_disabled_adapter",
|
||||
credential_key="alert_wecom_webhook_url",
|
||||
supports_media_attachment=False,
|
||||
disabled_reason="adapter_not_implemented" if has_credential else "credential_missing",
|
||||
)
|
||||
if normalized == CHANNEL_EMAIL:
|
||||
has_credential = bool(settings.alert_email_smtp_url and settings.alert_email_to)
|
||||
missing_key = "alert_email_smtp_url_or_to"
|
||||
return _external_capability(
|
||||
channel=CHANNEL_EMAIL,
|
||||
enabled=False,
|
||||
adapter="email_disabled_adapter",
|
||||
credential_key=missing_key,
|
||||
supports_media_attachment=False,
|
||||
disabled_reason="adapter_not_implemented" if has_credential else "credential_missing",
|
||||
)
|
||||
return {
|
||||
"channel": normalized,
|
||||
"enabled": False,
|
||||
"status": "disabled",
|
||||
"reason": "unknown_channel",
|
||||
"supports_text": False,
|
||||
"supports_media_attachment": False,
|
||||
"supports_media_link": False,
|
||||
"adapter": "unknown",
|
||||
}
|
||||
|
||||
|
||||
def build_channel_health_summary(channels: list[str] | None = None) -> dict:
|
||||
capabilities = [get_channel_capability(channel) for channel in list(channels or KNOWN_CHANNELS)]
|
||||
disabled = [item for item in capabilities if not item.get("enabled")]
|
||||
return {
|
||||
"channels": {item["channel"]: item for item in capabilities},
|
||||
"enabled_channels": [item["channel"] for item in capabilities if item.get("enabled")],
|
||||
"disabled_channels": [item["channel"] for item in disabled],
|
||||
"credential_missing_channels": [item["channel"] for item in disabled if item.get("reason") == "credential_missing"],
|
||||
"disabled_count": len(disabled),
|
||||
}
|
||||
|
||||
|
||||
def _external_capability(
|
||||
*,
|
||||
channel: str,
|
||||
enabled: bool,
|
||||
adapter: str,
|
||||
credential_key: str,
|
||||
supports_media_attachment: bool,
|
||||
disabled_reason: str = "credential_missing",
|
||||
) -> dict:
|
||||
return {
|
||||
"channel": channel,
|
||||
"enabled": enabled,
|
||||
"status": "enabled" if enabled else "disabled",
|
||||
"reason": None if enabled else disabled_reason,
|
||||
"credential_key": credential_key,
|
||||
"supports_text": True,
|
||||
"supports_media_attachment": supports_media_attachment,
|
||||
"supports_media_link": True,
|
||||
"adapter": adapter,
|
||||
}
|
||||
|
||||
|
||||
def _normalize_channel(channel: str) -> str:
|
||||
return str(channel or CHANNEL_WORKBENCH).strip().lower() or CHANNEL_WORKBENCH
|
||||
|
||||
@@ -549,6 +549,9 @@ class TradingRuntimeScheduler:
|
||||
"media_attached": int(payload.get("media_attached") or 0),
|
||||
"media_linked": int(payload.get("media_linked") or 0),
|
||||
"media_unavailable": int(payload.get("media_unavailable") or 0),
|
||||
"channel_disabled": int(payload.get("channel_disabled") or 0),
|
||||
"channel_summary": dict(payload.get("channel_summary") or {}),
|
||||
"channel_health": dict(payload.get("channel_health") or {}),
|
||||
"delivered": sum(1 for item in items if item.get("status") == "delivered"),
|
||||
"skipped": sum(1 for item in items if item.get("status") == "skipped"),
|
||||
"failed": len(failed),
|
||||
|
||||
@@ -589,6 +589,8 @@ function renderAlertQuality(summary) {
|
||||
['transport retry failed', String(delivery.transport_retry_failed_count ?? 0)],
|
||||
['policy external allowed', String(delivery.policy_external_allowed_count ?? 0)],
|
||||
['media attached / linked / unavailable', `${delivery.media_attached_count ?? 0} / ${delivery.media_linked_count ?? 0} / ${delivery.media_unavailable_count ?? 0}`],
|
||||
['channel disabled', String(delivery.channel_disabled_count ?? 0)],
|
||||
['channel status counts', formatJsonInline(delivery.channel_status_counts || {})],
|
||||
['external send attempts', String(delivery.external_attempt_count ?? 0)],
|
||||
['latest delivery', `${delivery.latest_status || 'n/a'} @ ${delivery.latest_channel || 'n/a'}`],
|
||||
['latest attempted', formatTimestamp(delivery.latest_attempted_ts)],
|
||||
|
||||
@@ -312,6 +312,41 @@ class AlertVisualDispatchTests(unittest.TestCase):
|
||||
"suppressed_noise_policy",
|
||||
)
|
||||
|
||||
def test_dispatcher_records_disabled_channel_when_default_adapter_has_no_credentials(self) -> None:
|
||||
decision_repository = StubDecisionRepository()
|
||||
decision_repository.alerts[12] = {
|
||||
"id": 12,
|
||||
"decision_packet_id": 55,
|
||||
"alert_type": "candidate_blocked",
|
||||
"status": "queued",
|
||||
"payload": {"action_recommendation": "wait_for_confirmations", "status": "blocked"},
|
||||
}
|
||||
decision_repository.packets[55] = {"id": 55, "payload": {"symbol": "BTC-USDT"}}
|
||||
delivery_repository = StubDeliveryRepository()
|
||||
dispatcher = AlertDispatcherService(
|
||||
decision_repository=decision_repository,
|
||||
delivery_repository=delivery_repository,
|
||||
visual_watch_service=StubVisualWatchService(),
|
||||
feedback_service=StubFeedbackService(),
|
||||
)
|
||||
|
||||
with patch("src.services.alerts.channel_router_service.settings.alert_default_channel", "email"), patch(
|
||||
"src.services.alerts.channel_router_service.settings.alert_email_smtp_url",
|
||||
"",
|
||||
), patch("src.services.alerts.channel_router_service.settings.alert_email_to", ""), patch.object(
|
||||
AlertDispatcherService,
|
||||
"_send",
|
||||
) as send_mock:
|
||||
result = dispatcher.dispatch_alert(12)
|
||||
|
||||
send_mock.assert_not_called()
|
||||
self.assertEqual(result["status"], "skipped")
|
||||
self.assertEqual(result["reason"], "channel_disabled")
|
||||
self.assertEqual(result["channel"], "email")
|
||||
self.assertEqual(result["channel_route"]["capability"]["reason"], "credential_missing")
|
||||
self.assertEqual(delivery_repository.delivery_calls[0]["error"], "channel_disabled")
|
||||
self.assertEqual(delivery_repository.delivery_calls[0]["payload"]["channel_route"]["selected_channel"], "email")
|
||||
|
||||
def test_dispatch_pending_retries_failed_alert_after_backoff(self) -> None:
|
||||
decision_repository = StubDecisionRepository()
|
||||
decision_repository.alerts[21] = {
|
||||
|
||||
@@ -83,6 +83,10 @@ class AlertQualitySummaryServiceTests(unittest.TestCase):
|
||||
self.assertEqual(summary["delivery_summary"]["media_attached_count"], 2)
|
||||
self.assertEqual(summary["delivery_summary"]["media_linked_count"], 2)
|
||||
self.assertEqual(summary["delivery_summary"]["media_unavailable_count"], 1)
|
||||
self.assertEqual(summary["delivery_summary"]["channel_status_counts"]["webhook"]["delivered"], 1)
|
||||
self.assertEqual(summary["delivery_summary"]["channel_status_counts"]["webhook"]["failed"], 1)
|
||||
self.assertEqual(summary["delivery_summary"]["channel_status_counts"]["workbench"]["skipped"], 2)
|
||||
self.assertEqual(summary["delivery_summary"]["channel_disabled_count"], 0)
|
||||
self.assertTrue(summary["silent_window"]["active"])
|
||||
self.assertEqual(summary["delivery_summary"]["success_rate"], 0.5)
|
||||
self.assertEqual(summary["alert_feedback_summary"]["positive_count"], 1)
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from src.services.alerts.channel_router_service import (
|
||||
build_channel_health_summary,
|
||||
get_channel_capability,
|
||||
select_alert_route,
|
||||
)
|
||||
|
||||
|
||||
class ChannelRouterServiceTests(unittest.TestCase):
|
||||
def test_high_priority_alert_routes_to_webhook_when_configured(self) -> None:
|
||||
alert = {"alert_type": "candidate_actionable", "payload": {"action_recommendation": "confirm_ticket"}}
|
||||
|
||||
with patch("src.services.alerts.channel_router_service.settings.alert_webhook_url", "https://example.test/hook"):
|
||||
route = select_alert_route(alert)
|
||||
|
||||
self.assertEqual(route["selected_channel"], "webhook")
|
||||
self.assertTrue(route["capability"]["enabled"])
|
||||
self.assertEqual(route["route_rule"], "webhook_high_priority_alert")
|
||||
|
||||
def test_missing_wecom_credentials_surface_disabled_adapter(self) -> None:
|
||||
with patch("src.services.alerts.channel_router_service.settings.alert_wecom_webhook_url", ""):
|
||||
capability = get_channel_capability("wecom")
|
||||
|
||||
self.assertFalse(capability["enabled"])
|
||||
self.assertEqual(capability["reason"], "credential_missing")
|
||||
self.assertEqual(capability["adapter"], "wecom_disabled_adapter")
|
||||
|
||||
def test_channel_health_reports_disabled_external_channels(self) -> None:
|
||||
with patch("src.services.alerts.channel_router_service.settings.alert_webhook_url", ""), patch(
|
||||
"src.services.alerts.channel_router_service.settings.alert_wecom_webhook_url",
|
||||
"",
|
||||
), patch("src.services.alerts.channel_router_service.settings.alert_email_smtp_url", ""), patch(
|
||||
"src.services.alerts.channel_router_service.settings.alert_email_to",
|
||||
"",
|
||||
):
|
||||
summary = build_channel_health_summary()
|
||||
|
||||
self.assertIn("workbench", summary["enabled_channels"])
|
||||
self.assertIn("webhook", summary["credential_missing_channels"])
|
||||
self.assertIn("wecom", summary["credential_missing_channels"])
|
||||
self.assertIn("email", summary["credential_missing_channels"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -136,6 +136,9 @@ class FakeAlertDispatcherService:
|
||||
"media_attached": 0,
|
||||
"media_linked": 0,
|
||||
"media_unavailable": 0,
|
||||
"channel_disabled": 0,
|
||||
"channel_summary": {},
|
||||
"channel_health": {"enabled_channels": ["workbench"], "disabled_channels": ["webhook", "wecom", "email"]},
|
||||
}
|
||||
|
||||
|
||||
@@ -243,6 +246,8 @@ class TradingRuntimeSchedulerTests(unittest.TestCase):
|
||||
self.assertFalse(payload["tasks"]["alert_dispatch"]["details"]["silent_window_active"])
|
||||
self.assertEqual(payload["tasks"]["alert_dispatch"]["details"]["suppressed_noise_policy"], 0)
|
||||
self.assertEqual(payload["tasks"]["alert_dispatch"]["details"]["media_unavailable"], 0)
|
||||
self.assertEqual(payload["tasks"]["alert_dispatch"]["details"]["channel_disabled"], 0)
|
||||
self.assertIn("enabled_channels", payload["tasks"]["alert_dispatch"]["details"]["channel_health"])
|
||||
self.assertEqual(payload["runtime_watchdog"]["stalled_task_count"], 0)
|
||||
self.assertEqual(payload["latest_broker_state_by_symbol"]["BTC-USDT"]["account"]["equity"], 1000.0)
|
||||
self.assertTrue(payload["tasks"]["watch"]["next_run_ts"])
|
||||
|
||||
Reference in New Issue
Block a user