yuqei/ai-exchange
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Harden exchange context fallbacks

Browse Source
This commit is contained in:
Codex
2026-05-14 07:30:57 +08:00
parent e8e8a16755
commit d4232501ae
6 changed files with 283 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
SESSION.md
+6 -3
View File
@@ -2,7 +2,7 @@
## Current Goal
Automation execution-layer PXX remains closed and its known scope stays frozen. Under explicit user direction, the active product phase is a dense `Decision Assistant` operator workspace with grouped recall, broader replay baselines, note templates, quick-save shortcuts, read-only market context, and a continuous operator cockpit. This run closed H28 as a real focus-workspace batch: a new `focus_workspace_context` now carries the current focus title/session/model/tier plus cross-section workspace prompts, watchlist cards can pin the active focus locally, and the cockpit exposes a dedicated `Focus Workspace` section that bridges Watchlist, Evidence, Review Entry, Digest, and Replay around the same object. Local validation passed, the current working tree was deployed, and remote smoke confirmed the latest Decision Assistant fields on the live host.
Automation execution-layer PXX remains closed and its known scope stays frozen. Under explicit user direction, the active product phase is a dense `Decision Assistant` operator workspace with grouped recall, broader replay baselines, note templates, quick-save shortcuts, read-only market context, and a continuous operator cockpit. This run closed E8 as a stability-hardening batch: OKX system status and announcements now retry across proxy/direct paths, fall back to cached snapshots when available, degrade to empty fallback instead of hard error when no cache exists, and no longer poison the external-context chart inventory when the upstream endpoint flakes. Local validation passed, the current working tree was deployed, and remote smoke confirmed the latest Decision Assistant fields on the live host.
## Current Branch
@@ -134,6 +134,7 @@ Automation execution-layer PXX remains closed and its known scope stays frozen.
- latest_decision_assistant_workflow_persistence_slice: workflow_history_archive_persistence
- latest_decision_assistant_workflow_archive_slice: persisted_workflow_archive_visibility
- latest_decision_assistant_focus_workspace_slice: focus_workspace_context
- latest_decision_assistant_exchange_hardening_slice: okx_status_announcement_cached_fallbacks
- latest_decision_assistant_summary_status: implemented
- latest_decision_assistant_change_summary_status: implemented
- latest_decision_assistant_evidence_status: implemented
@@ -252,10 +253,11 @@ Automation execution-layer PXX remains closed and its known scope stays frozen.
- latest_decision_assistant_workflow_persistence_status: implemented
- latest_decision_assistant_workflow_archive_status: implemented
- latest_decision_assistant_focus_workspace_status: implemented
- latest_decision_assistant_exchange_hardening_status: implemented
- latest_decision_assistant_workbench_status: visible_local_smoke
- latest_remote_decision_assistant_deploy_status: deployed_current_head
- latest_remote_decision_assistant_smoke_status: passed
- latest_remote_decision_assistant_review_packet_status: focus_workspace_context_visible
- latest_remote_decision_assistant_review_packet_status: exchange_context_fallback_visible
- latest_p7_selector_choice: continue_periodic_analysis_decision_monitoring_with_maturity_rollup
- latest_p14_slice: webhook_transport_retry
- latest_p15_pxx_tasklist_status: ready
@@ -673,11 +675,12 @@ Automation execution-layer PXX remains closed and its known scope stays frozen.
- Latest C3/G20/H26 validation passed: focused operator workspace tests, 68 related unit/API/static tests, 713 full unit tests, 73 script static checks, `node --check src/web/app.js`, deploy smoke passed 11 endpoints, remote ops smoke passed 9 checks, and remote field smoke confirmed `watchlist focus matrix`, `review_entry_action_queue`, `review_entry_handoff_summary`, and watchlist-focus presence inside `assistant_loop_queue`.
- Latest F26/W5/H27 validation passed: focused operator workspace/API/deploy tests, 75 related unit/API/static tests, 715 full unit tests, 73 script static checks, `node --check src/web/app.js`, remote migrations `044_create_decision_assistant_digest_state.sql` and `045_create_decision_assistant_workflow_history.sql` applied successfully, deploy completed, remote ops smoke passed 9 checks, and remote field smoke confirmed `/trading/alerts/digest-preferences`, persisted `digest_preference_profile`, `digest_history_archive`, and `workflow_history_archive`.
- Latest H28 validation passed: focused operator workspace tests, 69 related unit/API/static tests, 715 full unit tests, 73 script static checks, `node --check src/web/app.js`, deploy completed, remote ops smoke passed 9 checks, and remote field smoke confirmed `focus_workspace_context` plus the deployed `Focus Workspace` UI.
- Latest E8 validation passed: focused exchange/provider tests, 71 related unit/API/static tests, 717 full unit tests, 73 script static checks, deploy completed, remote ops smoke passed 9 checks, and remote field smoke confirmed `exchange_status/announcements = idle`, `cache_state = empty_fallback`, and `chart_stale_source_count = 0`.
- Real-live order submission remains intentionally disabled because the OKX order adapter is still not a real-live submitter.
## Exact Next Step
H28 focus workspace batch is closed. External chart URLs remain optional reference enhancements and are not required for the current analysis-only loop. Internal non-trading backlog still exists, but PXX remains closed; do not create P25/P26 automatically, and do not enable real trading.
E8 exchange-context hardening batch is closed. External chart URLs remain optional reference enhancements and are not required for the current analysis-only loop. The remaining unfinished part of item 4 is the real external-chart adapter itself; PXX remains closed and you still must not create P25/P26 automatically or enable real trading.
## Validation Commands Used Recently
TASKS.md
+6 -1
View File
@@ -164,6 +164,7 @@
- [x] W5 workflow artifact archive persistence slice
- [x] H27 persisted digest archive / cockpit sync slice
- [x] H28 focus workspace / cross-section current-focus sync slice
- [x] E8 exchange context cached/empty fallback hardening slice
- [x] P8 local closure complete
- [x] P9 local closure complete
- [x] P10 local closure complete
@@ -291,6 +292,7 @@
- [x] latest_decision_assistant_workflow_persistence_slice: workflow_history_archive_persistence
- [x] latest_decision_assistant_workflow_archive_slice: persisted_workflow_archive_visibility
- [x] latest_decision_assistant_focus_workspace_slice: focus_workspace_context
- [x] latest_decision_assistant_exchange_hardening_slice: okx_status_announcement_cached_fallbacks
- [x] latest_decision_assistant_summary_status: implemented
- [x] latest_decision_assistant_change_summary_status: implemented
- [x] latest_decision_assistant_evidence_status: implemented
@@ -406,10 +408,11 @@
- [x] latest_decision_assistant_workflow_persistence_status: implemented
- [x] latest_decision_assistant_workflow_archive_status: implemented
- [x] latest_decision_assistant_focus_workspace_status: implemented
- [x] latest_decision_assistant_exchange_hardening_status: implemented
- [x] latest_decision_assistant_workbench_status: visible_local_smoke
- [x] latest_remote_decision_assistant_deploy_status: deployed_current_head
- [x] latest_remote_decision_assistant_smoke_status: passed
- [x] latest_remote_decision_assistant_review_packet_status: focus_workspace_context_visible
- [x] latest_remote_decision_assistant_review_packet_status: exchange_context_fallback_visible
- [x] latest_p7_selector_choice: continue_periodic_analysis_decision_monitoring_with_maturity_rollup
- [x] latest_p14_slice: webhook_transport_retry
- [x] latest_p15_pxx_tasklist_status: ready
@@ -725,6 +728,8 @@
- [x] Deployed F26/W5/H27 current tree, applied remote migrations `044_create_decision_assistant_digest_state.sql` and `045_create_decision_assistant_workflow_history.sql`, and passed remote smoke for `/trading/alerts/digest-preferences`, persisted `digest_preference_profile`, `digest_history_archive`, and `workflow_history_archive`
- [x] Completed H28 focused validation: operator workspace tests, 69 related unit/API/static tests, 715 full unit tests, 73 script static checks, and dashboard JS syntax
- [x] Deployed H28 current tree and passed remote smoke: 9 remote ops checks, `focus_workspace_context` field smoke, and deployed `Focus Workspace` UI visibility
- [x] Completed E8 focused validation: exchange/provider tests, 71 related unit/API/static tests, 717 full unit tests, 73 script static checks, and dashboard JS syntax
- [x] Deployed E8 current tree and passed remote smoke: 9 remote ops checks, `exchange_status/announcements` fallback field smoke, and `chart_stale_source_count = 0`
## Blocked
src/repositories/market_state_repository.py
+14
View File
@@ -339,3 +339,17 @@ class MarketStateRepository:
with SessionLocal() as session:
row = session.execute(sql).mappings().first()
return dict(row) if row else None
def fetch_news_provider_snapshot(self, provider_name: str) -> Optional[dict]:
sql = text(
"""
select id, provider_name, fetched_ts, freshness_code, event_count, is_active,
code, active_events, error, raw_payload, updated_at
from news_provider_snapshots
where provider_name = :provider_name
limit 1
"""
)
with SessionLocal() as session:
row = session.execute(sql, {"provider_name": provider_name}).mappings().first()
return dict(row) if row else None
src/services/market_state/exchange_status_provider.py
+184 -47
View File
@@ -7,6 +7,7 @@ from typing import Optional
import httpx
from src.config.settings import settings
from src.repositories.market_state_repository import MarketStateRepository
DEFAULT_OKX_SYSTEM_STATUS_URL = "https://www.okx.com/api/v5/system/status"
@@ -22,10 +23,14 @@ class OkxSystemStatusProvider:
url: Optional[str] = None,
timeout: Optional[int] = None,
proxy_url: Optional[str] = None,
repository: Optional[MarketStateRepository] = None,
cache_max_age_seconds: Optional[int] = None,
) -> None:
self.url = url or DEFAULT_OKX_SYSTEM_STATUS_URL
self.timeout = timeout or settings.news_provider_timeout
self.proxy_url = proxy_url or settings.market_truth_proxy_url or getenv("HTTPS_PROXY") or getenv("HTTP_PROXY") or ""
self.repository = repository or MarketStateRepository()
self.cache_max_age_seconds = cache_max_age_seconds or settings.news_provider_cache_max_age_seconds
def fetch_snapshot(self, *, now: Optional[datetime] = None, payload: Optional[dict] = None) -> dict:
observed_now = now or datetime.now(timezone.utc)
@@ -33,38 +38,88 @@ class OkxSystemStatusProvider:
data = dict(payload or self._fetch_payload())
events = [dict(item or {}) for item in list(data.get("data") or []) if isinstance(item, dict)]
active_events = [_normalize_status_event(item) for item in events if _status_event_is_active(item)]
return {
snapshot = {
"provider_name": self.provider_name,
"fetched_ts": observed_now.isoformat(),
"status": "active" if active_events else "idle",
"freshness_code": "provider_fresh",
"cache_state": "fresh",
"event_count": len(events),
"active_events": active_events[:6],
"error": None,
"warning": None,
"raw_payload": data,
}
self._persist_snapshot(snapshot)
return snapshot
except Exception as exc:
return {
"provider_name": self.provider_name,
"fetched_ts": observed_now.isoformat(),
"status": "error",
"event_count": 0,
"active_events": [],
"error": str(exc),
"raw_payload": {},
}
cached = self._cached_snapshot(observed_now=observed_now, warning=str(exc))
if cached is not None:
return cached
return self._empty_fallback_snapshot(observed_now=observed_now, warning=str(exc))
def _fetch_payload(self) -> dict:
kwargs = {"timeout": self.timeout, "follow_redirects": True}
if self.proxy_url:
kwargs["proxy"] = self.proxy_url
kwargs["trust_env"] = False
with httpx.Client(**kwargs) as client:
response = client.get(self.url, headers={"User-Agent": "ai-ict/exchange-status"})
response.raise_for_status()
payload = response.json()
if not isinstance(payload, dict):
raise ValueError("unsupported_okx_system_status_payload")
return payload
return _fetch_okx_json(
url=self.url,
timeout=self.timeout,
proxy_url=self.proxy_url,
headers={"User-Agent": "ai-ict/exchange-status"},
payload_error="unsupported_okx_system_status_payload",
)
def _persist_snapshot(self, snapshot: dict) -> None:
try:
self.repository.upsert_news_provider_snapshot(
provider_name=self.provider_name,
fetched_ts=snapshot.get("fetched_ts"),
freshness_code=snapshot.get("freshness_code") or "provider_fresh",
event_count=int(snapshot.get("event_count") or 0),
is_active=bool(snapshot.get("status") == "active"),
code=str(snapshot.get("status") or "idle"),
active_events=list(snapshot.get("active_events") or []),
error=snapshot.get("error"),
raw_payload=dict(snapshot.get("raw_payload") or {}),
)
except Exception:
return
def _cached_snapshot(self, *, observed_now: datetime, warning: str) -> dict | None:
try:
row = self.repository.fetch_news_provider_snapshot(self.provider_name)
except Exception:
return None
if not row or not row.get("fetched_ts"):
return None
fetched_ts = _ensure_aware(row.get("fetched_ts"))
age_seconds = max((observed_now - fetched_ts).total_seconds(), 0.0)
if age_seconds > max(int(self.cache_max_age_seconds or 0), 1):
return None
return {
"provider_name": self.provider_name,
"fetched_ts": fetched_ts.isoformat(),
"status": str(row.get("code") or "idle"),
"freshness_code": "provider_cached",
"cache_state": "cached_fallback",
"event_count": int(row.get("event_count") or 0),
"active_events": list(row.get("active_events") or []),
"error": None,
"warning": warning,
"raw_payload": dict(row.get("raw_payload") or {}),
}
def _empty_fallback_snapshot(self, *, observed_now: datetime, warning: str) -> dict:
return {
"provider_name": self.provider_name,
"fetched_ts": observed_now.isoformat(),
"status": "idle",
"freshness_code": "provider_fallback_empty",
"cache_state": "empty_fallback",
"event_count": 0,
"active_events": [],
"error": None,
"warning": warning,
"raw_payload": {},
}
def _status_event_is_active(row: dict) -> bool:
@@ -93,11 +148,15 @@ class OkxAnnouncementsProvider:
timeout: Optional[int] = None,
proxy_url: Optional[str] = None,
language: str = "en-US",
repository: Optional[MarketStateRepository] = None,
cache_max_age_seconds: Optional[int] = None,
) -> None:
self.url = url or DEFAULT_OKX_ANNOUNCEMENTS_URL
self.timeout = timeout or settings.news_provider_timeout
self.proxy_url = proxy_url or settings.market_truth_proxy_url or getenv("HTTPS_PROXY") or getenv("HTTP_PROXY") or ""
self.language = language
self.repository = repository or MarketStateRepository()
self.cache_max_age_seconds = cache_max_age_seconds or settings.news_provider_cache_max_age_seconds
def fetch_snapshot(self, *, payload: Optional[dict] = None, now: Optional[datetime] = None) -> dict:
observed_now = now or datetime.now(timezone.utc)
@@ -107,44 +166,91 @@ class OkxAnnouncementsProvider:
for row in list(data.get("data") or []):
if isinstance(row, dict):
details.extend([dict(item or {}) for item in list(row.get("details") or []) if isinstance(item, dict)])
return {
snapshot = {
"provider_name": self.provider_name,
"fetched_ts": observed_now.isoformat(),
"status": "active" if details else "idle",
"freshness_code": "provider_fresh",
"cache_state": "fresh",
"announcement_count": len(details),
"announcements": [_normalize_announcement_item(item) for item in details[:6]],
"error": None,
"warning": None,
"raw_payload": data,
}
self._persist_snapshot(snapshot)
return snapshot
except Exception as exc:
return {
"provider_name": self.provider_name,
"fetched_ts": observed_now.isoformat(),
"status": "error",
"announcement_count": 0,
"announcements": [],
"error": str(exc),
"raw_payload": {},
}
cached = self._cached_snapshot(observed_now=observed_now, warning=str(exc))
if cached is not None:
return cached
return self._empty_fallback_snapshot(observed_now=observed_now, warning=str(exc))
def _fetch_payload(self) -> dict:
kwargs = {"timeout": self.timeout, "follow_redirects": True}
if self.proxy_url:
kwargs["proxy"] = self.proxy_url
kwargs["trust_env"] = False
with httpx.Client(**kwargs) as client:
response = client.get(
self.url,
headers={
"User-Agent": "ai-ict/exchange-announcements",
"Accept-Language": self.language,
},
return _fetch_okx_json(
url=self.url,
timeout=self.timeout,
proxy_url=self.proxy_url,
headers={
"User-Agent": "ai-ict/exchange-announcements",
"Accept-Language": self.language,
},
payload_error="unsupported_okx_announcements_payload",
)
def _persist_snapshot(self, snapshot: dict) -> None:
try:
self.repository.upsert_news_provider_snapshot(
provider_name=self.provider_name,
fetched_ts=snapshot.get("fetched_ts"),
freshness_code=snapshot.get("freshness_code") or "provider_fresh",
event_count=int(snapshot.get("announcement_count") or 0),
is_active=bool(snapshot.get("status") == "active"),
code=str(snapshot.get("status") or "idle"),
active_events=list(snapshot.get("announcements") or []),
error=snapshot.get("error"),
raw_payload=dict(snapshot.get("raw_payload") or {}),
)
response.raise_for_status()
payload = response.json()
if not isinstance(payload, dict):
raise ValueError("unsupported_okx_announcements_payload")
return payload
except Exception:
return
def _cached_snapshot(self, *, observed_now: datetime, warning: str) -> dict | None:
try:
row = self.repository.fetch_news_provider_snapshot(self.provider_name)
except Exception:
return None
if not row or not row.get("fetched_ts"):
return None
fetched_ts = _ensure_aware(row.get("fetched_ts"))
age_seconds = max((observed_now - fetched_ts).total_seconds(), 0.0)
if age_seconds > max(int(self.cache_max_age_seconds or 0), 1):
return None
return {
"provider_name": self.provider_name,
"fetched_ts": fetched_ts.isoformat(),
"status": str(row.get("code") or "idle"),
"freshness_code": "provider_cached",
"cache_state": "cached_fallback",
"announcement_count": int(row.get("event_count") or 0),
"announcements": list(row.get("active_events") or []),
"error": None,
"warning": warning,
"raw_payload": dict(row.get("raw_payload") or {}),
}
def _empty_fallback_snapshot(self, *, observed_now: datetime, warning: str) -> dict:
return {
"provider_name": self.provider_name,
"fetched_ts": observed_now.isoformat(),
"status": "idle",
"freshness_code": "provider_fallback_empty",
"cache_state": "empty_fallback",
"announcement_count": 0,
"announcements": [],
"error": None,
"warning": warning,
"raw_payload": {},
}
def _normalize_announcement_item(row: dict) -> dict:
@@ -155,3 +261,34 @@ def _normalize_announcement_item(row: dict) -> dict:
"business_published_ts": str(row.get("businessPTime") or ""),
"url": str(row.get("url") or ""),
}
def _fetch_okx_json(*, url: str, timeout: int, proxy_url: str, headers: dict, payload_error: str) -> dict:
attempt_timeout = max(1, min(int(timeout or 1), 4))
attempts = []
if proxy_url:
attempts.append({"timeout": attempt_timeout, "follow_redirects": True, "proxy": proxy_url, "trust_env": False})
attempts.append({"timeout": attempt_timeout, "follow_redirects": True, "trust_env": False})
errors = []
for kwargs in attempts:
try:
with httpx.Client(**kwargs) as client:
response = client.get(url, headers=headers)
response.raise_for_status()
payload = response.json()
if not isinstance(payload, dict):
raise ValueError(payload_error)
return payload
except Exception as exc:
errors.append(str(exc))
raise RuntimeError(" | ".join(errors))
def _ensure_aware(value) -> datetime:
if isinstance(value, datetime):
parsed = value
else:
parsed = datetime.fromisoformat(str(value).replace("Z", "+00:00"))
if parsed.tzinfo is None:
return parsed.replace(tzinfo=timezone.utc)
return parsed.astimezone(timezone.utc)
src/services/operator/operator_workspace_service.py
+2 -2
View File
@@ -3327,7 +3327,7 @@ def _build_external_context_chart_readonly_sources(
"source_id": "exchange_status_context",
"source_type": "exchange_status_context",
"status": exchange_status.get("status") or "unknown",
"freshness_status": "source_current" if exchange_status.get("status") == "active" else "source_degraded",
"freshness_status": "source_degraded" if exchange_status.get("error") else "source_current",
"stale_source_evidence": exchange_status.get("error") if exchange_status.get("status") == "error" else None,
"prompt": "Use exchange status only as context, not execution permission.",
},
@@ -3335,7 +3335,7 @@ def _build_external_context_chart_readonly_sources(
"source_id": "announcement_context",
"source_type": "announcement_context",
"status": announcements_snapshot.get("status") or "unknown",
"freshness_status": "source_current" if announcements_snapshot.get("status") == "active" else "source_degraded",
"freshness_status": "source_degraded" if announcements_snapshot.get("error") else "source_current",
"stale_source_evidence": announcements_snapshot.get("error") if announcements_snapshot.get("status") == "error" else None,
"prompt": "Use announcements as read-only chart context when evaluating event windows.",
},
tests/unit/test_exchange_status_provider.py
+71
View File
@@ -0,0 +1,71 @@
import os
import unittest
from datetime import datetime, timezone
os.environ.setdefault("DATABASE_URL", "postgresql+psycopg://user:pass@localhost:5432/ai_ict_test")
from src.services.market_state.exchange_status_provider import OkxAnnouncementsProvider, OkxSystemStatusProvider
class ExchangeStatusProviderTests(unittest.TestCase):
def test_system_status_provider_uses_cached_snapshot_when_live_fetch_fails(self) -> None:
repository = _SnapshotRepo(
{
"provider_name": "okx_system_status",
"fetched_ts": "2026-05-14T02:00:00+00:00",
"event_count": 1,
"code": "active",
"active_events": [{"title": "API maintenance"}],
"raw_payload": {"data": [{"title": "API maintenance"}]},
}
)
provider = OkxSystemStatusProvider(repository=repository, cache_max_age_seconds=21600)
provider._fetch_payload = lambda: (_ for _ in ()).throw(RuntimeError("ssl eof")) # type: ignore[attr-defined]
snapshot = provider.fetch_snapshot(now=datetime(2026, 5, 14, 2, 30, tzinfo=timezone.utc))
self.assertEqual(snapshot["status"], "active")
self.assertEqual(snapshot["freshness_code"], "provider_cached")
self.assertEqual(snapshot["cache_state"], "cached_fallback")
self.assertIsNone(snapshot["error"])
self.assertIn("ssl eof", snapshot["warning"])
def test_announcements_provider_uses_cached_snapshot_when_live_fetch_fails(self) -> None:
repository = _SnapshotRepo(
{
"provider_name": "okx_announcements",
"fetched_ts": "2026-05-14T02:00:00+00:00",
"event_count": 1,
"code": "active",
"active_events": [{"title": "Listing schedule update"}],
"raw_payload": {"data": [{"details": [{"title": "Listing schedule update"}]}]},
}
)
provider = OkxAnnouncementsProvider(repository=repository, cache_max_age_seconds=21600)
provider._fetch_payload = lambda: (_ for _ in ()).throw(RuntimeError("ssl eof")) # type: ignore[attr-defined]
snapshot = provider.fetch_snapshot(now=datetime(2026, 5, 14, 2, 30, tzinfo=timezone.utc))
self.assertEqual(snapshot["status"], "active")
self.assertEqual(snapshot["freshness_code"], "provider_cached")
self.assertEqual(snapshot["cache_state"], "cached_fallback")
self.assertIsNone(snapshot["error"])
self.assertIn("ssl eof", snapshot["warning"])
class _SnapshotRepo:
def __init__(self, row: dict) -> None:
self.row = row
def fetch_news_provider_snapshot(self, provider_name: str):
if provider_name == self.row.get("provider_name"):
return dict(self.row)
return None
def upsert_news_provider_snapshot(self, **kwargs):
self.row = dict(kwargs)
return 1
if __name__ == "__main__":
unittest.main()