yuqei/ai-contract-platform
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat: schedule low-noise production watchdog

Browse Source
This commit is contained in:
yuqei
2026-05-17 18:37:43 +08:00
parent a4986096de
commit c7f5718932
5 changed files with 136 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
project-docs/AI合同起草与审查助手-当前工作进度保存-2026-05-11.md
+56
View File
@@ -2063,6 +2063,62 @@
- 本轮无 required 阻塞项。
- 当前剩余均为 non-required 运维增强项：真实外部 Webhook 地址、可选容量清理、后续法条批次扩展。
## 2026-05-17 低负载生产值守调度与 Webhook 通知入口
### 本轮完成项
- 已把生产值守调度改为适合资源紧张服务器的低负载模式：
- 计划任务：`AIContractProductionWatchdog`
- 执行时间：每天 `02:00`
- 通知策略：`WarnOrFail`，正常 PASS 不通知，只有 WARN/FAIL 才通知。
- 日常巡检只做轻量生产 ops check；容量治理 dry-run 不放进每日任务，保留给维护窗口单独执行。
- 日常任务跳过本机开发工作区 clean 检查，避免开发机未提交前端改动造成假告警。
- 已增强 `scripts/run-production-watchdog-scheduler.ps1`：
- 默认时间从 `07:30` 调整为 `02:00`。
- 新增 `WebhookKind=auto`，可按 URL 自动识别企业微信、钉钉或通用 Webhook。
- 新增钉钉 markdown payload。
- 注册任务不再硬编码 `-RequireCleanWorktree` 和 `-FailOnWarn`。
- 新增 `-SkipWorktreeCheck`、`-SkipCapacityGovernance` 透传能力。
- 已增强相关子脚本：
- `scripts/run-production-watchdog.ps1`
- `scripts/run-production-ops-check.ps1`
- `scripts/run-capacity-governance.ps1`
- 支持日常巡检场景跳过本机工作区噪声。
### 本轮验证
- 企业微信 payload dry-run：
- `PHASE7_WATCHDOG_SCHEDULER_PASS`
- 产物：`project-docs/qa-artifacts/phase7.5-watchdog-scheduler/2026-05-17T18-34-32-390/watchdog-notification-payload.json`
- 钉钉 payload dry-run：
- `PHASE7_WATCHDOG_SCHEDULER_PASS`
- 产物：`project-docs/qa-artifacts/phase7.5-watchdog-scheduler/2026-05-17T18-34-32-389/watchdog-notification-payload.json`
- 计划任务 dry-run：
- `PHASE7_WATCHDOG_SCHEDULER_PASS`
- 产物：`project-docs/qa-artifacts/phase7.5-watchdog-scheduler/2026-05-17T18-34-32-406/watchdog-scheduler-summary.md`
- 真实注册计划任务：
- `PHASE7_WATCHDOG_SCHEDULER_PASS`
- 产物：`project-docs/qa-artifacts/phase7.5-watchdog-scheduler/2026-05-17T18-35-03-112/watchdog-scheduler-summary.md`
- Windows 任务下一次执行时间：`2026-05-18 02:00:00`
- 同配置手动轻量巡检：
- `PHASE7_WATCHDOG_SCHEDULER_PASS`
- 产物：`project-docs/qa-artifacts/phase7.5-watchdog-scheduler/2026-05-17T18-35-27-004/watchdog-scheduler-summary.md`
- 子巡检：production ops check PASS，capacity governance SKIP，working tree SKIP。
### 当前状态
- 每日凌晨 2 点低负载生产巡检已经启用。
- 计划任务命令只引用 `AI_CONTRACT_WATCHDOG_WEBHOOK_URL`，不把真实 Webhook URL 写进 Git、脚本或任务命令。
- 当前没有真实 Webhook URL 时，巡检仍会正常产出本地证据；配置 URL 后，企业微信或钉钉会在 WARN/FAIL 时收到通知。
### 仍需配合项
- 需要提供一个真实 Webhook URL：
- 企业微信机器人 URL，或
- 钉钉机器人 URL，或
- 其他能接收 JSON POST 的通用 Webhook。
- URL 应只配置到运行环境变量 `AI_CONTRACT_WATCHDOG_WEBHOOK_URL`，不能写入仓库文档或代码。
- 如果使用钉钉且机器人开启“关键词”安全策略，关键词建议包含 `AI Contract Platform`，否则钉钉会拒收消息。
### 阻塞项
- 本轮无 required 阻塞项。
- 真实外部告警到人仍等待 Webhook URL；在 URL 提供前，系统已完成本地证据闭环和 02:00 低负载巡检调度。
## 2026-05-16 Phase14 生产巡检与 Chat/报告法条引用闭环
### 本轮完成项
- 已完成 Chat/报告法条引用闭环代码补齐并推送远程 `main`：
scripts/run-capacity-governance.ps1
+6 -2
View File
@@ -17,7 +17,8 @@ param(
[switch]$Apply,
[switch]$SkipBackupCleanup,
[switch]$SkipDockerPrune,
[switch]$AllowDirtyWorktree
[switch]$AllowDirtyWorktree,
[switch]$SkipWorktreeCheck
)
$ErrorActionPreference = "Stop"
@@ -132,7 +133,9 @@ $commit = (git -C $repoRoot rev-parse HEAD).Trim()
$remoteUrl = (git -C $repoRoot remote get-url origin).Trim()
$gitStatus = git -C $repoRoot status --short
($gitStatus -join [Environment]::NewLine) | Set-Content -Encoding UTF8 -Path $gitStatusPath
if ($gitStatus) {
if ($SkipWorktreeCheck) {
Add-Check -Key "working_tree" -Label "working tree" -Status "skip" -Required $false -Detail "Skipped for low-noise scheduled runtime monitoring" -Meta @{ artifact = $gitStatusPath }
} elseif ($gitStatus) {
if ($AllowDirtyWorktree) {
Add-Check -Key "working_tree" -Label "working tree" -Status "warn" -Required $false -Detail "Working tree has uncommitted changes; capacity governance allowed by -AllowDirtyWorktree" -Meta @{ artifact = $gitStatusPath }
} else {
@@ -386,6 +389,7 @@ $summary = [ordered]@{
skip_backup_cleanup = [bool]$SkipBackupCleanup
skip_docker_prune = [bool]$SkipDockerPrune
allow_dirty_worktree = [bool]$AllowDirtyWorktree
skip_worktree_check = [bool]$SkipWorktreeCheck
}
checks = $checks
totals = [ordered]@{
scripts/run-production-ops-check.ps1
+6 -2
View File
@@ -15,7 +15,8 @@ param(
[int]$DiskFailPercent = 90,
[int]$MinFrontendBackupCount = 1,
[int]$MaxNewestBackupAgeHours = 168,
[switch]$RequireCleanWorktree
[switch]$RequireCleanWorktree,
[switch]$SkipWorktreeCheck
)
$ErrorActionPreference = "Stop"
@@ -177,7 +178,9 @@ $commit = (git -C $repoRoot rev-parse HEAD).Trim()
$remoteUrl = (git -C $repoRoot remote get-url origin).Trim()
$gitStatus = git -C $repoRoot status --short
($gitStatus -join [Environment]::NewLine) | Set-Content -Encoding UTF8 -Path $gitStatusPath
if ($gitStatus) {
if ($SkipWorktreeCheck) {
Add-Check -Key "working_tree" -Label "working tree" -Status "skip" -Required $false -Detail "Skipped for low-noise scheduled runtime monitoring" -Meta @{ artifact = $gitStatusPath }
} elseif ($gitStatus) {
if ($RequireCleanWorktree) {
Add-Check -Key "working_tree" -Label "working tree" -Status "fail" -Detail "Working tree has uncommitted changes and -RequireCleanWorktree is set" -Meta @{ artifact = $gitStatusPath }
} else {
@@ -388,6 +391,7 @@ $summary = [ordered]@{
backend_base_url = $BackendBaseUrl
remote_backup_root = $RemoteBackupRoot
remote_rollback_root = $RemoteRollbackRoot
skip_worktree_check = [bool]$SkipWorktreeCheck
}
thresholds = [ordered]@{
disk_warn_percent = $DiskWarnPercent
scripts/run-production-watchdog-scheduler.ps1
+57 -9
View File
@@ -5,14 +5,14 @@ param(
[string]$TaskDescription = "AI Contract Platform production watchdog",
[ValidateSet("Daily", "Hourly")]
[string]$Schedule = "Daily",
[string]$At = "07:30",
[string]$At = "02:00",
[int]$IntervalMinutes = 60,
[string]$ArtifactRoot = "",
[string]$WatchdogArtifactRoot = "",
[string]$WebhookUrl = "",
[string]$WebhookUrlEnvVar = "AI_CONTRACT_WATCHDOG_WEBHOOK_URL",
[ValidateSet("generic", "wecom")]
[string]$WebhookKind = "generic",
[ValidateSet("auto", "generic", "wecom", "dingtalk")]
[string]$WebhookKind = "auto",
[ValidateSet("All", "WarnOrFail", "FailOnly", "Never")]
[string]$NotifyOnStatus = "WarnOrFail",
[int]$WebhookTimeoutSeconds = 10,
@@ -21,6 +21,8 @@ param(
[switch]$SkipWebhook,
[switch]$RequireCleanWorktree,
[switch]$AllowDirtyWorktree,
[switch]$SkipWorktreeCheck,
[switch]$SkipCapacityGovernance,
[switch]$IncludeReleaseGate,
[switch]$IncludeBackupDrill,
[switch]$FailOnWarn,
@@ -102,6 +104,20 @@ function Test-ShouldNotify {
}
}
function Get-EffectiveWebhookKind {
$effectiveWebhookUrl = Get-EffectiveWebhookUrl
if ($WebhookKind -ne "auto") {
return $WebhookKind
}
if ($effectiveWebhookUrl -match "qyapi\.weixin\.qq\.com") {
return "wecom"
}
if ($effectiveWebhookUrl -match "oapi\.dingtalk\.com|dingtalk\.com") {
return "dingtalk"
}
return "generic"
}
function Get-WebhookPayload {
param(
[string]$Status,
@@ -111,7 +127,8 @@ function Get-WebhookPayload {
)
$text = "AI Contract Platform production watchdog $Status ($Severity). Artifact: $WatchdogArtifactDir"
if ($WebhookKind -eq "wecom") {
$effectiveWebhookKind = Get-EffectiveWebhookKind
if ($effectiveWebhookKind -eq "wecom") {
return [ordered]@{
msgtype = "markdown"
markdown = [ordered]@{
@@ -119,6 +136,15 @@ function Get-WebhookPayload {
}
}
}
if ($effectiveWebhookKind -eq "dingtalk") {
return [ordered]@{
msgtype = "markdown"
markdown = [ordered]@{
title = "AI Contract Platform Watchdog"
text = "AI Contract Platform Watchdog: $text"
}
}
}
return [ordered]@{
project = "ai-contract-platform"
@@ -167,7 +193,7 @@ function Add-WebhookResult {
Add-Check -Key "webhook" -Label "webhook notification" -Status "pass" -Detail "Webhook payload generated in dry-run mode" -Meta @{
payload = $notificationPayloadJson
url_source = if ([string]::IsNullOrWhiteSpace($WebhookUrl)) { $WebhookUrlEnvVar } else { "parameter" }
webhook_kind = $WebhookKind
webhook_kind = Get-EffectiveWebhookKind
dry_run = $true
}
return
@@ -179,14 +205,14 @@ function Add-WebhookResult {
Add-Check -Key "webhook" -Label "webhook notification" -Status "pass" -Detail "Webhook notification delivered" -Meta @{
payload = $notificationPayloadJson
url_source = if ([string]::IsNullOrWhiteSpace($WebhookUrl)) { $WebhookUrlEnvVar } else { "parameter" }
webhook_kind = $WebhookKind
webhook_kind = Get-EffectiveWebhookKind
dry_run = $false
}
} catch {
Add-Check -Key "webhook" -Label "webhook notification" -Status "fail" -Detail "Webhook delivery failed: $($_.Exception.Message)" -Meta @{
payload = $notificationPayloadJson
url_source = if ([string]::IsNullOrWhiteSpace($WebhookUrl)) { $WebhookUrlEnvVar } else { "parameter" }
webhook_kind = $WebhookKind
webhook_kind = Get-EffectiveWebhookKind
dry_run = $false
}
}
@@ -204,8 +230,21 @@ function Get-TaskRunArgumentLine {
$args.Add("RunOnce") | Out-Null
$args.Add("-ArtifactRoot") | Out-Null
$args.Add(('"{0}"' -f $ArtifactRoot)) | Out-Null
$args.Add("-RequireCleanWorktree") | Out-Null
$args.Add("-FailOnWarn") | Out-Null
if ($RequireCleanWorktree) {
$args.Add("-RequireCleanWorktree") | Out-Null
}
if ($AllowDirtyWorktree) {
$args.Add("-AllowDirtyWorktree") | Out-Null
}
if ($SkipWorktreeCheck) {
$args.Add("-SkipWorktreeCheck") | Out-Null
}
if ($SkipCapacityGovernance) {
$args.Add("-SkipCapacityGovernance") | Out-Null
}
if ($FailOnWarn) {
$args.Add("-FailOnWarn") | Out-Null
}
if ($SkipWebhook) {
$args.Add("-SkipWebhook") | Out-Null
}
@@ -240,6 +279,12 @@ function Invoke-RunOnce {
if ($AllowDirtyWorktree) {
$watchdogArgs += "-AllowDirtyWorktree"
}
if ($SkipWorktreeCheck) {
$watchdogArgs += "-SkipWorktreeCheck"
}
if ($SkipCapacityGovernance) {
$watchdogArgs += "-SkipCapacityGovernance"
}
if ($IncludeReleaseGate) {
$watchdogArgs += "-IncludeReleaseGate"
}
@@ -438,12 +483,15 @@ $summary = [ordered]@{
webhook_url_env_var = $WebhookUrlEnvVar
webhook_url_configured = -not [string]::IsNullOrWhiteSpace((Get-EffectiveWebhookUrl))
webhook_kind = $WebhookKind
effective_webhook_kind = Get-EffectiveWebhookKind
notify_on_status = $NotifyOnStatus
dry_run_webhook = [bool]$DryRunWebhook
require_webhook = [bool]$RequireWebhook
skip_webhook = [bool]$SkipWebhook
require_clean_worktree = [bool]$RequireCleanWorktree
allow_dirty_worktree = [bool]$AllowDirtyWorktree
skip_worktree_check = [bool]$SkipWorktreeCheck
skip_capacity_governance = [bool]$SkipCapacityGovernance
include_release_gate = [bool]$IncludeReleaseGate
include_backup_drill = [bool]$IncludeBackupDrill
fail_on_warn = [bool]$FailOnWarn
scripts/run-production-watchdog.ps1
+11 -1
View File
@@ -16,6 +16,7 @@ param(
[int]$DiskFailPercent = 90,
[switch]$RequireCleanWorktree,
[switch]$AllowDirtyWorktree,
[switch]$SkipWorktreeCheck,
[switch]$IncludeReleaseGate,
[switch]$IncludeBackupDrill,
[switch]$SkipOpsCheck,
@@ -133,7 +134,9 @@ $commit = (git -C $repoRoot rev-parse HEAD).Trim()
$remoteUrl = (git -C $repoRoot remote get-url origin).Trim()
$gitStatus = git -C $repoRoot status --short
($gitStatus -join [Environment]::NewLine) | Set-Content -Encoding UTF8 -Path $gitStatusPath
if ($gitStatus) {
if ($SkipWorktreeCheck) {
Add-Check -Key "working_tree" -Label "working tree" -Status "skip" -Required $false -Detail "Skipped for low-noise scheduled runtime monitoring" -Meta @{ artifact = $gitStatusPath }
} elseif ($gitStatus) {
if ($RequireCleanWorktree -and -not $AllowDirtyWorktree) {
Add-Check -Key "working_tree" -Label "working tree" -Status "fail" -Detail "Working tree has uncommitted changes and -RequireCleanWorktree is set" -Meta @{ artifact = $gitStatusPath }
} else {
@@ -181,6 +184,9 @@ if ($failedPreflight.Count -eq 0) {
if ($RequireCleanWorktree -and -not $AllowDirtyWorktree) {
$opsArgs += "-RequireCleanWorktree"
}
if ($SkipWorktreeCheck) {
$opsArgs += "-SkipWorktreeCheck"
}
Invoke-WatchStep -Key "ops_check" -Label "production ops check" -ScriptName "run-production-ops-check.ps1" -Arguments $opsArgs
}
@@ -201,6 +207,9 @@ if ($failedPreflight.Count -eq 0) {
if ($AllowDirtyWorktree) {
$capacityArgs += "-AllowDirtyWorktree"
}
if ($SkipWorktreeCheck) {
$capacityArgs += "-SkipWorktreeCheck"
}
Invoke-WatchStep -Key "capacity_governance" -Label "capacity governance dry run" -ScriptName "run-capacity-governance.ps1" -Arguments $capacityArgs
}
@@ -262,6 +271,7 @@ $summary = [ordered]@{
include_backup_drill = [bool]$IncludeBackupDrill
skip_ops_check = [bool]$SkipOpsCheck
skip_capacity_governance = [bool]$SkipCapacityGovernance
skip_worktree_check = [bool]$SkipWorktreeCheck
fail_on_warn = [bool]$FailOnWarn
}
thresholds = [ordered]@{