fix: align release benchmark with abuse limits
Build and Test Quality Gate / Contract Web typecheck and build (pull_request) Successful in 2m58s
Build and Test Quality Gate / Python tests and Contract API migration (pull_request) Failing after 15m25s
Security Quality Gate / npm audit, Bandit, pip-audit, detect-secrets (pull_request) Successful in 5m14s
Build and Test Quality Gate / Contract Web typecheck and build (pull_request) Successful in 2m58s
Build and Test Quality Gate / Python tests and Contract API migration (pull_request) Failing after 15m25s
Security Quality Gate / npm audit, Bandit, pip-audit, detect-secrets (pull_request) Successful in 5m14s
This commit is contained in:
@@ -16,7 +16,8 @@
|
||||
4. Nginx 为同步 `/contract/api/v1/legal/qa` 设置独立 660 秒上限,覆盖 Contract 允许的 600 秒最大值;其他 Contract 路由仍保持 120 秒。
|
||||
5. Compose 的 migrate、Contract API 和 review worker 三个服务均映射 timeout;共享预发布部署脚本显式写入 Contract 240 秒和 Provider 180 秒,避免重部署恢复默认 30 秒。
|
||||
6. 生产 Web 使用 `/contract`、`/ai` 边缘前缀时,Contract 与 AI 请求均保留 base path。统一出口还会规范化 33 个旧 AI 代理路由生成的无前缀 URL;角色判断和 Contract 公共健康检查使用移除边缘前缀后的逻辑 API 路径,避免修复路由时引入权限回归。
|
||||
7. 身份执行器、support access API 和破坏性身份写接口继续关闭。共享预发布 OpenAPI 中包含 `execute`、`support-access` 或 `support_access` 的路由数为 0。
|
||||
7. release gate 的实时 benchmark 默认调整为每场景 15 次、并发 3、预热 2;包含探测请求后仍低于 enforce 模式最严格的每用户 20 次/分钟读取门槛,不关闭或绕过反滥用。独立性能压测仍保留显式大负载能力。
|
||||
8. 身份执行器、support access API 和破坏性身份写接口继续关闭。共享预发布 OpenAPI 中包含 `execute`、`support-access` 或 `support_access` 的路由数为 0。
|
||||
|
||||
## 自动化验证
|
||||
|
||||
@@ -24,7 +25,7 @@
|
||||
| --- | --- |
|
||||
| Contract API 全量 | `286 passed, 3 skipped`;跳过项仅为需显式 PostgreSQL URL 的并发专项 |
|
||||
| AI Platform API、AI worker、Python SDK | `162 passed` |
|
||||
| Infra production release tools | `62 passed` |
|
||||
| Infra production release tools | `63 passed` |
|
||||
| 本轮 timeout 定向回归 | `28 passed` |
|
||||
| Contract Web 单元测试 | `17 passed`,其中既有入口/会话/工作空间 12 项,生产 API base path、幂等规范化和逻辑权限路径 5 项 |
|
||||
| Contract Web typecheck | 通过 |
|
||||
@@ -47,12 +48,13 @@
|
||||
- Contract timeout:240 秒;API 与 review worker 容器均实际读取为 `240.0`
|
||||
- Contract API 镜像:`sha256:5072971b7bdda62036791cc5836faece78b5a8615a98fd98ad5e9b4999e3f51a`
|
||||
- Contract review worker 镜像:`sha256:35a76a9f6dd5dd44f6545d029cd0e621ea909251c7aabccd6e4cd860a37f11be`
|
||||
- Contract Web 镜像:`sha256:ed6b931d7a104e042da21c5516728cd193190b3cee11e74e41d98a9f517fba4f`;已定向部署生产 base-path 修复,容器 healthy,`/health` 与 `/login` 均为 HTTP 200,容器内回归 `5/5`
|
||||
- 真实审查:合同 `contract-096914efc203`、任务 `review-e8100aa210d4`,约 93 秒后状态为 `succeeded`,并持久化 AI audit run id
|
||||
- post-deploy release gate:`pass`;Contract 与 AI schema-check 均为 `ok`
|
||||
- post-deploy release gate:`pass`;Contract 与 AI schema-check 均为 `ok`,实时 benchmark 以 15 次/场景、并发 3、预热 2 完成 7/7,错误率均为 0,p95 均低于门槛
|
||||
- 8 个共享预发布服务全部 healthy
|
||||
- 根盘:92% 使用,约 17 GiB 可用;未自动清理镜像、卷或业务数据
|
||||
|
||||
远端变更前备份位于 `/opt/apps/ai-contract-platform-v2-smoke/.runtime/pre-contract-timeout-20260811-1650/`。本地 release gate 证据位于忽略目录 `.runtime/postdeploy-release-gate-timeout-fix-20260811/`,不包含服务密钥。
|
||||
远端 Contract 变更前备份位于 `/opt/apps/ai-contract-platform-v2-smoke/.runtime/pre-contract-timeout-20260811-1650/`;Web base-path 定向部署前备份位于 `/opt/apps/ai-contract-platform-v2-smoke/backups/pre-web-base-path-effbefc.tgz`。最新 release gate 证据位于 `project-docs/qa-artifacts/v2-release-quality-gate/2026-08-11T17-34-10/`;此前 timeout 专项证据位于忽略目录 `.runtime/postdeploy-release-gate-timeout-fix-20260811/`。证据不包含服务密钥。
|
||||
|
||||
## 正式生产阻断项
|
||||
|
||||
|
||||
@@ -194,6 +194,14 @@ python yuqei-infra\scripts\run-release-quality-gate.py `
|
||||
--allow-ai-provider-warning
|
||||
```
|
||||
|
||||
When `--run-benchmark` is enabled, the release gate defaults to 15 measured
|
||||
requests per scenario, concurrency 3, and 2 warmup requests. Including the
|
||||
benchmark probes, this remains below the strictest enforced read limit of 20
|
||||
requests per user per minute. The standalone interface benchmark retains its
|
||||
larger load-test defaults; only raise the release-gate values in an isolated
|
||||
performance environment where the intended anti-abuse response is evaluated
|
||||
separately.
|
||||
|
||||
To require fresh summaries without rerunning every check:
|
||||
|
||||
```powershell
|
||||
|
||||
@@ -28,6 +28,9 @@ REQUIRED_SMOKE_CHECKS = (
|
||||
"remote_db_check",
|
||||
"review_worker_smoke",
|
||||
)
|
||||
DEFAULT_RELEASE_BENCHMARK_REQUESTS_PER_SCENARIO = 15
|
||||
DEFAULT_RELEASE_BENCHMARK_CONCURRENCY = 3
|
||||
DEFAULT_RELEASE_BENCHMARK_WARMUP = 2
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -481,9 +484,21 @@ def parse_args() -> argparse.Namespace:
|
||||
parser.add_argument("--max-smoke-summary-age-hours", type=float, default=0)
|
||||
parser.add_argument("--max-benchmark-summary-age-hours", type=float, default=0)
|
||||
parser.add_argument("--run-benchmark", action="store_true")
|
||||
parser.add_argument("--benchmark-requests-per-scenario", type=int, default=60)
|
||||
parser.add_argument("--benchmark-concurrency", type=int, default=6)
|
||||
parser.add_argument("--benchmark-warmup", type=int, default=3)
|
||||
parser.add_argument(
|
||||
"--benchmark-requests-per-scenario",
|
||||
type=int,
|
||||
default=DEFAULT_RELEASE_BENCHMARK_REQUESTS_PER_SCENARIO,
|
||||
)
|
||||
parser.add_argument(
|
||||
"--benchmark-concurrency",
|
||||
type=int,
|
||||
default=DEFAULT_RELEASE_BENCHMARK_CONCURRENCY,
|
||||
)
|
||||
parser.add_argument(
|
||||
"--benchmark-warmup",
|
||||
type=int,
|
||||
default=DEFAULT_RELEASE_BENCHMARK_WARMUP,
|
||||
)
|
||||
parser.add_argument("--include-heavy-explain", action="store_true")
|
||||
parser.add_argument("--tenant-id", default="default")
|
||||
parser.add_argument("--user-id", default="user-current-admin")
|
||||
@@ -599,6 +614,9 @@ def main() -> int:
|
||||
"contract_api_base": args.contract_api_base,
|
||||
"ai_api_base": args.ai_api_base,
|
||||
"run_benchmark": args.run_benchmark,
|
||||
"benchmark_requests_per_scenario": args.benchmark_requests_per_scenario,
|
||||
"benchmark_concurrency": args.benchmark_concurrency,
|
||||
"benchmark_warmup": args.benchmark_warmup,
|
||||
"max_smoke_summary_age_hours": args.max_smoke_summary_age_hours,
|
||||
"max_benchmark_summary_age_hours": args.max_benchmark_summary_age_hours,
|
||||
"tenant_id": args.tenant_id,
|
||||
|
||||
@@ -39,6 +39,10 @@ contract_product_golden = load_script(
|
||||
"yuqei_contract_product_golden",
|
||||
"yuqei-infra/scripts/run-contract-product-golden.py",
|
||||
)
|
||||
release_quality_gate = load_script(
|
||||
"yuqei_release_quality_gate",
|
||||
"yuqei-infra/scripts/run-release-quality-gate.py",
|
||||
)
|
||||
|
||||
from yuqei_ai_platform_api.provider_acceptance_receipt import ( # noqa: E402
|
||||
RECEIPT_DOMAIN,
|
||||
@@ -243,6 +247,17 @@ def test_shared_compose_and_smoke_deploy_wire_contract_ai_timeout() -> None:
|
||||
assert "AI_PLATFORM_PROVIDER_REQUEST_TIMEOUT_SECONDS=180" in deploy_script
|
||||
|
||||
|
||||
def test_release_gate_benchmark_defaults_fit_enforced_anti_abuse_window(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
monkeypatch.setattr(sys, "argv", ["run-release-quality-gate.py"])
|
||||
|
||||
args = release_quality_gate.parse_args()
|
||||
|
||||
assert args.benchmark_requests_per_scenario + args.benchmark_warmup + 1 < 20
|
||||
assert 1 <= args.benchmark_concurrency <= args.benchmark_requests_per_scenario
|
||||
|
||||
|
||||
def test_legal_qa_edge_timeout_covers_contract_client_maximum() -> None:
|
||||
nginx = (
|
||||
REPO_ROOT / "yuqei-infra/nginx/templates/default.conf.template"
|
||||
|
||||
Reference in New Issue
Block a user