feat: add structured baseline history
This commit is contained in:
Codex
@@ -0,0 +1,158 @@
|
||||
# OCR 后续能力扩展阶段 1 第八批执行记录（2026-05-04）
|
||||
|
||||
## 1. 本批目标
|
||||
|
||||
基于阶段 1 第七批已经完成的样本独立样本夹与滚动基线入口，继续推进第八批：
|
||||
|
||||
1. 让真实样本按批次正式进入样本库。
|
||||
2. 让滚动基线开始自动留存历史快照。
|
||||
3. 让滚动基线开始自动输出趋势对比。
|
||||
|
||||
## 2. 本批改造
|
||||
|
||||
### 样本库
|
||||
|
||||
1. [structured-review-samples/manifest.json](D:/ai-contract-platform/project-docs/structured-review-samples/manifest.json)
|
||||
- 每条样本正式增加 `batch_id`
|
||||
|
||||
2. [structured-review-samples/batches.json](D:/ai-contract-platform/project-docs/structured-review-samples/batches.json)
|
||||
- 新增批次索引文件
|
||||
- 当前批次：
|
||||
- `B-001` 基础结构化样本批
|
||||
- `B-002` 复杂关系规则样本批
|
||||
- `B-003` 扩展合同类型样本批
|
||||
|
||||
3. [真实业务合同批量样本库-2026-05-04.csv](D:/ai-contract-platform/project-docs/真实业务合同批量样本库-2026-05-04.csv)
|
||||
- 样本总表新增 `batch_id`
|
||||
|
||||
### 门禁工具
|
||||
|
||||
1. `backend/tools/review_structured_acceptance_gate.py`
|
||||
- 运行时样本真相源正式切换为：
|
||||
- `manifest.json`
|
||||
- `samples/*.txt`
|
||||
- 新增自动历史留存：
|
||||
- `review-structured-baseline-history.jsonl`
|
||||
- `snapshots/*.json`
|
||||
- 新增自动趋势文件：
|
||||
- `baseline-trend-overall.csv`
|
||||
- `baseline-trend-relationship-overall.csv`
|
||||
- `baseline-trend-by-contract.csv`
|
||||
- `baseline-trend-by-batch.csv`
|
||||
- `baseline-trend-by-relationship-type.csv`
|
||||
- `baseline-trend-summary.md`
|
||||
- 新增稳定输出：
|
||||
- `latest-summary.json`
|
||||
- `latest-field-baseline.csv`
|
||||
- `latest-relationship-baseline.csv`
|
||||
- `latest-field-baseline-by-contract.csv`
|
||||
- `latest-field-baseline-by-batch.csv`
|
||||
- `latest-relationship-baseline-by-type.csv`
|
||||
- `latest-sample-inventory.csv`
|
||||
|
||||
2. `backend/tests/test_review_structured_acceptance_gate.py`
|
||||
- 新增 / 补强：
|
||||
- manifest 读取测试
|
||||
- 按合同类型滚动基线测试
|
||||
- 按关系类型滚动基线测试
|
||||
|
||||
### 文档
|
||||
|
||||
1. [structured-review-samples/README.md](D:/ai-contract-platform/project-docs/structured-review-samples/README.md)
|
||||
- 增加样本库、批次和 `baselines/` 用法说明
|
||||
|
||||
2. [structured-review-samples/baselines/README.md](D:/ai-contract-platform/project-docs/structured-review-samples/baselines/README.md)
|
||||
- 新增基线目录说明
|
||||
|
||||
3. [结构化字段与关系准确率基线-2026-05-04.md](D:/ai-contract-platform/project-docs/结构化字段与关系准确率基线-2026-05-04.md)
|
||||
- 新增按批次滚动基线入口说明
|
||||
|
||||
## 3. 当前批次结构
|
||||
|
||||
### B-001 基础结构化样本批
|
||||
|
||||
- `S-001`
|
||||
- `S-002`
|
||||
- `S-003`
|
||||
- `S-004`
|
||||
|
||||
### B-002 复杂关系规则样本批
|
||||
|
||||
- `S-005`
|
||||
- `S-006`
|
||||
- `S-007`
|
||||
- `S-008`
|
||||
|
||||
### B-003 扩展合同类型样本批
|
||||
|
||||
- `S-009`
|
||||
- `S-010`
|
||||
|
||||
## 4. 验证结果
|
||||
|
||||
### 本地验证
|
||||
|
||||
1. 后端：
|
||||
- `backend\\.venv\\Scripts\\python.exe -m unittest tests.test_review_structured_acceptance_gate tests.test_review_service tests.test_review_service_rules tests.test_reviews_api`
|
||||
- `PASS`
|
||||
|
||||
2. 前端：
|
||||
- `frontend\\npm.cmd run build`
|
||||
- `PASS`
|
||||
|
||||
### 线上验证
|
||||
|
||||
1. 结构化门禁：
|
||||
- `project-docs/qa-artifacts/review-structured-acceptance-gate/2026-05-04T19-49-41-392/`
|
||||
- 结果：`PASS`
|
||||
|
||||
2. PDF 门禁：
|
||||
- `project-docs/qa-artifacts/pdf-acceptance-gate/2026-05-04T19-49-41-585/`
|
||||
- 结果：`PASS`
|
||||
|
||||
## 5. 当前新增资产
|
||||
|
||||
### 历史留存
|
||||
|
||||
- `project-docs/structured-review-samples/baselines/review-structured-baseline-history.jsonl`
|
||||
- `project-docs/structured-review-samples/baselines/snapshots/*.json`
|
||||
|
||||
### 最新基线
|
||||
|
||||
- `project-docs/structured-review-samples/baselines/latest-field-baseline.csv`
|
||||
- `project-docs/structured-review-samples/baselines/latest-relationship-baseline.csv`
|
||||
- `project-docs/structured-review-samples/baselines/latest-field-baseline-by-contract.csv`
|
||||
- `project-docs/structured-review-samples/baselines/latest-field-baseline-by-batch.csv`
|
||||
- `project-docs/structured-review-samples/baselines/latest-relationship-baseline-by-type.csv`
|
||||
- `project-docs/structured-review-samples/baselines/latest-sample-inventory.csv`
|
||||
|
||||
### 趋势对比
|
||||
|
||||
- `project-docs/structured-review-samples/baselines/baseline-trend-overall.csv`
|
||||
- `project-docs/structured-review-samples/baselines/baseline-trend-relationship-overall.csv`
|
||||
- `project-docs/structured-review-samples/baselines/baseline-trend-by-contract.csv`
|
||||
- `project-docs/structured-review-samples/baselines/baseline-trend-by-batch.csv`
|
||||
- `project-docs/structured-review-samples/baselines/baseline-trend-by-relationship-type.csv`
|
||||
- `project-docs/structured-review-samples/baselines/baseline-trend-summary.md`
|
||||
|
||||
## 6. 当前结论
|
||||
|
||||
阶段 1 第八批已经闭环：
|
||||
|
||||
1. 样本库已经具备批次语义。
|
||||
2. 基线已经开始自动留存历史快照。
|
||||
3. 基线已经开始自动产出趋势对比。
|
||||
4. 结构化门禁已经能对当前表现和历史表现做出持续留痕。
|
||||
|
||||
## 7. 下一步建议
|
||||
|
||||
下一步最自然的是进入：
|
||||
|
||||
**阶段 1 第九批：真实来源批量扩容与批次趋势可视化**
|
||||
|
||||
建议优先处理：
|
||||
|
||||
1. 再引入一批新的真实脱敏合同样本。
|
||||
2. 给每批样本加来源批次说明和业务线标签。
|
||||
3. 把趋势结果做成更直观的汇总页或图表。
|
||||
4. 开始识别字段准确率的波动点和弱关系热点。
|
||||
@@ -8,13 +8,19 @@
|
||||
- 样本元数据清单
|
||||
- 包含：
|
||||
- 样本编号
|
||||
- 批次编号
|
||||
- 合同族
|
||||
- 样本来源说明
|
||||
- 预期字段
|
||||
- 预期关系
|
||||
- 预期置信度
|
||||
- `batches.json`
|
||||
- 样本批次说明
|
||||
- 描述每个批次的目标、关注点和包含的样本
|
||||
- `samples/`
|
||||
- 每条样本对应一份独立文本文件
|
||||
- `baselines/`
|
||||
- 结构化门禁的最新基线、副本快照、趋势文件与历史记录
|
||||
|
||||
## 当前样本
|
||||
|
||||
@@ -45,3 +51,13 @@
|
||||
当前主要被以下工具读取：
|
||||
|
||||
- `backend/tools/review_structured_acceptance_gate.py`
|
||||
|
||||
## 基线与趋势
|
||||
|
||||
当结构化门禁运行后，会在 `baselines/` 下生成：
|
||||
|
||||
1. `latest-*.csv / latest-summary.json`
|
||||
2. `snapshots/*.json`
|
||||
3. `review-structured-baseline-history.jsonl`
|
||||
4. `baseline-trend-*.csv`
|
||||
5. `baseline-trend-summary.md`
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
# Structured Review Baselines
|
||||
|
||||
## 目录用途
|
||||
|
||||
本目录用于留存结构化审查门禁的历史快照、最新基线与趋势文件。
|
||||
|
||||
## 文件说明
|
||||
|
||||
### 最新基线
|
||||
|
||||
1. `latest-summary.json`
|
||||
2. `latest-field-baseline.csv`
|
||||
3. `latest-relationship-baseline.csv`
|
||||
4. `latest-field-baseline-by-contract.csv`
|
||||
5. `latest-field-baseline-by-batch.csv`
|
||||
6. `latest-relationship-baseline-by-type.csv`
|
||||
7. `latest-sample-inventory.csv`
|
||||
|
||||
### 历史记录
|
||||
|
||||
1. `review-structured-baseline-history.jsonl`
|
||||
2. `snapshots/*.json`
|
||||
|
||||
### 趋势文件
|
||||
|
||||
1. `baseline-trend-overall.csv`
|
||||
2. `baseline-trend-relationship-overall.csv`
|
||||
3. `baseline-trend-by-contract.csv`
|
||||
4. `baseline-trend-by-batch.csv`
|
||||
5. `baseline-trend-by-relationship-type.csv`
|
||||
6. `baseline-trend-summary.md`
|
||||
|
||||
## 使用方式
|
||||
|
||||
1. 运行 `scripts/run-review-structured-acceptance-gate.ps1`
|
||||
2. 门禁完成后会自动刷新本目录内容
|
||||
3. 使用 `latest-*` 查看当前基线
|
||||
4. 使用 `baseline-trend-*` 查看与前一次快照的对比
|
||||
5. 使用 `review-structured-baseline-history.jsonl` 查看历史留存
|
||||
@@ -0,0 +1,4 @@
|
||||
batch_id,current_contract_type_accuracy,previous_contract_type_accuracy,delta_contract_type_accuracy,current_payment_milestone_accuracy,previous_payment_milestone_accuracy,delta_payment_milestone_accuracy
|
||||
B-001,100.0,100.0,0.0,100.0,100.0,0.0
|
||||
B-002,100.0,100.0,0.0,100.0,100.0,0.0
|
||||
B-003,100.0,100.0,0.0,100.0,100.0,0.0
|
||||
|
@@ -0,0 +1,8 @@
|
||||
contract_family,current_contract_type_accuracy,previous_contract_type_accuracy,delta_contract_type_accuracy,current_payment_milestone_accuracy,previous_payment_milestone_accuracy,delta_payment_milestone_accuracy,current_relationship_presence_accuracy,previous_relationship_presence_accuracy,delta_relationship_presence_accuracy
|
||||
service,100.0,100.0,0.0,100.0,100.0,0.0,100.0,100.0,0.0
|
||||
procurement,100.0,100.0,0.0,100.0,100.0,0.0,0.0,0.0,0.0
|
||||
sales,100.0,100.0,0.0,100.0,100.0,0.0,100.0,100.0,0.0
|
||||
lease,100.0,100.0,0.0,0.0,0.0,0.0,100.0,100.0,0.0
|
||||
development,100.0,100.0,0.0,100.0,100.0,0.0,100.0,100.0,0.0
|
||||
license,100.0,100.0,0.0,100.0,100.0,0.0,100.0,100.0,0.0
|
||||
entrustment,100.0,100.0,0.0,100.0,100.0,0.0,100.0,100.0,0.0
|
||||
|
+8
@@ -0,0 +1,8 @@
|
||||
relationship_type,current_presence_accuracy,previous_presence_accuracy,delta_presence_accuracy,current_confidence_accuracy,previous_confidence_accuracy,delta_confidence_accuracy
|
||||
payment_depends_on_acceptance,100.0,100.0,0.0,100.0,100.0,0.0
|
||||
payment_depends_on_invoice,100.0,100.0,0.0,0.0,0.0,0.0
|
||||
liability_cap_conflicts_with_breach,100.0,100.0,0.0,100.0,100.0,0.0
|
||||
payment_depends_on_go_live,100.0,100.0,0.0,0.0,0.0,0.0
|
||||
payment_after_warranty_period,100.0,100.0,0.0,100.0,100.0,0.0
|
||||
liability_cap_conflicts_with_confidentiality,100.0,100.0,0.0,0.0,0.0,0.0
|
||||
liability_cap_conflicts_with_ip,100.0,100.0,0.0,100.0,100.0,0.0
|
||||
|
@@ -0,0 +1,9 @@
|
||||
metric,current_accuracy,previous_accuracy,delta_accuracy
|
||||
contract_type,100.0,100.0,0.0
|
||||
amount,100.0,100.0,0.0
|
||||
term_start,100.0,100.0,0.0
|
||||
term_end,100.0,100.0,0.0
|
||||
payment_milestones,100.0,100.0,0.0
|
||||
clause_types,100.0,100.0,0.0
|
||||
intellectual_property_owner,100.0,100.0,0.0
|
||||
liability_carve_outs,100.0,100.0,0.0
|
||||
|
+6
@@ -0,0 +1,6 @@
|
||||
metric,current_accuracy,previous_accuracy,delta_accuracy
|
||||
relationship_presence,100.0,100.0,0.0
|
||||
relationship_absence,100.0,100.0,0.0
|
||||
relationship_confidence,100.0,100.0,0.0
|
||||
relationship_high_confidence_expected,100.0,100.0,0.0
|
||||
relationship_weak_hint_expected,100.0,100.0,0.0
|
||||
|
@@ -0,0 +1,13 @@
|
||||
# Structured Baseline Trend Summary
|
||||
|
||||
- Latest snapshot: `2026-05-04T11:49:42.278492+00:00`
|
||||
- Sample count: `10`
|
||||
- Previous snapshot: `2026-05-04T11:40:35.844773+00:00`
|
||||
|
||||
Generated trend files:
|
||||
|
||||
1. `baseline-trend-overall.csv`
|
||||
2. `baseline-trend-relationship-overall.csv`
|
||||
3. `baseline-trend-by-contract.csv`
|
||||
4. `baseline-trend-by-batch.csv`
|
||||
5. `baseline-trend-by-relationship-type.csv`
|
||||
@@ -0,0 +1,4 @@
|
||||
batch_id,contract_type_passed,contract_type_total,contract_type_accuracy,payment_milestone_passed,payment_milestone_total,payment_milestone_accuracy
|
||||
B-001,4,4,100.0,3,3,100.0
|
||||
B-002,4,4,100.0,4,4,100.0
|
||||
B-003,2,2,100.0,2,2,100.0
|
||||
|
@@ -0,0 +1,8 @@
|
||||
contract_family,contract_type_passed,contract_type_total,contract_type_accuracy,payment_milestone_passed,payment_milestone_total,payment_milestone_accuracy,relationship_presence_passed,relationship_presence_total,relationship_presence_accuracy
|
||||
service,4,4,100.0,4,4,100.0,13,13,100.0
|
||||
procurement,1,1,100.0,1,1,100.0,0,0,0.0
|
||||
sales,1,1,100.0,1,1,100.0,2,2,100.0
|
||||
lease,1,1,100.0,0,0,0.0,1,1,100.0
|
||||
development,1,1,100.0,1,1,100.0,2,2,100.0
|
||||
license,1,1,100.0,1,1,100.0,1,1,100.0
|
||||
entrustment,1,1,100.0,1,1,100.0,2,2,100.0
|
||||
|
@@ -0,0 +1,9 @@
|
||||
metric,passed,total,accuracy
|
||||
contract_type,10,10,100.0
|
||||
amount,8,8,100.0
|
||||
term_start,6,6,100.0
|
||||
term_end,6,6,100.0
|
||||
payment_milestones,9,9,100.0
|
||||
clause_types,32,32,100.0
|
||||
intellectual_property_owner,3,3,100.0
|
||||
liability_carve_outs,7,7,100.0
|
||||
|
+8
@@ -0,0 +1,8 @@
|
||||
relationship_type,presence_passed,presence_total,presence_accuracy,confidence_passed,confidence_total,confidence_accuracy
|
||||
payment_depends_on_acceptance,7,7,100.0,3,3,100.0
|
||||
payment_depends_on_invoice,7,7,100.0,0,0,0.0
|
||||
liability_cap_conflicts_with_breach,3,3,100.0,1,1,100.0
|
||||
payment_depends_on_go_live,1,1,100.0,0,0,0.0
|
||||
payment_after_warranty_period,1,1,100.0,1,1,100.0
|
||||
liability_cap_conflicts_with_confidentiality,1,1,100.0,0,0,0.0
|
||||
liability_cap_conflicts_with_ip,1,1,100.0,1,1,100.0
|
||||
|
@@ -0,0 +1,6 @@
|
||||
metric,passed,total,accuracy
|
||||
relationship_presence,21,21,100.0
|
||||
relationship_absence,3,3,100.0
|
||||
relationship_confidence,6,6,100.0
|
||||
relationship_high_confidence_expected,4,4,100.0
|
||||
relationship_weak_hint_expected,2,2,100.0
|
||||
|
@@ -0,0 +1,11 @@
|
||||
case_id,batch_id,label,contract_family,source_origin,redaction_level,expected_relationships,expected_payment_milestones
|
||||
S-001,B-001,服务合同结构化摘要,service,deidentified_realistic,sanitized,2,1
|
||||
S-002,B-001,采购合同缺失条款提示,procurement,deidentified_realistic,sanitized,0,1
|
||||
S-003,B-001,购销合同编号条款分段,sales,deidentified_realistic,sanitized,2,1
|
||||
S-004,B-001,租赁合同责任限制识别,lease,deidentified_realistic,sanitized,1,0
|
||||
S-005,B-002,开发合同付款节点拆解,development,deidentified_realistic,sanitized,2,2
|
||||
S-006,B-002,复杂实施合同关系深化,service,deidentified_realistic,sanitized,7,4
|
||||
S-007,B-002,实施合同责任限制含 carve-out,service,deidentified_realistic,sanitized,3,1
|
||||
S-008,B-002,弱提示付款依赖样本,service,deidentified_realistic,sanitized,1,1
|
||||
S-009,B-003,软件许可协议归属与许可关系,license,deidentified_realistic,sanitized,1,2
|
||||
S-010,B-003,委托服务合同多节点验收关系,entrustment,deidentified_realistic,sanitized,2,3
|
||||
|
@@ -0,0 +1,369 @@
|
||||
{
|
||||
"generated_at": "2026-05-04T11:49:42.278492+00:00",
|
||||
"artifact_dir": "D:\\ai-contract-platform\\project-docs\\qa-artifacts\\review-structured-acceptance-gate\\2026-05-04T19-49-41-392",
|
||||
"sample_count": 10,
|
||||
"field_baseline": [
|
||||
{
|
||||
"metric": "contract_type",
|
||||
"passed": 10,
|
||||
"total": 10,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "amount",
|
||||
"passed": 8,
|
||||
"total": 8,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "term_start",
|
||||
"passed": 6,
|
||||
"total": 6,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "term_end",
|
||||
"passed": 6,
|
||||
"total": 6,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "payment_milestones",
|
||||
"passed": 9,
|
||||
"total": 9,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "clause_types",
|
||||
"passed": 32,
|
||||
"total": 32,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "intellectual_property_owner",
|
||||
"passed": 3,
|
||||
"total": 3,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "liability_carve_outs",
|
||||
"passed": 7,
|
||||
"total": 7,
|
||||
"accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"relationship_baseline": [
|
||||
{
|
||||
"metric": "relationship_presence",
|
||||
"passed": 21,
|
||||
"total": 21,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "relationship_absence",
|
||||
"passed": 3,
|
||||
"total": 3,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "relationship_confidence",
|
||||
"passed": 6,
|
||||
"total": 6,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "relationship_high_confidence_expected",
|
||||
"passed": 4,
|
||||
"total": 4,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "relationship_weak_hint_expected",
|
||||
"passed": 2,
|
||||
"total": 2,
|
||||
"accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"field_baseline_by_contract": [
|
||||
{
|
||||
"contract_family": "service",
|
||||
"contract_type_passed": 4,
|
||||
"contract_type_total": 4,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 4,
|
||||
"payment_milestone_total": 4,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 13,
|
||||
"relationship_presence_total": 13,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "procurement",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 0,
|
||||
"relationship_presence_total": 0,
|
||||
"relationship_presence_accuracy": 0.0
|
||||
},
|
||||
{
|
||||
"contract_family": "sales",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 2,
|
||||
"relationship_presence_total": 2,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "lease",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 0,
|
||||
"payment_milestone_total": 0,
|
||||
"payment_milestone_accuracy": 0.0,
|
||||
"relationship_presence_passed": 1,
|
||||
"relationship_presence_total": 1,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "development",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 2,
|
||||
"relationship_presence_total": 2,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "license",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 1,
|
||||
"relationship_presence_total": 1,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "entrustment",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 2,
|
||||
"relationship_presence_total": 2,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"field_baseline_by_batch": [
|
||||
{
|
||||
"batch_id": "B-001",
|
||||
"contract_type_passed": 4,
|
||||
"contract_type_total": 4,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 3,
|
||||
"payment_milestone_total": 3,
|
||||
"payment_milestone_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"batch_id": "B-002",
|
||||
"contract_type_passed": 4,
|
||||
"contract_type_total": 4,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 4,
|
||||
"payment_milestone_total": 4,
|
||||
"payment_milestone_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"batch_id": "B-003",
|
||||
"contract_type_passed": 2,
|
||||
"contract_type_total": 2,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 2,
|
||||
"payment_milestone_total": 2,
|
||||
"payment_milestone_accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"relationship_baseline_by_type": [
|
||||
{
|
||||
"relationship_type": "payment_depends_on_acceptance",
|
||||
"presence_passed": 7,
|
||||
"presence_total": 7,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 3,
|
||||
"confidence_total": 3,
|
||||
"confidence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "payment_depends_on_invoice",
|
||||
"presence_passed": 7,
|
||||
"presence_total": 7,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 0,
|
||||
"confidence_total": 0,
|
||||
"confidence_accuracy": 0.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "liability_cap_conflicts_with_breach",
|
||||
"presence_passed": 3,
|
||||
"presence_total": 3,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 1,
|
||||
"confidence_total": 1,
|
||||
"confidence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "payment_depends_on_go_live",
|
||||
"presence_passed": 1,
|
||||
"presence_total": 1,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 0,
|
||||
"confidence_total": 0,
|
||||
"confidence_accuracy": 0.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "payment_after_warranty_period",
|
||||
"presence_passed": 1,
|
||||
"presence_total": 1,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 1,
|
||||
"confidence_total": 1,
|
||||
"confidence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "liability_cap_conflicts_with_confidentiality",
|
||||
"presence_passed": 1,
|
||||
"presence_total": 1,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 0,
|
||||
"confidence_total": 0,
|
||||
"confidence_accuracy": 0.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "liability_cap_conflicts_with_ip",
|
||||
"presence_passed": 1,
|
||||
"presence_total": 1,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 1,
|
||||
"confidence_total": 1,
|
||||
"confidence_accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"sample_inventory": [
|
||||
{
|
||||
"case_id": "S-001",
|
||||
"batch_id": "B-001",
|
||||
"label": "服务合同结构化摘要",
|
||||
"contract_family": "service",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 2,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-002",
|
||||
"batch_id": "B-001",
|
||||
"label": "采购合同缺失条款提示",
|
||||
"contract_family": "procurement",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 0,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-003",
|
||||
"batch_id": "B-001",
|
||||
"label": "购销合同编号条款分段",
|
||||
"contract_family": "sales",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 2,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-004",
|
||||
"batch_id": "B-001",
|
||||
"label": "租赁合同责任限制识别",
|
||||
"contract_family": "lease",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 1,
|
||||
"expected_payment_milestones": 0
|
||||
},
|
||||
{
|
||||
"case_id": "S-005",
|
||||
"batch_id": "B-002",
|
||||
"label": "开发合同付款节点拆解",
|
||||
"contract_family": "development",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 2,
|
||||
"expected_payment_milestones": 2
|
||||
},
|
||||
{
|
||||
"case_id": "S-006",
|
||||
"batch_id": "B-002",
|
||||
"label": "复杂实施合同关系深化",
|
||||
"contract_family": "service",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 7,
|
||||
"expected_payment_milestones": 4
|
||||
},
|
||||
{
|
||||
"case_id": "S-007",
|
||||
"batch_id": "B-002",
|
||||
"label": "实施合同责任限制含 carve-out",
|
||||
"contract_family": "service",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 3,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-008",
|
||||
"batch_id": "B-002",
|
||||
"label": "弱提示付款依赖样本",
|
||||
"contract_family": "service",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 1,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-009",
|
||||
"batch_id": "B-003",
|
||||
"label": "软件许可协议归属与许可关系",
|
||||
"contract_family": "license",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 1,
|
||||
"expected_payment_milestones": 2
|
||||
},
|
||||
{
|
||||
"case_id": "S-010",
|
||||
"batch_id": "B-003",
|
||||
"label": "委托服务合同多节点验收关系",
|
||||
"contract_family": "entrustment",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 2,
|
||||
"expected_payment_milestones": 3
|
||||
}
|
||||
]
|
||||
}
|
||||
+2
@@ -0,0 +1,2 @@
|
||||
{"generated_at": "2026-05-04T11:40:35.844773+00:00", "artifact_dir": "D:\\ai-contract-platform\\project-docs\\qa-artifacts\\review-structured-acceptance-gate\\2026-05-04T19-40-34-945", "snapshot_path": "D:\\ai-contract-platform\\project-docs\\structured-review-samples\\baselines\\snapshots\\2026-05-04T11-40-35-844773+00-00-review-structured-baseline-snapshot.json", "sample_count": 10, "field_metric_count": 8, "relationship_metric_count": 5}
|
||||
{"generated_at": "2026-05-04T11:49:42.278492+00:00", "artifact_dir": "D:\\ai-contract-platform\\project-docs\\qa-artifacts\\review-structured-acceptance-gate\\2026-05-04T19-49-41-392", "snapshot_path": "D:\\ai-contract-platform\\project-docs\\structured-review-samples\\baselines\\snapshots\\2026-05-04T11-49-42-278492+00-00-review-structured-baseline-snapshot.json", "sample_count": 10, "field_metric_count": 8, "relationship_metric_count": 5}
|
||||
+369
@@ -0,0 +1,369 @@
|
||||
{
|
||||
"generated_at": "2026-05-04T11:40:35.844773+00:00",
|
||||
"artifact_dir": "D:\\ai-contract-platform\\project-docs\\qa-artifacts\\review-structured-acceptance-gate\\2026-05-04T19-40-34-945",
|
||||
"sample_count": 10,
|
||||
"field_baseline": [
|
||||
{
|
||||
"metric": "contract_type",
|
||||
"passed": 10,
|
||||
"total": 10,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "amount",
|
||||
"passed": 8,
|
||||
"total": 8,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "term_start",
|
||||
"passed": 6,
|
||||
"total": 6,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "term_end",
|
||||
"passed": 6,
|
||||
"total": 6,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "payment_milestones",
|
||||
"passed": 9,
|
||||
"total": 9,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "clause_types",
|
||||
"passed": 32,
|
||||
"total": 32,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "intellectual_property_owner",
|
||||
"passed": 3,
|
||||
"total": 3,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "liability_carve_outs",
|
||||
"passed": 7,
|
||||
"total": 7,
|
||||
"accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"relationship_baseline": [
|
||||
{
|
||||
"metric": "relationship_presence",
|
||||
"passed": 21,
|
||||
"total": 21,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "relationship_absence",
|
||||
"passed": 3,
|
||||
"total": 3,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "relationship_confidence",
|
||||
"passed": 6,
|
||||
"total": 6,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "relationship_high_confidence_expected",
|
||||
"passed": 4,
|
||||
"total": 4,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "relationship_weak_hint_expected",
|
||||
"passed": 2,
|
||||
"total": 2,
|
||||
"accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"field_baseline_by_contract": [
|
||||
{
|
||||
"contract_family": "service",
|
||||
"contract_type_passed": 4,
|
||||
"contract_type_total": 4,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 4,
|
||||
"payment_milestone_total": 4,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 13,
|
||||
"relationship_presence_total": 13,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "procurement",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 0,
|
||||
"relationship_presence_total": 0,
|
||||
"relationship_presence_accuracy": 0.0
|
||||
},
|
||||
{
|
||||
"contract_family": "sales",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 2,
|
||||
"relationship_presence_total": 2,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "lease",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 0,
|
||||
"payment_milestone_total": 0,
|
||||
"payment_milestone_accuracy": 0.0,
|
||||
"relationship_presence_passed": 1,
|
||||
"relationship_presence_total": 1,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "development",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 2,
|
||||
"relationship_presence_total": 2,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "license",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 1,
|
||||
"relationship_presence_total": 1,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "entrustment",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 2,
|
||||
"relationship_presence_total": 2,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"field_baseline_by_batch": [
|
||||
{
|
||||
"batch_id": "B-001",
|
||||
"contract_type_passed": 4,
|
||||
"contract_type_total": 4,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 3,
|
||||
"payment_milestone_total": 3,
|
||||
"payment_milestone_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"batch_id": "B-002",
|
||||
"contract_type_passed": 4,
|
||||
"contract_type_total": 4,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 4,
|
||||
"payment_milestone_total": 4,
|
||||
"payment_milestone_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"batch_id": "B-003",
|
||||
"contract_type_passed": 2,
|
||||
"contract_type_total": 2,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 2,
|
||||
"payment_milestone_total": 2,
|
||||
"payment_milestone_accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"relationship_baseline_by_type": [
|
||||
{
|
||||
"relationship_type": "payment_depends_on_acceptance",
|
||||
"presence_passed": 7,
|
||||
"presence_total": 7,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 3,
|
||||
"confidence_total": 3,
|
||||
"confidence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "payment_depends_on_invoice",
|
||||
"presence_passed": 7,
|
||||
"presence_total": 7,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 0,
|
||||
"confidence_total": 0,
|
||||
"confidence_accuracy": 0.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "liability_cap_conflicts_with_breach",
|
||||
"presence_passed": 3,
|
||||
"presence_total": 3,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 1,
|
||||
"confidence_total": 1,
|
||||
"confidence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "payment_depends_on_go_live",
|
||||
"presence_passed": 1,
|
||||
"presence_total": 1,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 0,
|
||||
"confidence_total": 0,
|
||||
"confidence_accuracy": 0.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "payment_after_warranty_period",
|
||||
"presence_passed": 1,
|
||||
"presence_total": 1,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 1,
|
||||
"confidence_total": 1,
|
||||
"confidence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "liability_cap_conflicts_with_confidentiality",
|
||||
"presence_passed": 1,
|
||||
"presence_total": 1,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 0,
|
||||
"confidence_total": 0,
|
||||
"confidence_accuracy": 0.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "liability_cap_conflicts_with_ip",
|
||||
"presence_passed": 1,
|
||||
"presence_total": 1,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 1,
|
||||
"confidence_total": 1,
|
||||
"confidence_accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"sample_inventory": [
|
||||
{
|
||||
"case_id": "S-001",
|
||||
"batch_id": "B-001",
|
||||
"label": "服务合同结构化摘要",
|
||||
"contract_family": "service",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 2,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-002",
|
||||
"batch_id": "B-001",
|
||||
"label": "采购合同缺失条款提示",
|
||||
"contract_family": "procurement",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 0,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-003",
|
||||
"batch_id": "B-001",
|
||||
"label": "购销合同编号条款分段",
|
||||
"contract_family": "sales",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 2,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-004",
|
||||
"batch_id": "B-001",
|
||||
"label": "租赁合同责任限制识别",
|
||||
"contract_family": "lease",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 1,
|
||||
"expected_payment_milestones": 0
|
||||
},
|
||||
{
|
||||
"case_id": "S-005",
|
||||
"batch_id": "B-002",
|
||||
"label": "开发合同付款节点拆解",
|
||||
"contract_family": "development",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 2,
|
||||
"expected_payment_milestones": 2
|
||||
},
|
||||
{
|
||||
"case_id": "S-006",
|
||||
"batch_id": "B-002",
|
||||
"label": "复杂实施合同关系深化",
|
||||
"contract_family": "service",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 7,
|
||||
"expected_payment_milestones": 4
|
||||
},
|
||||
{
|
||||
"case_id": "S-007",
|
||||
"batch_id": "B-002",
|
||||
"label": "实施合同责任限制含 carve-out",
|
||||
"contract_family": "service",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 3,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-008",
|
||||
"batch_id": "B-002",
|
||||
"label": "弱提示付款依赖样本",
|
||||
"contract_family": "service",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 1,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-009",
|
||||
"batch_id": "B-003",
|
||||
"label": "软件许可协议归属与许可关系",
|
||||
"contract_family": "license",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 1,
|
||||
"expected_payment_milestones": 2
|
||||
},
|
||||
{
|
||||
"case_id": "S-010",
|
||||
"batch_id": "B-003",
|
||||
"label": "委托服务合同多节点验收关系",
|
||||
"contract_family": "entrustment",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 2,
|
||||
"expected_payment_milestones": 3
|
||||
}
|
||||
]
|
||||
}
|
||||
+369
@@ -0,0 +1,369 @@
|
||||
{
|
||||
"generated_at": "2026-05-04T11:49:42.278492+00:00",
|
||||
"artifact_dir": "D:\\ai-contract-platform\\project-docs\\qa-artifacts\\review-structured-acceptance-gate\\2026-05-04T19-49-41-392",
|
||||
"sample_count": 10,
|
||||
"field_baseline": [
|
||||
{
|
||||
"metric": "contract_type",
|
||||
"passed": 10,
|
||||
"total": 10,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "amount",
|
||||
"passed": 8,
|
||||
"total": 8,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "term_start",
|
||||
"passed": 6,
|
||||
"total": 6,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "term_end",
|
||||
"passed": 6,
|
||||
"total": 6,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "payment_milestones",
|
||||
"passed": 9,
|
||||
"total": 9,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "clause_types",
|
||||
"passed": 32,
|
||||
"total": 32,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "intellectual_property_owner",
|
||||
"passed": 3,
|
||||
"total": 3,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "liability_carve_outs",
|
||||
"passed": 7,
|
||||
"total": 7,
|
||||
"accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"relationship_baseline": [
|
||||
{
|
||||
"metric": "relationship_presence",
|
||||
"passed": 21,
|
||||
"total": 21,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "relationship_absence",
|
||||
"passed": 3,
|
||||
"total": 3,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "relationship_confidence",
|
||||
"passed": 6,
|
||||
"total": 6,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "relationship_high_confidence_expected",
|
||||
"passed": 4,
|
||||
"total": 4,
|
||||
"accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"metric": "relationship_weak_hint_expected",
|
||||
"passed": 2,
|
||||
"total": 2,
|
||||
"accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"field_baseline_by_contract": [
|
||||
{
|
||||
"contract_family": "service",
|
||||
"contract_type_passed": 4,
|
||||
"contract_type_total": 4,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 4,
|
||||
"payment_milestone_total": 4,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 13,
|
||||
"relationship_presence_total": 13,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "procurement",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 0,
|
||||
"relationship_presence_total": 0,
|
||||
"relationship_presence_accuracy": 0.0
|
||||
},
|
||||
{
|
||||
"contract_family": "sales",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 2,
|
||||
"relationship_presence_total": 2,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "lease",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 0,
|
||||
"payment_milestone_total": 0,
|
||||
"payment_milestone_accuracy": 0.0,
|
||||
"relationship_presence_passed": 1,
|
||||
"relationship_presence_total": 1,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "development",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 2,
|
||||
"relationship_presence_total": 2,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "license",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 1,
|
||||
"relationship_presence_total": 1,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"contract_family": "entrustment",
|
||||
"contract_type_passed": 1,
|
||||
"contract_type_total": 1,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 1,
|
||||
"payment_milestone_total": 1,
|
||||
"payment_milestone_accuracy": 100.0,
|
||||
"relationship_presence_passed": 2,
|
||||
"relationship_presence_total": 2,
|
||||
"relationship_presence_accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"field_baseline_by_batch": [
|
||||
{
|
||||
"batch_id": "B-001",
|
||||
"contract_type_passed": 4,
|
||||
"contract_type_total": 4,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 3,
|
||||
"payment_milestone_total": 3,
|
||||
"payment_milestone_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"batch_id": "B-002",
|
||||
"contract_type_passed": 4,
|
||||
"contract_type_total": 4,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 4,
|
||||
"payment_milestone_total": 4,
|
||||
"payment_milestone_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"batch_id": "B-003",
|
||||
"contract_type_passed": 2,
|
||||
"contract_type_total": 2,
|
||||
"contract_type_accuracy": 100.0,
|
||||
"payment_milestone_passed": 2,
|
||||
"payment_milestone_total": 2,
|
||||
"payment_milestone_accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"relationship_baseline_by_type": [
|
||||
{
|
||||
"relationship_type": "payment_depends_on_acceptance",
|
||||
"presence_passed": 7,
|
||||
"presence_total": 7,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 3,
|
||||
"confidence_total": 3,
|
||||
"confidence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "payment_depends_on_invoice",
|
||||
"presence_passed": 7,
|
||||
"presence_total": 7,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 0,
|
||||
"confidence_total": 0,
|
||||
"confidence_accuracy": 0.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "liability_cap_conflicts_with_breach",
|
||||
"presence_passed": 3,
|
||||
"presence_total": 3,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 1,
|
||||
"confidence_total": 1,
|
||||
"confidence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "payment_depends_on_go_live",
|
||||
"presence_passed": 1,
|
||||
"presence_total": 1,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 0,
|
||||
"confidence_total": 0,
|
||||
"confidence_accuracy": 0.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "payment_after_warranty_period",
|
||||
"presence_passed": 1,
|
||||
"presence_total": 1,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 1,
|
||||
"confidence_total": 1,
|
||||
"confidence_accuracy": 100.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "liability_cap_conflicts_with_confidentiality",
|
||||
"presence_passed": 1,
|
||||
"presence_total": 1,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 0,
|
||||
"confidence_total": 0,
|
||||
"confidence_accuracy": 0.0
|
||||
},
|
||||
{
|
||||
"relationship_type": "liability_cap_conflicts_with_ip",
|
||||
"presence_passed": 1,
|
||||
"presence_total": 1,
|
||||
"presence_accuracy": 100.0,
|
||||
"confidence_passed": 1,
|
||||
"confidence_total": 1,
|
||||
"confidence_accuracy": 100.0
|
||||
}
|
||||
],
|
||||
"sample_inventory": [
|
||||
{
|
||||
"case_id": "S-001",
|
||||
"batch_id": "B-001",
|
||||
"label": "服务合同结构化摘要",
|
||||
"contract_family": "service",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 2,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-002",
|
||||
"batch_id": "B-001",
|
||||
"label": "采购合同缺失条款提示",
|
||||
"contract_family": "procurement",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 0,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-003",
|
||||
"batch_id": "B-001",
|
||||
"label": "购销合同编号条款分段",
|
||||
"contract_family": "sales",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 2,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-004",
|
||||
"batch_id": "B-001",
|
||||
"label": "租赁合同责任限制识别",
|
||||
"contract_family": "lease",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 1,
|
||||
"expected_payment_milestones": 0
|
||||
},
|
||||
{
|
||||
"case_id": "S-005",
|
||||
"batch_id": "B-002",
|
||||
"label": "开发合同付款节点拆解",
|
||||
"contract_family": "development",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 2,
|
||||
"expected_payment_milestones": 2
|
||||
},
|
||||
{
|
||||
"case_id": "S-006",
|
||||
"batch_id": "B-002",
|
||||
"label": "复杂实施合同关系深化",
|
||||
"contract_family": "service",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 7,
|
||||
"expected_payment_milestones": 4
|
||||
},
|
||||
{
|
||||
"case_id": "S-007",
|
||||
"batch_id": "B-002",
|
||||
"label": "实施合同责任限制含 carve-out",
|
||||
"contract_family": "service",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 3,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-008",
|
||||
"batch_id": "B-002",
|
||||
"label": "弱提示付款依赖样本",
|
||||
"contract_family": "service",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 1,
|
||||
"expected_payment_milestones": 1
|
||||
},
|
||||
{
|
||||
"case_id": "S-009",
|
||||
"batch_id": "B-003",
|
||||
"label": "软件许可协议归属与许可关系",
|
||||
"contract_family": "license",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 1,
|
||||
"expected_payment_milestones": 2
|
||||
},
|
||||
{
|
||||
"case_id": "S-010",
|
||||
"batch_id": "B-003",
|
||||
"label": "委托服务合同多节点验收关系",
|
||||
"contract_family": "entrustment",
|
||||
"source_origin": "deidentified_realistic",
|
||||
"redaction_level": "sanitized",
|
||||
"expected_relationships": 2,
|
||||
"expected_payment_milestones": 3
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"batches": [
|
||||
{
|
||||
"batch_id": "B-001",
|
||||
"label": "基础结构化样本批",
|
||||
"focus": "基础字段、缺失条款、编号条款切分",
|
||||
"sample_ids": ["S-001", "S-002", "S-003", "S-004"]
|
||||
},
|
||||
{
|
||||
"batch_id": "B-002",
|
||||
"label": "复杂关系规则样本批",
|
||||
"focus": "付款分期、质保金、责任限制、carve-out、弱提示关系",
|
||||
"sample_ids": ["S-005", "S-006", "S-007", "S-008"]
|
||||
},
|
||||
{
|
||||
"batch_id": "B-003",
|
||||
"label": "扩展合同类型样本批",
|
||||
"focus": "许可协议、委托服务合同与滚动基线扩展",
|
||||
"sample_ids": ["S-009", "S-010"]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -2,6 +2,7 @@
|
||||
"samples": [
|
||||
{
|
||||
"case_id": "S-001",
|
||||
"batch_id": "B-001",
|
||||
"label": "服务合同结构化摘要",
|
||||
"contract_type": "service",
|
||||
"review_as": "party_a",
|
||||
@@ -20,6 +21,7 @@
|
||||
},
|
||||
{
|
||||
"case_id": "S-002",
|
||||
"batch_id": "B-001",
|
||||
"label": "采购合同缺失条款提示",
|
||||
"contract_type": "procurement",
|
||||
"review_as": "party_a",
|
||||
@@ -35,6 +37,7 @@
|
||||
},
|
||||
{
|
||||
"case_id": "S-003",
|
||||
"batch_id": "B-001",
|
||||
"label": "购销合同编号条款分段",
|
||||
"contract_type": null,
|
||||
"review_as": "party_a",
|
||||
@@ -50,6 +53,7 @@
|
||||
},
|
||||
{
|
||||
"case_id": "S-004",
|
||||
"batch_id": "B-001",
|
||||
"label": "租赁合同责任限制识别",
|
||||
"contract_type": null,
|
||||
"review_as": "party_a",
|
||||
@@ -66,6 +70,7 @@
|
||||
},
|
||||
{
|
||||
"case_id": "S-005",
|
||||
"batch_id": "B-002",
|
||||
"label": "开发合同付款节点拆解",
|
||||
"contract_type": null,
|
||||
"review_as": "party_a",
|
||||
@@ -84,6 +89,7 @@
|
||||
},
|
||||
{
|
||||
"case_id": "S-006",
|
||||
"batch_id": "B-002",
|
||||
"label": "复杂实施合同关系深化",
|
||||
"contract_type": null,
|
||||
"review_as": "party_a",
|
||||
@@ -98,7 +104,15 @@
|
||||
"expected_term_start": "2026-08-01",
|
||||
"expected_term_end": "2027-08-01",
|
||||
"expected_payment_milestones": 4,
|
||||
"expected_relationship_types": ["payment_depends_on_acceptance", "payment_depends_on_invoice", "payment_depends_on_go_live", "payment_after_warranty_period", "liability_cap_conflicts_with_breach", "liability_cap_conflicts_with_confidentiality", "liability_cap_conflicts_with_ip"],
|
||||
"expected_relationship_types": [
|
||||
"payment_depends_on_acceptance",
|
||||
"payment_depends_on_invoice",
|
||||
"payment_depends_on_go_live",
|
||||
"payment_after_warranty_period",
|
||||
"liability_cap_conflicts_with_breach",
|
||||
"liability_cap_conflicts_with_confidentiality",
|
||||
"liability_cap_conflicts_with_ip"
|
||||
],
|
||||
"expected_relationship_confidence": {
|
||||
"payment_depends_on_acceptance": "high",
|
||||
"payment_after_warranty_period": "high",
|
||||
@@ -108,6 +122,7 @@
|
||||
},
|
||||
{
|
||||
"case_id": "S-007",
|
||||
"batch_id": "B-002",
|
||||
"label": "实施合同责任限制含 carve-out",
|
||||
"contract_type": null,
|
||||
"review_as": "party_a",
|
||||
@@ -130,6 +145,7 @@
|
||||
},
|
||||
{
|
||||
"case_id": "S-008",
|
||||
"batch_id": "B-002",
|
||||
"label": "弱提示付款依赖样本",
|
||||
"contract_type": null,
|
||||
"review_as": "party_a",
|
||||
@@ -149,6 +165,7 @@
|
||||
},
|
||||
{
|
||||
"case_id": "S-009",
|
||||
"batch_id": "B-003",
|
||||
"label": "软件许可协议归属与许可关系",
|
||||
"contract_type": null,
|
||||
"review_as": "party_a",
|
||||
@@ -170,6 +187,7 @@
|
||||
},
|
||||
{
|
||||
"case_id": "S-010",
|
||||
"batch_id": "B-003",
|
||||
"label": "委托服务合同多节点验收关系",
|
||||
"contract_type": null,
|
||||
"review_as": "party_a",
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
sample_id,title,contract_family,source_origin,redaction_level,source_mode,focus,expected_highlights,confidence_focus
|
||||
S-001,服务合同结构化摘要,service,deidentified_realistic,sanitized,text_contract,基础结构化摘要,合同类型/金额/期限/付款/保密/违约,高置信关系
|
||||
S-002,采购合同缺失条款提示,procurement,deidentified_realistic,sanitized,text_contract,缺失条款识别,付款/交付/缺失验收/缺失保密,弱提示关系
|
||||
S-003,购销合同编号条款分段,sales,deidentified_realistic,sanitized,numbered_contract,编号条款切分,交货/付款/争议解决,高置信关系
|
||||
S-004,租赁合同责任限制识别,lease,deidentified_realistic,sanitized,liability_cap_contract,责任限制识别,交付/责任限制/违约,高置信关系
|
||||
S-005,开发合同付款节点拆解,development,deidentified_realistic,sanitized,milestone_contract,付款分期拆解,首期款/尾款/验收依赖/发票依赖,高置信关系
|
||||
S-006,复杂实施合同关系深化,service,deidentified_realistic,sanitized,complex_realistic_contract,复杂关系组合,付款分期/质保金/保密/知识产权/责任限制/违约,高置信关系
|
||||
S-007,实施合同责任限制含 carve-out,service,deidentified_realistic,sanitized,carveout_contract,carve-out 规则校正,责任限制/保密/知识产权/违约,高置信与反例
|
||||
S-008,弱提示付款依赖样本,service,deidentified_realistic,sanitized,weak_hint_contract,弱提示关系校正,付款依赖验收但无独立验收条款,弱提示关系
|
||||
S-009,软件许可协议归属与许可关系,license,deidentified_realistic,sanitized,license_contract,许可协议归属识别,许可费/知识产权归乙方/授权甲方使用/责任限制,高置信与反例
|
||||
S-010,委托服务合同多节点验收关系,entrustment,deidentified_realistic,sanitized,entrustment_contract,多节点验收关系,预付款/进度款/尾款/验收依赖/争议解决,高置信关系
|
||||
sample_id,batch_id,title,contract_family,source_origin,redaction_level,source_mode,focus,expected_highlights,confidence_focus
|
||||
S-001,B-001,服务合同结构化摘要,service,deidentified_realistic,sanitized,text_contract,基础结构化摘要,合同类型/金额/期限/付款/保密/违约,高置信关系
|
||||
S-002,B-001,采购合同缺失条款提示,procurement,deidentified_realistic,sanitized,text_contract,缺失条款识别,付款/交付/缺失验收/缺失保密,弱提示关系
|
||||
S-003,B-001,购销合同编号条款分段,sales,deidentified_realistic,sanitized,numbered_contract,编号条款切分,交货/付款/争议解决,高置信关系
|
||||
S-004,B-001,租赁合同责任限制识别,lease,deidentified_realistic,sanitized,liability_cap_contract,责任限制识别,交付/责任限制/违约,高置信关系
|
||||
S-005,B-002,开发合同付款节点拆解,development,deidentified_realistic,sanitized,milestone_contract,付款分期拆解,首期款/尾款/验收依赖/发票依赖,高置信关系
|
||||
S-006,B-002,复杂实施合同关系深化,service,deidentified_realistic,sanitized,complex_realistic_contract,复杂关系组合,付款分期/质保金/保密/知识产权/责任限制/违约,高置信关系
|
||||
S-007,B-002,实施合同责任限制含 carve-out,service,deidentified_realistic,sanitized,carveout_contract,carve-out 规则校正,责任限制/保密/知识产权/违约,高置信与反例
|
||||
S-008,B-002,弱提示付款依赖样本,service,deidentified_realistic,sanitized,weak_hint_contract,弱提示关系校正,付款依赖验收但无独立验收条款,弱提示关系
|
||||
S-009,B-003,软件许可协议归属与许可关系,license,deidentified_realistic,sanitized,license_contract,许可协议归属识别,许可费/知识产权归乙方/授权甲方使用/责任限制,高置信与反例
|
||||
S-010,B-003,委托服务合同多节点验收关系,entrustment,deidentified_realistic,sanitized,entrustment_contract,多节点验收关系,预付款/进度款/尾款/验收依赖/争议解决,高置信关系
|
||||
|
||||
|
@@ -13,6 +13,7 @@
|
||||
3. `review-structured-sample-inventory.csv`
|
||||
4. `review-structured-baseline.md`
|
||||
5. `review-structured-field-baseline-by-contract.csv`
|
||||
6. `review-structured-field-baseline-by-batch.csv`
|
||||
6. `review-structured-relationship-baseline-by-type.csv`
|
||||
|
||||
## 2. 字段准确率基线
|
||||
@@ -63,7 +64,9 @@
|
||||
|
||||
1. 按合同类型：
|
||||
- `review-structured-field-baseline-by-contract.csv`
|
||||
2. 按关系类型：
|
||||
2. 按批次：
|
||||
- `review-structured-field-baseline-by-batch.csv`
|
||||
3. 按关系类型：
|
||||
- `review-structured-relationship-baseline-by-type.csv`
|
||||
|
||||
## 6. 基线意义
|
||||
|
||||
Reference in New Issue
Block a user