Files
ai-contract-platform/project-docs/V2-phase3-provider-ocr-secret-ref-pgvector-schema-2026-07-02.md
T
yuqei a3bb78e06b
Security Quality Gate / npm audit, Bandit, pip-audit, detect-secrets (pull_request) Failing after 16m35s
release: freeze V2 release candidate
2026-07-12 15:32:24 +08:00

3.9 KiB

V2 Phase 3 Provider/OCR Secret Ref And pgvector Schema Check

Date: 2026-07-02

Status

Completed and smoke verified on 192.168.3.90.

This slice closes two delivery-hardening items:

  1. Provider/OCR key compatibility through non-sensitive secret refs.
  2. AI Platform pgvector/search runtime schema-check hardening.

Final smoke artifact:

  • project-docs/qa-artifacts/v2-smoke-docker-deploy/2026-07-02T20-39-49

Secret Ref Compatibility

Added api_key_secret_ref support for:

  • ai_provider_configs
  • ocr_provider_configs

Supported reference formats:

  • env:VARIABLE_NAME
  • env://VARIABLE_NAME

Behavior:

  • Provider calls resolve ProviderConfig.api_key_secret_ref first.
  • OCR runtime resolves OcrProviderConfig.api_key_secret_ref when include_secret=True.
  • Existing OCR api_key remains compatible for old callers and old data.
  • Public API responses expose api_key_set / api_key_ref_set, but do not expose plaintext api_key.
  • Provider config change history records the secret ref, not a plaintext key.

Migration:

  • 20260702_0020_add_provider_ocr_secret_refs.py

pgvector Schema Check

Runtime schema-check now includes:

  • secret_ref_columns
  • pgvector_schema

pgvector_schema verifies:

  • knowledge_chunks.embedding
  • knowledge_chunks.embedding_dimension
  • knowledge_chunks.search_vector
  • knowledge_documents.search_vector
  • ix_knowledge_chunks_embedding_hnsw
  • ix_knowledge_chunks_search_vector_gin
  • ix_knowledge_documents_search_vector_gin
  • default indexed embedding dimension consistency: 64

Smoke Verification

Schema endpoint:

GET http://192.168.3.90:18101/api/v1/runtime/schema-check

Result:

{
  "status": "ok",
  "alembic": {
    "actual": "20260702_0020",
    "expected_head": "20260702_0020"
  },
  "secret_ref_columns": "ok",
  "postgres_extensions": "ok",
  "pgvector_schema": {
    "status": "ok",
    "embedded_count": 60002,
    "expected_dimension_count": 60002,
    "non_default_dimension_count": 0,
    "min_embedding_dimension": 64,
    "max_embedding_dimension": 64,
    "missing_indexes": ""
  }
}

Secret ref API smoke:

{
  "provider": {
    "id": "provider-04d78637886c",
    "provider": "mock",
    "model": "secret-ref-smoke-20260702",
    "api_key_secret_ref": "env:SMOKE_PROVIDER_KEY",
    "api_key_set": true,
    "enabled": false,
    "is_default": false
  },
  "provider_change": {
    "count": 1,
    "after_api_key_secret_ref": "env:SMOKE_PROVIDER_KEY",
    "after_api_key_set": true
  },
  "ocr_get": {
    "provider": "disabled",
    "display_name": "Smoke OCR Secret Ref",
    "api_key_secret_ref": "env:SMOKE_OCR_KEY",
    "api_key_set": true,
    "api_key_ref_set": true,
    "enabled": false
  },
  "ocr_plain_key_exposed": false
}

Local Validation

python -m py_compile `
  yuqei-ai-platform\services\ai-platform-api\src\yuqei_ai_platform_api\models.py `
  yuqei-ai-platform\services\ai-platform-api\src\yuqei_ai_platform_api\repository.py `
  yuqei-ai-platform\services\ai-platform-api\src\yuqei_ai_platform_api\persistence\provider_configs.py `
  yuqei-ai-platform\services\ai-platform-api\src\yuqei_ai_platform_api\persistence\worker_runtime.py `
  yuqei-ai-platform\services\ai-platform-api\src\yuqei_ai_platform_api\providers.py `
  yuqei-ai-platform\services\ai-platform-api\src\yuqei_ai_platform_api\schema_check.py `
  yuqei-ai-platform\services\ai-platform-api\src\yuqei_ai_platform_api\secret_refs.py `
  yuqei-ai-platform\services\ai-platform-api\migrations\versions\20260702_0020_add_provider_ocr_secret_refs.py

python -m pytest yuqei-ai-platform\services\ai-platform-api\tests -q

Result:

  • AI Platform API tests: 55 passed.

Remaining Notes

  • This slice does not implement an external secret manager client.
  • Secret refs currently resolve from environment variables only.
  • Old OCR plaintext api_key is retained for compatibility; removal should be handled by a later cleanup migration after callers have moved to refs.