feature/initial-capture-tooling #1

Merged

Xeplosion merged 16 commits from feature/initial-capture-tooling into dev

2026-08-26 23:56:04 +00:00

Xeplosion commented

2026-08-26 23:55:23 +00:00

Owner

What changed and why

Here's the filled template — since there's no tracked issue number for this work, I left that section noting it, but swap in a real number if one exists.
What changed and why

Introduces the initial capture tooling for the Provider Offering Dataset: the JSON Schema (v2.3) providers must conform to, a bootstrap pipeline that generates skeleton capture files from the project's tracking spreadsheets, and the supporting docs and CI gates.

Specifically:

Schema v2.3 (schema/provider-schema-v2.3.json) — discrete and parametric product lines, all three pricing-rate models (linear/piecewise/tiered) including irregular step ranges, ratio/table couplings, add-ons, promotions, cart and interval-discount witnesses, and every feature block. Enforces mode separation, the absent/none/null convention, and a bidirectional currency rule (USD ⇔ null fx_conversion_id). Validated against a reference file (docs/hexanode-example.json).
capture_tools.bootstrap — reads DATA_Providers, REF_Sources, and FORM_Retrieval_Responses (joined on Source_Id) via the Sheets API and generates one PRV-NNNN.wip.json skeleton per provider, prefilled with everything the sheets know and deliberately-invalid placeholders for everything they don't — so the schema-validation report is a per-provider fill-out checklist.
capture_tools.validate — the .wip.json / .json promotion contract: skeletons are exempt from schema validation, completed files (renamed, no .wip) are enforced strictly. This is what the CI gate checks on every push.
utils.sheets_api / utils.drive_api — general-purpose Google API helper libraries (official google-api-python-client, ADC auth), shared with the future reviews-analysis work.
.forgejo/workflows/ — ci-python.yml (format/lint/test on push and PR), validate-captures.yml (schema gate on completed captures), bootstrap.yml (one-click skeleton generation for non-technical contributors, pushing results to a review branch).
docs/capture-guide.md and docs/bootstrap-guide.md — how to hand-fill a capture file, and how to set up/run/troubleshoot the bootstrap tool respectively.

No tracked issue for this work yet — this PR establishes the tooling baseline itself. (Replace with Closes #N if one gets filed before merge.)

How to test

pip install -e ".[dev]"
ruff format --check . && ruff check . && pytest -q — expect 63 passing, no lint/format issues
capture-validate --dir docs --schema schema/provider-schema-v2.3.json — confirms the reference example (hexanode-example.json) validates cleanly
With a service-account key configured (GOOGLE_APPLICATION_CREDENTIALS) and both tracking sheets shared to it: capture-bootstrap — confirms live-sheet generation produces skeletons with a ~5-item fill-out checklist per provider (not 7 — currency and product lines are the only genuine manual items)
From the Forgejo UI: Actions → "Bootstrap provider files" → Run workflow — confirms the one-click path works for a non-technical user with zero local setup

Checklist

[x ] Tests added or updated
[x ] Docs updated if behaviour changed
[x ] No unrelated changes included
[x ] PR is focused on one thing

Release notes

Adds the initial provider capture schema (v2.3) and tooling — sheet-driven skeleton generation, schema validation gates, and setup docs — for the Provider Offering Dataset.

## What changed and why  Here's the filled template — since there's no tracked issue number for this work, I left that section noting it, but swap in a real number if one exists. What changed and why Introduces the initial capture tooling for the Provider Offering Dataset: the JSON Schema (v2.3) providers must conform to, a bootstrap pipeline that generates skeleton capture files from the project's tracking spreadsheets, and the supporting docs and CI gates. Specifically: Schema v2.3 (schema/provider-schema-v2.3.json) — discrete and parametric product lines, all three pricing-rate models (linear/piecewise/tiered) including irregular step ranges, ratio/table couplings, add-ons, promotions, cart and interval-discount witnesses, and every feature block. Enforces mode separation, the absent/none/null convention, and a bidirectional currency rule (USD ⇔ null fx_conversion_id). Validated against a reference file (docs/hexanode-example.json). capture_tools.bootstrap — reads DATA_Providers, REF_Sources, and FORM_Retrieval_Responses (joined on Source_Id) via the Sheets API and generates one PRV-NNNN.wip.json skeleton per provider, prefilled with everything the sheets know and deliberately-invalid placeholders for everything they don't — so the schema-validation report is a per-provider fill-out checklist. capture_tools.validate — the .wip.json / .json promotion contract: skeletons are exempt from schema validation, completed files (renamed, no .wip) are enforced strictly. This is what the CI gate checks on every push. utils.sheets_api / utils.drive_api — general-purpose Google API helper libraries (official google-api-python-client, ADC auth), shared with the future reviews-analysis work. .forgejo/workflows/ — ci-python.yml (format/lint/test on push and PR), validate-captures.yml (schema gate on completed captures), bootstrap.yml (one-click skeleton generation for non-technical contributors, pushing results to a review branch). docs/capture-guide.md and docs/bootstrap-guide.md — how to hand-fill a capture file, and how to set up/run/troubleshoot the bootstrap tool respectively. ## Related issues  No tracked issue for this work yet — this PR establishes the tooling baseline itself. (Replace with Closes #N if one gets filed before merge.) ## How to test  1. pip install -e ".[dev]" 2. ruff format --check . && ruff check . && pytest -q — expect 63 passing, no lint/format issues 3. capture-validate --dir docs --schema schema/provider-schema-v2.3.json — confirms the reference example (hexanode-example.json) validates cleanly 4. With a service-account key configured (GOOGLE_APPLICATION_CREDENTIALS) and both tracking sheets shared to it: capture-bootstrap — confirms live-sheet generation produces skeletons with a ~5-item fill-out checklist per provider (not 7 — currency and product lines are the only genuine manual items) 5. From the Forgejo UI: Actions → "Bootstrap provider files" → Run workflow — confirms the one-click path works for a non-technical user with zero local setup ## Checklist - [x ] Tests added or updated - [x ] Docs updated if behaviour changed - [x ] No unrelated changes included - [x ] PR is focused on one thing ## Release notes  Adds the initial provider capture schema (v2.3) and tooling — sheet-driven skeleton generation, schema validation gates, and setup docs — for the Provider Offering Dataset.

Xeplosion added 16 commits

2026-08-26 23:55:23 +00:00