Manual workflow to push local provider capture files to the shared Drive folder. #2

Merged

Xeplosion merged 23 commits from dev into main 2026-08-27 04:20:54 +00:00

Conversation 0 Commits 23 Files changed 135 +10991 -445

Xeplosion commented 2026-08-27 04:20:40 +00:00

Owner

Copy link

What changed and why

Adds a workflow to push local provider capture files (providers/*.json,
*.wip.json) out to the Drive folder that tools/capture-form/index.html
reads from and writes to. Previously, getting newly-bootstrapped skeletons
into that folder required uploading them by hand.

• src/capture_tools/push_drive.py (new capture-push-drive CLI) — for
  each local provider file, uploads it if no same-named file exists in
  Drive yet, and skips it if one does, unless --overwrite is passed.
• src/utils/drive_api.py — added list_folder_files() and
  update_file_content(), using the full drive scope rather than
  drive.file, since the service account isn't the one that created these
  files (a researcher's own OAuth session in the capture-form is).
• .forgejo/workflows/push-drive.yml — manual workflow_dispatch job,
  mirroring bootstrap.yml's pattern, with overwrite and dry_run
  boolean inputs. Reuses the existing GCP_SA_KEY secret.
• tests/test_push_drive.py — 8 tests covering create/skip/overwrite/dry-run
  behavior and partial-failure handling, with the Drive API mocked out.
• pyproject.toml / src/capture_tools/__init__.py — register the new
  capture-push-drive console script.
• README.md / tools/capture-form/README.md — documented the new
  workflow and pointed the old "upload by hand" note at it instead.

The skip-by-default behavior is deliberate: a file already in Drive may
hold in-progress edits or a promotion made directly in the browser tool
that haven't been pulled back into git yet, and blindly overwriting it
would destroy that work.

Related issues

Closes #

How to test

1. Ensure the service account (GCP_SA_KEY) is shared as Editor on the
   target Drive folder — this is a separate share from its read access to
   the tracking sheet.
2. Run pytest tests/test_push_drive.py — all 8 should pass without any
   network access (the Drive API is mocked).
3. From Actions, run Push provider files to Drive with dry_run
   checked and confirm the log lists CREATE/SKIP decisions without
   error.
4. Run it again for real (both inputs unchecked) and confirm the new
   files appear in the Drive folder, and pre-existing ones are untouched.

Checklist

• Tests added or updated
• Docs updated if behaviour changed
• PR is focused on one thing
• No unrelated changes included

Release notes

Add a manual workflow to push local provider capture files to the shared Drive folder.

## What changed and why Adds a workflow to push local provider capture files (`providers/*.json`, `*.wip.json`) out to the Drive folder that `tools/capture-form/index.html` reads from and writes to. Previously, getting newly-bootstrapped skeletons into that folder required uploading them by hand. - `src/capture_tools/push_drive.py` (new `capture-push-drive` CLI) — for each local provider file, uploads it if no same-named file exists in Drive yet, and skips it if one does, unless `--overwrite` is passed. - `src/utils/drive_api.py` — added `list_folder_files()` and `update_file_content()`, using the full `drive` scope rather than `drive.file`, since the service account isn't the one that created these files (a researcher's own OAuth session in the capture-form is). - `.forgejo/workflows/push-drive.yml` — manual `workflow_dispatch` job, mirroring `bootstrap.yml`'s pattern, with `overwrite` and `dry_run` boolean inputs. Reuses the existing `GCP_SA_KEY` secret. - `tests/test_push_drive.py` — 8 tests covering create/skip/overwrite/dry-run behavior and partial-failure handling, with the Drive API mocked out. - `pyproject.toml` / `src/capture_tools/__init__.py` — register the new `capture-push-drive` console script. - `README.md` / `tools/capture-form/README.md` — documented the new workflow and pointed the old "upload by hand" note at it instead. The skip-by-default behavior is deliberate: a file already in Drive may hold in-progress edits or a promotion made directly in the browser tool that haven't been pulled back into git yet, and blindly overwriting it would destroy that work. ## Related issues Closes # ## How to test 1. Ensure the service account (`GCP_SA_KEY`) is shared as **Editor** on the target Drive folder — this is a separate share from its read access to the tracking sheet. 2. Run `pytest tests/test_push_drive.py` — all 8 should pass without any network access (the Drive API is mocked). 3. From Actions, run **Push provider files to Drive** with `dry_run` checked and confirm the log lists `CREATE`/`SKIP` decisions without error. 4. Run it again for real (both inputs unchecked) and confirm the new files appear in the Drive folder, and pre-existing ones are untouched. ## Checklist - [x] Tests added or updated - [x] Docs updated if behaviour changed - [x] PR is focused on one thing - [ ] No unrelated changes included ## Release notes Add a manual workflow to push local provider capture files to the shared Drive folder.

Xeplosion added 23 commits 2026-08-27 04:20:40 +00:00

feature: added provider data folder. d172461509

feat: added increased functionality for the Goolge API utilities. 3b792fa90d

feat: rebuild bootstrap around DATA_Providers and REF_Sources sheets. 858d24a68f

test: rewrite bootstrap tests for the two-sheet flow. 70efbba8df

ci: simplify bootstrap workflow env now that sheet IDs default in-code. 8712603c2b

fix: typo in module name. 4f172ffbba

fix: updated bootstrap-capture to use 'PRV-NNN' instead of 'PRV-NNNN'. 39d43dd7ac

fix: incorrect column names for DATA_Providers sheet. 5a4fc1c7a6

fix: accidentally deleted bootstrap files. 2d8aaf8a14

feat: source researcher and capture_datetime from REF_Sources and FORM_Retrieval_Responses. eb13857a6b

fix: source provider.name from Provider_Name column instead of leaving it a placeholder. fce71765dc

docs: add bootstrap tool guide. a228a11693

fix: added missing name update to the bootstrap Python file. de5c52d984

fix: added fixed wip files with provider.name field. 0de5daaf23

chore: normalize line endings to LF and add .gitattributes. 58aa3c8e63

ci: add format/lint/test check on push and pull request. 30899de525

Merge pull request 'feature/initial-capture-tooling' (#1) from feature/initial-capture-tooling into dev ... 748da5ed49

Reviewed-on: #1

ci: tests in ci-python workflow now look for runner with label 'ubuntu-latest'. 69a777e5ad

feat: add browser-based provider capture form (Drive-backed, schema v2.3). 44cfcd9ac0

fix: correct OAuth client ID whitespace, surface popup errors, and restore token callback after silent reauth. 5a8ad9a28b

feat: add push-drive workflow to sync local provider files to the shared Drive folder. 7976e7b5ea

feat: add push-drive workflow to sync local provider files to the shared Drive folder. f2cf342b31

reformat: ran ruff format. 07e11bdba1

Xeplosion merged commit ab4d0d9358 into main 2026-08-27 04:20:54 +00:00

Xeplosion referenced this pull request from a commit 2026-08-27 04:20:56 +00:00

Merge pull request 'Manual workflow to push local provider capture files to the shared Drive folder.' (#2) from dev into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

  

Compat/Breaking

Introduces a breaking change — not backward compatible

  

Compat/Deprecation

Deprecates something that will be removed in a future release

  

Kind

Bug

Something is not working

  

Kind

Chore

Maintenance, dependencies, tooling, tech debt

  

Kind

Documentation

Documentation changes

  

Kind

Enhancement

Improve existing functionality

  

Kind

Feature

New functionality

  

Kind

Investigation

Research or spike — output is findings, not necessarily a PR

  

Kind

Question

Question or discussion — no code change intended

  

Kind

Security

Security-related issue or fix

  

Kind

Testing

Related to tests or test infrastructure

  

Meta/Good First Issue

Good entry point for new contributors

  

Meta/Help Wanted

Extra attention or outside help welcome

  

Meta/Stale

No activity for 60+ days — will be closed if no response

  

Priority

Critical

Drop everything — production is broken or data is at risk

  

Priority

High

Next up — significant impact, should ship soon

  

Priority

Low

Nice to have — pick up when bandwidth allows

  

Priority

Medium

Important but not blocking anything

  

Reviewed

Confirmed

Issue is valid and ready to be worked on

  

Reviewed

Duplicate

Already tracked in another issue

  

Reviewed

Invalid

Cannot reproduce, out of scope, or not a real issue

  

Reviewed

Won't Fix

Acknowledged but will not be addressed

  

Status

Abandoned

Work was started but the contributor stepped away

  

Status

Blocked

Cannot proceed — waiting on something external

  

Status

Need More Info

Waiting on additional information from the reporter

  

Status/Needs Triage

Newly opened — awaiting review, kind, and priority assignment

No labels

Compat/Breaking

Compat/Deprecation

Kind

Bug

Kind

Chore

Kind

Documentation

Kind

Enhancement

Kind

Feature

Kind

Investigation

Kind

Question

Kind

Security

Kind

Testing

Meta/Good First Issue

Meta/Help Wanted

Meta/Stale

Priority

Critical

Priority

High

Priority

Low

Priority

Medium

Reviewed

Confirmed

Reviewed

Duplicate

Reviewed

Invalid

Reviewed

Won't Fix

Status

Abandoned

Status

Blocked

Status

Need More Info

Status/Needs Triage

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

XeplosionGames/minecraft-provider-analysis!2

No description provided.