Architecture Decision Records¶
An ADR records one significant decision: the context, what we decided, the alternatives, and the consequences. ADRs are never edited after acceptance; a new ADR supersedes an old one instead.
Format: Michael Nygard's ADR template.
| # | Decision | Status |
|---|---|---|
| 0001 | Python 3.12+, uv, src layout | Accepted |
| 0002 | Apache License 2.0 | Accepted |
| 0003 | Branches, PRs, squash merges, Conventional Commits | Accepted |
| 0004 | Docs with Material for MkDocs, with a path to Zensical | Accepted |
| 0005 | Versioning and releases with release-please | Accepted |
| 0006 | Retirement registry as YAML + pydantic schema | Accepted |
| 0007 | Keep rescheduled retirements as superseded plans | Accepted |
| 0008 | Research code lives in research/, separate from the product |
Accepted |
| 0009 | Mine with one search per model ID and split windows at the cap | Accepted, amended by 0010 |
| 0010 | Search hugely popular model IDs together with migration words | Superseded by 0011 |
| 0011 | Search hugely popular model IDs only from shortly before the deprecation notice | Accepted |
| 0012 | Label migration commits with one flag per rule, and keep AI-assisted commits separate | Accepted |
| 0013 | Check the migration filter on a blind, stratified sample of 300 commits | Accepted |
| 0014 | Filter rules v2, a migration_away label, and a fresh sample to measure them |
Accepted |
| 0015 | Measure rules v2 on a fresh, blind sample of 200 commits | Accepted |
| 0016 | Place each migration on its model's retirement timeline | Accepted |
| 0017 | Measure how late projects migrate, and what they migrate to | Accepted |
| 0018 | Check what migration commits change on a random sample of 400 | Accepted |
| 0019 | Build every number, table and figure in the paper with one script | Accepted |
| 0020 | Publish the dataset on Zenodo as one versioned zip, without commit messages | Accepted |
| 0021 | Test the timing results against the objections a reviewer would raise | Accepted |
| 0022 | Write Paper 1 in LaTeX in this repository, with the Springer Nature template | Accepted |
| 0023 | The scanner finds model IDs as text first, and parses code later | Accepted |
| 0024 | Severity levels, cloud forms of model IDs, and a CI exit code | Accepted |
| 0025 | JSON and SARIF reports, with UNKNOWN IDs left out of SARIF | Accepted |
| 0026 | A composite GitHub Action in this repository | Accepted |
| 0027 | Give Python findings their context with tree-sitter | Accepted |
| 0028 | TypeScript and JavaScript context, and option objects as call arguments | Accepted |
| 0029 | Evaluate the scanner on snapshots taken just before real migrations | Accepted |
| 0030 | Score the scanner's precision by group, weighted by group size | Accepted |
| 0031 | Mark IDs that only describe a model, and let users skip test files | Accepted |
| 0032 | Re-score the same labels after a scanner change | Accepted |
| 0033 | Replay the scanner on the snapshots before each migration | Accepted |
| 0034 | Say why each flagged project was warned, and re-check labels the sheet cut off | Accepted |
| 0035 | Publish each release to PyPI from the release workflow, with trusted publishing | Accepted |
| 0036 | The runner calls OpenAI-compatible endpoints with httpx, not LiteLLM, and uses only free models | Accepted |
| 0037 | Cache model answers in one SQLite file, keyed by a hash of the whole request | Accepted |
| 0038 | Limit spending in tokens and requests, not dollars, and count cached answers as savings | Accepted |
| 0039 | Pin local models to their Ollama digest, and let callers turn thinking off | Accepted |
| 0040 | Add three free cloud tiers, paced under their per-minute limits, and warn before prompts leave the machine | Accepted |
| 0041 | Decide each test case from pass counts with a continuity-corrected Newcombe interval and an equivalence margin | Accepted |
| 0042 | Stop asking as soon as the verdict is clear, with test-by-betting wealths that are safe to check after every answer | Accepted |
| 0043 | Control the false discovery rate across a suite with e-BH on the betting wealths | Accepted |
| 0044 | Check the stats engine with a seeded simulation study and property tests, against a single-run diff and fixed-k baselines | Accepted |
| 0045 | Accept scores between 0 and 1, and optionally stop test cases that look fine | Accepted |
| 0046 | Test the stats engine on real local models with computed answers, a null pair and injected regressions | Accepted |
| 0047 | Pick fixed k + BH or sequential + e-BH by budget, and report regressions per test case | Accepted |