Engineering challenges¶
Problems that took real effort to solve, written up so the lesson is not lost. One file per challenge: NNN-short-title.md.
Template¶
# NNN: Short title
- **Date:** YYYY-MM-DD
- **Area:** scanner / runner / stats / CI / ...
## Symptom
What we saw going wrong.
## Investigation
What we checked, in order, and what each step ruled out.
## Root cause
The actual reason.
## Options considered
The possible fixes and their trade-offs.
## Fix
What we did (link the PR).
## Lesson
What we'd do differently from now on.
Challenges¶
| # | Challenge | Area |
|---|---|---|
| 001 | Release PRs would never pass CI | CI / releases |
| 002 | One model, two shutdown dates | Registry data |
| 003 | GitHub's commit search is fuzzy | Research mining |
| 004 | Popular model IDs would take days to mine | Research mining |
| 005 | The hand labels needed checking too | Research validation |
| 006 | A model switch is not always a migration away | Research validation |