Weekly Update

Date: 2026-06-22 To: Isaac Botkin

Summary

The apps can now report their own crashes. If any app — iPhone, Android, the web app, or the admin dashboard — hits a problem, the team is notified with enough detail to fix it, with personal information stripped out before it ever leaves the device; it’s proven out on our staging environment, and it settles the crash-reporting question that’s been open for a few weeks. We also kept building out match-style scoring — squad scoring and live standings for multi-stage courses — though that work is still moving through testing. The hit-factor feature got more attention as well, including the migration that carries roughly 26,000 legacy records across. Behind the scenes we stood up our own build machines, which makes our automated checks faster and cheaper, and we ran a thorough pre-flight audit of the data migration ahead of moving the 70,000 existing users — the bulk of the data checks out, but the audit deliberately surfaced a set of smaller, lower-volume gaps that still need fixing, plus a handful of product decisions to settle before we throw the switch (details below). The system is now backed by 971 automated tests, all passing — up from 831 the week prior.

Our first error report landing in Sentry Cloud — a synthetic test crash sent through the staging → throwaway-Cloud validation (note the staging environment and the scrubbed [email]). The production cutover to Cloud is still ahead — see This Week.

Why it matters — for beta, and especially the cutover: This is the safety net that surfaces a failure on a real user’s device the moment it happens — exactly the kind our in-house tests can’t see (this week’s Android startup crash compiled cleanly and only showed up on a real phone). It matters for beta, where testers run devices and setups we can’t reproduce in-house. And it’s critical for the move off the old Flutter app: when we bring the 70,000 existing users onto the new app, this is how we catch anything that breaks for them right away — instead of flying blind through the single riskiest moment of the project.

Environments

Staging

Production

Beta Program & Discovery

  • The pipeline held at five training organizations this week — no new orgs, but engagement deepened and the Patriot Academy channel opened for real: the welcome to Jesse Nelson went out (the first of roughly four instructors expected there), and the follow-ups drafted last week reached all four active orgs. Two replied with new substance — Cry Havoc and GOAT — while XMTG has gone quiet as James Ferguson travels.
  • Signal this week:
    • The acquisition pain got sharper. Neil (Cry Havoc) — one of last week’s three who named “getting students” as the core pain — pinned it down: scheduling a class is easy; filling it is the hard part. He markets through a website, Instagram, and free USCCA/NRA class listings, with no paid advertising. That refines the thesis into a specific bottleneck — demand generation, not calendar work — and keeps last week’s open scope question live: should RDP help with student acquisition, or is that deliberately out of bounds?
    • The deliberate non-tracker, confirmed harder. Last week Neil told us he doesn’t track between sessions; this week he went further — even advancement is a subjective call (range time watching for safety and fundamentals, sometimes a certificate from a recognized school), never a scored qualification. A standing reminder that the measurable-progress hook lands unevenly across instructor types, and barely at all for the solo pass/fail teacher.
    • A net-new feature idea and a real bug, from GOAT. Jeremiah described his “Thunderdome” as a live head-to-head duel between two shooters — distinct from the legacy multi-shooter drill (run sequentially, ranked by hit factor) and net-new for us. He also caught a concrete bug — the app rejects an email address with a trailing space — for us to fix.
  • Status by org:
    • Code Red Solutions (Ann Marie Suter — also an A Girl & A Gun chapter lead) — follow-up sent 6/17 asking how she organizes her per-student videos and whether students ever see them; awaiting her reply
    • Cry Havoc Training (Neil Meyers) — replied 6/22; a warm wind-down (he’s a deliberate non-tracker), graceful close drafted. Sharpened the filling-classes pain and the subjective-advancement point
    • Go Outside And Train (GOAT) (Jeremiah) — live and engaged; the Thunderdome duel idea plus the email bug; our reply went out 6/17, ball back with him
    • XMTG Firearms Training Academy (James Ferguson, CEO/Director of Training — former PD Range Master) — follow-up sent 6/17; quiet while traveling, may be slow to respond
    • Patriot Academy (Jesse Nelson) — welcome sent 6/17; awaiting his first reply; one of roughly four Patriot Academy testers incoming

Last Week

  • Got crash and error reporting working end-to-end across all four apps (iPhone, Android, web app, dashboard) and validated on staging — when an app fails, the team gets a detailed report with personal data scrubbed on the device before it’s sent; production turn-on is still ahead. This resolves the Analytics & Crash Reporting decision that’s carried on the list for three weeks
  • Kept building out match-style scoring — squad scoring (one range officer entering a whole squad in a single pass) and live standings for multi-stage course runs — now working through testing
  • Put more work into the hit-factor feature — a calculator with a saved log in the web app, and the one-to-one migration to carry roughly 26,000 legacy records across at cutover
  • Stood up our own continuous-integration build machines (self-hosted runners) — faster, lower-cost automated checks, with deployments and releases now running on them
  • Ran a source-exhaustive completeness audit of the data migration ahead of cutover — confirmed the large tables reconcile (user counts, hit-factor counts, scoring fields), and deliberately surfaced a long tail of smaller gaps plus ten product decisions to settle before the one-shot move of the 70,000 existing users
  • Added a forward-compatibility safeguard so an older app shows a graceful message instead of breaking when the server is ahead of it
  • Tracked down and fixed a stretch where our deployment pipeline had quietly broken — several separate problems that each slipped past the normal checks because they only surfaced at deploy time, leaving the staging server stale — and added new gates (rebuilding the full server package and compiling the web apps in release mode on every change) so that class of failure is caught before merge
  • Built a 2D drill-builder demo in the playground — laying out a drill on a 2D range using the same catalog icons as the existing app, across mobile and desktop layouts
  • The system is now backed by 971 automated tests, all passing — up from 831 — including repairing a migration test that had quietly gone stale when the hit-factor work landed

This Week

  • Work through the ten migration decisions the audit surfaced (achievements, stage layouts, gear loadouts, embedded images, challenge-status, admin re-grant, email-verification continuity) and land the gap fixes plus self-defending reconciliation checks — all before cutover
  • Re-run the full migration dry-run from a fresh build against the real exported dataset and confirm it comes back clean
  • Flip production crash reporting over to Sentry Cloud — staging stays on our own in-house monitoring as the white-label reference; this waits on the iOS symbol-upload step so crash reports stay readable
  • Continue wiring the deployment pipeline for beta iOS and Android builds, and add a startup smoke-test in CI so a launch-time crash can’t slip through unseen
  • Continue styling and polishing the Dashboard and PWA ahead of opening them to beta users
  • Full audit write-up: 2026-06-22-migration-completeness-audit — every gap, its disposition, and the ten decisions owed

Changelog

New since the 2026-06-15 update.

Client Crash & Error Telemetry — across all four apps, staging-validated (resolves the Analytics & Crash Reporting decision)

  • One crash/error pipeline for iOS, Android, PWA, and dashboard — every client reports through our own server, never to a third party directly, so no tracking credentials ever ship inside an app
  • iOS (sentry-cocoa) and Android (sentry-android) crash + error capture, with the signed-in user attributed on staging — which on iOS took a device-specific fix, since the standard way of tagging the user works in the simulator but silently fails on a real iPhone
  • Because no off-the-shelf crash reporter exists for our web apps’ technology (Rust compiled to WebAssembly), we built one by hand — it captures the failure, scrubs personal data on the device, and sends the report even as the app is shutting down, sharing the same scrubbing code as the phone apps and server
  • A hard-won catch on Android: the crash reporter tried to initialize before our own app and crashed it on every launch — invisible to our automated checks because the code compiled cleanly, caught only by running on a real device. We fixed the startup order, and it’s the concrete reason for the device launch-test in This Week
  • Personal-data scrubbing on both ends (device and server) — tokens, headers, email, and IP stripped; no session replay, no request bodies
  • License-verification events ride the same pipeline (tagged, non-fatal) so instance-licensing activity will be visible during the migration
  • Phase 2 — faceted monitoring dashboard (Grafana/Loki) splitting crashes from verifications, per environment and license
  • Phase 3 — production forwarding to Sentry Cloud built and validated against a throwaway project on staging; crash-only and scrubbed. Production go-live (real credential + release) and the iOS symbol-upload step are the remaining gates
  • Follow-ups filed: iOS/Android symbol upload (readable crash locations) and a deeper web-app call-stack option

Match Scoring — Course Runs & Squad Scoring

  • Squad scoring entry — one range officer scores N shooters in a single batch, with a reusable score-input component shared across flows
  • Course-run grouping in the data model (course-run + course identifiers on scores), a live standings aggregator, and a deterministic run tag with a reconciliation check
  • Course-run UI in the web app — pick a course, run it, watch the live standing; squad course runs go per-shooter → per-stage grid → leaderboard
  • Course listing now defaults to the caller’s own organization; scoring fields threaded through the native call sites
  • Resolved the open course-scoring decision (#1) — folded into the MVP via course-run grouping
  • These flows are built and now validating on staging; the dashboard squad-entry surface remains an open question

USPSA Hit-Factor Calculator

  • Hit-factor calculator with a saved log in the web app
  • New hit-factor data model and service (caller-scoped create/read), plus the protobuf definitions
  • Built the one-to-one migration for ~26,000 legacy hit-factor records (preserved as recorded, not recomputed; runs at cutover)
  • Scoring correctness fix — a USPSA miss now scores −10 per official rules, aligned across the core and the seed data
  • Resolved the open hit-factor decision (#3) — calculator + log, with the native version moved to the mobile track

Self-Hosted CI & Release Pipeline

  • Stood up our own Linux build machines and moved CI, deployments, and releases onto them — this also brought our automated checks back online after they’d hit GitHub’s hosted-build spending cap (self-hosted minutes aren’t metered), on top of the speed and cost win
  • Found and fixed a stretch where deployments had quietly broken — the regular checks stayed green because the failures only happened at deploy time. Untangled a stack of them: a build artifact landing where the deploy couldn’t find it (so server deploys had been failing silently and staging went stale), the code-generation step misfiring inside the release image, an outdated system library that blocked the web build, and reused scratch space on the new machines that broke every docs publish
  • Closed that whole “passes the checks, breaks at deploy” class going forward — the server package is now rebuilt and the web apps compiled in release mode on every change, so these fail before merge instead of after
  • Safe weekly cleanup of old container images — the off-the-shelf tools would have deleted image layers still in use, so this walks the image graph and removes only truly-unreferenced versions

Data Migration — Completeness Audit & Cutover Prep

  • Source-exhaustive completeness audit (14-finder, adversarially verified): 188 elements migrated, 78 dropped-with-reason, 82 gaps (7 high-severity) — the large tables reconcile; the risk is a long tail of silent drops, now ledgered with dispositions and ten decisions owed
  • Corrected four factually-wrong claims in the migration spec that the audit caught
  • Ongoing media changes in the old app can now carry over through sync to the new system (coexistence media forwarding)
  • Sync outbox cleanup and a guarded pending-media reaper; a sync-status command for coexistence health
  • An asserted local migration dry-run rehearsal command (pre-cutover)
  • Legacy hit-factor records folded into the migration; the migration now carries legacy no-shoot counts into scores — and caught that this count is stored under two different field names, where reading only the one the earlier audit had seen would have silently dropped roughly three-quarters of those records
  • Revised the staging → cutover plan — all data-placing runs in-cluster, dual-outbox recorded

Web Client Forward-Compatibility

  • When the web client meets a server that’s ahead of it, it shows a clear, graceful message instead of breaking (forward-compatibility safeguard)
  • Proto JSON serialization for the shared types, with a parity + forward-compatibility gate in CI
  • Moved protobuf code generation onto our own machines — removing a dependency on an outside service whose rate limits had been intermittently blocking the whole team’s builds, with the generated output verified byte-identical

Reliability & Static Hosting

  • Tracked down a stale-cache bug that could strand users on a broken or outdated version of a web app after a deploy — the app checks each piece of its code against a checksum before running it, and a stale cached piece no longer matched, so the app wouldn’t start. The same root cause was hiding in four separate caching layers (the web server, the CDN edge, the offline service worker, and even the updater meant to replace it — which was caching the very fix), and we closed it at each one; one layer is on a conservative stopgap with a low-priority cleanup tracked to optimize later
  • Content-aware protobuf code generation so raw builds and the IDE regenerate correctly

Playground — 2D Drill Builder

  • New 2D drill builder with a drill-scene schema, matching the legacy app’s catalog glyphs
  • Mobile/desktop view toggle and touch support (pointer events)
  • A discipline-first mobile UI skeleton

Tests & Docs

  • 971 automated tests, all passing (up from 831); repaired a stale migration ordering test exposed by the hit-factor work
  • Roadmap updated throughout — decisions resolved (course-scoring, hit-factor, crash reporting), parity and audit items filed, follow-ups recorded