Cruiser Aviation on AWS, Part 4: The AI Pipeline and the Rest of the Goodies

This is the final part of a four-part series on rebuilding the Cruiser Aviation platform on AWS. Parts 1 - 3 covered the decision, the platform architecture, and WhatsApp self-service. Here we get into the AI document pipeline, a self-auditing flight logbook, the ops war stories that kept things honest, and a reflection on what it actually looks like to build with AI agents.

---

AI Document Pipeline: Licenses, Medicals, Radio Certificates

Pilots photograph their EASA license or upload a PDF. Claude on Bedrock (Opus, InvokeModel, with forced-tool structured output so every answer is machine-readable) extracts every field with per-field confidence and verbatim source text. The pilot lands on a pre-filled form next to the scan, with uncertain fields flagged amber, and confirms or corrects.

Wrong documents are rejected rather than hallucinated into fields. Blurry photos get a named retake prompt - e.g. "the ratings tables aren't visible - retake side 2." If Bedrock is down, the wizard silently degrades to manual entry.

The accuracy engineering is what makes this worth writing about.

Tiling beat downsampling

Claude vision downsamples to roughly 1568 px on the long edge, which lost faint 90°-rotated endorsement-table rows. The fix: large images are sent as a full-page overview plus overlapping ~2000 px native-resolution tiles. PDFs are rasterized at 200 DPI using poppler, installed in the app's Docker image for exactly this purpose.

Tiling took rating recall to 100% on the production eval set - including a Night rating row that both the model and the original human data entry had missed. Each license costs roughly 25k input tokens. That's spent deliberately: licenses renew every ~2 years, volume is low, and accuracy wins.

An adversarial second pass

A second model call re-reads the scan trying to disprove the first reading, character by character - dates, diacritic-heavy names, and classic OCR confusions like 1/7, 0/8, 3/8, and 5/6. Disagreements become amber flags.

This pass runs for backfills. The interactive path skips it because the pilot's own review is the human verification step, and the extraction has to fit within CloudFront's origin timeout.

The eval harness shares bytes with production

etl/license-extraction-eval.js loads the app's own prompt.json and schema.json off disk, runs extraction-vs-DB diffs over every production scan, and has ground-truth regression assertions baked in.

First run surfaced 20 suspect DB values across 13 licenses: a wrong rating expiry (2025 in the DB, 30.04.2027 on the document), a transposed digit in a license number, duplicate and missing ratings - and one license uploaded into the wrong member's account, caught by the holder-name identity cross-check. Corrections went through a reviewed, hand-approved apply mode.

One engine, multiple document types

The same DocumentTypeSpec pipeline powers medical and radio certificate wizards. Each document type gets a spec object; the extraction engine is shared.

Domain rules are encoded, not prompted around: LEVEL VI language proficiency is valid for life, so a stray expiry would wrongly invalidate a license. Name and date of birth are read for cross-checking but never stored.

Historical document recovery

Alongside the interactive wizard, a 5-stage Gmail harvest recovered historical documents from the school mailbox:

1. IMAP sweep of the mailbox
2. Deterministic, no-AI triage by content-hash and keywords
3. Sonnet classification
4. Human-reviewable proposal
5. Opus extraction with idempotent import for 96 matched users

Policy is encoded in the importer: expired medicals attach as files only; identity mismatches are never auto-imported. A bilingual SES reminder campaign ran alongside it, ledgered so re-runs never double-send.

---

Flight Tracking and the Self-Auditing Logbook

A GPS tracker vendor (AviTracer) feeds a live fleet map - MapLibre, grey positron cartography with terrain relief and an openflightmaps VFR overlay, 10-second polling, hidden-tab aware - on every member dashboard. Roughly 2,800 historical trips were ingested per environment back to February 2024, downloaded from the vendor once on staging and copied cross-account to production. Around 82% auto-matched to flight logs, with an admin console handling the rest.

The follow-up feature is the kind of thing worth ending the series on.

The flight-log list walks each aircraft's Hobbs meter chronologically and injects synthetic rows wherever the numbers don't line up:

• An amber "Flight missing" row for any gap ≥ 0.2 h - an unlogged flight.
• A red row where the meter runs backwards - a data-entry error.

A missing-flight row is matched against the unmatched GPS trip in its time window and rendered like a real flight. A one-click "Create from tracked flight" pre-fills the log form so the gap closes and the trip auto-links on save.

The logbook audits itself against physics.

---

Ops War Stories

The ip-10-0-x-x:3000 dead-end

Any page refresh with an expired token occasionally stranded browsers on an unresolvable internal hostname. Root cause: a Node route handler behind CloudFront/ALB derives request.nextUrl from the container's host header, and the auth-refresh redirect was built from it. Fix: route-handler redirects always use the canonical APP_BASE_URL. This same bug had also been the "unexplained post-checkout dead-end" weeks earlier.

"Invite user" never worked

The ECS task role had no cognito-idp permissions. AdminCreateUser got AccessDenied and the app silently filed "invite failed." The same permissions audit found AdminUserGlobalSignOut silently failing too - meaning role and status changes were never actually revoking sessions.

The MapLibre pin

maplibre-gl 5.9+/6.x never requested vector tiles on initial load (upstream regression #6649) - style and DEM loaded, roads and labels didn't. Pinned to 5.8.0 with the issue number in the commit message.

Deploy settling

A live test 60 seconds after CI-green hit still-draining ECS tasks. The runbook now says: wait for the rollout to settle before diagnosing "bugs."

Meta message ordering

Rapid separate sends can arrive reordered. That's why booking confirmations and payment links share one message.

---

How It Was Actually Built

The repo supports an honest "built with AI agents" angle.

194 commits in 11 days is the visible output of a human + Claude Code loop. The decisions log records the working agreement:

"Claude runs deploys. Julian granted standing allow-rules for cdk deploy (both envs), gh workflow run, and AWS secrets/Data-API writes - deploy handoffs are over; cdk diff change-set review remains mandatory before anything touching the Cognito pool."

• DECISIONS.md, 2026-07-29

app/AGENTS.md exists solely to warn coding agents that Next.js 16 post-dates their training data and to read the vendored docs first.

The product itself uses Claude three ways: conversation over WhatsApp, forced-tool availability parsing, and vision extraction - each with a different trust posture. There's a pleasing symmetry here: the same skepticism applied to the building agent (mandatory cdk diff) is applied to the runtime models (deterministic transaction paths, echo-back confirmation, adversarial second reads, human review screens).