Less Is More Reliable: Deleting 2,400 Lines and a Whole Class of Bugs

A case study in deleting infrastructure. How one of our company's internal LinkedIn content-automation platforms went from a two-process, Redis-backed job system to a single-runtime, database-as-queue design on Vercel Cron + Next.js 16 - and shipped ~2,400 fewer lines, ~85% fewer scheduled invocations, and an entire class of bugs eliminated.

Summary

• V1 ran a long-lived Railway worker that pulled jobs from Upstash Redis via BullMQ (publishing, RSS ingestion, AI generation, planning, token monitoring). The web app (Next.js 15 on Vercel) enqueued jobs; the worker consumed them.
• A production incident - approved posts silently stuck in QUEUED, never publishing - exposed the core weakness: two processes coordinating through a third system (Redis) is a distributed-systems problem, and it failed in the quiet, hard-to-debug way distributed systems do.
• V2 deletes the worker, Redis, and BullMQ. The Postgres database becomes the queue. Five Vercel Cron routes drain it on a schedule. The app upgraded to Next.js 16 (Turbopack by default, middleware → proxy), and the migration surfaced a dead next-auth dependency that we removed outright.
• Result: fewer moving parts, lower cost, simpler mental model, better observability, and a safe, zero-downtime cutover enabled by idempotent claims.

---

What the Product Does

fara.social schedules and publishes content to LinkedIn (and Instagram) for multiple "projects." It:

• Ingests RSS feeds, dedupes articles, and summarizes them with an LLM in a brand voice.
• Generates original posts from AI themes on a weekly plan.
• Lets a user approve content, which schedules it for publishing.
• Publishes to the platform APIs at the scheduled time, handling OAuth tokens, image uploads, and first comments.
• Monitors token expiry and emails the user before posts start failing.

Every one of those bullets was, in V1, a background job.

---

The V1 Architecture (and Why It Looked Reasonable)

┌───────────────────────┐         enqueue          ┌──────────────────────┐
│  Next.js 15 (Vercel)  │ ──────────────────────► │   Upstash Redis      │
│  - UI + API routes    │                          │   (BullMQ queues)    │
│  - "approve"→enqueue │ ◄─── queue stats ─────  │                      │
└───────────────────────┘                          └──────────┬───────────┘
                                                               │ consume
                                                    ┌──────────▼──────────┐
                                                    │  Railway worker      │
                                                    │  (long-running Node) │
                                                    │  - publisher         │
                                                    │  - rss-fetcher       │
                                                    │  - news-summarizer   │
                                                    │  - ai-generator      │
                                                    │  - ai-planner        │
                                                    │  + 3 polling loops   │
                                                    └──────────────────────┘

Why this is a defensible first design: BullMQ is excellent. It gives you retries with exponential backoff, delayed jobs, concurrency limits, rate limiting, and dead-letter semantics out of the box. If you're coming from a "real" backend, reaching for Redis + a queue is the obvious move. Vercel functions are ephemeral and time-limited, so "put the long work on an always-on worker" feels correct.

The hidden tax, in roughly increasing order of pain:

1. Two runtimes to deploy, secret-sync, and reason about (Vercel + Railway), plus a third managed service (Upstash).
2. Two sources of truth. The DB had Post.status; Redis had job state. They could disagree.
3. In-memory state in a process that restarts. The publisher used an in-memory Set to dedupe posts it had already enqueued. A post that failed and was re-queued stayed in the Set until the worker restarted → permanently stuck. The Set was also lost on restart → double-post risk.
4. Encryption coupling. OAuth tokens were encrypted with a single ENCRYPTION_KEY_BASE64. If the app and worker ever had a different key (or it was rotated), decrypt() failed the AES-GCM auth-tag check and every token became unreadable - silently, as a publish failure.

The Incident

Approved posts piled up in QUEUED and never reached PUBLISHED - and crucially, never reached FAILED either. The web app's view of the queue (/api/worker/status, which read BullMQ counts from Redis) disagreed with the database. Publishing wasn't even being attempted; the posts were claimed into an intermediate state and stranded.

That "no error, just stuck" signature is the fingerprint of a distributed handoff problem: something between claim and execute was dropping, across the process/Redis boundary, in a way no single component logged as a failure.

The fix wasn't a patch. It was a question: do we need any of this machinery?

---

The V2 Insight: Your Database Is Already a Queue

A scheduler's job queue needs three things: durable state, atomic claiming (so two workers don't grab the same job), and idempotency (so a retry doesn't double-execute). Postgres gives you all three with a status column and a conditional UPDATE. You don't need Redis for that - you needed it for BullMQ's ergonomics, not its guarantees.

So V2 makes the database the only queue and uses Vercel Cron as the thing that "turns the crank":

┌─────────────────────────────────────────────────────────────────┐
│  Next.js 16 (Vercel)                                            │
│                                                                 │
│  User action ──► writes a DB row (Post.status = APPROVED /     │
│                  SCHEDULED, or a draft stub)                    │
│                                                                 │
│  Vercel Cron ──► /api/cron/publish        (every 5 min)        │
│              ──► /api/cron/rss            (every 15 min)       │
│              ──► /api/cron/generate-drafts (every 30 min)      │
│              ──► /api/cron/weekly-plan    (weekly)             │
│              ──► /api/cron/token-monitor  (daily)              │
│                       │                                         │
│                       └─► plain functions in src/lib/ ──► DB  │
│                                                  └─► LinkedIn/ │
│                                                      LLM APIs   │
└─────────────────────────────────────────────────────────────────┘

No Redis. No BullMQ. No second process. The same Postgres row that the UI reads is the job.

The State Machine

Post.status moves through:

DRAFT ─► APPROVED ─┐
                    ├─► QUEUED ─► PUBLISHING ─► PUBLISHED
SCHEDULED ──────────┘                    └────► FAILED
        (scheduleAt ≤ now)

Backed by a handful of columns that make the queue work: scheduleAt, lockedAt, enqueuedAt, attemptCount, publishedAt, errorCode, errorMessage.

Atomic Claiming = The Whole Trick

The "don't double-process" guarantee is a single conditional update that only succeeds for the worker that wins the race:

// src/lib/supabase-db.ts
async function claimPost(postId: string, fromStatus: string) {
  const now = new Date().toISOString()
  const { data } = await supabase
    .from('Post')
    .update({ status: 'QUEUED', enqueuedAt: now, lockedAt: now, updatedAt: now })
    .eq('id', postId)
    .eq('status', fromStatus)   // ← only claims if still in the expected state
    .select()
  return data && data.length > 0 ? data[0] : null  // null = someone else got it
}

This is the Postgres equivalent of a compare-and-swap. Two concurrent cron runs (or, during migration, the cron and the old worker) can both try to claim the same post; exactly one UPDATE affects a row, the other gets null and moves on. No locks, no Redis, no double-post.

Draining Due Work

// src/lib/publish/publish-due.ts
export async function publishDuePosts({ limit = 10 } = {}) {
  // 1) Recover anything orphaned mid-flight by a prior timed-out run.
  const reclaimed = await reclaimStuckPosts(15 /* minutes */)

  // 2) Claim due work: APPROVED (publish asap) + SCHEDULED whose time has passed.
  const [approved, overdue] = await Promise.all([
    claimApprovedPosts(limit),
    claimOverduePosts(limit),
  ])

  // 3) Publish each claimed post inline.
  for (const post of [...approved, ...overdue]) {
    await publishOnePost(post.id)
  }
}

reclaimStuckPosts resets any QUEUED/PUBLISHING row whose lockedAt is older than 15 minutes back to APPROVED. That single function replaces BullMQ's stalled-job recovery - if a serverless function times out mid-publish, the next run picks the post back up.

Retries Without BullMQ

BullMQ gave us "5 attempts with exponential backoff." In a stateless cron world, the database row carries the attempt count across runs:

// src/lib/publish/publish-post.ts (error path)
const attempt = (post.attemptCount ?? 0) + 1
// ...
catch (error) {
  const errorCode = classifyPublishError(error)        // DECRYPT_FAILED, EXPIRED_ACCESS_TOKEN, ...
  if (attempt >= MAX_ATTEMPTS) {
    await finalizePostFailed(postId, errorCode, message) // terminal
  } else {
    // hand back to APPROVED so the *next* cron tick retries it
    await supabase.from('Post')
      .update({ status: 'APPROVED', lockedAt: null, errorCode, errorMessage: message })
      .eq('id', postId)
  }
}

The retry "backoff" is simply the cron interval. It's coarser than BullMQ's millisecond backoff, but for a social-posting scheduler that publishes a few times a day, "retry in 5 minutes" is exactly right - and it survives restarts for free because it's in Postgres.

---

Vercel Cron, in Depth

How It Works

You declare schedules in vercel.json. Vercel hits the route on schedule; that's it.

// vercel.json
{
  "functions": {
    "src/app/api/cron/**/route.ts": { "maxDuration": 300 }
  },
  "crons": [
    { "path": "/api/cron/publish",         "schedule": "*/5 * * * *" },
    { "path": "/api/cron/rss",             "schedule": "*/15 * * * *" },
    { "path": "/api/cron/generate-drafts", "schedule": "*/30 * * * *" },
    { "path": "/api/cron/weekly-plan",     "schedule": "0 14 * * 0" },
    { "path": "/api/cron/token-monitor",   "schedule": "0 8 * * *" }
  ]
}

Each route is a thin, authenticated handler over a src/lib function:

// src/app/api/cron/publish/route.ts
export const runtime = "nodejs";
export const dynamic = "force-dynamic";
export const maxDuration = 300;

export async function GET(request: Request) {
  const denied = assertCron(request);            // 401 if the secret is wrong/missing
  if (denied) return denied;
  const summary = await publishDuePosts({ limit: 10 });
  return NextResponse.json({ ok: true, ...summary });
}

Authentication

Vercel automatically attaches Authorization: Bearer ${CRON_SECRET} to cron invocations when you set the CRON_SECRET env var. The guard is ~15 lines:

// src/lib/cron-auth.ts
export function assertCron(request: Request): NextResponse | null {
  const secret = process.env.CRON_SECRET ?? process.env.ADMIN_API_SECRET;
  if (!secret) return json401("not configured");
  const bearer = request.headers.get("authorization")?.replace(/^Bearer\s+/i, "");
  const provided = request.headers.get("x-admin-secret") || bearer || "";
  return provided === secret ? null : json401("Unauthorized");
}

Accepting a manual x-admin-secret too means you can curl any cron route to trigger it on demand - which became the basis for operational tooling.

The Optimization That Matters Most: Interval ≠ Cadence

The single most useful insight from V2: a cron's interval is polling latency, not business frequency. The spacing between posts lives in scheduleAt. The publish cron just notices due rows.

V1's instinct was "publish every minute so things feel instant." But posts are scheduled hours apart, so a 1-minute cron is ~1,439 no-op runs between every real publish. Tuning the schedules to match reality:

That's ~85% fewer invocations with zero user-visible change - pure waste removal. The lesson generalizes: when you move polling to a metered platform, the interval becomes a cost dial. Set it to your latency tolerance, not to "fast."

Honest Constraints of Vercel Cron

• Schedules are static. They live in vercel.json and only change on deploy. There's no runtime API to retune a cron from a settings UI. If you want user-configurable cadence, you put the cadence in your data (scheduleAt spacing), or you let the cron fire at a fixed floor and early-exit based on a DB setting - you don't rewrite the crontab at runtime.
• Sub-daily schedules require Vercel Pro. Hobby runs crons at most once per day. (For Hobby, an external pinger hitting the same authenticated route is a valid escape hatch.)
• Function limits apply. maxDuration caps a run (300s on Pro here), so each cron processes a bounded batch and lets the next tick continue any backlog. No single invocation tries to drain everything.
• Cold starts + no shared memory. Every run is fresh. That's a feature for correctness (no stale in-memory Set) but means anything "remembered" must be in the DB.

None of these bit us, because the DB-as-queue model is designed around bounded, stateless, idempotent runs. That alignment is the point: Vercel Cron rewards a job design that's already a good idea.

---

Next.js 16: What the Upgrade Actually Bought

The framework jump from 15.5 to 16.2 was deliberately a separate PR from the cron work (different files, clean rollback, no confounding a build regression with a publishing one).

Turbopack Is the Default Builder

In 16, next build uses Turbopack with no flag. We already had no custom webpack config, so there was nothing to migrate - builds just got faster (production compiles in ~10 - 20s here). If you do have a webpack config, 16 makes you choose: migrate it, or opt back out with --webpack. The forcing function is healthy.

middleware.ts → proxy.ts

Next renamed the convention to clarify it as a network boundary, and proxy runs on the Node.js runtime (the Edge runtime isn't supported there). For us that was strictly better - the file does async Supabase session refresh and injects project-scoping headers, none of which needed Edge:

// src/proxy.ts  (was src/middleware.ts)
export async function proxy(request: NextRequest) {   // was: export async function middleware
  const { supabase, response } = createClient(request);
  await supabase.auth.getSession();                    // refresh cookies
  // ...validate /{namespace}/{project} and inject x-user-namespace / x-project-slug
  return response;
}

The official codemod (npx @next/codemod@canary upgrade latest) handles the rename and config migration; the rest was a 10-minute audit.

Async Request APIs Are Now Enforced

15 introduced async params/cookies()/headers() with a temporary sync shim; 16 removes the shim. Because the codebase already used await params and await cookies(), this was a no-op for us - but it's the change most likely to break an older App Router app, so it's worth calling out as the first thing to grep for.

React 19.2

Comes along for the ride (View Transitions, useEffectEvent, Activity, and stabilized React Compiler support). We didn't adopt new APIs, but the version bump is part of the 16 story.

The Upgrade Paid for Itself by Deleting a Dependency

Here's the best insight from the 16 work, and it's a process insight, not a code one:

next-auth@4 declares a peer dependency of next ^12–15 - it doesn't list 16 - so npm's strict resolver refused to install. The reflexive fix is .npmrc with legacy-peer-deps=true. We did that to unblock, then asked why next-auth was there at all. It turned out to be dead code: a shim file exporting an authOptions placeholder (empty providers, imported by nothing) and a local getServerSession() that actually wrapped Supabase. The real auth was 100% Supabase the whole time.

So instead of migrating to NextAuth v5, we removed next-auth entirely, which let us delete the .npmrc workaround too (a fresh npm install then resolved cleanly). The framework upgrade's friction was a smoke detector that found a dependency we could just delete.

Insight: a major-version bump is a free dependency audit. Every peer-conflict is the ecosystem asking "do you still need this?" Sometimes the answer is no.

---

The Encryption Foot-Gun (a V1→V2 Reliability Fix)

OAuth access tokens are stored AES-256-GCM encrypted. V1 used a single ENCRYPTION_KEY_BASE64 for both encrypt and decrypt. Rotating or diverging that key (e.g., app vs worker) makes the GCM auth tag fail verification, and every existing token becomes undecryptable - surfacing only as publish failures.

V2 introduces a versioned keyring so new writes can use a new key while old ciphertext still decrypts with its original key:

// src/lib/crypto.ts
export function encrypt(plain: string): { iv: Buffer; data: Buffer; keyVersion: number } { /* active key */ }
export function decrypt(iv: Buffer, blob: Buffer, keyVersion = 1): string { /* key for that version */ }

• keyVersion = NULL on a stored row means "version 1" (the legacy key) - existing rows keep working untouched.
• A new ENCRYPTION_KEYS JSON map + ENCRYPTION_ACTIVE_KEY_VERSION let you introduce a v2 key for new writes and migrate old rows lazily.

This converts "rotating a secret silently bricks production" into a safe, reversible operation - the kind of latent V1 hazard that only a rewrite gives you the excuse to fix.

---

Observability as Code

Without a worker process tailing logs, "why is this post stuck?" needed a first-class answer. We built read-only diagnostic endpoints that turn questions into one-line answers.

A single-article probe returns the article, its linked post's full state, the project's lastPublishedAt (a cron-liveness signal), and a plain-English diagnosis:

GET /api/{ns}/{project}/recover-stuck-posts?articleId=<uuid>
→ {
    "post": { "status": "SCHEDULED", "scheduleAt": "2026-06-17T15:54:36Z", "attemptCount": 0 },
    "lastPublishedAt": "2026-06-07T11:10:45Z",
    "diagnosis": "Post is SCHEDULED for 2026-06-17 (future) — it will publish then.
                  Add &publish=now to this URL to post it within ~1 minute."
  }

That output settled a real "it's broken!" report in seconds: the post wasn't broken, it was #24 in an 8-hour-spaced queue, and the cron was demonstrably alive (lastPublishedAt was minutes old). The same family of endpoints powers a "Recover stuck posts" panel and a ?publish=now one-click override - operational levers that exist precisely because the queue is just rows you can query and nudge with SQL-shaped updates.

Insight: when your queue is your database, your debugger is SELECT. That's a quietly huge ergonomic win over inspecting an opaque Redis-backed queue.

---

Migrating Safely: Idempotent Claims Enable a Zero-Downtime Cutover

Because claiming is an atomic conditional update, the old worker and the new cron could run at the same time without double-posting. The cutover was staged:

1. Ship the cron routes and src/lib functions while the Railway worker still ran. Both drained the same DB; whoever claimed a row first won, the other got null.
2. Set CRON_SECRET, confirm cron publishes (watch lastPublishedAt advance).
3. Flip user actions to drop enqueue* (they already wrote the DB row).
4. Stop the worker; remove Redis.

No maintenance window, no "big bang." The property that made V1 hard to reason about (concurrent claimers) is the same property that made the migration safe - once claiming was correct.

It shipped as a sequence of small PRs (cron refactor → Next 16 → remove next-auth → retune intervals → diagnostics), each independently reviewable and revertible.

---

Benefits and Trade-offs

What Got Better

What You Give Up (Be Honest)

• No millisecond-precise scheduling or backoff. Cron granularity is your latency floor (5 min here). Fine for social posting; wrong for, say, payment retries.
• No high-throughput fan-out. BullMQ shines at thousands of jobs/sec with concurrency controls. DB-as-queue with SELECT ... FOR-style claims is great into the low thousands and simpler, but it's not a stream processor. Match the tool to the volume.
• Static schedules. Cadence changes that aren't expressible in data require a redeploy.
• Per-run cost on a metered platform. Even no-op cron runs cost a little; you trade a fixed Railway/Upstash bill for usage-based function invocations. (For this workload, far cheaper - and you tune it with the interval dial.)
• Bounded batches. Function timeouts mean very large backlogs drain over several ticks rather than in one shot. Usually invisible; occasionally something to design around.

When DB-as-Queue Is the Right Call

✅ Scheduled/periodic work, modest throughput, work that's already DB-centric, a team that values one runtime and one source of truth, deployment on a platform with native cron.

🚫 High-frequency real-time jobs, very high throughput, complex DAG/fan-out orchestration, or sub-second latency requirements - keep a real queue.

For a content scheduler that publishes a handful of times a day per project, the database was the queue all along. V1 paid for infrastructure to avoid noticing that.

---

The Stack, End to End (V2)

• Framework: Next.js 16 (App Router, Turbopack, proxy.ts), React 19.2, TypeScript.
• Data + Auth: Supabase (Postgres + Supabase Auth). Service-role client for cron/server work.
• Background work: Vercel Cron → /api/cron/* → plain src/lib functions. No Redis/BullMQ.
• UI: Tailwind + shadcn/ui, sonner toasts.
• Integrations: LinkedIn & Instagram Graph APIs, an LLM provider (OpenAI-compatible) for summaries/generation, Resend for email.
• Crypto: versioned AES-256-GCM keyring for OAuth tokens.
• Hosting: Vercel (Pro, for minute-level cron). Railway + Upstash decommissioned.

---

Lessons

1. Your database is probably already a queue. A status column + a conditional UPDATE gives you durable, atomic, idempotent claiming. Reach for Redis/BullMQ for throughput and backoff ergonomics, not for the basic guarantees - Postgres has those.
2. Two processes coordinating through a third system is a distributed-systems problem. It will fail quietly. Collapsing to one runtime deletes a category of bugs, not just a bill.
3. Cron interval ≠ business cadence. Put cadence in your data; set the interval to your latency tolerance. On a metered platform, the interval is a cost dial - don't set it to "fast" by reflex.
4. Idempotent claims are what make migrations safe. If claiming is correct, the old and new systems can run side by side, and cutover needs no downtime.
5. A major-version upgrade is a free dependency audit. Peer-conflicts are the ecosystem asking what you still need. next-auth turned out to be deletable, not migratable.
6. Serverless changes the unit of state. No in-memory Set, no long-lived process - push "memory" into the row. This is more annoying and more correct.
7. When the queue is the database, the debugger is SELECT. Build read-only diagnostic endpoints early; they turn "why is it stuck?" into a one-line, shareable answer.
8. Delete to improve. The headline metric of V2 isn't a feature - it's -2,400 lines, -1 runtime, -2 services, -1 dependency. The best refactors are subtractive.

---

Key files in V2: src/app/api/cron/*/route.ts (5 cron entry points) · src/lib/cron-auth.ts · src/lib/publish/{publish-due,publish-post,platform-publishers}.ts · src/lib/supabase-db.ts (claim/finalize/reclaim helpers) · src/lib/rss/process.ts · src/lib/ai/{generate,weekly-plan}.ts · src/lib/crypto.ts (keyring) · src/proxy.ts · vercel.json (crons) · src/app/api/.../recover-stuck-posts/route.ts (diagnostics).