Development Story

A comprehensive record of the GospeLib project: what was built, how it was built, and what was learned along the way.

---

1. What Is GospeLib?

GospeLib is a scripture study platform providing full-text, graph-based, AI-powered access to LDS canon and scholarly resources. It is a polyglot microservices monorepo designed around a single thesis: buildable by one person on day one, never requiring a rewrite to reach enterprise scale.

The platform offers:

• A web-based scripture reader with plugin architecture
• A FalkorDB knowledge graph connecting passages, topics, lexicon entries, commentary, and entities
• An AI study assistant powered by Anthropic Claude and OpenAI
• A Study Map for visual graph exploration of scripture connections
• A 14-stage ingest pipeline that transforms source corpus data into a fully connected graph
• A plugin ecosystem with QuickJS WASM sandbox for community extensions
• Subscription-based monetization with Stripe integration

Architecture

The system comprises 8 backend services (Go and Python), 9 shared TypeScript/Python packages, and 3 frontend applications, orchestrated by Nx + pnpm workspaces:

• Gateway (Go/Chi) -- API gateway with JWT validation, rate limiting, reverse proxy
• Content (Python/FastAPI) -- scripture graph queries against FalkorDB with 13 route groups
• Auth (Go/Chi) -- Clerk wrapper, user sync, JWT middleware
• Billing (Go/Chi) -- Stripe subscriptions and entitlements
• AI (Python/FastAPI) -- LLM passage explanation and study questions
• Notifications (Go/Chi) -- push, email, Redis Streams
• Plugin Registry (Python/FastAPI) -- plugin manifest submission, discovery, versioning
• Ingest (Python/Click CLI) -- 14-stage data pipeline from corpus JSON to FalkorDB

Data stores: FalkorDB (graph), PostgreSQL (users/subscriptions), Redis (cache/sessions), Typesense (full-text search).

---

2. The Development Journey

Phase 0: Foundation (M00-M02, M13-M14) -- March-May 2026

The project began from a fully scaffolded monorepo with zero business logic: health-only service stubs, empty package barrels, and a complete corpus dataset (~966 files, ~215 MB across 7 schema families). The monorepo tooling (Nx, pnpm workspaces, CI pipeline, Docker Compose, commitlint, Release Please) was already functional.

M00 (Tech Prep) validated the development environment, established coding standards, and set up the CI pipeline. All 13 issues completed.

M01 (Data Pipeline) built the core ingest pipeline -- the first 7 stages that load lexicon entries, scripture text, topical guide, Bible Dictionary, and commentary into FalkorDB. 22 issues completed.

M02 (Content API) built the content service with passage retrieval, lexicon queries, and graph connections. The Gateway service was scaffolded with JWT middleware and rate limiting. 19 issues completed.

M13 (Multi-Translation Schema) ran concurrently, defining the schema evolution for 31 languages and 4.4 million Translation nodes. 26 issues completed.

M14 (Corpus Harmonization) aligned the corpus data across editions and translations. 24 issues completed.

Phase 1: Reader (M03-M05) -- May-June 2026

M03 (Scripture Reader) built the core reading experience: ScripturePage component, verse rendering, chapter navigation, translation switching. The UI foundation package was established with cross-platform React Native components. 24 issues completed.

M04 (Annotations) added bookmarks, highlights, and notes with Clerk authentication integration. 10 issues completed.

M05 (Search & Staging) integrated Typesense for full-text and faceted search, plus cross-reference footnotes. 13 issues completed.

Phase 2: Enhancement (M09-M12) -- June 2026

M09 (AI Study Assistant) built the provider-abstracted LLM endpoint with corpus filtering, supporting both Anthropic and OpenAI. 10 issues completed.

M10 (Monetization) implemented Stripe webhook handling, subscription CRUD, entitlement APIs, and the Whisper Card (contextual insight per reading). 12 issues completed.

M11 (Launch Readiness) hardened the platform for production: PWA support, service workers, offline caching, install prompts, user onboarding, UI polish, and production infrastructure. 18 issues completed.

M12 (Documentation Site) built the Docusaurus engineering docs site, migrating all milestone specs and product documentation. 21 issues completed.

A1 Amendment: The Specification Gap (M15-M21) -- June 2026

A thorough specification audit revealed gaps between the original milestone specs and what had been built. The A1 Amendment created milestones M15-M21 to close these gaps systematically.

M15 (Cleanup & Data) addressed contradictions in the codebase and aligned ingest pipeline labels and directions. 14 of 15 issues completed (1 partial -- GLDL data population ongoing).

M16 (GUI Infrastructure) was the architectural foundation for the web app rebuild:

• Three-tier composition model (sessions, workspaces, panes)
• Plugin manifest with 14 contribution types and 10 core plugins
• Dexie schema v6 for client-side persistence
• URL schema for deep linking
• Widget dock zone system
• Navigator panel decomposition
• Region resizing and focus cycling

13 of 15 issues completed, 2 partial.

M17 (Core UX) built the interaction layer:

• Three-layer hover card system with 6 trigger types
• Vertical verse-aligned toolbar and horizontal toolbar
• Pinboard widget for pinning scripture connections
• Compound tab comparison (2-4 column scroll-synced)
• Pane discovery and instantiation via Cmd+P
• Session color ambient accent

10 of 11 issues completed, 1 partial (comparison entry points).

M18 (Navigation Surfaces) delivered the full navigation experience:

• Verse Map (canvas left-edge strip, 8 marker types)
• Three-Column Minimap
• Breadcrumb navigation panel
• Infinite scroll across chapter boundaries
• Page gutters with verse-aligned annotation markers
• Branching history tree with staleness engine
• Command palette with 3-tier results and 5 prefix modes
• core.notes plugin (rich-text notes + annotations pane)

All 10 issues completed.

M19 (Integration & Polish) finished the web app surface:

• Corpus filtering, cloud sync, help system, focus modes
• Undo model, pane lifecycle, dock zone visuals
• Accessibility audit (WCAG 2.1 AA via axe-core)
• Motion and transitions verification
• Mobile-responsive layout

All 13 issues completed.

M20 (Pipeline & Schema) brought the graph to 100% parity with corpus.db:

• Talk processing stage for conference talks
• Church content stages (curriculum, books, periodicals, hymns, proclamations)
• Typed entity projection (Person, Place, Event, PeopleGroup)
• Density materialization (xrefCount, commentaryCount, entityMentionCount)
• CLI --mode flag (full/csv-only/incremental/validate)
• 8 validation rule sets
• corpus.db-to-graph parity check (the done-gate for the milestone)

All 17 issues completed.

M21 (Performance & Mobile) established the testing and performance infrastructure:

• Lighthouse CI performance budgets
• Bundle size verification with @next/bundle-analyzer
• Playwright E2E configuration
• Storybook visual regression testing
• Vitest performance test suites

All 8 issues completed.

Study Map (M22-M24) -- June 2026

M22 (Session Advanced) added workspace presets and session management. All 3 issues completed.

M23 (Study Map Core) built the graph visualization:

• Sigma.js v3 + Graphology integration with WebGL canvas
• ForceAtlas2 layout with Web Worker convergence detection
• Type-specific node rendering with LOD labels
• Graph seeding, expansion, and semantic zoom
• Pin/unpin, multi-select, rubber-band drag
• Dexie-based persistence with 500ms debounced auto-save
• Staleness gradient engine with navigation edges

8 of 9 issues completed, 1 partial (nav-graph edge ingestion).

M24 (Study Map Integration) connected the Study Map to the reader:

• Reader-to-map integration via vertical toolbar and hover card actions
• Bidirectional Pinboard sync
• Pane-local toolbar with node/edge toggles
• Accessible map outline (role="tree", keyboard navigation)
• Pane-local undo stack (50 entries, Cmd+Z/Cmd+Shift+Z)

All 5 active issues completed (2 obsolete: Link Sets and Inspector).

Scholarly Features (M25) -- June 2026

M25 (Advanced Plugins) delivered the 5 core study plugins:

• core.interlinear -- word-level Hebrew/Greek interlinear layer
• core.witnesses -- manuscript viewer pane (ELIMINATED -- witness concept removed; manuscripts are editions**)**
• core.ai -- AI study assistant (sidebar widget + pane dual form)
• core.commentary -- commentary browser
• core.journal -- daily study journal with streak tracking

4 of 5 completed, 1 partial (core.ai -- SSE streaming and paywall deferred).

Plugin Ecosystem (M26) -- June 2026

M26 (Plugin Ecosystem) built the full plugin infrastructure:

• QuickJS WASM sandbox for community plugin isolation
• Plugin manifest and distribution via npm registry
• Subscription gating with feature flags and entitlement checks
• Session-specific plugin enable/disable
• Toolbar customization (drag-to-reorder, hide/show)
• Plugin Registry FastAPI service
• Permissions model ADR + @gospelib/plugin-sdk
• Semver versioning policy with breaking change detection

All 8 issues completed.

Witness Apparatus (M30) -- June 2026

M30 (Witness Apparatus) was created to track the witness/critical apparatus pipeline work. Pipeline stubs are in place (1 of 3 issues done), but witness data acquisition and the critical apparatus UI remain.

---

3. The Batch/Phase Execution Model

The most distinctive aspect of GospeLib's development was its parallel batch execution model, documented in the PHASE-MAP.md. This model enabled massive parallelism while maintaining correctness.

How It Worked

1. Batch = one branch, one PR, one agent session. Each batch was sized to fit a single AI coding agent's context window.

2. Phase = a set of conflict-free parallel batches. All batches in a phase were guaranteed to edit disjoint files, so they could run simultaneously without merge conflicts.

3. Track = a sequence of phases with dependencies. Eight parallel tracks (A through X) advanced independently except at explicit cross-track gates.

The Conflict-Freedom Strategy

The key insight was that merge conflicts concentrate in a small number of "hot" registry files. Two foundational refactors eliminated this contention:

• Ingest stage registry (Track A, A0.1): Converted the 5 shared ingest files into a decorator-based auto-discovery registry. After this, new ingest stages were self-contained modules that touched no shared files.

• Plugin contribution registry (Track B, B0.2/M16-002): Built the 14-contribution-type manifest and host registry so each plugin self-registered via its manifest. After this, every core plugin was new-files-only.

Track Structure

Track	Theme	Phases	Purpose
A	Corpus-to-graph parity	A0, A1, A2	Complete the FalkorDB knowledge graph
B	Web GUI	B0, B1, B2, B3, B4	Build the entire web application
C	Scholarly features	C1, C2	Interlinear, JST, knowledge graph UI
D	Study Map	D1, D2	Graph visualization and integration
E	Plugins & ecosystem	E0, E1, E2	Plugin sandbox, registry, core plugins
F	Desktop & mobile	F1, F2, F3	Electron, Expo/React Native
W	Witness apparatus	W1, W2	Critical apparatus pipeline
X	Cross-cutting	(any time)	Security, docs, site regeneration

Universal Merge Checklist

Every batch, without exception, had to satisfy:

• Branch rebased onto latest origin/stage with zero merge conflicts
• All CI tasks pass (lint, type-check, unit/integration tests)
• New code has tests; coverage does not regress
• Lint clean for every language touched
• Every issue has completed frontmatter, PR blockquote, and updated counts
• No new skip/allowlist entries to make CI pass
• For graph batches: count parity against corpus.db source

Phase Exit Validation

Between phases, a formal gate validated that parallel batches composed correctly:

• Clean rebuild from fresh worktree
• End-to-end boot (full ingest for backend; pnpm build for frontend)
• Integration checks specific to the phase
• No regression on prior phase checks

---

4. Key Technical Decisions

Plugin Host and Composition Model

The three-tier composition model (sessions > workspaces > panes) was a deliberate architectural choice. Sessions represent study contexts (e.g., "Sunday School," "Personal Study"), workspaces are layout configurations within a session, and panes are individual content views. This hierarchy enables:

• Per-session plugin enable/disable
• Workspace presets that can be shared
• Pane-local undo stacks
• Session color ambient accents for visual context switching

Study Map: Sigma.js Over D3

The Study Map chose Sigma.js v3 with Graphology over D3.js for graph rendering because:

• WebGL rendering handles large graphs (200+ nodes) without performance degradation
• ForceAtlas2 layout runs in a Web Worker with convergence detection
• Graphology provides a mature graph data structure with serialization
• The same renderer can display both Study Map and Knowledge Graph data (resolving scope decision SD-1: M08's ConstellationView collapsed into the Study Map)

Plugin Sandbox: QuickJS WASM

Community plugins run in a QuickJS WASM sandbox with:

• Time-bounded execution
• Default-deny permissions (per ADR-002)
• UI rendered in sandboxed iframes
• Ed25519 signature verification for published plugins

This enables a community plugin ecosystem without compromising security.

Graph Database: FalkorDB

FalkorDB (Redis-compatible graph database) was chosen because:

• The scripture knowledge graph IS the product -- relationships between passages, topics, entities, and commentary are first-class
• Redis-compatible wire protocol enables familiar tooling
• Single-threaded writes simplify the idempotent MERGE strategy
• The graph model naturally represents cross-references, topical connections, and entity relationships

Ingest Pipeline: Self-Registering Stage Registry

The ingest pipeline evolved from a 7-stage hardcoded list to a 14-stage self-registering system. This was the critical enabler for parallel development: each new content domain (talks, curriculum, periodicals, hymns, publications, etc.) could be implemented as a standalone module with zero edits to shared files.

---

5. Issues Discovered and Resolved

CI Pipeline Corrections

Multiple CI issues were identified and fixed:

• Dependency security: 187 Dependabot CVEs were triaged and resolved (9 critical, 91 high) in Track X security remediation
• Lint matrix coverage: Expanded lint matrices to cover all languages (Python/Ruff, TypeScript/ESLint, Go/golangci-lint) with zero exceptions
• Test determinism: Fixed non-deterministic test failures caused by timing-dependent graph queries and Redis state

Merge Process Corrections

The parallel batch execution model surfaced several merge process issues:

• Hot file contention: Early batches occasionally collided on shared files (e.g., Dexie db.ts, command palette). The owned-files roster in PHASE-MAP.md was tightened to serialize these edits explicitly.
• Phase gate enforcement: Initial phase transitions were informal. After a composability failure (two Phase B2 batches both editing the session store), formal phase exit validation was instituted.
• Frontmatter drift: Issue completed-dates and milestone README counts frequently drifted from reality. A full reconciliation audit on 2026-06-12 corrected counts across all 31 milestones.

Scope Decisions (SD-1 and SD-2)

Two scope decisions gated later tracks:

• SD-1 (M08 vs M23/M24): Should M08's ConstellationView be a separate graph renderer or collapse into the Sigma.js Study Map? Resolution: Collapse into Study Map. This eliminated a duplicate renderer and ensured a single, well-tested graph visualization.

• SD-2 (M06/M07 vs M25): Should interlinear/JST/witness UIs be delivered as reader components (M06/M07) or A1 plugins (M25)? Resolution: M06/M07 own data and API (Track C1); plugins (M25) own the UI. This kept the data layer clean while enabling the plugin architecture for the UI.

Specification Audit (A1 Amendment)

A thorough specification audit on 2026-06-08 identified gaps between the original milestone specs (M00-M14) and what had been built. Key findings:

• Missing milestones for GUI infrastructure, navigation, integration/polish, and performance
• No specification for the plugin composition model
• Ingest pipeline documentation described 7 stages; actual implementation had grown to 14
• Content service had 13 route groups but only 6 were documented

The A1 Amendment (M15-M21) was created to close these gaps, adding 84 new issues across 7 milestones.

---

6. The Adversarial Review Process

GospeLib employed a formal adversarial review process for critical artifacts. The process deployed three parallel reviewer subagents:

1. Advocate (friendly): Identified strengths and validated the approach
2. Adversary (hostile): Attacked assumptions, found weaknesses, challenged claims
3. Analyst (neutral): Provided data-driven assessment with specific evidence

The reviewers' findings were then cross-examined against each other, and a single adjudicated report was produced where every claim was substantiated or rebutted with evidence.

Impact

The adversarial review process caught several significant issues:

• M16-014 (@gospelib/scripture-ref): The initial implementation had edge cases with multi-chapter ranges and non-standard book ID formats that were caught and fixed before merge
• Ingest stage registry design: The initial decorator-based approach had a circular import issue that was identified during adversarial review of the A0.1 design
• Plugin permission model: The original permissions enum was too coarse-grained; adversarial review led to the more nuanced capability-based model in ADR-002
• Milestone count reconciliation: The 2026-06-12 audit that reconciled all milestone counts was triggered by adversarial review findings showing systematic undercounting

---

7. Current State

What Is Complete (332 of 365 issues, 91%)

22 of 31 milestones are at 100%:

• Foundation: M00, M01, M02, M13, M14 (complete)
• Reader: M03, M04, M05 (complete)
• Enhancement: M09, M10, M11, M12 (complete)
• A1 Amendment: M18, M19, M20, M21, M22 (complete)
• Study Map: M24 (complete)
• Plugins: M26 (complete)

5 milestones are at 80-93%:

• M15 (93% -- GLDL data population partial)
• M16 (87% -- 2 issues partial: focused-pane context, dock zones)
• M17 (91% -- 1 issue partial: comparison entry points)
• M23 (89% -- 1 issue partial: nav-graph edge ingestion)
• M25 (80% -- 1 issue partial: core.ai streaming/paywall)

What Remains (33 issues across 26 remaining)

Interlinear and Scholarly Features (16 issues):

• M06 (7 to do): interlinear UI components, lexicon bubble, study mode
• M07 (3 to do): JST diff view, manuscript attribution, witness layer UI
• M08 (6 to do): knowledge graph visualization (now collapsed into Study Map)

Desktop and Mobile (10 issues):

• M27 (2 to do): Electron shell + desktop features
• M28 (3 to do + 1 partial): Expo project setup, mobile nav, reader, state
• M29 (5 to do): mobile annotations, notes, study map, search, tablet

Witness Data (2 issues):

• M30 (2 to do): witness data acquisition, critical apparatus UI

Deferred Work

• Desktop application (M27): Electron shell deferred; PWA covers the immediate need (M27-003 superseded by M11-003)
• Mobile application (M28-M29): React Native/Expo deferred until web platform stabilizes
• Witness data (M30): Blocked on open-source critical apparatus data acquisition

---

8. Statistics

Metric	Count
Total milestones	31 (M00-M30)
Issues completed	332 of 365 (91%)
Issues partial	7
Issues remaining	26
Milestones at 100%	22
Backend services	8
Shared packages	9
Frontend apps	3 (web, admin, mobile)
Ingest pipeline stages	14+ (with self-registering extensions)
Core plugins	10 (interlinear, witnesses, ai, commentary, journal, notes, +4 infrastructure)
Content service route groups	13
PRs on GitHub	700+
Commits on stage branch	2,200+
Corpus data files	~966 files (~215 MB)
Graph node types	22
Graph edge types	15+
Supported languages (translations)	31
Development execution tracks	8 (A through X)

---

9. Lessons Learned

What Worked Well

1. The batch/phase model was transformative. By decomposing work into conflict-free parallel batches with formal phase gates, the project achieved high throughput while maintaining composability guarantees.

2. The stage registry pattern was the key enabler. Converting shared files into self-registering registries eliminated the hot-file contention that would have serialized all ingest and plugin work.

3. The parity gate (M20-016) was the right done-definition. Having a formal, automated check for corpus.db-to-graph parity prevented "done enough" from creeping in.

4. Adversarial review caught real issues. The three-perspective review process identified problems that single-reviewer code review would have missed.

What Could Be Improved

1. Milestone count tracking needs automation. Manual reconciliation of issue counts across READMEs was error-prone and time-consuming. An automated script should derive counts from issue frontmatter.

2. Scope decisions should be made earlier. SD-1 and SD-2 gated entire tracks (C and E). Earlier resolution would have enabled more parallel work.

3. Documentation-as-code should be enforced. Several docs pages (content service endpoints, ingest pipeline stages) diverged significantly from reality. CI checks comparing docs against code would catch drift.

4. The "current state" section in design docs rots quickly. Statements like "30+ stub files" in the MVP doc and M11 launch readiness doc became false as soon as the first implementation landed. These sections should either be auto-generated or clearly marked as point-in-time snapshots.

---

Last updated: 2026-06-12