Publish dev database Docker images to GHCR#837
Conversation
Add a GitHub Actions workflow to build and publish the postgres-postgis and db-setup Docker images to GHCR, enabling CI to use the same database environment as local development. Update CI workflows and docker-compose to reference the published images. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Claude Review✅ Ready to merge - No significant issues found. The workflow is well-structured with appropriate permissions, multi-platform builds, and proper conditional push logic for PRs vs main branch. |
Build a multi-stage Dockerfile (Dockerfile.dev-db) that downloads Kilter/Tension APKs, extracts SQLite databases, fixes compatibility issues, and imports all data via pgloader at build time. This eliminates the ~400MB download + import step that previously ran on every developer's first `db:up`. - Stage 1: postgres:17 + PostGIS (self-contained, no GHCR dependency) - Stage 2: Downloads APKs, extracts/fixes SQLite DBs, imports via pgloader - Stage 3: Clean image with only the pre-populated PGDATA - Uses PGDATA=/var/lib/postgresql/pgdata to avoid Docker VOLUME discard - docker-compose.yml uses new dev-db image, removes db_setup service - GitHub Actions workflow builds/publishes ghcr.io/marcodejongh/boardsesh-dev-db Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Review✅ Ready to merge - Minor issues noted below, but nothing blocking. Issues
Notes
|
- Remove db:setup script (referenced removed db_setup service) - Fix db:up: remove stale db-setup-complete.flag check, auto-download MoonBoard data if not already present - Add explicit verification after Tension APK extraction to fail fast with a clear error instead of silently continuing - Apply drizzle migrations inside Docker build via psql + journal.json, recording SHA-256 hashes in __drizzle_migrations table so drizzle-orm recognizes them as already applied - Widen build context to packages/db/ so Dockerfile can access both docker/ scripts and drizzle/ migrations Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Review✅ Ready to merge - Minor issues noted below, but nothing blocking. Issues
Minor
|
Without this flag, psql silently continues past SQL errors and returns exit code 0, meaning broken migrations could be applied without failing the build. Also adds progress counter (e.g. [3/56]) and a final success message to confirm all migrations completed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Review✅ Ready to merge - Minor issues noted below, but nothing blocking. Issues
DocumentationNo documentation files appear to need updates for this change. |
- Increase max_wal_size to 2GB to suppress "checkpoints occurring too frequently" warnings during heavy import/migration writes - Add verification step after migrations: queries __drizzle_migrations count and compares to expected, fails build on mismatch - Prints table count for quick visual confirmation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Review✅ Ready to merge - Minor issues noted below, but nothing blocking. Issues
Notes
|
PostgreSQL TCP socket binding fails under QEMU emulation in Docker
buildx arm64 builds ("could not bind IPv4 address 127.0.0.1: Address
already in use"). Switch to Unix sockets exclusively:
- listen_addresses='' (disable TCP entirely)
- unix_socket_directories=/var/run/postgresql
- All psql calls use -h /var/run/postgresql
- pgloader DB_URL uses URL-encoded socket path (%2Fvar%2Frun%2Fpostgresql)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Review✅ Ready to merge - Minor issues noted below, but nothing blocking. Issues
Notes
|
- pgloader can't parse %2F-encoded Unix socket URLs (DIGIT-CHAR-P error). Switch to localhost TCP for pgloader while keeping Unix sockets for psql. - Move complex db:up one-liner to scripts/dev-db-up.sh for maintainability. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude ReviewIssues
|
Port 5432 is claimed under QEMU emulation (buildx arm64 on amd64 runner), causing "Address already in use" for both IPv4 and IPv6. Using a non-standard port (15432) avoids the conflict while keeping TCP for pgloader and Unix sockets for psql. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Review✅ Ready to merge - Minor issues noted below, but nothing blocking. Issues
|
QEMU emulation on amd64 runners cannot bind any TCP sockets, which pgloader requires for its connection. Build amd64 only in CI. Apple Silicon users can pull the amd64 image (Docker Desktop runs via Rosetta) or build locally where native arm64 has no QEMU limitation. Also reverts port 15432 workaround since arm64 is no longer built in CI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Review✅ Ready to merge - Minor issues noted below, but nothing blocking. Issues
DocumentationCLAUDE.md was appropriately updated with the new database setup process. |
Explain the faster dev setup with the pre-built Docker image, how to pull/reset/build-locally, and that MoonBoard data is imported on first run. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary
Dockerfile.dev-dbbuilds a PostgreSQL + PostGIS image with all Kilter/Tension board data and drizzle migrations pre-applied. Developers pull and run — no more waiting for ~400MB APK downloads and pgloader imports on first setup.db:up: Moved from a fragile one-liner toscripts/dev-db-up.shshell script. It starts containers, runs migrations (to pick up any newer than the image), and downloads/imports MoonBoard data.build-dev-dbjob to publish the image to GHCR on merge to main.Breaking changes
Volume path changed
The postgres data directory moved from
/var/lib/postgresql/datato/var/lib/postgresql/pgdata(to work around Docker's VOLUME declaration that discards build-time data). Existingdb_datavolumes won't be picked up by the new image. To reset:db:setupscript removedThe
npm run db:setupcommand and thedb_setupDocker service have been removed. All board data (Kilter/Tension) is now baked into the postgres image. MoonBoard data is imported bynpm run db:up.MoonBoard data not in the image
MoonBoard data requires the Neon HTTP proxy (TypeScript import script), so it can't be imported during Docker build. It's downloaded and imported by
npm run db:upon first run instead.Technical details
ENV PGDATA=/var/lib/postgresql/pgdataavoids the officialpostgres:17image'sVOLUME /var/lib/postgresql/datawhich would discard build-time data%2F-encoded Unix socket URLs, so postgres listens onlocalhostTCP for pgloader while psql uses Unix socketsjq+psql+sha256sumto parse the journal, apply SQL files, and record hasheslinux/amd64andlinux/arm64Test plan
build-dev-dbworkflow builds successfully on this PRworkflow_dispatchand verify images appear in GHCRdocker compose down -v && docker compose up -d && npm run db:uplocallypsql -h localhost -U postgres main -c "\dt kilter_*"🤖 Generated with Claude Code