Installation¶

Docker¶

Images for linux/amd64 and linux/arm64 are published to Docker Hub: latest from main, X.Y.Z from each release.

docker pull polynyadev/pg2iceberg:latest

To build one yourself from the repository's multi-stage Dockerfile:

docker build -t pg2iceberg-rust:dev .

The image's entrypoint is the pg2iceberg binary; pass a subcommand, and the settings as environment variables (or mount a config file):

docker run --rm \
  -e POSTGRES_URL=postgres://user:password@db:5432/app \
  -e ICEBERG_CATALOG_URL=https://catalog.example.com \
  -e ICEBERG_WAREHOUSE=s3://my-bucket/warehouse/ \
  polynyadev/pg2iceberg:latest run

Build from source¶

Requires Rust 1.85+ (matching rust-toolchain.toml).

git clone https://github.com/polynya-dev/pg2iceberg.git
cd pg2iceberg/pg2iceberg-rust
cargo build --release --bin pg2iceberg --features prod
./target/release/pg2iceberg --help

The default build is sim-only (no S3, no real PG client). The prod feature wires the real backends (tokio-postgres-rustls, iceberg-rust REST catalog, object_store for S3, iceberg-storage-opendal for catalog-side IO).

Helm¶

Not yet published.

Prerequisites¶

PostgreSQL¶

PostgreSQL 14 or newer is required; pg2iceberg refuses to start against an older server.

Logical replication must be enabled. Set the following in postgresql.conf:

wal_level = logical
max_replication_slots = 10
max_wal_senders = 10

pg2iceberg auto-creates the replication slot and publication on first run; you don't need to create them manually. If you do want to pre-create:

SELECT pg_create_logical_replication_slot('pg2iceberg_slot', 'pgoutput');
CREATE PUBLICATION pg2iceberg_pub FOR TABLE my_schema.my_table;

Each replicated table needs a primary key. Tables without one fail at startup with a clear error pointing at the missing column.

Note

The replication slot retains WAL until pg2iceberg consumes it. Monitor slot lag to avoid excessive disk usage on the primary. The startup validation refuses to start if slot.wal_status = lost (PG 13+) — at that point the slot is unrecoverable and you must pg2iceberg cleanup and re-snapshot.

Local development on macOS (Colima)¶

Integration tests use testcontainers and need a Docker daemon. On macOS we recommend Colima:

brew install colima
colima start

export DOCKER_HOST=unix://${HOME}/.colima/default/docker.sock
export TESTCONTAINERS_RYUK_DISABLED=true   # Colima doesn't run Ryuk reliably

Then:

cargo test --features integration -- --test-threads=1

Single-threaded execution is required because the integration tests share a single PG container per test binary; parallelism races on slot/publication names.