Cloudflare R2 Data Catalog (Basin Catalog)¶

Cloudflare's Iceberg REST catalog for R2 buckets — renamed Basin Catalog — vends storage credentials: pg2iceberg gets them from the catalog for each table, so no S3 keys are needed.

Prerequisites¶

• An R2 bucket with its catalog enabled: npx wrangler r2 bucket catalog enable <bucket>
• An R2 API token with Admin Read & Write (R2 → Manage API tokens; it can be limited to the bucket)

npx wrangler r2 bucket catalog get <bucket> shows the catalog URI and the warehouse name.

Configuration¶

export POSTGRES_URL=postgres://user:password@host:5432/db
export ICEBERG_CATALOG_URL=https://catalog.cloudflarestorage.com/<account-id>/<bucket>
export ICEBERG_CATALOG_TOKEN=<R2 API token>
export ICEBERG_WAREHOUSE=<account-id>_<bucket>
pg2iceberg run

A token means bearer auth, and a warehouse that's a name rather than an s3:// path means vended credentials, so nothing else is needed. The same in a config file:

sink:
  catalog_uri: https://catalog.cloudflarestorage.com/<account-id>/<bucket>
  catalog_token: ${ICEBERG_CATALOG_TOKEN}
  warehouse: <account-id>_<bucket>

Vended credentials¶

The catalog returns, with each table, the R2 endpoint and credentials for the bucket. pg2iceberg writes each table's files under the table's location: its staged change chunks in staged/, its data and delete files in data/. pg2iceberg maintain's orphan cleanup only looks in data/.

Maintenance¶

R2 Data Catalog can compact tables and expire their snapshots. Turn it on, and set sink.maintenance: managed (or PG2ICEBERG_MAINTENANCE=managed) so pg2iceberg leaves that to it. See Table Maintenance.