fluxer-pg-to-cassandra/README.md

fluxer-pg-to-cassandra

Migrate a self-hosted Fluxer instance from the PostgreSQL key-value backend to the native Apache Cassandra schema, and switch the running stack over to it.

Fluxer stores everything in PostgreSQL as one fluxer_kv table of __fluxer_type-tagged JSON. Cassandra mode uses ~200 native CQL tables. This tool copies the data across, coercing each value to its destination column's real CQL type.

Cassandra vs ScyllaDB: Fluxer's Rust services use the scylla crate, which speaks the Cassandra wire protocol. Everything here works against Apache Cassandra (what the official instance runs) or ScyllaDB unchanged.

Prerequisites

1. A running Cassandra node, with a keyspace and login role created:

   CREATE ROLE fluxer WITH PASSWORD = '…' AND LOGIN = true;
   CREATE KEYSPACE fluxer WITH replication =
     {'class': 'NetworkTopologyStrategy', '<your-dc>': 1} AND durable_writes = true;
   GRANT ALL PERMISSIONS ON KEYSPACE fluxer TO fluxer;
   ALTER KEYSPACE system_auth WITH replication =
     {'class': 'NetworkTopologyStrategy', '<your-dc>': 1};
   

   The datacenter name (dc=… in cassandra-rackdc.properties) must match FLUXER_CASSANDRA_LOCAL_DC / SCYLLA_DC everywhere. Mismatch → the driver throws localDataCenter was configured as 'datacenter1', but only found hosts in [<dc>].

2. The Cassandra schema applied. Fluxer does not create it on api startup. Use the fluxer-dev tool from a source checkout:

   docker run --rm -it --network host -v ~/fluxer-src:/app -w /app \
     -e FLUXER_CASSANDRA_HOSTS=<host> -e FLUXER_CASSANDRA_PORT=9042 \
     -e FLUXER_CASSANDRA_KEYSPACE=fluxer -e FLUXER_CASSANDRA_LOCAL_DC=<dc> \
     -e FLUXER_CASSANDRA_USERNAME=fluxer -e FLUXER_CASSANDRA_PASSWORD='…' \
     rust:1-bookworm cargo run --release -p fluxer-dev -- cassandra apply
   

   On a slow/low-RAM node the apply may hit a 30s driver timeout part-way; it's idempotent (IF NOT EXISTS), so just re-run until cassandra verify is clean.

3. Docker on a host that can reach both PostgreSQL and Cassandra.

Running the migration

cp .env.example migrate.env      # fill in PG + Cassandra creds (literal values, no quotes)

# dry run — reads Postgres, writes nothing, reports per-table counts + any errors
docker run --rm -it --network host -v "$PWD":/app -w /app --env-file migrate.env \
  node:20 sh -c "npm install && npm run dry-run"

# real run — same, with migrate
docker run --rm -it --network host -v "$PWD":/app -w /app --env-file migrate.env \
  node:20 sh -c "npm install && npm run migrate"

--network host lets the container reach LAN IPs directly. The env-file passes passwords literally, sidestepping URL-encoding and shell !/$ expansion headaches.

How it handles types

The generic JSON transform can't know a column's real CQL type, so the script reads system_schema.columns at startup and coerces per column (coerceForColumn):

• varint ← bigint Long is rejected by the driver; sent as a string.
• map<…> ← Postgres stores some maps as [[k,v],…] arrays; rebuilt into a Map.
• timeuuid ← Postgres has no timeuuid (it stored a plain timestamp); reconstructed with TimeUuid.fromDate(date, 0, fixedNode, fixedClock) — deterministic, so re-runs upsert the same row instead of duplicating the clustering key.

Idempotency & the delete caveat

Every run does a full re-copy; INSERTs upsert by primary key, so re-running is safe and converges on the source state. It syncs inserts and updates but not deletes — a row deleted in Postgres after a prior run stays in Cassandra. For a clean cutover, run the final pass during a maintenance window (Postgres quiet), or TRUNCATE the Cassandra tables first.

Cutover

1. Build the scylla-enabled shard images (the public images are Postgres-only — the Cassandra code is behind #[cfg(feature = "scylla")]):

   ./images/build-cassandra-images.sh v1 ~/fluxer-src
   

2. Final migration run (delta) during a maintenance window.
3. In docker-compose.yml:
   • x-fluxer-env: set FLUXER_DATABASE_BACKEND: cassandra and add the FLUXER_CASSANDRA_* block (incl. FLUXER_CASSANDRA_LOCAL_DC).
   • messages-shard / users-shard: point image: at fluxer-messages-cassandra:<tag> / fluxer-users-cassandra:<tag>.
4. docker compose config -q to validate, then docker compose up -d.

Rollback is a one-word change: set FLUXER_DATABASE_BACKEND: postgres (leave the postgres vars in place) and restart.

Upgrades

The custom shard images are pinned to the commit you built from — they do not update with docker compose pull. On every Fluxer upgrade, re-run ./images/build-cassandra-images.sh <new-tag> ~/fluxer-src so they track the same version as the stock images and avoid protocol skew. (Ideal long-term fix: ask upstream to publish Cassandra-enabled image variants.)

Files

Path	Purpose
migrate.mjs	The migration script (type-aware PG→Cassandra copy).
.env.example	Template for migrate.env (gitignored).
images/Dockerfile.{messages,users}.cassandra	Stock Dockerfile + --features scylla.
images/build-cassandra-images.sh	One-command pinned build of both shard images.