Migrating Realtime Database to Firestore With Dual Writes and a Parallel Run

Most teams still on the Realtime Database are there for a good reason: it was the default in 2017, the app works, and the JSON tree is fast. The migration request usually arrives with a specific trigger — a query that RTDB cannot serve, a fan-out that has gotten expensive in downloaded bytes, or a security model that outgrew .read/.write on nodes.

RTDB is not deprecated and Firestore is not automatically better. But when the trigger is real, the mistake is treating the move as an export/import job. RTDB and Firestore have different query models, different billing units, and different listener semantics. The data is the easy part.

Here is the sequence we run, and the check that has to pass before each step.

Step 0: decide whether to move at all

Move if you need composite queries, per-document security, or billing that tracks operations instead of bytes. Stay if your workload is presence, cursors, high-frequency counters, or anything where you write the same small value many times per second — RTDB bills bandwidth and storage, Firestore bills per document write at roughly $0.18 per 100,000 writes.

Do the arithmetic on the hottest path before anything else. A cursor-position node written 4 times a second per active user is 14,400 writes per user-hour. Ten concurrent users is 144,000 writes an hour, about $0.26 — call it $190 a month for ten users, and it scales linearly. In RTDB that same traffic is a few megabytes of bandwidth. That path should stay in RTDB, permanently, and a hybrid is a legitimate end state.

So the deliverable from step 0 is not "migrate" or "don't." It is a table: every top-level node, its write frequency, its read pattern, and a destination of firestore, rtdb, or drop.

Step 1: translate the model, don't transcribe it

RTDB trees are denormalized for one access path because that is all RTDB gives you. Firestore lets you index, which means some of the fan-out you built can go away — and some of it must stay, because Firestore has its own limits.

The patterns that translate directly:

RTDB	Firestore
/users/$uid	users/{uid} document
/messages/$roomId/$msgId	rooms/{roomId}/messages/{msgId} subcollection
/userRooms/$uid/$roomId: true (index node)	array field memberUids with array-contains, or a members subcollection

The ones that need a decision:

• Deep trees. A get() on an RTDB node returns the whole subtree. A Firestore document read returns that document only, and subcollections are separate reads. If your UI relied on one fetch of /rooms/$id pulling members, settings, and the last 50 messages, that becomes four reads and four listeners unless you copy a summary onto the parent.
• Index nodes. /userRooms/$uid/$roomId: true existed to answer "which rooms is this user in." In Firestore, memberUids: [uid, ...] plus where('memberUids', 'array-contains', uid) replaces it — but arrays cap at the 1 MiB document limit and each element is indexed, so a room with 20,000 members needs a members subcollection instead. Under a few hundred, use the array; above that, use the subcollection and accept the extra read.
• Counters. RTDB transactions on a single node are cheap. Firestore gives you about one sustained write per second per document, so a global counter needs FieldValue.increment with sharding, or a scheduled aggregation, or it stays in RTDB.

Write this translation down as a document-by-document mapping before you write any code. Every later step consumes it.

Step 2: backfill with an idempotent, resumable job

Export RTDB to JSON, then transform and write into Firestore in batches. Two properties matter more than speed.

Idempotent. Derive the Firestore document ID from the RTDB key, and use set() with merge rather than add(). Re-running the job over the same slice must produce the same result, because you will re-run it.

Resumable. Checkpoint the last completed key so a crash at 80% resumes at 80%.

// one shard of the backfill; run many, keyed by RTDB key prefix
const BATCH = 400; // under the 500-op limit, leaves room for retries

async function backfillRooms(startAfterKey) {
  const snap = await rtdb.ref('rooms')
    .orderByKey().startAfter(startAfterKey).limitToFirst(BATCH).get();

  let batch = firestore.batch();
  let lastKey = startAfterKey;

  snap.forEach((child) => {
    const room = child.val();
    batch.set(firestore.doc(`rooms/${child.key}`), {
      name: room.name,
      memberUids: Object.keys(room.members ?? {}),
      createdAt: new Date(room.createdAt),
      migratedFrom: 'rtdb',
      migratedAt: FieldValue.serverTimestamp(),
    }, { merge: true });
    lastKey = child.key;
  });

  await batch.commit();
  await firestore.doc('_migration/rooms').set({ lastKey }, { merge: true });
  return snap.size === BATCH ? lastKey : null; // null = done
}

Budget the writes. Two million documents is 2,000,000 writes, roughly $3.60, plus index writes. The backfill is almost never the expensive part of a migration; the dual-write window is.

Keep the migratedFrom marker. When you find a document with the wrong shape three weeks later, you want to know whether it came from the backfill or from a live write.

Step 3: dual-write, source of truth still RTDB

Now make every write land in both stores while RTDB stays authoritative. Two ways to do it, and the difference matters.

Mirror with a trigger (preferred). An RTDB-triggered Gen 2 function projects each change into Firestore. Clients change nothing, so there is no client release to coordinate and no partial rollout of write logic.

import { onValueWritten } from 'firebase-functions/v2/database';

export const mirrorRoom = onValueWritten(
  { ref: '/rooms/{roomId}', region: 'us-central1' },
  async (event) => {
    const { roomId } = event.params;
    const after = event.data.after.val();

    if (after === null) {
      await firestore.doc(`rooms/${roomId}`).delete();
      return;
    }

    await firestore.doc(`rooms/${roomId}`).set({
      name: after.name,
      memberUids: Object.keys(after.members ?? {}),
      updatedAt: FieldValue.serverTimestamp(),
      mirrorSeq: event.time, // for ordering checks
    }, { merge: true });
  },
);

The cost is one function invocation and one Firestore write per RTDB change — and RTDB triggers fire on every node write under the ref, so a tree that gets 30 small writes per room update produces 30 invocations. Narrow the trigger ref, or debounce, or you will see the invoice before you see the benefit.

Dual-write in application code. Honest, but every client and every backend path needs the second write, and old app versions in the wild will not have it. Use this only when all writes already flow through your own API.

Either way, do not point clients at Firestore yet. This window exists to prove the projection is correct, nothing else.

Step 4: shadow reads and a divergence report

The step teams skip. While RTDB still serves the UI, read both stores and compare. Sample, don't compare everything: 1% of reads, or a scheduled job over a random slice.

async function compareRoom(roomId) {
  const [rtdbSnap, fsSnap] = await Promise.all([
    rtdb.ref(`rooms/${roomId}`).get(),
    firestore.doc(`rooms/${roomId}`).get(),
  ]);

  const expected = projectRoom(rtdbSnap.val()); // same function the mirror uses
  const actual = fsSnap.data();
  const diff = diffFields(expected, actual, ['name', 'memberUids']);

  if (diff.length) {
    await firestore.collection('_migration_divergence')
      .add({ roomId, diff, at: FieldValue.serverTimestamp() });
  }
}

Reuse the same projection function in the mirror and in the comparison. If they differ, you are testing two implementations against each other and will chase phantom diffs.

Expect real findings here, and they are consistent across engagements:

• Deletes that didn't propagate. RTDB deletion by setting null on a parent removes the subtree in one event; the mirror has to translate that into multiple Firestore deletes, and the ones it misses become orphans.
• Numbers becoming arrays. RTDB stores an object with keys 0,1,2 as an array on read. Round-trip that and a map turns into a list.
• Timestamp drift. RTDB ServerValue.TIMESTAMP is epoch millis; Firestore Timestamp is not. Pick one representation at the boundary and assert it in the comparison.
• Ordering under bursts. Two writes to the same node within milliseconds can mirror out of order. mirrorSeq plus a guard that drops stale events fixes it.

The gate: the divergence rate on your sampled slice has to sit at zero for a full business cycle — including whatever your weekly batch jobs do — before you move reads. Not "low." Zero, with every class of diff explained.

Step 5: move reads, one surface at a time

Firestore becomes the read source behind a remote-config flag, per surface and per cohort. Start with the lowest-consequence screen and a small percentage.

Before flipping anything, deploy the Firestore rules with a tested denial suite, and check the read arithmetic on the new path. A screen that was one RTDB subtree fetch may now be four document reads plus a query; at 40,000 sessions a day that is the difference between a rounding error and a line item. Fix it with a summary document on the parent before rollout, not after.

Writes stay on RTDB through this step. That is what makes rollback trivial: flip the flag back and the authoritative data was never wrong.

Step 6: flip writes, keep the reverse mirror

When reads have been 100% Firestore for a couple of weeks, move writes. Then install the mirror in reverse — Firestore triggers projecting back into RTDB — and leave it running.

import { onDocumentWritten } from 'firebase-functions/v2/firestore';

export const reverseMirrorRoom = onDocumentWritten('rooms/{roomId}', async (event) => {
  const after = event.data?.after?.data();
  const ref = rtdb.ref(`rooms/${event.params.roomId}`);
  await after ? ref.update(toRtdbShape(after)) : ref.remove();
});

This costs function invocations and RTDB bandwidth, and it buys you a rollback path that survives a bad week. Keep it for one full billing cycle. Then delete the reverse mirror, delete the forward mirror, and delete the RTDB nodes you migrated — in that order, with a backup taken first.

The part that is not technical

Every step above has a gate, and the gates are the method. Backfill is done when a re-run produces no changes. Dual writes are done when divergence is zero across a business cycle. Reads are done when error rates and read counts hold at 100%. Writes are done when you have gone a billing cycle without needing the reverse mirror.

Migrations fail when someone collapses steps 3 through 5 into one release because the backfill looked fine. The backfill always looks fine. It is the live writes that diverge.

If you are planning an RTDB-to-Firestore move and want the translation table and gate criteria reviewed before you write the backfill, that is a short engagement and it is much cheaper than discovering the counter problem after cutover.