Snapshots

Cache bootstrap payloads for 10x faster initial sync.

The Problem

Bootstrap queries can be slow for large datasets:


// Without snapshots
GET /sync/v1/bootstrap
// Queries database every time
// 10,000 rows → 1-2 seconds 😔

The Solution

Cache bootstrap payloads using snapshot adapters:


import { SyncServer } from '@gluonic/server'
import { S3Snapshots } from '@gluonic/snapshots-s3'
 
const server = SyncServer({
  database,
  auth: JWTAuth({ secret: process.env.JWT_SECRET }),
  snapshots: S3Snapshots({
    bucket: 'my-snapshots',
    intervalMinutes: 10
  })
})
// 10,000 rows → 50-100ms ✓ (was 1-2 seconds)

Available Snapshot Adapters

File-Based (Development)


await registerSyncRoutes(app, {
  snapshot: {
    type: 'file',
    filePath: './data/snapshots',
    intervalMs: 600000  // Regenerate every 10 minutes
  },
  listOrgIds: async () => {
    return await prisma.organization.findMany({ select: { id: true } })
      .then(orgs => orgs.map(o => o.id))
  }
})

S3-Based (Production)


snapshot: {
  type: 's3',
  bucket: 'my-app-snapshots',
  keyPrefix: 'prod/',
  intervalMs: 3600000  // Regenerate every hour
}

Redis-Based (Low Latency)


snapshot: {
  type: 'redis',
  url: 'redis://localhost:6379',
  keyPrefix: 'snapshots:',
  ttlSeconds: 3600  // 1 hour TTL
}

How It Works

Snapshot Generation


// Every intervalMs:
for (const orgId of await listOrgIds()) {
  const payload = {
    lastSid: await adapter.getLastSyncId(orgId),
    rows: await adapter.bootstrap(orgId)
  }
  
  // Save to storage backend
  await saveSnapshot(orgId, payload)
}

Client Request


// Client requests snapshot
GET /sync/v1/snapshot
 
// Server returns cached payload (fast!)
{
  "lastSid": 1000,
  "rows": [ ... ]
}

Performance: 10-50x faster than querying database!

Worker Coordination

Multiple pods can generate snapshots collaboratively:


// Each pod registers as worker
// Consistent hashing assigns orgs to pods
// No duplicate work ✓
// Auto-rebalancing on scale
 
// Configuration (automatic)
snapshot: {
  type: 's3',
  bucket: 'snapshots',
  intervalMs: 600000
}

See existing docs for complete implementation.