End-to-End Guide

Step 1 — Choose a test mode

Combine multiple modes in one testSuite — they run sequentially.

Step 2 — Configure reconciliation

Every comparison accepts three snapline options (processing order: ignore → transform → map → compare).

import { snapline } from '@vaagatech/snapline-core';

const { match, diff, processed } = snapline(liveData, expectedData, {
  ignoreFields: ['metadata.traceId', 'pincode'],
  transformations: {
    lastLogin: (v) => isValidDate(v) ? 'VALID_DATE' : 'INVALID_DATE',
    role: (v) => String(v).toUpperCase(),
  },
  dataMapping: {
    status: { synced: 'SYNCED', ACTIVE: 'ACTV' },
    planCode: { PRO: 'premium' },
  },
});

Step 3 — API vs file (all protocols)

REST

await testSuite('REST snapshot', {
  auth: auth.oauth2({ tokenUrl, clientId, clientSecret }),
  baseUrl: 'https://api.example.com',
  api: {
    endpoint: '/api/v1/user/sync',
    method: 'POST',
    inputFile: './fixtures/input.json',
    expectedFile: './fixtures/expected.json',
    ignoreFields: ['pincode'],
  },
});

GraphQL

import { api } from '@vaagatech/snapline-core';

await testSuite('GraphQL snapshot', {
  baseUrl: 'https://api.example.com',
  api: {
    ...api.graphql({
      endpoint: '/graphql',
      queryFile: './fixtures/query.graphql',
      variablesFile: './fixtures/variables.json',
      dataPath: 'customerAccount',
    }),
    expectedFile: './fixtures/expected.json',
    ignoreFields: ['metadata.traceId'],
    dataMapping: { status: { synced: 'ACTIVE' } },
  },
});

SOAP

await testSuite('SOAP snapshot', {
  baseUrl: 'https://api.example.com',
  api: {
    ...api.soap({
      endpoint: '/soap/user',
      soapAction: 'GetUser',
      inputFile: './fixtures/request.xml',
    }),
    expectedFile: './fixtures/expected.json',
  },
});

Step 4 — Database comparison

Implement DbConnectionLike with your SQL driver, or use the in-memory example from packages/core/examples/in-memory-db.mjs.

Same query on both sides

import { createInMemoryDb } from '@vaagatech/snapline-core/examples/in-memory-db.mjs';

const sourceDb = createInMemoryDb([{ status: 'ABC', email: 'alice@example.com' }]);
const targetDb = createInMemoryDb([{ status: 'CBA', email: 'alice@example.com' }]);

await testSuite('Warehouse sync', {
  dbComparison: {
    sourceDb,
    targetDb,
    query: 'SELECT status, email FROM users WHERE email = :email',
    params: { email: 'alice@example.com' },
    dataMapping: { status: { ABC: 'CBA' } },
  },
});

Different queries with linkKeys

await testSuite('Orders sync', {
  dbComparison: {
    sourceDb, targetDb,
    sourceQuery: `SELECT o.order_id AS orderId, o.status FROM orders o WHERE o.email = :email`,
    sourceParams: { email: 'alice@example.com' },
    targetQuery: `SELECT order_id AS orderId, status FROM orders WHERE order_id = :orderId`,
    linkKeys: { orderId: 'orderId' },
  },
});

NoSQL documents

import { nosql } from '@vaagatech/snapline-core';

const sourceStore = nosql.memory();
const targetStore = nosql.memory();

await testSuite('Document sync', {
  dbComparison: {
    sourceDb: sourceStore,
    targetDb: targetStore,
    sourceCollection: 'customers',
    targetCollection: 'snapshots',
    sourceFilter: { email: 'alice@example.com' },
    linkKeys: { id: 'customerId' },
  },
});

Step 5 — Cross-system (API ↔ DB)

API vs DB

import { api } from '@vaagatech/snapline-core';
import { createInMemoryDb } from '@vaagatech/snapline-core/examples/in-memory-db.mjs';

const appDb = createInMemoryDb([{ email: 'alice@example.com', status: 'SYNCED', role: 'member' }]);

await testSuite('Profile — API matches DB', {
  baseUrl: 'https://api.example.com',
  apiToDb: {
    api: api.rest({ endpoint: '/users/profile?email=alice@example.com', method: 'GET' }),
    db: {
      db: appDb,
      query: 'SELECT email, status, role FROM users WHERE email = :email',
      params: { email: 'alice@example.com' },
    },
    ignoreFields: ['traceId'],
    dataMapping: { status: { synced: 'SYNCED' } },
  },
});

DB vs API (inputFromDb)

await testSuite('DB row matches API', {
  baseUrl: 'https://api.example.com',
  dbToApi: {
    db: {
      db: appDb,
      query: 'SELECT email, status FROM users WHERE email = :email',
      params: { email: 'alice@example.com' },
    },
    api: api.rest({ endpoint: '/api/v1/users/profile', method: 'GET' }),
    inputFromDb: true,
    dataMapping: { status: { SYNCED: 'synced' } },
  },
});

Step 6 — Warehouse comparison (SQL → NoSQL)

For ETL pipelines that replicate many SQL tables into a document store, use runWarehouseComparison. It processes rows in chunks and streams a JSONL report — see demo project-db.

import { runWarehouseComparison, nosql } from '@vaagatech/snapline-core';

await runWarehouseComparison({
  suiteName: 'Warehouse sync',
  sourceDb,                    // DbConnectionLike — your SQL warehouse
  targetDb: nosql.memory(),   // DocumentStoreLike — e.g. MongoDB adapter
  tables: [{
    id: 'wh_customers',
    sourceQuery: 'SELECT customer_id AS customerId, email, status FROM wh_customers',
    targetCollection: 'customers',
    linkKeys: { customerId: 'customerId' },
  }],
  chunkSize: 50,
  report: { outputPath: './reports/warehouse.jsonl', format: 'jsonl' },
});

Step 6b — Queue → poll (async pipelines)

When your system processes work asynchronously — via Kafka, SQS, RabbitMQ, or an internal queue — use publishAndPoll to publish a message and poll for the result in SQL or on disk.

npm install @vaagatech/snapline-messaging-adapters

import { messaging } from '@vaagatech/snapline-messaging-adapters';
import { runPublishAndPoll } from '@vaagatech/snapline-core';

const { publisher } = messaging.memory();

await runPublishAndPoll({
  publish: {
    publisher,
    topic: 'jobs.submit',
    payload: { jobId: 'JOB-42' },
  },
  poll: { file: { directory: '/var/results' } },
  expectedFile: './fixtures/job-complete.json',
  pollOptions: { timeoutMs: 30_000 },
});

// Or poll SQL:
await testSuite('Order pipeline', {
  publishAndPoll: {
    publish: { publisher, topic: 'orders.request', payload: { orderId: 'ORD-1' } },
    poll: {
      db: {
        db: appDb,
        query: 'SELECT * FROM orders WHERE correlationId = :correlationId',
        until: (rows) => rows[0]?.status === 'PROCESSED',
      },
    },
    expected: { orderId: 'ORD-1', status: 'PROCESSED' },
  },
});

Kafka: messaging.kafkaPublisher({ brokers: ['localhost:9092'] }) (requires optional kafkajs peer).

Step 7 — Fixture-driven cases

For many scenarios (happy path, expected failures, auth errors), use fixture case runners instead of one testSuite per case.

fixtures/
└── cases/
    ├── 01-happy-path/
    │   ├── case.json
    │   ├── query.graphql
    │   ├── variables.json
    │   └── expected.json
    └── 02-wrong-status-mapping/
        ├── case.json
        └── expected.json

Scenario file:

import { fixturesDir, runApiFixtureCases } from '@vaagatech/snapline-core';

const result = await runApiFixtureCases({
  suiteName: 'Customer account — GraphQL cases',
  fixturesRoot: fixturesDir(import.meta.url, { relativePath: '../fixtures' }),
  baseUrl: process.env.API_BASE_URL,
  defaults: {
    endpoint: '/graphql',
    protocol: 'graphql',
    dataPath: 'customerAccount',
    ignoreFields: ['metadata.traceId'],
    transformations: 'accountTransforms',
    dataMapping: 'accountMapping',
  },
  presets: {
    transformations: { accountTransforms: { /* ... */ } },
    dataMapping: { accountMapping: { /* ... */ } },
  },
});

Minimal case.json:

{ "name": "Happy path", "expectMatch": true }

Failure case:

{
  "name": "Fail — wrong status mapping",
  "expectMatch": false,
  "failureType": "dataMapping",
  "expectedDiffPath": "status"
}

Offline cases (no HTTP):

import { runSnaplineFixtureCases } from '@vaagatech/snapline-core';

await runSnaplineFixtureCases({
  suiteName: 'Transformations — offline cases',
  fixturesRoot: fixturesDir(import.meta.url),
  defaults: { transformations: 'enrichment' },
  presets: { transformations: { enrichment: { /* ... */ } } },
});

Step 8 — Reports and CI

import { resolveReportConfig, writeTestReport } from '@vaagatech/snapline-core';

const result = await testSuite('User sync', { /* ... */ });

const reportConfig = resolveReportConfig({
  defaultOutputPath: './reports/integration.json',
});
if (reportConfig) {
  writeTestReport([result], reportConfig);
}

process.exitCode = result.passed ? 0 : 1;

CLI / environment:

node tests/run.mjs --report-format=html --report-output=./reports/run.html
REPORT_FORMAT=json REPORT_OUTPUT=./reports/run.json node tests/run.mjs
REPORT_REDACT_FIELDS=data,processed node tests/run.mjs

Snapline Hub (optional)

Push reports to Snapline Hub for a centralized dashboard with run history and diff visualization. Hub is optional — file-based reports above work without it.

import { buildReport, pushTestReportToHub, resolveHubConfig } from '@vaagatech/snapline-core';

const report = buildReport([result], { durationMs: 1200 });
await pushTestReportToHub(report, {
  hubUrl: 'http://localhost:3847',
  project: 'my-app',
  tags: ['node', 'ci'],
});

// Or via env / CLI:
// SNAPLINE_HUB_URL=http://localhost:3847 node tests/run.mjs
// node tests/run.mjs --hub-url=http://localhost:3847 --hub-label="CI run"

Step 9 — Auth adapters

import { auth } from '@vaagatech/snapline-core';

auth.oauth2({ tokenUrl, clientId, clientSecret });
auth.openid({ /* issuer, clientId, clientSecret */ });
auth.basic({ username, password });

Pass auth to testSuite or runApiFixtureCases. Headers are applied to all API calls. Per-case skipAuth: true in case.json tests 401 paths.