# Totals Data Sync — backend contract

Totals is a privacy-first personal-finance app. Its Data Sync feature pushes
the user's records — transactions, accounts, and budgets — to a server the
user controls, as plain JSON over HTTPS. It is strictly one-way: Totals never
reads anything back. This document is the complete contract a receiving
backend must satisfy.

## 1. Overview

The backend has three jobs:

1. **Accept JSON** — `POST`, `PUT`, or `PATCH` requests (chosen per rule)
   at paths the user defines.
2. **Store idempotently** — delivery is at-least-once, so upsert on the
   identity keys below; never blind-insert.
3. **Return 2xx** — anything else means retry-later or give-up (see
   "Responses & retries").

That is a webhook receiver, not a protocol — about thirty lines in any
framework. A complete single-file reference server is included at the end.

## 2. Configuration (app side)

Data Sync lives at Settings → Advanced → Data Sync, is off by default, and
starts with an explicit consent screen. Two concepts do all the work:

- **Destinations are servers.** A base URL plus auth: an API-key header (any
  header name, `X-API-Key` by default), a bearer token, basic auth, or none.
  Secrets are kept in the platform keystore, never in the app database.
- **Rules say what goes where.** One entity per rule — transactions, accounts,
  or budgets — with optional filters (bank, account, amount range, date range,
  credit/debit, profile), the HTTP method and path, an optional field map, and
  either one record per request or bulk arrays.
- **Sends are batched, not streamed.** Rules fire manually, on every new
  transaction, when connectivity returns, on an interval (15-minute minimum),
  or at fixed times daily. Everything flows through a durable outbox drained
  in the background, so expect bursts. When a rule is first enabled, Totals
  offers to backfill everything that already matches, and a reset-and-resend
  action can replay all data from scratch.

## 3. Requests

```http
POST /transactions HTTP/1.1
Host: api.your-server.dev
Content-Type: application/json
X-API-Key: <your secret>

{ "amount": -350.0, "reference": "FT26TEST0001", ... }
```

- **Method** — `POST`, `PUT`, or `PATCH`, chosen per rule. Never `GET`
  or `DELETE`.
- **URL** — destination base URL plus the rule's path. Paths may embed record
  fields: `/transactions/{reference}` resolves to
  `/transactions/FT26189QWK1`, percent-encoded. Bulk rules cannot use
  placeholders.
- **Headers** — `Content-Type: application/json`, plus the configured auth:
  `X-API-Key: <secret>`, `Authorization: Bearer <secret>`, or
  `Authorization: Basic <base64>`.
- **Body** — one JSON object per request, or a JSON array in bulk mode,
  chunked at 500 records per request.
- **Limits** — 1 MB per request (anything larger is never sent and is marked
  failed on-device). 15-second timeout: respond fast, defer slow work.
- **Transport** — HTTPS required in release builds; localhost and private-LAN
  addresses are refused. A home server needs a tunnel (Cloudflare Tunnel,
  Tailscale Funnel) or a small VPS.
- **Probe** — the app's "Test connection" button sends `HEAD` to the base
  URL with auth headers attached; any HTTP response counts as reachable.

## 4. Responses & retries

| Your response | What Totals does |
| --- | --- |
| `2xx` | Success — the record is marked sent. The response body is ignored, except the first ~200 characters shown in the app's sync log. |
| `408`, `429`, `5xx`, timeout / network error | Retried with exponential backoff — doubling from 30 seconds, ±20% jitter, `Retry-After` honored (seconds or HTTP date). After 8 failed attempts the record is dropped. |
| Anything else (3xx, 400, 401, 403, 404, 422, …) | Dropped immediately — no retry. |

- The give-up-on-4xx rule is deliberate — a record your schema rejects should
  not loop forever — but it makes misconfigured auth expensive: every record
  sent with a wrong key is dropped. Fix the key, then use reset-and-resend on
  the Data Sync screen.
- Do not redirect. `3xx` counts as give-up — respond `2xx` directly at the
  configured URL.
- Retries also wait for the rule's next trigger or schedule window, so they
  can land later than the backoff alone suggests.
- On failure, return a short plain-text reason — it shows up in the app's
  sync log, where the user can actually read it.

## 5. Delivery semantics

**At-least-once, so upsert.** The same record will reach you more than once —
retries, edits, backfills, and reset-and-resend all resend it. Key your
writes on the identity fields:

| Entity | Identity |
| --- | --- |
| transactions | `reference` |
| accounts | `accountNumber` + `bank` |
| budgets | `id` |

**State, not events.** Payloads are built from the live record at send time.
Each arrival is the record's current state, not a change event: rapid edits
collapse into one send, and there is no ordering guarantee across records.
Model your store as a mirror, not an event log.

## 6. Payloads

The default shapes, straight from the app's models. Keys are present on every
upsert unless marked otherwise; values can be `null`. New optional fields
may be added over time — ignore keys you don't recognize.

Transaction — upsert body:

```jsonc
{
  "amount": 2500.0,               // positive = credit, negative = debit
  "reference": "FT26189QWK1",     // bank reference — the record's stable id
  "creditor": "ABEBE KEBEDE",
  "receiver": null,
  "note": "July rent",
  "time": "2026-07-01T09:30:00",
  "status": "Completed",
  "currentBalance": "12,345.67",  // a string, as parsed from the SMS
  "bankId": 1,                    // Totals' internal bank id
  "type": "CREDIT",               // "CREDIT" | "DEBIT"
  "transactionLink": null,        // bank receipt URL, when the SMS has one
  "accountNumber": "1234",        // often just the last digits from the SMS
  "categoryId": 7,                // primary category
  "categoryIds": [7, 9],          // all assigned categories
  "categoryNames": ["Rent", "Household"]  // resolved names, always present
}
// present only when set: profileId, serviceCharge, vat,
// sourceType, sourceMessageId, sourceFingerprint
```

Account — upsert body:

```jsonc
{
  "accountNumber": "1000123456789",    // full number — identity, with bank
  "bank": 1,                           // same id space as bankId above
  "balance": 12345.67,
  "accountHolderName": "ABEBE KEBEDE",
  "settledBalance": 12000.0,           // nullable
  "pendingCredit": 345.67,             // nullable
  "profileId": 1                       // present only when set
}
```

Budget — upsert body:

```jsonc
{
  "id": 3,                        // identity
  "name": "Groceries",
  "type": "monthly",
  "amount": 8000.0,
  "categoryId": 7,
  "categoryIds": [7, 9],
  "startDate": "2026-07-01T00:00:00.000",
  "endDate": null,
  "rollover": false,
  "alertThreshold": 80.0,         // percent
  "isActive": true,
  "createdAt": "2026-05-01T10:00:00.000",
  "updatedAt": "2026-06-28T18:12:00.000",
  "timeFrame": "monthly",
  "calendar": "gregorian"
}
```

**Field mapping caveat:** each rule can rename fields
(`{ totalsField: yourField }`). When a map is set, only mapped fields are
sent unless the rule opts into unmapped ones too — so a configured endpoint
may receive renamed or fewer keys than shown here. The reference server below
assumes default payloads.

## 7. Deletes

Deleting a record sends the same request — same method, same path — with a
body containing only the record's identity fields:

```jsonc
{ "reference": "FT26189QWK1" }                    // transactions
{ "accountNumber": "1000123456789", "bank": 1 }   // accounts
{ "id": 3 }                                       // budgets
```

That is also how you tell them apart: upserts always carry the full field
set, deletes only the identity. In bulk mode the same rule applies per
element — a delete rides inside the array as an identity-only object next to
full upsert objects. If you use a field map, keep the identity fields
mapped — otherwise delete bodies arrive empty and indistinguishable.

Deletes skip rule filters (the record is gone, so there is nothing to filter
on). A delete for a record you never stored is normal — treat it as a no-op
and return `2xx`.

## 8. Reference server

A complete backend in one file: zero dependencies, Node 22.13+ (built-in
`node:sqlite`), every record stored in `totals.db`. It handles both batch
modes, deletes, auth, the `HEAD` probe, and the status-code contract.

```js
// totals-sync-server.mjs — a minimal self-hosted backend for Totals Data Sync.
// Zero dependencies. Node 22.13+ (uses the built-in node:sqlite).
//
//   API_KEY=change-me node totals-sync-server.mjs
//
// Point Totals at it (Settings → Advanced → Data Sync):
//   Destination   Base URL: your public https URL
//                 Auth: API key header · Header name: X-API-Key
//   Rules         transactions → POST /transactions
//                 accounts     → POST /accounts
//                 budgets      → POST /budgets
// Works with per-record and bulk-array rules alike.

import { createServer } from "node:http";
import { DatabaseSync } from "node:sqlite";

const PORT = process.env.PORT ?? 8787;
const API_KEY = process.env.API_KEY ?? "change-me";

const db = new DatabaseSync("totals.db");
db.exec(`
  CREATE TABLE IF NOT EXISTS records (
    entity     TEXT NOT NULL,
    ref        TEXT NOT NULL,
    body       TEXT NOT NULL,
    updated_at TEXT NOT NULL DEFAULT (datetime('now')),
    PRIMARY KEY (entity, ref)
  )
`);

const upsert = db.prepare(`
  INSERT INTO records (entity, ref, body) VALUES (?, ?, ?)
  ON CONFLICT (entity, ref)
  DO UPDATE SET body = excluded.body, updated_at = datetime('now')
`);
const remove = db.prepare("DELETE FROM records WHERE entity = ? AND ref = ?");

// How Totals identifies each record. Upserts MUST be keyed on these —
// the same record can arrive more than once (retries, edits, backfills).
const identityOf = {
  transactions: (r) => r.reference,
  accounts: (r) =>
    r.accountNumber != null && r.bank != null
      ? r.accountNumber + "|" + r.bank
      : undefined,
  budgets: (r) => (r.id != null ? "budget:" + r.id : undefined),
};
const identityFields = {
  transactions: ["reference"],
  accounts: ["accountNumber", "bank"],
  budgets: ["id"],
};

// A delete arrives as the same request, but its body carries ONLY the
// identity fields. Upserts always include the full field set.
const isDelete = (entity, record) =>
  Object.keys(record).every((key) => identityFields[entity].includes(key));

const server = createServer((req, res) => {
  // Totals' "Test connection" button sends HEAD to the base URL.
  if (req.method === "HEAD") return res.writeHead(204).end();

  if (req.headers["x-api-key"] !== API_KEY) {
    // Any 4xx other than 408/429 tells Totals to give up on the record —
    // correct for bad auth. Fix the key in the app, then reset & resend
    // from the Data Sync screen to recover dropped records.
    return res.writeHead(401).end();
  }

  const [entity] = req.url.split("?")[0].split("/").filter(Boolean);
  if (!identityOf[entity] || !["POST", "PUT", "PATCH"].includes(req.method)) {
    return res.writeHead(404).end();
  }

  let raw = "";
  req.on("data", (chunk) => (raw += chunk));
  req.on("end", () => {
    let parsed;
    try {
      parsed = JSON.parse(raw);
    } catch {
      return res.writeHead(400).end("body must be JSON");
    }

    // Per-record rules send one object; bulk-array rules send an array
    // of up to 500 records.
    const records = Array.isArray(parsed) ? parsed : [parsed];
    try {
      for (const record of records) {
        const ref = identityOf[entity](record);
        if (ref == null) continue;
        if (isDelete(entity, record)) remove.run(entity, String(ref));
        else upsert.run(entity, String(ref), JSON.stringify(record));
      }
      res.writeHead(200, { "content-type": "application/json" });
      res.end(JSON.stringify({ ok: true, received: records.length }));
    } catch (error) {
      console.error(error);
      // 5xx means "try again later" — Totals retries with backoff.
      res.writeHead(500).end();
    }
  });
});

server.listen(PORT, () => {
  console.log("totals sync server listening on port " + PORT);
});
```

Run it:

```bash
# pick your runtime
API_KEY=change-me node totals-sync-server.mjs
API_KEY=change-me bun totals-sync-server.bun.mjs
API_KEY=change-me python3 totals_sync_server.py

# In another terminal — get a temporary public https URL for it:
cloudflared tunnel --url http://localhost:8787
```

`cloudflared` quick tunnels are handy while you experiment — you get a
public HTTPS URL in one command. Give the server a stable home (VPS, named
tunnel, Fly, Render, …) before relying on it.

## 9. Testing

Impersonate the app from a terminal:

```bash
curl -i -X POST https://api.your-server.dev/transactions \
  -H 'Content-Type: application/json' \
  -H 'X-API-Key: change-me' \
  -d '{
    "amount": -350.0,
    "reference": "FT26TEST0001",
    "creditor": null,
    "receiver": "Ethio Telecom",
    "note": "Airtime",
    "time": "2026-07-05T09:30:00",
    "status": "Completed",
    "currentBalance": "11,995.67",
    "bankId": 1,
    "type": "DEBIT",
    "transactionLink": null,
    "accountNumber": "1234",
    "categoryId": null,
    "categoryIds": null,
    "categoryNames": []
  }'
```

From the app: "Test connection" verifies reachability and auth wiring; then
trigger a manual sync and watch the run log — every send records your
server's status code and the start of its response.

## 10. Security & source

Data Sync is opt-in and off by default. Data goes only to servers the user
configures, over HTTPS. The engine is open source — the durable outbox,
backoff logic, and state machine live in `lib/services/data_sync` at
https://github.com/detached-space/totals, with unit tests that double as the
spec. If this document and the code ever disagree, the code wins.