Datasets and sync.

A dataset is a named collection of records that ArdaForm holds and each device mirrors, encrypted, on its own disk. The app reads what the device holds; it writes by pushing records up through a durable outbox. Both work offline.

Working — describes ArdaPlayer v1.2.3

Reading: Arda.db.dataset(name)

app/app.js
function fields(r) { return (r && r.seq !== undefined && r.data !== undefined) ? r.data : r; }

function loadProducts() {
  return Arda.db.dataset('products').then(function (rows) {
    rows = rows || {};                       // null until anything has arrived
    return Object.keys(rows).map(function (k) { return fields(rows[k]); });
  });
}

loadProducts().then(render);
Arda.on('sync', function (name) {           // one dataset finished refreshing
  if (name === 'products') loadProducts().then(render);
});
  • It resolves what the device holds right now, as an object keyed by record key — or null when nothing for that dataset has arrived yet. Always write rows || {}.
  • What a device syncs. From v1.1.17 a device syncs every dataset the manifest declares as soon as the app is installed or updated, before the app runs. Asking for a dataset the manifest does not declare still works: the device syncs it until the next update. Up to v1.1.16 a device synced a dataset only once the app had asked for it, and then at every start. Either way, call dataset() at start-up for every dataset you need, even before you draw it; on a v1.1.16 device expect the first answer to be empty the first time it starts.
  • Re-read on 'sync'. There is no change feed of single records: read the dataset again.
  • Datasets are ArdaForm's truth. The device converges to it; nothing local is authoritative. Names follow the manifest's rule (letters, digits, . _ -).

What a record looks like

On a device, every record synced from ArdaForm is an envelope:

one value of Arda.db.dataset('products') on a device
{
  "key": "tea",
  "data": { "name": "Tea", "price": 250 },
  "deviceId": null,
  "seq": 1042,
  "createdAt": "2026-10-12T09:14:03.120Z",
  "updatedAt": "2026-10-12T09:14:03.120Z"
}
  • data is the record as it was written (by the HTTP API, the portal, or a device's push).
  • deviceId is the device that pushed it, or null for a record written through the API or the portal.
  • In a local run on your desk, seed records and records you pushed are stored as written, without the envelope. The fields() helper above reads both.

Writing: Arda.db.push(dataset, data, key)

app/app.js
function recordSale(item, qty) {
  var id = 'sale-' + Date.now() + '-' + Math.floor(Math.random() * 1e6);
  return Arda.db.push('sales', { item: item, qty: qty, at: new Date().toISOString() }, id)
    .then(function (r) { console.log('queued', id, 'seq', r.seq); return id; },
          function (e) { console.error('not queued:', e.message); throw e; });
}
  • The only way up. The record is written to a durable outbox on the device's disk and resolves {seq}, its local sequence number. It survives restarts, power cuts and days offline, and travels when the device is connected — batched, at least once, deduplicated by seq.
  • Give every record a key. The third argument is your record's key. ArdaForm refuses a record pushed without one (NO_KEY, in 'ack''s refused), developer mode reads a pushed record back only if it has one, and a key is how you recognise the record when it comes back down.
  • One record must fit in 128 KB. A bigger one is refused when you push, where your code can still act. Pictures and other bulk belong in files, not in a record.
  • Only datasets with "access": "rw" (the default) accept pushes; ArdaForm refuses a push into an "ro" dataset.
  • A push resolving means queued on this device, not delivered. From v1.2.0 the 'ack' event names your record's seq once ArdaForm has stored it, and status() counts what is still queued per dataset. Up to v1.1.17 nothing in Arda.db reports the outbox's progress.

Your own records, coming back

On a device, a record you push is not in Arda.db.dataset() straight away. It goes up through the outbox, ArdaForm stores it, and it comes back down with the dataset's next sync — if the app reads that dataset, and if the record is in the device's partition. Keep what you just did in your own variables; do not wait for the dataset to show it. (Developer mode's local echo shows it at once, which is the one place the desk and the device differ here.)

Arda.db.status()

app/app.js
Arda.db.status().then(function (s) {
  // s.ready                    the datasets the app waits for are current
  // s.connected, s.since       the control connection right now, and since when (ms)   from v1.2.0
  // s.datasets.products        {synced: true, boot: true, records: 412}
  // s.datasets.orders          {..., pending: 2, lastAckAt: 1760950000000, syncedAt: ...}   from v1.2.0
  // s.files                    {done, total, failed, complete, bootDone, bootTotal, ...} or null
  document.body.classList.toggle('syncing', !s.ready);
});
It tells youIt does not tell you
ready: every dataset the app waits for has been walked to the latest since the player started.Up to v1.1.17, whether your pushed records have reached ArdaForm. From v1.2.0 each dataset's pending is how many of this device's records are still queued (zero: every write you ever made to it is up, restarts included) and lastAckAt when ArdaForm last answered one. See below.
Per dataset: synced (walked at least once since start), boot (the app waits for it), records (how many the device holds).Up to v1.1.17, whether the device is online now. From v1.2.0 connected is the control connection itself and since when it went up or down; 'device' fires when it changes. Arda.device().network is only the network, and phase is whether the device is set up.
files: the file sweep's progress, or null before the first sweep.That ArdaForm has no newer records: synced is "since start", not "this second". From v1.2.0 syncedAt says when the last walk finished, and Arda.db.sync() walks now.

In a local run on your desk nothing is walked, so ready and every synced stay false, and connected is false. Do not gate your screen on them in development; gate on the records being there.

Knowing your records arrived: 'ack', pending and sync() from v1.2.0

app/app.js
var waiting = {};                                   // seq: what we promised the customer
Arda.on('ack', function (a) {
  a.acked.forEach(function (seq) { var w = waiting[seq]; delete waiting[seq]; if (w) w.done(); });
  a.refused.forEach(function (r) { var w = waiting[r.seq]; delete waiting[r.seq]; if (w) w.failed(r.code, r.message); });
});
function takeDeposit(order, amount) {
  return Arda.db.push('tillorders', { order: order.id, deposit: amount }, order.id + '-dep-' + Date.now())
    .then(function (r) { return new Promise(function (ok, no) {
      waiting[r.seq] = { done: ok, failed: function (code, msg) { no(new Error(code + ': ' + msg)); } };
    }); });
}
// Money on an EXISTING order: are this till's earlier writes up, and are we connected?
Arda.db.status().then(function (s) {
  var d = s.datasets.tillorders || {};
  if (!s.connected || d.pending > 0) askSupervisor(); else takeDeposit(order, 20);
});
  • 'ack' fires when ArdaForm answers a batch of your pushes, with {highWater, acked, refused, pending}. acked lists the seqs that push() resolved with and that are now stored; refused lists {seq, dataset, code, message} for records ArdaForm will never hold (a read-only dataset, a record without a key) — final, and the only place you hear of it; pending is what is still queued per dataset, the same number status() shows.
  • A held record is in neither list. A push into a dataset ArdaForm does not have yet is kept on the device and retried (see the order of operations); it stays in pending and is answered later.
  • pending is the truth across restarts. The outbox is on disk, so a count of zero after the player starts means every write the app ever made to that dataset was answered. Do not guess with a timer.
  • Arda.db.sync() walks every dataset and sweeps the files now, and resolves {kicked}: false when there is no connection (the next one walks everything anyway) or you asked within the last five seconds. Nothing waits on it; 'sync' fires per dataset as always. It is a "Sync now" button, not a thing to call in a loop.
  • Up to v1.1.17 none of this exists: feature-check with typeof Arda.db.sync === 'function'.

When the app starts: needForBoot

  • When any dataset in the manifest has "needForBoot": true, the player starts the app once those are current and syncs the rest behind it.
  • When none has, every dataset the device syncs must be current before the app starts. A large dataset can then hold the first screen back: mark the small ones you need and let the rest follow.
  • From v1.1.17 it holds on a device's very first start too. The device syncs every declared dataset when the app is installed, and the boot screen waits for the needForBoot ones before it starts the app. Up to v1.1.16 it applied only to datasets the app had already asked for: on a brand-new device's first start that was none, so the app started at once on empty data. Proved in the player's tests, not yet on a device.
  • An app already running that receives an update adding a needForBoot dataset is started in the new version straight away, not sent back to the boot screen. Check Arda.db.status() before you rely on it.
  • Files the app cannot start without are named separately: files.needForBoot.
app/manifest.json datasets
{
  "datasets": [
    { "key": "shops",    "partition": "shop", "needForBoot": true },
    { "key": "prices",   "partition": "shop", "needForBoot": true },
    { "key": "catalogue" },
    { "key": "sales",    "partition": "shop", "retention": { "days": 400 } }
  ]
}

Partitions

A partitioned dataset is cut per device. Each device has one data partition (set in the portal on its Setup tab, or by the install code that registered it). A dataset's partition names the record field that is matched against it.

  • A device receives only the records whose partition field equals its partition. A record may carry a list of partitions ("shop": ["11", "15"]) and then reaches every device in any of them.
  • A record a device pushes into a partitioned dataset is filed under that device's partition, taken from the device, not from the record.
  • The manifest's top-level partition block turns the portal's partition box into a list of named choices, and can make it required — once the project's Setup has Data partition: use the app's list turned on (off by default). The choices may be written in the manifest or read from a dataset's records (devices read that form from v1.1.17).
  • Changing a device's partition or branch in the portal resyncs it: the player drops its datasets and files, shows "Configuration changed — resyncing", and starts the app again on the new data. The app is told nothing new: it is a fresh page load followed by 'sync' events.

Offline

  • Reads keep working: the device answers from its own encrypted copy.
  • Pushes keep working: they queue in the outbox and leave when the connection returns.
  • Datasets and files sync on separate drivers, so a sale never waits behind pictures.
  • A push into a dataset ArdaForm does not have yet (the manifest has not reached it) is held on the device and retried — 5 s, then 15, 30, 60 — not dropped.

On your desk

Local run (-dev)Device (or "device": true)
Which datasetsevery one in the manifest, at startevery one in the manifest (from v1.1.17; before, the ones the app had asked for)
Where records come fromseed/<dataset>.json, loaded into an empty storeArdaForm
Record shapeas writtenthe envelope
Your pushesreadable at once (records with a key)after the round trip
Partition"partition" in arda.dev.json, matched on the record's top-level fieldthe device's partition
status().readyalways falsetrue once the boot datasets are current
seed/prices.json flat records, each with its key
[
  { "key": "tea-11",  "shop": "11", "item": "tea", "price": 250 },
  { "key": "tea-15",  "shop": "15", "item": "tea", "price": 260 },
  { "key": "cake-all", "shop": ["11", "15"], "item": "cake", "price": 400 }
]

A seed may also be an object keyed by record key ({"tea-11": {…}}). It is loaded only into an empty store; delete data/ to load it again. A dataset you drop from the manifest is dropped from the local stores; the outbox is never touched.

Checked: observed in a local ardaplayer run: dataset() resolving null for an empty store, seeds in both forms, the local echo needing a key (a keyless push is queued but not readable locally — the log says so), status() staying not ready, and the shape of status(). The 128 KB refusal at push was observed too (a 140 KB record: "record exceeds the deliverable size"); the partition filter on a seed was observed with "partition": "11". The envelope, the device's partition being taken from the device row and the held-and-retried push are read from the control plane's lib/sync.js and the player's internal/dataset. A record carrying a list of partitions is matched on a desk from ArdaPlayer v1.1.17.