Integrate Dataplicity with your Linux image #

Most products keep the distribution, BSP, kernel, bootloader, service manager, and application runtime they already ship. Dataplicity OS is a CM5 quickstart and a worked reference; it is not a requirement to adopt a Dataplicity-owned distribution.

This page is the accepted image contract. Use it when you bring your own OS. Use Dataplicity OS only when the ready-made CM5 image is a useful starting point.

Choose only the components you need #

The Dataplicity agent, Class Software containers, and RAUC image management are separate choices. You can use any row in this table; installing the agent does not require you to adopt the other two components. Class Software does not require RAUC, and RAUC does not require Class Software.

Product path	Required on the device	Not required
Remote access and fleet operations only	Rescue agent, outbound HTTPS, accurate time, and durable identity storage	Container runtime, supervisor, RAUC
Streams, settings, and actions from your existing process	Rescue agent plus supervisor, local broker, durable agent state, and RAM-backed volatile storage	Container runtime and RAUC
Dataplicity-managed Class Software	Full agent, a qualified OCI runtime, matching CPU architecture, persistent container store, local-broker networking, and the required narrow runtime privileges	RAUC
Dataplicity-managed RAUC OS images	Full agent in both rootfs slots, qualified RAUC A/B boot chain, persistent data outside both slots, and rollback evidence	Container runtime
Class Software and RAUC together	All container and RAUC requirements; runtime state and container storage must remain outside both rootfs slots	Nothing from the two rows may be omitted

Dataplicity can also coexist with a container or OS updater that you already operate. In that case, keep its ownership and release path rather than enabling the corresponding Dataplicity-managed component.

Choose the agent mode #

First identify the generation. New product integrations use the private modern agent release path. The public dataplicity 0.5.x source is the legacy remote-access agent and is not the source used to build Dataplicity OS. See Choose a Dataplicity agent generation.

Mode	Use it for	Minimum runtime
Rescue only	Device identity, Remote Shell, Wormhole, and file retrieval	Python 2.7 or newer, outbound HTTPS, accurate time
Rescue plus supervisor	Streams, settings, actions, Class Software, and managed RAUC integration	Python 3.8 or newer with `pip` and `venv`

Published modern management packages cover x86_64, aarch64, and armv7. Qualify the exact architecture, Python ABI, agent release, and image together.

A container runtime is optional. Qualify its CLI and privilege model only when the product uses Class Software. Dataplicity OS uses containerd with nerdctl, not Docker Engine; your own image may use another supported runtime.

Network and operating requirements #

The image must provide:

trusted CA certificates, DNS, and outbound HTTPS;
an accurate clock, normally through NTP;
a writable persistent state location;
a RAM-backed location for non-durable agent runtime data;
service ordering that waits for the network and persistent storage;
enough process, storage, and memory capacity for the selected agent mode.

Normal Dataplicity access needs no inbound port. See Firewall requirements for restricted customer networks.

Filesystem and service layout #

The modern full-agent convention is:

text

/opt/dataplicity/venv
    Rescue, supervisor, and fixed dp-converge helper

/var/lib/dataplicity
    Device identity, durable state, cache, features, config/current,
    image-update state, and persistent managed-container storage

/run/dataplicity
    RuntimeDirectory for the supervisor (tmpfs via systemd)

/tmp/dataplicity
    Default volatile_dir for non-durable host scratch (must be on tmpfs)

/etc/dataplicity/agent.yaml
    Agent configuration; symlink into persistent state on A/B images

/usr/local/bin/
    dataplicity, prelude-agent, dp-provision, dp-supervisor,
    dp-converge, and dp-rauc-ready entry points

On a conventional single-root image, /var/lib/dataplicity can be an ordinary directory. On an A/B image it must be a separate persistent mount outside both root filesystem slots.

Keep /opt/dataplicity/venv in every signed rootfs slot. An OS rollback then restores the matching agent runtime. Do not move executables onto the persistent data partition.

If Class Software is used on an A/B product, keep the container store, config/current, and content-addressed cache on the persistent mount as well.

Flash wear and volatile storage #

Dataplicity expects non-durable work to stay in memory. Do not point high-rate agent scratch, default stream buffering, or Faceplate status snapshots at eMMC, NAND, or other wear-limited media unless you deliberately opt in.

Required RAM-backed paths #

Path / setting	Role	Requirement
`/run/dataplicity`	Supervisor runtime directory	Provided by systemd `RuntimeDirectory=dataplicity`, or an equivalent tmpfs mount owned for the agent users
`volatile_dir` (default `/tmp/dataplicity`)	Non-durable host scratch configured in `agent.yaml`	Must resolve onto `tmpfs` or `ramfs`. The agent gates this path; do not quietly relocate it to disk
Default stream buffer	High-rate telemetry buffer	Defaults to RAM. Durable SQLite buffering is an explicit OEM choice

Faceplate and related status publishers also refuse to write snapshots unless the target directory is on tmpfs or ramfs. That is intentional: status pixels must not become a flash-write path.

Configure the agent storage map #

Edit /etc/dataplicity/agent.yaml. On a standard installation this is a symlink to the durable file at /var/lib/dataplicity/state/agent.yaml.

yaml

# Agent identity, desired state, caches, image-update state, and other data
# which must survive a reboot or RAUC slot change.
data_root: /var/lib/dataplicity

# Live-only stream samples and other high-rate, no-retention scratch data.
# This path must be backed by tmpfs or ramfs.
volatile_dir: /tmp/dataplicity
volatile_allow_disk: false

# Optional. Set this only when Device Class streams configured to Keep history
# must be buffered across a reboot or long network outage.
# durable_mount: /mnt/oem-data/dataplicity-durable

The Device Class stream History setting decides which buffer the agent uses:

Stream History	Agent storage	Behaviour
Live only	Process memory, with bounded spill under `volatile_dir` when it is RAM-backed	No flash writes for samples. Data may be lost on reboot or when the bounded offline buffer fills
Keep history with no `durable_mount`	Volatile buffer	Fails flash-safe: the agent warns and does not silently start writing samples to flash
Keep history with `durable_mount` set	SQLite under the selected persistent mount	Survives reboot subject to the configured bounds; the OEM owns capacity and media-endurance qualification

Do not set durable_mount merely because /var/lib/dataplicity is persistent. It is an explicit decision to let retained stream samples write to the selected medium. A storageProfile value sent by a publisher is ignored; configure retention on the Device Class stream and configure physical storage in the agent.

After mounting the paths and before enabling the services, verify the actual filesystems rather than the directory names:

findmnt -T /var/lib/dataplicity
findmnt -T /tmp/dataplicity
dataplicity status

The first path must be on the intended durable filesystem. The second must report tmpfs or ramfs. On a managed RAUC product, also run dataplicity rauc-ready.

What may live on flash #

Persist only what must survive reboot or an A/B slot change:

device identity and enrolment state;
config/current and other desired-state material the product requires offline;
content-addressed caches and managed container images when Class Software is used;
write-on-change identity, limits, and status files the agent already treats as durable.

Do not:

turn every high-frequency sample into a synchronous durable write;
put unbounded logs or scratch under /var/lib/dataplicity;
assume a writable overlay root is durable because a lab reboot succeeded;
disable the tmpfs gate on volatile_dir without recording the flash-wear consequence for the product.

See Telemetry delivery guarantees and What runs on my device for the durability design.

Journal and logs #

If you keep a persistent journal for first-boot diagnosis, cap it. Dataplicity OS uses a small persistent journal on the active rootfs slot (Storage=persistent with bounded SystemMaxUse / RuntimeMaxUse). Prefer journald or another capped sink over an ever-growing file on flash.

Containers when you use Class Software #

Skip this section when the product does not run Class Software. Missing runtime is a skipped feature, not a failure to fake Docker.

When Class Software is used, the image must provide:

Requirement	Accepted shape
Runtime	A supported OCI runtime the supervisor can drive. Dataplicity OS uses `containerd` + `nerdctl`; other images may qualify a different supported CLI
Durable store	Images, layers, snapshots, and client state on persistent storage, not on an A/B rootfs slot
Slot-local binaries	Runtime binaries in each signed rootfs; mutable state on the persistent mount
Privilege boundary	Narrow elevation for the supervisor user only. Dataplicity OS uses allowlisted `nerdctl` / `ctr` sudo wrappers for `dpagent`; do not grant unrestricted root or setuid clients
Networking	Host networking is the Dataplicity OS default so containers can reach the local broker; qualify any other model before shipping

The container itself must also match the Device Class architecture, start without interactive setup, tolerate restarts and broker/network outages, and run without Dataplicity cloud credentials. Expose only the serial devices, GPIO, sockets, files, and other host interfaces it actually needs. Qualify its rollback against any durable application data it changes.

Reference layout from Dataplicity OS:

text

/var/lib/dataplicity/containerd   containerd root
/var/lib/dataplicity/nerdctl      nerdctl data_root
/var/lib/dataplicity/home/dpagent supervisor HOME
/var/lib/containerd               symlink → persistent containerd root

containerd must start only after the persistent mount is ready. Do not point containerd root at a path that RAUC will replace.

See Dataplicity OS software runtime for the worked CM5 example.

Linux users and privileges #

The rescue service runs as dataplicity. The supervisor runs as dpagent. Grant container-runtime group membership only when the product uses that runtime.

Managed RAUC uses the fixed /opt/dataplicity/venv/bin/dp-converge privileged helper for install, mark-good, and reboot. Runtime operations should follow the same allowlist principle. Do not replace these boundaries with blanket passwordless sudo or configurable executable paths.

Provision each device #

In the Dataplicity app, select Add device and run the complete generated command on the target image. The command contains organisation-specific provisioning material; never put it in a public image or repository.

For production:

bake the agent packages into the image or make them available to a controlled first-boot step;
provision each physical unit independently during factory, first-boot, or site commissioning;
preserve the resulting identity under /var/lib/dataplicity;
remove temporary provisioning material;
confirm the unique device in inventory.

Clone the base image before registration, not after. See Installing for mass production.

Add RAUC only when you need managed OS images #

Most Dataplicity devices do not need RAUC. If your product already has another OS update mechanism, keep it unless you deliberately choose to qualify RAUC.

For Dataplicity-managed RAUC delivery, the image also needs:

two real A/B root filesystem slots;
RAUC connected to the actual bootloader;
bounded boot-attempt counting and automatic fallback;
a product-specific compatible value;
a public verification CA baked into the RAUC keyring;
the Dataplicity runtime in every slot;
/var/lib/dataplicity and managed application state outside the slots;
physical failed-boot and power-loss qualification.

RAUC and Class Software remain independent release lines. A RAUC slot change does not select a new container Build, and a container rollout does not change the active OS slot. When you use both, /var/lib/dataplicity, the container store, config/current, and any selected durable_mount must remain available across either kind of update.

Use dataplicity rauc-ready to check what userspace can prove. Use Build OS firmware and configure RAUC signing to create the signing trust and connect it to a Device Class.

The device keyring and Device Class keyring normally contain the same public CA but are checked in different places: RAUC checks the bundle on the device, while Dataplicity checks it before delivery. Private signing keys belong in the protected release environment only.

Custom Yocto, Buildroot, and unusual platforms #

Custom embedded Linux is expected integration work. Email support@dataplicity.com with:

distribution or build system and release;
CPU architecture and Python version;
init/service manager;
read-only and persistent filesystem layout;
how /tmp, /run, and any volatile_dir are backed;
container runtime, when used;
OS updater and bootloader design;
network and certificate constraints.

We can help map the agent runtime, CA certificates, users, permissions, services, connectivity, persistent paths, flash-wear policy, and RAUC integration to the product.

Qualification checklist #

Before a fleet rollout:

install on the exact production-equivalent image and hardware;
confirm DNS, HTTPS, and time synchronisation;
verify the device enrols and reconnects after cold boot and network delay;
confirm its identity does not change;
confirm /tmp/dataplicity (or the configured volatile_dir) is on tmpfs;
confirm /run/dataplicity exists as a runtime tmpfs directory;
test Remote Shell with the intended Linux permissions;
verify the supervisor and container deployment when used, including durable container storage off the A/B rootfs;
test agent and OS upgrades while preserving state;
repeat qualification when the base image, architecture, service policy, or boot chain changes.