Edge AI operations #

Dataplicity can operate the Linux device, deliver an inference application as OCI Software, carry typed Product events, and run a bounded Edge Process while the WAN is unavailable. You supply and qualify the inference code, model, runtime, hardware access, resource limits, and safety behaviour.

Dataplicity does not currently provide a first-class Model registry, managed inference runner, accelerator inventory, logical model slots, model-only rollout, AI-specific fleet health, or an evidence export service. Do not design a production release as though those controls exist.

Start with the availability boundary #

Need	Available path today	Boundary
Package an inference workload	Attach an OCI container to a Device Class and select it in a Software Build	The model and runtime travel inside or beside customer-owned application Software; Model is not an independent Dataplicity artefact
Use your own model	Build it into the container or provide it through storage and configuration that your product owns	Dataplicity does not validate ONNX, TensorRT, Hailo, OpenVINO, or TFLite compatibility
Use an accelerator	Grant the container the minimum device access and install the vendor runtime in the qualified image	Dataplicity does not discover accelerators or select a vendor adapter
Fall back to CPU	Implement, test, and report fallback in your application	There is no platform fallback policy or automatic fallback visibility
Act on an inference result	Publish a versioned Product event and let an Edge Process consume it	Your publisher owns schema validation, event identity, queue bounds, and model provenance
Continue while disconnected	Run a bounded Edge Process through the Product Runtime	Cloud records and integrations are unavailable to the local segment
Stage a release	Assign a Software Build to a tag cohort	This rolls the whole Build, not an independent model slot
Observe health	Publish bounded Streams/events and configure monitors; use Pulse for general peer drift	Pulse does not currently calculate model, accelerator, fallback, or inference SLO health
Retain evidence	Keep a bounded, privacy-reviewed spool in customer-owned software and export it through a customer-owned path	Product events must contain references or metadata, not image, audio, or model bytes

The model-free edge video analytics path is a shipped example of this current architecture. It deliberately requires no neural model and must not be presented as person detection, semantic vehicle detection, or evidentiary-grade number-plate recognition.

Bring your model with customer-owned Software #

Today, bring-your-model means bringing the complete qualified workload:

Select a model format supported by the runtime in your product image.
Pin the model bytes, labels, preprocessing, postprocessing, and runtime versions together.
Record the model digest and application version in every typed result.
Build and test one OCI image for each supported CPU architecture and accelerator family.
Publish the image as Class Software, compose a Software Build, and target a bounded cohort.

Dataplicity does not train, annotate, convert, quantise, benchmark, or judge the accuracy of a model. Accuracy and fitness for a customer scene remain part of your product qualification.

Managed and customer-owned runtime boundary #

The supported path is a customer-owned runtime in your daemon or container. It may use ONNX Runtime, TensorRT, HailoRT, OpenVINO, TFLite, or another runtime that you install and qualify. Dataplicity supplies no managed adapter for those runtimes today.

Keep the runtime least-privileged:

no Dataplicity cloud credentials;
no Docker or containerd socket;
read-only access to model artefacts and input sources where practical;
only the accelerator device nodes needed by the workload;
explicit limits for memory, concurrency, input rate, queue depth, and inference timeout;
a bounded restart policy and a visible degraded state.

Treat accelerator selection and fallback as product state #

Probe accelerator and runtime compatibility before starting inference. Keep the states distinct: accelerator absent, driver missing, runtime incompatible, accelerator unhealthy, and telemetry unknown.

If CPU fallback is allowed, declare it in product configuration and include execution_target, fallback_active, and fallback_reason in health and result events. Never fall back silently. If fallback cannot meet the latency or thermal envelope, fail closed into an explicit unavailable state.

There is no Dataplicity accelerator capability inventory today. Use Device Class boundaries and cohort tags to avoid targeting incompatible hardware, and verify the actual device before expanding a rollout.

Keep application bindings stable #

A logical model slot such as vehicle-detection is the intended abstraction for changing a model without changing application code. Dataplicity does not currently resolve or atomically advance logical model slots.

Until that lifecycle exists, keep a stable model key in your own application configuration, but treat its bytes as part of the Software release. Do not point production code at an unversioned filename or mutable public URL. Record the exact digest loaded by the runtime and compare it with the digest expected by the image or configuration.

Publish typed Product events #

Use a stable event type and a versioned payload. A useful result contains:

deterministic event and correlation identifiers;
Device Class, device, and source identity;
source capture timestamp and inference completion timestamp;
application, model, model digest, and runtime versions;
execution target and explicit fallback/degraded state;
a typed result with honest unknown or uncertain values;
an evidence reference when your own evidence service has retained a sample;
simulated: true for every fixture-generated result.

Structured events are authoritative for operational decisions. Project selected low-cardinality numbers such as latency, queue depth, and result rate into Streams for charting and monitors. Do not put images, audio, video, model bytes, or unbounded high-cardinality output in Product events.

The Product Runtime and Edge Process vocabulary supports waiting for a Product event, branching, invoking a typed local Command, waiting for its result, and emitting a follow-up event. It does not run arbitrary inference, Python, shell, HTTP, hardware drivers, or cloud integrations.

Run the decision offline #

An offline path should be:

the customer-owned runtime publishes a deterministic Product event locally;
an Edge Process waits for that exact event type;
the Process checks only data and capabilities available on the device;
it invokes a typed OEM Command;
the OEM handler revalidates authority, deduplicates the invocation, applies physical interlocks, and reports the real outcome;
events and outcomes reconcile after connectivity returns.

Loss of WAN does not move cloud work to Edge. Keep emergency stops, hard real-time loops, protective controls, and the only copy of an authority rule below the Process in qualified product software.

A deterministic current-format example is available as edge-ai-offline-process.json. It uses the shipped model-free edge.vision.observation event so it can be tested without inference hardware.

Roll out and reverse safely #

Use tag cohorts for staging devices, hardware revisions, sites, and canaries. Before changing a cohort:

prove the image on representative CPU and accelerator hardware;
include an incompatible-device test and a no-accelerator test;
verify model digest, runtime load, first result, latency, queue, and source freshness from application-owned telemetry;
define a soak window and a manual stop decision;
retain a known-good Build and prove its data/configuration compatibility;
expand only after inspecting exact devices, including offline and unknown members.

Dataplicity does not currently pause a model rollout on AI health regression or roll back one model independently. To reverse today's path, assign a newer Build containing the known-good application and model combination. A Software Build change does not itself reverse schema, database, or physical side effects.

Operate fleet health honestly #

Publish an explicit inference service state such as healthy, degraded, unavailable, or unknown. Useful reasons and metrics include input age, model-load failure, runtime restarts, queue saturation, dropped inputs, result rate, error rate, latency percentiles, execution target, fallback reason, and accelerator temperature where your runtime can measure it.

Use monitors for declared thresholds and Device Class Pulse for general peer-relative host drift. Pulse can help find an unusual process, package, resource posture, or software version. It does not currently understand model accuracy or provide AI-specific cohort SLOs. Never turn missing telemetry into zero, or a confidence shift into an accuracy claim.

Bound evidence and protect privacy #

Evidence capture is customer-owned today and should default to off. If enabled:

set byte, item, age, and rate quotas;
define deterministic eviction under disk pressure;
capture only the required region or time window;
redact or mask identifiers before upload where required;
keep the event payload to metadata and a customer-owned reference;
audit access and export in the system that stores the evidence;
keep original inference output separate from later operator feedback.

Images, audio, number plates, faces, and other identifiers may be personal data. Dataplicity does not currently provide an AI evidence spool, feedback UI, or export destination.

Continue with a reference workload #

Use the five Edge AI reference workloads to qualify the same operating path for object detection, vehicle/plate workflow, occupancy, visual inspection, and non-vision sensor inference. Only the model-free video analytics workload is supplied today; the model-backed workloads are deterministic acceptance patterns, not bundled applications.