Dataplicity Developer Hub

The Dataplicity Developer Hub

Here you'll find announcements for new features and comprehensive guides to help you integrate Dataplicity into your Raspberry Pi projects.

Get Started    Announcements

Installing for mass production

How to bake Dataplicity into a standard build image

Dataplicity's quick one-line installation is fantastic to get started on your first few Raspberry Pi devices. But if you are assembling and imaging Pi devices in their hundreds, manually typing in the command to each Pi is pretty time consuming.

Here we discuss how to incorporate Dataplicity into your standard build image.

This installation process is most suitable for production of larger numbers of devices.

For hobbyist users, we highly recommend our standard one-line installation process described here Remotely connect to Raspberry Pi.

Prerequisites

Dataplicity is a cloud service which we will be installing as part of a first-time boot sequence for your flash image. As such, your Pi will require internet access at first boot.

Additionally, there is some assumed technical knowledge required for this process:

  • A broad understanding of how to prepare firmware images for mass deployment
  • A basic understanding of manufacturing and assembly procedures

Finally, and assuming that you are using an SDCard-based method to replicate your images, you'll need a bunch of SDCards and an SDCard reader.

Overview

There are two ways to include Dataplicity as part of a factory image:

  1. Boot the master image and install Dataplicity, delete the device registration keys, then clone the image, and re-register each device as part of a first time boot script.
  2. Install Dataplicity as part of a first time boot script.

Since both options require a first time boot script, we recommend the latter single-step process and that's what we'll cover in this tutorial.

If you install Dataplicity then clone the image, don't forget to delete the auth keys!

Every device that is registered with Dataplicity obtains a random identifier and an authorisation token. These two items uniquely identify the device to the system. Where an existing image is cloned, these two values are also typically cloned, which leads to the problem where the cloned device is confused with the original.

Should you accidentally find yourself in this position, delete the authorisation key and serial from the new device (/opt/dataplicity/tuxtunnel/auth and /opt/dataplicity/tuxtunnel/serial respectively), and re-run the installation process to re-register the device as new.

Insert a one-time installation script into your master image

In this tutorial we'll prepare a system image that contains Dataplicity installer which runs when the system is first booted.

This script will do the following at first time boot:

  1. Assign Raspberry Pi's serial number as hostname for the device. This ensures that all new devices will have a unique name on the device list.
  2. Wait for 30 se a viable network interface to be configured - typically eth0.
  3. Check if Dataplicity has already been installed (and do not re-install)
  4. Check if there is a working internet connection
  5. Download packages and register the device
  6. After successful installation delete the script and it's associated files. The only thing left will be "/var/log/mass-install-dp.log" file.

If at any point the script fails (for example because the internet connection was not viable), the installation process could be restarted simply by rebooting the device, causing the script to run again.

Create the first boot installation script

Navigate to the if-up.d scripts directory (this is the home for scripts that run after an interface is set to 'up')..

Inside that directory create a new file called mass-install-dp with your favourite editor, we'll use nano.

sudo nano mass-install-dp

Copy the script below into that file and be sure to modify ACCT_ID with the account ID in your Dataplicity account. For example, if your account ID is 12AABBAA, then we would replace EXAMPLE123 in the script below with 12AABBAA.

#!/bin/sh

ACCT_ID="EXAMPLE123"
INSTALL_URL="https://www.dataplicity.com/$ACCT_ID.py | sudo python"
LOG_FILE="/var/log/mass-install-dp.log"

#Wait maximum 30 seconds on network connectivity before giving up
limit=30
retry=0

#Don't bother to run when lo is configured
if [ "$IFACE" = lo ]; then
    exit 0
fi

#Only run from ifup
if [ "$MODE" != start ]; then
    exit 0
fi

if [ ! -e /opt/dataplicity/mass-install-hostname ]; then
    echo "Configuring hostname..." >> $LOG_FILE 2>&1

    procinfo=$(cat /proc/cpuinfo | grep Serial)
    rpi_serial=$(echo $procinfo | tr " " "\n" | tail -1)    

    if [ -z $rpi_serial ]; then
	echo "Raspberry Pi serial number not found" >> $LOG_FILE 2>&1
    else
	echo $rpi_serial | sudo tee /etc/hostname
	
	sudo sed -i '$d' /etc/hosts
	printf "127.0.0.1\t$rpi_serial\n" | sudo tee --append /etc/hosts
	
	sudo mkdir /opt/dataplicity
	sudo touch /opt/dataplicity/mass-install-hostname
	
	echo "Rebooting..." >> $LOG_FILE 2>&1
	sudo reboot
    fi
fi

if [ ! -e /opt/dataplicity/tuxtunnel/auth ]; then
    echo $IFACE >> $LOG_FILE 2>&1

    until ping -c 1 www.dataplicity.com > /dev/null ; do
        sleep 1
        retry=$(($retry+1))
        if [ $retry -eq $limit ]; then
            echo "Interface not connected and limit reached..." >> $LOG_FILE
            exit 0
        fi
    done
       
    echo "Dataplicity will now be installed..." >> $LOG_FILE 2>&1

    /bin/sh -c "curl -k $INSTALL_URL" >> $LOG_FILE 2>&1
    
    #Self deletion (cleanup)
    /bin/sh -c "sudo rm /etc/network/if-up.d/mass-install-dp /opt/dataplicity/mass-install-hostname"
fi

Your account ID is visible after login to the Dataplicity website

It is listed under "My devices" > "Add device".

Ensure the file permissions are set correctly (particularly that it is set executable).

sudo chmod 755 mass-install-dp

Clone the image

Using Windows

To clone an SDCard using Windows:

  1. Power off your Raspberry Pi and take out the SDCard.
  2. Insert the card into your computer's card reader
  3. Run Win32 Disk Image (available here https://sourceforge.net/projects/win32diskimager/)
  4. Inside "Image File" field set where your image is to be stored, for example:
  1. Press Read. Wait until finished.
Create an image from an SDCard in Windows

Create an image from an SDCard in Windows

  1. Replace the current SDCard with a blank one.
  2. Press Write and wait until the imaging is complete.

Using Ubuntu Linux

After putting the SDCard into the card reader there should be 3 removable devices that show up on the side bar - "SETTINGS", "boot" and "root". You can then use blkid to identify the device ID to which these new partitions belong.

Which should yield output similar to the following:

Display partition information

Display partition information

In the above case, the new partitions belong to /dev/sdb: Note that your device may not be the same.

Next, in addition to dd, you may also wish to install pv (progress viewer) to see the progress of dd as it takes or creates an image. Like so:

sudo apt-get install pv

To take an image of the SDCard in /dev/sdb using pv and dd:

sudo pv /dev/sdb | dd of=~/Desktop/output.img

For which the output will be something that resembles this:

Create an image to an SDCard in Linux

Create an image to an SDCard in Linux

To mint a new SDCard using your gold image, swap out the SDCard for a fresh one and swap the input and output path for the above commands. Assuming the new card is still visible as /dev/sdb, we prepare the clone as follows:

sudo pv ~/Desktop/output.img | dd of=/dev/sdb

Booting your new Pi for the first time

Connect your new SDCard to your new Pi, attach a working network connection and boot up your Pi!

At this point, if all goes well, your system will load for the first time, install Dataplicity, and register the device with your Dataplicity account.

A friendly reminder

Please ensure your Raspberry Pi has a working internet connection prior to boot.

Should the installation fail for any reason, an 'auth' key will not have been created, which means you can retry the installation by simply rebooting the device.

Validating a successful installation

The main indicator of a successful installation is the presence of a device key ("serial") and authorisation file ("auth") in /opt/dataplicity/tuxtunnel/.

Should you encounter any issues, the above installation script will create an installation log file in /var/log/install-dp.log which can be viewed with 'tail', 'less' or just 'cat'.

tail -F /var/log/mass-install-dp.log

Finally you should be able to see a device similar to the one below on your device list.

Successfully added device

Successfully added device

Installing for mass production

How to bake Dataplicity into a standard build image