Skip to main content

How to Create a DIP

The LabScale Agent is a locally running service that constantly monitors the state of devices, whether via USB/Serial for those connected to the localhost, SSH for networked devices, or some other mechanism, it does not matter. The agent makes no assumptions on how these devices function, or how to connect to them, rather it relies on a hardware abstraction that provides these functions for the agent. These abstractions are known as Device Integration Packages, or DIPs for short. They are simply an archive that contains code and configurations required to probe a particular type of device. The configuration file contained within the package describes the hooks that the Agent can call to acquire information from the device.

This guide will lead you through the process of creating a custom DIP.

Terminology​

  • Agent - A locally running service that executes jobs on behalf of the LabScale service.
  • Device - The hardware under test.
  • Host - The machine that an agent is running on.
  • DIP - Device Integration Package, provides hooks required by the agent for probing target devices.
  • YAML - A standardized, human readable, data serialization file format. Visit the YAML page on The Wikipedia for more information.

Anatomy of a DIP​

The LabScale automation platform does not limit DIPs to any particular programming language or framework. The only requirements is that the package contain a dip.yaml file that describes the entry points required by the agent to interrogate the target hardware, and 1 or more executables that do the interrogate.

DIPs have direct relationship to the type of hardware being tested and are assigned to the target at the time a device is created in the LabScale service. If you have not created a device, please review the Quick Start guide.

The Call-in Hooks​

The purpose of a DIP is to interrogate a target device or piece of hardware on behalf of the Agent. To do this, we must provide executable call-in hooks that the Agent can call and that will return data required by LabScale service for monitoring and scheduling. These hooks can be in the form of executable scripts or command lines, the only requirement is that they print out structured data as JSON strings that can be read by the Agent.

There are 3 primary hooks, get_state, get_status, and two optional hooks, dip_install and get_metrics. Each hook performs a very specific function and returns data in the form of a JSON string that is interpreted by the Agent and relayed to the server.

  • get_status - Simply gets the online or offline status of the device. It is intended to be a non-invasive operation. The response must be JSON in the form:
{ "status": "online" }

There are two supported states, online and offline.

  • get_state - Similar to get_status but is not required to be non-invasive and returns more information about the device, such as platform name, OS version, etc. The response must be JSON in the form:
{
  "status": "online",
  "serialNumber": "serial no",
  "softwareVersion": "version"
}

There are two supported states, online and offline.

  • dip_install - An optional hook to install any dependencies required by the DIP for proper execution. This is called only once after a DIP is initially downloaded from LabScale.

  • get_metrics - An optional hook that provides device metrics or telemetry to the agent and ultimately the server. The implementation for this is too large for a simple guide therefore we will cover how to implement this hook in the How to Add Metrics to a DIP guide.

The Agent also provides device configuration data as environment variables or as a JSON string that can be passed to the executables as a command line parameter.

The Executable(s)​

A dip can contain one or more executable resources which are called by the agent and perform one or more of the operations required by the call-in hooks. DIPs can be written in any language, they simply need to print their output in a format that can be read by the agent when it calls the desired hook. In this example, we will be writing a DIP for a 'virtual device' in the Python programming language.

Implement the get_status hook​

First, we want to implement the get_status hook. The get_status function is intended to be a non-invasive check of the device status and only returns the current state. Using the previous dip.py example from above, update it and add the get_status() function along with argument and parameter parsing:

import json
import os

def get_status() -> dict:
    """
    The get_state function will return an online or offline
    state of the virtual device.  Since a virtual device is
    shell on the local host, we will assume that it will
    always be online.
    """
    return {"status": "online"}


if __name__ == "__main__":
    from argparse import ArgumentParser

    # For simplicity, we will create a direct
    # mapping between the name of the command
    # and the function that will carry it out.
    commands = {
        "get_status": get_status, # Add get_status function
    }

    # Read the 'get_status' command from the command line.
    parser = ArgumentParser()
    parser.add_argument("command", nargs=1, choices=commands.keys())
    args = parser.parse_args()

    data = commands[args.command[0]]()
    if data:
       print(json.dumps(data))

To test this, save the code to a file named dip.py. Open a terminal window in the same folder as the file and type:

% python3 dip.py get_status
{"state": "online"}
%

And then update the dip.yaml file:

name: example_dip
version: 1.0.0
commands:
  get_status: "python3 ${DIP_ROOT}/dip.py get_status"

Notice that the get_status hook has ${DIP_ROOT} in the invocation string, this will be replaced by the absolute path to the DIP at runtime. Also of note, the current working directory will also be automatically set to the same folder thus allowing relative paths instead.

Implement the get_state hook​

The get_state hook is very similar to get_status but is allowed to be invasive to the device. Not only does this function return state, but it is allowed to return essentially any piece of data about the device. Using the previous dip.py example from above, add the get_state() function and add it to the list of recognized commands:

...

def get_state() -> dict:
    """
    The get_state call is an invasive function that
    probes the device and returns not only status,
    but any other bits of platform metadata.
    """
    import platform

    data = get_status()
    data.update({
        'serialNumber': platform.node(),
        'softwareVersion': platform.release(),
    })
    return data


if __name__ == "__main__":
    from argparse import ArgumentParser

    commands = {
        "get_status": get_status,
        "get_state": get_state, # Add get_state
    }

...

To test this, save the code to a file named dip.py. Open a terminal window in the same folder as the file and type:

% python3 dip.py get_state'
{"state": "online", "serialNumber": "binbox.attlocal.net", "softwareVersion": "22.6.0"}
%

And then update the dip.yaml file and add the get_state hook:

name: example_dip
version: 1.0.0
commands:
  get_status: "python3 ${DIP_ROOT}/dip.py get_status"
  get_state: "python3 ${DIP_ROOT}/dip.py get_state"

Create the DIP package​

Now that we have functional code, we need to create a DIP bundle that can be distributed to Agents whenever required to probe a type of device your DIP was create for. To do this, we will create a Gzipped Tar archive.

tar czf example_dip.tgz dip.yaml dip.py

Create a Device Type​

  1. In the LabScale home screen, select the Labs & Devices menu item in the navbar on the left side of the page.
  2. Once in the Labs & Devices page, select the Device Types menu item in the nav bar.
  3. Click on the ADD DEVICE TYPE button in the top right corner.
  4. Enter a Name of the device type and the name of the DIP, in this case example.
  5. Click on the SUBMIT button.

Upload the DIP package​

Now that we have a DIP, we need bind it to a device type.

  1. In the LabScale home screen, select the Labs & Devices menu item in the navbar on the left side of the page.
  2. Once in the Labs & Devices page, select the Device Types menu item in the nav bar.
  3. Again on the Device Types page, click on the name of the device type you created above.
  4. In the Details page, click on the UPLOAD DIP.
  5. Click on the Upload box and select the example_dip.tgz package you created from the previous examples, or drag and drop it onto the box.
  6. Click on the Submit button.

The DIP environment​

When DIP hooks are invoked, the agent will provide environment variables that can be used within the DIP code. These environment variables come from the Environment Configuration property of the target device which was set at the time the device was created, but the specification is defined within the Device Type. In this example we describe how to create a new Device Type with custom Config variables and how to use them within the DIP.

How to use the environment variable in code​

We will update the previous example DIP to require an ip_address variable defined by the device. At run time, the agent will transform the name to LS_DIP_ip_address and put it into the task environment when calling the dip. We will update the code to check if the ip_address is available by making an HTTP request:

...
from http import client

def get_status() -> dict:
    """
    The get_state function will return an online or offline
    state of the virtual device.
    """
    # In a real world implementation, the ip_address
    # would be used to connected to the device with SSH.
    conn = client.HTTPSConnection(os.environ["LS_DIP_ip_address"], timeout=5)
    conn.request("HEAD","/")
    resp = conn.getresponse()
    if resp.status == 200:
        return {"status": "online"}
    else:
        return {"status": "offline"}
...

Create a Device Type​

Now we need to define a new device type.

  1. In the LabScale home screen, select the Labs & Devices menu item in the navbar on the left side of the page.
  2. Once in the Labs & Devices page, select the Device Types menu item in the nav bar.
  3. Click on the ADD DEVICE TYPE button in the top right corner.
  4. Enter a Name of the device type and the name of the DIP, in this case example.
  5. Click on the SUBMIT button.

Update the Device Type with an environment specification​

Then we must configure a DIP environment specification. This environment specification defines the form that is provided to the user during device creation where they can input the values of the environment varaibles.

  1. In the LabScale home screen, select the Labs & Devices menu item in the navbar on the left side of the page.
  2. Once in the Labs & Devices page, select the Device Types menu item in the nav bar.
  3. On the Device Types page, click on the name of the device type you created above.
  4. In the Details page, click on the CONFIGURE DIP ENV SPEC.
  5. Once the modal dialog panel appears, click on the ADD FIELD button.
  6. Enter the name of variable, for this example we will use ip_address.
  7. Next the type of value this field will accept, select String.
  8. Click on the Submit button.

Upload the DIP package​

Now that we have a DIP, we need bind it to a device type.

  1. In the LabScale home screen, select the Labs & Devices menu item in the navbar on the left side of the page.
  2. Once in the Labs & Devices page, select the Device Types menu item in the nav bar.
  3. Again on the Device Types page, click on the name of the device type you created above.
  4. In the Details page, click on the UPLOAD DIP.
  5. Click on the Upload box and select the example_dip.tgz package you created from the previous examples, or drag and drop it onto the box.
  6. Click on the Submit button.

Create a device​

Once we have a device type, we can now create a device of that type.

  1. In the LabScale home screen, select the Labs & Devices menu item in the navbar on the left side of the page.
  2. Once in the Labs & Devices page, select the Devices menu item in the nav bar.
  3. Click on the ADD DEVICE button.
  4. Enter the Name of the device.
  5. Select which agent will be monitoring and executing tests on the device.
  6. Select a Pool if one is available.
  7. Select the example device type.
  8. You should also notice that in the Environment Configuration on the modal dialog, there is a field labeled ip_address, enter an IP address. For this example, use www.python.org.
  9. Click on the SUBMIT button.

The device you created above should now appear as online. If you create a new device and enter an ip_address to some other location, for instance one that does not exist or does not have an index.html, it will appear offline. In a real world scenario, the ip_address could be used to connect to a device with SSH, HTTP, or perhaps even telnet.