Blocks
The blocks CLI tool creates different blocks types that are used in organizational features such as:
  • Transformation blocks - to transform large sets of data efficiently.
  • Deployment blocks - to build personalized firmware using your own data or to create custom libraries.
  • Custom DSP blocks - to create and host your custom signal processing techniques and use it directly in your projects.
  • Custom learning models - to use your custom neural networks architectures and load pretrained weights.
With the blocks CLI tool, you can create new blocks, run them locally, and push them to Edge Impulse infrastructure so we can host them for you. Edge Impulse blocks can be written in any language, and are based on Docker container for maximum flexibility.
As an example here, we will show how to create a transformation block.
You can create a new block by running:
1
$ edge-impulse-blocks init
2
? What is your user name or e-mail address (edgeimpulse.com)? [email protected]
3
? What is your password? [hidden]
4
? In which organization do you want to create this block? EdgeImpulse Inc.
5
Attaching block to organization 'EdgeImpulse Inc.'
6
? Choose a type of block Transformation block
7
? Choose an option Create a new block
8
? Enter the name of your block Extract voice
9
? Enter the description of your block Extracts voice from video files
10
Creating block with config: {
11
name: 'Extract voice',
12
type: 'transform',
13
description: 'Extracts voice from video files',
14
organizationId: 4
15
}
16
? Would you like to download and load the example repository (Python)? yes
17
Template repository fetched!
18
Your new block 'Extract voice' has been created in '/Users/janjongboom/repos/custom-transform-block'.
19
When you have finished building your transformation block, run "edge-impulse-blocks push" to update the block in Edge Impulse.
Copied!
When you're done developing the block you can push it to Edge Impulse via:
1
$ edge-impulse-blocks push
2
Archiving 'custom-transform-block'...
3
Archiving 'custom-transform-block' OK (2 KB)
4
5
Uploading block 'Extract voice' to organization 'EdgeImpulse Inc.'...
6
Uploading block 'Extract voice' to organization 'EdgeImpulse Inc.' OK
7
8
Building transformation block 'Extract voice'...
9
INFO[0000] Retrieving image manifest python:3.7.5-stretch
10
INFO[0000] Retrieving image python:3.7.5-stretch
11
12
...
13
14
Building transformation block 'Extract voice' OK
15
16
Your block has been updated, go to https://studio.edgeimpulse.com/organization/4/data to run a new transformation
Copied!
The metadata about the block (which organization it belongs to, block ID) is saved in .ei-block-config, which you should commit. To view this data in a convenient format, run:
1
$ edge-impulse-blocks info
2
Name: TestDataItemTransform
3
Description: Data item transformation example
4
Organization ID: 1
5
Not pushed
6
Block type: transform
7
Operates on: dataitem
8
Bucket mount points:
9
- ID: 1, Mount point: /path/to/bucket
Copied!

Block runner

Rather than only running custom blocks in the cloud, the edge-impulse-blocks runner command lets developers download, configure, and run custom blocks entirely on their local machine, making testing and development much faster. The options depend on the type of block being run, and they can be viewed by using the help menu:
1
$ edge-impulse-blocks runner -h
2
Usage: edge-impulse-blocks runner [options]
3
Run the current block locally
4
Options:
5
--data-item <dataItem> Tranformation block: Name of data item
6
--file <filename> File tranformation block: Name of file in data item
7
--epochs <number> Transfer learning: # of epochs to train
8
--learning-rate <learningRate> Transfer learning: Learning rate while training
9
--validation-set-size <size> Transfer learning: Size of validation set
10
--input-shape <shape> Transfer learning: List of axis dimensions. Example: "(1, 4, 2)"
11
--download-data Transfer learning or deploy: Only download data and don't run the block
12
--port <number> DSP: Port to host DSP block on
13
--extra-args <args> Pass extra arguments/options to the Docker container
14
-h, --help display help for command
Copied!
As seen above, the runner accepts a list of relevant option flags along with a variable number of extra arguments that get passed to the Docker container at runtime for extra flexibility. As an example, here is what happens when edge-impulse-blocks runner is used on a file transformation block:
1
$ edge-impulse-blocks runner --data-item item1 --file sample_1.cbor
2
Found data item item1 with id=1, metadata={}
3
Downloading file sample_1.cbor to /path/to/block/data/dataset_1/item1...
4
File downloaded
5
...
Copied!
Best of all, the runner only downloads data when it isn't present locally, thus saving time and bandwidth.
1
$ edge-impulse-blocks runner --data-item item1 --file sample_1.cbor
2
Found data item item1 with id=1, metadata={}
3
File already present; skipping download...
4
...
Copied!

Block structure

Transformation blocks use Docker containers, a virtualization technique which lets developers package up an application with all dependencies in a single package. Thus, every block needs at least a Dockerfile. This is a file describing how to build the container that powers the block, and it has information about the dependencies for the block - like a list of Python packages your block needs. This Dockerfile needs to declare an ENTRYPOINT: a command that needs to run when the container starts.
An example of a Python container is:
1
FROM python:3.7.5-stretch
2
3
WORKDIR /app
4
5
# Python dependencies
6
COPY requirements.txt ./
7
RUN pip3 --no-cache-dir install -r requirements.txt
8
9
COPY . ./
10
11
ENTRYPOINT [ "python3", "transform.py" ]
Copied!
Which takes a base-image with Python 3.7.5, then installs all dependencies listed in requirements.txt, and finally starts a script called transform.py.
Note: Do not use a WORKDIR under /home! The /home path will be mounted in by Edge Impulse, making your files inaccessible.
Note: If you use a different programming language, make sure to use ENTRYPOINT to specify the application to execute, rather than RUN or CMD.
Besides your Dockerfile you'll also need the application files, in the example above transform.py and requirements.txt. You can place these in the same folder.

Excluding files

When pushing a new block all files in your folder are archived and sent to Edge Impulse, where the container is built. You can exclude files by creating a file called .ei-ignore in the root folder of your block. You can either set absolute paths here, or use wildcards to exclude many files. For example:
1
a-large-folder/*
2
some-path-to-a-text-file.txt
Copied!

Clearing configuration

To clear the configuration, run:
1
$ edge-impulse-blocks --clean
Copied!
This resets the CLI configuration and will prompt you to log in again.

API Key

You can use an API key to authenticate with:
1
$ edge-impulse-blocks --api-key ei_...
Copied!
Note that this resets the CLI configuration and automatically configures your organization.

Other options

  • --dev - lists development servers, use in conjunction with --clean.
Export as PDF
Copy link