Upload portals

Upload portals are a secure way to let external parties upload data to your datasets. Through an upload portal they get an easy user interface to add data, but they have no access to the content of the dataset, nor can they delete any files. Data that is uploaded through the portal can be stored on-premise or in your own cloud infrastructure.

In this tutorial we'll set up an upload portal, show you how to add new data, and how to show this data in Edge Impulse for further processing.

📘

Only available for enterprise customers

Organizational features are only available for enterprise customers. Contact us for more information.

1. Configuring a storage bucket

Data is stored in storage buckets, which can either be hosted by Edge Impulse, or in your own infrastructure. Follow the Building your first dataset tutorial to set up your storage bucket.

2. Creating an upload portal

With your storage bucket configured you're ready to set up your first upload portal. In your organization go to Data > Upload portals and choose Create new upload portal. Here, select a name, a description, the storage bucket, and a path in the storage bucket.

Creating an upload portalCreating an upload portal

Creating an upload portal

Note: You'll need to enable CORS headers on the bucket. If these are not configured you'll get prompted with instructions. Talk to your user success engineer (when your data is hosted by Edge Impulse), or your system administrator to configure this.

After your portal is created a link is shown. This link contains an authentication token, and can be shared directly with the third party.

An active upload portalAn active upload portal

An active upload portal

Click the link to open the portal. If you ever forget the link: no worries. Click the next to your portal, and choose View portal.

3. Uploading data to the portal

An upload portal with two folders.An upload portal with two folders.

An upload portal with two folders.

To upload data you can now drag & drop files or folders to the drop zone on the right, or use Create new folder to first create a folder structure. There's no limit to the amount of files you can upload here, and all files are hashed, so if you upload a file that's already present the file will be skipped.

Note: Files with the same name but with a different hash are overwritten.

4. Adding the data to your dataset

To view the uploaded data in your dataset now go to your organization, and select Data. Then select Add data > Add dataset from bucket. Here, enter a name for the dataset, and select the same bucket path as you used for the portal. Then click Add data.

Adding the data to your datasetAdding the data to your dataset

Adding the data to your dataset

You now have the data in your Edge Impulse organization, ready to be applied to your next machine learning project.

The data in Edge ImpulseThe data in Edge Impulse

The data in Edge Impulse

You can run the 'Add dataset from bucket' step any time new data was added. Data is automatically deduplicated and new files will be picked up. You can also automate this through our API.

5. Recap

If you need a secure way for external parties to contribute data to your datasets then upload portals are the way to go. They offer a friendly user interface, upload data directly into your storage buckets, and give you an easy way to use the data directly in Edge Impulse. :rocket:

Any questions, or interested in the enterprise version of Edge Impulse? Contact us for more information.


Did this page help you?