Vision Deployment Guide
1. About Vision
Vision is a decentralized machine vision platform built on Gridium, designed to provide efficient image recognition and computer vision capabilities. It leverages distributed computing resources to handle large-scale image data and enhance processing efficiency. Vision uses advanced algorithms to perform real-time image recognition tasks, optimizing performance by dynamically allocating resources across the decentralized network.
Users can deploy Vision on Gridium’s network, where it provides seamless access to machine vision services. Once deployed, the platform allows users to access trained models and perform image processing tasks through a dedicated service. As Vision scales, the platform will track user contributions and incentivize them with rewards based on task completion, encouraging collaborative development within the decentralized network.
2. Hardware Requirement
CPU: Intel Xeon series with a base frequency of no less than 2.0 GHz and at least 8 cores
GPU: Nvidia Pascal or higher, video memory of no less than 12 GB
Internet: Internal bandwidth over 10 Gbps, external bandwidth over 10 Gbps
Harddrive: At least 700G SSD storage (500G for storing Matrix Mainnet data and 200G for storing models and training logs)
Storage: At least 32 GB
Network Configuration: A server that can be accessed through public network with an address in the format of http://{IP}.
(For instance, http://{IP}:8052. IP part should be accessible through public network.)
3. Distributed Auto-ML Web Service Configuration
A. The project will utilise two datasets for image categorisation and the training of parallax estimation models.
Image categorisation: ImageNet
a. Dataset Download
Download the datasets at https://www.image-net.org/challenges/LSVRC/index.php.
For image recognition, download the two files in red brackets
Training images (Task 1 & 2)
(https://image-net.org/data/ILSVRC/2012/ILSVRC2012_img_train.tar) Validation images (all tasks)
(https://image-net.org/data/ILSVRC/2012/ILSVRC2012_img_val.tar)
b. Dataset processing
Having downloaded the training and verification datasets, now we need to convert these datasets into a format that can be directly loaded by models.
First, decompress ILSVRC2012_img_train.tar to train. The decompressed folder should contain 1,000 tar files, each representing one category of images. The files are named accordingly, so don’t rename the files. Simply decompress these tar files.
Decompress.
As there are too many to decompress, write a script unzip.sh as below:
Give the script execution permissions.
As ILSVRC2012_img_train.tar may be a bit large, you may delete it afterwards. Move train.tar to train first, before proceeding.
The final training dataset should be something like the following.
The verification dataset is relatively simple. It only contains 50,000 images. We could simply decompress ILSVRC2012_img_val.tar, but for ease of use afterwards, we should divide these images into 1,000 categories. (Just like what we did with the training dataset, create 1,000 folders and put the images in their corresponding categories.)
First decompress.
Enter val. Download the script and execute.
The final form of the verification dataset should look like the following screenshot.
After processing, the dataset should be in the following format.
Parallax estimation: SceneFlow
Load the six datasets in the following screenshot and decompress them.
After downloading is complete, modify the route through the following method for easy recognition by web services.
(1) ImageNet
/mnt/imagenet |
|—train
|--val
(2) SceneFlow
/mnt/SceneFlow |
|--FlyingThings3D
|--Monkaa
|--Driving
B. Launching Web Service Training
（2）Decompress dist-automl.
Command: Unzip -d dist-automl
（3）Enter dist-automl catalogue.
Command: cd dist-automl
（4）Install the python database required by the project.
Command: pip install -r requirements.txt
（5）Execute the “launch service” order.
Command: streamlit run training_manager.py
If the Terminal should show the following, it means the service has been successfully launched.
Replace the part in the red bracket with the public IP of the server where you wish to deploy the service, and you will be able to access the training web service.
You may also offer your deployed service as a distributed Auto-ML service on the Mainnet.
Last updated
