Optimizing the operation of neural analytics on GPU

Starting from DetectorPack 3.9, a utility was added to the Neuro Pack add-ons (see Installing DetectorPack add-ons), which allows you to create GPU network caches without using Axxon One. The presence of the cache speeds up the initialization and optimizes video memory consumption.

To optimize the operation of the neural analytics on GPU, do the following:

1. Stop the Server (see Shutting down a Server).
   

   Attention!

   If the system has the software running on GPU, it is necessary to stop its operation.

2. Create the GPU_CACHE_DIR system variable (see Appendix 10. Creating system variable) by specifying in the Variable value field the path to the cache location with an arbitrary folder name. For example, D:\AN_GPU_cache. The specified directory will store the cache for all used detection tools and neural networks.

3. To call the utility, open in the command line: C:\Program Files\Common Files\AxxonSoft\DetectorPack\NeuroPackGpuCacheGenerator.exe and press Enter.

4. Specify the ID of the required GPU (see General Information on Configuring Detection) and press Enter.

Optimizing the operation of the neural analytics on GPU is complete. The utility will create the caches of four networks included in the Neuro Pack add-ons:

• GeneralNMHuman_v1.0GPU_onnx.ann—human;
• smokeScanned_v1_onnx.ann—smoke detection;
• fireScanned_v1_onnx.ann—fire detection;
• reid_15_0_256__osnetfpn_segmentation_noise_20_common_29_onnx.ann—search for the similar in the Neural Tracker (see Image Search).
  
  

Creating GPU network caches using parameters:

1. -p is a parameter to create a cache for a particular network.
   Command example:
   

   C:\Program Files\Common Files\AxxonSoft\DetectorPack\NeuroPackGpuCacheGenerator.exe -p "<System disk>\<Network location directory>\Network_name.ann"

   To create a cache for multiple networks, list the paths to the selected networks, separated by a space.
   Command example:

   C:\Program Files\Common Files\AxxonSoft\DetectorPack\NeuroPackGpuCacheGenerator.exe -p "<System disk>\<Network location directory>\Network_name.ann" "C:\Program Files\Common Files\AxxonSoft\DetectorPack\NeuroSDK\WaterLevelRuleNet_origin_onnx.ann"

   
   

2. -v is a parameter to output the procedure log to the console during cache generation.
   Command example to automatically create caches of four networks included in the Neuro Pack add-ons with log output:
   

   C:\Program Files\Common Files\AxxonSoft\DetectorPack\NeuroPackGpuCacheGenerator.exe -v

   Command example:
   

   C:\Program Files\Common Files\AxxonSoft\DetectorPack\NeuroPackGpuCacheGenerator.exe -p "<System disk>\<Network location directory>\Network_name.ann" -v

   
   

3. --int8=1 is a parameter to create a quantized version of the cache for those networks for which quantization is available. By default, the --int8=0 parameter is disabled.
   Command example:
   

   C:\Program Files\Common Files\AxxonSoft\DetectorPack\NeuroPackGpuCacheGenerator.exe --int8=1

   Attention!

   The networks for which the quantization mode is available are included in the Neuro Pack add-ons together with the *.info file.

   The networks for which the quantization mode is available:
   

   1. GeneralNMCar_v1.0GPU_onnx.ann—vehicles.
   2. GeneralNMHuman_v1.0GPU_onnx.ann—human.
   3. GeneralNMHumanTopView_v0.8GPU_onnx.ann—human, top view.