Page History

General information

It can take several minutes to launch neural analytics algorithms on Nvidia GPU after server restart. Meanwhile, the neural models are optimized for the current GPU type.

You can use the caching function to ensure that this operation is performed only once. Caching saves the optimization results on the hard drive and uses it them for the subsequent analytics runs. 

Starting with DetectorPack 3.9, a utility was added to the Neuro Pack add-ons (see Installing DetectorPack add-ons), which allows you to create GPU neural network caches without using Axxon One. The presence of the cache speeds up the initialization and optimizes video memory consumption.

Optimizing the operation of neural analytics on GPU

To optimize the operation of the neural analytics on GPU, do the following:

1. Stop the server (see Starting and stopping the Axxon One Server in Linux OS).
   

   Note
   title Attention!
   If the system has the software running on GPU,

it is necessary to

2. you must stop its operation.

3. Login as ngp superuser:

   1. In the command prompt, run the command:

      Code Block
      language bash
      sudo su ngp

   2. Enter the password for the

root

4. 1. superuser.

5. Create a folder with a custom name to store the cache. For example:
   

   Code Block
   language bash
   mkdir /opt/AxxonSoft/AxxonOne/gpucache

6. Change folder permissions:
   

   Code Block
   language bash
   chmod -R 777 /opt/AxxonSoft/AxxonOne/gpucache

Create the GPU_CACHE_DIR system variable:

7. Go to the /opt/AxxonSoft/AxxonOne

/ folder

8.  folder:
   

   Code Block
   language bash
   cd /opt/AxxonSoft/AxxonOne

9. Open the server configuration file for editing:
   

   Code Block
   language bash
   nano instance.conf

   Note
   title Attention!
   When you use the server in failover mode, you should: Open the /etc/AxxonSoft folder: Code Block language bash cd /etc/AxxonSoft Open the server configuration file for editing: Code Block language bash nano

instance

10. axxon-one.conf Info title Attention!

Add the following line to the

11. Before you make changes to the axxon-one.conf file, we recommend reading the instructions on the page Creating system variables for the Axxon One server in the failover mode in Linux OS. Add the GPU_CACHE_DIR system variable in the configuration file, wherethevaluewillspecify the pathto the cachelocationfolder: Code Block language bash export GPU_CACHE_DIR="/opt/AxxonSoft/AxxonOne/gpucache" Save the changes in the server configuration file. Add the GPU_CACHE_DIR system variable in the /etc/profile file: Code Block language bash export GPU_CACHE_DIR="/opt/AxxonSoft/AxxonOne/gpucache" Run the command: Code Block language bash source /etc/profile

12. Add the GPU_CACHE_DIR system variable,wherethevaluewillspecify the pathto the cachelocationfolder. For example:

    Code Block
    language bash
    export GPU_CACHE_DIR="/opt/AxxonSoft/AxxonOne/gpucache"

13. Save the server configuration file using the Ctrl+O keyboard shortcut.

14. Exit file editing mode using the Ctrl+X keyboard shortcut.
    

    Note
    title Attention!

If

15. When you change the server configuration (see Changing the configuration of the Axxon One Server in Linux OS) or update to

a

16. the new

version of

17. Axxon One version, the system variables previously added to the instance.conf configuration file will be deleted (see Creating system variables for the Axxon One server in Linux OS).

18. Save the file using the Ctrl+O keyboard shortcut.

19. Exit file editing mode using the Ctrl+X keyboard shortcut.

Run the following command in the terminal
20. In the command prompt, run the command that was usedtoadd the systemvariablewiththepathto the cache location folder. For example:
    

    Code Block
    language bash
    export GPU_CACHE_DIR="/opt/AxxonSoft/AxxonOne/gpucache"

21. Go to the /opt/AxxonSoft/DetectorPack/ folder:

    Code Block
    language bash
    cd /opt/AxxonSoft/DetectorPack

22. Run the following command:

    Code Block
    language bash
    ./NeuroPackGpuCacheGenerator

    Note
    title Attention!
    If more than one Nvidia GPU is available, you will be able to select one. To do this, specify a number from 0 to 3, which corresponds to the required device in the list.

Optimizing the operation of the neural analytics on GPU is complete. The utility will create the caches of four neural networks included in the Neuro Pack add-ons:

• GeneralNMHuman_v1.0GPU_onnx.ann—person;
• smokeScanned_v1_onnx.ann (or bestSmoke_v1.ann starting with Detector Pack 3.14)—smoke detection;
• fireScanned_v1_onnx.ann (or bestFire_v1.ann starting with Detector Pack 3.14)—fire detection;
• reid_15_0_256__osnetfpn_segmentation_noise_20_common_29_onnx.ann—search for the similar in the Neural tracker (see Similitude search).
  

Creating GPU neural network caches using parameters

1. -p is a parameter to create a cache for a particular neural network.
   Command example:
   

   Code Block
   ./NeuroPackGpuCacheGenerator -p /opt/AxxonSoft/DetectorPack/NeuroSDK/GeneralNMHumanAndVehicle_Nano_v1.0_GPU_onnx.ann

2. -v is a parameter to output the procedure log to the console during cache generation.
   Command example to automatically create caches of four neural networks included in the Neuro Pack add-ons with log output:
   

   Code Block
   ./NeuroPackGpuCacheGenerator -v

3. --int8=1 is a parameter to create a quantized version of the cache for those neural networks for which quantization is available. By default, the --int8=0 parameter is disabled.
   Command example:
   

   Code Block
   ./NeuroPackGpuCacheGenerator -p /opt/AxxonSoft/DetectorPack/NeuroSDK/GeneralNMHumanAndVehicle_Nano_v1.0_GPU_onnx.ann --int8=1

   Note
   title Attention!
   The neural networks for which the quantization mode is available are included in the Neuro Pack add-ons together with the *.info file.

The neural networks for which the quantization mode is available (see Neural tracker, Stopped object detector, Neural counter):

• GeneralNMCar_v1.0GPU_onnx.ann—Vehicle.
• GeneralNMHuman_v1.0GPU_onnx.ann—Person.
• GeneralNMHumanTopView_v0.8GPU_onnx.ann—Person (top-down view).

Starting with DetectorPack 3.11, the following neural networks were added:

• GeneralNMHumanAndVehicle_Nano_v1.0_GPU_onnx.ann—Person and vehicle (Nano).
• GeneralNMHumanAndVehicle_Medium_v1.0_GPU_onnx.ann—Person and vehicle (Medium).
• GeneralNMHumanAndVehicle_Large_v1.0_GPU_onnx.ann—Person and vehicle (Large).

Starting with DetectorPack 3.12, the following neural networks were added:

• GeneralNMHumanTopView_Nano_v1.0_GPU_onnx.ann—Person (top-down view Nano).
• GeneralNMHumanTopView_Medium_v1.0_GPU_onnx.ann—Person (top-down view Medium).
• GeneralNMHumanTopView_Large_v1.0_GPU_onnx.ann—Person (top-down view Large).