ZTE LiDAR-Fusion Chapter 1 - Environment Preparation

This project highly relies on CUDA, so Windows and macOS are not supported.

Install PyTorch

All the frameworks and libraries used in our realization are based on PyTorch, install it first.

Create Environment

Due to the laziness of some developers, not all the packages support the latest Python(especially open3d), we choose to create an environment with Python 3.10.

conda create -n zte python==3.10 git

Activate Environment

conda activate zte

Install PyTorch

The version may change in the future, please refer to PyTorch’s official website for the latest information.

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

Install OpenPCDet

OpenPCDet is a modular and versatile framework for 3D object detection that supports various state-of-the-art methods such as PointPillars and PV-RCNN. OpenPCDet also provides a suite of data augmentation techniques to improve the robustness and diversity of training datasets, including random flip, scaling, rotation, jittering, and more. It can train networks with distribution technology and offers benchmark datasets like KITTI, Waymo, and NuScenes. Overall, OpenPCDet is a comprehensive and flexible platform for developing, training and evaluating 3D object detection methods. Whether you’re building a new state-of-the-art algorithm or trying to replicate and improve an existing one, OpenPCDet can provide the necessary tools and infrastructure to accomplish these tasks.

Clone OpenPCDet Repository

This repository contains the modified version of the OpenPCDet project, in which the Painted Kitti Dataset has already been added. It also contains many bug fixes.

git clone git@github.com:psc1453/OpenPCDet.git

Install spconv Library

This library only supports Linux systems with CUDA.

Since OpenPCDet relies on spconv to handle sparse convolution, we need to install this library. In my Python environment, PyTorch with CUDA 11.8 is installed. So I install spconv-cu118.

pip install spconv-cu118

Install OpenPCDet Dependencies

The requirements.txt offers the list of packages that will be used in OpenPCDet. Since PyTorch-related packages were already installed. We can use the command below to install the rest of them.

pip install numpy llvmlite numba tensorboardX pyyaml scikit-image tqdm easydict SharedArray opencv-python pyquaternion

Install OpenPCDet Package

To install the OpenPCDet and obtain an importable package, go to the root directory of its git repository and run:

python setup.py install

Then, OpenPCDet will become a package inside your Python environment just like being installed conda or pip.

Dependency Version Conflict(Skip if You Successfully Installed OpenPCDet)

If the CUDA version of the installed PyTorch doesn’t match the version in your system, it may cause errors. The mismatch of the GCC version between the system and the maximum version that CUDA can support will also lead to errors.

The suggestion is to install the latest version of PyTorch with CUDA, then manually download and install the corresponding CUDA version from NVIDIA CUDA Archive. Make sure to install the cuDNN library to support DNN computing from NVIDIA cuDNN Archive(You may need to sign up and login first).

Besides the CUDA download link, there is another link named Versioned Online Documentation. Click the link and find the Installation Guide Linux to find the appropriate GCC version. Install the recommended version of GCC.

Both of the necessary CUDA and GCC can be installed in /usr/local/, then use the following command to temporarily change their version.

export PATH=/usr/local/cuda-11.8/bin:$PATH && export CUDA_PATH=/usr/local/cuda-11.8 && export CC=/usr/bin/gcc-11 && export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.8/lib64

You may change the path according to your system. You can make it an alias in your Bash or ZSH profile.

Then, use the flowing command to clean the previous build:

python setup.py clean

Now, install OpenPCDet again:

python setup.py install

You only need to do this step during the installation. In your later usage, you do not need to change the path variable again. What you need to use the environment is only simply activating it.

Test OpenPCDet Installation by Official Demo

This test only relies on the supplemental files that I provided on GitHub.

Go to the tool directory of the OpenPCDet repository and run one of the two commands:

• GUI (If you are using a graphical interface):

python demo_gui.py --cfg_file cfgs/kitti_models/pointpillar.yaml --ckpt demo_files/pointpillar_7728.pth --data_path demo_files/pp_test.bin

• Shell (If can only use shell environment like ssh server):

python demo_shell.py --cfg_file cfgs/kitti_models/pointpillar.yaml --ckpt demo_files/pointpillar_7728.pth --data_path demo_files/pp_test.bin

If the configurations are correct, you will see the result.

Potential Problems

You may find some errors while executing this command which tells you it cannot find some packages. Just use pip to install them, and the problems will be solved.

Here I provide a command that will install the missing packages in my test. It may be different in your environment. Just put it here as a reference.

pip install mayavi av2 pyqt5 pyqtwebengine kornia open3d

open3d is optional if your system is purely using a shell environment. It is just used to graphically demonstrate the result.

It may also tell you it needs a UI toolkit like PyQt/PySide or wxPython. Just use pip to install PyQT5. The latest version may cause some conflicts. Older version may solve the problem:

pip install pyqt5==5.12.3 pyqtwebengine==5.12.1

Then run it again, and you can view the result.