Yolov3 to OpenCV C++

Refer to
(1.) https://blog.francium.tech/custom-object-training-and-detection-with-yolov3-darknet-and-opencv-41542f2ff44e
(2.) https://learnopencv.com/training-yolov3-deep-learning-based-custom-object-detector
(3.) https://github.com/pjreddie/darknet/issues/2257
(4.) https://github.com/pjreddie/darknet/issues/1458
(5.) https://blog.csdn.net/ll_master/article/details/81487844

Start Training
1.
git clone --recursive https://github.com/pjreddie/darknet

2.
wget https://github.com/arnoldfychen/darknet/blob/master/src/convolutional_layer.c
replace ~/darknet/src/convolutional_layer.c with new one

3. build OpenCV (option)
download branches https://github.com/opencv/opencv/tree/5.x
if you have cuda then -DWITH_CUDA=OFF to -DWITH_CUDA=ON

////////////////////////opencv x64 debug////////////////////////
CPU_NUM=$(grep processor /proc/cpuinfo | awk '{field=$NF};END{print field+1}')
echo "CPU number = "$CPU_NUM

mkdir -p build
cd build
cmake .. -G"Unix Makefiles" \
-DCMAKE_INSTALL_PREFIX=/opt/opencv/x64/debug \
-DCMAKE_BUILD_TYPE=Debug \
-DCMAKE_C_FLAGS_DEBUG="-O0 -g" \
-DCMAKE_CXX_FLAGS_DEBUG="-O0 -g" \
-DBUILD_PERF_TESTS=OFF \
-DBUILD_TESTS=OFF \
-DBUILD_EXAMPLES=OFF \
-DBUILD_JPEG=ON \
-DBUILD_PNG=ON \
-DWITH_GSTREAMER=OFF \
-DWITH_CUDA=OFF \
-DENABLE_SSE=OFF \
-DENABLE_SSE2=OFF \
-DENABLE_SSE3=OFF \
-DENABLE_SSSE3=OFF \
-DENABLE_SSE41=OFF \
-DENABLE_SSE42=OFF \
-DENABLE_AVX=OFF \
-DENABLE_AVX2=OFF \
-DENABLE_AVX512F=OFF \
-DENABLE_NEON=OFF \
-DENABLE_VFPV3=OFF \
-DENABLE_FAST_MATH=OFF

cmake --build . --config Debug --target install -- -j$CPU_NUM VERBOSE=1

////////////////////////opencv x64 release////////////////////////
CPU_NUM=$(grep processor /proc/cpuinfo | awk '{field=$NF};END{print field+1}')
echo "CPU number = "$CPU_NUM

mkdir -p build
cd build
cmake .. -G"Unix Makefiles" \
-DCMAKE_INSTALL_PREFIX=/opt/opencv/x64/release \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_FLAGS_RELEASE="-Ofast -g" \
-DCMAKE_CXX_FLAGS_RELEASE="-Ofast -g" \
-DBUILD_PERF_TESTS=OFF \
-DBUILD_TESTS=OFF \
-DBUILD_EXAMPLES=OFF \
-DBUILD_JPEG=ON \
-DBUILD_PNG=ON \
-DWITH_GSTREAMER=OFF \
-DWITH_CUDA=OFF \
-DENABLE_SSE=ON \
-DENABLE_SSE2=ON \
-DENABLE_SSE3=ON \
-DENABLE_SSSE3=ON \
-DENABLE_SSE41=ON \
-DENABLE_SSE42=ON \
-DENABLE_AVX=ON \
-DENABLE_AVX2=ON \
-DENABLE_AVX512F=OFF \
-DENABLE_NEON=OFF \
-DENABLE_VFPV3=OFF \
-DENABLE_FAST_MATH=ON

cmake --build . --config Release --target install -- -j$CPU_NUM VERBOSE=1

4.
enable follow options for training faster if you have hardware cuda
gedit ~/darknet/Makefile
GPU=1
CUDNN=1
OPENCV=1

CFLAGS = -Wno-unused-result -Wno-unknown-pragmas -Wfatal-errors -fPIC
#if you have sse sse2 sse3 ssse3 sse4.1 sse4.2 avx avx2 avx512f
OPTS = -Ofast -msse -msse2 -msse3 -mssse3 -msse4 -msse4.1 -msse4.2 -mavx -mavx2 -mavx512f -pthread

LDFLAGS = -Wl,-rpath,$(shell pwd)
LDFLAGS += -lm -lpthread

#exit and save
make -j8

5.
training way "Refer to (1.)"
create directory ~/darknet/custom_data
create directories cfg, images, labels
create files classes.names, detector.data, test.txt, train.txt
In [labels] there are many xxxxx.txt that are created by labelImg in yolo format

note:
#function fill_truth_detection of data.c will find directory labels automatically
#sudo pip3 install --break-system-packages --force-reinstall labelImg

6. single class modify
gedit ~/darknet/custom_data/cfg/yolov3-custom.cfg
classes=1 in line number 610, 696, 783
filters=18 in line number 603, 689, 776

note:
classes=1 in every [yolo] layer
filters=18 in first [convolutional] before the [yolo] layer
filters=(classes+5)*3=6*3=18

ex:
[convolutional]
size=1
stride=1
pad=1
filters=18
activation=linear

[yolo]
mask = 6,7,8
anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
classes=1
num=9
jitter=.3
ignore_thresh = .7
truth_thresh = 1
random=1

7.
cd ~/darknet
wget https://pjreddie.com/media/files/darknet53.conv.74

8. start train
./darknet detector train custom_data/detector.data custom_data/cfg/yolov3-custom.cfg darknet53.conv.74

#line number 138 of ~/darkinet/examples/detector.c
if(i%10000==0 || (i < 1000 && i%100 == 0))
saved for every 100 iterations until 1000 iterations and then continues to save for every 10000 iteration
can add or decrease for saving weight

9. resume training (option)
./darknet detector train custom_data/detector.data custom_data/cfg/yolov3-custom.cfg backup/yolov3-custom_final.weights

10.
cp -f ~/darknet/custom_data/cfg/yolov3-custom.cfg ~/darknet/opencv_cpp/cfg/yolov3-custom.cfg
cp -f ~/darknet/backup/yolov3-custom_final.weights ~/darknet/opencv_cpp/cfg/yolov3-custom.weights

11.
gedit ~/darknet/opencv_cpp/source/opencv_yolo.cpp
make and run
#width and height of opencv_yolo.cpp need match with ~/darknet/custom_data/cfg/yolov3-custom.cfg

12. opencv_yolo.cpp code clip explanation
cv::Mat scores = onerow.colRange(5, onerow.cols);
cv::minMaxLoc(scores, 0, &confidence, 0, &classIdPoint);
if (confidence > threshold)
{
    //int classId = classIdPoint.x;
    int centerX = static_cast<int>(onerow.at<float>(0) * resize_3840_bgr.cols);
    int centerY = static_cast<int>(onerow.at<float>(1) * resize_3840_bgr.rows);
    int width   = static_cast<int>(onerow.at<float>(2) * resize_3840_bgr.cols);
    int height  = static_cast<int>(onerow.at<float>(3) * resize_3840_bgr.rows);
    int left    = centerX - width   / 2;
    int top     = centerY - height  / 2;

    boxes.push_back(cv::Rect(left, top, width, height));
    confidences.push_back((float)confidence);
}

Total columns=4(bounding box)+1(confidence)+N(class probabilities)
Example:
Column Index    Value    Description
0    0.5    Normalized center x of bounding box.
1    0.4    Normalized center y of bounding box.
2    0.2    Normalized width of bounding box.
3    0.3    Normalized height of bounding box.
4    0.85    Object confidence (probability of object).
5    0.1    Class 1 probability (e.g., "Car").
6    0.05    Class 2 probability (e.g., "Person").
7    0.9    Class 3 probability (e.g., "Truck").
8    0.02    Class 4 probability (e.g., "Dog").
object Confidence = [0.85]
scores = [0.1, 0.05, 0.9, 0.02]
Object Confidence (0.85): How confident YOLO is that there’s an object in the bounding box.
Class scores (0.9 for "Truck"): How confident YOLO is that the detected object belongs to a specific class.
Final confidence=0.85×0.9=0.765

note: YOLOv3 Architecture. Detection occurs at layers 82, 94, and 106
https://www.researchgate.net/figure/YOLOv3-Architecture-Detection-occurs-at-layers-82-94-and-106-21_fig2_370069501

Bug fixed and full source:
https://www.mediafire.com/file/3fsz5rn9f2n58k8/darknet.zip