Jetson Nano

tensorRT + yolov4 (2022/03/07)

目標：
了解如何使用gpu加速（tensorRT)
學如何訓練模型

repo
video tutorial

中文教學

實作成果

需購買web cam
sd card 容量太小
走過官方給的demo流程，需要自己再熟悉一下

開始訓練自己的模型

Cave education 對於jetson nano教學的系列文

開會筆記

trace code
買兩台web cam
做hackMD下次咪挺報告

trace code(2022/03/21)

目標：
買web cam *2
trace code

用pre-trained model做一遍由.weights to .trt的流程

1.訓練

YOLOv4實作教學(DarkNet)

了解如何用DarkNet做訓練
但這次時做是使用pre-trained model

2.weight -> ONNX -> tensorRT

i. 權重轉ONNX

使用Google colab實作流程

卡在onnx轉成trt:
tensorRT在google collab安裝時遇到問題，但依據上禮拜我在jetson nano跑demo後，tensorRT是原本就被裝好的，在jetson nano不會遇到這個問題。

方法一: weight file轉onnx file

路線： .weights 轉成.pth再用pytorch的函式轉成onnx


from tool import darknet2pytorch
import torch

# load weights from darknet format
model = darknet2pytorch.Darknet('path/to/cfg/yolov4-416.cfg', inference=True)
model.load_weights('path/to/weights/yolov4-416.weights')

# save weights to pytorch format
torch.save(model.state_dict(), 'path/to/save/yolov4-pytorch.pth')

# reload weights from pytorch format
model_pt = darknet2pytorch.Darknet('path/to/cfg/yolov4-416.cfg', inference=True)
model_pt.load_state_dict(torch.load('path/to/save/yolov4-pytorch.pth'))

方法二: pth file轉onnx file

ref 、pytorch doc

使用Torch.onnx.export函式

torch.onnx.export(model, args, f, export_params=True,
                  verbose=False, training=False, 
                  input_names=None, output_names=None,
                  aten=False, export_raw_ir=False, 
                  operator_export_type=None, 
                  opset_version=None, 
                  _retain_param_name=True, 
                  do_constant_folding=False, 
                  example_outputs=None, 
                  strip_doc_string=True, 
                  dynamic_axes=None, 
                  keep_initializers_as_inputs=None)

官方範例

使用pre-trained model AlexNet轉換成onnx檔案

import torch
import torchvision

dummy_input = torch.randn(1, 3, 224, 224, device='cuda')
model = torchvision.models.alexnet(pretrained=True).cuda()

input_names = [ "actual_input_1" ] + [ "learned_%d" % i for i in range(16) ]
output_names = [ "output1" ]

torch.onnx.export(model, dummy_input, "alexnet.onnx", verbose=True, input_names=input_names, output_names=output_names)

ii. ONNX 轉 trt (還沒在google collab上實作過)

builder

# Builder：將模型導入TensorRT並且建構TensorRT的引擎。
def build_engine(onnx_path, shape = [1,224,224,3]):
    
    with trt.Builder(TRT_LOGGER) as builder, builder.create_network(1) as network, trt.OnnxParser(network, TRT_LOGGER) as parser:
        
        builder.max_workspace_size = (256 << 20)    
# 256MiB model任一層的最大可用空間
        #使用fp16的精度
        builder.fp16_mode = True 
# fp32_mode -> False

        
        with open(onnx_path, 'rb') as model:
            parser.parse(model.read())

        engine = builder.build_cuda_engine(network)

        return engine

main function


if __name__ == "__main__":
    
    onnx_path = '/content/yolov4_1_3_608_608_static.onnx'
    trt_path = '/content/yolov4_1_3_608_608_static.trt'
    input_shape = [1, 224, 224, 3]
    
    build_trt = timer('Parser ONNX & Build TensorRT Engine')
    engine = build_engine(onnx_path, input_shape)
    build_trt.end()
    
    save_trt = timer('Save TensorRT Engine')
    save_engine(engine, trt_path)
    save_trt.end()

3.inference and visualize 成果

還沒看QQ

結論與問題

上週是使用google colab實作這個流程，這周在jetson nano裝好openCV,pytorch等套件後有根據google colab上的程式去跑跑看。
但是在跑DarkNet to ONNX的步驟沒辦法抓到onnx的library(已經有install過)還不確定是甚麼原因。
我有裝好archiconda，之後會用虛擬環境的方式再做一遍，也繼續看tensorrt_demos的程式和環境設定。不過jetson nano 裝那些套件比較麻煩，應該要先找好或自己寫script去跑。

Finish .weights to .trt(2022/03/28)

解決無法import onnx問題 –>輸出.onnx成功
使用trtexec 將.onnx轉成.trt
~~轉換完後執行demo_trt.py無法框出物件位置~~
(2022/3/28已解決)

在跑.weights轉.onnx很常遇到的錯誤訊息

會導致程式中斷

1
2
3

>  - Can't parse 'pt2'. Sequence item with index 0 has a wrong type
>  - Can't parse 'rec'. Expected sequence length 4, got 2
>  - Can't parse 'rec'. Expected sequence length 4, got 2

可能原因
cv2.rectangle(img,(x1,y1),(x2,y2),rgb,3);
其中x1,x2,y1,y2需要為type int

已解決
參考此連結修改類似的地方即可

Try to change following lines in core/utils.py as defined below:

Line 152 -> c1, c2 = (int(coor[1]), int(coor[0])), (int(coor[3]), int(coor[2]))
Line 159 -> cv2.rectangle(image, c1, (int(np.float32(c3[0])), int(np.float32(c3[1]))), bbox_color, -1)
Line 161 -> cv2.putText(image, bbox_mess, (c1[0], int(np.float32(c1[1] - 2))), cv2.FONT_HERSHEY_SIMPLEX,
fontScale, (0, 0, 0), bbox_thick // 2, lineType=cv2.LINE_AA)

.onnx轉.trt方法(可行)

1	trtexec --onnx=resnet50/model.onnx --saveEngine=resnet_engine.trt

下次進度：開始訓練

Labeling tool
玩幾次kaggle object detect相關
讀yolo理論
問專題可以做出大概怎樣的東西？