nx上darknet的使用-目标检测-在python中的使用

1  内置的代码

在darknet中已经内置了两个py文件

darknet_video.py与darknet_images.py用法类似,都是改一改给的参数就行了,我们说一下几个关键的参数

  • input 要预测哪张图像
  • weights 要使用哪个权重
  • config_file 要使用哪个cfg文件
  • data_file 要使用哪个data文件
  • thresh 置信度给多少

改完上面这些参数就可以直接运行了

用鼠标点一下图片然后按q可以关闭图片

调用视频有时可以正常用,有时不可以,会报下面的错,可能是opencv的版本问题

这两个代码都很好,想的很周道,但是看起来太麻烦,我通常用下面两个简化版的

2  opencv使用模型

优点:只要有python,有opencv-python,有numpy就能用(不需要编译darknet,只要有names、cfg、weights就行)

缺点:只能搞CPU版,用摄像头搞效果较差

2.1  识别图像

你需要一张预测图像,训练时的classes.names yolov4-tiny-cfg yolov4-tiny_final.weights

yolo_opencv.py的内容如下

import cv2
import numpy as npLABELS = open("classes.names").read().strip().split("\n")
net = cv2.dnn.readNetFromDarknet('yolov4-tiny.cfg', 'yolov4-tiny_final.weights')
layer = net.getUnconnectedOutLayersNames()frame = cv2.imread('000003.jpg')
(H, W) = frame.shape[:2]
blob = cv2.dnn.blobFromImage(frame, 1 / 255.0, (416, 416),swapRB=True, crop=False)
net.setInput(blob)
layerOutputs = net.forward(layer)
boxes = []
confidences = []
classIDs = []for output in layerOutputs:for detection in output:scores = detection[5:]classID = np.argmax(scores)confidence = scores[classID]box = detection[0:4] * np.array([W, H, W, H])(centerX, centerY, width, height) = box.astype("int")x = int(centerX - (width / 2))y = int(centerY - (height / 2))boxes.append([x, y, int(width), int(height)])confidences.append(float(confidence))classIDs.append(classID)idxs = cv2.dnn.NMSBoxes(boxes, confidences, 0.5, 0.3)
if len(idxs) > 0:for i in idxs.flatten():(x, y) = (boxes[i][0], boxes[i][1])(w, h) = (boxes[i][2], boxes[i][3])cv2.rectangle(frame, (x, y), (x + w, y + h), (0,255,0), 1, lineType=cv2.LINE_AA)text = "{}: {:.4f}".format(LABELS[classIDs[i]], confidences[i])cv2.putText(frame, text, (x, y - 5), cv2.FONT_HERSHEY_SIMPLEX,0.5, (255,0,0), 1, lineType=cv2.LINE_AA)cv2.imshow('frame',frame)
cv2.waitKey(0)
cv2.destroyAllWindows()

运行后可以显示预测结果

2.2  flask起服务

我们可以搞一个接口来处理识别的功能

服务端

import numpy as np
import cv2
from flask import Flask,request
import base64LABELS = open("classes.names").read().strip().split("\n")
net = cv2.dnn.readNetFromDarknet('yolov4-tiny.cfg', 'yolov4-tiny_final.weights')
layer = net.getUnconnectedOutLayersNames()app = Flask(__name__)
@app.route('/predict',methods=["POST"])
def predict():if request.method == 'POST':image_base64 = request.json['img_b64']img = base64.b64decode(image_base64)img = np.fromstring(img,np.uint8)frame = cv2.imdecode(img,cv2.IMREAD_COLOR)(H, W) = frame.shape[:2]blob = cv2.dnn.blobFromImage(frame, 1 / 255.0, (416, 416),swapRB=True, crop=False)net.setInput(blob)layerOutputs = net.forward(layer)boxes = []confidences = []classIDs = []for output in layerOutputs:for detection in output:scores = detection[5:]classID = np.argmax(scores)confidence = scores[classID]box = detection[0:4] * np.array([W, H, W, H])(centerX, centerY, width, height) = box.astype("int")x = int(centerX - (width / 2))y = int(centerY - (height / 2))boxes.append([x, y, int(width), int(height)])confidences.append(float(confidence))classIDs.append(classID)idxs = cv2.dnn.NMSBoxes(boxes, confidences, 0.5, 0.3)result = []if len(idxs) > 0:for i in idxs.flatten():(x, y) = (boxes[i][0], boxes[i][1])(w, h) = (boxes[i][2], boxes[i][3])result_obj = {}result_obj['x'] = xresult_obj['y'] = yresult_obj['w'] = wresult_obj['h'] = hresult_obj['name'] = LABELS[classIDs[i]]result_obj['confidence'] = confidences[i]result.append(result_obj)return {'result':result}if __name__ == '__main__':app.run(host='192.168.0.105')

客户端

import cv2
import requests
import base64def get_result(url,frame):retval, buffer = cv2.imencode('.jpg', frame)image = str(base64.b64encode(buffer), 'utf-8')json_data = {'img_b64': image}response = eval(requests.post(url, json=json_data).text).get('result')return response
if __name__ == '__main__':url = 'http://192.168.0.105:5000/predict'frame = cv2.imread('000003.jpg')response = get_result(url,frame)for result in response:confidence = result.get('confidence')x = int(result.get('x'))y = int(result.get('y'))w = int(result.get('w'))h = int(result.get('h'))name = result.get('name')frame = cv2.rectangle(frame, (x, y), (x + w, y + h), (255, 0, 0), 2)frame = cv2.putText(frame, name, (x, y), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)cv2.imshow('img',frame)cv2.waitKey(0)cv2.destroyAllWindows()

请求结果

2.3  摄像头识别

原理是图像识别+多线程

import cv2
import numpy as np
import queue
import threading
import timeLABELS = open("classes.names").read().strip().split("\n")
net = cv2.dnn.readNetFromDarknet('yolov4-tiny.cfg', 'yolov4-tiny_final.weights')
layer = net.getUnconnectedOutLayersNames()cap = cv2.VideoCapture(0)frame_queue = queue.Queue()
detection_result_queue = queue.Queue(maxsize=1)def video_capture():while cap.isOpened():ret, frame = cap.read()if ret:frame_queue.put(frame)cap.release()def predict():while cap.isOpened():start_time = time.time()predict_frame = frame_queue.get()(H, W) = predict_frame.shape[:2]blob = cv2.dnn.blobFromImage(predict_frame, 1 / 255.0, (416, 416), swapRB=True, crop=False)net.setInput(blob)layerOutputs = net.forward(layer)boxes = []confidences = []classIDs = []for output in layerOutputs:for detection in output:scores = detection[5:]classID = np.argmax(scores)confidence = scores[classID]box = detection[0:4] * np.array([W, H, W, H])(centerX, centerY, width, height) = box.astype("int")x = int(centerX - (width / 2))y = int(centerY - (height / 2))boxes.append([x, y, int(width), int(height)])confidences.append(float(confidence))classIDs.append(classID)idxs = cv2.dnn.NMSBoxes(boxes, confidences, 0.5, 0.3)if len(idxs) > 0:result_list = []for i in idxs.flatten():result_dic = {}(x, y) = (boxes[i][0], boxes[i][1])(w, h) = (boxes[i][2], boxes[i][3])label = LABELS[classIDs[i]]confidence = confidences[i]result_dic.__setitem__('x',x)result_dic.__setitem__('y',y)result_dic.__setitem__('w',w)result_dic.__setitem__('h',h)result_dic.__setitem__('label',label)result_dic.__setitem__('confidence',confidence)result_list.append(result_dic)detection_result_queue.put(result_list)print(time.time()-start_time)cap.release()def draw():while cap.isOpened():draw_frame = frame_queue.get()try:predict_results = detection_result_queue.get(block=False)for predict_result in predict_results:x = predict_result.get('x')y = predict_result.get('y')w = predict_result.get('w')h = predict_result.get('h')label = predict_result.get('label')confidence = predict_result.get('confidence')cv2.rectangle(draw_frame, (x, y), (x + w, y + h), (0,255,0), 1, lineType=cv2.LINE_AA)text = "{}: {:.4f}".format(label, confidence)cv2.putText(draw_frame, text, (x, y - 5), cv2.FONT_HERSHEY_SIMPLEX,0.5, (255,0,0), 1, lineType=cv2.LINE_AA)except:passcv2.imshow('draw_frame',draw_frame)cv2.waitKey(1)cap.release()threading.Thread(target=video_capture).start()
threading.Thread(target=predict).start()
threading.Thread(target=draw).start()

由于是CPU运行,效果根据CPU的性能会有差异,我是用NX上的6核CPU运行yolov4-tiny,大约0.5s一张。这个速度如果在摄像头上的连续识别就非常慢了,至少要1秒30帧才能有流畅的感觉,也就是0.03秒需要预测一张

  • 0.5/0.03 = 16.6 或许搞个17线程能够达成流畅的效果,没尝试过

3  darknet使用模型

优点:可以使用GPU

缺点:需要编译darknet才能使用

下面在代码中import darknet指的是import下面这个文件。下面这个py文件依赖了darknet文件夹下的其他文件,所以建议把下面的代码放在darknet的根目录下使用

3.1  识别图像

在模型读取的时候是通过data文件找names文件,而不是直接找names文件

import cv2
import darknet
import timenetwork, class_names, class_colors = darknet.load_network('/home/suyu/darknet/yolo_opencv/yolov7-tiny.cfg','/home/suyu/darknet/custom_training/custom_training.data','/home/suyu/darknet/yolo_opencv/yolov7-tiny_final_origin.weights',batch_size=1
)width = darknet.network_width(network)
height = darknet.network_height(network)
darknet_image = darknet.make_image(width, height, 3)image = cv2.imread('/home/suyu/darknet/yolo_opencv/280.jpg')
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
image_resized = cv2.resize(image_rgb, (width, height),interpolation=cv2.INTER_LINEAR)
darknet.copy_image_from_bytes(darknet_image, image_resized.tobytes())
start_time = time.time()
detections = darknet.detect_image(network, class_names, darknet_image, thresh=0.8)
print(time.time()-start_time)darknet.free_image(darknet_image)
image = darknet.draw_boxes(detections, image_resized, class_colors)
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)cv2.imshow('image',image)
cv2.waitKey(0)
cv2.destroyAllWindows()

在终端上可以看到对模型进行了读取

画框的颜色每次都是随机的

在预测中,如果出现了下面这种情况,我们就需要减少预测的结果,可以通过标签后面的置信度减少,但是我们可以看到有两个框的置信度都在0.99以上,这个时候我们就需要用到其他减少框的办法

darknet.detect_image()这个方法,除了通过置信度减少预测结果,还可以通过非最大值抑制消除冗余,也就是最后一个的nms

  • hier_thresh也是消除冗余的一个参数,叫控制层次性阈值(与nms类似),没用过,如果nms不行的化再尝试使用它

nms数值越小,消除冗余的效果就越好,我这里直接改成了0.05

改完之后可以得到还不错的效果

3.2  摄像头识别

原理是识别图像+多线程,我用yolov7-tiny大概能到0.03秒预测一张,这个预测速度不加多线程应该也可以

import cv2
import numpy as np
import queue
import threading
import time
import darknetnetwork, class_names, class_colors = darknet.load_network('/home/suyu/darknet/yolo_opencv/yolov7-tiny.cfg','/home/suyu/darknet/custom_training/custom_training.data','/home/suyu/darknet/yolo_opencv/yolov7-tiny_final_origin.weights',batch_size=1
)width = darknet.network_width(network)
height = darknet.network_height(network)
darknet_image = darknet.make_image(width, height, 3)cap = cv2.VideoCapture(0)frame_queue = queue.Queue()
detection_result_queue = queue.Queue(maxsize=1)def video_capture():while cap.isOpened():ret, frame = cap.read()if ret:image_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)image_resized = cv2.resize(image_rgb, (width, height), interpolation=cv2.INTER_LINEAR)frame_queue.put(image_resized)cap.release()def predict():while cap.isOpened():start_time = time.time()image_resized = frame_queue.get()darknet.copy_image_from_bytes(darknet_image, image_resized.tobytes())detections = darknet.detect_image(network, class_names, darknet_image, thresh=0.8)# darknet.free_image(darknet_image)print(time.time()-start_time)detection_result_queue.put(detections)cap.release()def draw():while cap.isOpened():draw_frame = frame_queue.get()# print(draw_frame)try:detections = detection_result_queue.get(block=False)draw_frame = darknet.draw_boxes(detections, draw_frame, class_colors)except:passdraw_frame = cv2.cvtColor(draw_frame, cv2.COLOR_BGR2RGB)cv2.imshow('draw_frame',draw_frame)cv2.waitKey(1)cap.release()threading.Thread(target=video_capture).start()
threading.Thread(target=predict).start()
threading.Thread(target=draw).start()

如果你想对预测的结果进行别的操作你可以用到detection_result_queue中的detections,打印出来是这样的

列表套元组,元组的第一个值是label,第二个值是置信

操作的时候建议多开一个线程,然后将detection_result_queue置为2或者更高,避免两个线程抢数据的情况

4  onnx使用模型

onnx(Open Neural Network eXchange) 开放式神经网络交换,好多模型都可以转换为onnx类型的模型,相当于是人工智能模型界的docker了。以高适配性而著名。在性能上并没有优于其他模型。

4.1  用到的库

onnx,这个是将其他类型的模型转换为onnx类型的库,在arm端上安装可能会有些麻烦,但在amd上直接用pip就可以安装了

onnxruntime是跑onnx模型用的,onnxruntime在arm端可以直接用pip安装

4.2  darknet的weight转onnx

参考 https://zhuanlan.zhihu.com/p/543345367

源码 GitHub - Tianxiaomo/pytorch-YOLOv4: PyTorch ,ONNX and TensorRT implementation of YOLOv4

安装完onnx与onnxruntime后运行demo_darknet2onnx.py,第一个参数是cfg,第二个参数是names,第三个参数是weights,第四个参数是图,第五个参数是batch_size,直接写1就行了

python demo_darknet2onnx.py /home/suyu/darknet/cfg/yolov4.cfg /home/suyu/darknet/data/coco.names /home/suyu/darknet/yolov4.weights /home/suyu/darknet/data/dog.jpg 1

如果opencv版本过高会爆出下面两个问题,是cv2.rectangele()与cv2.putText()的参数需为int

我们需要更改 /pytorch-YOLOv4-master/tool/utils.py

把画红线的地方改成int

成功使用会显示下面这些东西

在文件夹中可以找到通过onnx预测成功的图像和onnx模型

4.3  识别图像

识别的代码是用的上面提供的源码中的东西,有的代码直接从utils中复制过来了,用的时候不需要引入其他py文件了,只需要.onnx文件与.names文件

import sys
import onnx
import os
import argparse
import numpy as np
import cv2
import onnxruntime
import time
import mathdef load_class_names(namesfile):class_names = []with open(namesfile, 'r') as fp:lines = fp.readlines()for line in lines:line = line.rstrip()class_names.append(line)return class_namesdef nms_cpu(boxes, confs, nms_thresh=0.5, min_mode=False):# print(boxes.shape)x1 = boxes[:, 0]y1 = boxes[:, 1]x2 = boxes[:, 2]y2 = boxes[:, 3]areas = (x2 - x1) * (y2 - y1)order = confs.argsort()[::-1]keep = []while order.size > 0:idx_self = order[0]idx_other = order[1:]keep.append(idx_self)xx1 = np.maximum(x1[idx_self], x1[idx_other])yy1 = np.maximum(y1[idx_self], y1[idx_other])xx2 = np.minimum(x2[idx_self], x2[idx_other])yy2 = np.minimum(y2[idx_self], y2[idx_other])w = np.maximum(0.0, xx2 - xx1)h = np.maximum(0.0, yy2 - yy1)inter = w * hif min_mode:over = inter / np.minimum(areas[order[0]], areas[order[1:]])else:over = inter / (areas[order[0]] + areas[order[1:]] - inter)inds = np.where(over <= nms_thresh)[0]order = order[inds + 1]return np.array(keep)def post_processing(img, conf_thresh, nms_thresh, output):# anchors = [12, 16, 19, 36, 40, 28, 36, 75, 76, 55, 72, 146, 142, 110, 192, 243, 459, 401]# num_anchors = 9# anchor_masks = [[0, 1, 2], [3, 4, 5], [6, 7, 8]]# strides = [8, 16, 32]# anchor_step = len(anchors) // num_anchors# [batch, num, 1, 4]box_array = output[0]# [batch, num, num_classes]confs = output[1]t1 = time.time()if type(box_array).__name__ != 'ndarray':box_array = box_array.cpu().detach().numpy()confs = confs.cpu().detach().numpy()num_classes = confs.shape[2]# [batch, num, 4]box_array = box_array[:, :, 0]# [batch, num, num_classes] --> [batch, num]max_conf = np.max(confs, axis=2)max_id = np.argmax(confs, axis=2)t2 = time.time()bboxes_batch = []for i in range(box_array.shape[0]):argwhere = max_conf[i] > conf_threshl_box_array = box_array[i, argwhere, :]l_max_conf = max_conf[i, argwhere]l_max_id = max_id[i, argwhere]bboxes = []# nms for each classfor j in range(num_classes):cls_argwhere = l_max_id == jll_box_array = l_box_array[cls_argwhere, :]ll_max_conf = l_max_conf[cls_argwhere]ll_max_id = l_max_id[cls_argwhere]keep = nms_cpu(ll_box_array, ll_max_conf, nms_thresh)if (keep.size > 0):ll_box_array = ll_box_array[keep, :]ll_max_conf = ll_max_conf[keep]ll_max_id = ll_max_id[keep]for k in range(ll_box_array.shape[0]):bboxes.append([ll_box_array[k, 0], ll_box_array[k, 1], ll_box_array[k, 2], ll_box_array[k, 3], ll_max_conf[k], ll_max_conf[k], ll_max_id[k]])bboxes_batch.append(bboxes)t3 = time.time()print('-----------------------------------')print('       max and argmax : %f' % (t2 - t1))print('                  nms : %f' % (t3 - t2))print('Post processing total : %f' % (t3 - t1))print('-----------------------------------')return bboxes_batchdef plot_boxes_cv2(img, boxes, savename=None, class_names=None, color=None):import cv2img = np.copy(img)colors = np.array([[1, 0, 1], [0, 0, 1], [0, 1, 1], [0, 1, 0], [1, 1, 0], [1, 0, 0]], dtype=np.float32)def get_color(c, x, max_val):ratio = float(x) / max_val * 5i = int(math.floor(ratio))j = int(math.ceil(ratio))ratio = ratio - ir = (1 - ratio) * colors[i][c] + ratio * colors[j][c]return int(r * 255)width = img.shape[1]height = img.shape[0]for i in range(len(boxes)):box = boxes[i]x1 = int(box[0] * width)y1 = int(box[1] * height)x2 = int(box[2] * width)y2 = int(box[3] * height)bbox_thick = int(0.6 * (height + width) / 600)if color:rgb = colorelse:rgb = (255, 0, 0)if len(box) >= 7 and class_names:cls_conf = box[5]cls_id = box[6]print('%s: %f' % (class_names[cls_id], cls_conf))classes = len(class_names)offset = cls_id * 123457 % classesred = get_color(2, offset, classes)green = get_color(1, offset, classes)blue = get_color(0, offset, classes)if color is None:rgb = (red, green, blue)msg = str(class_names[cls_id])+" "+str(round(cls_conf,3))t_size = cv2.getTextSize(msg, 0, 0.7, thickness=bbox_thick // 2)[0]c1, c2 = (x1,y1), (x2, y2)c3 = (c1[0] + t_size[0], c1[1] - t_size[1] - 3)cv2.rectangle(img, (int(x1),int(y1)), (int(np.float32(c3[0])), int(np.float32(c3[1]))), rgb, -1)img = cv2.putText(img, msg, (int(c1[0]), int(np.float32(c1[1] - 2))), cv2.FONT_HERSHEY_SIMPLEX,0.7, (0,0,0), bbox_thick//2,lineType=cv2.LINE_AA)img = cv2.rectangle(img, (int(x1), int(y1)), (int(x2), int(y2)), rgb, bbox_thick)if savename:print("save plot results to %s" % savename)cv2.imwrite(savename, img)return imgdef detect(session, image_src, namesfile):IN_IMAGE_H = session.get_inputs()[0].shape[2]IN_IMAGE_W = session.get_inputs()[0].shape[3]# Inputresized = cv2.resize(image_src, (IN_IMAGE_W, IN_IMAGE_H), interpolation=cv2.INTER_LINEAR)img_in = cv2.cvtColor(resized, cv2.COLOR_BGR2RGB)img_in = np.transpose(img_in, (2, 0, 1)).astype(np.float32)img_in = np.expand_dims(img_in, axis=0)img_in /= 255.0#print("Shape of the network input: ", img_in.shape)# Computeinput_name = session.get_inputs()[0].nameoutputs = session.run(None, {input_name: img_in})boxes = post_processing(img_in, 0.4, 0.6, outputs)class_names = load_class_names(namesfile)return plot_boxes_cv2(image_src, boxes[0], class_names=class_names)if __name__ == '__main__':session = onnxruntime.InferenceSession('yolov4_1_3_640_640_static.onnx')namesfile = 'classes.names'image_src = cv2.imread('3.png')detected_img = detect(session, image_src, namesfile)cv2.imshow('detected_img',detected_img)cv2.waitKey(0)cv2.destroyAllWindows()

4.4  摄像头识别

用的采集、预测、画 三线程。模型输入width与height都为320,两分类,在树莓派4B中大概是0.5s一张,大概是1秒2个识别帧

import cv2
import numpy as np
import queue
import threading
import timeimport sys
import onnx
import os
import argparse
import onnxruntime
import mathdef load_class_names(namesfile):class_names = []with open(namesfile, 'r') as fp:lines = fp.readlines()for line in lines:line = line.rstrip()class_names.append(line)return class_namesdef nms_cpu(boxes, confs, nms_thresh=0.5, min_mode=False):x1 = boxes[:, 0]y1 = boxes[:, 1]x2 = boxes[:, 2]y2 = boxes[:, 3]areas = (x2 - x1) * (y2 - y1)order = confs.argsort()[::-1]keep = []while order.size > 0:idx_self = order[0]idx_other = order[1:]keep.append(idx_self)xx1 = np.maximum(x1[idx_self], x1[idx_other])yy1 = np.maximum(y1[idx_self], y1[idx_other])xx2 = np.minimum(x2[idx_self], x2[idx_other])yy2 = np.minimum(y2[idx_self], y2[idx_other])w = np.maximum(0.0, xx2 - xx1)h = np.maximum(0.0, yy2 - yy1)inter = w * hif min_mode:over = inter / np.minimum(areas[order[0]], areas[order[1:]])else:over = inter / (areas[order[0]] + areas[order[1:]] - inter)inds = np.where(over <= nms_thresh)[0]order = order[inds + 1]return np.array(keep)def post_processing(img, conf_thresh, nms_thresh, output):box_array = output[0]confs = output[1]if type(box_array).__name__ != 'ndarray':box_array = box_array.cpu().detach().numpy()confs = confs.cpu().detach().numpy()num_classes = confs.shape[2]box_array = box_array[:, :, 0]max_conf = np.max(confs, axis=2)max_id = np.argmax(confs, axis=2)bboxes_batch = []for i in range(box_array.shape[0]):argwhere = max_conf[i] > conf_threshl_box_array = box_array[i, argwhere, :]l_max_conf = max_conf[i, argwhere]l_max_id = max_id[i, argwhere]bboxes = []for j in range(num_classes):cls_argwhere = l_max_id == jll_box_array = l_box_array[cls_argwhere, :]ll_max_conf = l_max_conf[cls_argwhere]ll_max_id = l_max_id[cls_argwhere]keep = nms_cpu(ll_box_array, ll_max_conf, nms_thresh)if (keep.size > 0):ll_box_array = ll_box_array[keep, :]ll_max_conf = ll_max_conf[keep]ll_max_id = ll_max_id[keep]for k in range(ll_box_array.shape[0]):bboxes.append([ll_box_array[k, 0], ll_box_array[k, 1], ll_box_array[k, 2], ll_box_array[k, 3], ll_max_conf[k], ll_max_conf[k], ll_max_id[k]])bboxes_batch.append(bboxes)return bboxes_batchdef get_color(c, x, max_val):colors = np.array([[1, 0, 1], [0, 0, 1], [0, 1, 1], [0, 1, 0], [1, 1, 0], [1, 0, 0]], dtype=np.float32)ratio = float(x) / max_val * 5i = int(math.floor(ratio))j = int(math.ceil(ratio))ratio = ratio - ir = (1 - ratio) * colors[i][c] + ratio * colors[j][c]return int(r * 255)def detect(session, image_src, namesfile):IN_IMAGE_H = session.get_inputs()[0].shape[2]IN_IMAGE_W = session.get_inputs()[0].shape[3]# Inputresized = cv2.resize(image_src, (IN_IMAGE_W, IN_IMAGE_H), interpolation=cv2.INTER_LINEAR)img_in = cv2.cvtColor(resized, cv2.COLOR_BGR2RGB)img_in = np.transpose(img_in, (2, 0, 1)).astype(np.float32)img_in = np.expand_dims(img_in, axis=0)img_in /= 255.0# Computeinput_name = session.get_inputs()[0].nameoutputs = session.run(None, {input_name: img_in})boxes = post_processing(img_in, 0.4, 0.6, outputs)return boxes[0]session = onnxruntime.InferenceSession('yolov4_1_3_640_640_static.onnx')
namesfile = 'classes.names'
class_names = load_class_names(namesfile)cap = cv2.VideoCapture(0)frame_queue = queue.Queue()
detection_result_queue = queue.Queue(maxsize=1)def video_capture():while cap.isOpened():ret, frame = cap.read()if ret:frame_queue.put(frame)cap.release()def predict():while cap.isOpened():start_time = time.time()predict_frame = frame_queue.get()result_list = detect(session, predict_frame, namesfile)detection_result_queue.put(result_list)print(time.time()-start_time)cap.release()def draw():while cap.isOpened():draw_frame = frame_queue.get()try:boxes = detection_result_queue.get(block=False)img = draw_framewidth = img.shape[1]height = img.shape[0]for i in range(len(boxes)):box = boxes[i]x1 = int(box[0] * width)y1 = int(box[1] * height)x2 = int(box[2] * width)y2 = int(box[3] * height)bbox_thick = int(0.6 * (height + width) / 600)rgb = (255, 0, 0)if len(box) >= 7 and class_names:cls_conf = box[5]cls_id = box[6]print('%s: %f' % (class_names[cls_id], cls_conf))classes = len(class_names)offset = cls_id * 123457 % classesred = get_color(2, offset, classes)green = get_color(1, offset, classes)blue = get_color(0, offset, classes)rgb = (red, green, blue)msg = str(class_names[cls_id])+" "+str(round(cls_conf,3))t_size = cv2.getTextSize(msg, 0, 0.7, thickness=bbox_thick // 2)[0]c1, c2 = (x1,y1), (x2, y2)c3 = (c1[0] + t_size[0], c1[1] - t_size[1] - 3)cv2.rectangle(img, (int(x1),int(y1)), (int(np.float32(c3[0])), int(np.float32(c3[1]))), rgb, -1)img = cv2.putText(img, msg, (int(c1[0]), int(np.float32(c1[1] - 2))), cv2.FONT_HERSHEY_SIMPLEX,0.7, (0,0,0), bbox_thick//2,lineType=cv2.LINE_AA)img = cv2.rectangle(img, (int(x1), int(y1)), (int(x2), int(y2)), rgb, bbox_thick)draw_frame = img#except Exception as e:#print(e)except:passcv2.imshow('draw_frame',draw_frame)cv2.waitKey(1)cap.release()threading.Thread(target=video_capture).start()
threading.Thread(target=predict).start()
threading.Thread(target=draw).start()

在树莓派4B上是下面这个效果

 

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/pingmian/45650.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

基于AT89C51单片机篮球计时计分器的设计(含文档、源码与proteus仿真,以及系统详细介绍)

本篇文章论述的是基于AT89C51单片机篮球计时计分器的设计的详情介绍&#xff0c;如果对您有帮助的话&#xff0c;还请关注一下哦&#xff0c;如果有资源方面的需要可以联系我。 目录 绪论 原理图 ​编辑 仿真图 系统总体设计图 代码实现 系统论文 资源下载 绪论 本次…

通用详情页的打造

背景介绍 大家都知道&#xff0c;详情页承载了站内的核心流量。它的量级到底有多大呢&#xff1f; 我们来看一下&#xff0c;日均播放次数数亿次&#xff0c;这么大的流量&#xff0c;其重要程度可想而知。 在这样一个页面&#xff0c;每一个功能都是大量业务的汇总点。 作为…

【Web开发手礼】探索Web开发的魅力(三)-html基础标签(3)

上述主要是对html标签的介绍和一些基本练习可以当作日常笔记收藏一下&#xff01;&#xff01;&#xff01; 目录 前言 html基础标签 前言 上述主要是对html标签的介绍和一些基本练习可以当作日常笔记收藏一下&#xff01;&#xff01;&#xff01; 提示&#xff1a;以下是本…

PostgreSQL 怎样处理数据仓库中维度表和事实表的关联性能?

文章目录 PostgreSQL 中维度表和事实表关联性能的处理 PostgreSQL 中维度表和事实表关联性能的处理 在数据仓库的领域中&#xff0c;PostgreSQL 作为一款强大的关系型数据库管理系统&#xff0c;对于处理维度表和事实表的关联性能是一个关键的问题。维度表和事实表的关联是数据…

【手写数据库内核组件】0301 动态内存池,频繁malloc/free让系统不堪重负,动态内存池让应用自由使用动态内存

动态内存管理 ​专栏内容&#xff1a; postgresql使用入门基础手写数据库toadb并发编程 个人主页&#xff1a;我的主页 管理社区&#xff1a;开源数据库 座右铭&#xff1a;天行健&#xff0c;君子以自强不息&#xff1b;地势坤&#xff0c;君子以厚德载物. 文章目录 动态内存管…

RSA算法详解:万字文章详解RSA的加密与解密

本文目录 文章前言一、RSA的诞生1、加密算法的前世今生① 《六韬龙韬》中的阴符与阴书② 古罗马&#xff1a;凯撒密码③ 斯巴达&#xff1a;塞塔式密码&#xff08;Scytale&#xff09; 2、对称加密的脆弱性3、非对称加密算法的出现 二、RSA中的数学概念与定理1、质数理论2、关…

使用Qt和mitmproxy开发一个抓取网页短视频的万能工具

目录 实现原理 mitmproxy介绍 功能简介 安装 脚本示例 如何使用 解释 注意事项 QT工具实现 其他资源 实现原理 使用WebView组件造一工具,工具可输入网页地址并显示网页内容及播放视频。把工具的代理设置指向mitmproxy的端口服务。配合使用mitmproxy的MITM技术,监…

7.8~7.10练习

目录 1.扑克牌游戏 2.链表基本功能的实现&#xff08;单项链表&#xff09; 3.移除链表元素力扣 4.反转链表力扣 5.链表的中间结点 5.返回倒数第k个节点​编辑 6.合并两个有序链表 7.链表基本功能的实现&#xff08;双向链表&#xff09; 8.链表分割 1.扑克牌游戏 public…

LightRAG:高效构建和优化大型语言模型应用的 PyTorch 框架

一、前言 随着大语言模型 (LLM) 的蓬勃发展&#xff0c;检索增强生成 (RAG) 技术作为一种将 LLM 与外部知识库结合的有效途径&#xff0c;受到了越来越多的关注。 然而&#xff0c;构建 LLM 应用的真正挑战在于开发者需要根据具体需求进行高度定制化&#xff0c;而现有的 RAG …

Vscode ssh远程连接Linux服务器登录时密码password无法输入

问题 最近在用Vscode远程连接Linux服务器时&#xff0c;在终端提示输入密码password的时候用键盘输入没有反应。 以为是键盘坏了&#xff0c;然后尝试复制粘贴没有用。 后来找到了原因以及解决方法&#xff0c;感谢原帖作者&#xff08;原贴链接粘在下面&#xff09; 原因 …

flutter 列表下拉框加搜索

1.使用控件搜索加下拉框dropdown_search: ^0.4.9和获取中文拼音lpinyin: ^1.1.1 2.加入中文查询和首字查询 在当中找到相应的packages&#xff0c;再在SelectDialog.dart当中加入引入拼音搜索 import package:lpinyin/lpinyin.dart; 更改匹配方法manageItemsByFilter使其可…

有必要把共享服务器升级到VPS吗?

根据自己的需求来选择是否升级&#xff0c;虚拟专用服务器 (VPS) 是一种托管解决方案&#xff0c;它以低得多的成本提供专用服务器的大部分功能。使用 VPS&#xff0c;您的虚拟服务器将与在其上运行的其他虚拟服务器共享硬件服务器的资源。但是&#xff0c;与传统的共享托管&am…

Oracle数据库加密与安全

Wallet简介&#xff1a; Oracle Wallet(即内部加密技术TDE( Transparent DataEncryption&#xff09; TDE是 Oracle10gR2中推出的一个新功能,使用时要保证Oracle版本是在10gR2或者以上 Wallet配置&#xff1a; 1.创建一个新目录&#xff0c;并指定为Wallet目录 /home/oracle…

Python爬虫技术从去哪儿网获取旅游数据,对攻略进行可视化分析,提供全面的旅游攻略和个性化的出行建议

背景 随着信息技术的快速发展和互联网的普及&#xff0c;旅游行业也迎来了数字化和智能化的变革。去哪儿网作为中国领先的在线旅游平台之一&#xff0c;提供了丰富的旅游产品和服务&#xff0c;涵盖了机票、酒店、旅游度假等各个方面。用户通过去哪儿网可以方便地查询、预订和…

STM32HAL库+ESP8266+cJSON+微信小程序_连接华为云物联网平台

STM32HAL库ESP8266cJSON微信小程序_连接华为云物联网平台 实验使用资源&#xff1a;正点原子F407 USART1&#xff1a;PA9P、A10&#xff08;串口打印调试&#xff09; USART3&#xff1a;PB10、PB11&#xff08;WiFi模块&#xff09; DHT11&#xff1a;PG9&#xff08;采集数据…

阿里云操作系统智能助手OS Copilot的实验测评报告

什么是OS Copilot 在老师的介绍下我了解到了阿里云OS Copilot这个产品&#xff0c;它是阿里云推出的一项基于人工智能技术的操作系统&#xff0c;设计用于阿里云Linux操作系统以及其他可能的云上操作系统环境&#xff0c;为用户提供智能化的系统管理和运维支持。 阿里云提供了…

Python数据分析-Excel和 Text 文件的读写操作

1.Excel和 Text 文件的读写操作 1. Text 文件读写包 import sys print(sys.argv[0]) print(__file__) print(sys.path[0]) qopen(sys.path[0] "\out.txt","w",encodingutf-8) q.write(这个是测试一下) q.close() print(done)open 语句可以打开的创建text…

【吊打面试官系列-ZooKeeper面试题】简述 Zookeeper 文件系统?

大家好&#xff0c;我是锋哥。今天分享关于 【简述 Zookeeper 文件系统?】面试题&#xff0c;希望对大家有帮助&#xff1b; 简述 Zookeeper 文件系统? Zookeeper 提供一个多层级的节点命名空间&#xff08;节点称为 znode&#xff09;。与文件系统不同的是&#xff0c;这些节…

白平衡说明

白平衡 相机白平衡的起源原理以及作用起源作用 白平衡的原理白平衡的类型应用说明 工业相机的白平衡效果对比一键白平衡的必要性一键白平衡的实现方式 相机白平衡的起源原理以及作用 起源 白平衡&#xff08;White Balance, WB&#xff09;概念的起源与色温理论密切相关。色温…

【eNSP模拟实验】单臂路由实现VLAN间通信

实验需求 如下图所示&#xff0c;辅导员办公室需要访问处在不同vlan的学生管理服务器的文件&#xff0c;那么如何实现两台终端相互通信呢&#xff1f;我们可以使用单臂路由的方式来实现。 单臂路由&#xff08;router-on-a-stick&#xff09;是指在路由器的一个接口上通过配置…