1.AidLux介绍
官网:https://aidlux.com/
- AidLux是基于ARM架构的跨生态（Android/鸿蒙+Linux）一站式AIoT利用疾速开发和部署平台APP
- 通过共享 Linux 内核实现Android 和 Linux 生态交融，为繁多设施同时提供Android和Linux运行环境
- 集成国内支流AI框架和多种开发环境、后盾服务程序、编译器及图形开发组件，毋庸配置开箱即用，极大地简化了开发步骤；自主研发的AI智能减速技术可实现CPU+GPU+NPU智能减速，大幅提高AI利用运行效率；平台宽泛而深度的适配外设接口，省去大量调试工作；内置残缺的跨平台桌面和命令行终端连贯（SSH），一个平台实现多终端协同开发、测试、部署
- 可应用APK包装置形式疾速部署在ARM架构的手机、平板、电脑和板卡等智能终端上
- AidLux能广泛应用在智能工业、AI教育、智慧人居、智慧城市、智慧物流、智慧交通、智慧批发和机器人等诸多场景中
博主感觉，AidLux最大的劣势在于：惯例Al利用开发须要C++、Java、Python、Linux等不同工程师相互配合，而在AidLux平台，开发者仅需应用Python一种编程语言即可进行开发，并反对将其余平台（PC、服务器）上应用Python开发的AI利用间接迁徙至AidLux调试运行。
2.环境搭建
2.1 手机版AidLux的装置
目前应用Aidlux次要有两种形式：
- 边缘设施的形式：阿加犀用高通芯片的S855，和S865制作了两款边缘设施，一款提供7T算力，一款提供15T算力。
手机设施的形式：没有边缘设施的状况下，也能够应用手机版本的Aidlux，尝试边缘设施的所有性能。
并且目前Aidlux已对根本市面上所有的芯片都进行了适配，在手机上运行算法模型，也能够体验优化的成果。当然在应用过程中，有个共同点，即手机设施和边缘设施采纳的Aidlux软件，都是一样的。因而能够先尝试手机设施的形式，在前期须要更多算力的时候，应用边缘设施，就能够无缝连接。所以咱们先下载一下手机Aidlux的APP软件。关上安卓手机的利用商城，搜寻Aidlux即可下载安装。
关上手机版本的Aidlux软件APP，第一次进入的时候，APP自带的零碎会进行初始化。
初始化好后，进入零碎登录页面，这一步最好能够用手机注册一下，当然也能够间接点击“我已浏览并批准”，而后点击跳过登录。
留神：软件获取存储权限后会有提醒，如果是安卓版本12的手机，请点击确认，跟着提醒实现相干步骤，以便后续开发。
进入主页面后，能够点击左上角的红色叉号，将阐明页面敞开。
为了不便编程，aidlux还提供了电脑接口（须要手机和电脑连贯同一网络）：
点击下图中的Cloud_ip，即可显示电脑链接
在电脑中输出第一个网址即可，登陆时须要明码，默认明码为: aidlux
登录胜利后能够发现，电脑的界面和手机端是同步的：
因而在开发的时候咱们将写好的代码上传到文件中，就能够利用VSCode运行和调试 .2.2 Aidlux软件设置默认后盾运行
这里列举了多款手机设置的形式，大家能够参照相应的设置教程：
（1）小米手机和平板设置教程：
https://community.aidlux.com/...
（2）OPPO手机与平板设置教程：
https://community.aidlux.com/...
（3）vivo手机与平板设置教程：
https://community.aidlux.com/...
（4）华为鸿蒙/HarmonyOS 2.0设置教程：
https://community.aidlux.com/...
（5）华为鸿蒙/HarmonyOS 3.0设置教程：
https://community.aidlux.com/...
2.3 VSCode近程连贯
新建近程连贯：config如图所示，连贯的明码同样是aidlux
HostName即上文提到的Cloud_ip地址，Port默认9022，连贯胜利后，关上咱们上传到aidlux中的文件夹：注：关上文件夹的时候会再次输出明码 aidlux
接下来就能够通过vscode进行代码调试了。
3 人流统计实战
而后咱们来看一个运行YOLOv5+指标追踪bytetrack进行人流统计的案例，话不多说，先上成果：
https://www.bilibili.com/vide...
3.1 目录构造：
其中yolov5n_best-fp16.tflite是从yolov5训练好的best.pt权重文件转换来的。
3.2 相干代码：
- yolov5_overstep.py
# aidlux相干from cvs import *import aidlite_gpufrom utils import detect_postprocess, preprocess_img, draw_detect_res,is_passing_lineimport cv2# bytetrackfrom track.tracker.byte_tracker import BYTETrackerfrom track.utils.visualize import plot_trackingimport requestsimport time # 加载模型model_path = '/home/lesson5_codes/aidlux/yolov5n_best-fp16.tflite'in_shape = [1 * 640 * 640 * 3 * 4]out_shape = [1 * 25200 * 6 * 4] # 载入模型aidlite = aidlite_gpu.aidlite()# 载入yolov5检测模型aidlite.ANNModel(model_path, in_shape, out_shape, 4, 0) tracker = BYTETracker(frame_rate=30)track_id_status = {}cap = cvs.VideoCapture("/home/lesson5_codes/aidlux/video.mp4")frame_id = 0count_person = 0while True: frame = cap.read() if frame is None: print('camera is over!') # 统计打印人数流量 # 填写对应的喵码 id = 'tOqH04S' # 填写喵揭示中，发送的音讯，这里放上后面提到的图片外链 text = "人流统计数："+str(count_person) ts = str(time.time()) # 工夫戳 type = 'json' # 返回内容格局 request_url = "http://miaotixing.com/trigger?" headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.67 Safari/537.36 Edg/87.0.664.47'} result = requests.post(request_url + "id=" + id + "&text=" + text + "&ts=" + ts + "&type=" + type,headers=headers) break frame_id += 1 if frame_id % 3 != 0: continue # 预处理 img = preprocess_img(frame, target_shape=(640, 640), div_num=255, means=None, stds=None) # 数据转换：因为setTensor_Fp32()须要的是float32类型的数据，所以送入的input的数据需为float32,大多数的开发者都会遗记将图像的数据类型转换为float32 aidlite.setInput_Float32(img, 640, 640) # 模型推理API aidlite.invoke() # 读取返回的后果 pred = aidlite.getOutput_Float32(0) # 数据维度转换 pred = pred.reshape(1, 25200, 6)[0] # 模型推理后处理 pred = detect_postprocess(pred, frame.shape, [640, 640, 3], conf_thres=0.4, iou_thres=0.45) # 绘制推理后果 res_img = draw_detect_res(frame, pred) # 指标追踪相干性能 det = [] # Process predictions for box in pred[0]: # per image box[2] += box[0] box[3] += box[1] det.append(box) if len(det): # Rescale boxes from img_size to im0 size online_targets = tracker.update(det, [frame.shape[0], frame.shape[1]]) online_tlwhs = [] online_ids = [] online_scores = [] # 取出每个指标的追踪信息 for t in online_targets: # 指标的检测框信息 tlwh = t.tlwh # 指标的track_id信息 tid = t.track_id online_tlwhs.append(tlwh) online_ids.append(tid) online_scores.append(t.score) # 针对指标绘制追踪相干信息 res_img = plot_tracking(res_img, online_tlwhs, online_ids, 0,0) ### 人流计数辨认性能实现 ### # 1.绘制直线 lines = [[186,249],[1235,366]] cv2.line(res_img,(186,249),(1235,266),(255,255,0),3) # 2.计算失去人体下方中心点的地位（人体检测监测点调整） pt = [tlwh[0]+1/2*tlwh[2],tlwh[1]+tlwh[3]] # 3. 人体和违规区域的判断（人体状态追踪判断） track_info = is_passing_line(pt, lines) if tid not in track_id_status.keys(): track_id_status.update( {tid:[track_info]}) else: if track_info != track_id_status[tid][-1]: track_id_status[tid].append(track_info) # 4. 判断是否有track_id越界，有的话保留成图片 # 当某个track_id的状态，上一帧是-1，然而这一帧是1时，阐明越界了 if track_id_status[tid][-1] == 1 and len(track_id_status[tid]) >1: # 判断上一个状态是否是-1，是否的话阐明越界，为了避免持续判断，随机的赋了一个3的值 if track_id_status[tid][-2] == -1: track_id_status[tid].append(3) count_person+=1 cv2.putText(res_img,"-1 to 1 person_count: "+str(count_person),(50,105),cv2.FONT_HERSHEY_SIMPLEX,1,(0,255,255),2) cvs.imshow(res_img) - utils.py
import time import cv2import numpy as np coco_class = ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light', 'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard', 'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear', 'hair drier', 'toothbrush'] def xywh2xyxy(x): ''' Box (center x, center y, width, height) to (x1, y1, x2, y2) ''' y = np.copy(x) y[:, 0] = x[:, 0] - x[:, 2] / 2 # top left x y[:, 1] = x[:, 1] - x[:, 3] / 2 # top left y y[:, 2] = x[:, 0] + x[:, 2] / 2 # bottom right x y[:, 3] = x[:, 1] + x[:, 3] / 2 # bottom right y return y def xyxy2xywh(box): ''' Box (left_top x, left_top y, right_bottom x, right_bottom y) to (left_top x, left_top y, width, height) ''' box[:, 2:] = box[:, 2:] - box[:, :2] return box def NMS(dets, thresh): ''' 单类NMS算法 dets.shape = (N, 5), (left_top x, left_top y, right_bottom x, right_bottom y, Scores) ''' x1 = dets[:,0] y1 = dets[:,1] x2 = dets[:,2] y2 = dets[:,3] areas = (y2-y1+1) * (x2-x1+1) scores = dets[:,4] keep = [] index = scores.argsort()[::-1] while index.size >0: i = index[0] # every time the first is the biggst, and add it directly keep.append(i) x11 = np.maximum(x1[i], x1[index[1:]]) # calculate the points of overlap y11 = np.maximum(y1[i], y1[index[1:]]) x22 = np.minimum(x2[i], x2[index[1:]]) y22 = np.minimum(y2[i], y2[index[1:]]) w = np.maximum(0, x22-x11+1) # the weights of overlap h = np.maximum(0, y22-y11+1) # the height of overlap overlaps = w*h ious = overlaps / (areas[i]+areas[index[1:]] - overlaps) idx = np.where(ious<=thresh)[0] index = index[idx+1] # because index start from 1 return dets[keep] def letterbox(img, new_shape=(640, 640), color=(114, 114, 114), auto=True, scaleFill=False, scaleup=True, stride=32): # Resize and pad image while meeting stride-multiple constraints shape = img.shape[:2] # current shape [height, width] if isinstance(new_shape, int): new_shape = (new_shape, new_shape) # Scale ratio (new / old) r = min(new_shape[0] / shape[0], new_shape[1] / shape[1]) if not scaleup: # only scale down, do not scale up (for better test mAP) r = min(r, 1.0) # Compute padding ratio = r, r # width, height ratios new_unpad = int(round(shape[1] * r)), int(round(shape[0] * r)) dw, dh = new_shape[1] - new_unpad[0], new_shape[0] - new_unpad[1] # wh padding if auto: # minimum rectangle dw, dh = np.mod(dw, stride), np.mod(dh, stride) # wh padding elif scaleFill: # stretch dw, dh = 0.0, 0.0 new_unpad = (new_shape[1], new_shape[0]) ratio = new_shape[1] / shape[1], new_shape[0] / shape[0] # width, height ratios dw /= 2 # divide padding into 2 sides dh /= 2 if shape[::-1] != new_unpad: # resize img = cv2.resize(img, new_unpad, interpolation=cv2.INTER_LINEAR) top, bottom = int(round(dh - 0.1)), int(round(dh + 0.1)) left, right = int(round(dw - 0.1)), int(round(dw + 0.1)) img = cv2.copyMakeBorder(img, top, bottom, left, right, cv2.BORDER_CONSTANT, value=color) # add border return img, ratio, (dw, dh) def preprocess_img(img, target_shape:tuple=None, div_num=255, means:list=[0.485, 0.456, 0.406], stds:list=[0.229, 0.224, 0.225]): ''' 图像预处理: target_shape: 指标shape div_num: 归一化除数 means: len(means)==图像通道数，通道均值, None不进行zscore stds: len(stds)==图像通道数，通道方差, None不进行zscore ''' img_processed = np.copy(img) # resize if target_shape: # img_processed = cv2.resize(img_processed, target_shape) img_processed = letterbox(img_processed, target_shape, stride=None, auto=False)[0] img_processed = img_processed.astype(np.float32) img_processed = img_processed/div_num # z-score if means is not None and stds is not None: means = np.array(means).reshape(1, 1, -1) stds = np.array(stds).reshape(1, 1, -1) img_processed = (img_processed-means)/stds # unsqueeze img_processed = img_processed[None, :] return img_processed.astype(np.float32) def convert_shape(shapes:tuple or list, int8=False): ''' 转化为aidlite须要的格局 ''' if isinstance(shapes, tuple): shapes = [shapes] out = [] for shape in shapes: nums = 1 if int8 else 4 for n in shape: nums *= n out.append(nums) return out def scale_coords(img1_shape, coords, img0_shape, ratio_pad=None): # Rescale coords (xyxy) from img1_shape to img0_shape if ratio_pad is None: # calculate from img0_shape gain = min(img1_shape[0] / img0_shape[0], img1_shape[1] / img0_shape[1]) # gain = old / new pad = (img1_shape[1] - img0_shape[1] * gain) / 2, (img1_shape[0] - img0_shape[0] * gain) / 2 # wh padding else: gain = ratio_pad[0][0] pad = ratio_pad[1] coords[:, [0, 2]] -= pad[0] # x padding coords[:, [1, 3]] -= pad[1] # y padding coords[:, :4] /= gain clip_coords(coords, img0_shape) return coords def clip_coords(boxes, img_shape): # Clip bounding xyxy bounding boxes to image shape (height, width) boxes[:, 0].clip(0, img_shape[1], out=boxes[:, 0]) # x1 boxes[:, 1].clip(0, img_shape[0], out=boxes[:, 1]) # y1 boxes[:, 2].clip(0, img_shape[1], out=boxes[:, 2]) # x2 boxes[:, 3].clip(0, img_shape[0], out=boxes[:, 3]) # y2 def detect_postprocess(prediction, img0shape, img1shape, conf_thres=0.25, iou_thres=0.45): ''' 检测输入后处理 prediction: aidlite模型预测输入 img0shape: 原始图片shape img1shape: 输出图片shape conf_thres: 置信度阈值 iou_thres: IOU阈值 return: list[np.ndarray(N, 5)], 对应类别的坐标框信息, xywh、conf ''' h, w, _ = img1shape cls_num = prediction.shape[-1] - 5 valid_condidates = prediction[prediction[..., 4] > conf_thres] valid_condidates[:, 0] *= w valid_condidates[:, 1] *= h valid_condidates[:, 2] *= w valid_condidates[:, 3] *= h valid_condidates[:, :4] = xywh2xyxy(valid_condidates[:, :4]) valid_condidates = valid_condidates[(valid_condidates[:, 0] > 0) & (valid_condidates[:, 1] > 0) & (valid_condidates[:, 2] > 0) & (valid_condidates[:, 3] > 0)] box_cls = valid_condidates[:, 5:].argmax(1) cls_box = [] for i in range(cls_num): temp_boxes = valid_condidates[box_cls == i] if(len(temp_boxes) == 0): cls_box.append([]) continue temp_boxes = NMS(temp_boxes, iou_thres) temp_boxes[:, :4] = scale_coords([h, w], temp_boxes[:, :4] , img0shape).round() temp_boxes[:, :4] = xyxy2xywh(temp_boxes[:, :4]) cls_box.append(temp_boxes[:, :5]) return cls_box def draw_detect_res(img, all_boxes): ''' 检测后果绘制 ''' img = img.astype(np.uint8) color_step = int(255/len(all_boxes)) for bi in range(len(all_boxes)): if len(all_boxes[bi]) == 0: continue for box in all_boxes[bi]: x, y, w, h = [int(t) for t in box[:4]] score = str(box[4:5][0]) cv2.putText(img, str(round(float(score),2)), (x, y-5), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2) cv2.rectangle(img, (x,y), (x+w, y+h),(0, bi*color_step, 255-bi*color_step),thickness = 2) return img def process_points(img,points,color_light_green): points = np.array([points], dtype=np.int32) ###绘制mask zeros = np.zeros((img.shape), dtype=np.uint8) mask = cv2.fillPoly(zeros, points, color=color_light_green) ####填充色彩 ##绘制轮廓 cv2.drawContours(img, points, -1, (144, 238, 144), 5) ###绘制轮廓 ##叠加mask和一般图片 mask_img = 0.01 * mask + img return mask_img def is_in_poly(p, poly): """ :param p: [x, y] :param poly: [[], [], [], [], ...] :return: """ px, py = p is_in = False for i, corner in enumerate(poly): next_i = i + 1 if i + 1 < len(poly) else 0 x1, y1 = corner x2, y2 = poly[next_i] if (x1 == px and y1 == py) or (x2 == px and y2 == py): # if point is on vertex is_in = True break if min(y1, y2) < py <= max(y1, y2): # find horizontal edges of polygon x = x1 + (py - y1) * (x2 - x1) / (y2 - y1) if x == px: # if point is on edge is_in = True break elif x > px: # if point is on left-side of line is_in = not is_in if is_in == True: person_status = 1 else: person_status = -1 return person_status def is_passing_line(point, polyline): # 在直线下方,status =-1 # 在直线上方,status =1 status = 1 # 点映射在直线的高度 poly_y = ((polyline[1][1] - polyline[0][1]) * (point[0] - polyline[0][0])) / (polyline[1][0] - polyline[0][0]) + \ polyline[0][1] if point[1] > poly_y: status = -1 return status3.3 喵揭示公众号的应用
在yolov5_overstep.py中咱们能够看到，有这样一段代码：
它能够用来实现如下成果：
即：当咱们预测完一段视频，或者一个时段之后，通过公众号告诉咱们人流量为多少。
应用办法咱们曾经在下面给出，大家只须要把喵码ID换成本人的，这样才会发送到本人的微信上。
上面来介绍一下如何获取喵码id：
（1）关注“喵揭示”公众号，关注胜利后，公众号会给咱们发送一条音讯，能够间接点击注册账号，或者给公众号发送“注册账号”。
（2）在底部菜单栏中，点击揭示--新建
（3）填写完题目和备注后，点击保留，零碎会主动生成喵码id。
（4）此时，咱们只须要复制喵码id，并且替换代码中原有的id即可
4 AidLux精彩案例
当然，aidlux的性能不只于此，在examples中有各种各样的AI我的项目案例，只须要下载运行即可体验。