python使用AprilTag 3

最近想测试一下AprilTag精度，看看能不能用的上。

1 安装

法1：github源码编译安装（放弃）

一开始找到了AprilTag 3的官方github网址https://github.com/AprilRobotics/apriltag，但是按着操作下来不会把python库链接到conda虚拟环境，所以尝试看看有没有别的办法。

法2：pip

在pypi.org官网上找到了两种apriltag安装方法

法2.1：pip install apriltag（放弃）

https://pypi.org/project/apriltag/，看到写的更新时间为2018年，所以应该不是最新版的AprilTag 3，故放弃

法2.2：pip install pyapriltags（可用）

再一番寻找，找到了2023更新的版本，https://pypi.org/project/pyapriltags/，应该比较靠谱，并且有如下描述，可以判断是AprilTag原作者AprilRobotics开发的官方库。

These are Python bindings for the Apriltags 3 library developed by AprilRobotics.

这个包的github链接为https://github.com/WillB97/pyapriltags，可以git clone一下里面有pyapriltags/test/test.py测试例程序。

运行时应当注意yaml包版本，过新版本可能会出现报错。

pip install pyyaml==5.3 

2 测试

进入pyapriltags/test/文件夹下，终端中执行

python test.py

得到如下结果，还没有集成到摄像头里面，但应该是跑通了

TESTING WITH A SAMPLE IMAGE
[Detection object:
tag_family = b'tag36h11'
tag_id = 60
tag_size = 0.065
hamming = 0
decision_margin = 24.732816696166992
homography = [[1.12852846e+01 1.35218628e+01 1.50604021e+02][4.12931891e-01 1.34012105e+01 9.97275963e+01][7.54358942e-03 3.97360209e-02 1.00000000e+00]]
center = [150.60402144  99.72759634]
corners = [[148.07374573 109.20044708][167.49220276 108.4158783 ][153.30262756  89.62454987][132.03965759  90.17698669]]
pose_R = [[ 0.97275506  0.17189059  0.1555674 ][ 0.04820706  0.50638753 -0.86095746][-0.22676788  0.84500017  0.48430469]]
pose_t = [[-0.4970639 ][-0.30799945][ 0.9148219 ]]
pose_err = 1.4234227187581564e-06
, Detection object:
tag_family = b'tag36h11'
tag_id = 82
tag_size = 0.065
hamming = 0
decision_margin = 24.040637969970703
homography = [[ 8.02054019e+00  3.10314575e+00  4.66621306e+02][-8.49221724e-01  1.06309763e+01  8.20201682e+01][-3.96118917e-03  1.89261753e-02  1.00000000e+00]]
center = [466.62130564  82.02016819]
corners = [[451.37316895  91.40827179][470.70095825  90.44836426][482.58377075  72.19226074][462.41769409  73.33588409]]
pose_R = [[ 0.96003341 -0.15463188 -0.2332913 ][-0.14024405  0.45556232 -0.87908736][ 0.24221365  0.87667095  0.41566885]]
pose_t = [[ 0.37945231][-0.37270953][ 0.95850859]]
pose_err = 3.8481665025688905e-07
, Detection object:
tag_family = b'tag36h11'
tag_id = 318
tag_size = 0.065
hamming = 0
decision_margin = 49.61515426635742
homography = [[-1.18572593e+01  3.08253464e+01  5.37597981e+02][-2.09246950e+01  1.09739259e+01  3.75029839e+02][-1.38257708e-02  3.30089272e-02  1.00000000e+00]]
center = [537.59798111 375.02983943]
corners = [[554.31921387 388.7227478 ][546.09033203 358.20751953][519.23352051 359.99130249][528.7734375  392.51019287]]
pose_R = [[-0.08183852  0.99100632 -0.1058722 ][-0.92256889 -0.03513853  0.38422901][ 0.37705318  0.12911913  0.91714729]]
pose_t = [[0.46192837][0.36769891][0.76352648]]
pose_err = 5.429954298671284e-06
, Detection object:
tag_family = b'tag36h11'
tag_id = 328
tag_size = 0.065
hamming = 0
decision_margin = 30.794836044311523
homography = [[ 1.15647124e+01 -1.31398782e+00  3.19180294e+02][ 5.44862875e-01  6.30841077e+00  9.93377081e+01][ 8.41156717e-03 -4.81280377e-03  1.00000000e+00]]
center = [319.18029385  99.33770814]
corners = [[310.40652466 106.50978088][328.24972534 105.81019592][327.72503662  92.35285187][310.04534912  92.81846619]]
pose_R = [[ 9.99085975e-01  4.27458458e-02 -8.01169633e-05][-3.87858311e-02  9.07314404e-01  4.18660042e-01][ 1.79686689e-02 -4.18274269e-01  9.08143030e-01]]
pose_t = [[-0.0521486 ][-0.41622112][ 1.23252283]]
pose_err = 3.00796680270929e-07
, Detection object:
tag_family = b'tag36h11'
tag_id = 387
tag_size = 0.065
hamming = 0
decision_margin = 39.92521286010742
homography = [[-1.31081417e+01  6.81016759e+00  1.79359887e+02][ 3.18874693e-01 -6.31109107e+00  3.22140884e+02][-1.26834699e-03  2.44129945e-02  1.00000000e+00]]
center = [179.35988748 322.14088375]
corners = [[194.28860474 307.61105347][169.14706421 308.99703979][163.6441803  337.43667603][190.05665588 335.90756226]]
pose_R = [[-0.99045956 -0.03036217  0.13441724][ 0.08804465 -0.88980218  0.4477725 ][ 0.10600941  0.45533528  0.88398631]]
pose_t = [[-0.36597334][ 0.26055146][ 0.79989467]]
pose_err = 2.613317301680534e-07
]TESTING WITH ROTATION IMAGES
Testing image  test_image_rotation_0.png
[[ 0.00855086][-0.03378784][ 0.20410712]] 0.3 0.0 0.05
[[ 0.99958323 -0.02017469  0.02064826][ 0.02462415  0.96918519 -0.2450994 ][-0.01506718  0.2455057   0.96927805]] 0.0 0.0 0.0
Testing image  test_image_rotation_10.png
[[ 0.00899355][-0.03396286][ 0.20464217]] 0.3 0.0 0.05
[[ 0.99553509 -0.01814129 -0.09263244][-0.00525816  0.96917553 -0.24631514][ 0.09424557  0.24570244  0.96475286]] 0.0 0.0 10.0
Testing image  test_image_rotation_20.png
[[ 0.01026933][-0.03436307][ 0.20649642]] 0.3 0.0 0.05
[[ 0.96053902 -0.00980882 -0.27797227][-0.05970563  0.96881099 -0.24050054][ 0.27166161  0.24760666  0.92999511]] 0.0 0.0 20.0
Testing image  test_image_rotation_30.png
[[ 0.01173215][-0.03442039][ 0.20700676]] 0.3 0.0 0.05
[[ 0.9001428  -0.01497421 -0.43533746][-0.09601522  0.96800711 -0.23182602][ 0.42488117  0.25047555  0.86990689]] 0.0 0.0 30.0
Testing image  test_image_rotation_40.png
[[ 0.012491  ][-0.03455821][ 0.20762498]] 0.3 0.0 0.05
[[ 0.81339466 -0.00986884 -0.58162852][-0.13883209  0.96766979 -0.21057264][ 0.56490245  0.25202736  0.78572732]] 0.0 0.0 40.0
Testing image  test_image_rotation_50.png
[[ 0.01392287][-0.03490017][ 0.20890351]] 0.3 0.0 0.05
[[ 0.68644151 -0.01404759 -0.72704932][-0.1685925   0.96949743 -0.17790814][ 0.70737163  0.2446986   0.66313496]] 0.0 0.0 50.0
Testing image  test_image_rotation_60.png
[[ 0.01452253][-0.0352799 ][ 0.2101661 ]] 0.3 0.0 0.05
[[ 0.55337696 -0.00760598 -0.83289621][-0.19765949  0.9701953  -0.14018492][ 0.80913822  0.24220495  0.53538033]] 0.0 0.0 60.0
Testing image  test_image_rotation_70.png
[[ 0.01504444][-0.03561829][ 0.211431  ]] 0.3 0.0 0.05
[[ 0.39452178 -0.00315769 -0.91888116][-0.22194234  0.97005917 -0.09862454][ 0.89168053  0.24284816  0.38200865]] 0.0 0.0 70.0
Testing image  test_image_rotation_-10.png
[[ 0.00737225][-0.03395985][ 0.20490716]] 0.3 0.0 0.05
[[ 0.98644522 -0.02056676  0.16279692][ 0.05954953  0.96934681 -0.23837075][-0.15290416  0.24483417  0.95743227]] 0.0 0.0 -10.0
Testing image  test_image_rotation_-20.png
[[ 0.00605699][-0.03420006][ 0.20600093]] 0.3 0.0 0.05
[[ 0.94790959 -0.01801473  0.31802968][ 0.09644923  0.96776511 -0.23265478][-0.30358682  0.25120942  0.91909133]] 0.0 0.0 -20.0
Testing image  test_image_rotation_-30.png
[[ 0.00486003][-0.03435319][ 0.20670285]] 0.3 0.0 0.05
[[ 0.87250798 -0.01693225  0.48830639][ 0.14092693  0.96564692 -0.21832458][-0.46783483  0.25930546  0.84492085]] 0.0 0.0 -30.0
Testing image  test_image_rotation_-40.png
[[ 0.00344044][-0.03460087][ 0.20756462]] 0.3 0.0 0.05
[[ 0.76359383 -0.01288116  0.64556838][ 0.17905511  0.9648149  -0.19253954][-0.62037386  0.26261432  0.73903315]] 0.0 0.0 -40.0
Testing image  test_image_rotation_-50.png
[[ 0.00257899][-0.0345974 ][ 0.20762641]] 0.3 0.0 0.05
[[ 0.62773692 -0.01039825  0.77835611][ 0.21986308  0.96156606 -0.16447172][-0.7467306   0.27437674  0.6058967 ]] 0.0 0.0 -50.0
Testing image  test_image_rotation_-60.png
[[ 0.00175676][-0.0346137 ][ 0.20774601]] 0.3 0.0 0.05
[[ 0.47993701 -0.00589266  0.87728316][ 0.25294356  0.9584421  -0.13194053][-0.84004763  0.28522627  0.46148234]] 0.0 0.0 -60.0
Testing image  test_image_rotation_-70.png
[[ 0.00084808][-0.03494523][ 0.20876179]] 0.3 0.0 0.05
[[ 0.31758472 -0.00758214  0.94819959][ 0.27671098  0.9571842  -0.08502607][-0.90695699  0.28938022  0.30608514]] 0.0 0.0 -70.0
AVG time per detection:  0.01741302808125814TESTING WITH MULTIPLE TAGS IMAGES
Testing image  test_image_multiple_01.png
6  tags found:  [22, 24, 58, 85, 144, 198]
Testing image  test_image_multiple_02.png
5  tags found:  [22, 24, 85, 144, 198]
Testing image  test_image_multiple_03.png
6  tags found:  [22, 24, 58, 85, 144, 198]
Testing image  test_image_multiple_04.png
6  tags found:  [22, 24, 58, 85, 144, 198]
Testing image  test_image_multiple_05.png
4  tags found:  [22, 24, 85, 198]
Testing image  test_image_multiple_06.png
6  tags found:  [22, 24, 58, 85, 144, 198]
Testing image  test_image_multiple_07.png
6  tags found:  [22, 24, 58, 85, 144, 198]
Testing image  test_image_multiple_08.png
3  tags found:  [85, 144, 198]
Testing image  test_image_multiple_09.png
5  tags found:  [22, 24, 58, 85, 144]
Testing image  test_image_multiple_10.png
4  tags found:  [24, 58, 85, 198]
AVG time per detection:  0.01161332130432129

贴一下测试例程：

from pyapriltags import Detector
import numpy
import ostest_images_path = 'test_files'visualization = True
try:import cv2
except:raise Exception('You need cv2 in order to run the demo. However, you can still use the library without it.')try:from cv2 import imshow
except:print("The function imshow was not implemented in this installation. Rebuild OpenCV from source to use it")print("VIsualization will be disabled.")visualization = Falsetry:import yaml
except:raise Exception('You need yaml in order to run the tests. However, you can still use the library without it.')at_detector = Detector(families='tag36h11',nthreads=1,quad_decimate=1.0,quad_sigma=0.0,refine_edges=1,decode_sharpening=0.25,debug=0)with open(test_images_path + '/test_info.yaml', 'r') as stream:parameters = yaml.load(stream)#### test WITH THE SAMPLE IMAGE ####print("\n\nTESTING WITH A SAMPLE IMAGE")img = cv2.imread(test_images_path+'/'+parameters['sample_test']['file'], cv2.IMREAD_GRAYSCALE)
cameraMatrix = numpy.array(parameters['sample_test']['K']).reshape((3,3))
camera_params = ( cameraMatrix[0,0], cameraMatrix[1,1], cameraMatrix[0,2], cameraMatrix[1,2] )if visualization:cv2.imshow('Original image',img)tags = at_detector.detect(img, True, camera_params, parameters['sample_test']['tag_size'])
print(tags)color_img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)for tag in tags:for idx in range(len(tag.corners)):cv2.line(color_img, tuple(tag.corners[idx-1, :].astype(int)), tuple(tag.corners[idx, :].astype(int)), (0, 255, 0))cv2.putText(color_img, str(tag.tag_id),org=(tag.corners[0, 0].astype(int)+10,tag.corners[0, 1].astype(int)+10),fontFace=cv2.FONT_HERSHEY_SIMPLEX,fontScale=0.8,color=(0, 0, 255))if visualization:cv2.imshow('Detected tags', color_img)k = cv2.waitKey(0)if k == 27:         # wait for ESC key to exitcv2.destroyAllWindows()#### test WITH THE ROTATION IMAGES ####import timeprint("\n\nTESTING WITH ROTATION IMAGES")time_num = 0
time_sum = 0test_images_path = 'test_files'
image_names = parameters['rotation_test']['files']for image_name in image_names:print("Testing image ", image_name)ab_path = test_images_path + '/' + image_nameif(not os.path.isfile(ab_path)):continuegroundtruth = float(image_name.split('_')[-1].split('.')[0])  # name of test_files image should be set to its groundtruthparameters['rotation_test']['rotz'] = groundtruthcameraMatrix = numpy.array(parameters['rotation_test']['K']).reshape((3,3))camera_params = ( cameraMatrix[0,0], cameraMatrix[1,1], cameraMatrix[0,2], cameraMatrix[1,2] )img = cv2.imread(ab_path, cv2.IMREAD_GRAYSCALE)start = time.time()tags = at_detector.detect(img, True, camera_params, parameters['rotation_test']['tag_size'])time_sum+=time.time()-starttime_num+=1print(tags[0].pose_t, parameters['rotation_test']['posx'], parameters['rotation_test']['posy'], parameters['rotation_test']['posz'])print(tags[0].pose_R, parameters['rotation_test']['rotx'], parameters['rotation_test']['roty'], parameters['rotation_test']['rotz'])print("AVG time per detection: ", time_sum/time_num)#### test WITH MULTIPLE TAGS IMAGES ####print("\n\nTESTING WITH MULTIPLE TAGS IMAGES")time_num = 0
time_sum = 0image_names = parameters['multiple_tags_test']['files']for image_name in image_names:print("Testing image ", image_name)ab_path = test_images_path + '/' + image_nameif(not os.path.isfile(ab_path)):continuecameraMatrix = numpy.array(parameters['multiple_tags_test']['K']).reshape((3,3))camera_params = ( cameraMatrix[0,0], cameraMatrix[1,1], cameraMatrix[0,2], cameraMatrix[1,2] )img = cv2.imread(ab_path, cv2.IMREAD_GRAYSCALE)start = time.time()tags = at_detector.detect(img, True, camera_params, parameters['multiple_tags_test']['tag_size'])time_sum+=time.time()-starttime_num+=1tag_ids = [tag.tag_id for tag in tags]print(len(tags), " tags found: ", tag_ids)color_img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)for tag in tags:for idx in range(len(tag.corners)):cv2.line(color_img, tuple(tag.corners[idx-1, :].astype(int)), tuple(tag.corners[idx, :].astype(int)), (0, 255, 0))cv2.putText(color_img, str(tag.tag_id),org=(tag.corners[0, 0].astype(int)+10,tag.corners[0, 1].astype(int)+10),fontFace=cv2.FONT_HERSHEY_SIMPLEX,fontScale=0.8,color=(0, 0, 255))if visualization:cv2.imshow('Detected tags for ' + image_name    , color_img)k = cv2.waitKey(0)if k == 27:         # wait for ESC key to exitcv2.destroyAllWindows()print("AVG time per detection: ", time_sum/time_num)