Image sharpening method of automatic loading and unloading drill pipe target in underground coal mine
-
摘要:
受煤矿井运输巷道空间限制，将钻孔机器人设计为灵活布局的钻进和钻杆加卸2个独立系统，利用基于合作靶标的单目视觉识别，将钻进和钻杆加卸2个系统关联，解决两系统递送钻杆的协同作业问题。然而，煤矿井下钻孔施工中场景复杂，存在孔口煤渣、污水污染合作靶标、粉尘浓度大能见度低、人造光源亮度低、光照不均匀使靶标图像对比度低，以及矿灯直射合作靶标反光等问题，会影响合作靶标图像清晰度，使基于靶标识别加卸钻杆的位置的精度降低，进而影响钻孔机器人自动加卸钻杆的成功率。因此，首先针对施工中孔口返出的煤渣、煤泥污染合作靶标的问题，利用形态学滤波中先腐蚀，再膨胀的开运算，清除煤渣间断斑块和突出靶标像素点；先膨胀再腐蚀的闭运算，消除小的孔洞和间断细小鸿沟，达到消除固体污染物对合作靶标的影响；其次针对粉尘、岩尘、水雾和污水影响图像对比度问题，通过暗通道先验求出初始透射率，采用梯度域导向滤波精细化透射率，利用导向滤波得到改进暗通道，并求大气光值，利用大气散射模型得到除雾后的图像，实现清晰化图像、增强图像对比度目的；最后对亚像素级角点和方向进行精细化，优化能量函数，使用种子棋盘格沿4个边缘生长棋盘格，用能量变化情况确认是否生长出最大的棋盘，补全因强光反射缺失的合作靶标图像。实验结果表明：将形态学滤波、去雾算法和棋盘格生长等方法相融合，可增强煤矿井下复杂场景钻孔施工中合作靶标图像的清晰度，提高基于视觉识别的图像精度，为钻孔机器人自动加卸钻杆的精确控制奠定基础。
Abstract:Due to the space limitation of underground roadway in coal mine, the drilling robot is designed with two independent systems, i.e., the drilling system and the drilling pipe loading and unloading system, which are in flexible layout. Using the monocular visual identification based on cooperative target, the drilling system is associated with the drilling pipe loading and unloading system for the cooperative operation of the two systems to deliver the drill pipe. However, there are problems such as borehole cinder, cooperative target polluted by sewage, high dust concentration, low visibility, low luminance of artificial light source, uneven illumination resulting in low contrast of target image, and the reflection of cooperative target under the direct lighting of mining lamp in the complex scenarios of drilling construction in the underground coal mine, which will affect the resolution of cooperative target image and reduce the accuracy of target identification and the position to load and unload the drill pipe, thus influencing the success rate of automatic loading and unloading of drill pipe by drilling robot. Therefore, in view of the problem of cooperative target pollution by coal cinder and coal slime returned from the borehole in construction, the open operation of first corrosion and then expansion in morphological filtering was used to remove the discontinuous patches and prominent target pixels of coal cinder. Meanwhile, the closing operation of first expansion and then corrosion was also adopted to remove the small holes and intermittent small gaps, thereby eliminating the impact of solid pollutants on the cooperative targets. Besides, to eliminate the effect of dust, rock dust, water mist and sewage on image contrast, the initial transmittance was obtained by dark channel prior, the transmittance was refined by gradient domain guided filtering, the dark channel was improved by the guided filtering, and the atmospheric light value was solved. On this basis, the defogged image was obtained with the atmospheric scattering model, so as to realize the purpose of sharpening the image and enhancing the image contrast. Finally, sub-pixel corners and directions were refined, and the energy function was optimized. In addition, the seed checkerboard was used to grow checkerboard along the four edges, and the energy changes were used to confirm whether the largest checkerboard was grown, and thus the missing cooperative target image due to strong light reflection was completed. The experimental results show that the combination of morphological filtering, defogging algorithm and growth based checkerboard corner detection method could enhance the resolution of the cooperative target image in the complex scenarios of drilling construction in underground coal mine and improve the image accuracy based on visual identification, laying a foundation for the precise control of automatic loading and unloading of drill pipe by the drilling robot.
-
-
![]()
图 6 强光照射情况下合作靶标图像清晰化
Fig. 6 Cooperative target image sharpening under strong light exposure
表 1 形态学滤波前后特征点坐标值和误差值对比
Table 1 Comparison of coordinates and errors of feature points before and after morphological filtering
组 未滤波角点坐标 形态滤波角点坐标 理论角点坐标 像素差 1 (1 028.30,830.58) (1 028.20,830.47) (1 028.0,830.5) (0.20,0.03) 2 (894.59,823.64) (894.55,823.62) (894.5,823.6) (0.05,0.02) 3 ( 1020.1,1 006.1) (1 020.1,1006.1) (1 020.0,1006.0) (0.1,0.1) 4 (891.05,998.54) (891.05,998.87) (891.0,998.9) (0.05,0.03) 
下载: 导出CSV
表 2 去雾前后特征点坐标值和误差值对比
Table 2 Comparison of coordinates and errors of feature points before and after defogging algorithm
组 未去雾角点坐标 去雾后角点坐标 理论角点坐标 像素差 1 (1 029.0,8310) (1 029.1,831.2) (1 029.0,831.2) (0.1,0) 2 (913.9,825.1) (895.6,824.2) (895.6,824.2) (0,0) 3 (1 021,1007) (1 020.8,1006.8) (1 021.0,1007.0) (0.2,0.2) 4 (910,1 000) (891.5,999.1) (891.5,999.1) (0,0)
下载: 导出CSV
表 3 棋盘格生长前后特征点坐标值和误差值对比
Table 3 Comparison of coordinates and errors of feature points before and after growth based checkerboard corner detection method
组 去雾角点坐标 生长角点坐标 理论角点坐标 像素差 1 (1 029.0,8310) (1 029.1,831.2) (1 029.0,831.2) (0.1,0) 2 (913.9,825.1) (895.6,824.2) (895.6,824.2) (0,0) 3 (1 021,1007) (1 020.8,1006.8) (1 021.0,1007.0) (0.2,0.2) 4 (910,1000) (891.5,999.1) (891.5,999.1) (0,0)
下载: 导出CSV
-
[1] 董洪波,姚宁平,马斌,等. 煤矿井下坑道钻机电控自动化技术研究[J]. 煤田地质与勘探,2020,48(3):219−224. DOI: 10.3969/j.issn.1001-1986.2020.03.031 DONG Hongbo,YAO Ningping,MA Bin,et al. Research on electronically controlled automation technology of underground drilling rig for coal mine[J]. Coal Geology & Exploration,2020,48(3):219−224. DOI: 10.3969/j.issn.1001-1986.2020.03.031
[2] 罗光强,李扬,周策,等. 深孔智能化钻井参数远程监控及工况识别系统研究[J]. 地质与勘探,2022,58(3):696−703. LUO Guangqiang,LI Yang,ZHOU Ce,et al. Intelligent remote monitoring and the work-condition identification system for deep drilling[J]. Geology and Exploration,2022,58(3):696−703.
[3] 王清峰,陈航. 瓦斯抽采智能化钻探技术及装备的发展与展望[J]. 工矿自动化,2018,44(11):18−24. DOI: 10.13272/j.issn.1671-251x.17370 WANG Qingfeng,CHEN Hang. Development and prospect on intelligent drilling technology and equipment for gas drainage[J]. Industry and Mine Automation,2018,44(11):18−24. DOI: 10.13272/j.issn.1671-251x.17370
[4] AUGENSTEIN S, ROCK S M. Improved frame–to–frame pose tracking during vision–only SLAM/SFM with a tumbling target[C]//2011 IEEE International Conference on Robotics and Automation (ICRA). Shanghai, 2011: 3131–3138.
[5] DONG Gangqi,ZHU Z H. Position–based visual servo control of autonomous robotic manipulators[J]. Acta Astronautica,2015,115:291−302. DOI: 10.1016/j.actaastro.2015.05.036
[6] WEN Zhuoman,WANG Yanjie,KUIJPER A,et al. On−orbit real−time robust cooperative target identification in complex background[J]. Chinese Journal of Aeronautics,2015,28(5):1451−1463. DOI: 10.1016/j.cja.2015.08.006
[7] WEN Zhuoman,WANG Yanjie,LUO Jun,et al. Robust,fast and accurate vision–based localization of a cooperative target used for space robotic arm[J]. Acta Astronautica,2017,136:101−114. DOI: 10.1016/j.actaastro.2017.03.008
[8] 王晓鸣,吴高升. 基于单目视觉的水下机器人相对位姿精确控制[J]. 水下无人系统学报,2021,29(3):299−307. WANG Xiaoming,WU Gaosheng. Relative position and attitude precise control of underwater robot based on monocular vision[J]. Journal of Unmanned Undersea Systems,2021,29(3):299−307.
[9] 雷孟宇,张旭辉,杨文娟,等. 煤矿掘进装备视觉位姿检测与控制研究现状与趋势[J]. 煤炭学报,2021,46(增刊2):1135−1148. DOI: 10.13225/j.cnki.jccs.jj21.0534 LEI Mengyu,ZHANG Xuhui,YANG Wenjuan,et al. Current status and trend of research on visual pose detection and control of heading equipment in coal mines[J]. Journal of China Coal Society,2021,46(Sup.2):1135−1148. DOI: 10.13225/j.cnki.jccs.jj21.0534
[10] 吴佳奇, 张文琪, 陈伟, 等. 基于改进CycleGAN的煤矿井下低照度图像增强方法[J/OL]. 华中科技大学学报(自然科学版), 2022: 1–10 [2022-11-16]. DOI: 10.13245/j.hust.229323. WU Jiaqi, ZHANG Wenqi, CHEN Wei, et al. Image enhancement method of low illumination in coal mine based on improved CycleGAN[J/OL]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2022: 1–10 [2022-11-16]. DOI: 10.13245/j.hust.229323.
[11] 智宁,毛善君,李梅. 基于双伽马函数的煤矿井下低亮度图像增强算法[J]. 辽宁工程技术大学学报(自然科学版),2018,37(1):191−197. DOI: 10.11956/j.issn.1008-0562.2018.01.034 ZHI Ning,MAO Shanjun,LI Mei. An enhancement algorithm for coal mine low illumination images based on bi–Gamma function[J]. Journal of Liaoning Technical University(Natural Science),2018,37(1):191−197. DOI: 10.11956/j.issn.1008-0562.2018.01.034
[12] 张谢华,张申,方帅,等. 煤矿智能视频监控中雾尘图像的清晰化研究[J]. 煤炭学报,2014,39(1):198−204. DOI: 10.13225/j.cnki.jccs.2013.0150 ZHANG Xiehua,ZHANG Shen,FANG Shuai,et al. Clearing research on fog and dust images in coalmine intelligent video surveillance[J]. Journal of China Coal Society,2014,39(1):198−204. DOI: 10.13225/j.cnki.jccs.2013.0150
[13] 王启明,李季. 煤矿井下高清图像快速去雾算法研究[J]. 小型微型计算机系统,2018,39(11):2557−2560. DOI: 10.3969/j.issn.1000-1220.2018.11.038 WANG Qiming,LI Ji. Study on fast haze removal algorithm for underground high definition image[J]. Journal of Chinese Computer Systems,2018,39(11):2557−2560. DOI: 10.3969/j.issn.1000-1220.2018.11.038
[14] 曹陇鑫,马宗方,石晶. 基于Retinex的自适应非均匀低照度图像增强算法[J]. 计算机测量与控制,2020,28(10):155−159. DOI: 10.16526/j.cnki.11-4762/tp.2020.10.032 CAO Longxin,MA Zongfang,SHI Jing. Retinex–based adaptive non–uniform low light image enhancement algorithm[J]. Computer Measurement & Control,2020,28(10):155−159. DOI: 10.16526/j.cnki.11-4762/tp.2020.10.032
[15] 马丽娜. 基于机器视觉的磁体器件表面缺陷识别方法研究[D]. 西安: 西安理工大学, 2021. MA Lina. Research on surface defect recognition method of magnet device based on machine vision[D]. Xi’an: Xi’an University of Technology, 2021.
[16] 齐咏生,樊佶,李永亭,等. 基于增强型形态学滤波的风电机组轴承故障诊断方法[J]. 振动与冲击,2021,40(4):212−220. DOI: 10.13465/j.cnki.jvs.2021.04.029 QI Yongsheng,FAN Ji,LI Yongting,et al. A fault diagnosis method of wind turbine bearings based on an enhanced morphological filter[J]. Journal of Vibration and Shock,2021,40(4):212−220. DOI: 10.13465/j.cnki.jvs.2021.04.029
[17] HE Kaiming, SUN Jian, TANG Xiao’ou. Single image haze removal using dark channel prior[C]//2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009). Miami, 2009: 1956–1963.
[18] HE Kaiming,SUN Jian,TANG Xiao’ou. Guided image filtering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(6):1397−1409. DOI: 10.1109/TPAMI.2012.213
[19] KOU Fei,CHEN Weihai,WEN Changyun,et al. Gradient domain guided image filtering[J]. IEEE Transactions on Image Processing,2015,24(11):4528−4539. DOI: 10.1109/TIP.2015.2468183
[20] 火元莲,郑海亮,李明,等. 基于融合亮度模型和梯度域滤波的图像去雾[J]. 计算机工程与科学,2021,43(9):1623−1633. HOU Yuanlian,ZHENG ailian,LI Ming,et al. Image dehazing based on fusiong luminance model and gradient domain filter[J]. Computer Eengineering & Science,2021,43(9):1623−1633.
[21] 王健,何自豪,刘洁,等. 基于梯度域导向滤波器和改进PCNN的图像融合算法[J]. 系统工程与电子技术,2022,44(8):2381−2392. WANG Jian,HE Zihao,LIU Jie,et al. Image fusion algorithm based on gradient domain guided filtering and improved PCNN[J]. Systems Engineering and Electronics,2022,44(8):2381−2392.
[22] GEIGER A, MOOSMANN F, CAR O, et al. Automatic camera and range sensor calibration using a single shot[C]//2012 IEEE International Conference on Robotics & Automation. Saint Paul, 2012: 3936–3943.
[23] RUFLI M, SCARAMUZZA D, SIEGWART R. Automatic detection of checkerboards on blurred and distorted images[C]//2008 IEEE/RSJ International Conference on Intelligent Robots and Systems. Nice, 2008.
[24] ORGHIDAN R,MOUADDIB E M,SALVI J,et al. Catadioptric single–shot rangefinder for textured map building in robot navigation[J]. IET Computer Vision,2007,1(2):43−53. DOI: 10.1049/iet-cvi:20065003
计量
- 文章访问数: 159
- HTML全文浏览量: 12
- PDF下载量: 28
