This paper discusses integrating Unmanned Aerial Vehicle (UAV) data, Artificial Intelligence (AI), and Mixed Reality (MR) as a hybrid system for monitoring mine site rehabilitation. UAV systems are reviewed and the application of MR and AI to mining is discussed. The most common UAVs are low-cost multi-rotor systems however, their capabilities may be limited for long-term autonomous operation. The application scope is broad but identifies many trial UAV applications and sensors that could be integrated with MR and AI to produce an autonomous system. A simplified model for autonomous UAV, AI, and MR integrated systems is suggested along with some core modules. Via collaborative, modular development, the integration of UAV data, AI, and MR may be possible. Such a system will provide a powerful tool for mine site rehabilitation and compliance sign-off via a visual tool that allows all stakeholders to comprehend processes and data and agree when final rehabilitation is achieved.
AliD, FrimpongS.2020. Artificial intelligence, machine learning and process automation: existing knowledge frontier and way forward for mining sector. Artif Intell Rev. 53 (8): 6025 – 6042.
3.
Al-TurjmanF, ZahmatkeshH.2020. A comprehensive review on the use of AI in UAV communications: enabling technologies, applications, and challenges. In: Al-TurjmanF.Unmanned aerial vehicles in smart cities. Cham: Springer ; p. 1–26.
AntoniouV, BonaliFL, NomikouP, TibaldiA, MelissinosP, MariottoFP, VitelloFR, KrokosM, WhitworthM.2020. Integrating virtual reality and GIS tools for geological mapping, data collection and analysis: an example from the Metaxa Mine, Santorini (Greece). Appl Sci. 10 (23): 8317.
6.
BamfordT, MedinacF, EsmaeiliK.2020. Continuous monitoring and improvement of the blasting process in open pit mines using unmanned aerial vehicle techniques. Remote Sens. 12 (17): 2801.
7.
BanerjeeBP, RavalS, CullenPJ.2020. UAV-hyperspectral imaging of spectrally complex environments. Int J Remote Sens. 41 (11): 4136 – 4159.
8.
CastendykD, StraightB, VoorhisJV, SomogyiM, JepsonWP, KučeraB.2019. Using aerial drones to select sample depths in pit lakes. Proceedings of the 13th International Conference on Mine Closure.
9.
DonnellyLJ, De La CruzH, AsmarI, ZapataO, PerezJD.2001. The monitoring and prediction of mining subsidence in the Amaga, Angelopolis, Venecia and Bolombolo regions, Antioquia, Colombia. Eng Geol. 59 (1): 103 – 114.
10.
ElmokademT, SavkinAV.2021. Towards fully autonomous UAVs: a survey. Sensors (Basel, Switzerland). 21 (18): 6223.
11.
GangadharA, MurugaiahM, RajaramD, MavrisD.2021. Conceptual design and feasibility study of winged hybrid airship. Aerospace. 9 (1): 8.
12.
GiacomoABD, BarretoJ, TeixeiraJB, OliveiraL, CajaíbaL, JoyeuxJ-C, BarcelosN, MartinsAS.2021. Using drones and ROV to assess the vulnerability of marine megafauna to the Fundão tailings dam collapse. Sci Total Environ. 800: 149302.
13.
GiordanD, AdamsM, AicardiI, AlicandroM, AllasiaP, BaldoM, BerardinisP, DominiciD, GodoneD, HobbsP,2020. The use of unmanned aerial vehicles (UAVs) for engineering geology applications. Bull Eng Geol Environ. 79: 3437 – 3481.
14.
HirschmüllerH.2011. Semi-global matching: motivation, development and applications.
15.
HuangHM.2008. Autonomy levels for unmanned systems (ALFUS) framework. Gaithersburg (MD): Volume I: Terminology.
16.
HugenholtzC, WalkerJ, BrownO, MyshakS.2015. Earthwork volumetrics with an unmanned aerial vehicle and softcopy photogrammetry. J Surveying Eng. 141: 06014003.
17.
IversonS, SignerS.2014. Assessment and detection of loose rock hazards in underground metal mines using thermal imaging. In: SME Annual Meeting & Exhibit (SME 2014): Leadership in Uncertain Times; Salt Lake City. SME; p. 666–672.
18.
JangH, TopalE.2020. Transformation of the Australian mining industry and future prospects. Min Technol. 129 (3): 120 – 134.
19.
JaniszewskiM, UotinenL, MerkelJ, LeveinenJ, RinneM.. 2020. Virtual reality learning environments for rock engineeringIn 54th US Rock Mechanics/Geomechanics Symposium. OnePetro.
JohansenK, ErskinePD, McCabeMF.2019. Using unmanned aerial vehicles to assess the rehabilitation performance of open cut coal mines. J Cleaner Prod. 209: 819 – 833.
22.
KaturuzaM, BirchC.2019. The use of unmanned aircraft system technology for highwall mapping at Isibonelo Colliery, South Africa. J South Afr Inst Min Metall. 119: 291 – 295.
23.
KongD, SaroglouH, WuF, ShaP, LiB.2021. Development and application of UAV-SfM photogrammetry for quantitative characterization of rock mass discontinuities. Int J Rock Mech Min Sci. 141: 104729.
24.
LaksonoD, AdityaT.2019. Utilizing a game engine for interactive 3D topographic data visualization. ISPRS Int J Geoinf. 8 (8): 361.
25.
LaurenceDC.2011. Establishing a sustainable mining operation: an overview. J Cleaner Prod. 19 (2-3): 278 – 284.
26.
LeBlanc-SmithG, CarisC, CarterG.1998. Interactive evaluation of mine plans with integrated geological exploration data. Virtual mine prototype: interactive visualisation methods using VRML, Java and an internet browser, CSIRO.
27.
LeeS, ChoiY.2016. Reviews of unmanned aerial vehicle (drone) technology trends and its applications in the mining industry. Geosyst Eng. 19 (4): 197 – 204.
28.
Lyons-BaralJ, KemenyJ.2016. Applications of point cloud technology in geomechanical characterization, analysis and predictive modeling. In: 2016 SME Annual Conference and Expo: The Future for Mining in a Data-Driven World, Society for Mining, Metallurgy and Exploration; p. 657–663.
29.
McKennaPB, LechnerAM, PhinnS, ErskinePD.2020. Remote sensing of mine site rehabilitation for ecological outcomes: a global systematic review. Remote Sens (Basel). 12 (21): 3535.
30.
MilgramP, KishinoF.1994. A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst. E77-D (12): 1321 – 1329.
31.
NeighCSR, BoltonDK, DiabateM, WilliamsJJ, CarvalhaisN.2014. An automated approach to map the history of forest disturbance from insect mortality and harvest with landsat time-series data. Remote Sens (Basel). 6 (4): 2782 – 2808.
32.
OnifadeM, GencB.2019. A review of spontaneous combustion studies – South African context. Int J Min Reclam Environ. 33 (8): 527 – 547.
PadróJ-C, CarabassaV, BalaguéJ, BrotonsL, AlcañizJM, PonsX.2019. Monitoring opencast mine restorations using unmanned aerial system (UAS) imagery. Sci Total Environ. 657: 1602 – 1614.
35.
PuY, ApelDB, LiuV, MitriH.2019. Machine learning methods for rockburst prediction-state-of-the-art review. Int J Min Sci Technol. 29 (4): 565 – 570.
36.
PuniachE, GruszczyńskiW, ĆwiąkałaP, MatwijW.2021. Application of UAV-based orthomosaics for determination of horizontal displacement caused by underground mining. ISPRS J Photogramm Remote Sens. 174: 282 – 303.
37.
RavalS, BanerjeeB.2018. Improved structural mapping of pit walls using UAV-based mobile laser scanning.
38.
RavalS, BanerjeeBP, ShenX, MasoumiH, TannantD.2018. Evaluation of a UAV-LiDAR system for mapping geological structures in an open pit highwall. In: AusRock 2018: The Fourth Australasian Ground Control in Mining Conference; Sydney, NSW. AusIMM. p. 496–503.
39.
RossiP, ManciniF, DubbiniM, MazzoneF, CapraA.2017. Combining nadir and oblique UAV imagery to reconstruct quarry topography: methodology and feasibility analysis. Eur J Remote Sens. 50 (1): 211 – 221.
40.
RussellS, NorvigP.2022. Artificial intelligence: a modern approach. Uniteted States: Pearson.
41.
SaidK, OnifadeM, LawalA, GithiriaJ.2020. An artificial intelligence-based model for the prediction of spontaneous combustion liability of coal based on its proximate analysis. Combust Sci Technol. 19: 2350 – 2367.
42.
SalviniR, MastroroccoG, EspositoG, Di BartoloS, CogganJ, VanneschiC.2018. Use of a remotely piloted aircraft system for hazard assessment in a rocky mining area (Lucca, Italy). Nat Hazards Earth Syst Sci Discuss. 18: 287 – 302.
43.
SayabM, AerdenD, PaananenM, SaarelaP.2018. Virtual structural analysis of Jokisivu open pit using ‘structure-from-motion’ unmanned aerial vehicles (UAV) photogrammetry: implications for structurally-controlled gold deposits in southwest Finland. Remote Sens (Basel). 10: 1296.
44.
SeoJ.2015. A study on generation of 3D model and mesh image of excavation work using UAV. In: Proceedings of the 32nd International Symposium on Automation and Robotics in Construction and Mining (ISARC 2015). International Association for Automation and Robotics in Construction (IAARC). p. 1–7.
45.
ShahmoradiJ, TalebiE, RoghanchiP, HassanalianM.2020. A comprehensive review of applications of drone technology in the mining industry. Drones. 4 (3): 34.
46.
Shirani FaradonbehR, TaheriA, KarakusM.2022. The propensity of the over-stressed rock masses to different failure mechanisms based on a hybrid probabilistic approach. Tunnelling Underground Space Technol. 119: 104214.
47.
SofiaG, MarinelloF.2014. A new landscape metric for the identification of terraced sites: the slope local length of auto-correlation (SLLAC). ISPRS J Photogramm Remote Sens. 96: 123 – 133.
48.
SofoniaJ, GrayJ.2021. How mobile lidar data delivers better outcomes at BHP's Olympic dam mine. AusIMM bulletin. Melbourne: AusIMM.
49.
StraightBJ, CastendykDN, McKnightDM, NewmanCP, FiliatreaultP, PinoA.2021. Using an unmanned aerial vehicle water sampler to gather data in a pit-lake mining environment to assess closure and monitoring. Environ Monit Assess. 193 (9): 572.
50.
TscharfA, RumplerM, FraundorferF, MayerG, BischofH.2015. On the use of UAVs in mining and archaeology-geo-accurate 3D reconstructions using various platforms and terrestrial views. ISPRS Ann Photogrammetry Remote Sens Spatial Inf Sci. 2 (W1): 15 – 22.
51.
VasukiY, HoldenE-J, KovesiP, MicklethwaiteS.2014. Semi-automatic mapping of geological structures using UAV-based photogrammetric data: an image analysis approach. Comput Geosci. 69: 22 – 32.
52.
VuT, BaoT, HoangQV, DrebenstetdC, HoaPV, ThangHH.2021. Measuring blast fragmentation at Nui Phao open-pit mine, Vietnam using the Mask R-CNN deep learning model. Min Technol. 130 (4): 232 – 243.
53.
WangY, HuX, ZhongY, MaA, WeiL, ZhangL.2019. Tailings reservoir disaster and environmental monitoring using the UAV-ground hyperspectral joint observation and processing: a case of study in Xinjiang, the Belt and Road. IGARSS 2019 – 2019 IEEE International Geoscience and Remote Sensing Symposium.
54.
WangY, TianF, HuangY, WangJ, WeiC.2015. Monitoring coal fires in Datong coalfield using multi-source remote sensing data. Trans Nonferrous Metals Soc China. 25: 3421 – 3428.
55.
YangL, YangD, WuJ.2020. An automated visible / infrared image analysis system of unmanned aerial vehicles (UAVs). In: 2020 IEEE 9th Data Driven Control and Learning Systems Conference (DDCLS). p. 1103–1108.
