Publications

Version: 1.0 Approved

LDO-ID 2021-06-08-09

Title A framework based on Deep Learning techniques for multi-drone ISR missions performance evaluation in different synthetic environments

Abstract

This study aims to focus on how the synthetic world of today's simulators can act in synergy with that of neural networks and Deep Learning applied to video analysis, in particular using multi-drone/swarm systems for ISR missions. It is a fact that modern virtual engines used in both simulation and gaming have reached a level of realism that confuses a not so attentive observer. The question that arises spontaneously is, therefore, whether even an AI can be "cheated", and consequently the behaviour and decisions on-board the drone, changing the resulting actions of the fleet. That is, in more scientific terms, to evaluate whether the type and number of features that automatic learning systems on-board the drones, such as neural networks, can extract from synthetic images and can be reflected in the continuous world with significant advantages in the delicate and / or expensive phase of the training process, such as dataset creation and pre-exercise test. In fact, being able to model the elements in the simulated world at will, it is possible to reliably recreate situations and scenarios even impossible to recreate in real life (a network for the detection of lightning or explosions, for example), allowing the creation of datasets congruent in size according to a modern Deep Learning approach and reducing the physical times for the recovery of these images, also taking into account the constraints of the on-board computing power and capacity. Furthermore, it is worth asking whether the minor detail due to the discretization of the real scenario can act, in some circumstances, as a Principal Component Analysis (PCA) filter within the pre-processing of a dataset, during the dimensionality reduction process. The approach of the presented study will be experimental and will foresee a double direction of application. In a first phase we want to understand how neural networks trained on real datasets, on-board of one or more drones, behave in different synthetic environments. Three different simulators will be examined, namely VRForces, ROS Gazebo and VBS4, in order to also understand how the increasing of the graphical details will affect the accuracy and the precision-recall curve. The study presented in the proposed paper involves the area of AI object recognition and tracking, with particular focus on the problem of localization and therefore the detection of particular classes of objects such as people and vehicles. In a second phase of our research, the networks will be ready to be deployed taking into account the possible preparation of a hardware-in-the-loop use of COTS or custom autopilots, simulating real scenarios of ISR missions using a cooperative and intelligent fleet of drones. For this stage, our expertise is centred on a project called SWARM: a large industrial R&D Vitrociset project. It is an AI-Enabled Command and Control (C&C) system, able to execute and review ISR missions for mini/micro cooperative fleets of heterogeneous UAVs. SWARM will be used as testbed for the presented framework, testing and evaluating Deep Learning techniques for multi-drone ISR missions in different synthetic environments.

Authors Antenucci Alessio, Messina Luca, Palumbo Andrea, Mazzaro Silvio, Matta Walter

Type Paper for Seminar/Symposium/Conference

Media NATO S&TO (Science & Technology) SCI-341 Research Symposium on "Situation Awareness of Swarms and Autonomous Systems" (18-19 May 2021, Virtual Conference)

Web site

https://www.sto.nato.int/publications/STO Meeting Proceedings/STO-MP-SCI-341/MP-SCI-341-07.pdf

Anno 2021

Cancel