Cooperative teams of Unmanned Aerial Vehicles (UAVs) transporting a cable-suspended pay- load offer many advantages over single carriers such as higher payload capacity, longer range and more robustness. While multirotor UAVs are more agile and have the ability to hover, fixed- wing UAVs have better energy efficiency and therefore higher operational range. However, they cannot hover, each carrier must maintain a strictly positive velocity. Accordingly, if the payload is required to remain stationary, the carriers must coordinate their movement such that the forces they exert on the payload by their movement gets cancelled out. Such coordination is likewise necessary in other phases of motion. Prior work solved this coordination with a centralized con- trol architecture that calculates the carriers' trajectories by solving a non-linear program (NLP) online for each time step. This approach, however, is computationally expensive and difficult to deploy onboard. This thesis asks whether a fully decentralized, learning-based approach can accomplish the same task. The solution is developed in two phases. In the first phase, the NLP-based optimizer that was previously presented is distilled using Imitation Learning (IL) into a neural network/student policy. Consequently, the computational cost is greatly reduced while also allowing the calculation to be executed on a GPU. The resulting student policy is able to mimic the optimizer accurately and shows good generalization across various durations, movement magnitudes and directions. In the second phase, the student policy along with a load- wrench controller are deployed in a decentralized manner using only local, potentially noisy and asynchronous observations. A residual multi-agent reinforcement learning (MARL) policy is trained on top of them to compensate for the loss and corruption of global information resulting from the removal of the centralized architecture. The residual MARL policy enables the de- centralized control architecture to operate effectively, substantially stabilizing the load tracking and the carrier trajectories on some trajectories. The results indicate that the policy generalizes well across different flight durations and movement magnitudes. On the contrary, generaliza- tion across movement directions remains more limited. Diagnosis traces this limitation to a high variance in the training process when using a diverse set of trajectories. In this regard, larger batch sizes may help achieve more stable training. The current CPU-based plant model limits the number of parallel environments, and hence the batch sizes that can be collected within a reasonable time. Addressing this limitation will therefore require migrating the plant model to the GPU and investigating other strategies that improve generalization.
Squadre cooperative di Unmanned Aerial Vehicles (UAV) che trasportano un carico sospeso tramite un cavo offrono numerosi vantaggi rispetto ai singoli vettori, come una maggiore capac- ità di carico, una maggiore autonomia e una maggiore robustezza. Mentre i UAV multirotore sono più agili e hanno la capacità di stazionare in volo, i UAV ad ala fissa hanno una migliore effi- cienza energetica e quindi una maggiore autonomia operativa. Tuttavia, non possono stazionare in volo e ogni vettore deve mantenere una velocità strettamente positiva. Di conseguenza, se il carico deve rimanere fermo, i vettori devono coordinare il loro movimento in modo tale che le forze che esercitano sul carico attraverso il loro movimento si annullino. Tale coordinazione è necessaria anche nelle altre fasi del movimento. Lavori precedenti hanno risolto questo prob- lema di coordinazione mediante un'architettura di controllo centralizzata che calcola le traiettorie dei vettori risolvendo online, a ogni istante temporale, un programma non lineare (NLP). Questo approccio, tuttavia, è computazionalmente oneroso e difficile da implementare a bordo. Questa tesi si chiede se un approccio completamente decentralizzato e basato sull'apprendimento possa svolgere lo stesso compito. La soluzione è sviluppata in due fasi. Nella prima fase, l'ottimiz- zatore basato su NLP precedentemente presentato viene distillato, mediante Imitation Learning (IL), in una rete neurale/student policy. Di conseguenza, il costo computazionale è notevolmente ridotto, consentendo inoltre di eseguire il calcolo su una GPU. La student policy risultante è in grado di imitare accuratamente l'ottimizzatore e mostra una buona generalizzazione attraverso diverse durate, ampiezze di movimento e direzioni. Nella seconda fase, la student policy in- sieme a un controllore load-wrench vengono implementati in modo decentralizzato utilizzando solamente osservazioni locali, potenzialmente rumorose e asincrone. Una policy di multi-agent reinforcement learning (MARL) residuale viene addestrata al di sopra di essi per compensare la perdita e la corruzione delle informazioni globali derivanti dalla rimozione dell'architettura centralizzata. La policy MARL residuale consente all'architettura di controllo decentralizzata di operare efficacemente, stabilizzando significativamente il tracking del carico e le traiettorie dei vettori su alcune traiettorie. I risultati indicano che la policy generalizza bene attraverso diverse durate di volo e ampiezze di movimento. Al contrario, la generalizzazione attraverso le direzioni di movimento rimane più limitata. La diagnosi riconduce questa limitazione a un'ele- vata varianza nel processo di addestramento quando viene utilizzato un insieme diversificato di traiettorie. In questo senso, batch di dimensioni maggiori possono aiutare a ottenere un addestra- mento più stabile. L'attuale modello della dinamica basato su CPU limita il numero di ambienti paralleli e, di conseguenza, le dimensioni dei batch che possono essere raccolti in un tempo ragionevole. Affrontare questa limitazione richiederà di migrare il modello della dinamica su GPU e di investigare altre strategie che migliorino la generalizzazione.
Learning Coordinated Trajectories for Non-Stop Flying Carriers Holding a Cable-Suspended Load
ABOULENIEN, AHMED HISHAM ABDELGHAFFAR
2025/2026
Abstract
Cooperative teams of Unmanned Aerial Vehicles (UAVs) transporting a cable-suspended pay- load offer many advantages over single carriers such as higher payload capacity, longer range and more robustness. While multirotor UAVs are more agile and have the ability to hover, fixed- wing UAVs have better energy efficiency and therefore higher operational range. However, they cannot hover, each carrier must maintain a strictly positive velocity. Accordingly, if the payload is required to remain stationary, the carriers must coordinate their movement such that the forces they exert on the payload by their movement gets cancelled out. Such coordination is likewise necessary in other phases of motion. Prior work solved this coordination with a centralized con- trol architecture that calculates the carriers' trajectories by solving a non-linear program (NLP) online for each time step. This approach, however, is computationally expensive and difficult to deploy onboard. This thesis asks whether a fully decentralized, learning-based approach can accomplish the same task. The solution is developed in two phases. In the first phase, the NLP-based optimizer that was previously presented is distilled using Imitation Learning (IL) into a neural network/student policy. Consequently, the computational cost is greatly reduced while also allowing the calculation to be executed on a GPU. The resulting student policy is able to mimic the optimizer accurately and shows good generalization across various durations, movement magnitudes and directions. In the second phase, the student policy along with a load- wrench controller are deployed in a decentralized manner using only local, potentially noisy and asynchronous observations. A residual multi-agent reinforcement learning (MARL) policy is trained on top of them to compensate for the loss and corruption of global information resulting from the removal of the centralized architecture. The residual MARL policy enables the de- centralized control architecture to operate effectively, substantially stabilizing the load tracking and the carrier trajectories on some trajectories. The results indicate that the policy generalizes well across different flight durations and movement magnitudes. On the contrary, generaliza- tion across movement directions remains more limited. Diagnosis traces this limitation to a high variance in the training process when using a diverse set of trajectories. In this regard, larger batch sizes may help achieve more stable training. The current CPU-based plant model limits the number of parallel environments, and hence the batch sizes that can be collected within a reasonable time. Addressing this limitation will therefore require migrating the plant model to the GPU and investigating other strategies that improve generalization.| File | Dimensione | Formato | |
|---|---|---|---|
|
Aboulenien_Ahmed Hisham Abdelghaffar.pdf
accesso aperto
Dimensione
14.7 MB
Formato
Adobe PDF
|
14.7 MB | Adobe PDF | Visualizza/Apri |
The text of this website © Università degli studi di Padova. Full Text are published under a non-exclusive license. Metadata are under a CC0 License
https://hdl.handle.net/20.500.12608/119109