The present paper introduces a smart trajectories generation algorithm for unmanned aerial vehicles under various environments. Dynamic movement primitive is extended by adding jerk to mock the kinematics, particularly for unmanned aerial vehicles. Combining the improved dynamic movement primitive with policy learning by weighted exploration with the returns, we propose the new algorithm producing optimal trajectories under different scenarios. Furthermore, numerical simulations under several scenarios are performed, demonstrating the ability of the proposed algorithm.
GurdanDStumpfJAchtelikM. Energy-efficient autonomous four-rotor flying robot controlled at 1 khz. In: Robotics and automation, 2007 IEEE international conference, Roma, Italy, 10–14 April 2007, pp.361–366. IEEE.
3.
HeRBachrachAAchtelikM. On the design and use of a micro air vehicle to track and avoid adversaries. Int J Robot Res2009; 29(5): 529–546.
4.
LupashinSScholligASherbackM. A simple learning strategy for high-speed quadrocopter multi-flips. In: Robotics and automation (ICRA), 2010 IEEE international conference, Anchorage, Alaska, USA, 3–8 May 2010, pp.1642–1648.
5.
SchouwenaarsT. Safe trajectory planning of autonomous vehicles. PhD Thesis, Massachusetts Institute of Technology, USA, 2006.
6.
LatombeJC.Robot motion planning. Springer Science & Business Media, 2012.
7.
ShinKGMcKayND. A dynamic programming approach to trajectory planning of robotic manipulators. IEEE T Automat Contr1986; 31(6): 491–500.
8.
HargravesCRParisSW. Direct trajectory optimization using nonlinear programming and collocation. J Guid Control Dynam1987; 10(4): 338–342.
9.
BettsJT. Survey of numerical methods for trajectory optimization. J Guid Control Dynam1998; 21(2): 193–207.
10.
RichardsAHowJSchouwenaarsT. Plume avoidance maneuver planning using mixed integer linear programming. In: Proceedings of the AIAA guidance, navigation, and control conference, 6–9 August 2001, Montreal, Canada, pp.6–9.
11.
RichardsASchouwenaarsTHowJP. Spacecraft trajectory planning with avoidance constraints using mixed-integer linear programming. J Guid Control Dynam2002; 25(4): 755–764.
12.
SchouwenaarsTDe MoorBFeronE. Mixed integer programming for multi-vehicle path planning. In: 2001 European control conference (ECC), Porto, Portugal, 4 September 2001, pp.2603–2608. IEEE.
13.
MellingerDKumarV. Minimum snap trajectory generation and control for quadrotors. In: Robotics and automation (ICRA), 2011 IEEE international conference, 9–13 May 2011Shanghai, China, pp.2520–2525.
14.
MellingerDKushleyevAKumarV. Mixed-integer quadratic program trajectory generation for heterogeneous quadrotor teams. Robotics and automation (ICRA), 2012 IEEE international conference, 14–18 May 2012, St. Paul, Minnesota, USA, pp.477–483.
15.
RichterCBryARoyN. Polynomial trajectory planning for quadrotor flight. In: Robotics and automation (ICRA), 2013 IEEE international conference, Karlsruhe, 6–10 May 2013.
16.
YaoPWangHSuZ. UAV feasible path planning based on disturbed fluid and trajectory propagation. Chinese J Aeronaut2015; 28(4): 1163–1177.
17.
PopovaMGLiuHHT. Position-based visual servoing for target tracking by a quadrotor UAV. In: AIAA guidance, navigation, and control conference, San Diego, California, USA, 4–8 January 2016, p.2092.
18.
PetersJSchaalS. Reinforcement learning of motor skills with policy gradients. Neural Netw2008; 21(4), 682–697.
19.
SpicaRFranchiAOrioloG. Aerial grasping of a moving target with a quadrotor UAV. In: Intelligent robots and systems (IROS), 2012, IEEE/RSJ international conference, Vilamoura, Algarve, Portugal, 7–12 October 2012, pp.4985–4992.
20.
SaripalliSSukhatmeG. Landing on a moving target using an autonomous helicopter. In: Field and service robotics, 2006, pp.277–286. Berlin, Heidelberg: Springer.
21.
GarimellaGKobilarovM. Towards model-predictive control for aerial pick-and-place. In: IEEE international conference on robotics and automation (ICRA), Seattle, Washington, USA, 26–30 May 2015, pp.4692–4697.
22.
SuttonRSBartoAGWilliamsRJ. Reinforcement learning is direct adaptive optimal control. IEEE Contr Syst Mag1992; 12(2): 19–22.
23.
ArgallBDChernovaSVelosoM. A survey of robot learning from demonstration. Robot Auton Syst2009; 57(5): 469–483.
24.
KoberJPetersJ. Reinforcement learning in robotics: A survey. In: Reinforcement learning 2012, pp.579–610. Berlin, Heidelberg: Springer.
25.
SuttonRSBartoAG. Reinforcement learning: An introduction. Cambridge: MIT Press, 1998.
26.
AbbeelPCoatesAQuigleyM. An application of reinforcement learning to aerobatic helicopter flight. Adv Neural Inf Process Syst2007; 19: 1–8.
NgACoatesADielM. Autonomous inverted helicopter flight via reinforcement learning. Experimental Robotics IX. 2006; 363–372.
29.
PalunkoIFaustACruzP. A reinforcement learning approach towards autonomous suspended load manipulation using aerial robots. In: Robotics and automation (ICRA), 2013 IEEE international conference, Karlsruhe, 6–10 May 2013, pp.4896–4901.
30.
LouWGuoX. Adaptive trajectory tracking control using reinforcement learning for quadrotor. Intl J Advanced Robot Syst2016; 13(1): 38.
31.
Bing-QiangHGuang-YiCMinG. Reinforcement learning neural network to the problem of autonomous mobile robot obstacle avoidance. In: Proceedings of 2005 international conference on machine learning and cybernetics2005; 1: 85–89.
32.
MartinTHHowardBDOrlando DeJ. An introduction to the use of neural networks in control systems. Int J Robust Nonlin2002; 12(11): 959–985.
33.
IjspeertAJNakanishiJSchaalS. Movement imitation with nonlinear dynamical systems in humanoid robots. In: IEEE international conference on robotics and automation, Washington, DC, USA, 11–15 May 2002, pp.1398–1403.
34.
BagnellJANgAYKakadeS. Policy search by dynamic programming. Adv Neural Inf Process Syst2003; 1–8.
35.
KoberJPetersJ. Policy search for motor primitives in robotics. Mach Learn2011; 84(1–2): 171–203.
36.
PetersJSchaalS. Reinforcement learning by reward-weighted regression for operational space control. In: Proceedings of the 24th international conference on machine learning, Corvallis, OR, USA, 20–24 June 2007, pp.745–750.
37.
Auke JanIJunNStefanS. Learning attractor landscapes for learning motor primitives. Adv Neural Inf Process Syst2003; 1523–1530.
38.
KoberJMüllingKKrmerO. Movement templates for learning of hitting and batting. In: Robotics and automation (ICRA), 2010 IEEE international conference on Egan convention center, Anchorage, AK, USA, 3–8 May 2010, pp. 853–858. IEEE.
39.
ParkDHHoffmannHPastorP. Movement reproduction and obstacle avoidance with dynamic movement primitives and potential fields. In: Humanoid robots, 2008 8th IEEE-RAS international conference, Hotel Riviera & KAIST Daejeon, Korea (South), 1–3 December 2008, pp.91–98.
40.
ZuoZ. Trajectory tracking control design with command-filtered compensation for a quadrotor. IET Control Theory A2010; 4(11): 2343–2355.
41.
TayebiAMcGilvrayS. Attitude stabilization of a four-rotor aerial robot. In: Decision and control, 2004. CDC. 43rd IEEE conference, Atlantis, Paradise Island, Bahamas, 14–17 December 2004, vol. 2, pp.1216–1221.
42.
YangquanCZhongminW. Formation control: A review and a new consideration. In: Intelligent robots and system, IEEE/RSJ international conference, 2005.
