id sid tid token lemma pos fcis-32134 1 1 frontiers frontier NOUN fcis-32134 1 2 in in ADP fcis-32134 1 3 computing computing NOUN fcis-32134 1 4 and and CCONJ fcis-32134 1 5 intelligent intelligent ADJ fcis-32134 1 6 systems system NOUN fcis-32134 1 7 issn issn VERB fcis-32134 1 8 : : PUNCT fcis-32134 1 9 2832 2832 NUM fcis-32134 1 10 - - SYM fcis-32134 1 11 6024 6024 NUM fcis-32134 1 12 | | NOUN fcis-32134 1 13 vol vol NOUN fcis-32134 1 14 . . PUNCT fcis-32134 2 1 14 14 NUM fcis-32134 2 2 , , PUNCT fcis-32134 2 3 no no INTJ fcis-32134 2 4 . . NOUN fcis-32134 2 5 1 1 NUM fcis-32134 2 6 , , PUNCT fcis-32134 2 7 2025 2025 NUM fcis-32134 2 8 42 42 NUM fcis-32134 2 9 research research NOUN fcis-32134 2 10 on on ADP fcis-32134 2 11 a a DET fcis-32134 2 12 sac‐based sac‐base VERB fcis-32134 2 13 algorithm algorithm NOUN fcis-32134 2 14 for for ADP fcis-32134 2 15 uav uav PROPN fcis-32134 2 16 autonomous autonomous PROPN fcis-32134 2 17 navigation navigation PROPN fcis-32134 2 18 and and CCONJ fcis-32134 2 19 obstacle obstacle NOUN fcis-32134 2 20 avoidance avoidance NOUN fcis-32134 2 21 zewen zewen PROPN fcis-32134 2 22 cui cui PROPN fcis-32134 2 23 * * PUNCT fcis-32134 2 24 college college NOUN fcis-32134 2 25 of of ADP fcis-32134 2 26 intelligent intelligent ADJ fcis-32134 2 27 systems system NOUN fcis-32134 2 28 science science NOUN fcis-32134 2 29 and and CCONJ fcis-32134 2 30 engineering engineering NOUN fcis-32134 2 31 , , PUNCT fcis-32134 2 32 harbin harbin PROPN fcis-32134 2 33 engineering engineering PROPN fcis-32134 2 34 university university PROPN fcis-32134 2 35 , , PUNCT fcis-32134 2 36 harbin harbin PROPN fcis-32134 2 37 , , PUNCT fcis-32134 2 38 heilongjiang heilongjiang PROPN fcis-32134 2 39 province province PROPN fcis-32134 2 40 , , PUNCT fcis-32134 2 41 150001 150001 NUM fcis-32134 2 42 , , PUNCT fcis-32134 2 43 china china PROPN fcis-32134 2 44 * * PUNCT fcis-32134 2 45 corresponding correspond VERB fcis-32134 2 46 author author NOUN fcis-32134 2 47 email email NOUN fcis-32134 2 48 : : PUNCT fcis-32134 3 1 cuizewan_123@163.com cuizewan_123@163.com X fcis-32134 3 2 abstract abstract NOUN fcis-32134 3 3 : : PUNCT fcis-32134 3 4 the the DET fcis-32134 3 5 traditional traditional ADJ fcis-32134 3 6 reinforcement reinforcement NOUN fcis-32134 3 7 learning learning NOUN fcis-32134 3 8 algorithms algorithm NOUN fcis-32134 3 9 face face VERB fcis-32134 3 10 the the DET fcis-32134 3 11 problems problem NOUN fcis-32134 3 12 of of ADP fcis-32134 3 13 poor poor ADJ fcis-32134 3 14 complex complex ADJ fcis-32134 3 15 spatial spatial ADJ fcis-32134 3 16 adaptability adaptability NOUN fcis-32134 3 17 and and CCONJ fcis-32134 3 18 inefficient inefficient ADJ fcis-32134 3 19 exploration exploration NOUN fcis-32134 3 20 in in ADP fcis-32134 3 21 the the DET fcis-32134 3 22 autonomous autonomous PROPN fcis-32134 3 23 uav uav PROPN fcis-32134 3 24 pathfinding pathfinding PROPN fcis-32134 3 25 and and CCONJ fcis-32134 3 26 obstacle obstacle NOUN fcis-32134 3 27 avoidance avoidance NOUN fcis-32134 3 28 tasks task NOUN fcis-32134 3 29 in in ADP fcis-32134 3 30 three three NUM fcis-32134 3 31 - - PUNCT fcis-32134 3 32 dimensional dimensional ADJ fcis-32134 3 33 unknown unknown ADJ fcis-32134 3 34 environments environment NOUN fcis-32134 3 35 . . PUNCT fcis-32134 4 1 to to ADP fcis-32134 4 2 this this DET fcis-32134 4 3 end end NOUN fcis-32134 4 4 , , PUNCT fcis-32134 4 5 this this DET fcis-32134 4 6 paper paper NOUN fcis-32134 4 7 combines combine VERB fcis-32134 4 8 the the DET fcis-32134 4 9 dynamically dynamically ADV fcis-32134 4 10 decayingπœ€-greedy decayingπœ€-greedy ADJ fcis-32134 4 11 exploration exploration NOUN fcis-32134 4 12 strategy strategy NOUN fcis-32134 4 13 , , PUNCT fcis-32134 4 14 the the DET fcis-32134 4 15 dynamic dynamic ADJ fcis-32134 4 16 learning learning NOUN fcis-32134 4 17 rate rate NOUN fcis-32134 4 18 adjustment adjustment NOUN fcis-32134 4 19 mechanism mechanism NOUN fcis-32134 4 20 and and CCONJ fcis-32134 4 21 the the DET fcis-32134 4 22 offline offline ADJ fcis-32134 4 23 strategy strategy NOUN fcis-32134 4 24 sac sac NOUN fcis-32134 4 25 ( ( PUNCT fcis-32134 4 26 soft soft ADJ fcis-32134 4 27 actor actor NOUN fcis-32134 4 28 - - PUNCT fcis-32134 4 29 critic critic NOUN fcis-32134 4 30 ) ) PUNCT fcis-32134 4 31 algorithm algorithm NOUN fcis-32134 4 32 based base VERB fcis-32134 4 33 on on ADP fcis-32134 4 34 the the DET fcis-32134 4 35 maximum maximum PROPN fcis-32134 4 36 entropy entropy PROPN fcis-32134 4 37 reinforcement reinforcement NOUN fcis-32134 4 38 learning learning NOUN fcis-32134 4 39 framework framework NOUN fcis-32134 4 40 , , PUNCT fcis-32134 4 41 and and CCONJ fcis-32134 4 42 proposes propose VERB fcis-32134 4 43 an an DET fcis-32134 4 44 autonomous autonomous ADJ fcis-32134 4 45 pathfinding pathfinding NOUN fcis-32134 4 46 and and CCONJ fcis-32134 4 47 obstacle obstacle NOUN fcis-32134 4 48 avoidance avoidance NOUN fcis-32134 4 49 algorithm algorithm NOUN fcis-32134 4 50 based base VERB fcis-32134 4 51 on on ADP fcis-32134 4 52 sac sac NOUN fcis-32134 4 53 , , PUNCT fcis-32134 4 54 aiming aim VERB fcis-32134 4 55 to to PART fcis-32134 4 56 enhance enhance VERB fcis-32134 4 57 the the DET fcis-32134 4 58 obstacle obstacle NOUN fcis-32134 4 59 avoidance avoidance NOUN fcis-32134 4 60 ability ability NOUN fcis-32134 4 61 of of ADP fcis-32134 4 62 uavs uavs NOUN fcis-32134 4 63 in in ADP fcis-32134 4 64 complex complex ADJ fcis-32134 4 65 environments.πœ€-greedy environments.πœ€-greedy PROPN fcis-32134 4 66 exploration exploration NOUN fcis-32134 4 67 strategy strategy NOUN fcis-32134 4 68 promotes promote VERB fcis-32134 4 69 exploration exploration NOUN fcis-32134 4 70 in in ADP fcis-32134 4 71 the the DET fcis-32134 4 72 early early ADJ fcis-32134 4 73 stage stage NOUN fcis-32134 4 74 of of ADP fcis-32134 4 75 training training NOUN fcis-32134 4 76 and and CCONJ fcis-32134 4 77 exploitation exploitation NOUN fcis-32134 4 78 in in ADP fcis-32134 4 79 the the DET fcis-32134 4 80 later later ADJ fcis-32134 4 81 stage stage NOUN fcis-32134 4 82 , , PUNCT fcis-32134 4 83 which which PRON fcis-32134 4 84 helps help VERB fcis-32134 4 85 the the DET fcis-32134 4 86 intelligent intelligent ADJ fcis-32134 4 87 body body NOUN fcis-32134 4 88 to to PART fcis-32134 4 89 find find VERB fcis-32134 4 90 the the DET fcis-32134 4 91 globally globally ADV fcis-32134 4 92 optimal optimal ADJ fcis-32134 4 93 strategy strategy NOUN fcis-32134 4 94 in in ADP fcis-32134 4 95 complex complex ADJ fcis-32134 4 96 environments environment NOUN fcis-32134 4 97 and and CCONJ fcis-32134 4 98 avoids avoid VERB fcis-32134 4 99 falling fall VERB fcis-32134 4 100 into into ADP fcis-32134 4 101 the the DET fcis-32134 4 102 local local ADJ fcis-32134 4 103 optimum optimum NOUN fcis-32134 4 104 in in ADP fcis-32134 4 105 the the DET fcis-32134 4 106 early early ADJ fcis-32134 4 107 stage stage NOUN fcis-32134 4 108 ; ; PUNCT fcis-32134 4 109 the the DET fcis-32134 4 110 dynamic dynamic ADJ fcis-32134 4 111 learning learning NOUN fcis-32134 4 112 rate rate NOUN fcis-32134 4 113 adjustment adjustment NOUN fcis-32134 4 114 mechanism mechanism NOUN fcis-32134 4 115 can can AUX fcis-32134 4 116 refine refine VERB fcis-32134 4 117 the the DET fcis-32134 4 118 strategy strategy NOUN fcis-32134 4 119 and and CCONJ fcis-32134 4 120 improve improve VERB fcis-32134 4 121 the the DET fcis-32134 4 122 training training NOUN fcis-32134 4 123 stability stability NOUN fcis-32134 4 124 and and CCONJ fcis-32134 4 125 final final ADJ fcis-32134 4 126 performance performance NOUN fcis-32134 4 127 . . PUNCT fcis-32134 5 1 the the DET fcis-32134 5 2 results result NOUN fcis-32134 5 3 in in ADP fcis-32134 5 4 the the DET fcis-32134 5 5 test test NOUN fcis-32134 5 6 environment environment NOUN fcis-32134 5 7 show show VERB fcis-32134 5 8 that that SCONJ fcis-32134 5 9 this this DET fcis-32134 5 10 algorithm algorithm NOUN fcis-32134 5 11 makes make VERB fcis-32134 5 12 the the DET fcis-32134 5 13 uav uav PROPN fcis-32134 5 14 collision collision PROPN fcis-32134 5 15 rate rate NOUN fcis-32134 5 16 decrease decrease NOUN fcis-32134 5 17 significantly significantly ADV fcis-32134 5 18 , , PUNCT fcis-32134 5 19 and and CCONJ fcis-32134 5 20 the the DET fcis-32134 5 21 timeout timeout NOUN fcis-32134 5 22 rate rate NOUN fcis-32134 5 23 is be AUX fcis-32134 5 24 close close ADJ fcis-32134 5 25 to to ADP fcis-32134 5 26 0 0 NUM fcis-32134 5 27 , , PUNCT fcis-32134 5 28 which which PRON fcis-32134 5 29 effectively effectively ADV fcis-32134 5 30 enhances enhance VERB fcis-32134 5 31 the the DET fcis-32134 5 32 algorithm algorithm NOUN fcis-32134 5 33 's 's PART fcis-32134 5 34 adaptability adaptability NOUN fcis-32134 5 35 and and CCONJ fcis-32134 5 36 stability stability NOUN fcis-32134 5 37 to to ADP fcis-32134 5 38 the the DET fcis-32134 5 39 environment environment NOUN fcis-32134 5 40 . . PUNCT fcis-32134 6 1 keywords keyword NOUN fcis-32134 6 2 : : PUNCT fcis-32134 6 3 unmanned unmanned ADJ fcis-32134 6 4 aircraft aircraft NOUN fcis-32134 6 5 ; ; PUNCT fcis-32134 6 6 obstacle obstacle NOUN fcis-32134 6 7 avoidance avoidance NOUN fcis-32134 6 8 ; ; PUNCT fcis-32134 6 9 sac sac NOUN fcis-32134 6 10 ; ; PUNCT fcis-32134 6 11 autonomous autonomous ADJ fcis-32134 6 12 navigation navigation NOUN fcis-32134 6 13 . . PUNCT fcis-32134 7 1 1 1 X fcis-32134 7 2 . . X fcis-32134 7 3 introduction introduction NOUN fcis-32134 7 4 in in ADP fcis-32134 7 5 recent recent ADJ fcis-32134 7 6 years year NOUN fcis-32134 7 7 , , PUNCT fcis-32134 7 8 the the DET fcis-32134 7 9 basic basic ADJ fcis-32134 7 10 research research NOUN fcis-32134 7 11 as as ADV fcis-32134 7 12 well well ADV fcis-32134 7 13 as as ADP fcis-32134 7 14 practical practical ADJ fcis-32134 7 15 application application NOUN fcis-32134 7 16 of of ADP fcis-32134 7 17 unmanned unmanned ADJ fcis-32134 7 18 aerial aerial ADJ fcis-32134 7 19 vehicles vehicle NOUN fcis-32134 7 20 ( ( PUNCT fcis-32134 7 21 uav uav PROPN fcis-32134 7 22 ) ) PUNCT fcis-32134 7 23 has have AUX fcis-32134 7 24 been be AUX fcis-32134 7 25 gradually gradually ADV fcis-32134 7 26 and and CCONJ fcis-32134 7 27 widely widely ADV fcis-32134 7 28 emphasized emphasize VERB fcis-32134 7 29 by by ADP fcis-32134 7 30 scholars scholar NOUN fcis-32134 7 31 at at ADP fcis-32134 7 32 home home NOUN fcis-32134 7 33 and and CCONJ fcis-32134 7 34 abroad[1 abroad[1 PRON fcis-32134 7 35 ] ] PUNCT fcis-32134 7 36 . . PUNCT fcis-32134 8 1 with with ADP fcis-32134 8 2 the the DET fcis-32134 8 3 in in ADP fcis-32134 8 4 - - PUNCT fcis-32134 8 5 depth depth NOUN fcis-32134 8 6 research research NOUN fcis-32134 8 7 in in ADP fcis-32134 8 8 the the DET fcis-32134 8 9 field field NOUN fcis-32134 8 10 of of ADP fcis-32134 8 11 optimal optimal ADJ fcis-32134 8 12 control control NOUN fcis-32134 8 13 theory theory NOUN fcis-32134 8 14 , , PUNCT fcis-32134 8 15 new new ADJ fcis-32134 8 16 sensor sensor NOUN fcis-32134 8 17 technology technology NOUN fcis-32134 8 18 and and CCONJ fcis-32134 8 19 artificial artificial ADJ fcis-32134 8 20 intelligence intelligence NOUN fcis-32134 8 21 , , PUNCT fcis-32134 8 22 the the DET fcis-32134 8 23 autonomous autonomous ADJ fcis-32134 8 24 flight flight NOUN fcis-32134 8 25 technology technology NOUN fcis-32134 8 26 of of ADP fcis-32134 8 27 uav uav PROPN fcis-32134 8 28 is be AUX fcis-32134 8 29 also also ADV fcis-32134 8 30 developing develop VERB fcis-32134 8 31 vigorously vigorously ADV fcis-32134 8 32 . . PUNCT fcis-32134 9 1 at at ADP fcis-32134 9 2 present present ADJ fcis-32134 9 3 , , PUNCT fcis-32134 9 4 uavs uavs PROPN fcis-32134 9 5 have have AUX fcis-32134 9 6 been be AUX fcis-32134 9 7 widely widely ADV fcis-32134 9 8 used use VERB fcis-32134 9 9 in in ADP fcis-32134 9 10 military military ADJ fcis-32134 9 11 , , PUNCT fcis-32134 9 12 civil civil ADJ fcis-32134 9 13 and and CCONJ fcis-32134 9 14 commercial commercial ADJ fcis-32134 9 15 fields field NOUN fcis-32134 9 16 , , PUNCT fcis-32134 9 17 such such ADJ fcis-32134 9 18 as as ADP fcis-32134 9 19 military military ADJ fcis-32134 9 20 reconnaissance reconnaissance NOUN fcis-32134 9 21 , , PUNCT fcis-32134 9 22 disaster disaster NOUN fcis-32134 9 23 monitoring monitoring NOUN fcis-32134 9 24 , , PUNCT fcis-32134 9 25 express express VERB fcis-32134 9 26 delivery delivery NOUN fcis-32134 9 27 , , PUNCT fcis-32134 9 28 aerial aerial ADJ fcis-32134 9 29 photography photography NOUN fcis-32134 9 30 , , PUNCT fcis-32134 9 31 etc.[2 etc.[2 X fcis-32134 9 32 ] ] PUNCT fcis-32134 9 33 . . PUNCT fcis-32134 10 1 therefore therefore ADV fcis-32134 10 2 , , PUNCT fcis-32134 10 3 how how SCONJ fcis-32134 10 4 uavs uavs NOUN fcis-32134 10 5 can can AUX fcis-32134 10 6 efficiently efficiently ADV fcis-32134 10 7 and and CCONJ fcis-32134 10 8 accurately accurately ADV fcis-32134 10 9 avoid avoid VERB fcis-32134 10 10 obstacles obstacle NOUN fcis-32134 10 11 and and CCONJ fcis-32134 10 12 reach reach VERB fcis-32134 10 13 the the DET fcis-32134 10 14 end end NOUN fcis-32134 10 15 point point NOUN fcis-32134 10 16 in in ADP fcis-32134 10 17 these these DET fcis-32134 10 18 complex complex ADJ fcis-32134 10 19 scenarios scenario NOUN fcis-32134 10 20 is be AUX fcis-32134 10 21 still still ADV fcis-32134 10 22 an an DET fcis-32134 10 23 important important ADJ fcis-32134 10 24 issue issue NOUN fcis-32134 10 25 . . PUNCT fcis-32134 11 1 for for ADP fcis-32134 11 2 uav uav PROPN fcis-32134 11 3 autonomous autonomous PROPN fcis-32134 11 4 obstacle obstacle PROPN fcis-32134 11 5 avoidance avoidance NOUN fcis-32134 11 6 technology technology NOUN fcis-32134 11 7 , , PUNCT fcis-32134 11 8 researchers researcher NOUN fcis-32134 11 9 have have AUX fcis-32134 11 10 proposed propose VERB fcis-32134 11 11 a a DET fcis-32134 11 12 variety variety NOUN fcis-32134 11 13 of of ADP fcis-32134 11 14 algorithms algorithm NOUN fcis-32134 11 15 , , PUNCT fcis-32134 11 16 which which PRON fcis-32134 11 17 can can AUX fcis-32134 11 18 be be AUX fcis-32134 11 19 roughly roughly ADV fcis-32134 11 20 categorized categorize VERB fcis-32134 11 21 into into ADP fcis-32134 11 22 traditional traditional ADJ fcis-32134 11 23 path path NOUN fcis-32134 11 24 planning planning NOUN fcis-32134 11 25 algorithms algorithm NOUN fcis-32134 11 26 and and CCONJ fcis-32134 11 27 reinforcement reinforcement NOUN fcis-32134 11 28 learning learning NOUN fcis-32134 11 29 algorithms algorithm NOUN fcis-32134 11 30 . . PUNCT fcis-32134 12 1 traditional traditional ADJ fcis-32134 12 2 path path NOUN fcis-32134 12 3 planning planning NOUN fcis-32134 12 4 methods method NOUN fcis-32134 12 5 are be AUX fcis-32134 12 6 divided divide VERB fcis-32134 12 7 into into ADP fcis-32134 12 8 graph graph NOUN fcis-32134 12 9 search search NOUN fcis-32134 12 10 - - PUNCT fcis-32134 12 11 based base VERB fcis-32134 12 12 path path NOUN fcis-32134 12 13 planning planning NOUN fcis-32134 12 14 algorithms algorithm NOUN fcis-32134 12 15 , , PUNCT fcis-32134 12 16 sampling sample VERB fcis-32134 12 17 - - PUNCT fcis-32134 12 18 based base VERB fcis-32134 12 19 path path NOUN fcis-32134 12 20 planning planning NOUN fcis-32134 12 21 algorithms algorithm NOUN fcis-32134 12 22 and and CCONJ fcis-32134 12 23 artificial artificial ADJ fcis-32134 12 24 potential potential ADJ fcis-32134 12 25 field field NOUN fcis-32134 12 26 methods method NOUN fcis-32134 12 27 . . PUNCT fcis-32134 13 1 although although SCONJ fcis-32134 13 2 graph graph NOUN fcis-32134 13 3 search search NOUN fcis-32134 13 4 - - PUNCT fcis-32134 13 5 based base VERB fcis-32134 13 6 path path NOUN fcis-32134 13 7 planning planning NOUN fcis-32134 13 8 algorithms algorithm NOUN fcis-32134 13 9 ( ( PUNCT fcis-32134 13 10 e.g. e.g. ADV fcis-32134 13 11 , , PUNCT fcis-32134 13 12 a a DET fcis-32134 13 13 * * X fcis-32134 13 14 algorithm algorithm NOUN fcis-32134 13 15 , , PUNCT fcis-32134 13 16 dijkstra dijkstra PROPN fcis-32134 13 17 's 's PART fcis-32134 13 18 algorithm algorithm NOUN fcis-32134 13 19 , , PUNCT fcis-32134 13 20 etc etc X fcis-32134 13 21 . . X fcis-32134 13 22 ) ) PUNCT fcis-32134 13 23 perform perform VERB fcis-32134 13 24 well well ADV fcis-32134 13 25 in in ADP fcis-32134 13 26 static static ADJ fcis-32134 13 27 environments environment NOUN fcis-32134 13 28 , , PUNCT fcis-32134 13 29 they they PRON fcis-32134 13 30 tend tend VERB fcis-32134 13 31 to to PART fcis-32134 13 32 lack lack VERB fcis-32134 13 33 flexibility flexibility NOUN fcis-32134 13 34 and and CCONJ fcis-32134 13 35 adaptability adaptability NOUN fcis-32134 13 36 in in ADP fcis-32134 13 37 complex complex ADJ fcis-32134 13 38 and and CCONJ fcis-32134 13 39 uncertain uncertain ADJ fcis-32134 13 40 environments environment NOUN fcis-32134 13 41 , , PUNCT fcis-32134 13 42 and and CCONJ fcis-32134 13 43 are be AUX fcis-32134 13 44 highly highly ADV fcis-32134 13 45 dependent dependent ADJ fcis-32134 13 46 on on ADP fcis-32134 13 47 environmental environmental ADJ fcis-32134 13 48 information information NOUN fcis-32134 13 49 [ [ X fcis-32134 13 50 3 3 NUM fcis-32134 13 51 , , PUNCT fcis-32134 13 52 4 4 NUM fcis-32134 13 53 ] ] X fcis-32134 13 54 samplingbased samplingbased ADJ fcis-32134 13 55 methods method NOUN fcis-32134 13 56 ( ( PUNCT fcis-32134 13 57 e.g. e.g. ADV fcis-32134 13 58 , , PUNCT fcis-32134 13 59 rrt rrt NOUN fcis-32134 13 60 ) ) PUNCT fcis-32134 13 61 are be AUX fcis-32134 13 62 effective effective ADJ fcis-32134 13 63 in in ADP fcis-32134 13 64 realizing realize VERB fcis-32134 13 65 obstacle obstacle NOUN fcis-32134 13 66 avoidance avoidance NOUN fcis-32134 13 67 navigation navigation NOUN fcis-32134 13 68 through through ADP fcis-32134 13 69 the the DET fcis-32134 13 70 asymptotic asymptotic ADJ fcis-32134 13 71 optimality optimality NOUN fcis-32134 13 72 property property NOUN fcis-32134 13 73 , , PUNCT fcis-32134 13 74 but but CCONJ fcis-32134 13 75 such such ADJ fcis-32134 13 76 algorithms algorithm NOUN fcis-32134 13 77 rely rely VERB fcis-32134 13 78 on on ADP fcis-32134 13 79 random random ADJ fcis-32134 13 80 sampling sampling NOUN fcis-32134 13 81 resulting result VERB fcis-32134 13 82 in in ADP fcis-32134 13 83 low low ADJ fcis-32134 13 84 efficiency efficiency NOUN fcis-32134 14 1 [ [ X fcis-32134 14 2 5 5 NUM fcis-32134 14 3 ] ] PUNCT fcis-32134 14 4 . . PUNCT fcis-32134 15 1 artificial artificial ADJ fcis-32134 15 2 potential potential ADJ fcis-32134 15 3 field field NOUN fcis-32134 15 4 method method NOUN fcis-32134 15 5 by by ADP fcis-32134 15 6 constructing construct VERB fcis-32134 15 7 an an DET fcis-32134 15 8 artificial artificial ADJ fcis-32134 15 9 potential potential ADJ fcis-32134 15 10 field field NOUN fcis-32134 15 11 the the DET fcis-32134 15 12 target target NOUN fcis-32134 15 13 and and CCONJ fcis-32134 15 14 obstacles obstacle NOUN fcis-32134 15 15 are be AUX fcis-32134 15 16 regarded regard VERB fcis-32134 15 17 as as ADP fcis-32134 15 18 objects object NOUN fcis-32134 15 19 with with ADP fcis-32134 15 20 gravitational gravitational ADJ fcis-32134 15 21 and and CCONJ fcis-32134 15 22 repulsive repulsive ADJ fcis-32134 15 23 forces force NOUN fcis-32134 15 24 on on ADP fcis-32134 15 25 the the DET fcis-32134 15 26 uav uav PROPN fcis-32134 15 27 respectively respectively ADV fcis-32134 15 28 , , PUNCT fcis-32134 15 29 and and CCONJ fcis-32134 15 30 the the DET fcis-32134 15 31 uav uav PROPN fcis-32134 15 32 moves move VERB fcis-32134 15 33 along along ADP fcis-32134 15 34 the the DET fcis-32134 15 35 combined combined ADJ fcis-32134 15 36 force force NOUN fcis-32134 15 37 of of ADP fcis-32134 15 38 gravitational gravitational ADJ fcis-32134 15 39 and and CCONJ fcis-32134 15 40 repulsive repulsive ADJ fcis-32134 15 41 forces force NOUN fcis-32134 15 42 , , PUNCT fcis-32134 15 43 this this DET fcis-32134 15 44 algorithm algorithm NOUN fcis-32134 15 45 is be AUX fcis-32134 15 46 a a DET fcis-32134 15 47 feedback feedback NOUN fcis-32134 15 48 control control NOUN fcis-32134 15 49 strategy strategy NOUN fcis-32134 15 50 , , PUNCT fcis-32134 15 51 which which PRON fcis-32134 15 52 is be AUX fcis-32134 15 53 robust robust ADJ fcis-32134 15 54 to to PART fcis-32134 15 55 control control VERB fcis-32134 15 56 and and CCONJ fcis-32134 15 57 sensing sense VERB fcis-32134 15 58 errors error NOUN fcis-32134 15 59 , , PUNCT fcis-32134 15 60 but but CCONJ fcis-32134 15 61 it it PRON fcis-32134 15 62 does do AUX fcis-32134 15 63 not not PART fcis-32134 15 64 necessarily necessarily ADV fcis-32134 15 65 find find VERB fcis-32134 15 66 the the DET fcis-32134 15 67 optimal optimal ADJ fcis-32134 15 68 solution solution NOUN fcis-32134 15 69 [ [ X fcis-32134 15 70 6 6 NUM fcis-32134 15 71 ] ] PUNCT fcis-32134 15 72 . . PUNCT fcis-32134 16 1 reinforcement reinforcement NOUN fcis-32134 16 2 learning learning NOUN fcis-32134 16 3 ( ( PUNCT fcis-32134 16 4 rl rl NOUN fcis-32134 16 5 ) ) PUNCT fcis-32134 16 6 , , PUNCT fcis-32134 16 7 as as ADP fcis-32134 16 8 a a DET fcis-32134 16 9 method method NOUN fcis-32134 16 10 in in ADP fcis-32134 16 11 which which PRON fcis-32134 16 12 an an DET fcis-32134 16 13 agent agent NOUN fcis-32134 16 14 learns learn VERB fcis-32134 16 15 optimal optimal ADJ fcis-32134 16 16 strategies strategy NOUN fcis-32134 16 17 through through ADP fcis-32134 16 18 interaction interaction NOUN fcis-32134 16 19 with with ADP fcis-32134 16 20 the the DET fcis-32134 16 21 environment environment NOUN fcis-32134 16 22 , , PUNCT fcis-32134 16 23 is be AUX fcis-32134 16 24 particularly particularly ADV fcis-32134 16 25 suitable suitable ADJ fcis-32134 16 26 for for ADP fcis-32134 16 27 tasks task NOUN fcis-32134 16 28 where where SCONJ fcis-32134 16 29 the the DET fcis-32134 16 30 environment environment NOUN fcis-32134 16 31 is be AUX fcis-32134 16 32 dynamically dynamically ADV fcis-32134 16 33 changing change VERB fcis-32134 16 34 , , PUNCT fcis-32134 16 35 the the DET fcis-32134 16 36 model model NOUN fcis-32134 16 37 is be AUX fcis-32134 16 38 unknown unknown ADJ fcis-32134 16 39 , , PUNCT fcis-32134 16 40 or or CCONJ fcis-32134 16 41 optimal optimal ADJ fcis-32134 16 42 solutions solution NOUN fcis-32134 16 43 can can AUX fcis-32134 16 44 not not PART fcis-32134 16 45 be be AUX fcis-32134 16 46 directly directly ADV fcis-32134 16 47 obtained obtain VERB fcis-32134 16 48 . . PUNCT fcis-32134 17 1 therefore therefore ADV fcis-32134 17 2 , , PUNCT fcis-32134 17 3 it it PRON fcis-32134 17 4 has have AUX fcis-32134 17 5 gradually gradually ADV fcis-32134 17 6 been be AUX fcis-32134 17 7 applied apply VERB fcis-32134 17 8 in in ADP fcis-32134 17 9 the the DET fcis-32134 17 10 field field NOUN fcis-32134 17 11 of of ADP fcis-32134 17 12 drone drone NOUN fcis-32134 17 13 path path NOUN fcis-32134 17 14 planning planning NOUN fcis-32134 17 15 . . PUNCT fcis-32134 18 1 amala amala PROPN fcis-32134 18 2 sonny sonny PROPN fcis-32134 18 3 et et PROPN fcis-32134 18 4 al al PROPN fcis-32134 18 5 . . PROPN fcis-32134 18 6 proposed propose VERB fcis-32134 18 7 a a DET fcis-32134 18 8 q q ADV fcis-32134 18 9 - - PUNCT fcis-32134 18 10 learning learn VERB fcis-32134 18 11 method method NOUN fcis-32134 18 12 based base VERB fcis-32134 18 13 on on ADP fcis-32134 18 14 shortest short ADJ fcis-32134 18 15 distance distance NOUN fcis-32134 18 16 priority priority NOUN fcis-32134 18 17 strategy strategy NOUN fcis-32134 18 18 to to PART fcis-32134 18 19 improve improve VERB fcis-32134 18 20 the the DET fcis-32134 18 21 accuracy accuracy NOUN fcis-32134 18 22 and and CCONJ fcis-32134 18 23 efficiency efficiency NOUN fcis-32134 18 24 of of ADP fcis-32134 18 25 drone drone NOUN fcis-32134 18 26 path path NOUN fcis-32134 18 27 planning planning NOUN fcis-32134 18 28 [ [ X fcis-32134 18 29 7 7 NUM fcis-32134 18 30 ] ] PUNCT fcis-32134 18 31 . . PUNCT fcis-32134 19 1 qi qi PROPN fcis-32134 19 2 et et PROPN fcis-32134 19 3 al al PROPN fcis-32134 19 4 . . PROPN fcis-32134 19 5 proposed propose VERB fcis-32134 19 6 an an DET fcis-32134 19 7 fd fd PROPN fcis-32134 19 8 - - PUNCT fcis-32134 19 9 ppo ppo PROPN fcis-32134 19 10 algorithm algorithm NOUN fcis-32134 19 11 based base VERB fcis-32134 19 12 on on ADP fcis-32134 19 13 frequency frequency NOUN fcis-32134 19 14 decomposition decomposition NOUN fcis-32134 19 15 , , PUNCT fcis-32134 19 16 which which PRON fcis-32134 19 17 efficiently efficiently ADV fcis-32134 19 18 guides guide VERB fcis-32134 19 19 drones drone NOUN fcis-32134 19 20 to to PART fcis-32134 19 21 complete complete VERB fcis-32134 19 22 path path NOUN fcis-32134 19 23 planning planning NOUN fcis-32134 19 24 tasks task NOUN fcis-32134 19 25 by by ADP fcis-32134 19 26 decomposing decompose VERB fcis-32134 19 27 rewards reward NOUN fcis-32134 19 28 into into ADP fcis-32134 19 29 multi multi ADJ fcis-32134 19 30 - - ADJ fcis-32134 19 31 dimensional dimensional ADJ fcis-32134 19 32 frequency frequency NOUN fcis-32134 19 33 rewards reward NOUN fcis-32134 19 34 and and CCONJ fcis-32134 19 35 then then ADV fcis-32134 19 36 calculating calculate VERB fcis-32134 19 37 frequency frequency NOUN fcis-32134 19 38 returns return NOUN fcis-32134 19 39 [ [ X fcis-32134 19 40 8 8 NUM fcis-32134 19 41 ] ] PUNCT fcis-32134 19 42 . . PUNCT fcis-32134 20 1 omar omar PROPN fcis-32134 20 2 bouhamed bouhame VERB fcis-32134 20 3 et et PROPN fcis-32134 20 4 al al PROPN fcis-32134 20 5 . . PROPN fcis-32134 21 1 combined combine VERB fcis-32134 21 2 the the DET fcis-32134 21 3 continuous continuous ADJ fcis-32134 21 4 deep deep ADJ fcis-32134 21 5 deterministic deterministic ADJ fcis-32134 21 6 policy policy NOUN fcis-32134 21 7 gradient gradient NOUN fcis-32134 21 8 ( ( PUNCT fcis-32134 21 9 ddpg ddpg ADJ fcis-32134 21 10 ) ) PUNCT fcis-32134 21 11 algorithm algorithm NOUN fcis-32134 21 12 with with ADP fcis-32134 21 13 drone drone NOUN fcis-32134 21 14 3d 3d NUM fcis-32134 21 15 obstacle obstacle NOUN fcis-32134 21 16 avoidance avoidance NOUN fcis-32134 21 17 to to PART fcis-32134 21 18 solve solve VERB fcis-32134 21 19 the the DET fcis-32134 21 20 problem problem NOUN fcis-32134 21 21 of of ADP fcis-32134 21 22 exploration exploration NOUN fcis-32134 21 23 efficiency efficiency NOUN fcis-32134 21 24 in in ADP fcis-32134 21 25 continuous continuous ADJ fcis-32134 21 26 control control NOUN fcis-32134 22 1 [ [ X fcis-32134 22 2 9 9 NUM fcis-32134 22 3 ] ] PUNCT fcis-32134 22 4 . . PUNCT fcis-32134 23 1 luo luo PROPN fcis-32134 23 2 et et PROPN fcis-32134 23 3 al al PROPN fcis-32134 23 4 . . PROPN fcis-32134 23 5 introduced introduce VERB fcis-32134 23 6 the the DET fcis-32134 23 7 priority priority NOUN fcis-32134 23 8 experience experience NOUN fcis-32134 23 9 playback playback NOUN fcis-32134 23 10 mechanism mechanism NOUN fcis-32134 23 11 into into ADP fcis-32134 23 12 the the DET fcis-32134 23 13 td3 td3 NOUN fcis-32134 23 14 algorithm algorithm NOUN fcis-32134 23 15 to to PART fcis-32134 23 16 transform transform VERB fcis-32134 23 17 its its PRON fcis-32134 23 18 experience experience NOUN fcis-32134 23 19 pool pool NOUN fcis-32134 23 20 , , PUNCT fcis-32134 23 21 so so SCONJ fcis-32134 23 22 that that SCONJ fcis-32134 23 23 the the DET fcis-32134 23 24 intelligent intelligent ADJ fcis-32134 23 25 physical physical ADJ fcis-32134 23 26 ability ability NOUN fcis-32134 23 27 can can AUX fcis-32134 23 28 distinguish distinguish VERB fcis-32134 23 29 the the DET fcis-32134 23 30 priority priority NOUN fcis-32134 23 31 of of ADP fcis-32134 23 32 different different ADJ fcis-32134 23 33 experience experience NOUN fcis-32134 23 34 samples sample NOUN fcis-32134 23 35 , , PUNCT fcis-32134 23 36 thus thus ADV fcis-32134 23 37 improving improve VERB fcis-32134 23 38 the the DET fcis-32134 23 39 sampling sample VERB fcis-32134 23 40 efficiency efficiency NOUN fcis-32134 23 41 and and CCONJ fcis-32134 23 42 shortening shorten VERB fcis-32134 23 43 the the DET fcis-32134 23 44 training training NOUN fcis-32134 23 45 time time NOUN fcis-32134 23 46 [ [ X fcis-32134 23 47 10 10 NUM fcis-32134 23 48 ] ] PUNCT fcis-32134 23 49 . . PUNCT fcis-32134 24 1 xue xue PROPN fcis-32134 24 2 et et PROPN fcis-32134 24 3 al al PROPN fcis-32134 24 4 . . PROPN fcis-32134 24 5 proposed propose VERB fcis-32134 24 6 a a DET fcis-32134 24 7 framework framework NOUN fcis-32134 24 8 of of ADP fcis-32134 24 9 deep deep ADJ fcis-32134 24 10 rl rl PROPN fcis-32134 24 11 and and CCONJ fcis-32134 24 12 developed develop VERB fcis-32134 24 13 a a DET fcis-32134 24 14 value value NOUN fcis-32134 24 15 network network NOUN fcis-32134 24 16 that that PRON fcis-32134 24 17 integrates integrate VERB fcis-32134 24 18 crowd crowd NOUN fcis-32134 24 19 and and CCONJ fcis-32134 24 20 static static ADJ fcis-32134 24 21 obstacle obstacle NOUN fcis-32134 24 22 information information NOUN fcis-32134 24 23 , , PUNCT fcis-32134 24 24 utilizing utilize VERB fcis-32134 24 25 spatiotemporal spatiotemporal ADJ fcis-32134 24 26 reasoning reasoning NOUN fcis-32134 24 27 and and CCONJ fcis-32134 24 28 lidar lidar NOUN fcis-32134 24 29 data datum NOUN fcis-32134 24 30 to to PART fcis-32134 24 31 help help VERB fcis-32134 24 32 the the DET fcis-32134 24 33 agent agent NOUN fcis-32134 24 34 effectively effectively ADV fcis-32134 24 35 understand understand VERB fcis-32134 24 36 the the DET fcis-32134 24 37 surrounding surround VERB fcis-32134 24 38 environment environment NOUN fcis-32134 24 39 [ [ X fcis-32134 24 40 11 11 NUM fcis-32134 24 41 ] ] PUNCT fcis-32134 24 42 . . PUNCT fcis-32134 25 1 in in ADP fcis-32134 25 2 summary summary NOUN fcis-32134 25 3 , , PUNCT fcis-32134 25 4 this this DET fcis-32134 25 5 paper paper NOUN fcis-32134 25 6 takes take VERB fcis-32134 25 7 the the DET fcis-32134 25 8 autonomous autonomous ADJ fcis-32134 25 9 uav uav PROPN fcis-32134 25 10 pathfinding pathfinding NOUN fcis-32134 25 11 and and CCONJ fcis-32134 25 12 obstacle obstacle NOUN fcis-32134 25 13 avoidance avoidance NOUN fcis-32134 25 14 task task NOUN fcis-32134 25 15 in in ADP fcis-32134 25 16 a a DET fcis-32134 25 17 threedimensional threedimensional ADJ fcis-32134 25 18 unknown unknown ADJ fcis-32134 25 19 environment environment NOUN fcis-32134 25 20 as as ADP fcis-32134 25 21 the the DET fcis-32134 25 22 research research NOUN fcis-32134 25 23 object object NOUN fcis-32134 25 24 , , PUNCT fcis-32134 25 25 and and CCONJ fcis-32134 25 26 through through ADP fcis-32134 25 27 the the DET fcis-32134 25 28 decision decision NOUN fcis-32134 25 29 - - PUNCT fcis-32134 25 30 making make VERB fcis-32134 25 31 ability ability NOUN fcis-32134 25 32 of of ADP fcis-32134 25 33 rl rl PROPN fcis-32134 25 34 , , PUNCT fcis-32134 25 35 the the DET fcis-32134 25 36 uav uav PROPN fcis-32134 25 37 can can AUX fcis-32134 25 38 autonomously autonomously ADV fcis-32134 25 39 avoid avoid VERB fcis-32134 25 40 all all DET fcis-32134 25 41 kinds kind NOUN fcis-32134 25 42 of of ADP fcis-32134 25 43 static static ADJ fcis-32134 25 44 and and CCONJ fcis-32134 25 45 dynamic dynamic ADJ fcis-32134 25 46 obstacles obstacle NOUN fcis-32134 25 47 and and CCONJ fcis-32134 25 48 reach reach VERB fcis-32134 25 49 the the DET fcis-32134 25 50 end end NOUN fcis-32134 25 51 point point NOUN fcis-32134 25 52 . . PUNCT fcis-32134 26 1 specifically specifically ADV fcis-32134 26 2 , , PUNCT fcis-32134 26 3 this this DET fcis-32134 26 4 paper paper NOUN fcis-32134 26 5 first first ADV fcis-32134 26 6 establishes establish VERB fcis-32134 26 7 the the DET fcis-32134 26 8 markov markov NOUN fcis-32134 26 9 decision decision NOUN fcis-32134 26 10 - - PUNCT fcis-32134 26 11 making make VERB fcis-32134 26 12 process process NOUN fcis-32134 26 13 of of ADP fcis-32134 26 14 autonomous autonomous PROPN fcis-32134 26 15 uav uav PROPN fcis-32134 26 16 navigation navigation PROPN fcis-32134 26 17 task task PROPN fcis-32134 26 18 , , PUNCT fcis-32134 26 19 designs design VERB fcis-32134 26 20 the the DET fcis-32134 26 21 non non ADJ fcis-32134 26 22 - - ADJ fcis-32134 26 23 sparse sparse ADJ fcis-32134 26 24 reward reward NOUN fcis-32134 26 25 function function NOUN fcis-32134 26 26 , , PUNCT fcis-32134 26 27 and and CCONJ fcis-32134 26 28 realizes realize VERB fcis-32134 26 29 the the DET fcis-32134 26 30 autonomous autonomous ADJ fcis-32134 26 31 pathfinding pathfinding NOUN fcis-32134 26 32 and and CCONJ fcis-32134 26 33 obstacle obstacle NOUN fcis-32134 26 34 avoidance avoidance NOUN fcis-32134 26 35 of of ADP fcis-32134 26 36 uav uav PROPN fcis-32134 26 37 based base VERB fcis-32134 26 38 on on ADP fcis-32134 26 39 sac sac NOUN fcis-32134 26 40 algorithm algorithm NOUN fcis-32134 26 41 , , PUNCT fcis-32134 26 42 based base VERB fcis-32134 26 43 on on ADP fcis-32134 26 44 which which PRON fcis-32134 26 45 , , PUNCT fcis-32134 26 46 this this DET fcis-32134 26 47 paper paper NOUN fcis-32134 26 48 introduces introduce VERB fcis-32134 26 49 the the DET fcis-32134 26 50 dynamically dynamically ADV fcis-32134 26 51 decayingΞ΅ decayingΞ΅ VERB fcis-32134 26 52 43 43 NUM fcis-32134 26 53 greedy greedy ADJ fcis-32134 26 54 exploring explore VERB fcis-32134 26 55 strategy strategy NOUN fcis-32134 26 56 , , PUNCT fcis-32134 26 57 and and CCONJ fcis-32134 26 58 adds add VERB fcis-32134 26 59 the the DET fcis-32134 26 60 dynamic dynamic ADJ fcis-32134 26 61 learning learning NOUN fcis-32134 26 62 rate rate NOUN fcis-32134 26 63 adjustment adjustment NOUN fcis-32134 26 64 mechanism mechanism NOUN fcis-32134 26 65 , , PUNCT fcis-32134 26 66 so so SCONJ fcis-32134 26 67 that that SCONJ fcis-32134 26 68 the the DET fcis-32134 26 69 uav uav PROPN fcis-32134 26 70 can can AUX fcis-32134 26 71 explore explore VERB fcis-32134 26 72 a a DET fcis-32134 26 73 wider wide ADJ fcis-32134 26 74 action action NOUN fcis-32134 26 75 space space NOUN fcis-32134 26 76 and and CCONJ fcis-32134 26 77 improve improve VERB fcis-32134 26 78 the the DET fcis-32134 26 79 stability stability NOUN fcis-32134 26 80 of of ADP fcis-32134 26 81 the the DET fcis-32134 26 82 training training NOUN fcis-32134 26 83 , , PUNCT fcis-32134 26 84 and and CCONJ fcis-32134 26 85 finally finally ADV fcis-32134 26 86 , , PUNCT fcis-32134 26 87 compare compare VERB fcis-32134 26 88 and and CCONJ fcis-32134 26 89 validate validate VERB fcis-32134 26 90 its its PRON fcis-32134 26 91 performance performance NOUN fcis-32134 26 92 with with ADP fcis-32134 26 93 other other ADJ fcis-32134 26 94 algorithms algorithm NOUN fcis-32134 26 95 in in ADP fcis-32134 26 96 the the DET fcis-32134 26 97 testing testing NOUN fcis-32134 26 98 environment environment NOUN fcis-32134 26 99 . . PUNCT fcis-32134 27 1 2 2 X fcis-32134 27 2 . . X fcis-32134 27 3 automatic automatic ADJ fcis-32134 27 4 obstacle obstacle NOUN fcis-32134 27 5 avoidance avoidance NOUN fcis-32134 27 6 algorithm algorithm NOUN fcis-32134 27 7 based base VERB fcis-32134 27 8 on on ADP fcis-32134 27 9 reinforcement reinforcement NOUN fcis-32134 27 10 learning learn VERB fcis-32134 27 11 2.1 2.1 NUM fcis-32134 27 12 . . PUNCT fcis-32134 28 1 markov markov NOUN fcis-32134 28 2 decision decision NOUN fcis-32134 28 3 process process NOUN fcis-32134 28 4 reinforcement reinforcement NOUN fcis-32134 28 5 learning learning NOUN fcis-32134 28 6 ( ( PUNCT fcis-32134 28 7 rl rl NOUN fcis-32134 28 8 ) ) PUNCT fcis-32134 28 9 is be AUX fcis-32134 28 10 a a DET fcis-32134 28 11 method method NOUN fcis-32134 28 12 that that PRON fcis-32134 28 13 emphasizes emphasize VERB fcis-32134 28 14 the the DET fcis-32134 28 15 ability ability NOUN fcis-32134 28 16 of of ADP fcis-32134 28 17 an an DET fcis-32134 28 18 intelligent intelligent ADJ fcis-32134 28 19 body body NOUN fcis-32134 28 20 to to PART fcis-32134 28 21 maximize maximize VERB fcis-32134 28 22 its its PRON fcis-32134 28 23 expected expect VERB fcis-32134 28 24 benefits benefit NOUN fcis-32134 28 25 by by ADP fcis-32134 28 26 acting act VERB fcis-32134 28 27 based base VERB fcis-32134 28 28 on on ADP fcis-32134 28 29 its its PRON fcis-32134 28 30 environment environment NOUN fcis-32134 28 31 . . PUNCT fcis-32134 29 1 the the DET fcis-32134 29 2 agent agent NOUN fcis-32134 29 3 interacts interact VERB fcis-32134 29 4 with with ADP fcis-32134 29 5 the the DET fcis-32134 29 6 environment environment NOUN fcis-32134 29 7 through through ADP fcis-32134 29 8 policy policy NOUN fcis-32134 29 9 - - PUNCT fcis-32134 29 10 driven drive VERB fcis-32134 29 11 actions action NOUN fcis-32134 29 12 , , PUNCT fcis-32134 29 13 generates generate VERB fcis-32134 29 14 new new ADJ fcis-32134 29 15 data datum NOUN fcis-32134 29 16 , , PUNCT fcis-32134 29 17 and and CCONJ fcis-32134 29 18 then then ADV fcis-32134 29 19 uses use VERB fcis-32134 29 20 the the DET fcis-32134 29 21 new new ADJ fcis-32134 29 22 data datum NOUN fcis-32134 29 23 to to PART fcis-32134 29 24 modify modify VERB fcis-32134 29 25 its its PRON fcis-32134 29 26 own own ADJ fcis-32134 29 27 action action NOUN fcis-32134 29 28 strategies strategy NOUN fcis-32134 29 29 , , PUNCT fcis-32134 29 30 and and CCONJ fcis-32134 29 31 so so ADV fcis-32134 29 32 on on ADP fcis-32134 29 33 iteratively iteratively ADV fcis-32134 29 34 optimizes optimize VERB fcis-32134 29 35 its its PRON fcis-32134 29 36 strategies strategy NOUN fcis-32134 29 37 until until SCONJ fcis-32134 29 38 it it PRON fcis-32134 29 39 learns learn VERB fcis-32134 29 40 the the DET fcis-32134 29 41 action action NOUN fcis-32134 29 42 strategies strategy NOUN fcis-32134 29 43 needed need VERB fcis-32134 29 44 to to PART fcis-32134 29 45 complete complete VERB fcis-32134 29 46 the the DET fcis-32134 29 47 task task NOUN fcis-32134 29 48 . . PUNCT fcis-32134 30 1 since since SCONJ fcis-32134 30 2 the the DET fcis-32134 30 3 current current ADJ fcis-32134 30 4 state state NOUN fcis-32134 30 5 generated generate VERB fcis-32134 30 6 by by ADP fcis-32134 30 7 the the DET fcis-32134 30 8 intelligent intelligent ADJ fcis-32134 30 9 body body NOUN fcis-32134 30 10 interacting interact VERB fcis-32134 30 11 with with ADP fcis-32134 30 12 the the DET fcis-32134 30 13 environment environment NOUN fcis-32134 30 14 contains contain VERB fcis-32134 30 15 useful useful ADJ fcis-32134 30 16 information information NOUN fcis-32134 30 17 needed need VERB fcis-32134 30 18 for for ADP fcis-32134 30 19 future future ADJ fcis-32134 30 20 prediction prediction NOUN fcis-32134 30 21 , , PUNCT fcis-32134 30 22 and and CCONJ fcis-32134 30 23 the the DET fcis-32134 30 24 state state NOUN fcis-32134 30 25 generated generate VERB fcis-32134 30 26 in in ADP fcis-32134 30 27 the the DET fcis-32134 30 28 past past NOUN fcis-32134 30 29 is be AUX fcis-32134 30 30 irrelevant irrelevant ADJ fcis-32134 30 31 for for ADP fcis-32134 30 32 prediction prediction NOUN fcis-32134 30 33 , , PUNCT fcis-32134 30 34 which which PRON fcis-32134 30 35 satisfies satisfy VERB fcis-32134 30 36 markov markov NOUN fcis-32134 30 37 property property NOUN fcis-32134 30 38 , , PUNCT fcis-32134 30 39 reinforcement reinforcement NOUN fcis-32134 30 40 learning learning NOUN fcis-32134 30 41 can can AUX fcis-32134 30 42 be be AUX fcis-32134 30 43 described describe VERB fcis-32134 30 44 as as ADP fcis-32134 30 45 markov markov NOUN fcis-32134 30 46 decision decision NOUN fcis-32134 30 47 process process NOUN fcis-32134 30 48 ( ( PUNCT fcis-32134 30 49 mdp mdp NOUN fcis-32134 30 50 ) ) PUNCT fcis-32134 30 51 . . PUNCT fcis-32134 31 1 markov markov NOUN fcis-32134 31 2 decision decision NOUN fcis-32134 31 3 process process NOUN fcis-32134 31 4 is be AUX fcis-32134 31 5 denoted denote VERB fcis-32134 31 6 as as ADP fcis-32134 31 7 < < X fcis-32134 31 8 s s PROPN fcis-32134 31 9 , , PUNCT fcis-32134 31 10 a a DET fcis-32134 31 11 , , PUNCT fcis-32134 31 12 p p X fcis-32134 31 13 , , PUNCT fcis-32134 31 14 r r NOUN fcis-32134 31 15 , , PUNCT fcis-32134 31 16 Ξ³ Ξ³ X fcis-32134 31 17 > > X fcis-32134 31 18 , , PUNCT fcis-32134 31 19 where where SCONJ fcis-32134 31 20 s s NOUN fcis-32134 31 21 is be AUX fcis-32134 31 22 the the DET fcis-32134 31 23 set set NOUN fcis-32134 31 24 of of ADP fcis-32134 31 25 state state NOUN fcis-32134 31 26 space space NOUN fcis-32134 31 27 ; ; PUNCT fcis-32134 31 28 a a PRON fcis-32134 31 29 is be AUX fcis-32134 31 30 the the DET fcis-32134 31 31 set set NOUN fcis-32134 31 32 of of ADP fcis-32134 31 33 action action NOUN fcis-32134 31 34 space space NOUN fcis-32134 31 35 ; ; PUNCT fcis-32134 31 36 p p PRON fcis-32134 31 37 is be AUX fcis-32134 31 38 the the DET fcis-32134 31 39 state state NOUN fcis-32134 31 40 transfer transfer NOUN fcis-32134 31 41 probability,𝑃 probability,𝑃 NOUN fcis-32134 31 42 𝔼 𝔼 PROPN fcis-32134 31 43 𝑆 𝑆 PROPN fcis-32134 31 44 𝑠 𝑠 PROPN fcis-32134 31 45 |𝑆 |𝑆 PROPN fcis-32134 31 46 𝑠 𝑠 PROPN fcis-32134 31 47 , , PUNCT fcis-32134 31 48 𝐴 𝐴 PROPN fcis-32134 31 49 π‘Ž π‘Ž PROPN fcis-32134 31 50 οΌ› οΌ› NOUN fcis-32134 31 51 r r NOUN fcis-32134 31 52 is be AUX fcis-32134 31 53 the the DET fcis-32134 31 54 reward reward NOUN fcis-32134 31 55 function,𝑅 function,𝑅 NOUN fcis-32134 31 56 , , PUNCT fcis-32134 31 57 𝐸 𝐸 PROPN fcis-32134 31 58 𝑅 𝑅 PROPN fcis-32134 31 59 ∣ ∣ NOUN fcis-32134 31 60 𝑠 𝑠 PROPN fcis-32134 31 61 𝑠 𝑠 PROPN fcis-32134 31 62 , , PUNCT fcis-32134 31 63 π‘Ž π‘Ž PROPN fcis-32134 31 64 π‘Ž π‘Ž NOUN fcis-32134 31 65 ; ; PUNCT fcis-32134 31 66 Ξ³ Ξ³ X fcis-32134 31 67 is be AUX fcis-32134 31 68 the the DET fcis-32134 31 69 discount discount NOUN fcis-32134 31 70 factor factor NOUN fcis-32134 31 71 , , PUNCT fcis-32134 31 72 the the DET fcis-32134 31 73 ultimate ultimate ADJ fcis-32134 31 74 goal goal NOUN fcis-32134 31 75 of of ADP fcis-32134 31 76 this this DET fcis-32134 31 77 process process NOUN fcis-32134 31 78 is be AUX fcis-32134 31 79 to to PART fcis-32134 31 80 find find VERB fcis-32134 31 81 the the DET fcis-32134 31 82 strategy strategy NOUN fcis-32134 31 83 with with ADP fcis-32134 31 84 the the DET fcis-32134 31 85 greatest great ADJ fcis-32134 31 86 payoff payoff NOUN fcis-32134 31 87 among among ADP fcis-32134 31 88 the the DET fcis-32134 31 89 various various ADJ fcis-32134 31 90 strategies strategy NOUN fcis-32134 31 91 Ο€ Ο€ NOUN fcis-32134 31 92 . . PUNCT fcis-32134 32 1 the the DET fcis-32134 32 2 process process NOUN fcis-32134 32 3 is be AUX fcis-32134 32 4 based base VERB fcis-32134 32 5 on on ADP fcis-32134 32 6 a a DET fcis-32134 32 7 set set NOUN fcis-32134 32 8 of of ADP fcis-32134 32 9 strategies strategy NOUN fcis-32134 32 10 Ο€ Ο€ NOUN fcis-32134 32 11 , , PUNCT fcis-32134 32 12 which which PRON fcis-32134 32 13 is be AUX fcis-32134 32 14 the the DET fcis-32134 32 15 most most ADV fcis-32134 32 16 efficient efficient ADJ fcis-32134 32 17 one one NUM fcis-32134 32 18 . . PUNCT fcis-32134 33 1 the the DET fcis-32134 33 2 strategies strategy NOUN fcis-32134 33 3 satisfy satisfy VERB fcis-32134 33 4 the the DET fcis-32134 33 5 functionπœ‹ functionπœ‹ NOUN fcis-32134 33 6 π‘Ž π‘Ž PROPN fcis-32134 33 7 ∣ ∣ ADJ fcis-32134 33 8 𝑠 𝑠 PROPN fcis-32134 33 9 𝑝 𝑝 PROPN fcis-32134 33 10 𝐴 𝐴 PROPN fcis-32134 33 11 π‘Ž π‘Ž PROPN fcis-32134 33 12 ∣ ∣ ADJ fcis-32134 33 13 𝑆 𝑆 PROPN fcis-32134 33 14 𝑠 𝑠 PROPN fcis-32134 33 15 . . PUNCT fcis-32134 34 1 the the DET fcis-32134 34 2 task task NOUN fcis-32134 34 3 of of ADP fcis-32134 34 4 this this DET fcis-32134 34 5 paper paper NOUN fcis-32134 34 6 is be AUX fcis-32134 34 7 to to PART fcis-32134 34 8 let let VERB fcis-32134 34 9 the the DET fcis-32134 34 10 uav uav PROPN fcis-32134 34 11 interact interact VERB fcis-32134 34 12 with with ADP fcis-32134 34 13 the the DET fcis-32134 34 14 environment environment NOUN fcis-32134 34 15 during during ADP fcis-32134 34 16 flight flight NOUN fcis-32134 34 17 , , PUNCT fcis-32134 34 18 avoid avoid VERB fcis-32134 34 19 various various ADJ fcis-32134 34 20 static static ADJ fcis-32134 34 21 and and CCONJ fcis-32134 34 22 dynamic dynamic ADJ fcis-32134 34 23 obstacles obstacle NOUN fcis-32134 34 24 without without ADP fcis-32134 34 25 collision collision NOUN fcis-32134 34 26 , , PUNCT fcis-32134 34 27 and and CCONJ fcis-32134 34 28 finally finally ADV fcis-32134 34 29 reach reach VERB fcis-32134 34 30 the the DET fcis-32134 34 31 target target NOUN fcis-32134 34 32 point point NOUN fcis-32134 34 33 successfully successfully ADV fcis-32134 34 34 . . PUNCT fcis-32134 35 1 due due ADP fcis-32134 35 2 to to ADP fcis-32134 35 3 the the DET fcis-32134 35 4 local local ADJ fcis-32134 35 5 observability observability NOUN fcis-32134 35 6 of of ADP fcis-32134 35 7 the the DET fcis-32134 35 8 environment environment NOUN fcis-32134 35 9 , , PUNCT fcis-32134 35 10 the the DET fcis-32134 35 11 task task NOUN fcis-32134 35 12 can can AUX fcis-32134 35 13 be be AUX fcis-32134 35 14 modeled model VERB fcis-32134 35 15 as as ADP fcis-32134 35 16 a a DET fcis-32134 35 17 partiallyobservable partiallyobservable ADJ fcis-32134 35 18 markov markov NOUN fcis-32134 35 19 decision decision NOUN fcis-32134 35 20 - - PUNCT fcis-32134 35 21 making make VERB fcis-32134 35 22 process process NOUN fcis-32134 35 23 , , PUNCT fcis-32134 35 24 which which PRON fcis-32134 35 25 can can AUX fcis-32134 35 26 be be AUX fcis-32134 35 27 incorporated incorporate VERB fcis-32134 35 28 into into ADP fcis-32134 35 29 the the DET fcis-32134 35 30 framework framework NOUN fcis-32134 35 31 of of ADP fcis-32134 35 32 reinforcement reinforcement NOUN fcis-32134 35 33 learning learning NOUN fcis-32134 35 34 by by ADP fcis-32134 35 35 considering consider VERB fcis-32134 35 36 the the DET fcis-32134 35 37 use use NOUN fcis-32134 35 38 of of ADP fcis-32134 35 39 environment environment NOUN fcis-32134 35 40 observations observation NOUN fcis-32134 35 41 instead instead ADV fcis-32134 35 42 of of ADP fcis-32134 35 43 complete complete ADJ fcis-32134 35 44 state state NOUN fcis-32134 35 45 quantities quantity NOUN fcis-32134 35 46 . . PUNCT fcis-32134 36 1 2.2 2.2 NUM fcis-32134 36 2 . . PUNCT fcis-32134 36 3 sac sac NOUN fcis-32134 36 4 algorithm algorithm NOUN fcis-32134 36 5 framework framework NOUN fcis-32134 36 6 sac sac NOUN fcis-32134 36 7 ( ( PUNCT fcis-32134 36 8 soft soft ADJ fcis-32134 36 9 actor actor NOUN fcis-32134 36 10 - - PUNCT fcis-32134 36 11 critic critic NOUN fcis-32134 36 12 ) ) PUNCT fcis-32134 36 13 is be AUX fcis-32134 36 14 an an DET fcis-32134 36 15 off off ADJ fcis-32134 36 16 - - PUNCT fcis-32134 36 17 policy policy NOUN fcis-32134 36 18 actor actor NOUN fcis-32134 36 19 - - PUNCT fcis-32134 36 20 critic critic NOUN fcis-32134 36 21 deep deep ADJ fcis-32134 36 22 reinforcement reinforcement NOUN fcis-32134 36 23 learning learning NOUN fcis-32134 36 24 algorithm algorithm NOUN fcis-32134 36 25 grounded ground VERB fcis-32134 36 26 in in ADP fcis-32134 36 27 the the DET fcis-32134 36 28 maximum maximum PROPN fcis-32134 36 29 entropy entropy NOUN fcis-32134 36 30 framework framework NOUN fcis-32134 36 31 , , PUNCT fcis-32134 36 32 wherein wherein SCONJ fcis-32134 36 33 the the DET fcis-32134 36 34 actor actor NOUN fcis-32134 36 35 jointly jointly ADV fcis-32134 36 36 maximizes maximize VERB fcis-32134 36 37 expected expect VERB fcis-32134 36 38 reward reward NOUN fcis-32134 36 39 and and CCONJ fcis-32134 36 40 policy policy NOUN fcis-32134 36 41 entropy entropy NOUN fcis-32134 36 42 , , PUNCT fcis-32134 36 43 i.e. i.e. X fcis-32134 36 44 , , PUNCT fcis-32134 36 45 completing complete VERB fcis-32134 36 46 the the DET fcis-32134 36 47 task task NOUN fcis-32134 36 48 as as ADV fcis-32134 36 49 randomly randomly ADV fcis-32134 36 50 as as ADP fcis-32134 36 51 possible possible ADJ fcis-32134 36 52 [ [ X fcis-32134 36 53 12 12 NUM fcis-32134 36 54 ] ] X fcis-32134 37 1 [ [ X fcis-32134 37 2 13 13 NUM fcis-32134 37 3 ] ] PUNCT fcis-32134 37 4 . . PUNCT fcis-32134 38 1 maximizing maximize VERB fcis-32134 38 2 entropy entropy NOUN fcis-32134 38 3 allows allow VERB fcis-32134 38 4 the the DET fcis-32134 38 5 strategy strategy NOUN fcis-32134 38 6 to to PART fcis-32134 38 7 be be AUX fcis-32134 38 8 motivated motivate VERB fcis-32134 38 9 to to PART fcis-32134 38 10 explore explore VERB fcis-32134 38 11 the the DET fcis-32134 38 12 space space NOUN fcis-32134 38 13 more more ADV fcis-32134 38 14 extensively extensively ADV fcis-32134 38 15 , , PUNCT fcis-32134 38 16 capturing capture VERB fcis-32134 38 17 multiple multiple ADJ fcis-32134 38 18 near near ADP fcis-32134 38 19 - - PUNCT fcis-32134 38 20 optimal optimal ADJ fcis-32134 38 21 behavioral behavioral ADJ fcis-32134 38 22 patterns pattern NOUN fcis-32134 38 23 while while SCONJ fcis-32134 38 24 abandoning abandon VERB fcis-32134 38 25 apparently apparently ADV fcis-32134 38 26 unpromising unpromising ADJ fcis-32134 38 27 pathways pathway NOUN fcis-32134 38 28 , , PUNCT fcis-32134 38 29 making make VERB fcis-32134 38 30 the the DET fcis-32134 38 31 model model NOUN fcis-32134 38 32 more more ADV fcis-32134 38 33 robust robust ADJ fcis-32134 38 34 in in ADP fcis-32134 38 35 the the DET fcis-32134 38 36 face face NOUN fcis-32134 38 37 of of ADP fcis-32134 38 38 disturbances disturbance NOUN fcis-32134 38 39 . . PUNCT fcis-32134 39 1 the the DET fcis-32134 39 2 sac sac NOUN fcis-32134 39 3 algorithm algorithm NOUN fcis-32134 39 4 framework framework NOUN fcis-32134 39 5 is be AUX fcis-32134 39 6 shown show VERB fcis-32134 39 7 in in ADP fcis-32134 39 8 figure figure NOUN fcis-32134 39 9 1 1 NUM fcis-32134 39 10 and and CCONJ fcis-32134 39 11 contains contain VERB fcis-32134 39 12 the the DET fcis-32134 39 13 intelligences intelligence NOUN fcis-32134 39 14 , , PUNCT fcis-32134 39 15 the the DET fcis-32134 39 16 environment environment NOUN fcis-32134 39 17 , , PUNCT fcis-32134 39 18 the the DET fcis-32134 39 19 experience experience NOUN fcis-32134 39 20 buffer buffer NOUN fcis-32134 39 21 pool pool NOUN fcis-32134 39 22 , , PUNCT fcis-32134 39 23 one one NUM fcis-32134 39 24 policy policy NOUN fcis-32134 39 25 network network NOUN fcis-32134 39 26 , , PUNCT fcis-32134 39 27 two two NUM fcis-32134 39 28 evaluation evaluation NOUN fcis-32134 39 29 networks network NOUN fcis-32134 39 30 and and CCONJ fcis-32134 39 31 two two NUM fcis-32134 39 32 goal goal NOUN fcis-32134 39 33 evaluation evaluation NOUN fcis-32134 39 34 networks network NOUN fcis-32134 39 35 . . PUNCT fcis-32134 40 1 figure figure NOUN fcis-32134 40 2 1 1 NUM fcis-32134 40 3 . . PUNCT fcis-32134 40 4 sac sac NOUN fcis-32134 40 5 algorithm algorithm NOUN fcis-32134 40 6 framework framework NOUN fcis-32134 40 7 actor actor NOUN fcis-32134 40 8 network network NOUN fcis-32134 40 9 is be AUX fcis-32134 40 10 used use VERB fcis-32134 40 11 to to PART fcis-32134 40 12 output output VERB fcis-32134 40 13 mean mean VERB fcis-32134 40 14 πœ‡ πœ‡ ADP fcis-32134 40 15 and and CCONJ fcis-32134 40 16 standard standard ADJ fcis-32134 40 17 deviation𝜎 deviation𝜎 NOUN fcis-32134 40 18 conditioned condition VERB fcis-32134 40 19 on on ADP fcis-32134 40 20 current current ADJ fcis-32134 40 21 state state NOUN fcis-32134 40 22 𝑠 𝑠 PROPN fcis-32134 40 23 . . PUNCT fcis-32134 41 1 the the DET fcis-32134 41 2 policyπœ‹ policyπœ‹ PROPN fcis-32134 41 3 β‹…βˆ£ β‹…βˆ£ PROPN fcis-32134 41 4 𝑠 𝑠 PROPN fcis-32134 41 5 is be AUX fcis-32134 41 6 usually usually ADV fcis-32134 41 7 modeled model VERB fcis-32134 41 8 as as ADP fcis-32134 41 9 a a DET fcis-32134 41 10 gaussian gaussian ADJ fcis-32134 41 11 distribution distribution NOUN fcis-32134 41 12 with with ADP fcis-32134 41 13 meanπœ‡and meanπœ‡and NOUN fcis-32134 41 14 standard standard ADJ fcis-32134 41 15 deviation deviation NOUN fcis-32134 41 16 𝜎 𝜎 PRON fcis-32134 41 17 . . PUNCT fcis-32134 42 1 actions action NOUN fcis-32134 42 2 are be AUX fcis-32134 42 3 sampled sample VERB fcis-32134 42 4 by by ADP fcis-32134 42 5 reparameterization reparameterization NOUN fcis-32134 42 6 , , PUNCT fcis-32134 42 7 and and CCONJ fcis-32134 42 8 the the DET fcis-32134 42 9 intelligent intelligent ADJ fcis-32134 42 10 body body NOUN fcis-32134 42 11 receives receive VERB fcis-32134 42 12 a a DET fcis-32134 42 13 rewardπ‘Ÿ rewardπ‘Ÿ NOUN fcis-32134 42 14 after after ADP fcis-32134 42 15 executing execute VERB fcis-32134 42 16 the the DET fcis-32134 42 17 actionπ‘Ž actionπ‘Ž NOUN fcis-32134 42 18 , , PUNCT fcis-32134 42 19 and and CCONJ fcis-32134 42 20 the the DET fcis-32134 42 21 state state NOUN fcis-32134 42 22 changes change VERB fcis-32134 42 23 to to ADP fcis-32134 42 24 𝑠 𝑠 PROPN fcis-32134 42 25 +1 +1 PROPN fcis-32134 42 26 . . PUNCT fcis-32134 43 1 since since SCONJ fcis-32134 43 2 the the DET fcis-32134 43 3 sac sac NOUN fcis-32134 43 4 algorithm algorithm NOUN fcis-32134 43 5 takes take VERB fcis-32134 43 6 into into ADP fcis-32134 43 7 account account NOUN fcis-32134 43 8 the the DET fcis-32134 43 9 maximum maximum ADJ fcis-32134 43 10 entropy entropy NOUN fcis-32134 43 11 of of ADP fcis-32134 43 12 the the DET fcis-32134 43 13 output output NOUN fcis-32134 43 14 action action NOUN fcis-32134 43 15 , , PUNCT fcis-32134 43 16 the the DET fcis-32134 43 17 policy policy NOUN fcis-32134 43 18 optimization optimization NOUN fcis-32134 43 19 objective objective ADJ fcis-32134 43 20 formula formula NOUN fcis-32134 43 21 with with ADP fcis-32134 43 22 entropy entropy NOUN fcis-32134 43 23 regularization regularization NOUN fcis-32134 43 24 is be AUX fcis-32134 43 25 : : PUNCT fcis-32134 43 26 𝐽 𝐽 PROPN fcis-32134 43 27 πœ‹ πœ‹ NOUN fcis-32134 43 28 𝔼 𝔼 PROPN fcis-32134 43 29 , , PUNCT fcis-32134 43 30 𝛾 𝛾 PROPN fcis-32134 43 31 π‘Ÿ π‘Ÿ X fcis-32134 43 32 𝑠 𝑠 INTJ fcis-32134 43 33 , , PUNCT fcis-32134 43 34 π‘Ž π‘Ž DET fcis-32134 43 35 𝛼𝐻 𝛼𝐻 NOUN fcis-32134 43 36 πœ‹ πœ‹ NOUN fcis-32134 43 37 β‹…βˆ£ β‹…βˆ£ NOUN fcis-32134 43 38 𝑠 𝑠 PROPN fcis-32134 43 39 ( ( PUNCT fcis-32134 43 40 1 1 NUM fcis-32134 43 41 ) ) PUNCT fcis-32134 43 42 where where SCONJ fcis-32134 43 43 𝐻 𝐻 PROPN fcis-32134 43 44 πœ‹ πœ‹ NOUN fcis-32134 43 45 β‹…βˆ£ β‹…βˆ£ NOUN fcis-32134 43 46 𝑠 𝑠 PROPN fcis-32134 43 47 is be AUX fcis-32134 43 48 the the DET fcis-32134 43 49 entropy entropy NOUN fcis-32134 43 50 of of ADP fcis-32134 43 51 the the DET fcis-32134 43 52 strategy strategy NOUN fcis-32134 43 53 in in ADP fcis-32134 43 54 the the DET fcis-32134 43 55 state state NOUN fcis-32134 43 56 𝑠 𝑠 PROPN fcis-32134 43 57 , , PUNCT fcis-32134 43 58 𝐻 𝐻 PROPN fcis-32134 43 59 πœ‹ πœ‹ NOUN fcis-32134 43 60 β‹…βˆ£ β‹…βˆ£ NOUN fcis-32134 43 61 𝑠 𝑠 INTJ fcis-32134 43 62 π‘™π‘œπ‘”πœ‹ π‘™π‘œπ‘”πœ‹ VERB fcis-32134 43 63 β‹…βˆ£ β‹…βˆ£ NOUN fcis-32134 43 64 𝑠 𝑠 PROPN fcis-32134 43 65 , , PUNCT fcis-32134 43 66 𝛾is 𝛾is VERB fcis-32134 43 67 discount discount NOUN fcis-32134 43 68 factor factor NOUN fcis-32134 43 69 , , PUNCT fcis-32134 43 70 and and CCONJ fcis-32134 43 71 Ξ± Ξ± PRON fcis-32134 43 72 is be AUX fcis-32134 43 73 an an DET fcis-32134 43 74 automatically automatically ADV fcis-32134 43 75 adjustable adjustable ADJ fcis-32134 43 76 temperature temperature NOUN fcis-32134 43 77 coefficient coefficient NOUN fcis-32134 43 78 that that PRON fcis-32134 43 79 regulates regulate VERB fcis-32134 43 80 the the DET fcis-32134 43 81 entropy entropy NOUN fcis-32134 43 82 - - PUNCT fcis-32134 43 83 reward reward NOUN fcis-32134 43 84 trade trade NOUN fcis-32134 43 85 - - PUNCT fcis-32134 43 86 off off NOUN fcis-32134 43 87 to to PART fcis-32134 43 88 modulate modulate VERB fcis-32134 43 89 optimal optimal ADJ fcis-32134 43 90 policy policy NOUN fcis-32134 43 91 stochasticity stochasticity NOUN fcis-32134 43 92 . . PUNCT fcis-32134 44 1 the the DET fcis-32134 44 2 critic critic PROPN fcis-32134 44 3 network network NOUN fcis-32134 44 4 is be AUX fcis-32134 44 5 used use VERB fcis-32134 44 6 to to PART fcis-32134 44 7 estimate estimate VERB fcis-32134 44 8 the the DET fcis-32134 44 9 expected expect VERB fcis-32134 44 10 cumulative cumulative ADJ fcis-32134 44 11 reward reward NOUN fcis-32134 44 12 ( ( PUNCT fcis-32134 44 13 including include VERB fcis-32134 44 14 the the DET fcis-32134 44 15 entropy entropy NOUN fcis-32134 44 16 reward reward NOUN fcis-32134 44 17 ) ) PUNCT fcis-32134 44 18 for for ADP fcis-32134 44 19 a a DET fcis-32134 44 20 given give VERB fcis-32134 44 21 state state NOUN fcis-32134 44 22 and and CCONJ fcis-32134 44 23 action.sac action.sac NOUN fcis-32134 44 24 uses use VERB fcis-32134 44 25 two two NUM fcis-32134 44 26 critic critic ADJ fcis-32134 44 27 networks network NOUN fcis-32134 44 28 and and CCONJ fcis-32134 44 29 the the DET fcis-32134 44 30 44 44 NUM fcis-32134 44 31 corresponding correspond VERB fcis-32134 44 32 target target NOUN fcis-32134 44 33 network network NOUN fcis-32134 44 34 to to PART fcis-32134 44 35 reduce reduce VERB fcis-32134 44 36 the the DET fcis-32134 44 37 overestimation overestimation NOUN fcis-32134 44 38 bias bias NOUN fcis-32134 44 39 . . PUNCT fcis-32134 45 1 the the DET fcis-32134 45 2 network network NOUN fcis-32134 45 3 outputs output VERB fcis-32134 45 4 two two NUM fcis-32134 45 5 q q NOUN fcis-32134 45 6 - - PUNCT fcis-32134 45 7 values value NOUN fcis-32134 45 8 based base VERB fcis-32134 45 9 on on ADP fcis-32134 45 10 the the DET fcis-32134 45 11 𝑠 𝑠 PROPN fcis-32134 45 12 , , PUNCT fcis-32134 45 13 π‘Ž π‘Ž PROPN fcis-32134 45 14 of of ADP fcis-32134 45 15 the the DET fcis-32134 45 16 intelligent intelligent ADJ fcis-32134 45 17 body body NOUN fcis-32134 45 18 , , PUNCT fcis-32134 45 19 and and CCONJ fcis-32134 45 20 the the DET fcis-32134 45 21 minimum minimum NOUN fcis-32134 45 22 of of ADP fcis-32134 45 23 the the DET fcis-32134 45 24 two two NUM fcis-32134 45 25 q q NOUN fcis-32134 45 26 - - PUNCT fcis-32134 45 27 values value NOUN fcis-32134 45 28 is be AUX fcis-32134 45 29 taken take VERB fcis-32134 45 30 as as ADP fcis-32134 45 31 the the DET fcis-32134 45 32 target target NOUN fcis-32134 45 33 value value NOUN fcis-32134 45 34 when when SCONJ fcis-32134 45 35 updating update VERB fcis-32134 45 36 , , PUNCT fcis-32134 45 37 which which PRON fcis-32134 45 38 provides provide VERB fcis-32134 45 39 a a DET fcis-32134 45 40 more more ADV fcis-32134 45 41 conservative conservative ADJ fcis-32134 45 42 estimation estimation NOUN fcis-32134 45 43 . . PUNCT fcis-32134 46 1 the the DET fcis-32134 46 2 q q ADJ fcis-32134 46 3 - - PUNCT fcis-32134 46 4 function function NOUN fcis-32134 46 5 equation equation NOUN fcis-32134 46 6 is be AUX fcis-32134 46 7 : : PUNCT fcis-32134 46 8 𝑄 𝑄 PROPN fcis-32134 46 9 𝑠 𝑠 PROPN fcis-32134 46 10 , , PUNCT fcis-32134 46 11 π‘Ž π‘Ž PROPN fcis-32134 46 12 π‘Ÿ π‘Ÿ NOUN fcis-32134 46 13 𝑠 𝑠 INTJ fcis-32134 46 14 , , PUNCT fcis-32134 46 15 π‘Ž π‘Ž PROPN fcis-32134 46 16 𝛾 𝛾 NOUN fcis-32134 46 17     SPACE fcis-32134 46 18 𝔼 𝔼 PROPN fcis-32134 46 19 𝑉 𝑉 PROPN fcis-32134 46 20 𝑠 𝑠 PROPN fcis-32134 46 21 ( ( PUNCT fcis-32134 46 22 2 2 NUM fcis-32134 46 23 ) ) PUNCT fcis-32134 46 24 𝑉 𝑉 PROPN fcis-32134 46 25 𝑠 𝑠 PROPN fcis-32134 46 26 is be AUX fcis-32134 46 27 the the DET fcis-32134 46 28 state state NOUN fcis-32134 46 29 - - PUNCT fcis-32134 46 30 value value NOUN fcis-32134 46 31 function function NOUN fcis-32134 46 32 that that PRON fcis-32134 46 33 represents represent VERB fcis-32134 46 34 the the DET fcis-32134 46 35 value value NOUN fcis-32134 46 36 at at ADP fcis-32134 46 37 𝑠 𝑠 PROPN fcis-32134 46 38 . . PUNCT fcis-32134 47 1 the the DET fcis-32134 47 2 state state NOUN fcis-32134 47 3 - - PUNCT fcis-32134 47 4 value value NOUN fcis-32134 47 5 function function NOUN fcis-32134 47 6 formula formula NOUN fcis-32134 47 7 is be AUX fcis-32134 47 8 : : PUNCT fcis-32134 47 9 𝑉 𝑉 PROPN fcis-32134 47 10 𝑠 𝑠 NOUN fcis-32134 47 11 𝔼 𝔼 NOUN fcis-32134 47 12 ∼ ∼ NOUN fcis-32134 47 13 𝑄 𝑄 PROPN fcis-32134 47 14 𝑠 𝑠 PROPN fcis-32134 47 15 , , PUNCT fcis-32134 47 16 π‘Ž π‘Ž PROPN fcis-32134 47 17 𝛼 𝛼 NOUN fcis-32134 47 18 log log NOUN fcis-32134 47 19 πœ‹ πœ‹ NOUN fcis-32134 47 20 π‘Ž π‘Ž PROPN fcis-32134 47 21 ∣ ∣ ADJ fcis-32134 47 22 𝑠 𝑠 PROPN fcis-32134 47 23 ( ( PUNCT fcis-32134 47 24 3 3 NUM fcis-32134 47 25 ) ) PUNCT fcis-32134 47 26 in in ADP fcis-32134 47 27 addition addition NOUN fcis-32134 47 28 , , PUNCT fcis-32134 47 29 sac sac NOUN fcis-32134 47 30 constructs construct VERB fcis-32134 47 31 a a DET fcis-32134 47 32 critic critic ADJ fcis-32134 47 33 target target NOUN fcis-32134 47 34 network network NOUN fcis-32134 47 35 with with ADP fcis-32134 47 36 the the DET fcis-32134 47 37 same same ADJ fcis-32134 47 38 structure structure NOUN fcis-32134 47 39 as as ADP fcis-32134 47 40 the the DET fcis-32134 47 41 critic critic NOUN fcis-32134 47 42 network network NOUN fcis-32134 47 43 but but CCONJ fcis-32134 47 44 with with ADP fcis-32134 47 45 different different ADJ fcis-32134 47 46 parameters parameter NOUN fcis-32134 47 47 , , PUNCT fcis-32134 47 48 which which PRON fcis-32134 47 49 outputs output VERB fcis-32134 47 50 the the DET fcis-32134 47 51 target target NOUN fcis-32134 47 52 𝑄 𝑄 NOUN fcis-32134 47 53 value value NOUN fcis-32134 47 54 , , PUNCT fcis-32134 47 55 and and CCONJ fcis-32134 47 56 then then ADV fcis-32134 47 57 obtains obtain VERB fcis-32134 47 58 the the DET fcis-32134 47 59 target target NOUN fcis-32134 47 60 value𝑦 value𝑦 NOUN fcis-32134 47 61 , , PUNCT fcis-32134 47 62 which which PRON fcis-32134 47 63 improves improve VERB fcis-32134 47 64 the the DET fcis-32134 47 65 stability stability NOUN fcis-32134 47 66 of of ADP fcis-32134 47 67 neural neural ADJ fcis-32134 47 68 network network NOUN fcis-32134 47 69 training.the training.the DET fcis-32134 47 70 critic critic PROPN fcis-32134 47 71 target target NOUN fcis-32134 47 72 network network NOUN fcis-32134 47 73 also also ADV fcis-32134 47 74 takes take VERB fcis-32134 47 75 the the DET fcis-32134 47 76 smallest small ADJ fcis-32134 47 77 of of ADP fcis-32134 47 78 the the DET fcis-32134 47 79 two two NUM fcis-32134 47 80 𝑄 𝑄 PROPN fcis-32134 47 81 values value NOUN fcis-32134 47 82 as as ADP fcis-32134 47 83 the the DET fcis-32134 47 84 target target NOUN fcis-32134 47 85 value value NOUN fcis-32134 47 86 at at ADP fcis-32134 47 87 the the DET fcis-32134 47 88 time time NOUN fcis-32134 47 89 of of ADP fcis-32134 47 90 updating updating NOUN fcis-32134 47 91 , , PUNCT fcis-32134 47 92 which which PRON fcis-32134 47 93 mitigates mitigate VERB fcis-32134 47 94 overestimation overestimation NOUN fcis-32134 47 95 bias bias NOUN fcis-32134 47 96 at at ADP fcis-32134 47 97 the the DET fcis-32134 47 98 time time NOUN fcis-32134 47 99 of of ADP fcis-32134 47 100 training.the training.the DET fcis-32134 47 101 loss loss NOUN fcis-32134 47 102 function function NOUN fcis-32134 47 103 of of ADP fcis-32134 47 104 the the DET fcis-32134 47 105 critic critic NOUN fcis-32134 47 106 network network NOUN fcis-32134 47 107 adopts adopt VERB fcis-32134 47 108 the the DET fcis-32134 47 109 bellman bellman NOUN fcis-32134 47 110 equation equation NOUN fcis-32134 47 111 's 's PART fcis-32134 47 112 mean mean ADJ fcis-32134 47 113 - - PUNCT fcis-32134 47 114 square square NOUN fcis-32134 47 115 error error NOUN fcis-32134 47 116 : : PUNCT fcis-32134 47 117 𝐿 𝐿 PROPN fcis-32134 47 118 πœƒ πœƒ PROPN fcis-32134 47 119 𝔼 𝔼 PROPN fcis-32134 47 120 , , PUNCT fcis-32134 47 121 , , PUNCT fcis-32134 47 122 , , PUNCT fcis-32134 47 123 ∼ ∼ NOUN fcis-32134 47 124 𝑄 𝑄 PROPN fcis-32134 47 125 𝑠 𝑠 PROPN fcis-32134 47 126 , , PUNCT fcis-32134 47 127 π‘Ž π‘Ž PROPN fcis-32134 47 128 𝑦 𝑦 NOUN fcis-32134 47 129 ( ( PUNCT fcis-32134 47 130 4 4 NUM fcis-32134 47 131 ) ) PUNCT fcis-32134 47 132 the the DET fcis-32134 47 133 target target NOUN fcis-32134 47 134 value value NOUN fcis-32134 47 135 𝑦 𝑦 PRON fcis-32134 47 136 calculation calculation NOUN fcis-32134 47 137 formula formula NOUN fcis-32134 47 138 : : PUNCT fcis-32134 47 139 𝑦 𝑦 NUM fcis-32134 47 140 π‘Ÿ π‘Ÿ NOUN fcis-32134 47 141 𝛾 𝛾 ADP fcis-32134 47 142 min min NOUN fcis-32134 47 143 , , PUNCT fcis-32134 47 144 𝑄 𝑄 PROPN fcis-32134 47 145 𝑠 𝑠 PROPN fcis-32134 47 146 , , PUNCT fcis-32134 47 147 π‘Ž π‘Ž PROPN fcis-32134 47 148 𝛼 𝛼 NOUN fcis-32134 47 149 log log NOUN fcis-32134 47 150 πœ‹ πœ‹ NOUN fcis-32134 47 151 π‘Ž π‘Ž PROPN fcis-32134 47 152 ∣ ∣ ADJ fcis-32134 47 153 𝑠 𝑠 PROPN fcis-32134 47 154 ( ( PUNCT fcis-32134 47 155 5 5 NUM fcis-32134 47 156 ) ) PUNCT fcis-32134 47 157 the the DET fcis-32134 47 158 loss loss NOUN fcis-32134 47 159 function function NOUN fcis-32134 47 160 of of ADP fcis-32134 47 161 actor actor NOUN fcis-32134 47 162 network network NOUN fcis-32134 47 163 is be AUX fcis-32134 47 164 : : PUNCT fcis-32134 47 165 𝐿 𝐿 PROPN fcis-32134 47 166 πœ‘ πœ‘ NOUN fcis-32134 47 167 𝔼 𝔼 NOUN fcis-32134 47 168 ∼ ∼ NOUN fcis-32134 47 169 𝔼 𝔼 NOUN fcis-32134 47 170 ∼ ∼ NOUN fcis-32134 47 171 𝛼 𝛼 NOUN fcis-32134 47 172 log log NOUN fcis-32134 47 173 πœ‹ πœ‹ NOUN fcis-32134 47 174 π‘Ž π‘Ž PROPN fcis-32134 47 175 ∣ ∣ ADJ fcis-32134 47 176 𝑠 𝑠 ADP fcis-32134 47 177 𝑄 𝑄 PROPN fcis-32134 47 178 𝑠 𝑠 PROPN fcis-32134 47 179 , , PUNCT fcis-32134 47 180 π‘Ž π‘Ž X fcis-32134 47 181 ( ( PUNCT fcis-32134 47 182 6 6 NUM fcis-32134 47 183 ) ) PUNCT fcis-32134 47 184 according accord VERB fcis-32134 47 185 to to ADP fcis-32134 47 186 the the DET fcis-32134 47 187 loss loss NOUN fcis-32134 47 188 functions function NOUN fcis-32134 47 189 lq(ΞΈ lq(ΞΈ X fcis-32134 47 190 ) ) PUNCT fcis-32134 47 191 and and CCONJ fcis-32134 47 192 𝐿 𝐿 PROPN fcis-32134 47 193 πœ‘ πœ‘ PROPN fcis-32134 47 194 , , PUNCT fcis-32134 47 195 the the DET fcis-32134 47 196 parameters parameter NOUN fcis-32134 47 197 of of ADP fcis-32134 47 198 actor actor NOUN fcis-32134 47 199 network network NOUN fcis-32134 47 200 and and CCONJ fcis-32134 47 201 critic critic PROPN fcis-32134 47 202 network network PROPN fcis-32134 47 203 πœƒ πœƒ PROPN fcis-32134 47 204 , , PUNCT fcis-32134 47 205 πœ‘ πœ‘ PROPN fcis-32134 47 206 are be AUX fcis-32134 47 207 updated update VERB fcis-32134 47 208 inversely inversely ADV fcis-32134 47 209 , , PUNCT fcis-32134 47 210 and and CCONJ fcis-32134 47 211 the the DET fcis-32134 47 212 parameter parameter NOUN fcis-32134 47 213 πœƒ πœƒ PROPN fcis-32134 47 214 of of ADP fcis-32134 47 215 critic critic ADJ fcis-32134 47 216 target target NOUN fcis-32134 47 217 network network NOUN fcis-32134 47 218 is be AUX fcis-32134 47 219 softly softly ADV fcis-32134 47 220 updated:πœƒ updated:πœƒ ADJ fcis-32134 47 221 ← ← PROPN fcis-32134 47 222 πœπœƒ πœπœƒ ADP fcis-32134 47 223 1 1 NUM fcis-32134 47 224 𝜏 𝜏 PRON fcis-32134 47 225 πœƒ πœƒ NOUN fcis-32134 47 226 , , PUNCT fcis-32134 47 227 and and CCONJ fcis-32134 47 228 the the DET fcis-32134 47 229 softly softly ADV fcis-32134 47 230 update update VERB fcis-32134 47 231 coefficient coefficient NOUN fcis-32134 47 232 𝜏 𝜏 NOUN fcis-32134 47 233 is be AUX fcis-32134 47 234 0.005 0.005 NUM fcis-32134 47 235 . . PUNCT fcis-32134 48 1 sac sac NOUN fcis-32134 48 2 automatically automatically ADV fcis-32134 48 3 adjusts adjust VERB fcis-32134 48 4 the the DET fcis-32134 48 5 temperature temperature NOUN fcis-32134 48 6 coefficient𝛼 coefficient𝛼 NOUN fcis-32134 48 7 so so SCONJ fcis-32134 48 8 that that SCONJ fcis-32134 48 9 the the DET fcis-32134 48 10 entropy entropy NOUN fcis-32134 48 11 of of ADP fcis-32134 48 12 the the DET fcis-32134 48 13 strategy strategy NOUN fcis-32134 48 14 is be AUX fcis-32134 48 15 close close ADJ fcis-32134 48 16 to to ADP fcis-32134 48 17 the the DET fcis-32134 48 18 target target NOUN fcis-32134 48 19 entropy entropy NOUN fcis-32134 48 20 . . PUNCT fcis-32134 49 1 the the DET fcis-32134 49 2 loss loss NOUN fcis-32134 49 3 function function NOUN fcis-32134 49 4 is be AUX fcis-32134 49 5 : : PUNCT fcis-32134 49 6 𝐿 𝐿 PROPN fcis-32134 49 7 𝛼 𝛼 NOUN fcis-32134 49 8 𝔼 𝔼 NOUN fcis-32134 49 9 ∼ ∼ NOUN fcis-32134 49 10 𝛼 𝛼 PRON fcis-32134 49 11 log log NOUN fcis-32134 49 12 πœ‹ πœ‹ NOUN fcis-32134 49 13 π‘Ž π‘Ž PROPN fcis-32134 49 14 ∣ ∣ ADJ fcis-32134 49 15 𝑠 𝑠 PROPN fcis-32134 49 16 𝐻 𝐻 PROPN fcis-32134 49 17 ( ( PUNCT fcis-32134 49 18 7 7 X fcis-32134 49 19 ) ) PUNCT fcis-32134 49 20 h h NOUN fcis-32134 49 21 is be AUX fcis-32134 49 22 the the DET fcis-32134 49 23 target target NOUN fcis-32134 49 24 entropy entropy NOUN fcis-32134 49 25 , , PUNCT fcis-32134 49 26 which which PRON fcis-32134 49 27 is be AUX fcis-32134 49 28 usually usually ADV fcis-32134 49 29 set set VERB fcis-32134 49 30 to to ADP fcis-32134 49 31 the the DET fcis-32134 49 32 negative negative ADJ fcis-32134 49 33 action action NOUN fcis-32134 49 34 dimension dimension NOUN fcis-32134 49 35 , , PUNCT fcis-32134 49 36 and and CCONJ fcis-32134 49 37 automatic automatic ADJ fcis-32134 49 38 adjustment adjustment NOUN fcis-32134 49 39 is be AUX fcis-32134 49 40 achieved achieve VERB fcis-32134 49 41 by by ADP fcis-32134 49 42 minimizing minimize VERB fcis-32134 49 43 the the DET fcis-32134 49 44 entropy entropy NOUN fcis-32134 49 45 loss loss NOUN fcis-32134 49 46 . . PUNCT fcis-32134 50 1 3 3 X fcis-32134 50 2 . . X fcis-32134 50 3 experimental experimental ADJ fcis-32134 50 4 design design NOUN fcis-32134 50 5 3.1 3.1 NUM fcis-32134 50 6 . . PUNCT fcis-32134 50 7 experimental experimental ADJ fcis-32134 50 8 scene scene NOUN fcis-32134 50 9 due due ADP fcis-32134 50 10 to to ADP fcis-32134 50 11 its its PRON fcis-32134 50 12 small small ADJ fcis-32134 50 13 size size NOUN fcis-32134 50 14 and and CCONJ fcis-32134 50 15 convenient convenient ADJ fcis-32134 50 16 features feature NOUN fcis-32134 50 17 , , PUNCT fcis-32134 50 18 uav uav PROPN fcis-32134 50 19 is be AUX fcis-32134 50 20 often often ADV fcis-32134 50 21 used use VERB fcis-32134 50 22 to to PART fcis-32134 50 23 perform perform VERB fcis-32134 50 24 tasks task NOUN fcis-32134 50 25 in in ADP fcis-32134 50 26 complex complex ADJ fcis-32134 50 27 three three NUM fcis-32134 50 28 - - PUNCT fcis-32134 50 29 dimensional dimensional ADJ fcis-32134 50 30 space space NOUN fcis-32134 50 31 . . PUNCT fcis-32134 51 1 in in ADP fcis-32134 51 2 order order NOUN fcis-32134 51 3 to to PART fcis-32134 51 4 better well ADV fcis-32134 51 5 simulate simulate VERB fcis-32134 51 6 the the DET fcis-32134 51 7 real real ADJ fcis-32134 51 8 environment environment NOUN fcis-32134 51 9 , , PUNCT fcis-32134 51 10 this this DET fcis-32134 51 11 paper paper NOUN fcis-32134 51 12 designs design VERB fcis-32134 51 13 a a DET fcis-32134 51 14 diversified diversified ADJ fcis-32134 51 15 flight flight NOUN fcis-32134 51 16 scene scene NOUN fcis-32134 51 17 containing contain VERB fcis-32134 51 18 static static ADJ fcis-32134 51 19 and and CCONJ fcis-32134 51 20 dynamic dynamic ADJ fcis-32134 51 21 obstacles obstacle NOUN fcis-32134 51 22 , , PUNCT fcis-32134 51 23 and and CCONJ fcis-32134 51 24 the the DET fcis-32134 51 25 specific specific ADJ fcis-32134 51 26 settings setting NOUN fcis-32134 51 27 are be AUX fcis-32134 51 28 as as SCONJ fcis-32134 51 29 follows follow VERB fcis-32134 51 30 : : PUNCT fcis-32134 51 31 ( ( PUNCT fcis-32134 51 32 1 1 X fcis-32134 51 33 ) ) PUNCT fcis-32134 51 34 experimental experimental ADJ fcis-32134 51 35 environment environment NOUN fcis-32134 51 36 : : PUNCT fcis-32134 51 37 the the DET fcis-32134 51 38 spatial spatial ADJ fcis-32134 51 39 size size NOUN fcis-32134 51 40 of of ADP fcis-32134 51 41 the the DET fcis-32134 51 42 environment environment NOUN fcis-32134 51 43 in in ADP fcis-32134 51 44 this this DET fcis-32134 51 45 paper paper NOUN fcis-32134 51 46 is be AUX fcis-32134 51 47 set set VERB fcis-32134 51 48 to60 to60 PROPN fcis-32134 51 49 m m VERB fcis-32134 51 50 60 60 NUM fcis-32134 51 51 m m NOUN fcis-32134 51 52 30 30 NUM fcis-32134 51 53 m m NOUN fcis-32134 51 54 , , PUNCT fcis-32134 51 55 while while SCONJ fcis-32134 51 56 the the DET fcis-32134 51 57 flight flight NOUN fcis-32134 51 58 altitude altitude NOUN fcis-32134 51 59 of of ADP fcis-32134 51 60 the the DET fcis-32134 51 61 uav uav PROPN fcis-32134 51 62 is be AUX fcis-32134 51 63 limited limit VERB fcis-32134 51 64 to to ADP fcis-32134 51 65 no no DET fcis-32134 51 66 less less ADJ fcis-32134 51 67 than than ADP fcis-32134 51 68 0.5 0.5 NUM fcis-32134 51 69 m m NOUN fcis-32134 51 70 , , PUNCT fcis-32134 51 71 and and CCONJ fcis-32134 51 72 the the DET fcis-32134 51 73 safety safety NOUN fcis-32134 51 74 distance distance NOUN fcis-32134 51 75 is be AUX fcis-32134 51 76 set set VERB fcis-32134 51 77 to to ADP fcis-32134 51 78 2 2 NUM fcis-32134 51 79 m m NOUN fcis-32134 51 80 , , PUNCT fcis-32134 51 81 and and CCONJ fcis-32134 51 82 different different ADJ fcis-32134 51 83 obstacles obstacle NOUN fcis-32134 51 84 are be AUX fcis-32134 51 85 introduced introduce VERB fcis-32134 51 86 into into ADP fcis-32134 51 87 this this DET fcis-32134 51 88 environment environment NOUN fcis-32134 51 89 to to PART fcis-32134 51 90 enhance enhance VERB fcis-32134 51 91 the the DET fcis-32134 51 92 spatial spatial ADJ fcis-32134 51 93 complexity complexity NOUN fcis-32134 51 94 . . PUNCT fcis-32134 52 1 ( ( PUNCT fcis-32134 52 2 2 2 X fcis-32134 52 3 ) ) PUNCT fcis-32134 52 4 obstacle obstacle NOUN fcis-32134 52 5 setting setting NOUN fcis-32134 52 6 : : PUNCT fcis-32134 52 7 in in ADP fcis-32134 52 8 order order NOUN fcis-32134 52 9 to to PART fcis-32134 52 10 simulate simulate VERB fcis-32134 52 11 the the DET fcis-32134 52 12 complex complex ADJ fcis-32134 52 13 environment environment NOUN fcis-32134 52 14 in in ADP fcis-32134 52 15 reality reality NOUN fcis-32134 52 16 , , PUNCT fcis-32134 52 17 this this DET fcis-32134 52 18 paper paper NOUN fcis-32134 52 19 sets set VERB fcis-32134 52 20 up up ADP fcis-32134 52 21 5 5 NUM fcis-32134 52 22 static static ADJ fcis-32134 52 23 obstacles obstacle NOUN fcis-32134 52 24 and and CCONJ fcis-32134 52 25 8 8 NUM fcis-32134 52 26 dynamic dynamic ADJ fcis-32134 52 27 obstacles obstacle NOUN fcis-32134 52 28 as as SCONJ fcis-32134 52 29 shown show VERB fcis-32134 52 30 in in ADP fcis-32134 52 31 figure figure NOUN fcis-32134 52 32 2 2 NUM fcis-32134 52 33 , , PUNCT fcis-32134 52 34 in in ADP fcis-32134 52 35 which which PRON fcis-32134 52 36 the the DET fcis-32134 52 37 dynamic dynamic ADJ fcis-32134 52 38 obstacles obstacle NOUN fcis-32134 52 39 include include VERB fcis-32134 52 40 the the DET fcis-32134 52 41 motion motion NOUN fcis-32134 52 42 of of ADP fcis-32134 52 43 circular circular ADJ fcis-32134 52 44 motion motion NOUN fcis-32134 52 45 , , PUNCT fcis-32134 52 46 spiral spiral ADJ fcis-32134 52 47 motion motion NOUN fcis-32134 52 48 , , PUNCT fcis-32134 52 49 compound compound VERB fcis-32134 52 50 periodic periodic ADJ fcis-32134 52 51 motion motion NOUN fcis-32134 52 52 , , PUNCT fcis-32134 52 53 asymmetric asymmetric ADJ fcis-32134 52 54 motion motion NOUN fcis-32134 52 55 , , PUNCT fcis-32134 52 56 straight straight ADJ fcis-32134 52 57 line line NOUN fcis-32134 52 58 + + CCONJ fcis-32134 52 59 fluctuation fluctuation NOUN fcis-32134 52 60 , , PUNCT fcis-32134 52 61 and and CCONJ fcis-32134 52 62 other other ADJ fcis-32134 52 63 kinds kind NOUN fcis-32134 52 64 of of ADP fcis-32134 52 65 complex complex ADJ fcis-32134 52 66 trajectories trajectory NOUN fcis-32134 52 67 ( ( PUNCT fcis-32134 52 68 the the DET fcis-32134 52 69 motion motion NOUN fcis-32134 52 70 trajectory trajectory NOUN fcis-32134 52 71 is be AUX fcis-32134 52 72 shown show VERB fcis-32134 52 73 in in ADP fcis-32134 52 74 figure figure NOUN fcis-32134 52 75 3 3 NUM fcis-32134 52 76 ) ) PUNCT fcis-32134 52 77 , , PUNCT fcis-32134 52 78 and and CCONJ fcis-32134 52 79 the the DET fcis-32134 52 80 radii radius NOUN fcis-32134 52 81 and and CCONJ fcis-32134 52 82 speeds speed NOUN fcis-32134 52 83 of of ADP fcis-32134 52 84 these these DET fcis-32134 52 85 obstacles obstacle NOUN fcis-32134 52 86 vary vary VERB fcis-32134 52 87 , , PUNCT fcis-32134 52 88 and and CCONJ fcis-32134 52 89 randomly randomly ADV fcis-32134 52 90 select select VERB fcis-32134 52 91 5 5 NUM fcis-32134 52 92 static static ADJ fcis-32134 52 93 obstacles obstacle NOUN fcis-32134 52 94 and and CCONJ fcis-32134 52 95 8 8 NUM fcis-32134 52 96 dynamic dynamic ADJ fcis-32134 52 97 obstacles obstacle NOUN fcis-32134 52 98 from from ADP fcis-32134 52 99 them they PRON fcis-32134 52 100 when when SCONJ fcis-32134 52 101 training training NOUN fcis-32134 52 102 . . PUNCT fcis-32134 53 1 five five NUM fcis-32134 53 2 static static ADJ fcis-32134 53 3 obstacles obstacle NOUN fcis-32134 53 4 and and CCONJ fcis-32134 53 5 five five NUM fcis-32134 53 6 dynamic dynamic ADJ fcis-32134 53 7 obstacles obstacle NOUN fcis-32134 53 8 are be AUX fcis-32134 53 9 randomly randomly ADV fcis-32134 53 10 selected select VERB fcis-32134 53 11 for for ADP fcis-32134 53 12 training training NOUN fcis-32134 53 13 , , PUNCT fcis-32134 53 14 so so SCONJ fcis-32134 53 15 as as SCONJ fcis-32134 53 16 to to PART fcis-32134 53 17 further far ADV fcis-32134 53 18 test test VERB fcis-32134 53 19 the the DET fcis-32134 53 20 autonomous autonomous ADJ fcis-32134 53 21 obstacle obstacle NOUN fcis-32134 53 22 avoidance avoidance NOUN fcis-32134 53 23 ability ability NOUN fcis-32134 53 24 of of ADP fcis-32134 53 25 the the DET fcis-32134 53 26 uav uav PROPN fcis-32134 53 27 . . PUNCT fcis-32134 54 1 figure figure NOUN fcis-32134 54 2 2 2 NUM fcis-32134 54 3 . . PUNCT fcis-32134 54 4 various various ADJ fcis-32134 54 5 types type NOUN fcis-32134 54 6 of of ADP fcis-32134 54 7 obstacle obstacle NOUN fcis-32134 54 8 figure figure NOUN fcis-32134 54 9 3 3 NUM fcis-32134 54 10 . . PUNCT fcis-32134 54 11 dynamic dynamic ADJ fcis-32134 54 12 obstacle obstacle NOUN fcis-32134 54 13 trajectory trajectory NOUN fcis-32134 54 14 ( ( PUNCT fcis-32134 54 15 3 3 NUM fcis-32134 54 16 ) ) PUNCT fcis-32134 54 17 uav uav PROPN fcis-32134 54 18 flight flight PROPN fcis-32134 54 19 task task PROPN fcis-32134 54 20 : : PUNCT fcis-32134 54 21 the the DET fcis-32134 54 22 initial initial ADJ fcis-32134 54 23 coordinates coordinate NOUN fcis-32134 54 24 of of ADP fcis-32134 54 25 the the DET fcis-32134 54 26 uav uav PROPN fcis-32134 54 27 are be AUX fcis-32134 54 28 ( ( PUNCT fcis-32134 54 29 10,10,5 10,10,5 NUM fcis-32134 54 30 ) ) PUNCT fcis-32134 54 31 m m PROPN fcis-32134 54 32 , , PUNCT fcis-32134 54 33 and and CCONJ fcis-32134 54 34 the the DET fcis-32134 54 35 endpoint endpoint NOUN fcis-32134 54 36 coordinates coordinate NOUN fcis-32134 54 37 are be AUX fcis-32134 54 38 ( ( PUNCT fcis-32134 54 39 50,50,10)m 50,50,10)m NOUN fcis-32134 54 40 . . PUNCT fcis-32134 55 1 the the DET fcis-32134 55 2 uav uav PROPN fcis-32134 55 3 needs need VERB fcis-32134 55 4 to to PART fcis-32134 55 5 start start VERB fcis-32134 55 6 from from ADP fcis-32134 55 7 the the DET fcis-32134 55 8 starting starting NOUN fcis-32134 55 9 point point NOUN fcis-32134 55 10 , , PUNCT fcis-32134 55 11 fly fly VERB fcis-32134 55 12 and and CCONJ fcis-32134 55 13 avoid avoid VERB fcis-32134 55 14 all all DET fcis-32134 55 15 kinds kind NOUN fcis-32134 55 16 of of ADP fcis-32134 55 17 obstacles obstacle NOUN fcis-32134 55 18 , , PUNCT fcis-32134 55 19 and and CCONJ fcis-32134 55 20 arrive arrive VERB fcis-32134 55 21 at at ADP fcis-32134 55 22 the the DET fcis-32134 55 23 endpoint endpoint NOUN fcis-32134 55 24 without without ADP fcis-32134 55 25 collision collision NOUN fcis-32134 55 26 . . PUNCT fcis-32134 56 1 3.2 3.2 NUM fcis-32134 56 2 . . PUNCT fcis-32134 56 3 strategy strategy NOUN fcis-32134 56 4 network network NOUN fcis-32134 56 5 design design VERB fcis-32134 56 6 the the DET fcis-32134 56 7 input input NOUN fcis-32134 56 8 dimension dimension NOUN fcis-32134 56 9 of of ADP fcis-32134 56 10 the the DET fcis-32134 56 11 strategy strategy NOUN fcis-32134 56 12 network network NOUN fcis-32134 56 13 is be AUX fcis-32134 56 14 the the DET fcis-32134 56 15 same same ADJ fcis-32134 56 16 as as ADP fcis-32134 56 17 the the DET fcis-32134 56 18 state state NOUN fcis-32134 56 19 dimension dimension NOUN fcis-32134 56 20 , , PUNCT fcis-32134 56 21 and and CCONJ fcis-32134 56 22 the the DET fcis-32134 56 23 hidden hide VERB fcis-32134 56 24 layer layer NOUN fcis-32134 56 25 adopts adopt VERB fcis-32134 56 26 two two NUM fcis-32134 56 27 layers layer NOUN fcis-32134 56 28 of of ADP fcis-32134 56 29 full full ADJ fcis-32134 56 30 connectivity connectivity NOUN fcis-32134 56 31 ( ( PUNCT fcis-32134 56 32 256 256 NUM fcis-32134 56 33 neurons neuron NOUN fcis-32134 56 34 per per ADP fcis-32134 56 35 layer layer NOUN fcis-32134 56 36 ) ) PUNCT fcis-32134 56 37 , , PUNCT fcis-32134 56 38 processing process VERB fcis-32134 56 39 the the DET fcis-32134 56 40 input input NOUN fcis-32134 56 41 state state NOUN fcis-32134 56 42 vector vector NOUN fcis-32134 56 43 through through ADP fcis-32134 56 44 the the DET fcis-32134 56 45 relu relu NOUN fcis-32134 56 46 activation activation NOUN fcis-32134 56 47 function function NOUN fcis-32134 56 48 . . PUNCT fcis-32134 57 1 the the DET fcis-32134 57 2 output output NOUN fcis-32134 57 3 layer layer NOUN fcis-32134 57 4 adopts adopt VERB fcis-32134 57 5 a a DET fcis-32134 57 6 two two NUM fcis-32134 57 7 - - PUNCT fcis-32134 57 8 branch branch NOUN fcis-32134 57 9 output output NOUN fcis-32134 57 10 structure structure NOUN fcis-32134 57 11 : : PUNCT fcis-32134 57 12 one one NUM fcis-32134 57 13 branch branch NOUN fcis-32134 57 14 generates generate VERB fcis-32134 57 15 the the DET fcis-32134 57 16 action action NOUN fcis-32134 57 17 meanπœ‡ meanπœ‡ NOUN fcis-32134 57 18 , , PUNCT fcis-32134 57 19 and and CCONJ fcis-32134 57 20 the the DET fcis-32134 57 21 other other ADJ fcis-32134 57 22 branch branch NOUN fcis-32134 57 23 generates generate VERB fcis-32134 57 24 the the DET fcis-32134 57 25 bounded bounded ADJ fcis-32134 57 26 logarithmic logarithmic ADJ fcis-32134 57 27 standard standard ADJ fcis-32134 57 28 deviation deviation NOUN fcis-32134 57 29 log log VERB fcis-32134 57 30 𝜎 𝜎 PROPN fcis-32134 57 31 ∈ ∈ PROPN fcis-32134 57 32 20,2 20,2 NUM fcis-32134 57 33 , , PUNCT fcis-32134 57 34 which which PRON fcis-32134 57 35 constructs construct VERB fcis-32134 57 36 the the DET fcis-32134 57 37 action action NOUN fcis-32134 57 38 probability probability NOUN fcis-32134 57 39 distribution distribution NOUN fcis-32134 57 40 . . PUNCT fcis-32134 58 1 a a DET fcis-32134 58 2 stochastic stochastic ADJ fcis-32134 58 3 strategy strategy NOUN fcis-32134 58 4 is be AUX fcis-32134 58 5 used use VERB fcis-32134 58 6 in in ADP fcis-32134 58 7 the the DET fcis-32134 58 8 action action NOUN fcis-32134 58 9 sampling sample VERB fcis-32134 58 10 phase phase NOUN fcis-32134 58 11 : : PUNCT fcis-32134 58 12 sampling sample VERB fcis-32134 58 13 from from ADP fcis-32134 58 14 a a DET fcis-32134 58 15 gaussian gaussian ADJ fcis-32134 58 16 distribution distribution NOUN fcis-32134 58 17 is be AUX fcis-32134 58 18 processed process VERB fcis-32134 58 19 by by ADP fcis-32134 58 20 tanh tanh PROPN fcis-32134 58 21 transformation transformation NOUN fcis-32134 58 22 and and CCONJ fcis-32134 58 23 physical physical ADJ fcis-32134 58 24 constraints constraint NOUN fcis-32134 58 25 , , PUNCT fcis-32134 58 26 and and CCONJ fcis-32134 58 27 the the DET fcis-32134 58 28 modified modify VERB fcis-32134 58 29 log log NOUN fcis-32134 58 30 probability probability NOUN fcis-32134 58 31 is be AUX fcis-32134 58 32 computed compute VERB fcis-32134 58 33 to to PART fcis-32134 58 34 compensate compensate VERB fcis-32134 58 35 for for ADP fcis-32134 58 36 the the DET fcis-32134 58 37 change change NOUN fcis-32134 58 38 in in ADP fcis-32134 58 39 probability probability NOUN fcis-32134 58 40 density density NOUN fcis-32134 58 41 caused cause VERB fcis-32134 58 42 by by ADP fcis-32134 58 43 the the DET fcis-32134 58 44 tanh tanh PROPN fcis-32134 58 45 transformation transformation NOUN fcis-32134 58 46 . . PUNCT fcis-32134 59 1 meanwhile meanwhile ADV fcis-32134 59 2 , , PUNCT fcis-32134 59 3 the the DET fcis-32134 59 4 strategy strategy NOUN fcis-32134 59 5 network network NOUN fcis-32134 59 6 integrates integrate VERB fcis-32134 59 7 a a DET fcis-32134 59 8 safety safety NOUN fcis-32134 59 9 mechanism mechanism NOUN fcis-32134 59 10 : : PUNCT fcis-32134 59 11 when when SCONJ fcis-32134 59 12 the the DET fcis-32134 59 13 distance distance NOUN fcis-32134 59 14 to to ADP fcis-32134 59 15 the the DET fcis-32134 59 16 obstacle obstacle NOUN fcis-32134 59 17 is be AUX fcis-32134 59 18 detected detect VERB fcis-32134 59 19 to to PART fcis-32134 59 20 be be AUX fcis-32134 59 21 less less ADJ fcis-32134 59 22 than than ADP fcis-32134 59 23 80 80 NUM fcis-32134 59 24 % % NOUN fcis-32134 59 25 of of ADP fcis-32134 59 26 the the DET fcis-32134 59 27 safety safety NOUN fcis-32134 59 28 threshold threshold NOUN fcis-32134 59 29 , , PUNCT fcis-32134 59 30 it it PRON fcis-32134 59 31 immediately immediately ADV fcis-32134 59 32 45 45 NUM fcis-32134 59 33 switches switch NOUN fcis-32134 59 34 to to ADP fcis-32134 59 35 a a DET fcis-32134 59 36 preset preset ADJ fcis-32134 59 37 emergency emergency NOUN fcis-32134 59 38 obstacle obstacle NOUN fcis-32134 59 39 avoidance avoidance NOUN fcis-32134 59 40 action action NOUN fcis-32134 59 41 library library NOUN fcis-32134 59 42 . . PUNCT fcis-32134 60 1 this this DET fcis-32134 60 2 action action NOUN fcis-32134 60 3 library library NOUN fcis-32134 60 4 contains contain VERB fcis-32134 60 5 five five NUM fcis-32134 60 6 sets set NOUN fcis-32134 60 7 of of ADP fcis-32134 60 8 evasive evasive ADJ fcis-32134 60 9 actions action NOUN fcis-32134 60 10 ( ( PUNCT fcis-32134 60 11 e.g. e.g. ADV fcis-32134 60 12 , , PUNCT fcis-32134 60 13 left leave VERB fcis-32134 60 14 turn turn NOUN fcis-32134 60 15 , , PUNCT fcis-32134 60 16 right right ADJ fcis-32134 60 17 turn turn NOUN fcis-32134 60 18 , , PUNCT fcis-32134 60 19 dive dive NOUN fcis-32134 60 20 , , PUNCT fcis-32134 60 21 tilt tilt NOUN fcis-32134 60 22 , , PUNCT fcis-32134 60 23 and and CCONJ fcis-32134 60 24 climb climb VERB fcis-32134 60 25 ) ) PUNCT fcis-32134 60 26 for for ADP fcis-32134 60 27 obstacles obstacle NOUN fcis-32134 60 28 in in ADP fcis-32134 60 29 different different ADJ fcis-32134 60 30 directions direction NOUN fcis-32134 60 31 , , PUNCT fcis-32134 60 32 which which PRON fcis-32134 60 33 achieve achieve VERB fcis-32134 60 34 rapid rapid ADJ fcis-32134 60 35 extrication extrication NOUN fcis-32134 60 36 through through ADP fcis-32134 60 37 thrust thrust VERB fcis-32134 60 38 reinforcement reinforcement NOUN fcis-32134 60 39 . . PUNCT fcis-32134 61 1 in in ADP fcis-32134 61 2 addition addition NOUN fcis-32134 61 3 , , PUNCT fcis-32134 61 4 this this DET fcis-32134 61 5 experiment experiment NOUN fcis-32134 61 6 introduces introduce VERB fcis-32134 61 7 an an DET fcis-32134 61 8 exploration exploration NOUN fcis-32134 61 9 strategy strategy NOUN fcis-32134 61 10 in in ADP fcis-32134 61 11 the the DET fcis-32134 61 12 normal normal ADJ fcis-32134 61 13 decision decision NOUN fcis-32134 61 14 - - PUNCT fcis-32134 61 15 making make VERB fcis-32134 61 16 process process NOUN fcis-32134 61 17 : : PUNCT fcis-32134 61 18 10 10 NUM fcis-32134 61 19 % % NOUN fcis-32134 61 20 probability probability NOUN fcis-32134 61 21 of of ADP fcis-32134 61 22 completely completely ADV fcis-32134 61 23 randomized randomized ADJ fcis-32134 61 24 actions action NOUN fcis-32134 61 25 ( ( PUNCT fcis-32134 61 26 πœ€-greedy πœ€-greedy PROPN fcis-32134 61 27 ) ) PUNCT fcis-32134 61 28 , , PUNCT fcis-32134 61 29 and and CCONJ fcis-32134 61 30 the the DET fcis-32134 61 31 remaining remain VERB fcis-32134 61 32 90 90 NUM fcis-32134 61 33 % % NOUN fcis-32134 61 34 adds add VERB fcis-32134 61 35 attenuated attenuate VERB fcis-32134 61 36 gaussian gaussian ADJ fcis-32134 61 37 noise noise NOUN fcis-32134 61 38 ( ( PUNCT fcis-32134 61 39 noisy noisy ADV fcis-32134 61 40 at at ADP fcis-32134 61 41 the the DET fcis-32134 61 42 early early ADJ fcis-32134 61 43 stage stage NOUN fcis-32134 61 44 of of ADP fcis-32134 61 45 training training NOUN fcis-32134 61 46 and and CCONJ fcis-32134 61 47 gradually gradually ADV fcis-32134 61 48 reduced reduce VERB fcis-32134 61 49 at at ADP fcis-32134 61 50 the the DET fcis-32134 61 51 later later ADJ fcis-32134 61 52 stage stage NOUN fcis-32134 61 53 ) ) PUNCT fcis-32134 61 54 to to ADP fcis-32134 61 55 the the DET fcis-32134 61 56 network network NOUN fcis-32134 61 57 output output NOUN fcis-32134 61 58 actions action NOUN fcis-32134 61 59 , , PUNCT fcis-32134 61 60 thus thus ADV fcis-32134 61 61 helping help VERB fcis-32134 61 62 the the DET fcis-32134 61 63 intelligent intelligent ADJ fcis-32134 61 64 body body NOUN fcis-32134 61 65 to to PART fcis-32134 61 66 find find VERB fcis-32134 61 67 the the DET fcis-32134 61 68 globally globally ADV fcis-32134 61 69 optimal optimal ADJ fcis-32134 61 70 strategy strategy NOUN fcis-32134 61 71 in in ADP fcis-32134 61 72 a a DET fcis-32134 61 73 complex complex ADJ fcis-32134 61 74 environment environment NOUN fcis-32134 61 75 , , PUNCT fcis-32134 61 76 and and CCONJ fcis-32134 61 77 avoiding avoid VERB fcis-32134 61 78 falling fall VERB fcis-32134 61 79 into into ADP fcis-32134 61 80 the the DET fcis-32134 61 81 local local ADJ fcis-32134 61 82 optimum optimum NOUN fcis-32134 61 83 at at ADP fcis-32134 61 84 an an DET fcis-32134 61 85 early early ADJ fcis-32134 61 86 stage stage NOUN fcis-32134 61 87 . . PUNCT fcis-32134 62 1 on on ADP fcis-32134 62 2 this this DET fcis-32134 62 3 basis basis NOUN fcis-32134 62 4 , , PUNCT fcis-32134 62 5 this this DET fcis-32134 62 6 paper paper NOUN fcis-32134 62 7 utilizes utilize VERB fcis-32134 62 8 the the DET fcis-32134 62 9 lambdal lambdal NOUN fcis-32134 62 10 r r NOUN fcis-32134 62 11 scheduler scheduler NOUN fcis-32134 62 12 to to PART fcis-32134 62 13 achieve achieve VERB fcis-32134 62 14 dynamic dynamic ADJ fcis-32134 62 15 adjustment adjustment NOUN fcis-32134 62 16 of of ADP fcis-32134 62 17 the the DET fcis-32134 62 18 learning learning NOUN fcis-32134 62 19 rate rate NOUN fcis-32134 62 20 : : PUNCT fcis-32134 62 21 by by ADP fcis-32134 62 22 fast fast ADJ fcis-32134 62 23 learning learning NOUN fcis-32134 62 24 in in ADP fcis-32134 62 25 the the DET fcis-32134 62 26 early early ADJ fcis-32134 62 27 stage stage NOUN fcis-32134 62 28 and and CCONJ fcis-32134 62 29 fine fine ADJ fcis-32134 62 30 tuning tuning NOUN fcis-32134 62 31 in in ADP fcis-32134 62 32 the the DET fcis-32134 62 33 later later ADJ fcis-32134 62 34 stage stage NOUN fcis-32134 62 35 , , PUNCT fcis-32134 62 36 the the DET fcis-32134 62 37 strategy strategy NOUN fcis-32134 62 38 is be AUX fcis-32134 62 39 refined refine VERB fcis-32134 62 40 to to PART fcis-32134 62 41 improve improve VERB fcis-32134 62 42 the the DET fcis-32134 62 43 training training NOUN fcis-32134 62 44 stability stability NOUN fcis-32134 62 45 and and CCONJ fcis-32134 62 46 final final ADJ fcis-32134 62 47 performance performance NOUN fcis-32134 62 48 . . PUNCT fcis-32134 63 1 this this DET fcis-32134 63 2 architecture architecture NOUN fcis-32134 63 3 enables enable VERB fcis-32134 63 4 uavs uavs NOUN fcis-32134 63 5 to to PART fcis-32134 63 6 realize realize VERB fcis-32134 63 7 safe safe ADJ fcis-32134 63 8 navigation navigation NOUN fcis-32134 63 9 in in ADP fcis-32134 63 10 dynamic dynamic ADJ fcis-32134 63 11 obstacle obstacle NOUN fcis-32134 63 12 environments environment NOUN fcis-32134 63 13 while while SCONJ fcis-32134 63 14 obeying obey VERB fcis-32134 63 15 the the DET fcis-32134 63 16 laws law NOUN fcis-32134 63 17 of of ADP fcis-32134 63 18 physics physics NOUN fcis-32134 63 19 . . PUNCT fcis-32134 64 1 3.3 3.3 NUM fcis-32134 64 2 . . PUNCT fcis-32134 65 1 state state NOUN fcis-32134 65 2 space space NOUN fcis-32134 65 3 design design NOUN fcis-32134 65 4 in in ADP fcis-32134 65 5 this this DET fcis-32134 65 6 paper paper NOUN fcis-32134 65 7 , , PUNCT fcis-32134 65 8 a a DET fcis-32134 65 9 15 15 NUM fcis-32134 65 10 - - PUNCT fcis-32134 65 11 dimensional dimensional ADJ fcis-32134 65 12 state state NOUN fcis-32134 65 13 space space NOUN fcis-32134 65 14 is be AUX fcis-32134 65 15 designed design VERB fcis-32134 65 16 , , PUNCT fcis-32134 65 17 which which PRON fcis-32134 65 18 consists consist VERB fcis-32134 65 19 of of ADP fcis-32134 65 20 three three NUM fcis-32134 65 21 parts part NOUN fcis-32134 65 22 : : PUNCT fcis-32134 65 23 the the DET fcis-32134 65 24 first first ADJ fcis-32134 65 25 part part NOUN fcis-32134 65 26 is be AUX fcis-32134 65 27 the the DET fcis-32134 65 28 uav uav PROPN fcis-32134 65 29 's 's PART fcis-32134 65 30 own own ADJ fcis-32134 65 31 normalized normalize VERB fcis-32134 65 32 3d 3d NUM fcis-32134 65 33 position position NOUN fcis-32134 65 34 x x NOUN fcis-32134 65 35 , , PUNCT fcis-32134 65 36 y y PROPN fcis-32134 65 37 , , PUNCT fcis-32134 65 38 z z X fcis-32134 65 39 , , PUNCT fcis-32134 65 40 the the DET fcis-32134 65 41 second second ADJ fcis-32134 65 42 part part NOUN fcis-32134 65 43 is be AUX fcis-32134 65 44 the the DET fcis-32134 65 45 uav uav PROPN fcis-32134 65 46 's 's PART fcis-32134 65 47 own own ADJ fcis-32134 65 48 normalized normalize VERB fcis-32134 65 49 3d 3d NUM fcis-32134 65 50 velocity velocity NOUN fcis-32134 65 51 v v NOUN fcis-32134 65 52 , , PUNCT fcis-32134 65 53 v v INTJ fcis-32134 65 54 , , PUNCT fcis-32134 65 55 v v NOUN fcis-32134 65 56 , , PUNCT fcis-32134 65 57 and and CCONJ fcis-32134 65 58 the the DET fcis-32134 65 59 third third ADJ fcis-32134 65 60 part part NOUN fcis-32134 65 61 is be AUX fcis-32134 65 62 the the DET fcis-32134 65 63 relative relative ADJ fcis-32134 65 64 positions position NOUN fcis-32134 65 65 of of ADP fcis-32134 65 66 the the DET fcis-32134 65 67 three three NUM fcis-32134 65 68 nearest near ADJ fcis-32134 65 69 obstacles obstacle NOUN fcis-32134 65 70 , , PUNCT fcis-32134 65 71 each each PRON fcis-32134 65 72 contributing contribute VERB fcis-32134 65 73 three three NUM fcis-32134 65 74 dimensions dimension NOUN fcis-32134 65 75 , , PUNCT fcis-32134 65 76 for for ADP fcis-32134 65 77 a a DET fcis-32134 65 78 total total NOUN fcis-32134 65 79 of of ADP fcis-32134 65 80 9 9 NUM fcis-32134 65 81 dimensions dimension NOUN fcis-32134 65 82 . . PUNCT fcis-32134 66 1 based base VERB fcis-32134 66 2 on on ADP fcis-32134 66 3 the the DET fcis-32134 66 4 working working ADJ fcis-32134 66 5 principle principle NOUN fcis-32134 66 6 of of ADP fcis-32134 66 7 analog analog NOUN fcis-32134 66 8 sensors sensor NOUN fcis-32134 66 9 , , PUNCT fcis-32134 66 10 this this DET fcis-32134 66 11 paper paper NOUN fcis-32134 66 12 adopts adopt VERB fcis-32134 66 13 the the DET fcis-32134 66 14 design design NOUN fcis-32134 66 15 of of ADP fcis-32134 66 16 prioritizing prioritize VERB fcis-32134 66 17 just just ADV fcis-32134 66 18 - - PUNCT fcis-32134 66 19 in in ADP fcis-32134 66 20 obstacles obstacle NOUN fcis-32134 66 21 , , PUNCT fcis-32134 66 22 while while SCONJ fcis-32134 66 23 reducing reduce VERB fcis-32134 66 24 the the DET fcis-32134 66 25 computational computational ADJ fcis-32134 66 26 complexity complexity NOUN fcis-32134 66 27 and and CCONJ fcis-32134 66 28 ensuring ensure VERB fcis-32134 66 29 the the DET fcis-32134 66 30 stability stability NOUN fcis-32134 66 31 of of ADP fcis-32134 66 32 the the DET fcis-32134 66 33 neural neural ADJ fcis-32134 66 34 network network NOUN fcis-32134 66 35 structure structure NOUN fcis-32134 66 36 . . PUNCT fcis-32134 67 1 3.4 3.4 NUM fcis-32134 67 2 . . PUNCT fcis-32134 67 3 action action NOUN fcis-32134 67 4 space space NOUN fcis-32134 67 5 design design NOUN fcis-32134 67 6 in in ADP fcis-32134 67 7 order order NOUN fcis-32134 67 8 to to PART fcis-32134 67 9 ensure ensure VERB fcis-32134 67 10 the the DET fcis-32134 67 11 maneuverability maneuverability NOUN fcis-32134 67 12 of of ADP fcis-32134 67 13 the the DET fcis-32134 67 14 uav uav PROPN fcis-32134 67 15 in in ADP fcis-32134 67 16 complex complex ADJ fcis-32134 67 17 environments environment NOUN fcis-32134 67 18 , , PUNCT fcis-32134 67 19 this this DET fcis-32134 67 20 paper paper NOUN fcis-32134 67 21 designs design VERB fcis-32134 67 22 the the DET fcis-32134 67 23 continuous continuous ADJ fcis-32134 67 24 action action NOUN fcis-32134 67 25 space space NOUN fcis-32134 67 26 to to PART fcis-32134 67 27 meet meet VERB fcis-32134 67 28 the the DET fcis-32134 67 29 demand demand NOUN fcis-32134 67 30 . . PUNCT fcis-32134 68 1 as as SCONJ fcis-32134 68 2 shown show VERB fcis-32134 68 3 in in ADP fcis-32134 68 4 table table NOUN fcis-32134 68 5 1 1 NUM fcis-32134 68 6 , , PUNCT fcis-32134 68 7 the the DET fcis-32134 68 8 three three NUM fcis-32134 68 9 - - PUNCT fcis-32134 68 10 axis axis NOUN fcis-32134 68 11 continuous continuous ADJ fcis-32134 68 12 control control NOUN fcis-32134 68 13 volume volume NOUN fcis-32134 68 14 is be AUX fcis-32134 68 15 mapped map VERB fcis-32134 68 16 to to ADP fcis-32134 68 17 the the DET fcis-32134 68 18 threedimensional threedimensional ADJ fcis-32134 68 19 continuous continuous ADJ fcis-32134 68 20 space space NOUN fcis-32134 68 21 , , PUNCT fcis-32134 68 22 which which PRON fcis-32134 68 23 simulates simulate VERB fcis-32134 68 24 the the DET fcis-32134 68 25 maneuverability maneuverability NOUN fcis-32134 68 26 of of ADP fcis-32134 68 27 the the DET fcis-32134 68 28 uav uav PROPN fcis-32134 68 29 in in ADP fcis-32134 68 30 the the DET fcis-32134 68 31 real real ADJ fcis-32134 68 32 environment environment NOUN fcis-32134 68 33 . . PUNCT fcis-32134 69 1 table table NOUN fcis-32134 69 2 1 1 NUM fcis-32134 69 3 . . PUNCT fcis-32134 70 1 action action NOUN fcis-32134 70 2 space space NOUN fcis-32134 70 3 table table NOUN fcis-32134 70 4 maneuver maneuver NOUN fcis-32134 70 5 range range NOUN fcis-32134 70 6 of of ADP fcis-32134 70 7 maneuver maneuver NOUN fcis-32134 70 8 description description NOUN fcis-32134 70 9 vx vx INTJ fcis-32134 70 10 -1.5m -1.5m NOUN fcis-32134 70 11 / / SYM fcis-32134 70 12 s,1.5 s,1.5 PROPN fcis-32134 70 13 m/s m/s PROPN fcis-32134 70 14 x x PUNCT fcis-32134 70 15 velocity velocity NOUN fcis-32134 70 16 vy vy X fcis-32134 70 17 -1.5m -1.5m PROPN fcis-32134 70 18 / / SYM fcis-32134 70 19 s,1.5 s,1.5 PROPN fcis-32134 70 20 m/s m/s PROPN fcis-32134 70 21 y y PROPN fcis-32134 70 22 velocity velocity NOUN fcis-32134 70 23 vz vz INTJ fcis-32134 70 24 0.0m 0.0m PROPN fcis-32134 70 25 / / SYM fcis-32134 70 26 s,1.5 s,1.5 PROPN fcis-32134 70 27 m/s m/s PROPN fcis-32134 70 28 z z PROPN fcis-32134 70 29 velocity velocity NOUN fcis-32134 70 30 3.5 3.5 NUM fcis-32134 70 31 . . PUNCT fcis-32134 71 1 reward reward NOUN fcis-32134 71 2 function function NOUN fcis-32134 71 3 design design VERB fcis-32134 71 4 the the DET fcis-32134 71 5 reward reward NOUN fcis-32134 71 6 function function NOUN fcis-32134 71 7 is be AUX fcis-32134 71 8 a a DET fcis-32134 71 9 core core NOUN fcis-32134 71 10 element element NOUN fcis-32134 71 11 in in ADP fcis-32134 71 12 defining define VERB fcis-32134 71 13 environmental environmental ADJ fcis-32134 71 14 feedback feedback NOUN fcis-32134 71 15 in in ADP fcis-32134 71 16 reinforcement reinforcement NOUN fcis-32134 71 17 learning learning NOUN fcis-32134 71 18 , , PUNCT fcis-32134 71 19 which which PRON fcis-32134 71 20 directly directly ADV fcis-32134 71 21 affects affect VERB fcis-32134 71 22 the the DET fcis-32134 71 23 performance performance NOUN fcis-32134 71 24 of of ADP fcis-32134 71 25 learning learn VERB fcis-32134 71 26 algorithms algorithm NOUN fcis-32134 71 27 . . PUNCT fcis-32134 72 1 the the DET fcis-32134 72 2 objective objective NOUN fcis-32134 72 3 of of ADP fcis-32134 72 4 this this DET fcis-32134 72 5 study study NOUN fcis-32134 72 6 is be AUX fcis-32134 72 7 to to PART fcis-32134 72 8 enable enable VERB fcis-32134 72 9 the the DET fcis-32134 72 10 uav uav PROPN fcis-32134 72 11 to to PART fcis-32134 72 12 avoid avoid VERB fcis-32134 72 13 various various ADJ fcis-32134 72 14 static static ADJ fcis-32134 72 15 and and CCONJ fcis-32134 72 16 dynamic dynamic ADJ fcis-32134 72 17 obstacles obstacle NOUN fcis-32134 72 18 and and CCONJ fcis-32134 72 19 reach reach VERB fcis-32134 72 20 the the DET fcis-32134 72 21 end end NOUN fcis-32134 72 22 point point NOUN fcis-32134 72 23 safely safely ADV fcis-32134 72 24 in in ADP fcis-32134 72 25 a a DET fcis-32134 72 26 complex complex ADJ fcis-32134 72 27 environment environment NOUN fcis-32134 72 28 . . PUNCT fcis-32134 73 1 based base VERB fcis-32134 73 2 on on ADP fcis-32134 73 3 the the DET fcis-32134 73 4 above above ADJ fcis-32134 73 5 objectives objective NOUN fcis-32134 73 6 , , PUNCT fcis-32134 73 7 this this DET fcis-32134 73 8 paper paper NOUN fcis-32134 73 9 designs design VERB fcis-32134 73 10 a a DET fcis-32134 73 11 non non ADJ fcis-32134 73 12 - - ADJ fcis-32134 73 13 sparse sparse ADJ fcis-32134 73 14 reward reward NOUN fcis-32134 73 15 function function NOUN fcis-32134 73 16 to to PART fcis-32134 73 17 form form VERB fcis-32134 73 18 a a DET fcis-32134 73 19 safe safe ADJ fcis-32134 73 20 navigation navigation NOUN fcis-32134 73 21 strategy strategy NOUN fcis-32134 73 22 through through ADP fcis-32134 73 23 progressive progressive ADJ fcis-32134 73 24 reward reward NOUN fcis-32134 73 25 gradient gradient NOUN fcis-32134 73 26 and and CCONJ fcis-32134 73 27 asymmetric asymmetric ADJ fcis-32134 73 28 punishment punishment NOUN fcis-32134 73 29 mechanism mechanism NOUN fcis-32134 73 30 , , PUNCT fcis-32134 73 31 and and CCONJ fcis-32134 73 32 at at ADP fcis-32134 73 33 the the DET fcis-32134 73 34 same same ADJ fcis-32134 73 35 time time NOUN fcis-32134 73 36 with with ADP fcis-32134 73 37 stagnation stagnation NOUN fcis-32134 73 38 detection detection NOUN fcis-32134 73 39 to to PART fcis-32134 73 40 prevent prevent VERB fcis-32134 73 41 the the DET fcis-32134 73 42 training training NOUN fcis-32134 73 43 from from ADP fcis-32134 73 44 falling fall VERB fcis-32134 73 45 into into ADP fcis-32134 73 46 a a DET fcis-32134 73 47 local local ADJ fcis-32134 73 48 optimum optimum NOUN fcis-32134 73 49 , , PUNCT fcis-32134 73 50 to to PART fcis-32134 73 51 guide guide VERB fcis-32134 73 52 the the DET fcis-32134 73 53 uav uav PROPN fcis-32134 73 54 to to PART fcis-32134 73 55 explore explore VERB fcis-32134 73 56 the the DET fcis-32134 73 57 environment environment NOUN fcis-32134 73 58 more more ADV fcis-32134 73 59 efficiently efficiently ADV fcis-32134 73 60 and and CCONJ fcis-32134 73 61 to to PART fcis-32134 73 62 improve improve VERB fcis-32134 73 63 the the DET fcis-32134 73 64 convergence convergence NOUN fcis-32134 73 65 . . PUNCT fcis-32134 74 1 the the DET fcis-32134 74 2 specific specific ADJ fcis-32134 74 3 design design NOUN fcis-32134 74 4 is be AUX fcis-32134 74 5 as as SCONJ fcis-32134 74 6 follows follow VERB fcis-32134 74 7 : : PUNCT fcis-32134 74 8 ( ( PUNCT fcis-32134 74 9 1 1 X fcis-32134 74 10 ) ) PUNCT fcis-32134 74 11 base base NOUN fcis-32134 74 12 navigation navigation NOUN fcis-32134 74 13 reward reward NOUN fcis-32134 74 14 to to PART fcis-32134 74 15 guide guide VERB fcis-32134 74 16 uav uav PROPN fcis-32134 74 17 navigation navigation NOUN fcis-32134 74 18 toward toward ADP fcis-32134 74 19 targets target NOUN fcis-32134 74 20 , , PUNCT fcis-32134 74 21 the the DET fcis-32134 74 22 reward reward NOUN fcis-32134 74 23 scales scale VERB fcis-32134 74 24 inversely inversely ADV fcis-32134 74 25 with with ADP fcis-32134 74 26 distance distance NOUN fcis-32134 74 27 : : PUNCT fcis-32134 74 28 positive positive ADJ fcis-32134 74 29 incentives incentive NOUN fcis-32134 74 30 when when SCONJ fcis-32134 74 31 approaching approach VERB fcis-32134 74 32 the the DET fcis-32134 74 33 goal goal NOUN fcis-32134 74 34 and and CCONJ fcis-32134 74 35 negative negative ADJ fcis-32134 74 36 penalties penalty NOUN fcis-32134 74 37 when when SCONJ fcis-32134 74 38 moving move VERB fcis-32134 74 39 away away ADV fcis-32134 74 40 , , PUNCT fcis-32134 74 41 i.e. i.e. X fcis-32134 74 42 , , PUNCT fcis-32134 74 43 the the DET fcis-32134 74 44 uav uav PROPN fcis-32134 74 45 is be AUX fcis-32134 74 46 incentivized incentivize VERB fcis-32134 74 47 to to PART fcis-32134 74 48 move move VERB fcis-32134 74 49 towards towards ADP fcis-32134 74 50 the the DET fcis-32134 74 51 target target NOUN fcis-32134 74 52 point point NOUN fcis-32134 74 53 by by ADP fcis-32134 74 54 successive successive ADJ fcis-32134 74 55 distance distance NOUN fcis-32134 74 56 differences difference NOUN fcis-32134 74 57 , , PUNCT fcis-32134 74 58 so so SCONJ fcis-32134 74 59 the the DET fcis-32134 74 60 distance distance NOUN fcis-32134 74 61 reward reward NOUN fcis-32134 74 62 formula formula NOUN fcis-32134 74 63 is be AUX fcis-32134 74 64 : : PUNCT fcis-32134 74 65 𝑅 𝑅 PROPN fcis-32134 74 66 𝑑 𝑑 PROPN fcis-32134 74 67 𝑑 𝑑 PROPN fcis-32134 74 68 ( ( PUNCT fcis-32134 74 69 8) 8) NUM fcis-32134 74 70 where where SCONJ fcis-32134 74 71 𝑑 𝑑 PROPN fcis-32134 74 72 and and CCONJ fcis-32134 74 73 𝑑 𝑑 PROPN fcis-32134 74 74 denote denote VERB fcis-32134 74 75 the the DET fcis-32134 74 76 euclidean euclidean ADJ fcis-32134 74 77 distance distance NOUN fcis-32134 74 78 to to ADP fcis-32134 74 79 the the DET fcis-32134 74 80 target target NOUN fcis-32134 74 81 at at ADP fcis-32134 74 82 the the DET fcis-32134 74 83 previous previous ADJ fcis-32134 74 84 and and CCONJ fcis-32134 74 85 current current ADJ fcis-32134 74 86 moment moment NOUN fcis-32134 74 87 respectively respectively ADV fcis-32134 74 88 . . PUNCT fcis-32134 75 1 ( ( PUNCT fcis-32134 75 2 2 2 X fcis-32134 75 3 ) ) PUNCT fcis-32134 75 4 target target NOUN fcis-32134 75 5 area area NOUN fcis-32134 75 6 enhancement enhancement NOUN fcis-32134 75 7 bonus bonus NOUN fcis-32134 75 8 the the DET fcis-32134 75 9 uav uav PROPN fcis-32134 75 10 tends tend VERB fcis-32134 75 11 to to PART fcis-32134 75 12 rotate rotate VERB fcis-32134 75 13 around around ADP fcis-32134 75 14 the the DET fcis-32134 75 15 target target NOUN fcis-32134 75 16 point point NOUN fcis-32134 75 17 as as SCONJ fcis-32134 75 18 it it PRON fcis-32134 75 19 approaches approach VERB fcis-32134 75 20 it it PRON fcis-32134 75 21 instead instead ADV fcis-32134 75 22 of of ADP fcis-32134 75 23 reaching reach VERB fcis-32134 75 24 the the DET fcis-32134 75 25 end end NOUN fcis-32134 75 26 point point NOUN fcis-32134 75 27 directly directly ADV fcis-32134 75 28 , , PUNCT fcis-32134 75 29 so so CCONJ fcis-32134 75 30 it it PRON fcis-32134 75 31 needs need VERB fcis-32134 75 32 to to PART fcis-32134 75 33 be be AUX fcis-32134 75 34 given give VERB fcis-32134 75 35 an an DET fcis-32134 75 36 additional additional ADJ fcis-32134 75 37 gradient gradient ADJ fcis-32134 75 38 reward reward NOUN fcis-32134 75 39 to to PART fcis-32134 75 40 promote promote VERB fcis-32134 75 41 accurate accurate ADJ fcis-32134 75 42 navigation navigation NOUN fcis-32134 75 43 . . PUNCT fcis-32134 76 1 specifically specifically ADV fcis-32134 76 2 : : PUNCT fcis-32134 76 3 𝑅 𝑅 NOUN fcis-32134 76 4 10 10 NUM fcis-32134 76 5 3 3 NUM fcis-32134 76 6 𝑑 𝑑 PROPN fcis-32134 76 7 𝑑 𝑑 PROPN fcis-32134 76 8 3 3 NUM fcis-32134 76 9 20 20 NUM fcis-32134 76 10 2 2 NUM fcis-32134 76 11 𝑑 𝑑 NOUN fcis-32134 76 12 𝑑 𝑑 PROPN fcis-32134 76 13 2 2 NUM fcis-32134 76 14 200 200 NUM fcis-32134 76 15 π‘‡π‘’π‘Ÿπ‘šπ‘–π‘›π‘Žπ‘‘π‘’ π‘‡π‘’π‘Ÿπ‘šπ‘–π‘›π‘Žπ‘‘π‘’ PROPN fcis-32134 76 16 π‘Ÿπ‘œπ‘’π‘›π‘‘ π‘Ÿπ‘œπ‘’π‘›π‘‘ NOUN fcis-32134 76 17 𝑑 𝑑 NOUN fcis-32134 76 18 1 1 NUM fcis-32134 76 19 ( ( PUNCT fcis-32134 76 20 9 9 NUM fcis-32134 76 21 ) ) PUNCT fcis-32134 76 22 at at ADP fcis-32134 76 23 𝑑 𝑑 PROPN fcis-32134 76 24 3 3 NUM fcis-32134 76 25 , , PUNCT fcis-32134 76 26 a a DET fcis-32134 76 27 target target NOUN fcis-32134 76 28 attraction attraction NOUN fcis-32134 76 29 zone zone NOUN fcis-32134 76 30 is be AUX fcis-32134 76 31 established establish VERB fcis-32134 76 32 to to PART fcis-32134 76 33 attract attract VERB fcis-32134 76 34 the the DET fcis-32134 76 35 drone drone NOUN fcis-32134 76 36 towards towards ADP fcis-32134 76 37 the the DET fcis-32134 76 38 target target NOUN fcis-32134 76 39 point point NOUN fcis-32134 76 40 ; ; PUNCT fcis-32134 76 41 at at ADP fcis-32134 76 42 𝑑 𝑑 PROPN fcis-32134 76 43 2 2 NUM fcis-32134 76 44 , , PUNCT fcis-32134 76 45 the the DET fcis-32134 76 46 reward reward NOUN fcis-32134 76 47 strength strength NOUN fcis-32134 76 48 is be AUX fcis-32134 76 49 increased increase VERB fcis-32134 76 50 to to PART fcis-32134 76 51 reinforce reinforce VERB fcis-32134 76 52 end end NOUN fcis-32134 76 53 guidance guidance NOUN fcis-32134 76 54 so so SCONJ fcis-32134 76 55 that that SCONJ fcis-32134 76 56 the the DET fcis-32134 76 57 drone drone NOUN fcis-32134 76 58 reaches reach VERB fcis-32134 76 59 the the DET fcis-32134 76 60 end end NOUN fcis-32134 76 61 point point NOUN fcis-32134 76 62 more more ADV fcis-32134 76 63 quickly quickly ADV fcis-32134 76 64 ; ; PUNCT fcis-32134 76 65 at at ADP fcis-32134 76 66 𝑑 𝑑 PROPN fcis-32134 76 67 1 1 NUM fcis-32134 76 68 , , PUNCT fcis-32134 76 69 the the DET fcis-32134 76 70 drone drone NOUN fcis-32134 76 71 reaches reach VERB fcis-32134 76 72 the the DET fcis-32134 76 73 end end NOUN fcis-32134 76 74 point point NOUN fcis-32134 76 75 and and CCONJ fcis-32134 76 76 the the DET fcis-32134 76 77 round round NOUN fcis-32134 76 78 ends end VERB fcis-32134 76 79 . . PUNCT fcis-32134 77 1 ( ( PUNCT fcis-32134 77 2 3 3 X fcis-32134 77 3 ) ) PUNCT fcis-32134 77 4 obstacle obstacle NOUN fcis-32134 77 5 constraint constraint NOUN fcis-32134 77 6 penalty penalty NOUN fcis-32134 77 7 inversely inversely ADV fcis-32134 77 8 penalize penalize VERB fcis-32134 77 9 the the DET fcis-32134 77 10 drone drone NOUN fcis-32134 77 11 when when SCONJ fcis-32134 77 12 it it PRON fcis-32134 77 13 approaches approach VERB fcis-32134 77 14 an an DET fcis-32134 77 15 obstacle obstacle NOUN fcis-32134 77 16 ; ; PUNCT fcis-32134 77 17 when when SCONJ fcis-32134 77 18 it it PRON fcis-32134 77 19 collides collide VERB fcis-32134 77 20 with with ADP fcis-32134 77 21 an an DET fcis-32134 77 22 obstacle obstacle NOUN fcis-32134 77 23 , , PUNCT fcis-32134 77 24 give give VERB fcis-32134 77 25 a a DET fcis-32134 77 26 larger large ADJ fcis-32134 77 27 penalty penalty NOUN fcis-32134 77 28 and and CCONJ fcis-32134 77 29 end end VERB fcis-32134 77 30 the the DET fcis-32134 77 31 round round NOUN fcis-32134 77 32 . . PUNCT fcis-32134 78 1 the the DET fcis-32134 78 2 details detail NOUN fcis-32134 78 3 are be AUX fcis-32134 78 4 as as SCONJ fcis-32134 78 5 follows follow VERB fcis-32134 78 6 : : PUNCT fcis-32134 78 7 𝑅 𝑅 PROPN fcis-32134 78 8 20 20 NUM fcis-32134 78 9 π‘‡π‘’π‘Ÿπ‘šπ‘–π‘›π‘Žπ‘‘π‘’ π‘‡π‘’π‘Ÿπ‘šπ‘–π‘›π‘Žπ‘‘π‘’ PROPN fcis-32134 78 10 π‘Ÿπ‘œπ‘’π‘›π‘‘ π‘Ÿπ‘œπ‘’π‘›π‘‘ NOUN fcis-32134 78 11 𝑑 𝑑 PROPN fcis-32134 78 12 2.0 2.0 NUM fcis-32134 78 13 2 2 NUM fcis-32134 78 14 𝑑 𝑑 PROPN fcis-32134 78 15 2.0 2.0 NUM fcis-32134 78 16 𝑑 𝑑 PROPN fcis-32134 78 17 3.5 3.5 NUM fcis-32134 78 18 ( ( PUNCT fcis-32134 78 19 10 10 NUM fcis-32134 78 20 ) ) PUNCT fcis-32134 78 21 where where SCONJ fcis-32134 78 22 𝑑 𝑑 PRON fcis-32134 78 23 represents represent VERB fcis-32134 78 24 the the DET fcis-32134 78 25 distance distance NOUN fcis-32134 78 26 between between ADP fcis-32134 78 27 the the DET fcis-32134 78 28 drone drone NOUN fcis-32134 78 29 and and CCONJ fcis-32134 78 30 the the DET fcis-32134 78 31 nearest near ADJ fcis-32134 78 32 obstacle obstacle NOUN fcis-32134 78 33 , , PUNCT fcis-32134 78 34 set set VERB fcis-32134 78 35 2.0 2.0 NUM fcis-32134 78 36 𝑑 𝑑 PROPN fcis-32134 78 37 3.5 3.5 NUM fcis-32134 78 38 area area NOUN fcis-32134 78 39 as as ADP fcis-32134 78 40 the the DET fcis-32134 78 41 caution caution NOUN fcis-32134 78 42 area area NOUN fcis-32134 78 43 , , PUNCT fcis-32134 78 44 and and CCONJ fcis-32134 78 45 set set VERB fcis-32134 78 46 𝑑 𝑑 PROPN fcis-32134 78 47 2.0 2.0 NUM fcis-32134 78 48 as as ADP fcis-32134 78 49 the the DET fcis-32134 78 50 collision collision NOUN fcis-32134 78 51 area area NOUN fcis-32134 78 52 . . PUNCT fcis-32134 79 1 ( ( PUNCT fcis-32134 79 2 4 4 X fcis-32134 79 3 ) ) PUNCT fcis-32134 79 4 hover hover NOUN fcis-32134 79 5 penalty penalty NOUN fcis-32134 79 6 the the DET fcis-32134 79 7 drone drone NOUN fcis-32134 79 8 may may AUX fcis-32134 79 9 accumulate accumulate VERB fcis-32134 79 10 navigation navigation NOUN fcis-32134 79 11 rewards reward NOUN fcis-32134 79 12 by by ADP fcis-32134 79 13 hovering hover VERB fcis-32134 79 14 , , PUNCT fcis-32134 79 15 so so SCONJ fcis-32134 79 16 it it PRON fcis-32134 79 17 is be AUX fcis-32134 79 18 set set VERB fcis-32134 79 19 that that SCONJ fcis-32134 79 20 when when SCONJ fcis-32134 79 21 the the DET fcis-32134 79 22 maximum maximum ADJ fcis-32134 79 23 distance distance NOUN fcis-32134 79 24 moved move VERB fcis-32134 79 25 by by ADP fcis-32134 79 26 the the DET fcis-32134 79 27 drone drone NOUN fcis-32134 79 28 in in ADP fcis-32134 79 29 20 20 NUM fcis-32134 79 30 consecutive consecutive ADJ fcis-32134 79 31 steps step NOUN fcis-32134 79 32 is be AUX fcis-32134 79 33 less less ADJ fcis-32134 79 34 than than ADP fcis-32134 79 35 0.1 0.1 NUM fcis-32134 79 36 , , PUNCT fcis-32134 79 37 the the DET fcis-32134 79 38 drone drone NOUN fcis-32134 79 39 is be AUX fcis-32134 79 40 considered consider VERB fcis-32134 79 41 to to PART fcis-32134 79 42 be be AUX fcis-32134 79 43 in in ADP fcis-32134 79 44 a a DET fcis-32134 79 45 hovering hover VERB fcis-32134 79 46 state state NOUN fcis-32134 79 47 and and CCONJ fcis-32134 79 48 a a DET fcis-32134 79 49 hovering hover VERB fcis-32134 79 50 penalty penalty NOUN fcis-32134 79 51 is be AUX fcis-32134 79 52 given give VERB fcis-32134 79 53 to to ADP fcis-32134 79 54 it it PRON fcis-32134 79 55 . . PUNCT fcis-32134 80 1 the the DET fcis-32134 80 2 specifics specific NOUN fcis-32134 80 3 are be AUX fcis-32134 80 4 as as SCONJ fcis-32134 80 5 follows follow VERB fcis-32134 80 6 : : PUNCT fcis-32134 80 7 r r NOUN fcis-32134 80 8 10 10 NUM fcis-32134 80 9 π›₯𝑑 π›₯𝑑 PROPN fcis-32134 80 10 0.1 0.1 NUM fcis-32134 80 11 ( ( PUNCT fcis-32134 80 12 11 11 NUM fcis-32134 80 13 ) ) PUNCT fcis-32134 80 14 where where SCONJ fcis-32134 80 15 π›₯𝑑 π›₯𝑑 PROPN fcis-32134 80 16 is be AUX fcis-32134 80 17 the the DET fcis-32134 80 18 maximum maximum ADJ fcis-32134 80 19 distance distance NOUN fcis-32134 80 20 difference difference NOUN fcis-32134 80 21 of of ADP fcis-32134 80 22 20 20 NUM fcis-32134 80 23 consecutive consecutive ADJ fcis-32134 80 24 steps step NOUN fcis-32134 80 25 . . PUNCT fcis-32134 81 1 in in ADP fcis-32134 81 2 summary summary NOUN fcis-32134 81 3 , , PUNCT fcis-32134 81 4 the the DET fcis-32134 81 5 final final ADJ fcis-32134 81 6 reward reward NOUN fcis-32134 81 7 function function NOUN fcis-32134 81 8 is be AUX fcis-32134 81 9 : : PUNCT fcis-32134 81 10 𝑅 𝑅 PROPN fcis-32134 81 11 𝑅 𝑅 PROPN fcis-32134 81 12 𝑅 𝑅 PROPN fcis-32134 81 13 𝑅 𝑅 PROPN fcis-32134 81 14 𝑅 𝑅 PROPN fcis-32134 81 15 ( ( PUNCT fcis-32134 81 16 12 12 NUM fcis-32134 81 17 ) ) PUNCT fcis-32134 81 18 4 4 NUM fcis-32134 81 19 . . X fcis-32134 81 20 simulation simulation NOUN fcis-32134 81 21 and and CCONJ fcis-32134 81 22 analysis analysis NOUN fcis-32134 81 23 4.1 4.1 NUM fcis-32134 81 24 . . PUNCT fcis-32134 82 1 algorithm algorithm NOUN fcis-32134 82 2 parameter parameter NOUN fcis-32134 82 3 setting set VERB fcis-32134 82 4 the the DET fcis-32134 82 5 algorithm algorithm NOUN fcis-32134 82 6 parameters parameter NOUN fcis-32134 82 7 used use VERB fcis-32134 82 8 in in ADP fcis-32134 82 9 this this DET fcis-32134 82 10 paper paper NOUN fcis-32134 82 11 are be AUX fcis-32134 82 12 shown show VERB fcis-32134 82 13 in in ADP fcis-32134 82 14 table table NOUN fcis-32134 82 15 2 2 NUM fcis-32134 82 16 , , PUNCT fcis-32134 82 17 in in ADP fcis-32134 82 18 which which PRON fcis-32134 82 19 the the DET fcis-32134 82 20 exploration exploration NOUN fcis-32134 82 21 rate rate NOUN fcis-32134 82 22 is be AUX fcis-32134 82 23 linearly linearly ADV fcis-32134 82 24 decaying decay VERB fcis-32134 82 25 , , PUNCT fcis-32134 82 26 which which PRON fcis-32134 82 27 is be AUX fcis-32134 82 28 convenient convenient ADJ fcis-32134 82 29 for for SCONJ fcis-32134 82 30 the the DET fcis-32134 82 31 uav uav PROPN fcis-32134 82 32 to to PART fcis-32134 82 33 explore explore VERB fcis-32134 82 34 the the DET fcis-32134 82 35 state state NOUN fcis-32134 82 36 space space NOUN fcis-32134 82 37 with with ADP fcis-32134 82 38 a a DET fcis-32134 82 39 larger large ADJ fcis-32134 82 40 probability probability NOUN fcis-32134 82 41 at at ADP fcis-32134 82 42 the the DET fcis-32134 82 43 beginning beginning NOUN fcis-32134 82 44 of of ADP fcis-32134 82 45 the the DET fcis-32134 82 46 training training NOUN fcis-32134 82 47 to to PART fcis-32134 82 48 avoid avoid VERB fcis-32134 82 49 falling fall VERB fcis-32134 82 50 into into ADP fcis-32134 82 51 the the DET fcis-32134 82 52 local local ADJ fcis-32134 82 53 optimum optimum NOUN fcis-32134 82 54 too too ADV fcis-32134 82 55 early early ADJ fcis-32134 82 56 , , PUNCT fcis-32134 82 57 and and CCONJ fcis-32134 82 58 stabilize stabilize VERB fcis-32134 82 59 policies policy NOUN fcis-32134 82 60 throughout throughout ADP fcis-32134 82 61 later later ADJ fcis-32134 82 62 phases phase NOUN fcis-32134 82 63 then then ADV fcis-32134 82 64 improve improve VERB fcis-32134 82 65 the the DET fcis-32134 82 66 efficiency efficiency NOUN fcis-32134 82 67 . . PUNCT fcis-32134 83 1 4.2 4.2 NUM fcis-32134 83 2 . . PUNCT fcis-32134 84 1 experimental experimental ADJ fcis-32134 84 2 analysis analysis NOUN fcis-32134 84 3 to to PART fcis-32134 84 4 evaluate evaluate VERB fcis-32134 84 5 algorithmic algorithmic ADJ fcis-32134 84 6 performance performance NOUN fcis-32134 84 7 , , PUNCT fcis-32134 84 8 this this DET fcis-32134 84 9 paper paper NOUN fcis-32134 84 10 trains train NOUN fcis-32134 84 11 and and CCONJ fcis-32134 84 12 tests test VERB fcis-32134 84 13 the the DET fcis-32134 84 14 sac sac NOUN fcis-32134 84 15 algorithm algorithm NOUN fcis-32134 84 16 with with ADP fcis-32134 84 17 ddpg ddpg ADJ fcis-32134 84 18 and and CCONJ fcis-32134 84 19 td3 td3 VERB fcis-32134 84 20 in in ADP fcis-32134 84 21 the the DET fcis-32134 84 22 simulation simulation NOUN fcis-32134 84 23 environment environment NOUN fcis-32134 85 1 [ [ X fcis-32134 85 2 14 14 NUM fcis-32134 85 3 , , PUNCT fcis-32134 85 4 15 15 NUM fcis-32134 85 5 ] ] PUNCT fcis-32134 85 6 . . PUNCT fcis-32134 86 1 at at ADP fcis-32134 86 2 the the DET fcis-32134 86 3 beginning beginning NOUN fcis-32134 86 4 of of ADP fcis-32134 86 5 training training NOUN fcis-32134 86 6 , , PUNCT fcis-32134 86 7 the the DET fcis-32134 86 8 initial initial ADJ fcis-32134 86 9 reward reward NOUN fcis-32134 86 10 is be AUX fcis-32134 86 11 set set VERB fcis-32134 86 12 to to ADP fcis-32134 86 13 zero zero NUM fcis-32134 86 14 , , PUNCT fcis-32134 86 15 and and CCONJ fcis-32134 86 16 when when SCONJ fcis-32134 86 17 the the DET fcis-32134 86 18 uav uav PROPN fcis-32134 86 19 flies fly VERB fcis-32134 86 20 to to ADP fcis-32134 86 21 the the DET fcis-32134 86 22 end end NOUN fcis-32134 86 23 point point NOUN fcis-32134 86 24 or or CCONJ fcis-32134 86 25 close close ADJ fcis-32134 86 26 to to ADP fcis-32134 86 27 the the DET fcis-32134 86 28 obstacle obstacle NOUN fcis-32134 86 29 , , PUNCT fcis-32134 86 30 it it PRON fcis-32134 86 31 will will AUX fcis-32134 86 32 be be AUX fcis-32134 86 33 rewarded reward VERB fcis-32134 86 34 and and CCONJ fcis-32134 86 35 penalized penalize VERB fcis-32134 86 36 accordingly accordingly ADV fcis-32134 86 37 . . PUNCT fcis-32134 87 1 since since SCONJ fcis-32134 87 2 the the DET fcis-32134 87 3 reward reward NOUN fcis-32134 87 4 value value NOUN fcis-32134 87 5 is be AUX fcis-32134 87 6 the the DET fcis-32134 87 7 sum sum NOUN fcis-32134 87 8 of of ADP fcis-32134 87 9 the the DET fcis-32134 87 10 rewards reward NOUN fcis-32134 87 11 obtained obtain VERB fcis-32134 87 12 at at ADP fcis-32134 87 13 each each DET fcis-32134 87 14 step step NOUN fcis-32134 87 15 , , PUNCT fcis-32134 87 16 a a DET fcis-32134 87 17 larger large ADJ fcis-32134 87 18 reward reward NOUN fcis-32134 87 19 value value NOUN fcis-32134 87 20 indicates indicate VERB fcis-32134 87 21 a a DET fcis-32134 87 22 higher high ADJ fcis-32134 87 23 probability probability NOUN fcis-32134 87 24 of of ADP fcis-32134 87 25 the the DET fcis-32134 87 26 uav uav PROPN fcis-32134 87 27 avoiding avoid VERB fcis-32134 87 28 obstacles obstacle NOUN fcis-32134 87 29 and and CCONJ fcis-32134 87 30 reaching reach VERB fcis-32134 87 31 the the DET fcis-32134 87 32 end end NOUN fcis-32134 87 33 point point NOUN fcis-32134 87 34 , , PUNCT fcis-32134 87 35 i.e. i.e. X fcis-32134 87 36 , , PUNCT fcis-32134 87 37 a a DET fcis-32134 87 38 better well ADV fcis-32134 87 39 learned learn VERB fcis-32134 87 40 strategy strategy NOUN fcis-32134 87 41 . . PUNCT fcis-32134 88 1 since since SCONJ fcis-32134 88 2 46 46 NUM fcis-32134 88 3 the the DET fcis-32134 88 4 obstacle obstacle NOUN fcis-32134 88 5 avoidance avoidance NOUN fcis-32134 88 6 task task NOUN fcis-32134 88 7 may may AUX fcis-32134 88 8 end end VERB fcis-32134 88 9 early early ADV fcis-32134 88 10 at at ADP fcis-32134 88 11 a a DET fcis-32134 88 12 later later ADJ fcis-32134 88 13 stage stage NOUN fcis-32134 88 14 , , PUNCT fcis-32134 88 15 or or CCONJ fcis-32134 88 16 the the DET fcis-32134 88 17 penalty penalty NOUN fcis-32134 88 18 for for ADP fcis-32134 88 19 collision collision NOUN fcis-32134 88 20 is be AUX fcis-32134 88 21 not not PART fcis-32134 88 22 enough enough ADJ fcis-32134 88 23 to to PART fcis-32134 88 24 offset offset VERB fcis-32134 88 25 the the DET fcis-32134 88 26 accumulated accumulate VERB fcis-32134 88 27 rewards reward NOUN fcis-32134 88 28 resulting result VERB fcis-32134 88 29 in in ADP fcis-32134 88 30 a a DET fcis-32134 88 31 reward reward NOUN fcis-32134 88 32 value value NOUN fcis-32134 88 33 that that PRON fcis-32134 88 34 does do AUX fcis-32134 88 35 not not PART fcis-32134 88 36 reflect reflect VERB fcis-32134 88 37 the the DET fcis-32134 88 38 real real ADJ fcis-32134 88 39 situation situation NOUN fcis-32134 88 40 well well ADV fcis-32134 88 41 , , PUNCT fcis-32134 88 42 this this DET fcis-32134 88 43 paper paper NOUN fcis-32134 88 44 chooses choose VERB fcis-32134 88 45 the the DET fcis-32134 88 46 average average ADJ fcis-32134 88 47 reward reward NOUN fcis-32134 88 48 value value NOUN fcis-32134 88 49 calculated calculate VERB fcis-32134 88 50 by by ADP fcis-32134 88 51 the the DET fcis-32134 88 52 50 50 NUM fcis-32134 88 53 - - PUNCT fcis-32134 88 54 round round NOUN fcis-32134 88 55 sliding slide VERB fcis-32134 88 56 average average ADJ fcis-32134 88 57 method method NOUN fcis-32134 88 58 as as SCONJ fcis-32134 88 59 the the DET fcis-32134 88 60 evaluation evaluation NOUN fcis-32134 88 61 index index NOUN fcis-32134 88 62 to to PART fcis-32134 88 63 eliminate eliminate VERB fcis-32134 88 64 the the DET fcis-32134 88 65 above above ADJ fcis-32134 88 66 interference interference NOUN fcis-32134 88 67 . . PUNCT fcis-32134 89 1 table table NOUN fcis-32134 89 2 2 2 NUM fcis-32134 89 3 . . PUNCT fcis-32134 89 4 algorithm algorithm NOUN fcis-32134 89 5 parameters parameter NOUN fcis-32134 89 6 setting set VERB fcis-32134 89 7 table table NOUN fcis-32134 89 8 parameter parameter NOUN fcis-32134 89 9 value value NOUN fcis-32134 89 10 discount discount NOUN fcis-32134 89 11 factor factor NOUN fcis-32134 89 12 Ξ³ Ξ³ PROPN fcis-32134 89 13 0.99 0.99 NUM fcis-32134 89 14 soft soft ADJ fcis-32134 89 15 update update NOUN fcis-32134 89 16 factor factor NOUN fcis-32134 89 17 𝜏 𝜏 NOUN fcis-32134 89 18 0.005 0.005 NUM fcis-32134 89 19 initial initial ADJ fcis-32134 89 20 Ξ± Ξ± PROPN fcis-32134 89 21 0.2 0.2 NUM fcis-32134 89 22 experience experience NOUN fcis-32134 89 23 playback playback NOUN fcis-32134 89 24 buffer buffer NOUN fcis-32134 89 25 pool pool NOUN fcis-32134 89 26 100000 100000 NUM fcis-32134 89 27 training training NOUN fcis-32134 89 28 batch batch NOUN fcis-32134 89 29 size size NOUN fcis-32134 89 30 128 128 NUM fcis-32134 89 31 maximum maximum ADJ fcis-32134 89 32 step step NOUN fcis-32134 89 33 size size NOUN fcis-32134 89 34 for for ADP fcis-32134 89 35 a a DET fcis-32134 89 36 single single ADJ fcis-32134 89 37 training training NOUN fcis-32134 89 38 batch batch NOUN fcis-32134 89 39 500 500 NUM fcis-32134 89 40 initial initial ADJ fcis-32134 89 41 exploration exploration NOUN fcis-32134 89 42 rate rate NOUN fcis-32134 89 43 0.5 0.5 NUM fcis-32134 89 44 minimum minimum NOUN fcis-32134 89 45 exploration exploration NOUN fcis-32134 89 46 rate rate NOUN fcis-32134 89 47 0.01 0.01 NUM fcis-32134 89 48 exploration exploration NOUN fcis-32134 89 49 rate rate NOUN fcis-32134 89 50 decay decay NOUN fcis-32134 89 51 coefficient coefficient NOUN fcis-32134 89 52 0.995 0.995 NUM fcis-32134 89 53 figure figure NOUN fcis-32134 89 54 4 4 NUM fcis-32134 89 55 . . PUNCT fcis-32134 90 1 change change NOUN fcis-32134 90 2 curve curve NOUN fcis-32134 90 3 of of ADP fcis-32134 90 4 average average ADJ fcis-32134 90 5 reward reward NOUN fcis-32134 90 6 value value NOUN fcis-32134 90 7 ( ( PUNCT fcis-32134 90 8 50 50 NUM fcis-32134 90 9 - - PUNCT fcis-32134 90 10 round round NOUN fcis-32134 90 11 sliding slide VERB fcis-32134 90 12 average average ADJ fcis-32134 90 13 ) ) PUNCT fcis-32134 90 14 figure figure NOUN fcis-32134 90 15 4 4 NUM fcis-32134 90 16 shows show VERB fcis-32134 90 17 the the DET fcis-32134 90 18 average average ADJ fcis-32134 90 19 reward reward NOUN fcis-32134 90 20 value value NOUN fcis-32134 90 21 change change NOUN fcis-32134 90 22 curve curve NOUN fcis-32134 90 23 of of ADP fcis-32134 90 24 each each DET fcis-32134 90 25 algorithm algorithm NOUN fcis-32134 90 26 . . PUNCT fcis-32134 91 1 from from ADP fcis-32134 91 2 the the DET fcis-32134 91 3 figure figure NOUN fcis-32134 91 4 , , PUNCT fcis-32134 91 5 it it PRON fcis-32134 91 6 can can AUX fcis-32134 91 7 be be AUX fcis-32134 91 8 found find VERB fcis-32134 91 9 that that SCONJ fcis-32134 91 10 the the DET fcis-32134 91 11 ddpg ddpg ADJ fcis-32134 91 12 algorithm algorithm NOUN fcis-32134 91 13 reward reward NOUN fcis-32134 91 14 growth growth NOUN fcis-32134 91 15 is be AUX fcis-32134 91 16 gentle gentle ADJ fcis-32134 91 17 and and CCONJ fcis-32134 91 18 smooth smooth ADJ fcis-32134 91 19 , , PUNCT fcis-32134 91 20 with with ADP fcis-32134 91 21 no no DET fcis-32134 91 22 significant significant ADJ fcis-32134 91 23 fluctuations fluctuation NOUN fcis-32134 91 24 throughout throughout ADV fcis-32134 91 25 , , PUNCT fcis-32134 91 26 but but CCONJ fcis-32134 91 27 the the DET fcis-32134 91 28 reward reward NOUN fcis-32134 91 29 value value NOUN fcis-32134 91 30 is be AUX fcis-32134 91 31 the the DET fcis-32134 91 32 lowest low ADJ fcis-32134 91 33 , , PUNCT fcis-32134 91 34 which which PRON fcis-32134 91 35 indicates indicate VERB fcis-32134 91 36 that that SCONJ fcis-32134 91 37 the the DET fcis-32134 91 38 algorithm algorithm NOUN fcis-32134 91 39 strategy strategy NOUN fcis-32134 91 40 is be AUX fcis-32134 91 41 conservative conservative ADJ fcis-32134 91 42 , , PUNCT fcis-32134 91 43 weak weak ADJ fcis-32134 91 44 in in ADP fcis-32134 91 45 exploration exploration NOUN fcis-32134 91 46 , , PUNCT fcis-32134 91 47 and and CCONJ fcis-32134 91 48 prone prone ADJ fcis-32134 91 49 to to ADP fcis-32134 91 50 falling fall VERB fcis-32134 91 51 into into ADP fcis-32134 91 52 local local ADJ fcis-32134 91 53 optimization optimization NOUN fcis-32134 91 54 . . PUNCT fcis-32134 92 1 compared compare VERB fcis-32134 92 2 to to ADP fcis-32134 92 3 the the DET fcis-32134 92 4 ddpg ddpg ADJ fcis-32134 92 5 algorithm algorithm NOUN fcis-32134 92 6 , , PUNCT fcis-32134 92 7 the the DET fcis-32134 92 8 td3 td3 NOUN fcis-32134 92 9 algorithm algorithm NOUN fcis-32134 92 10 has have VERB fcis-32134 92 11 a a DET fcis-32134 92 12 higher high ADJ fcis-32134 92 13 average average ADJ fcis-32134 92 14 reward reward NOUN fcis-32134 92 15 value value NOUN fcis-32134 92 16 and and CCONJ fcis-32134 92 17 is be AUX fcis-32134 92 18 smooth smooth ADJ fcis-32134 92 19 in in ADP fcis-32134 92 20 the the DET fcis-32134 92 21 later later ADJ fcis-32134 92 22 stages stage NOUN fcis-32134 92 23 , , PUNCT fcis-32134 92 24 reflecting reflect VERB fcis-32134 92 25 the the DET fcis-32134 92 26 suppression suppression NOUN fcis-32134 92 27 of of ADP fcis-32134 92 28 action action NOUN fcis-32134 92 29 noise noise NOUN fcis-32134 92 30 by by ADP fcis-32134 92 31 the the DET fcis-32134 92 32 double double ADJ fcis-32134 92 33 q q NOUN fcis-32134 92 34 - - PUNCT fcis-32134 92 35 learning learn VERB fcis-32134 92 36 mechanism mechanism NOUN fcis-32134 92 37 , , PUNCT fcis-32134 92 38 but but CCONJ fcis-32134 92 39 the the DET fcis-32134 92 40 potential potential NOUN fcis-32134 92 41 is be AUX fcis-32134 92 42 limited limit VERB fcis-32134 92 43 compared compare VERB fcis-32134 92 44 to to ADP fcis-32134 92 45 the the DET fcis-32134 92 46 sac sac NOUN fcis-32134 92 47 algorithm algorithm NOUN fcis-32134 92 48 , , PUNCT fcis-32134 92 49 which which PRON fcis-32134 92 50 introduces introduce VERB fcis-32134 92 51 maximum maximum ADJ fcis-32134 92 52 entropy entropy NOUN fcis-32134 92 53 , , PUNCT fcis-32134 92 54 encourages encourage VERB fcis-32134 92 55 the the DET fcis-32134 92 56 exploration exploration NOUN fcis-32134 92 57 of of ADP fcis-32134 92 58 a a DET fcis-32134 92 59 greater great ADJ fcis-32134 92 60 variety variety NOUN fcis-32134 92 61 of of ADP fcis-32134 92 62 actions action NOUN fcis-32134 92 63 , , PUNCT fcis-32134 92 64 and and CCONJ fcis-32134 92 65 has have VERB fcis-32134 92 66 a a DET fcis-32134 92 67 higher high ADJ fcis-32134 92 68 efficiency efficiency NOUN fcis-32134 92 69 in in ADP fcis-32134 92 70 the the DET fcis-32134 92 71 complex complex ADJ fcis-32134 92 72 space space NOUN fcis-32134 92 73 , , PUNCT fcis-32134 92 74 so so CCONJ fcis-32134 92 75 the the DET fcis-32134 92 76 reward reward NOUN fcis-32134 92 77 value value NOUN fcis-32134 92 78 rises rise VERB fcis-32134 92 79 rapidly rapidly ADV fcis-32134 92 80 from from ADP fcis-32134 92 81 rounds round NOUN fcis-32134 92 82 0 0 NUM fcis-32134 92 83 to to ADP fcis-32134 92 84 750 750 NUM fcis-32134 92 85 and and CCONJ fcis-32134 92 86 has have VERB fcis-32134 92 87 a a DET fcis-32134 92 88 high high ADJ fcis-32134 92 89 reward reward NOUN fcis-32134 92 90 value value NOUN fcis-32134 92 91 after after ADP fcis-32134 92 92 rounds round NOUN fcis-32134 92 93 1400 1400 NUM fcis-32134 92 94 , , PUNCT fcis-32134 92 95 and and CCONJ fcis-32134 92 96 through through ADP fcis-32134 92 97 the the DET fcis-32134 92 98 double double ADJ fcis-32134 92 99 q q NOUN fcis-32134 92 100 - - PUNCT fcis-32134 92 101 learning learn VERB fcis-32134 92 102 mechanism mechanism NOUN fcis-32134 92 103 and and CCONJ fcis-32134 92 104 soft soft ADJ fcis-32134 92 105 updates update NOUN fcis-32134 92 106 , , PUNCT fcis-32134 92 107 the the DET fcis-32134 92 108 performance performance NOUN fcis-32134 92 109 fluctuation fluctuation NOUN fcis-32134 92 110 after after SCONJ fcis-32134 92 111 1500 1500 NUM fcis-32134 92 112 rounds round NOUN fcis-32134 92 113 is be AUX fcis-32134 92 114 made make VERB fcis-32134 92 115 to to PART fcis-32134 92 116 be be AUX fcis-32134 92 117 more more ADV fcis-32134 92 118 effective effective ADJ fcis-32134 92 119 , , PUNCT fcis-32134 92 120 exhibiting exhibit VERB fcis-32134 92 121 greater great ADJ fcis-32134 92 122 stability stability NOUN fcis-32134 92 123 compared compare VERB fcis-32134 92 124 to to ADP fcis-32134 92 125 earlier early ADJ fcis-32134 92 126 rounds round NOUN fcis-32134 92 127 . . PUNCT fcis-32134 93 1 this this PRON fcis-32134 93 2 shows show VERB fcis-32134 93 3 that that SCONJ fcis-32134 93 4 sac sac NOUN fcis-32134 93 5 algorithm algorithm NOUN fcis-32134 93 6 has have VERB fcis-32134 93 7 high high ADJ fcis-32134 93 8 sample sample NOUN fcis-32134 93 9 efficiency efficiency NOUN fcis-32134 93 10 , , PUNCT fcis-32134 93 11 high high ADJ fcis-32134 93 12 reward reward NOUN fcis-32134 93 13 limit limit NOUN fcis-32134 93 14 , , PUNCT fcis-32134 93 15 strong strong ADJ fcis-32134 93 16 adaptability adaptability NOUN fcis-32134 93 17 to to ADP fcis-32134 93 18 complex complex ADJ fcis-32134 93 19 environment environment NOUN fcis-32134 93 20 , , PUNCT fcis-32134 93 21 better well ADJ fcis-32134 93 22 performance performance NOUN fcis-32134 93 23 than than ADP fcis-32134 93 24 ddpg ddpg ADJ fcis-32134 93 25 algorithm algorithm NOUN fcis-32134 93 26 and and CCONJ fcis-32134 93 27 td3 td3 NOUN fcis-32134 93 28 algorithm algorithm NOUN fcis-32134 93 29 , , PUNCT fcis-32134 93 30 and and CCONJ fcis-32134 93 31 gives give VERB fcis-32134 93 32 full full ADJ fcis-32134 93 33 play play NOUN fcis-32134 93 34 to to ADP fcis-32134 93 35 the the DET fcis-32134 93 36 maneuverability maneuverability NOUN fcis-32134 93 37 of of ADP fcis-32134 93 38 uav uav PROPN fcis-32134 93 39 . . PUNCT fcis-32134 94 1 figure figure NOUN fcis-32134 94 2 5 5 NUM fcis-32134 94 3 shows show VERB fcis-32134 94 4 the the DET fcis-32134 94 5 success success NOUN fcis-32134 94 6 rate rate NOUN fcis-32134 94 7 curve curve NOUN fcis-32134 94 8 of of ADP fcis-32134 94 9 each each DET fcis-32134 94 10 algorithm algorithm NOUN fcis-32134 94 11 per per ADP fcis-32134 94 12 50 50 NUM fcis-32134 94 13 rounds round NOUN fcis-32134 94 14 of of ADP fcis-32134 94 15 mission mission NOUN fcis-32134 94 16 . . PUNCT fcis-32134 95 1 from from ADP fcis-32134 95 2 the the DET fcis-32134 95 3 figure figure NOUN fcis-32134 95 4 , , PUNCT fcis-32134 95 5 it it PRON fcis-32134 95 6 can can AUX fcis-32134 95 7 be be AUX fcis-32134 95 8 found find VERB fcis-32134 95 9 that that SCONJ fcis-32134 95 10 the the DET fcis-32134 95 11 success success NOUN fcis-32134 95 12 rate rate NOUN fcis-32134 95 13 of of ADP fcis-32134 95 14 ddpg ddpg ADJ fcis-32134 95 15 fluctuates fluctuate NOUN fcis-32134 95 16 from from ADP fcis-32134 95 17 40 40 NUM fcis-32134 95 18 % % NOUN fcis-32134 95 19 to to PART fcis-32134 95 20 60 60 NUM fcis-32134 95 21 % % NOUN fcis-32134 95 22 , , PUNCT fcis-32134 95 23 which which PRON fcis-32134 95 24 indicates indicate VERB fcis-32134 95 25 that that SCONJ fcis-32134 95 26 the the DET fcis-32134 95 27 algorithm algorithm NOUN fcis-32134 95 28 is be AUX fcis-32134 95 29 not not PART fcis-32134 95 30 ideal ideal ADJ fcis-32134 95 31 for for SCONJ fcis-32134 95 32 this this DET fcis-32134 95 33 task.td3 task.td3 PROPN fcis-32134 95 34 algorithm algorithm NOUN fcis-32134 95 35 has have VERB fcis-32134 95 36 a a DET fcis-32134 95 37 lower low ADJ fcis-32134 95 38 success success NOUN fcis-32134 95 39 rate rate NOUN fcis-32134 95 40 than than ADP fcis-32134 95 41 ddpg ddpg ADJ fcis-32134 95 42 in in ADP fcis-32134 95 43 the the DET fcis-32134 95 44 first first ADJ fcis-32134 95 45 150 150 NUM fcis-32134 95 46 rounds round NOUN fcis-32134 95 47 , , PUNCT fcis-32134 95 48 but but CCONJ fcis-32134 95 49 the the DET fcis-32134 95 50 success success NOUN fcis-32134 95 51 rate rate NOUN fcis-32134 95 52 gradually gradually ADV fcis-32134 95 53 increases increase VERB fcis-32134 95 54 in in ADP fcis-32134 95 55 the the DET fcis-32134 95 56 later later ADJ fcis-32134 95 57 rounds round NOUN fcis-32134 95 58 and and CCONJ fcis-32134 95 59 is be AUX fcis-32134 95 60 higher high ADJ fcis-32134 95 61 than than ADP fcis-32134 95 62 that that PRON fcis-32134 95 63 of of ADP fcis-32134 95 64 ddpg ddpg ADJ fcis-32134 95 65 in in ADP fcis-32134 95 66 most most ADJ fcis-32134 95 67 of of ADP fcis-32134 95 68 the the DET fcis-32134 95 69 cases case NOUN fcis-32134 95 70 , , PUNCT fcis-32134 95 71 with with ADP fcis-32134 95 72 a a DET fcis-32134 95 73 peak peak NOUN fcis-32134 95 74 of of ADP fcis-32134 95 75 76 76 NUM fcis-32134 95 76 % % NOUN fcis-32134 95 77 , , PUNCT fcis-32134 95 78 which which PRON fcis-32134 95 79 proves prove VERB fcis-32134 95 80 the the DET fcis-32134 95 81 effectiveness effectiveness NOUN fcis-32134 95 82 of of ADP fcis-32134 95 83 the the DET fcis-32134 95 84 delayed delay VERB fcis-32134 95 85 updating updating NOUN fcis-32134 95 86 strategy strategy NOUN fcis-32134 95 87 of of ADP fcis-32134 95 88 this this DET fcis-32134 95 89 algorithm algorithm NOUN fcis-32134 95 90 . . PUNCT fcis-32134 96 1 the the DET fcis-32134 96 2 success success NOUN fcis-32134 96 3 rate rate NOUN fcis-32134 96 4 of of ADP fcis-32134 96 5 the the DET fcis-32134 96 6 sac sac NOUN fcis-32134 96 7 algorithm algorithm NOUN fcis-32134 96 8 fluctuates fluctuate VERB fcis-32134 96 9 within within ADP fcis-32134 96 10 the the DET fcis-32134 96 11 range range NOUN fcis-32134 96 12 of of ADP fcis-32134 96 13 70 70 NUM fcis-32134 96 14 % % NOUN fcis-32134 96 15 to to PART fcis-32134 96 16 80 80 NUM fcis-32134 96 17 % % NOUN fcis-32134 96 18 , , PUNCT fcis-32134 96 19 generally generally ADV fcis-32134 96 20 outperforming outperform VERB fcis-32134 96 21 both both CCONJ fcis-32134 96 22 the the DET fcis-32134 96 23 ddpg ddpg ADJ fcis-32134 96 24 and and CCONJ fcis-32134 96 25 td3 td3 NOUN fcis-32134 96 26 algorithms algorithm NOUN fcis-32134 96 27 . . PUNCT fcis-32134 97 1 this this PRON fcis-32134 97 2 demonstrates demonstrate VERB fcis-32134 97 3 sac sac NOUN fcis-32134 97 4 's 's PART fcis-32134 97 5 strong strong ADJ fcis-32134 97 6 capacity capacity NOUN fcis-32134 97 7 for for ADP fcis-32134 97 8 rapid rapid ADJ fcis-32134 97 9 learning learning NOUN fcis-32134 97 10 , , PUNCT fcis-32134 97 11 effective effective ADJ fcis-32134 97 12 adaptation adaptation NOUN fcis-32134 97 13 to to ADP fcis-32134 97 14 complex complex ADJ fcis-32134 97 15 tasks task NOUN fcis-32134 97 16 , , PUNCT fcis-32134 97 17 and and CCONJ fcis-32134 97 18 superior superior ADJ fcis-32134 97 19 overall overall ADJ fcis-32134 97 20 performance performance NOUN fcis-32134 97 21 . . PUNCT fcis-32134 98 1 47 47 NUM fcis-32134 98 2 figure figure NOUN fcis-32134 98 3 5 5 NUM fcis-32134 98 4 . . PUNCT fcis-32134 98 5 success success NOUN fcis-32134 98 6 rate rate NOUN fcis-32134 98 7 change change NOUN fcis-32134 98 8 curve curve NOUN fcis-32134 98 9 in in ADP fcis-32134 98 10 order order NOUN fcis-32134 98 11 to to PART fcis-32134 98 12 further far ADV fcis-32134 98 13 verify verify VERB fcis-32134 98 14 the the DET fcis-32134 98 15 performance performance NOUN fcis-32134 98 16 of of ADP fcis-32134 98 17 the the DET fcis-32134 98 18 algorithms algorithm NOUN fcis-32134 98 19 , , PUNCT fcis-32134 98 20 this this DET fcis-32134 98 21 paper paper NOUN fcis-32134 98 22 tests test VERB fcis-32134 98 23 each each DET fcis-32134 98 24 algorithm algorithm NOUN fcis-32134 98 25 in in ADP fcis-32134 98 26 the the DET fcis-32134 98 27 test test NOUN fcis-32134 98 28 environment environment NOUN fcis-32134 98 29 . . PUNCT fcis-32134 99 1 the the DET fcis-32134 99 2 starting starting NOUN fcis-32134 99 3 coordinates coordinate NOUN fcis-32134 99 4 of of ADP fcis-32134 99 5 the the DET fcis-32134 99 6 uav uav PROPN fcis-32134 99 7 in in ADP fcis-32134 99 8 the the DET fcis-32134 99 9 testing testing NOUN fcis-32134 99 10 environment environment NOUN fcis-32134 99 11 are be AUX fcis-32134 99 12 ( ( PUNCT fcis-32134 99 13 10,10,5 10,10,5 NUM fcis-32134 99 14 ) ) PUNCT fcis-32134 99 15 m m PROPN fcis-32134 99 16 , , PUNCT fcis-32134 99 17 and and CCONJ fcis-32134 99 18 the the DET fcis-32134 99 19 ending end VERB fcis-32134 99 20 coordinates coordinate NOUN fcis-32134 99 21 are be AUX fcis-32134 99 22 ( ( PUNCT fcis-32134 99 23 50,50,10 50,50,10 NUM fcis-32134 99 24 ) ) PUNCT fcis-32134 99 25 m. m. NOUN fcis-32134 99 26 the the DET fcis-32134 99 27 static static ADJ fcis-32134 99 28 obstacles obstacle NOUN fcis-32134 99 29 are be AUX fcis-32134 99 30 set set VERB fcis-32134 99 31 to to ADP fcis-32134 99 32 five five NUM fcis-32134 99 33 , , PUNCT fcis-32134 99 34 and and CCONJ fcis-32134 99 35 the the DET fcis-32134 99 36 dynamic dynamic ADJ fcis-32134 99 37 obstacles obstacle NOUN fcis-32134 99 38 are be AUX fcis-32134 99 39 increased increase VERB fcis-32134 99 40 to to ADP fcis-32134 99 41 seven seven NUM fcis-32134 99 42 . . PUNCT fcis-32134 100 1 the the DET fcis-32134 100 2 task task NOUN fcis-32134 100 3 requires require VERB fcis-32134 100 4 the the DET fcis-32134 100 5 uav uav PROPN fcis-32134 100 6 to to PART fcis-32134 100 7 first first ADV fcis-32134 100 8 start start VERB fcis-32134 100 9 from from ADP fcis-32134 100 10 the the DET fcis-32134 100 11 starting starting NOUN fcis-32134 100 12 point point NOUN fcis-32134 100 13 , , PUNCT fcis-32134 100 14 automatically automatically ADV fcis-32134 100 15 avoid avoid VERB fcis-32134 100 16 obstacles obstacle NOUN fcis-32134 100 17 when when SCONJ fcis-32134 100 18 encountering encounter VERB fcis-32134 100 19 obstacles obstacle NOUN fcis-32134 100 20 , , PUNCT fcis-32134 100 21 and and CCONJ fcis-32134 100 22 finally finally ADV fcis-32134 100 23 arrive arrive VERB fcis-32134 100 24 at at ADP fcis-32134 100 25 the the DET fcis-32134 100 26 end end NOUN fcis-32134 100 27 point point NOUN fcis-32134 100 28 without without ADP fcis-32134 100 29 collision collision NOUN fcis-32134 100 30 . . PUNCT fcis-32134 101 1 table table NOUN fcis-32134 101 2 3 3 NUM fcis-32134 101 3 shows show VERB fcis-32134 101 4 the the DET fcis-32134 101 5 results result NOUN fcis-32134 101 6 of of ADP fcis-32134 101 7 collision collision NOUN fcis-32134 101 8 rate rate NOUN fcis-32134 101 9 and and CCONJ fcis-32134 101 10 timeout timeout NOUN fcis-32134 101 11 rate rate NOUN fcis-32134 101 12 of of ADP fcis-32134 101 13 each each DET fcis-32134 101 14 algorithm algorithm NOUN fcis-32134 101 15 for for ADP fcis-32134 101 16 2000 2000 NUM fcis-32134 101 17 rounds round NOUN fcis-32134 101 18 in in ADP fcis-32134 101 19 the the DET fcis-32134 101 20 test test NOUN fcis-32134 101 21 environment environment NOUN fcis-32134 101 22 . . PUNCT fcis-32134 102 1 the the DET fcis-32134 102 2 collision collision NOUN fcis-32134 102 3 rate rate NOUN fcis-32134 102 4 represents represent VERB fcis-32134 102 5 the the DET fcis-32134 102 6 ratio ratio NOUN fcis-32134 102 7 of of ADP fcis-32134 102 8 the the DET fcis-32134 102 9 number number NOUN fcis-32134 102 10 of of ADP fcis-32134 102 11 collisions collision NOUN fcis-32134 102 12 rounds round NOUN fcis-32134 102 13 to to ADP fcis-32134 102 14 the the DET fcis-32134 102 15 total total ADJ fcis-32134 102 16 number number NOUN fcis-32134 102 17 of of ADP fcis-32134 102 18 rounds round NOUN fcis-32134 102 19 , , PUNCT fcis-32134 102 20 and and CCONJ fcis-32134 102 21 the the DET fcis-32134 102 22 timeout timeout NOUN fcis-32134 102 23 rate rate NOUN fcis-32134 102 24 represents represent VERB fcis-32134 102 25 the the DET fcis-32134 102 26 ratio ratio NOUN fcis-32134 102 27 of of ADP fcis-32134 102 28 the the DET fcis-32134 102 29 number number NOUN fcis-32134 102 30 of of ADP fcis-32134 102 31 rounds round NOUN fcis-32134 102 32 that that PRON fcis-32134 102 33 have have AUX fcis-32134 102 34 not not PART fcis-32134 102 35 reached reach VERB fcis-32134 102 36 the the DET fcis-32134 102 37 end end NOUN fcis-32134 102 38 point point NOUN fcis-32134 102 39 after after ADP fcis-32134 102 40 taking take VERB fcis-32134 102 41 the the DET fcis-32134 102 42 whole whole ADJ fcis-32134 102 43 step step NOUN fcis-32134 102 44 length length NOUN fcis-32134 102 45 to to ADP fcis-32134 102 46 the the DET fcis-32134 102 47 total total ADJ fcis-32134 102 48 number number NOUN fcis-32134 102 49 of of ADP fcis-32134 102 50 rounds round NOUN fcis-32134 102 51 . . PUNCT fcis-32134 103 1 table table NOUN fcis-32134 103 2 3 3 NUM fcis-32134 103 3 . . PUNCT fcis-32134 103 4 algorithm algorithm NOUN fcis-32134 103 5 results result VERB fcis-32134 103 6 for for ADP fcis-32134 103 7 each each DET fcis-32134 103 8 metric metric NOUN fcis-32134 103 9 in in ADP fcis-32134 103 10 the the DET fcis-32134 103 11 test test NOUN fcis-32134 103 12 environment environment NOUN fcis-32134 103 13 algorithm algorithm NOUN fcis-32134 103 14 collision collision PROPN fcis-32134 103 15 rate rate NOUN fcis-32134 103 16 timeout timeout NOUN fcis-32134 103 17 rate rate NOUN fcis-32134 103 18 sac sac NOUN fcis-32134 103 19 28.50 28.50 NUM fcis-32134 103 20 % % NOUN fcis-32134 103 21 0.00 0.00 NUM fcis-32134 103 22 % % NOUN fcis-32134 103 23 td3 td3 NOUN fcis-32134 103 24 33.05 33.05 NUM fcis-32134 103 25 % % NOUN fcis-32134 103 26 6.15 6.15 NUM fcis-32134 103 27 % % NOUN fcis-32134 103 28 ddpg ddpg ADJ fcis-32134 103 29 34.60 34.60 NUM fcis-32134 103 30 % % NOUN fcis-32134 103 31 9.25 9.25 NUM fcis-32134 103 32 % % NOUN fcis-32134 103 33 from from ADP fcis-32134 103 34 table table NOUN fcis-32134 103 35 3 3 NUM fcis-32134 103 36 , , PUNCT fcis-32134 103 37 it it PRON fcis-32134 103 38 can can AUX fcis-32134 103 39 be be AUX fcis-32134 103 40 found find VERB fcis-32134 103 41 that that SCONJ fcis-32134 103 42 the the DET fcis-32134 103 43 collision collision NOUN fcis-32134 103 44 rate rate NOUN fcis-32134 103 45 of of ADP fcis-32134 103 46 sac sac NOUN fcis-32134 103 47 is be AUX fcis-32134 103 48 lower low ADJ fcis-32134 103 49 than than ADP fcis-32134 103 50 that that PRON fcis-32134 103 51 of of ADP fcis-32134 103 52 ddpg ddpg ADJ fcis-32134 103 53 algorithm algorithm NOUN fcis-32134 103 54 and and CCONJ fcis-32134 103 55 td3 td3 NOUN fcis-32134 103 56 algorithm algorithm NOUN fcis-32134 103 57 , , PUNCT fcis-32134 103 58 and and CCONJ fcis-32134 103 59 the the DET fcis-32134 103 60 timeout timeout NOUN fcis-32134 103 61 rate rate NOUN fcis-32134 103 62 is be AUX fcis-32134 103 63 even even ADV fcis-32134 103 64 0 0 NUM fcis-32134 103 65 , , PUNCT fcis-32134 103 66 which which PRON fcis-32134 103 67 indicates indicate VERB fcis-32134 103 68 that that SCONJ fcis-32134 103 69 sac sac NOUN fcis-32134 103 70 has have AUX fcis-32134 103 71 achieved achieve VERB fcis-32134 103 72 a a DET fcis-32134 103 73 good good ADJ fcis-32134 103 74 balance balance NOUN fcis-32134 103 75 between between ADP fcis-32134 103 76 environment environment NOUN fcis-32134 103 77 exploration exploration NOUN fcis-32134 103 78 and and CCONJ fcis-32134 103 79 obstacle obstacle NOUN fcis-32134 103 80 avoidance avoidance NOUN fcis-32134 103 81 decision decision NOUN fcis-32134 103 82 - - PUNCT fcis-32134 103 83 making making NOUN fcis-32134 103 84 , , PUNCT fcis-32134 103 85 and and CCONJ fcis-32134 103 86 further further ADJ fcis-32134 103 87 verifies verifie NOUN fcis-32134 103 88 that that SCONJ fcis-32134 103 89 the the DET fcis-32134 103 90 algorithm algorithm NOUN fcis-32134 103 91 's 's PART fcis-32134 103 92 maximum maximum ADJ fcis-32134 103 93 entropy entropy NOUN fcis-32134 103 94 strategy strategy NOUN fcis-32134 103 95 , , PUNCT fcis-32134 104 1 πœ€ πœ€ X fcis-32134 104 2 -greedy -greedy PROPN fcis-32134 104 3 exploration exploration NOUN fcis-32134 104 4 , , PUNCT fcis-32134 104 5 and and CCONJ fcis-32134 104 6 double double ADJ fcis-32134 104 7 q q ADJ fcis-32134 104 8 - - PUNCT fcis-32134 104 9 learning learn VERB fcis-32134 104 10 network network NOUN fcis-32134 104 11 design design NOUN fcis-32134 104 12 can can AUX fcis-32134 104 13 significantly significantly ADV fcis-32134 104 14 improve improve VERB fcis-32134 104 15 its its PRON fcis-32134 104 16 environment environment NOUN fcis-32134 104 17 adaptability adaptability NOUN fcis-32134 104 18 , , PUNCT fcis-32134 104 19 so so SCONJ fcis-32134 104 20 as as SCONJ fcis-32134 104 21 to to PART fcis-32134 104 22 find find VERB fcis-32134 104 23 the the DET fcis-32134 104 24 optimal optimal ADJ fcis-32134 104 25 strategy strategy NOUN fcis-32134 104 26 more more ADV fcis-32134 104 27 quickly quickly ADV fcis-32134 104 28 and and CCONJ fcis-32134 104 29 achieve achieve VERB fcis-32134 104 30 good good ADJ fcis-32134 104 31 training training NOUN fcis-32134 104 32 results result NOUN fcis-32134 104 33 . . PUNCT fcis-32134 105 1 5 5 X fcis-32134 105 2 . . X fcis-32134 105 3 conclusion conclusion NOUN fcis-32134 105 4 in in ADP fcis-32134 105 5 this this DET fcis-32134 105 6 paper paper NOUN fcis-32134 105 7 , , PUNCT fcis-32134 105 8 taking take VERB fcis-32134 105 9 the the DET fcis-32134 105 10 autonomous autonomous ADJ fcis-32134 105 11 uav uav PROPN fcis-32134 105 12 pathfinding pathfinding NOUN fcis-32134 105 13 and and CCONJ fcis-32134 105 14 obstacle obstacle NOUN fcis-32134 105 15 avoidance avoidance NOUN fcis-32134 105 16 task task NOUN fcis-32134 105 17 in in ADP fcis-32134 105 18 a a DET fcis-32134 105 19 three three NUM fcis-32134 105 20 - - PUNCT fcis-32134 105 21 dimensional dimensional ADJ fcis-32134 105 22 unknown unknown ADJ fcis-32134 105 23 environment environment NOUN fcis-32134 105 24 as as ADP fcis-32134 105 25 the the DET fcis-32134 105 26 research research NOUN fcis-32134 105 27 background background NOUN fcis-32134 105 28 , , PUNCT fcis-32134 105 29 the the DET fcis-32134 105 30 sac sac NOUN fcis-32134 105 31 algorithm algorithm NOUN fcis-32134 105 32 is be AUX fcis-32134 105 33 combined combine VERB fcis-32134 105 34 with with ADP fcis-32134 105 35 the the DET fcis-32134 105 36 dynamically dynamically ADV fcis-32134 105 37 decaying decay VERB fcis-32134 105 38 πœ€ πœ€ PRON fcis-32134 105 39 -greedy -greedy PROPN fcis-32134 105 40 exploration exploration NOUN fcis-32134 105 41 strategy strategy NOUN fcis-32134 105 42 and and CCONJ fcis-32134 105 43 the the DET fcis-32134 105 44 dynamic dynamic ADJ fcis-32134 105 45 learning learning NOUN fcis-32134 105 46 rate rate NOUN fcis-32134 105 47 adjustment adjustment NOUN fcis-32134 105 48 mechanism mechanism NOUN fcis-32134 105 49 , , PUNCT fcis-32134 105 50 and and CCONJ fcis-32134 105 51 a a DET fcis-32134 105 52 dynamic dynamic ADJ fcis-32134 105 53 obstacle obstacle NOUN fcis-32134 105 54 avoidance avoidance NOUN fcis-32134 105 55 algorithm algorithm NOUN fcis-32134 105 56 based base VERB fcis-32134 105 57 on on ADP fcis-32134 105 58 sac sac NOUN fcis-32134 105 59 for for ADP fcis-32134 105 60 uavs uavs NOUN fcis-32134 105 61 is be AUX fcis-32134 105 62 designed design VERB fcis-32134 105 63 . . PUNCT fcis-32134 106 1 simulation simulation NOUN fcis-32134 106 2 experiments experiment NOUN fcis-32134 106 3 show show VERB fcis-32134 106 4 that that SCONJ fcis-32134 106 5 this this DET fcis-32134 106 6 algorithm algorithm NOUN fcis-32134 106 7 can can AUX fcis-32134 106 8 effectively effectively ADV fcis-32134 106 9 cope cope VERB fcis-32134 106 10 with with ADP fcis-32134 106 11 the the DET fcis-32134 106 12 autonomous autonomous ADJ fcis-32134 106 13 obstacle obstacle NOUN fcis-32134 106 14 avoidance avoidance NOUN fcis-32134 106 15 task task NOUN fcis-32134 106 16 of of ADP fcis-32134 106 17 uavs uavs NOUN fcis-32134 106 18 in in ADP fcis-32134 106 19 complex complex ADJ fcis-32134 106 20 environments environment NOUN fcis-32134 106 21 , , PUNCT fcis-32134 106 22 especially especially ADV fcis-32134 106 23 in in ADP fcis-32134 106 24 multi multi ADJ fcis-32134 106 25 - - ADJ fcis-32134 106 26 dynamic dynamic ADJ fcis-32134 106 27 obstacle obstacle NOUN fcis-32134 106 28 environments environment NOUN fcis-32134 106 29 , , PUNCT fcis-32134 106 30 and and CCONJ fcis-32134 106 31 improves improve VERB fcis-32134 106 32 the the DET fcis-32134 106 33 training training NOUN fcis-32134 106 34 speed speed NOUN fcis-32134 106 35 and and CCONJ fcis-32134 106 36 stability stability NOUN fcis-32134 106 37 , , PUNCT fcis-32134 106 38 so so SCONJ fcis-32134 106 39 that that SCONJ fcis-32134 106 40 the the DET fcis-32134 106 41 maneuverability maneuverability NOUN fcis-32134 106 42 of of ADP fcis-32134 106 43 uavs uavs NOUN fcis-32134 106 44 can can AUX fcis-32134 106 45 be be AUX fcis-32134 106 46 effectively effectively ADV fcis-32134 106 47 utilized utilize VERB fcis-32134 106 48 . . PUNCT fcis-32134 107 1 although although SCONJ fcis-32134 107 2 the the DET fcis-32134 107 3 uav uav PROPN fcis-32134 107 4 can can AUX fcis-32134 107 5 complete complete VERB fcis-32134 107 6 the the DET fcis-32134 107 7 obstacle obstacle NOUN fcis-32134 107 8 avoidance avoidance NOUN fcis-32134 107 9 task task NOUN fcis-32134 107 10 well well INTJ fcis-32134 107 11 by by ADP fcis-32134 107 12 the the DET fcis-32134 107 13 sac sac NOUN fcis-32134 107 14 algorithm algorithm NOUN fcis-32134 107 15 , , PUNCT fcis-32134 107 16 the the DET fcis-32134 107 17 collision collision NOUN fcis-32134 107 18 rate rate NOUN fcis-32134 107 19 of of ADP fcis-32134 107 20 28.5 28.5 NUM fcis-32134 107 21 % % NOUN fcis-32134 107 22 is be AUX fcis-32134 107 23 still still ADV fcis-32134 107 24 high high ADJ fcis-32134 107 25 relative relative ADJ fcis-32134 107 26 to to ADP fcis-32134 107 27 the the DET fcis-32134 107 28 actual actual ADJ fcis-32134 107 29 application application NOUN fcis-32134 107 30 scenario scenario NOUN fcis-32134 107 31 , , PUNCT fcis-32134 107 32 in in ADP fcis-32134 107 33 addition addition NOUN fcis-32134 107 34 , , PUNCT fcis-32134 107 35 as as SCONJ fcis-32134 107 36 the the DET fcis-32134 107 37 performance performance NOUN fcis-32134 107 38 of of ADP fcis-32134 107 39 the the DET fcis-32134 107 40 algorithm algorithm NOUN fcis-32134 107 41 has have AUX fcis-32134 107 42 not not PART fcis-32134 107 43 been be AUX fcis-32134 107 44 validated validate VERB fcis-32134 107 45 in in ADP fcis-32134 107 46 real real ADJ fcis-32134 107 47 environments environment NOUN fcis-32134 107 48 , , PUNCT fcis-32134 107 49 its its PRON fcis-32134 107 50 generalization generalization NOUN fcis-32134 107 51 ability ability NOUN fcis-32134 107 52 still still ADV fcis-32134 107 53 needs need VERB fcis-32134 107 54 further further ADJ fcis-32134 107 55 evaluation evaluation NOUN fcis-32134 107 56 and and CCONJ fcis-32134 107 57 improvement improvement NOUN fcis-32134 107 58 . . PUNCT fcis-32134 108 1 in in ADP fcis-32134 108 2 the the DET fcis-32134 108 3 future future NOUN fcis-32134 108 4 , , PUNCT fcis-32134 108 5 it it PRON fcis-32134 108 6 can can AUX fcis-32134 108 7 be be AUX fcis-32134 108 8 further far ADV fcis-32134 108 9 tested test VERB fcis-32134 108 10 in in ADP fcis-32134 108 11 the the DET fcis-32134 108 12 real real ADJ fcis-32134 108 13 environment environment NOUN fcis-32134 108 14 , , PUNCT fcis-32134 108 15 and and CCONJ fcis-32134 108 16 a a DET fcis-32134 108 17 variety variety NOUN fcis-32134 108 18 of of ADP fcis-32134 108 19 environment environment NOUN fcis-32134 108 20 sensing sense VERB fcis-32134 108 21 methods method NOUN fcis-32134 108 22 can can AUX fcis-32134 108 23 be be AUX fcis-32134 108 24 combined combine VERB fcis-32134 108 25 with with ADP fcis-32134 108 26 this this DET fcis-32134 108 27 algorithm algorithm NOUN fcis-32134 108 28 to to PART fcis-32134 108 29 enhance enhance VERB fcis-32134 108 30 its its PRON fcis-32134 108 31 value value NOUN fcis-32134 108 32 for for ADP fcis-32134 108 33 practical practical ADJ fcis-32134 108 34 application application NOUN fcis-32134 108 35 . . PUNCT fcis-32134 109 1 references reference NOUN fcis-32134 109 2 [ [ X fcis-32134 109 3 1 1 NUM fcis-32134 109 4 ] ] X fcis-32134 109 5 tao tao PROPN fcis-32134 109 6 , , PUNCT fcis-32134 109 7 y. y. PROPN fcis-32134 109 8 , , PUNCT fcis-32134 109 9 & & CCONJ fcis-32134 109 10 li li PROPN fcis-32134 109 11 , , PUNCT fcis-32134 109 12 p. p. NOUN fcis-32134 109 13 ( ( PUNCT fcis-32134 109 14 2014 2014 NUM fcis-32134 109 15 ) ) PUNCT fcis-32134 109 16 . . PUNCT fcis-32134 110 1 an an DET fcis-32134 110 2 overview overview NOUN fcis-32134 110 3 of of ADP fcis-32134 110 4 unmanned unmanned ADJ fcis-32134 110 5 aircraft aircraft NOUN fcis-32134 110 6 system system NOUN fcis-32134 110 7 development development NOUN fcis-32134 110 8 and and CCONJ fcis-32134 110 9 key key ADJ fcis-32134 110 10 technologies technology NOUN fcis-32134 110 11 . . PUNCT fcis-32134 111 1 aeronautical aeronautical PROPN fcis-32134 111 2 manufacturing manufacturing NOUN fcis-32134 111 3 technology,57(20),34 technology,57(20),34 NOUN fcis-32134 111 4 - - PUNCT fcis-32134 111 5 39 39 NUM fcis-32134 111 6 . . PUNCT fcis-32134 112 1 [ [ X fcis-32134 112 2 2 2 NUM fcis-32134 112 3 ] ] X fcis-32134 112 4 liu liu PROPN fcis-32134 112 5 , , PUNCT fcis-32134 112 6 m. m. NOUN fcis-32134 112 7 , , PUNCT fcis-32134 112 8 & & CCONJ fcis-32134 112 9 shi shi PROPN fcis-32134 112 10 , , PUNCT fcis-32134 112 11 h. h. PROPN fcis-32134 112 12 ( ( PUNCT fcis-32134 112 13 2025 2025 NUM fcis-32134 112 14 ) ) PUNCT fcis-32134 112 15 . . PUNCT fcis-32134 113 1 autonomous autonomous ADJ fcis-32134 113 2 obstacle obstacle NOUN fcis-32134 113 3 avoidance avoidance NOUN fcis-32134 113 4 strategy strategy NOUN fcis-32134 113 5 for for ADP fcis-32134 113 6 uav uav PROPN fcis-32134 113 7 based base VERB fcis-32134 113 8 on on ADP fcis-32134 113 9 efre efre NOUN fcis-32134 113 10 - - PUNCT fcis-32134 113 11 sac sac NOUN fcis-32134 113 12 . . PUNCT fcis-32134 113 13 computer computer NOUN fcis-32134 113 14 system system NOUN fcis-32134 113 15 applications,34(06),53 applications,34(06),53 NOUN fcis-32134 113 16 - - PUNCT fcis-32134 113 17 61 61 NUM fcis-32134 113 18 . . PUNCT fcis-32134 114 1 [ [ X fcis-32134 114 2 3 3 NUM fcis-32134 114 3 ] ] X fcis-32134 114 4 wang wang PROPN fcis-32134 114 5 , , PUNCT fcis-32134 114 6 z. z. PROPN fcis-32134 114 7 , , PUNCT fcis-32134 114 8 & & CCONJ fcis-32134 114 9 xiang xiang PROPN fcis-32134 114 10 , , PUNCT fcis-32134 114 11 x. x. NOUN fcis-32134 114 12 ( ( PUNCT fcis-32134 114 13 2018 2018 NUM fcis-32134 114 14 ) ) PUNCT fcis-32134 114 15 . . PUNCT fcis-32134 115 1 improved improve VERB fcis-32134 115 2 astar astar NOUN fcis-32134 115 3 algorithm algorithm NOUN fcis-32134 115 4 for for ADP fcis-32134 115 5 path path NOUN fcis-32134 115 6 planning planning NOUN fcis-32134 115 7 of of ADP fcis-32134 115 8 marine marine ADJ fcis-32134 115 9 robot robot NOUN fcis-32134 115 10 . . PUNCT fcis-32134 116 1 in in ADP fcis-32134 116 2 2018 2018 NUM fcis-32134 116 3 37th 37th ADJ fcis-32134 116 4 chinese chinese ADJ fcis-32134 116 5 control control PROPN fcis-32134 116 6 conference conference PROPN fcis-32134 116 7 ( ( PUNCT fcis-32134 116 8 ccc ccc PROPN fcis-32134 116 9 ) ) PUNCT fcis-32134 116 10 , , PUNCT fcis-32134 116 11 ieee ieee NOUN fcis-32134 116 12 , , PUNCT fcis-32134 116 13 5410 5410 NUM fcis-32134 116 14 - - SYM fcis-32134 116 15 5414 5414 NUM fcis-32134 116 16 . . PUNCT fcis-32134 117 1 [ [ X fcis-32134 117 2 4 4 NUM fcis-32134 117 3 ] ] X fcis-32134 117 4 prasad prasad PROPN fcis-32134 117 5 , , PUNCT fcis-32134 117 6 n. n. PROPN fcis-32134 117 7 l. l. PROPN fcis-32134 117 8 , , PUNCT fcis-32134 117 9 & & CCONJ fcis-32134 117 10 ramkumar ramkumar PROPN fcis-32134 117 11 , , PUNCT fcis-32134 117 12 b. b. PROPN fcis-32134 117 13 ( ( PUNCT fcis-32134 117 14 2022 2022 NUM fcis-32134 117 15 ) ) PUNCT fcis-32134 117 16 . . PUNCT fcis-32134 118 1 3 3 NUM fcis-32134 118 2 - - SYM fcis-32134 118 3 d d NOUN fcis-32134 118 4 deployment deployment NOUN fcis-32134 118 5 and and CCONJ fcis-32134 118 6 trajectory trajectory NOUN fcis-32134 118 7 planning planning NOUN fcis-32134 118 8 for for ADP fcis-32134 118 9 relay relay NOUN fcis-32134 118 10 based base VERB fcis-32134 118 11 uav uav PROPN fcis-32134 118 12 assisted assist VERB fcis-32134 118 13 cooperative cooperative ADJ fcis-32134 118 14 communication communication NOUN fcis-32134 118 15 for for ADP fcis-32134 118 16 emergency emergency NOUN fcis-32134 118 17 scenarios scenario NOUN fcis-32134 118 18 using use VERB fcis-32134 118 19 dijkstra dijkstra PROPN fcis-32134 118 20 's 's PART fcis-32134 118 21 algorithm algorithm NOUN fcis-32134 118 22 . . PUNCT fcis-32134 119 1 ieee ieee NOUN fcis-32134 119 2 transactions transaction NOUN fcis-32134 119 3 on on ADP fcis-32134 119 4 vehicular vehicular ADJ fcis-32134 119 5 technology technology NOUN fcis-32134 119 6 , , PUNCT fcis-32134 119 7 72(4 72(4 NUM fcis-32134 119 8 ) ) PUNCT fcis-32134 119 9 , , PUNCT fcis-32134 119 10 5049 5049 NUM fcis-32134 119 11 - - SYM fcis-32134 119 12 5063 5063 NUM fcis-32134 119 13 . . PUNCT fcis-32134 120 1 [ [ X fcis-32134 120 2 5 5 NUM fcis-32134 120 3 ] ] SYM fcis-32134 120 4 du du X fcis-32134 120 5 , , PUNCT fcis-32134 120 6 z. z. PROPN fcis-32134 120 7 , , PUNCT fcis-32134 120 8 & & CCONJ fcis-32134 120 9 liu liu PROPN fcis-32134 120 10 , , PUNCT fcis-32134 120 11 s. s. PROPN fcis-32134 120 12 ( ( PUNCT fcis-32134 120 13 2018 2018 NUM fcis-32134 120 14 ) ) PUNCT fcis-32134 120 15 . . PUNCT fcis-32134 121 1 asymptotical asymptotical ADJ fcis-32134 121 2 rrt rrt NOUN fcis-32134 121 3 - - PUNCT fcis-32134 121 4 based base VERB fcis-32134 121 5 path path NOUN fcis-32134 121 6 planning planning NOUN fcis-32134 121 7 for for ADP fcis-32134 121 8 mobile mobile ADJ fcis-32134 121 9 robots robot NOUN fcis-32134 121 10 in in ADP fcis-32134 121 11 dynamic dynamic ADJ fcis-32134 121 12 environments environment NOUN fcis-32134 121 13 . . PUNCT fcis-32134 122 1 in in ADP fcis-32134 122 2 2018 2018 NUM fcis-32134 122 3 37th 37th ADJ fcis-32134 122 4 chinese chinese ADJ fcis-32134 122 5 control control PROPN fcis-32134 122 6 conference conference PROPN fcis-32134 122 7 ( ( PUNCT fcis-32134 122 8 ccc ccc PROPN fcis-32134 122 9 ) ) PUNCT fcis-32134 122 10 , , PUNCT fcis-32134 122 11 ieee ieee NOUN fcis-32134 122 12 , , PUNCT fcis-32134 122 13 5281 5281 NUM fcis-32134 122 14 - - SYM fcis-32134 122 15 5286 5286 NUM fcis-32134 122 16 . . PUNCT fcis-32134 123 1 [ [ X fcis-32134 123 2 6 6 NUM fcis-32134 123 3 ] ] X fcis-32134 123 4 chen chen PROPN fcis-32134 123 5 , , PUNCT fcis-32134 123 6 h. h. PROPN fcis-32134 123 7 , , PUNCT fcis-32134 123 8 chen chen PROPN fcis-32134 123 9 , , PUNCT fcis-32134 123 10 h. h. PROPN fcis-32134 123 11 , , PUNCT fcis-32134 123 12 & & CCONJ fcis-32134 123 13 qiang qiang PROPN fcis-32134 123 14 , , PUNCT fcis-32134 123 15 l. l. PROPN fcis-32134 123 16 ( ( PUNCT fcis-32134 123 17 2020 2020 NUM fcis-32134 123 18 ) ) PUNCT fcis-32134 123 19 . . PUNCT fcis-32134 124 1 multi multi PROPN fcis-32134 124 2 - - PROPN fcis-32134 124 3 uav uav ADJ fcis-32134 124 4 3d 3d PROPN fcis-32134 124 5 formation formation NOUN fcis-32134 124 6 path path NOUN fcis-32134 124 7 planning planning NOUN fcis-32134 124 8 based base VERB fcis-32134 124 9 on on ADP fcis-32134 124 10 improved improve VERB fcis-32134 124 11 artificial artificial ADJ fcis-32134 124 12 potential potential ADJ fcis-32134 124 13 field field NOUN fcis-32134 124 14 . . PUNCT fcis-32134 125 1 journal journal NOUN fcis-32134 125 2 of of ADP fcis-32134 125 3 system system NOUN fcis-32134 125 4 simulation simulation NOUN fcis-32134 125 5 , , PUNCT fcis-32134 125 6 32(3 32(3 NUM fcis-32134 125 7 ) ) PUNCT fcis-32134 125 8 , , PUNCT fcis-32134 125 9 414 414 NUM fcis-32134 125 10 - - SYM fcis-32134 125 11 420 420 NUM fcis-32134 125 12 . . PUNCT fcis-32134 126 1 48 48 NUM fcis-32134 127 1 [ [ X fcis-32134 127 2 7 7 NUM fcis-32134 127 3 ] ] X fcis-32134 127 4 sonny sonny PROPN fcis-32134 127 5 , , PUNCT fcis-32134 127 6 a. a. PROPN fcis-32134 127 7 , , PUNCT fcis-32134 127 8 yeduri yeduri PROPN fcis-32134 127 9 , , PUNCT fcis-32134 127 10 s. s. PROPN fcis-32134 127 11 r. r. PROPN fcis-32134 127 12 , , PUNCT fcis-32134 127 13 & & CCONJ fcis-32134 127 14 cenkeramaddi cenkeramaddi PROPN fcis-32134 127 15 , , PUNCT fcis-32134 127 16 l. l. PROPN fcis-32134 127 17 r. r. PROPN fcis-32134 127 18 ( ( PUNCT fcis-32134 127 19 2023 2023 NUM fcis-32134 127 20 ) ) PUNCT fcis-32134 127 21 . . PUNCT fcis-32134 128 1 qlearning qlearning NOUN fcis-32134 128 2 - - PUNCT fcis-32134 128 3 based base VERB fcis-32134 128 4 unmanned unmanned ADJ fcis-32134 128 5 aerial aerial ADJ fcis-32134 128 6 vehicle vehicle NOUN fcis-32134 128 7 path path NOUN fcis-32134 128 8 planning planning NOUN fcis-32134 128 9 with with ADP fcis-32134 128 10 dynamic dynamic ADJ fcis-32134 128 11 obstacle obstacle NOUN fcis-32134 128 12 avoidance avoidance NOUN fcis-32134 128 13 . . PUNCT fcis-32134 129 1 applied apply VERB fcis-32134 129 2 soft soft ADJ fcis-32134 129 3 computing computing NOUN fcis-32134 129 4 , , PUNCT fcis-32134 129 5 147 147 NUM fcis-32134 129 6 , , PUNCT fcis-32134 129 7 110773 110773 NUM fcis-32134 129 8 . . PUNCT fcis-32134 130 1 [ [ X fcis-32134 130 2 8 8 NUM fcis-32134 130 3 ] ] SYM fcis-32134 130 4 qi qi PROPN fcis-32134 130 5 , , PUNCT fcis-32134 130 6 c. c. PROPN fcis-32134 130 7 , , PUNCT fcis-32134 130 8 wu wu PROPN fcis-32134 130 9 , , PUNCT fcis-32134 130 10 c. c. PROPN fcis-32134 130 11 , , PUNCT fcis-32134 130 12 lei lei PROPN fcis-32134 130 13 , , PUNCT fcis-32134 130 14 l. l. PROPN fcis-32134 130 15 , , PUNCT fcis-32134 130 16 li li PROPN fcis-32134 130 17 , , PUNCT fcis-32134 130 18 x. x. NOUN fcis-32134 130 19 , , PUNCT fcis-32134 130 20 & & CCONJ fcis-32134 130 21 cong cong PROPN fcis-32134 130 22 , , PUNCT fcis-32134 130 23 p. p. NOUN fcis-32134 130 24 ( ( PUNCT fcis-32134 130 25 2022 2022 NUM fcis-32134 130 26 ) ) PUNCT fcis-32134 130 27 . . PUNCT fcis-32134 131 1 uav uav PROPN fcis-32134 131 2 path path PROPN fcis-32134 131 3 planning planning NOUN fcis-32134 131 4 based base VERB fcis-32134 131 5 on on ADP fcis-32134 131 6 the the DET fcis-32134 131 7 improved improved ADJ fcis-32134 131 8 ppo ppo PROPN fcis-32134 131 9 algorithm algorithm NOUN fcis-32134 131 10 . . PUNCT fcis-32134 132 1 in in ADP fcis-32134 132 2 2022 2022 NUM fcis-32134 132 3 asia asia PROPN fcis-32134 132 4 conference conference PROPN fcis-32134 132 5 on on ADP fcis-32134 132 6 advanced advanced ADJ fcis-32134 132 7 robotics robotic NOUN fcis-32134 132 8 , , PUNCT fcis-32134 132 9 automation automation NOUN fcis-32134 132 10 , , PUNCT fcis-32134 132 11 and and CCONJ fcis-32134 132 12 control control NOUN fcis-32134 132 13 engineering engineering NOUN fcis-32134 132 14 ( ( PUNCT fcis-32134 132 15 arace arace NOUN fcis-32134 132 16 ) ) PUNCT fcis-32134 132 17 , , PUNCT fcis-32134 132 18 ieee ieee NOUN fcis-32134 132 19 , , PUNCT fcis-32134 132 20 193 193 NUM fcis-32134 132 21 - - SYM fcis-32134 132 22 199 199 NUM fcis-32134 132 23 . . PUNCT fcis-32134 133 1 [ [ X fcis-32134 133 2 9 9 NUM fcis-32134 133 3 ] ] PUNCT fcis-32134 133 4 bouhamed bouhame VERB fcis-32134 133 5 , , PUNCT fcis-32134 133 6 o. o. INTJ fcis-32134 133 7 , , PUNCT fcis-32134 133 8 ghazzai ghazzai PROPN fcis-32134 133 9 , , PUNCT fcis-32134 133 10 h. h. PROPN fcis-32134 133 11 , , PUNCT fcis-32134 133 12 besbes besbes PROPN fcis-32134 133 13 , , PUNCT fcis-32134 133 14 h. h. PROPN fcis-32134 133 15 , , PUNCT fcis-32134 133 16 & & CCONJ fcis-32134 133 17 massoud massoud PROPN fcis-32134 133 18 , , PUNCT fcis-32134 133 19 y. y. PROPN fcis-32134 133 20 ( ( PUNCT fcis-32134 133 21 2020 2020 NUM fcis-32134 133 22 ) ) PUNCT fcis-32134 133 23 . . PUNCT fcis-32134 134 1 autonomous autonomous PROPN fcis-32134 134 2 uav uav PROPN fcis-32134 134 3 navigation navigation PROPN fcis-32134 134 4 : : PUNCT fcis-32134 134 5 a a DET fcis-32134 134 6 ddpg ddpg ADJ fcis-32134 134 7 - - PUNCT fcis-32134 134 8 based base VERB fcis-32134 134 9 deep deep ADJ fcis-32134 134 10 reinforcement reinforcement NOUN fcis-32134 134 11 learning learning NOUN fcis-32134 134 12 approach approach NOUN fcis-32134 134 13 . . PUNCT fcis-32134 135 1 in in ADP fcis-32134 135 2 2020 2020 NUM fcis-32134 135 3 ieee ieee NOUN fcis-32134 135 4 international international ADJ fcis-32134 135 5 symposium symposium NOUN fcis-32134 135 6 on on ADP fcis-32134 135 7 circuits circuit NOUN fcis-32134 135 8 and and CCONJ fcis-32134 135 9 systems system NOUN fcis-32134 135 10 ( ( PUNCT fcis-32134 135 11 iscas iscas PROPN fcis-32134 135 12 ) ) PUNCT fcis-32134 135 13 , , PUNCT fcis-32134 135 14 ieee ieee NOUN fcis-32134 135 15 , , PUNCT fcis-32134 135 16 1 1 NUM fcis-32134 135 17 - - SYM fcis-32134 135 18 5 5 NUM fcis-32134 135 19 . . PUNCT fcis-32134 136 1 [ [ X fcis-32134 136 2 10 10 NUM fcis-32134 136 3 ] ] X fcis-32134 136 4 luo luo PROPN fcis-32134 136 5 , , PUNCT fcis-32134 136 6 x. x. PROPN fcis-32134 136 7 , , PUNCT fcis-32134 136 8 wang wang PROPN fcis-32134 136 9 , , PUNCT fcis-32134 136 10 q. q. PROPN fcis-32134 136 11 , , PUNCT fcis-32134 136 12 gong gong PROPN fcis-32134 136 13 , , PUNCT fcis-32134 136 14 h. h. PROPN fcis-32134 136 15 , , PUNCT fcis-32134 136 16 & & CCONJ fcis-32134 136 17 tang tang PROPN fcis-32134 136 18 , , PUNCT fcis-32134 136 19 c. c. PROPN fcis-32134 136 20 ( ( PUNCT fcis-32134 136 21 2024 2024 NUM fcis-32134 136 22 ) ) PUNCT fcis-32134 136 23 . . PUNCT fcis-32134 137 1 uav uav PROPN fcis-32134 137 2 path path PROPN fcis-32134 137 3 planning planning NOUN fcis-32134 137 4 based base VERB fcis-32134 137 5 on on ADP fcis-32134 137 6 the the DET fcis-32134 137 7 average average ADJ fcis-32134 137 8 td3 td3 NOUN fcis-32134 137 9 algorithm algorithm NOUN fcis-32134 137 10 with with ADP fcis-32134 137 11 prioritized prioritize VERB fcis-32134 137 12 experience experience NOUN fcis-32134 137 13 replay replay NOUN fcis-32134 137 14 . . PUNCT fcis-32134 138 1 ieee ieee NOUN fcis-32134 138 2 access access NOUN fcis-32134 138 3 , , PUNCT fcis-32134 138 4 12 12 NUM fcis-32134 138 5 , , PUNCT fcis-32134 138 6 38017 38017 NUM fcis-32134 138 7 - - SYM fcis-32134 138 8 38029 38029 NUM fcis-32134 138 9 . . PUNCT fcis-32134 139 1 [ [ X fcis-32134 139 2 11 11 NUM fcis-32134 139 3 ] ] X fcis-32134 139 4 xue xue PROPN fcis-32134 139 5 , , PUNCT fcis-32134 139 6 b. b. PROPN fcis-32134 139 7 , , PUNCT fcis-32134 139 8 zhou zhou PROPN fcis-32134 139 9 , , PUNCT fcis-32134 139 10 f. f. PROPN fcis-32134 139 11 , , PUNCT fcis-32134 139 12 wang wang PROPN fcis-32134 139 13 , , PUNCT fcis-32134 139 14 c. c. PROPN fcis-32134 139 15 , , PUNCT fcis-32134 139 16 gao gao PROPN fcis-32134 139 17 , , PUNCT fcis-32134 139 18 m. m. NOUN fcis-32134 139 19 , , PUNCT fcis-32134 139 20 & & CCONJ fcis-32134 139 21 yin yin PROPN fcis-32134 139 22 , , PUNCT fcis-32134 139 23 l. l. PROPN fcis-32134 139 24 ( ( PUNCT fcis-32134 139 25 2024 2024 NUM fcis-32134 139 26 ) ) PUNCT fcis-32134 139 27 . . PUNCT fcis-32134 140 1 robot robot NOUN fcis-32134 140 2 mapless mapless PROPN fcis-32134 140 3 navigation navigation NOUN fcis-32134 140 4 in in ADP fcis-32134 140 5 vuca vuca ADJ fcis-32134 140 6 environments environment NOUN fcis-32134 140 7 via via ADP fcis-32134 140 8 deep deep ADJ fcis-32134 140 9 reinforcement reinforcement NOUN fcis-32134 140 10 learning learning NOUN fcis-32134 140 11 . . PUNCT fcis-32134 141 1 ieee ieee NOUN fcis-32134 141 2 transactions transaction NOUN fcis-32134 141 3 on on ADP fcis-32134 141 4 industrial industrial ADJ fcis-32134 141 5 electronics electronic NOUN fcis-32134 141 6 , , PUNCT fcis-32134 141 7 72(1 72(1 NOUN fcis-32134 141 8 ) ) PUNCT fcis-32134 141 9 , , PUNCT fcis-32134 141 10 639 639 NUM fcis-32134 141 11 - - SYM fcis-32134 141 12 649 649 NUM fcis-32134 141 13 . . PUNCT fcis-32134 142 1 [ [ X fcis-32134 142 2 12 12 NUM fcis-32134 142 3 ] ] X fcis-32134 142 4 haarnoja haarnoja NOUN fcis-32134 142 5 , , PUNCT fcis-32134 142 6 t. t. PROPN fcis-32134 142 7 , , PUNCT fcis-32134 142 8 zhou zhou PROPN fcis-32134 142 9 , , PUNCT fcis-32134 142 10 a. a. PROPN fcis-32134 142 11 , , PUNCT fcis-32134 142 12 hartikainen hartikainen PROPN fcis-32134 142 13 , , PUNCT fcis-32134 142 14 k. k. PROPN fcis-32134 142 15 , , PUNCT fcis-32134 142 16 tucker tucker PROPN fcis-32134 142 17 , , PUNCT fcis-32134 142 18 g. g. PROPN fcis-32134 142 19 , , PUNCT fcis-32134 142 20 ha ha INTJ fcis-32134 142 21 , , PUNCT fcis-32134 142 22 s. s. PROPN fcis-32134 142 23 , , PUNCT fcis-32134 142 24 tan tan PROPN fcis-32134 142 25 , , PUNCT fcis-32134 142 26 j. j. PROPN fcis-32134 142 27 , , PUNCT fcis-32134 142 28 ... ... PUNCT fcis-32134 142 29 & & CCONJ fcis-32134 142 30 levine levine PROPN fcis-32134 142 31 , , PUNCT fcis-32134 142 32 s. s. PROPN fcis-32134 142 33 ( ( PUNCT fcis-32134 142 34 2018 2018 NUM fcis-32134 142 35 ) ) PUNCT fcis-32134 142 36 . . PUNCT fcis-32134 143 1 soft soft ADJ fcis-32134 143 2 actor actor NOUN fcis-32134 143 3 - - PUNCT fcis-32134 143 4 critic critic NOUN fcis-32134 143 5 algorithms algorithm NOUN fcis-32134 143 6 and and CCONJ fcis-32134 143 7 applications application NOUN fcis-32134 143 8 . . PUNCT fcis-32134 144 1 arxiv arxiv PROPN fcis-32134 144 2 preprint preprint PROPN fcis-32134 144 3 arxiv:1812.05905 arxiv:1812.05905 NOUN fcis-32134 144 4 . . PUNCT fcis-32134 145 1 [ [ X fcis-32134 145 2 13 13 NUM fcis-32134 145 3 ] ] SYM fcis-32134 145 4 haarnoja haarnoja NOUN fcis-32134 145 5 , , PUNCT fcis-32134 145 6 t. t. PROPN fcis-32134 145 7 , , PUNCT fcis-32134 145 8 zhou zhou PROPN fcis-32134 145 9 , , PUNCT fcis-32134 145 10 a. a. NOUN fcis-32134 145 11 , , PUNCT fcis-32134 145 12 abbeel abbeel NOUN fcis-32134 145 13 , , PUNCT fcis-32134 145 14 p. p. NOUN fcis-32134 145 15 , , PUNCT fcis-32134 145 16 & & CCONJ fcis-32134 145 17 levine levine PROPN fcis-32134 145 18 , , PUNCT fcis-32134 145 19 s. s. PROPN fcis-32134 145 20 ( ( PUNCT fcis-32134 145 21 2018 2018 NUM fcis-32134 145 22 ) ) PUNCT fcis-32134 145 23 . . PUNCT fcis-32134 146 1 soft soft ADJ fcis-32134 146 2 actor actor NOUN fcis-32134 146 3 - - PUNCT fcis-32134 146 4 critic critic NOUN fcis-32134 146 5 : : PUNCT fcis-32134 146 6 off off ADP fcis-32134 146 7 - - PUNCT fcis-32134 146 8 policy policy NOUN fcis-32134 146 9 maximum maximum ADJ fcis-32134 146 10 entropy entropy NOUN fcis-32134 146 11 deep deep ADJ fcis-32134 146 12 reinforcement reinforcement NOUN fcis-32134 146 13 learning learning NOUN fcis-32134 146 14 with with ADP fcis-32134 146 15 a a DET fcis-32134 146 16 stochastic stochastic ADJ fcis-32134 146 17 actor actor NOUN fcis-32134 146 18 . . PUNCT fcis-32134 147 1 in in ADP fcis-32134 147 2 international international ADJ fcis-32134 147 3 conference conference NOUN fcis-32134 147 4 on on ADP fcis-32134 147 5 machine machine NOUN fcis-32134 147 6 learning learning NOUN fcis-32134 147 7 , , PUNCT fcis-32134 147 8 pmlr pmlr NOUN fcis-32134 147 9 , , PUNCT fcis-32134 147 10 1861 1861 NUM fcis-32134 147 11 - - SYM fcis-32134 147 12 1870 1870 NUM fcis-32134 147 13 . . PUNCT fcis-32134 148 1 [ [ X fcis-32134 148 2 14 14 NUM fcis-32134 148 3 ] ] PUNCT fcis-32134 148 4 lillicrap lillicrap NOUN fcis-32134 148 5 , , PUNCT fcis-32134 148 6 t. t. PROPN fcis-32134 148 7 p. p. PROPN fcis-32134 148 8 , , PUNCT fcis-32134 148 9 hunt hunt PROPN fcis-32134 148 10 , , PUNCT fcis-32134 148 11 j. j. PROPN fcis-32134 148 12 j. j. PROPN fcis-32134 148 13 , , PUNCT fcis-32134 148 14 pritzel pritzel NOUN fcis-32134 148 15 , , PUNCT fcis-32134 148 16 a. a. NOUN fcis-32134 148 17 , , PUNCT fcis-32134 148 18 heess heess PROPN fcis-32134 148 19 , , PUNCT fcis-32134 148 20 n. n. NOUN fcis-32134 148 21 , , PUNCT fcis-32134 148 22 erez erez PROPN fcis-32134 148 23 , , PUNCT fcis-32134 148 24 t. t. PROPN fcis-32134 148 25 , , PUNCT fcis-32134 148 26 tassa tassa PROPN fcis-32134 148 27 , , PUNCT fcis-32134 148 28 y. y. PROPN fcis-32134 148 29 , , PUNCT fcis-32134 148 30 ... ... PUNCT fcis-32134 148 31 & & CCONJ fcis-32134 148 32 wierstra wierstra PROPN fcis-32134 148 33 , , PUNCT fcis-32134 148 34 d. d. PROPN fcis-32134 148 35 ( ( PUNCT fcis-32134 148 36 2015 2015 NUM fcis-32134 148 37 ) ) PUNCT fcis-32134 148 38 . . PUNCT fcis-32134 149 1 continuous continuous ADJ fcis-32134 149 2 control control NOUN fcis-32134 149 3 with with ADP fcis-32134 149 4 deep deep ADJ fcis-32134 149 5 reinforcement reinforcement NOUN fcis-32134 149 6 learning learning NOUN fcis-32134 149 7 . . PUNCT fcis-32134 150 1 arxiv arxiv PROPN fcis-32134 150 2 preprint preprint NOUN fcis-32134 150 3 arxiv:1509.02971 arxiv:1509.02971 PROPN fcis-32134 150 4 . . PUNCT fcis-32134 151 1 [ [ X fcis-32134 151 2 15 15 NUM fcis-32134 151 3 ] ] X fcis-32134 151 4 fujimoto fujimoto PROPN fcis-32134 151 5 , , PUNCT fcis-32134 151 6 s. s. PROPN fcis-32134 151 7 , , PUNCT fcis-32134 151 8 hoof hoof NOUN fcis-32134 151 9 , , PUNCT fcis-32134 151 10 h. h. PROPN fcis-32134 151 11 , , PUNCT fcis-32134 151 12 & & CCONJ fcis-32134 151 13 meger meger PROPN fcis-32134 151 14 , , PUNCT fcis-32134 151 15 d. d. PROPN fcis-32134 151 16 ( ( PUNCT fcis-32134 151 17 2018 2018 NUM fcis-32134 151 18 ) ) PUNCT fcis-32134 151 19 . . PUNCT fcis-32134 152 1 addressing address VERB fcis-32134 152 2 function function NOUN fcis-32134 152 3 approximation approximation NOUN fcis-32134 152 4 error error NOUN fcis-32134 152 5 in in ADP fcis-32134 152 6 actor actor NOUN fcis-32134 152 7 - - PUNCT fcis-32134 152 8 critic critic NOUN fcis-32134 152 9 methods method NOUN fcis-32134 152 10 . . PUNCT fcis-32134 153 1 in in ADP fcis-32134 153 2 international international ADJ fcis-32134 153 3 conference conference NOUN fcis-32134 153 4 on on ADP fcis-32134 153 5 machine machine NOUN fcis-32134 153 6 learning learning NOUN fcis-32134 153 7 , , PUNCT fcis-32134 153 8 pmlr pmlr NOUN fcis-32134 153 9 , , PUNCT fcis-32134 153 10 15871596 15871596 NUM fcis-32134 153 11 . . PUNCT