id sid tid token lemma pos fcis-1076 1 1 frontiers frontier NOUN fcis-1076 1 2 in in ADP fcis-1076 1 3 computing computing NOUN fcis-1076 1 4 and and CCONJ fcis-1076 1 5 intelligent intelligent ADJ fcis-1076 1 6 systems system NOUN fcis-1076 1 7 issn issn VERB fcis-1076 1 8 : : PUNCT fcis-1076 1 9 2832 2832 NUM fcis-1076 1 10 - - SYM fcis-1076 1 11 6024 6024 NUM fcis-1076 1 12 | | NOUN fcis-1076 1 13 vol vol NOUN fcis-1076 1 14 . . PROPN fcis-1076 2 1 1 1 NUM fcis-1076 2 2 , , PUNCT fcis-1076 2 3 no no INTJ fcis-1076 2 4 . . NOUN fcis-1076 2 5 1 1 NUM fcis-1076 2 6 , , PUNCT fcis-1076 2 7 2022 2022 NUM fcis-1076 2 8 13 13 NUM fcis-1076 2 9 research research NOUN fcis-1076 2 10 on on ADP fcis-1076 2 11 multimodal multimodal NOUN fcis-1076 2 12 summarization summarization NOUN fcis-1076 2 13 based base VERB fcis-1076 2 14 on on ADP fcis-1076 2 15 sequence sequence NOUN fcis-1076 2 16 to to ADP fcis-1076 2 17 sequence sequence NOUN fcis-1076 2 18 model model NOUN fcis-1076 2 19 qiduo qiduo ADV fcis-1076 2 20 lu lu PROPN fcis-1076 2 21 * * NOUN fcis-1076 2 22 , , PUNCT fcis-1076 2 23 xia xia PROPN fcis-1076 2 24 ye ye PROPN fcis-1076 2 25 and and CCONJ fcis-1076 2 26 chenhao chenhao PROPN fcis-1076 2 27 zhu zhu PROPN fcis-1076 2 28 academy academy PROPN fcis-1076 2 29 of of ADP fcis-1076 2 30 combat combat PROPN fcis-1076 2 31 support support NOUN fcis-1076 2 32 rocket rocket NOUN fcis-1076 2 33 force force NOUN fcis-1076 2 34 university university PROPN fcis-1076 2 35 of of ADP fcis-1076 2 36 engineer engineer NOUN fcis-1076 2 37 , , PUNCT fcis-1076 2 38 xi’an xi’an PROPN fcis-1076 2 39 , , PUNCT fcis-1076 2 40 china china PROPN fcis-1076 2 41 abstract abstract PROPN fcis-1076 2 42 : : PUNCT fcis-1076 2 43 with with ADP fcis-1076 2 44 the the DET fcis-1076 2 45 rapid rapid ADJ fcis-1076 2 46 development development NOUN fcis-1076 2 47 of of ADP fcis-1076 2 48 information information NOUN fcis-1076 2 49 technology technology NOUN fcis-1076 2 50 in in ADP fcis-1076 2 51 the the DET fcis-1076 2 52 21st 21st ADJ fcis-1076 2 53 century century NOUN fcis-1076 2 54 , , PUNCT fcis-1076 2 55 the the DET fcis-1076 2 56 data datum NOUN fcis-1076 2 57 on on ADP fcis-1076 2 58 the the DET fcis-1076 2 59 internet internet NOUN fcis-1076 2 60 is be AUX fcis-1076 2 61 growing grow VERB fcis-1076 2 62 exponentially exponentially ADV fcis-1076 2 63 , , PUNCT fcis-1076 2 64 which which PRON fcis-1076 2 65 makes make VERB fcis-1076 2 66 it it PRON fcis-1076 2 67 difficult difficult ADJ fcis-1076 2 68 for for SCONJ fcis-1076 2 69 users user NOUN fcis-1076 2 70 to to PART fcis-1076 2 71 receive receive VERB fcis-1076 2 72 useful useful ADJ fcis-1076 2 73 information information NOUN fcis-1076 2 74 from from ADP fcis-1076 2 75 an an DET fcis-1076 2 76 endless endless ADJ fcis-1076 2 77 stream stream NOUN fcis-1076 2 78 of of ADP fcis-1076 2 79 information information NOUN fcis-1076 2 80 sources source NOUN fcis-1076 2 81 . . PUNCT fcis-1076 3 1 therefore therefore ADV fcis-1076 3 2 , , PUNCT fcis-1076 3 3 as as ADP fcis-1076 3 4 a a DET fcis-1076 3 5 kind kind NOUN fcis-1076 3 6 of of ADP fcis-1076 3 7 data data NOUN fcis-1076 3 8 compression compression NOUN fcis-1076 3 9 task task NOUN fcis-1076 3 10 , , PUNCT fcis-1076 3 11 automatic automatic ADJ fcis-1076 3 12 summarization summarization NOUN fcis-1076 3 13 technology technology NOUN fcis-1076 3 14 has have AUX fcis-1076 3 15 gradually gradually ADV fcis-1076 3 16 become become VERB fcis-1076 3 17 a a DET fcis-1076 3 18 research research NOUN fcis-1076 3 19 hotspot hotspot NOUN fcis-1076 3 20 in in ADP fcis-1076 3 21 academia academia NOUN fcis-1076 3 22 and and CCONJ fcis-1076 3 23 industry industry NOUN fcis-1076 3 24 . . PUNCT fcis-1076 4 1 this this DET fcis-1076 4 2 paper paper NOUN fcis-1076 4 3 mainly mainly ADV fcis-1076 4 4 introduces introduce VERB fcis-1076 4 5 the the DET fcis-1076 4 6 multimodal multimodal NOUN fcis-1076 4 7 summarization summarization NOUN fcis-1076 4 8 method method NOUN fcis-1076 4 9 from from ADP fcis-1076 4 10 sequence sequence NOUN fcis-1076 4 11 to to ADP fcis-1076 4 12 sequence sequence NOUN fcis-1076 4 13 model model NOUN fcis-1076 4 14 , , PUNCT fcis-1076 4 15 then then ADV fcis-1076 4 16 introduces introduce VERB fcis-1076 4 17 the the DET fcis-1076 4 18 current current ADJ fcis-1076 4 19 multimodal multimodal NOUN fcis-1076 4 20 summarization summarization NOUN fcis-1076 4 21 evaluation evaluation NOUN fcis-1076 4 22 methods method NOUN fcis-1076 4 23 and and CCONJ fcis-1076 4 24 finally finally ADV fcis-1076 4 25 gives give VERB fcis-1076 4 26 some some DET fcis-1076 4 27 conclusions conclusion NOUN fcis-1076 4 28 and and CCONJ fcis-1076 4 29 thoughts thought NOUN fcis-1076 4 30 . . PUNCT fcis-1076 5 1 keywords keyword NOUN fcis-1076 5 2 : : PUNCT fcis-1076 5 3 multimodal multimodal NOUN fcis-1076 5 4 summarization summarization NOUN fcis-1076 5 5 , , PUNCT fcis-1076 5 6 sequence sequence NOUN fcis-1076 5 7 to to ADP fcis-1076 5 8 sequence sequence NOUN fcis-1076 5 9 model model NOUN fcis-1076 5 10 , , PUNCT fcis-1076 5 11 attention attention NOUN fcis-1076 5 12 mechanism mechanism NOUN fcis-1076 5 13 . . PUNCT fcis-1076 6 1 1 1 X fcis-1076 6 2 . . X fcis-1076 6 3 introduction introduction NOUN fcis-1076 6 4 multimodal multimodal NOUN fcis-1076 6 5 summarization summarization NOUN fcis-1076 6 6 refers refer VERB fcis-1076 6 7 to to ADP fcis-1076 6 8 inputting inputte VERB fcis-1076 6 9 multiple multiple ADJ fcis-1076 6 10 modal modal ADJ fcis-1076 6 11 information information NOUN fcis-1076 6 12 , , PUNCT fcis-1076 6 13 usually usually ADV fcis-1076 6 14 including include VERB fcis-1076 6 15 text text NOUN fcis-1076 6 16 , , PUNCT fcis-1076 6 17 voice voice NOUN fcis-1076 6 18 , , PUNCT fcis-1076 6 19 image image NOUN fcis-1076 6 20 , , PUNCT fcis-1076 6 21 video video NOUN fcis-1076 6 22 and and CCONJ fcis-1076 6 23 other other ADJ fcis-1076 6 24 information information NOUN fcis-1076 6 25 , , PUNCT fcis-1076 6 26 and and CCONJ fcis-1076 6 27 outputting output VERB fcis-1076 6 28 a a DET fcis-1076 6 29 core core NOUN fcis-1076 6 30 summary summary NOUN fcis-1076 6 31 after after ADP fcis-1076 6 32 comprehensively comprehensively ADV fcis-1076 6 33 considering consider VERB fcis-1076 6 34 multiple multiple ADJ fcis-1076 6 35 modal modal ADJ fcis-1076 6 36 information information NOUN fcis-1076 6 37 . . PUNCT fcis-1076 7 1 the the DET fcis-1076 7 2 current current ADJ fcis-1076 7 3 summary summary NOUN fcis-1076 7 4 research research NOUN fcis-1076 7 5 usually usually ADV fcis-1076 7 6 takes take VERB fcis-1076 7 7 the the DET fcis-1076 7 8 text text NOUN fcis-1076 7 9 as as ADP fcis-1076 7 10 the the DET fcis-1076 7 11 processing processing NOUN fcis-1076 7 12 object object NOUN fcis-1076 7 13 , , PUNCT fcis-1076 7 14 and and CCONJ fcis-1076 7 15 generally generally ADV fcis-1076 7 16 does do AUX fcis-1076 7 17 not not PART fcis-1076 7 18 involve involve VERB fcis-1076 7 19 the the DET fcis-1076 7 20 processing processing NOUN fcis-1076 7 21 of of ADP fcis-1076 7 22 other other ADJ fcis-1076 7 23 modal modal ADJ fcis-1076 7 24 information information NOUN fcis-1076 7 25 . . PUNCT fcis-1076 8 1 however however ADV fcis-1076 8 2 , , PUNCT fcis-1076 8 3 various various ADJ fcis-1076 8 4 studies study NOUN fcis-1076 8 5 have have AUX fcis-1076 8 6 shown show VERB fcis-1076 8 7 that that SCONJ fcis-1076 8 8 taking take VERB fcis-1076 8 9 multimodal multimodal ADJ fcis-1076 8 10 data datum NOUN fcis-1076 8 11 as as ADP fcis-1076 8 12 input input NOUN fcis-1076 8 13 does do AUX fcis-1076 8 14 help help VERB fcis-1076 8 15 to to PART fcis-1076 8 16 improve improve VERB fcis-1076 8 17 the the DET fcis-1076 8 18 quality quality NOUN fcis-1076 8 19 of of ADP fcis-1076 8 20 abstracts abstract NOUN fcis-1076 9 1 [ [ X fcis-1076 9 2 1 1 NUM fcis-1076 9 3 , , PUNCT fcis-1076 9 4 2 2 NUM fcis-1076 9 5 ] ] PUNCT fcis-1076 9 6 . . PUNCT fcis-1076 10 1 therefore therefore ADV fcis-1076 10 2 , , PUNCT fcis-1076 10 3 as as ADP fcis-1076 10 4 an an DET fcis-1076 10 5 extension extension NOUN fcis-1076 10 6 of of ADP fcis-1076 10 7 the the DET fcis-1076 10 8 text text NOUN fcis-1076 10 9 summarization summarization NOUN fcis-1076 10 10 task task NOUN fcis-1076 10 11 , , PUNCT fcis-1076 10 12 multimodal multimodal NOUN fcis-1076 10 13 summarization summarization NOUN fcis-1076 10 14 task task NOUN fcis-1076 10 15 integrates integrate VERB fcis-1076 10 16 the the DET fcis-1076 10 17 information information NOUN fcis-1076 10 18 of of ADP fcis-1076 10 19 visual visual ADJ fcis-1076 10 20 and and CCONJ fcis-1076 10 21 auditory auditory ADJ fcis-1076 10 22 modes mode NOUN fcis-1076 10 23 , , PUNCT fcis-1076 10 24 and and CCONJ fcis-1076 10 25 uses use VERB fcis-1076 10 26 the the DET fcis-1076 10 27 complementary complementary ADJ fcis-1076 10 28 and and CCONJ fcis-1076 10 29 verification verification NOUN fcis-1076 10 30 of of ADP fcis-1076 10 31 different different ADJ fcis-1076 10 32 modal modal ADJ fcis-1076 10 33 data datum NOUN fcis-1076 10 34 to to PART fcis-1076 10 35 make make VERB fcis-1076 10 36 the the DET fcis-1076 10 37 application application NOUN fcis-1076 10 38 range range NOUN fcis-1076 10 39 of of ADP fcis-1076 10 40 summarization summarization NOUN fcis-1076 10 41 task task NOUN fcis-1076 10 42 wider wide ADJ fcis-1076 10 43 . . PUNCT fcis-1076 11 1 according accord VERB fcis-1076 11 2 to to ADP fcis-1076 11 3 the the DET fcis-1076 11 4 definition definition NOUN fcis-1076 11 5 of of ADP fcis-1076 11 6 wikipedia wikipedia PROPN fcis-1076 11 7 , , PUNCT fcis-1076 11 8 summary summary NOUN fcis-1076 11 9 refers refer VERB fcis-1076 11 10 to to ADP fcis-1076 11 11 the the DET fcis-1076 11 12 process process NOUN fcis-1076 11 13 of of ADP fcis-1076 11 14 shortening shorten VERB fcis-1076 11 15 a a DET fcis-1076 11 16 set set NOUN fcis-1076 11 17 of of ADP fcis-1076 11 18 data datum NOUN fcis-1076 11 19 through through ADP fcis-1076 11 20 calculation calculation NOUN fcis-1076 11 21 to to PART fcis-1076 11 22 create create VERB fcis-1076 11 23 a a DET fcis-1076 11 24 summary summary NOUN fcis-1076 11 25 that that PRON fcis-1076 11 26 represents represent VERB fcis-1076 11 27 the the DET fcis-1076 11 28 most most ADV fcis-1076 11 29 important important ADJ fcis-1076 11 30 or or CCONJ fcis-1076 11 31 relevant relevant ADJ fcis-1076 11 32 information information NOUN fcis-1076 11 33 in in ADP fcis-1076 11 34 the the DET fcis-1076 11 35 original original ADJ fcis-1076 11 36 content content NOUN fcis-1076 11 37 . . PUNCT fcis-1076 12 1 mathematically mathematically ADV fcis-1076 12 2 , , PUNCT fcis-1076 12 3 it it PRON fcis-1076 12 4 can can AUX fcis-1076 12 5 be be AUX fcis-1076 12 6 expressed express VERB fcis-1076 12 7 as as ADP fcis-1076 12 8 the the DET fcis-1076 12 9 process process NOUN fcis-1076 12 10 of of ADP fcis-1076 12 11 obtaining obtain VERB fcis-1076 12 12 π‘‹π‘ π‘’π‘š π‘‹π‘ π‘’π‘š PROPN fcis-1076 12 13 = = PUNCT fcis-1076 12 14 𝑓(𝐷 𝑓(𝐷 PROPN fcis-1076 12 15 ) ) PUNCT fcis-1076 12 16 that that PRON fcis-1076 12 17 makes make VERB fcis-1076 12 18 π‘™π‘’π‘›π‘”π‘‘β„Ž(π‘‹π‘ π‘’π‘š π‘™π‘’π‘›π‘”π‘‘β„Ž(π‘‹π‘ π‘’π‘š NOUN fcis-1076 12 19 ) ) PUNCT fcis-1076 12 20 ≀ ≀ NUM fcis-1076 12 21 π‘™π‘’π‘›π‘”π‘‘β„Ž(𝐷 π‘™π‘’π‘›π‘”π‘‘β„Ž(𝐷 NUM fcis-1076 12 22 ) ) PUNCT fcis-1076 12 23 , , PUNCT fcis-1076 12 24 in in ADP fcis-1076 12 25 which which PRON fcis-1076 12 26 sumx sumx NOUN fcis-1076 12 27 is be AUX fcis-1076 12 28 the the DET fcis-1076 12 29 output output NOUN fcis-1076 12 30 summary summary NOUN fcis-1076 12 31 , , PUNCT fcis-1076 12 32 d d PROPN fcis-1076 12 33 is be AUX fcis-1076 12 34 the the DET fcis-1076 12 35 input input NOUN fcis-1076 12 36 data datum NOUN fcis-1076 12 37 , , PUNCT fcis-1076 12 38 and and CCONJ fcis-1076 12 39 ( ( PUNCT fcis-1076 12 40 ) ) PUNCT fcis-1076 12 41 f f NOUN fcis-1076 12 42 β€’ β€’ NOUN fcis-1076 12 43 is be AUX fcis-1076 12 44 the the DET fcis-1076 12 45 summary summary NOUN fcis-1076 12 46 function function NOUN fcis-1076 12 47 . . PUNCT fcis-1076 13 1 multimodal multimodal ADJ fcis-1076 13 2 summary summary NOUN fcis-1076 13 3 task task NOUN fcis-1076 13 4 can can AUX fcis-1076 13 5 be be AUX fcis-1076 13 6 defined define VERB fcis-1076 13 7 as as ADP fcis-1076 13 8 " " PUNCT fcis-1076 13 9 a a DET fcis-1076 13 10 summary summary NOUN fcis-1076 13 11 task task NOUN fcis-1076 13 12 that that PRON fcis-1076 13 13 takes take VERB fcis-1076 13 14 more more ADJ fcis-1076 13 15 than than ADP fcis-1076 13 16 one one NUM fcis-1076 13 17 information information NOUN fcis-1076 13 18 representation representation NOUN fcis-1076 13 19 mode mode NOUN fcis-1076 13 20 as as ADP fcis-1076 13 21 input input NOUN fcis-1076 13 22 and and CCONJ fcis-1076 13 23 depends depend VERB fcis-1076 13 24 on on ADP fcis-1076 13 25 the the DET fcis-1076 13 26 information information NOUN fcis-1076 13 27 sharing sharing NOUN fcis-1076 13 28 between between ADP fcis-1076 13 29 different different ADJ fcis-1076 13 30 modes mode NOUN fcis-1076 13 31 to to PART fcis-1076 13 32 generate generate VERB fcis-1076 13 33 the the DET fcis-1076 13 34 final final ADJ fcis-1076 13 35 summary summary NOUN fcis-1076 13 36 " " PUNCT fcis-1076 13 37 . . PUNCT fcis-1076 14 1 mathematically mathematically ADV fcis-1076 14 2 speaking speak VERB fcis-1076 14 3 , , PUNCT fcis-1076 14 4 when when SCONJ fcis-1076 14 5 the the DET fcis-1076 14 6 input input NOUN fcis-1076 14 7 data datum NOUN fcis-1076 14 8 set set VERB fcis-1076 14 9 d d NOUN fcis-1076 14 10 can can AUX fcis-1076 14 11 be be AUX fcis-1076 14 12 decomposed decompose VERB fcis-1076 14 13 into into ADP fcis-1076 14 14 disjoint disjoint NOUN fcis-1076 14 15 sets set NOUN fcis-1076 14 16   PROPN fcis-1076 14 17 1 1 PROPN fcis-1076 14 18 2 2 NUM fcis-1076 14 19 nm nm NOUN fcis-1076 14 20 m m VERB fcis-1076 14 21 m m VERB fcis-1076 14 22 of of ADP fcis-1076 14 23 different different ADJ fcis-1076 14 24 modal modal ADJ fcis-1076 14 25 information information NOUN fcis-1076 14 26 , , PUNCT fcis-1076 14 27 where where SCONJ fcis-1076 14 28 2n 2n NUM fcis-1076 14 29 ο‚³ ο‚³ NUM fcis-1076 14 30 , , PUNCT fcis-1076 14 31 the the DET fcis-1076 14 32 task task NOUN fcis-1076 14 33 of of ADP fcis-1076 14 34 obtaining obtain VERB fcis-1076 14 35 ( ( PUNCT fcis-1076 14 36 ) ) PUNCT fcis-1076 14 37 sumx sumx PROPN fcis-1076 14 38 f f PROPN fcis-1076 14 39 d= d= NOUN fcis-1076 14 40 is be AUX fcis-1076 14 41 defined define VERB fcis-1076 14 42 as as ADP fcis-1076 14 43 a a DET fcis-1076 14 44 multimodal multimodal NOUN fcis-1076 14 45 summary summary NOUN fcis-1076 14 46 . . PUNCT fcis-1076 15 1 if if SCONJ fcis-1076 15 2 for for ADP fcis-1076 15 3   PROPN fcis-1076 15 4   PROPN fcis-1076 15 5 '' '' PUNCT fcis-1076 15 6 ' ' PUNCT fcis-1076 15 7 ' ' NUM fcis-1076 15 8 1 1 NUM fcis-1076 15 9 2sum 2sum NUM fcis-1076 15 10 n n NOUN fcis-1076 15 11 x x NOUN fcis-1076 15 12 m m VERB fcis-1076 15 13 m m VERB fcis-1076 15 14 m= m= ADJ fcis-1076 15 15 , , PUNCT fcis-1076 15 16 ' ' PUNCT fcis-1076 15 17 2n 2n NUM fcis-1076 15 18 ο‚³ ο‚³ NUM fcis-1076 15 19 , , PUNCT fcis-1076 15 20 the the DET fcis-1076 15 21 output output NOUN fcis-1076 15 22 summary summary NOUN fcis-1076 15 23 is be AUX fcis-1076 15 24 multimodal multimodal ADJ fcis-1076 15 25 , , PUNCT fcis-1076 15 26 otherwise otherwise ADV fcis-1076 15 27 it it PRON fcis-1076 15 28 is be AUX fcis-1076 15 29 singlemode singlemode NOUN fcis-1076 15 30 output output NOUN fcis-1076 15 31 . . PUNCT fcis-1076 16 1 this this DET fcis-1076 16 2 paper paper NOUN fcis-1076 16 3 mainly mainly ADV fcis-1076 16 4 introduces introduce VERB fcis-1076 16 5 the the DET fcis-1076 16 6 current current ADJ fcis-1076 16 7 most most ADV fcis-1076 16 8 popular popular ADJ fcis-1076 16 9 model model NOUN fcis-1076 16 10 of of ADP fcis-1076 16 11 multimodal multimodal NOUN fcis-1076 16 12 summarization summarization NOUN fcis-1076 16 13 method method NOUN fcis-1076 16 14 based base VERB fcis-1076 16 15 on on ADP fcis-1076 16 16 sequence sequence NOUN fcis-1076 16 17 to to ADP fcis-1076 16 18 sequence sequence NOUN fcis-1076 16 19 , , PUNCT fcis-1076 16 20 and and CCONJ fcis-1076 16 21 the the DET fcis-1076 16 22 summary summary NOUN fcis-1076 16 23 evaluation evaluation NOUN fcis-1076 16 24 methods method NOUN fcis-1076 16 25 2 2 NUM fcis-1076 16 26 . . PUNCT fcis-1076 16 27 method method NOUN fcis-1076 16 28 based base VERB fcis-1076 16 29 on on ADP fcis-1076 16 30 sequence sequence NOUN fcis-1076 16 31 to to ADP fcis-1076 16 32 sequence sequence NOUN fcis-1076 16 33 in in ADP fcis-1076 16 34 recent recent ADJ fcis-1076 16 35 years year NOUN fcis-1076 16 36 , , PUNCT fcis-1076 16 37 deep deep ADJ fcis-1076 16 38 learning learning NOUN fcis-1076 16 39 technology technology NOUN fcis-1076 16 40 has have AUX fcis-1076 16 41 provided provide VERB fcis-1076 16 42 new new ADJ fcis-1076 16 43 ideas idea NOUN fcis-1076 16 44 for for ADP fcis-1076 16 45 the the DET fcis-1076 16 46 research research NOUN fcis-1076 16 47 of of ADP fcis-1076 16 48 automatic automatic ADJ fcis-1076 16 49 summarization summarization NOUN fcis-1076 16 50 . . PUNCT fcis-1076 17 1 among among ADP fcis-1076 17 2 them they PRON fcis-1076 17 3 , , PUNCT fcis-1076 17 4 the the DET fcis-1076 17 5 research research NOUN fcis-1076 17 6 and and CCONJ fcis-1076 17 7 application application NOUN fcis-1076 17 8 of of ADP fcis-1076 17 9 sequence sequence NOUN fcis-1076 17 10 to to ADP fcis-1076 17 11 sequence sequence NOUN fcis-1076 17 12 ( ( PUNCT fcis-1076 17 13 seq2seq seq2seq NOUN fcis-1076 17 14 ) ) PUNCT fcis-1076 17 15 model model NOUN fcis-1076 17 16 is be AUX fcis-1076 17 17 the the DET fcis-1076 17 18 most most ADV fcis-1076 17 19 extensive extensive ADJ fcis-1076 17 20 . . PUNCT fcis-1076 18 1 this this DET fcis-1076 18 2 model model NOUN fcis-1076 18 3 was be AUX fcis-1076 18 4 proposed propose VERB fcis-1076 18 5 by by ADP fcis-1076 18 6 cho cho PROPN fcis-1076 18 7 et et PROPN fcis-1076 18 8 al al PROPN fcis-1076 18 9 . . PROPN fcis-1076 18 10 and and CCONJ fcis-1076 18 11 sutskever sutskever VERB fcis-1076 18 12 et et PROPN fcis-1076 18 13 al al PROPN fcis-1076 18 14 . . PROPN fcis-1076 19 1 for for ADP fcis-1076 19 2 machine machine NOUN fcis-1076 19 3 translation translation NOUN fcis-1076 19 4 tasks task NOUN fcis-1076 19 5 . . PUNCT fcis-1076 20 1 the the DET fcis-1076 20 2 basic basic ADJ fcis-1076 20 3 idea idea NOUN fcis-1076 20 4 is be AUX fcis-1076 20 5 to to PART fcis-1076 20 6 use use VERB fcis-1076 20 7 the the DET fcis-1076 20 8 global global ADJ fcis-1076 20 9 information information NOUN fcis-1076 20 10 of of ADP fcis-1076 20 11 the the DET fcis-1076 20 12 input input NOUN fcis-1076 20 13 sequence sequence NOUN fcis-1076 20 14 to to PART fcis-1076 20 15 infer infer VERB fcis-1076 20 16 the the DET fcis-1076 20 17 corresponding correspond VERB fcis-1076 20 18 output output NOUN fcis-1076 20 19 sequence sequence NOUN fcis-1076 20 20 , , PUNCT fcis-1076 20 21 which which PRON fcis-1076 20 22 is be AUX fcis-1076 20 23 composed compose VERB fcis-1076 20 24 of of ADP fcis-1076 20 25 an an DET fcis-1076 20 26 encoder encoder NOUN fcis-1076 20 27 and and CCONJ fcis-1076 20 28 a a DET fcis-1076 20 29 decoder decoder NOUN fcis-1076 20 30 . . PUNCT fcis-1076 21 1 the the DET fcis-1076 21 2 encoder encoder NOUN fcis-1076 21 3 encodes encode VERB fcis-1076 21 4 an an DET fcis-1076 21 5 input input NOUN fcis-1076 21 6 variable variable ADJ fcis-1076 21 7 length length NOUN fcis-1076 21 8 sequence sequence NOUN fcis-1076 21 9 1 1 NUM fcis-1076 21 10 ( ( PUNCT fcis-1076 21 11 , , PUNCT fcis-1076 21 12 ... ... PUNCT fcis-1076 21 13 , , PUNCT fcis-1076 21 14 ) ) PUNCT fcis-1076 21 15 tx tx PROPN fcis-1076 21 16 x x X fcis-1076 21 17 x= x= PROPN fcis-1076 21 18 into into ADP fcis-1076 21 19 a a DET fcis-1076 21 20 fixed fix VERB fcis-1076 21 21 semantic semantic ADJ fcis-1076 21 22 vector vector NOUN fcis-1076 21 23 . . PUNCT fcis-1076 22 1 using use VERB fcis-1076 22 2 recurrent recurrent ADJ fcis-1076 22 3 neural neural ADJ fcis-1076 22 4 network network NOUN fcis-1076 22 5 ( ( PUNCT fcis-1076 22 6 rnn rnn PROPN fcis-1076 22 7 ) ) PUNCT fcis-1076 22 8 , , PUNCT fcis-1076 22 9 the the DET fcis-1076 22 10 source source NOUN fcis-1076 22 11 sequence sequence NOUN fcis-1076 22 12 x x PUNCT fcis-1076 22 13 is be AUX fcis-1076 22 14 converted convert VERB fcis-1076 22 15 into into ADP fcis-1076 22 16 the the DET fcis-1076 22 17 form form NOUN fcis-1076 22 18 of of ADP fcis-1076 22 19 context context NOUN fcis-1076 22 20 c c NOUN fcis-1076 22 21 by by ADP fcis-1076 22 22 the the DET fcis-1076 22 23 following follow VERB fcis-1076 22 24 formula formula NOUN fcis-1076 22 25 : : PUNCT fcis-1076 22 26 1 1 NUM fcis-1076 22 27 ( ( PUNCT fcis-1076 22 28 , , PUNCT fcis-1076 22 29 ) ) PUNCT fcis-1076 22 30 t t PROPN fcis-1076 22 31 enc enc PROPN fcis-1076 22 32 t t PROPN fcis-1076 22 33 th th X fcis-1076 22 34 f f PROPN fcis-1076 22 35 x x PUNCT fcis-1076 22 36 h h PROPN fcis-1076 22 37 βˆ’= βˆ’= NOUN fcis-1076 22 38 ( ( PUNCT fcis-1076 22 39 2.1 2.1 NUM fcis-1076 22 40 ) ) PUNCT fcis-1076 22 41 1 1 NUM fcis-1076 22 42 ( ( PUNCT fcis-1076 22 43 , , PUNCT fcis-1076 22 44 ... ... PUNCT fcis-1076 22 45 , , PUNCT fcis-1076 22 46 ) ) PUNCT fcis-1076 23 1 t t PROPN fcis-1076 23 2 c c PROPN fcis-1076 23 3 tc tc PROPN fcis-1076 23 4 f f PROPN fcis-1076 23 5 h h PROPN fcis-1076 23 6 h= h= X fcis-1076 23 7 ( ( PUNCT fcis-1076 23 8 2.2 2.2 NUM fcis-1076 23 9 ) ) PUNCT fcis-1076 23 10 where where SCONJ fcis-1076 23 11 n n PRON fcis-1076 23 12 th th X fcis-1076 23 13 rοƒŽ rοƒŽ PROPN fcis-1076 23 14 is be AUX fcis-1076 23 15 the the DET fcis-1076 23 16 hidden hidden ADJ fcis-1076 23 17 state state NOUN fcis-1076 23 18 at at ADP fcis-1076 23 19 time time NOUN fcis-1076 23 20 t t PROPN fcis-1076 23 21 , , PUNCT fcis-1076 23 22 tc tc PROPN fcis-1076 23 23 is be AUX fcis-1076 23 24 the the DET fcis-1076 23 25 context context NOUN fcis-1076 23 26 vector vector NOUN fcis-1076 23 27 generated generate VERB fcis-1076 23 28 from from ADP fcis-1076 23 29 the the DET fcis-1076 23 30 hidden hidden ADJ fcis-1076 23 31 state state NOUN fcis-1076 23 32 sequence sequence NOUN fcis-1076 23 33 , , PUNCT fcis-1076 23 34 and and CCONJ fcis-1076 23 35 encf encf NOUN fcis-1076 23 36 and and CCONJ fcis-1076 23 37 cf cf NOUN fcis-1076 23 38 are be AUX fcis-1076 23 39 nonlinear nonlinear ADJ fcis-1076 23 40 activation activation NOUN fcis-1076 23 41 functions function NOUN fcis-1076 23 42 . . PUNCT fcis-1076 24 1 the the DET fcis-1076 24 2 decoder decoder NOUN fcis-1076 24 3 extracts extract VERB fcis-1076 24 4 semantic semantic ADJ fcis-1076 24 5 information information NOUN fcis-1076 24 6 from from ADP fcis-1076 24 7 this this DET fcis-1076 24 8 vector vector NOUN fcis-1076 24 9 and and CCONJ fcis-1076 24 10 outputs output NOUN fcis-1076 24 11 another another DET fcis-1076 24 12 variable variable ADJ fcis-1076 24 13 length length NOUN fcis-1076 24 14 sequence sequence NOUN fcis-1076 24 15 ' ' NOUN fcis-1076 24 16 1 1 NUM fcis-1076 24 17 ( ( PUNCT fcis-1076 24 18 , , PUNCT fcis-1076 24 19 ... ... PUNCT fcis-1076 24 20 , , PUNCT fcis-1076 24 21 ) ) PUNCT fcis-1076 24 22 t t PROPN fcis-1076 24 23 y y PROPN fcis-1076 24 24 y y PROPN fcis-1076 24 25 y= y= PROPN fcis-1076 24 26 . . PUNCT fcis-1076 25 1 generate generate VERB fcis-1076 25 2 ty ty PRON fcis-1076 25 3 from from ADP fcis-1076 25 4 the the DET fcis-1076 25 5 context context NOUN fcis-1076 25 6 vector vector NOUN fcis-1076 25 7 tc tc NOUN fcis-1076 25 8 and and CCONJ fcis-1076 25 9 the the DET fcis-1076 25 10 previously previously ADV fcis-1076 25 11 generated generate VERB fcis-1076 25 12 1 1 NUM fcis-1076 25 13 1 1 NUM fcis-1076 25 14 { { PUNCT fcis-1076 25 15 , , PUNCT fcis-1076 25 16 ... ... PUNCT fcis-1076 25 17 , , PUNCT fcis-1076 25 18 } } PUNCT fcis-1076 25 19 ty ty NOUN fcis-1076 25 20 y y NOUN fcis-1076 26 1 βˆ’ βˆ’ NOUN fcis-1076 26 2 : : PUNCT fcis-1076 26 3 1 1 NUM fcis-1076 26 4 1 1 NUM fcis-1076 26 5 1 1 NUM fcis-1076 26 6 ( ( PUNCT fcis-1076 26 7 | | ADV fcis-1076 26 8 { { PUNCT fcis-1076 26 9 , , PUNCT fcis-1076 26 10 ... ... PUNCT fcis-1076 26 11 , , PUNCT fcis-1076 26 12 } } PUNCT fcis-1076 26 13 , , PUNCT fcis-1076 26 14 ) ) PUNCT fcis-1076 26 15 ( ( PUNCT fcis-1076 26 16 , , PUNCT fcis-1076 26 17 , , PUNCT fcis-1076 26 18 ) ) PUNCT fcis-1076 26 19 t t PROPN fcis-1076 26 20 t t PROPN fcis-1076 26 21 t t PROPN fcis-1076 26 22 dec dec PROPN fcis-1076 26 23 t t PROPN fcis-1076 26 24 t t PROPN fcis-1076 26 25 tp tp ADP fcis-1076 26 26 y y PROPN fcis-1076 26 27 y y PROPN fcis-1076 27 1 y y PROPN fcis-1076 27 2 c c PROPN fcis-1076 27 3 f f PROPN fcis-1076 27 4 y y PROPN fcis-1076 27 5 s s PROPN fcis-1076 27 6 cβˆ’ cβˆ’ PROPN fcis-1076 27 7 βˆ’= βˆ’= X fcis-1076 27 8 ( ( PUNCT fcis-1076 27 9 2.3 2.3 NUM fcis-1076 27 10 ) ) PUNCT fcis-1076 27 11 where where SCONJ fcis-1076 27 12 ts ts ADV fcis-1076 27 13 is be AUX fcis-1076 27 14 the the DET fcis-1076 27 15 hidden hidden ADJ fcis-1076 27 16 state state NOUN fcis-1076 27 17 of of ADP fcis-1076 27 18 the the DET fcis-1076 27 19 decoder decoder NOUN fcis-1076 27 20 , , PUNCT fcis-1076 27 21 and and CCONJ fcis-1076 27 22 decf decf PROPN fcis-1076 27 23 is be AUX fcis-1076 27 24 a a DET fcis-1076 27 25 nonlinear nonlinear ADJ fcis-1076 27 26 activation activation NOUN fcis-1076 27 27 function function NOUN fcis-1076 27 28 used use VERB fcis-1076 27 29 to to PART fcis-1076 27 30 calculate calculate VERB fcis-1076 27 31 the the DET fcis-1076 27 32 probability probability NOUN fcis-1076 27 33 vector vector NOUN fcis-1076 27 34 of of ADP fcis-1076 27 35 the the DET fcis-1076 27 36 output output NOUN fcis-1076 27 37 word word NOUN fcis-1076 27 38 at at ADP fcis-1076 27 39 time time NOUN fcis-1076 27 40 t t PROPN fcis-1076 27 41 . . PUNCT fcis-1076 28 1 the the DET fcis-1076 28 2 loss loss NOUN fcis-1076 28 3 function function NOUN fcis-1076 28 4 tl tl PROPN fcis-1076 28 5 at at ADP fcis-1076 28 6 each each DET fcis-1076 28 7 time time NOUN fcis-1076 28 8 is be AUX fcis-1076 28 9 the the DET fcis-1076 28 10 negative negative ADJ fcis-1076 28 11 log log NOUN fcis-1076 28 12 likelihood likelihood NOUN fcis-1076 28 13 function function NOUN fcis-1076 28 14 of of ADP fcis-1076 28 15 ty ty NUM fcis-1076 28 16 : : PUNCT fcis-1076 28 17 1 1 NUM fcis-1076 28 18 1log 1log X fcis-1076 28 19 ( ( PUNCT fcis-1076 28 20 | | ADV fcis-1076 28 21 { { PUNCT fcis-1076 28 22 , , PUNCT fcis-1076 28 23 ... ... PUNCT fcis-1076 28 24 , , PUNCT fcis-1076 28 25 } } PUNCT fcis-1076 28 26 , , PUNCT fcis-1076 28 27 ) ) PUNCT fcis-1076 29 1 t t PROPN fcis-1076 29 2 t t PROPN fcis-1076 29 3 t t PROPN fcis-1076 29 4 tl tl PROPN fcis-1076 29 5 p p PROPN fcis-1076 29 6 y y PROPN fcis-1076 29 7 y y PROPN fcis-1076 29 8 y y PROPN fcis-1076 29 9 cβˆ’= cβˆ’= PROPN fcis-1076 29 10 βˆ’ βˆ’ PROPN fcis-1076 29 11 ( ( PUNCT fcis-1076 29 12 2.4 2.4 NUM fcis-1076 29 13 ) ) PUNCT fcis-1076 29 14 14 14 NUM fcis-1076 29 15 the the DET fcis-1076 29 16 disadvantage disadvantage NOUN fcis-1076 29 17 of of ADP fcis-1076 29 18 this this DET fcis-1076 29 19 model model NOUN fcis-1076 29 20 is be AUX fcis-1076 29 21 that that SCONJ fcis-1076 29 22 the the DET fcis-1076 29 23 encoder encoder NOUN fcis-1076 29 24 represents represent VERB fcis-1076 29 25 all all DET fcis-1076 29 26 the the DET fcis-1076 29 27 information information NOUN fcis-1076 29 28 in in ADP fcis-1076 29 29 the the DET fcis-1076 29 30 source source NOUN fcis-1076 29 31 text text NOUN fcis-1076 29 32 as as ADP fcis-1076 29 33 a a DET fcis-1076 29 34 fixed fix VERB fcis-1076 29 35 semantic semantic ADJ fcis-1076 29 36 vector vector NOUN fcis-1076 29 37 , , PUNCT fcis-1076 29 38 and and CCONJ fcis-1076 29 39 the the DET fcis-1076 29 40 decoder decoder NOUN fcis-1076 29 41 refers refer VERB fcis-1076 29 42 to to ADP fcis-1076 29 43 this this DET fcis-1076 29 44 vector vector NOUN fcis-1076 29 45 when when SCONJ fcis-1076 29 46 generating generate VERB fcis-1076 29 47 each each DET fcis-1076 29 48 word word NOUN fcis-1076 29 49 item item NOUN fcis-1076 29 50 , , PUNCT fcis-1076 29 51 which which PRON fcis-1076 29 52 makes make VERB fcis-1076 29 53 it it PRON fcis-1076 29 54 difficult difficult ADJ fcis-1076 29 55 for for SCONJ fcis-1076 29 56 neural neural ADJ fcis-1076 29 57 network network NOUN fcis-1076 29 58 to to PART fcis-1076 29 59 process process VERB fcis-1076 29 60 long long ADJ fcis-1076 29 61 text text NOUN fcis-1076 29 62 . . PUNCT fcis-1076 30 1 the the DET fcis-1076 30 2 experiment experiment NOUN fcis-1076 30 3 of of ADP fcis-1076 30 4 cho cho PROPN fcis-1076 30 5 et et PROPN fcis-1076 30 6 al al PROPN fcis-1076 30 7 . . PROPN fcis-1076 30 8 confirmed confirm VERB fcis-1076 30 9 that that SCONJ fcis-1076 30 10 the the DET fcis-1076 30 11 performance performance NOUN fcis-1076 30 12 of of ADP fcis-1076 30 13 the the DET fcis-1076 30 14 model model NOUN fcis-1076 30 15 decreased decrease VERB fcis-1076 30 16 rapidly rapidly ADV fcis-1076 30 17 with with ADP fcis-1076 30 18 the the DET fcis-1076 30 19 increase increase NOUN fcis-1076 30 20 of of ADP fcis-1076 30 21 text text NOUN fcis-1076 30 22 length length NOUN fcis-1076 30 23 . . PUNCT fcis-1076 31 1 in in ADP fcis-1076 31 2 this this DET fcis-1076 31 3 regard regard NOUN fcis-1076 31 4 , , PUNCT fcis-1076 31 5 bandanas bandanas PROPN fcis-1076 31 6 et et PROPN fcis-1076 31 7 al al PROPN fcis-1076 31 8 . . PROPN fcis-1076 31 9 introduced introduce VERB fcis-1076 31 10 the the DET fcis-1076 31 11 attention attention NOUN fcis-1076 31 12 mechanism mechanism NOUN fcis-1076 31 13 into into ADP fcis-1076 31 14 the the DET fcis-1076 31 15 model model NOUN fcis-1076 31 16 to to PART fcis-1076 31 17 make make VERB fcis-1076 31 18 the the DET fcis-1076 31 19 decoder decoder NOUN fcis-1076 31 20 focus focus VERB fcis-1076 31 21 on on ADP fcis-1076 31 22 specific specific ADJ fcis-1076 31 23 parts part NOUN fcis-1076 31 24 of of ADP fcis-1076 31 25 the the DET fcis-1076 31 26 source source NOUN fcis-1076 31 27 text text NOUN fcis-1076 31 28 when when SCONJ fcis-1076 31 29 generating generate VERB fcis-1076 31 30 each each DET fcis-1076 31 31 word word NOUN fcis-1076 31 32 item item NOUN fcis-1076 31 33 . . PUNCT fcis-1076 32 1 the the DET fcis-1076 32 2 attention attention NOUN fcis-1076 32 3 mechanism mechanism NOUN fcis-1076 32 4 estimates estimate VERB fcis-1076 32 5 the the DET fcis-1076 32 6 probability probability NOUN fcis-1076 32 7 distribution distribution NOUN fcis-1076 32 8 of of ADP fcis-1076 32 9 the the DET fcis-1076 32 10 encoder encoder NOUN fcis-1076 32 11 's 's PART fcis-1076 32 12 hidden hide VERB fcis-1076 32 13 state state NOUN fcis-1076 32 14 in in ADP fcis-1076 32 15 each each DET fcis-1076 32 16 decoding decode VERB fcis-1076 32 17 step step NOUN fcis-1076 32 18 , , PUNCT fcis-1076 32 19 which which PRON fcis-1076 32 20 is be AUX fcis-1076 32 21 used use VERB fcis-1076 32 22 to to PART fcis-1076 32 23 calculate calculate VERB fcis-1076 32 24 the the DET fcis-1076 32 25 weighted weighted ADJ fcis-1076 32 26 average average ADJ fcis-1076 32 27 value value NOUN fcis-1076 32 28 of of ADP fcis-1076 32 29 the the DET fcis-1076 32 30 context context NOUN fcis-1076 32 31 vector vector NOUN fcis-1076 32 32 and and CCONJ fcis-1076 32 33 the the DET fcis-1076 32 34 encoder encoder NOUN fcis-1076 32 35 's 's PART fcis-1076 32 36 hidden hide VERB fcis-1076 32 37 state state NOUN fcis-1076 32 38 as as ADP fcis-1076 32 39 an an DET fcis-1076 32 40 additional additional ADJ fcis-1076 32 41 input input NOUN fcis-1076 32 42 to to ADP fcis-1076 32 43 the the DET fcis-1076 32 44 decoder decoder NOUN fcis-1076 32 45 . . PUNCT fcis-1076 33 1 experimental experimental ADJ fcis-1076 33 2 results result NOUN fcis-1076 33 3 show show VERB fcis-1076 33 4 that that SCONJ fcis-1076 33 5 the the DET fcis-1076 33 6 model model NOUN fcis-1076 33 7 with with ADP fcis-1076 33 8 attention attention NOUN fcis-1076 33 9 mechanism mechanism NOUN fcis-1076 33 10 achieves achieve VERB fcis-1076 33 11 better well ADJ fcis-1076 33 12 results result NOUN fcis-1076 33 13 in in ADP fcis-1076 33 14 machine machine NOUN fcis-1076 33 15 translation translation NOUN fcis-1076 33 16 tasks task NOUN fcis-1076 33 17 and and CCONJ fcis-1076 33 18 is be AUX fcis-1076 33 19 more more ADV fcis-1076 33 20 robust robust ADJ fcis-1076 33 21 to to PART fcis-1076 33 22 sentence sentence NOUN fcis-1076 33 23 length length NOUN fcis-1076 33 24 changes change NOUN fcis-1076 33 25 . . PUNCT fcis-1076 34 1 the the DET fcis-1076 34 2 addition addition NOUN fcis-1076 34 3 of of ADP fcis-1076 34 4 attention attention NOUN fcis-1076 34 5 mechanism mechanism NOUN fcis-1076 34 6 makes make VERB fcis-1076 34 7 the the DET fcis-1076 34 8 seq2seq seq2seq NOUN fcis-1076 34 9 model model NOUN fcis-1076 34 10 more more ADV fcis-1076 34 11 perfect perfect ADJ fcis-1076 34 12 , , PUNCT fcis-1076 34 13 and and CCONJ fcis-1076 34 14 a a DET fcis-1076 34 15 large large ADJ fcis-1076 34 16 number number NOUN fcis-1076 34 17 of of ADP fcis-1076 34 18 relevant relevant ADJ fcis-1076 34 19 studies study NOUN fcis-1076 34 20 have have AUX fcis-1076 34 21 been be AUX fcis-1076 34 22 based base VERB fcis-1076 34 23 on on ADP fcis-1076 34 24 this this DET fcis-1076 34 25 model model NOUN fcis-1076 34 26 . . PUNCT fcis-1076 35 1 fig.1 fig.1 PROPN fcis-1076 35 2 is be AUX fcis-1076 35 3 the the DET fcis-1076 35 4 schematic schematic ADJ fcis-1076 35 5 diagram diagram NOUN fcis-1076 35 6 of of ADP fcis-1076 35 7 seq2seq seq2seq NOUN fcis-1076 35 8 model model NOUN fcis-1076 35 9 with with ADP fcis-1076 35 10 attention attention NOUN fcis-1076 35 11 mechanism mechanism NOUN fcis-1076 35 12 . . PUNCT fcis-1076 36 1 fig.1 fig.1 ADJ fcis-1076 36 2 schematic schematic ADJ fcis-1076 36 3 diagram diagram NOUN fcis-1076 36 4 of of ADP fcis-1076 36 5 seq2seq seq2seq NOUN fcis-1076 36 6 model model NOUN fcis-1076 36 7 with with ADP fcis-1076 36 8 attention attention NOUN fcis-1076 36 9 mechanism mechanism NOUN fcis-1076 36 10 rush rush NOUN fcis-1076 36 11 et et PROPN fcis-1076 36 12 al al PROPN fcis-1076 36 13 . . PROPN fcis-1076 36 14 applied apply VERB fcis-1076 36 15 this this DET fcis-1076 36 16 model model NOUN fcis-1076 36 17 to to PART fcis-1076 36 18 generative generative VERB fcis-1076 36 19 summarization summarization NOUN fcis-1076 36 20 for for ADP fcis-1076 36 21 the the DET fcis-1076 36 22 first first ADJ fcis-1076 36 23 time time NOUN fcis-1076 36 24 . . PUNCT fcis-1076 37 1 compared compare VERB fcis-1076 37 2 with with ADP fcis-1076 37 3 the the DET fcis-1076 37 4 previous previous ADJ fcis-1076 37 5 generative generative ADJ fcis-1076 37 6 method method NOUN fcis-1076 37 7 , , PUNCT fcis-1076 37 8 this this DET fcis-1076 37 9 model model NOUN fcis-1076 37 10 generates generate VERB fcis-1076 37 11 abstracts abstract NOUN fcis-1076 37 12 based base VERB fcis-1076 37 13 on on ADP fcis-1076 37 14 " " PUNCT fcis-1076 37 15 understanding understand VERB fcis-1076 37 16 " " PUNCT fcis-1076 37 17 the the DET fcis-1076 37 18 semantics semantic NOUN fcis-1076 37 19 of of ADP fcis-1076 37 20 text text NOUN fcis-1076 37 21 , , PUNCT fcis-1076 37 22 which which PRON fcis-1076 37 23 is be AUX fcis-1076 37 24 closer close ADJ fcis-1076 37 25 to to ADP fcis-1076 37 26 the the DET fcis-1076 37 27 generation generation NOUN fcis-1076 37 28 process process NOUN fcis-1076 37 29 of of ADP fcis-1076 37 30 artificial artificial ADJ fcis-1076 37 31 abstracts abstract NOUN fcis-1076 37 32 . . PUNCT fcis-1076 38 1 subsequently subsequently ADV fcis-1076 38 2 , , PUNCT fcis-1076 38 3 a a DET fcis-1076 38 4 series series NOUN fcis-1076 38 5 of of ADP fcis-1076 38 6 generative generative ADJ fcis-1076 38 7 summarization summarization NOUN fcis-1076 38 8 models model NOUN fcis-1076 38 9 based base VERB fcis-1076 38 10 on on ADP fcis-1076 38 11 seq2seq seq2seq NOUN fcis-1076 38 12 have have AUX fcis-1076 38 13 been be AUX fcis-1076 38 14 proposed propose VERB fcis-1076 38 15 , , PUNCT fcis-1076 38 16 and and CCONJ fcis-1076 38 17 fruitful fruitful ADJ fcis-1076 38 18 research research NOUN fcis-1076 38 19 work work NOUN fcis-1076 38 20 has have AUX fcis-1076 38 21 been be AUX fcis-1076 38 22 carried carry VERB fcis-1076 38 23 out out ADP fcis-1076 38 24 on on ADP fcis-1076 38 25 encoders encoder NOUN fcis-1076 38 26 , , PUNCT fcis-1076 38 27 decoders decoder NOUN fcis-1076 38 28 and and CCONJ fcis-1076 38 29 training training NOUN fcis-1076 38 30 methods method NOUN fcis-1076 38 31 . . PUNCT fcis-1076 39 1 the the DET fcis-1076 39 2 abstracts abstract NOUN fcis-1076 39 3 generated generate VERB fcis-1076 39 4 based base VERB fcis-1076 39 5 on on ADP fcis-1076 39 6 this this DET fcis-1076 39 7 model model NOUN fcis-1076 39 8 let let VERB fcis-1076 39 9 the the DET fcis-1076 39 10 academic academic ADJ fcis-1076 39 11 community community NOUN fcis-1076 39 12 see see VERB fcis-1076 39 13 the the DET fcis-1076 39 14 hope hope NOUN fcis-1076 39 15 of of ADP fcis-1076 39 16 the the DET fcis-1076 39 17 practicality practicality NOUN fcis-1076 39 18 of of ADP fcis-1076 39 19 automatic automatic ADJ fcis-1076 39 20 abstracts abstract NOUN fcis-1076 39 21 in in ADP fcis-1076 39 22 terms term NOUN fcis-1076 39 23 of of ADP fcis-1076 39 24 language language NOUN fcis-1076 39 25 fluency fluency NOUN fcis-1076 39 26 and and CCONJ fcis-1076 39 27 coherence coherence NOUN fcis-1076 39 28 . . PUNCT fcis-1076 40 1 compared compare VERB fcis-1076 40 2 with with ADP fcis-1076 40 3 text text NOUN fcis-1076 40 4 summarization summarization NOUN fcis-1076 40 5 , , PUNCT fcis-1076 40 6 multimodal multimodal NOUN fcis-1076 40 7 summarization summarization NOUN fcis-1076 40 8 extracts extract NOUN fcis-1076 40 9 higher high ADJ fcis-1076 40 10 quality quality NOUN fcis-1076 40 11 summarization summarization NOUN fcis-1076 40 12 by by ADP fcis-1076 40 13 adding add VERB fcis-1076 40 14 modal modal ADJ fcis-1076 40 15 information information NOUN fcis-1076 40 16 such such ADJ fcis-1076 40 17 as as ADP fcis-1076 40 18 image image NOUN fcis-1076 40 19 , , PUNCT fcis-1076 40 20 video video NOUN fcis-1076 40 21 , , PUNCT fcis-1076 40 22 audio audio NOUN fcis-1076 40 23 , , PUNCT fcis-1076 40 24 etc etc X fcis-1076 40 25 . . X fcis-1076 40 26 therefore therefore ADV fcis-1076 40 27 , , PUNCT fcis-1076 40 28 sequence sequence NOUN fcis-1076 40 29 to to ADP fcis-1076 40 30 sequence sequence NOUN fcis-1076 40 31 model model NOUN fcis-1076 40 32 is be AUX fcis-1076 40 33 gradually gradually ADV fcis-1076 40 34 applied apply VERB fcis-1076 40 35 to to ADP fcis-1076 40 36 the the DET fcis-1076 40 37 field field NOUN fcis-1076 40 38 of of ADP fcis-1076 40 39 multimodal multimodal NOUN fcis-1076 40 40 summarization summarization NOUN fcis-1076 40 41 . . PUNCT fcis-1076 41 1 the the DET fcis-1076 41 2 multimodal multimodal NOUN fcis-1076 41 3 summarization summarization NOUN fcis-1076 41 4 model model NOUN fcis-1076 41 5 based base VERB fcis-1076 41 6 on on ADP fcis-1076 41 7 sequence sequence NOUN fcis-1076 41 8 to to ADP fcis-1076 41 9 sequence sequence NOUN fcis-1076 41 10 is be AUX fcis-1076 41 11 mainly mainly ADV fcis-1076 41 12 composed compose VERB fcis-1076 41 13 of of ADP fcis-1076 41 14 three three NUM fcis-1076 41 15 modules module NOUN fcis-1076 41 16 : : PUNCT fcis-1076 41 17 encoder encoder NOUN fcis-1076 41 18 , , PUNCT fcis-1076 41 19 decoder decoder NOUN fcis-1076 41 20 and and CCONJ fcis-1076 41 21 multimodal multimodal NOUN fcis-1076 41 22 fusion fusion NOUN fcis-1076 41 23 module module NOUN fcis-1076 41 24 . . PUNCT fcis-1076 42 1 2.1 2.1 NUM fcis-1076 42 2 . . PUNCT fcis-1076 42 3 encoder encoder NOUN fcis-1076 42 4 due due ADP fcis-1076 42 5 to to ADP fcis-1076 42 6 the the DET fcis-1076 42 7 significant significant ADJ fcis-1076 42 8 progress progress NOUN fcis-1076 42 9 in in ADP fcis-1076 42 10 feature feature NOUN fcis-1076 42 11 extraction extraction NOUN fcis-1076 42 12 , , PUNCT fcis-1076 42 13 various various ADJ fcis-1076 42 14 encoders encoder NOUN fcis-1076 42 15 have have AUX fcis-1076 42 16 been be AUX fcis-1076 42 17 explored explore VERB fcis-1076 42 18 to to PART fcis-1076 42 19 encode encode VERB fcis-1076 42 20 context context NOUN fcis-1076 42 21 information information NOUN fcis-1076 42 22 in in ADP fcis-1076 42 23 the the DET fcis-1076 42 24 form form NOUN fcis-1076 42 25 of of ADP fcis-1076 42 26 text text NOUN fcis-1076 42 27 . . PUNCT fcis-1076 43 1 due due ADP fcis-1076 43 2 to to ADP fcis-1076 43 3 the the DET fcis-1076 43 4 long long ADJ fcis-1076 43 5 - - PUNCT fcis-1076 43 6 term term NOUN fcis-1076 43 7 dependence dependence NOUN fcis-1076 43 8 of of ADP fcis-1076 43 9 rnn rnn PROPN fcis-1076 43 10 , , PUNCT fcis-1076 43 11 it it PRON fcis-1076 43 12 is be AUX fcis-1076 43 13 difficult difficult ADJ fcis-1076 43 14 to to PART fcis-1076 43 15 learn learn VERB fcis-1076 43 16 information information NOUN fcis-1076 43 17 far far ADV fcis-1076 43 18 away away ADV fcis-1076 43 19 , , PUNCT fcis-1076 43 20 so so CCONJ fcis-1076 43 21 most most ADJ fcis-1076 43 22 models model NOUN fcis-1076 43 23 use use VERB fcis-1076 43 24 long long ADJ fcis-1076 43 25 short short ADJ fcis-1076 43 26 - - PUNCT fcis-1076 43 27 term term NOUN fcis-1076 43 28 memory memory NOUN fcis-1076 43 29 ( ( PUNCT fcis-1076 43 30 lstm lstm NOUN fcis-1076 43 31 ) ) PUNCT fcis-1076 43 32 or or CCONJ fcis-1076 43 33 gated gate VERB fcis-1076 43 34 recurrent recurrent ADJ fcis-1076 43 35 unit unit NOUN fcis-1076 43 36 ( ( PUNCT fcis-1076 43 37 gru gru PROPN fcis-1076 43 38 ) ) PUNCT fcis-1076 43 39 to to PART fcis-1076 43 40 replace replace VERB fcis-1076 43 41 it it PRON fcis-1076 43 42 . . PUNCT fcis-1076 44 1 lstm lstm NOUN fcis-1076 44 2 introduces introduce VERB fcis-1076 44 3 the the DET fcis-1076 44 4 concept concept NOUN fcis-1076 44 5 of of ADP fcis-1076 44 6 gating gating NOUN fcis-1076 44 7 , , PUNCT fcis-1076 44 8 which which PRON fcis-1076 44 9 can can AUX fcis-1076 44 10 learn learn VERB fcis-1076 44 11 the the DET fcis-1076 44 12 memory memory NOUN fcis-1076 44 13 / / PUNCT fcis-1076 44 14 forgetting forgetting NOUN fcis-1076 44 15 , , PUNCT fcis-1076 44 16 input input NOUN fcis-1076 44 17 and and CCONJ fcis-1076 44 18 output output NOUN fcis-1076 44 19 of of ADP fcis-1076 44 20 memory memory NOUN fcis-1076 44 21 units unit NOUN fcis-1076 44 22 , , PUNCT fcis-1076 44 23 so so SCONJ fcis-1076 44 24 that that SCONJ fcis-1076 44 25 the the DET fcis-1076 44 26 machine machine NOUN fcis-1076 44 27 knows know VERB fcis-1076 44 28 when when SCONJ fcis-1076 44 29 to to PART fcis-1076 44 30 remember remember VERB fcis-1076 44 31 or or CCONJ fcis-1076 44 32 discard discard VERB fcis-1076 44 33 some some DET fcis-1076 44 34 information information NOUN fcis-1076 44 35 . . PUNCT fcis-1076 45 1 gru gru PROPN fcis-1076 45 2 uses use VERB fcis-1076 45 3 gating gate VERB fcis-1076 45 4 network network NOUN fcis-1076 45 5 to to PART fcis-1076 45 6 learn learn VERB fcis-1076 45 7 the the DET fcis-1076 45 8 combination combination NOUN fcis-1076 45 9 of of ADP fcis-1076 45 10 new new ADJ fcis-1076 45 11 input input NOUN fcis-1076 45 12 and and CCONJ fcis-1076 45 13 previous previous ADJ fcis-1076 45 14 memory memory NOUN fcis-1076 45 15 and and CCONJ fcis-1076 45 16 the the DET fcis-1076 45 17 retention retention NOUN fcis-1076 45 18 degree degree NOUN fcis-1076 45 19 of of ADP fcis-1076 45 20 previous previous ADJ fcis-1076 45 21 memory memory NOUN fcis-1076 45 22 . . PUNCT fcis-1076 46 1 although although SCONJ fcis-1076 46 2 the the DET fcis-1076 46 3 principles principle NOUN fcis-1076 46 4 of of ADP fcis-1076 46 5 lstm lstm NOUN fcis-1076 46 6 and and CCONJ fcis-1076 46 7 gru gru PROPN fcis-1076 46 8 are be AUX fcis-1076 46 9 different different ADJ fcis-1076 46 10 , , PUNCT fcis-1076 46 11 they they PRON fcis-1076 46 12 can can AUX fcis-1076 46 13 learn learn VERB fcis-1076 46 14 long long ADJ fcis-1076 46 15 - - PUNCT fcis-1076 46 16 term term NOUN fcis-1076 46 17 dependent dependent ADJ fcis-1076 46 18 information information NOUN fcis-1076 46 19 and and CCONJ fcis-1076 46 20 perform perform VERB fcis-1076 46 21 better well ADV fcis-1076 46 22 than than SCONJ fcis-1076 46 23 rnn rnn VERB fcis-1076 46 24 on on ADP fcis-1076 46 25 many many ADJ fcis-1076 46 26 tasks task NOUN fcis-1076 46 27 . . PUNCT fcis-1076 47 1 compared compare VERB fcis-1076 47 2 with with ADP fcis-1076 47 3 lstm lstm PROPN fcis-1076 47 4 , , PUNCT fcis-1076 47 5 gru gru NOUN fcis-1076 47 6 parameters parameter NOUN fcis-1076 47 7 are be AUX fcis-1076 47 8 less less ADV fcis-1076 47 9 and and CCONJ fcis-1076 47 10 easier easy ADJ fcis-1076 47 11 to to PART fcis-1076 47 12 train train VERB fcis-1076 47 13 , , PUNCT fcis-1076 47 14 but but CCONJ fcis-1076 47 15 when when SCONJ fcis-1076 47 16 there there PRON fcis-1076 47 17 are be VERB fcis-1076 47 18 more more ADJ fcis-1076 47 19 training training NOUN fcis-1076 47 20 data datum NOUN fcis-1076 47 21 , , PUNCT fcis-1076 47 22 lstm lstm NOUN fcis-1076 47 23 with with ADP fcis-1076 47 24 stronger strong ADJ fcis-1076 47 25 expression expression NOUN fcis-1076 47 26 ability ability NOUN fcis-1076 47 27 will will AUX fcis-1076 47 28 perform perform VERB fcis-1076 47 29 better well ADV fcis-1076 47 30 . . PUNCT fcis-1076 48 1 zhu zhu PROPN fcis-1076 48 2 et et PROPN fcis-1076 48 3 al al PROPN fcis-1076 48 4 . . PUNCT fcis-1076 49 1 [ [ X fcis-1076 49 2 9 9 NUM fcis-1076 49 3 , , PUNCT fcis-1076 49 4 10 10 NUM fcis-1076 49 5 ] ] PUNCT fcis-1076 49 6 used use VERB fcis-1076 49 7 bidirectional bidirectional ADJ fcis-1076 49 8 lstm lstm NOUN fcis-1076 49 9 for for ADP fcis-1076 49 10 word word NOUN fcis-1076 49 11 level level NOUN fcis-1076 49 12 coding code VERB fcis-1076 49 13 with with ADP fcis-1076 49 14 reference reference NOUN fcis-1076 49 15 to to ADP fcis-1076 49 16 the the DET fcis-1076 49 17 pointer pointer NOUN fcis-1076 49 18 generator generator NOUN fcis-1076 49 19 network network NOUN fcis-1076 49 20 ; ; PUNCT fcis-1076 49 21 khullar khullar PROPN fcis-1076 49 22 et et PROPN fcis-1076 49 23 al al PROPN fcis-1076 49 24 . . PROPN fcis-1076 49 25 used use VERB fcis-1076 49 26 bidirectional bidirectional PROPN fcis-1076 49 27 gru gru PROPN fcis-1076 49 28 to to PART fcis-1076 49 29 encode encode VERB fcis-1076 49 30 text text NOUN fcis-1076 49 31 . . PUNCT fcis-1076 50 1 except except SCONJ fcis-1076 50 2 that that DET fcis-1076 50 3 gru gru NOUN fcis-1076 50 4 and and CCONJ fcis-1076 50 5 lstm lstm NOUN fcis-1076 50 6 are be AUX fcis-1076 50 7 encoded encode VERB fcis-1076 50 8 in in ADP fcis-1076 50 9 writing writing NOUN fcis-1076 50 10 , , PUNCT fcis-1076 50 11 the the DET fcis-1076 50 12 emergence emergence NOUN fcis-1076 50 13 of of ADP fcis-1076 50 14 transformer transformer NOUN fcis-1076 50 15 , , PUNCT fcis-1076 50 16 a a DET fcis-1076 50 17 variant variant NOUN fcis-1076 50 18 of of ADP fcis-1076 50 19 rnn rnn PROPN fcis-1076 50 20 , , PUNCT fcis-1076 50 21 has have AUX fcis-1076 50 22 quickly quickly ADV fcis-1076 50 23 attracted attract VERB fcis-1076 50 24 researchers researcher NOUN fcis-1076 50 25 ' ' PART fcis-1076 50 26 attention attention NOUN fcis-1076 50 27 . . PUNCT fcis-1076 51 1 table table NOUN fcis-1076 51 2 1 1 NUM fcis-1076 51 3 shows show VERB fcis-1076 51 4 the the DET fcis-1076 51 5 advantages advantage NOUN fcis-1076 51 6 and and CCONJ fcis-1076 51 7 disadvantages disadvantage NOUN fcis-1076 51 8 of of ADP fcis-1076 51 9 different different ADJ fcis-1076 51 10 text text NOUN fcis-1076 51 11 coding code VERB fcis-1076 51 12 methods method NOUN fcis-1076 51 13 . . PUNCT fcis-1076 52 1 table table NOUN fcis-1076 52 2 1 1 NUM fcis-1076 52 3 . . PUNCT fcis-1076 52 4 text text NOUN fcis-1076 52 5 coding coding NOUN fcis-1076 52 6 method method NOUN fcis-1076 52 7 and and CCONJ fcis-1076 52 8 its its PRON fcis-1076 52 9 advantages advantage NOUN fcis-1076 52 10 and and CCONJ fcis-1076 52 11 disadvantages disadvantage VERB fcis-1076 52 12 text text NOUN fcis-1076 52 13 encoding encoding NOUN fcis-1076 52 14 method method NOUN fcis-1076 52 15 advantages advantage NOUN fcis-1076 52 16 disadvantages disadvantage VERB fcis-1076 52 17 traditional traditional ADJ fcis-1076 52 18 rnn rnn NOUN fcis-1076 52 19 it it PRON fcis-1076 52 20 has have VERB fcis-1076 52 21 simple simple ADJ fcis-1076 52 22 internal internal ADJ fcis-1076 52 23 structure structure NOUN fcis-1076 52 24 , , PUNCT fcis-1076 52 25 low low ADJ fcis-1076 52 26 requirements requirement NOUN fcis-1076 52 27 for for ADP fcis-1076 52 28 computing compute VERB fcis-1076 52 29 resources resource NOUN fcis-1076 52 30 , , PUNCT fcis-1076 52 31 and and CCONJ fcis-1076 52 32 excellent excellent ADJ fcis-1076 52 33 performance performance NOUN fcis-1076 52 34 on on ADP fcis-1076 52 35 short short ADJ fcis-1076 52 36 sequence sequence NOUN fcis-1076 52 37 tasks task VERB fcis-1076 52 38 gradient gradient ADJ fcis-1076 52 39 disappearance disappearance NOUN fcis-1076 52 40 or or CCONJ fcis-1076 52 41 explosion explosion NOUN fcis-1076 52 42 is be AUX fcis-1076 52 43 easy easy ADJ fcis-1076 52 44 to to PART fcis-1076 52 45 occur occur VERB fcis-1076 52 46 when when SCONJ fcis-1076 52 47 processing process VERB fcis-1076 52 48 long long ADJ fcis-1076 52 49 sequences sequence NOUN fcis-1076 52 50 , , PUNCT fcis-1076 52 51 and and CCONJ fcis-1076 52 52 parallel parallel ADJ fcis-1076 52 53 computing computing NOUN fcis-1076 52 54 is be AUX fcis-1076 52 55 impossible impossible ADJ fcis-1076 52 56 lstm lstm ADJ fcis-1076 53 1 [ [ X fcis-1076 53 2 9 9 NUM fcis-1076 53 3 , , PUNCT fcis-1076 53 4 10 10 NUM fcis-1076 53 5 ] ] PUNCT fcis-1076 53 6 with with ADP fcis-1076 53 7 long long ADJ fcis-1076 53 8 - - PUNCT fcis-1076 53 9 term term NOUN fcis-1076 53 10 memory memory NOUN fcis-1076 53 11 function function NOUN fcis-1076 53 12 , , PUNCT fcis-1076 53 13 it it PRON fcis-1076 53 14 can can AUX fcis-1076 53 15 alleviate alleviate VERB fcis-1076 53 16 the the DET fcis-1076 53 17 gradient gradient ADJ fcis-1076 53 18 disappearance disappearance NOUN fcis-1076 53 19 or or CCONJ fcis-1076 53 20 explosion explosion NOUN fcis-1076 53 21 of of ADP fcis-1076 53 22 traditional traditional ADJ fcis-1076 53 23 rnn rnn NOUN fcis-1076 53 24 , , PUNCT fcis-1076 53 25 and and CCONJ fcis-1076 53 26 is be AUX fcis-1076 53 27 suitable suitable ADJ fcis-1076 53 28 for for ADP fcis-1076 53 29 processing process VERB fcis-1076 53 30 long long ADJ fcis-1076 53 31 sequences sequence NOUN fcis-1076 53 32 the the DET fcis-1076 53 33 internal internal ADJ fcis-1076 53 34 structure structure NOUN fcis-1076 53 35 is be AUX fcis-1076 53 36 complex complex ADJ fcis-1076 53 37 , , PUNCT fcis-1076 53 38 and and CCONJ fcis-1076 53 39 the the DET fcis-1076 53 40 training training NOUN fcis-1076 53 41 efficiency efficiency NOUN fcis-1076 53 42 is be AUX fcis-1076 53 43 far far ADV fcis-1076 53 44 lower low ADJ fcis-1076 53 45 than than ADP fcis-1076 53 46 that that PRON fcis-1076 53 47 of of ADP fcis-1076 53 48 traditional traditional ADJ fcis-1076 53 49 rnn rnn NOUN fcis-1076 53 50 under under ADP fcis-1076 53 51 the the DET fcis-1076 53 52 same same ADJ fcis-1076 53 53 computing computing NOUN fcis-1076 53 54 power power NOUN fcis-1076 53 55 , , PUNCT fcis-1076 53 56 so so SCONJ fcis-1076 53 57 parallel parallel ADJ fcis-1076 53 58 computing computing NOUN fcis-1076 53 59 is be AUX fcis-1076 53 60 impossible impossible ADJ fcis-1076 53 61 gru gru NOUN fcis-1076 53 62 with with ADP fcis-1076 53 63 long long ADJ fcis-1076 53 64 - - PUNCT fcis-1076 53 65 term term NOUN fcis-1076 53 66 memory memory NOUN fcis-1076 53 67 function function NOUN fcis-1076 53 68 , , PUNCT fcis-1076 53 69 it it PRON fcis-1076 53 70 can can AUX fcis-1076 53 71 alleviate alleviate VERB fcis-1076 53 72 the the DET fcis-1076 53 73 gradient gradient ADJ fcis-1076 53 74 disappearance disappearance NOUN fcis-1076 53 75 or or CCONJ fcis-1076 53 76 explosion explosion NOUN fcis-1076 53 77 of of ADP fcis-1076 53 78 traditional traditional ADJ fcis-1076 53 79 rnn rnn NOUN fcis-1076 53 80 . . PUNCT fcis-1076 54 1 it it PRON fcis-1076 54 2 is be AUX fcis-1076 54 3 suitable suitable ADJ fcis-1076 54 4 for for ADP fcis-1076 54 5 processing process VERB fcis-1076 54 6 long long ADJ fcis-1076 54 7 sequences sequence NOUN fcis-1076 54 8 , , PUNCT fcis-1076 54 9 and and CCONJ fcis-1076 54 10 the the DET fcis-1076 54 11 computational computational ADJ fcis-1076 54 12 complexity complexity NOUN fcis-1076 54 13 is be AUX fcis-1076 54 14 smaller small ADJ fcis-1076 54 15 than than ADP fcis-1076 54 16 lstm lstm NOUN fcis-1076 54 17 it it PRON fcis-1076 54 18 can can AUX fcis-1076 54 19 not not PART fcis-1076 54 20 completely completely ADV fcis-1076 54 21 solve solve VERB fcis-1076 54 22 the the DET fcis-1076 54 23 problem problem NOUN fcis-1076 54 24 of of ADP fcis-1076 54 25 gradient gradient ADJ fcis-1076 54 26 disappearance disappearance NOUN fcis-1076 54 27 and and CCONJ fcis-1076 54 28 parallel parallel ADJ fcis-1076 54 29 computing computing NOUN fcis-1076 54 30 transformer transformer NOUN fcis-1076 54 31 it it PRON fcis-1076 54 32 can can AUX fcis-1076 54 33 be be AUX fcis-1076 54 34 calculated calculate VERB fcis-1076 54 35 in in ADP fcis-1076 54 36 parallel parallel NOUN fcis-1076 54 37 and and CCONJ fcis-1076 54 38 has have VERB fcis-1076 54 39 infinite infinite ADJ fcis-1076 54 40 memory memory NOUN fcis-1076 54 41 function function NOUN fcis-1076 54 42 weak weak ADJ fcis-1076 54 43 local local ADJ fcis-1076 54 44 information information NOUN fcis-1076 54 45 acquisition acquisition NOUN fcis-1076 54 46 ability ability NOUN fcis-1076 54 47 , , PUNCT fcis-1076 54 48 unable unable ADJ fcis-1076 54 49 to to PART fcis-1076 54 50 characterize characterize VERB fcis-1076 54 51 location location NOUN fcis-1076 54 52 information information NOUN fcis-1076 54 53 most most ADJ fcis-1076 54 54 visual visual ADJ fcis-1076 54 55 encoders encoder NOUN fcis-1076 54 56 do do AUX fcis-1076 54 57 not not PART fcis-1076 54 58 train train VERB fcis-1076 54 59 parameter parameter NOUN fcis-1076 54 60 weights weight NOUN fcis-1076 54 61 from from ADP fcis-1076 54 62 scratch scratch NOUN fcis-1076 54 63 , , PUNCT fcis-1076 54 64 but but CCONJ fcis-1076 54 65 prefer prefer VERB fcis-1076 54 66 to to PART fcis-1076 54 67 use use VERB fcis-1076 54 68 cnn cnn PROPN fcis-1076 54 69 based base VERB fcis-1076 54 70 pre pre ADJ fcis-1076 54 71 - - ADJ fcis-1076 54 72 training training ADJ fcis-1076 54 73 embedding embedding NOUN fcis-1076 54 74 . . PUNCT fcis-1076 55 1 most most ADJ fcis-1076 55 2 existing exist VERB fcis-1076 55 3 work work NOUN fcis-1076 55 4 uses use VERB fcis-1076 55 5 pre pre ADJ fcis-1076 55 6 - - ADJ fcis-1076 55 7 trained train VERB fcis-1076 55 8 networks network NOUN fcis-1076 55 9 ( ( PUNCT fcis-1076 55 10 such such ADJ fcis-1076 55 11 as as ADP fcis-1076 55 12 resnet resnet NOUN fcis-1076 55 13 , , PUNCT fcis-1076 55 14 vggnet vggnet NOUN fcis-1076 55 15 , , PUNCT fcis-1076 55 16 googlenet googlenet NOUN fcis-1076 55 17 ) ) PUNCT fcis-1076 55 18 to to PART fcis-1076 55 19 train train VERB fcis-1076 55 20 on on ADP fcis-1076 55 21 large large ADJ fcis-1076 55 22 image image NOUN fcis-1076 55 23 classification classification NOUN fcis-1076 55 24 datasets dataset NOUN fcis-1076 55 25 such such ADJ fcis-1076 55 26 as as ADP fcis-1076 55 27 imagenet imagenet NOUN fcis-1076 55 28 . . PUNCT fcis-1076 56 1 table table NOUN fcis-1076 56 2 2 2 NUM fcis-1076 56 3 lists list VERB fcis-1076 56 4 the the DET fcis-1076 56 5 comparison comparison NOUN fcis-1076 56 6 , , PUNCT fcis-1076 56 7 advantages advantage NOUN fcis-1076 56 8 and and CCONJ fcis-1076 56 9 disadvantages disadvantage NOUN fcis-1076 56 10 of of ADP fcis-1076 56 11 visual visual ADJ fcis-1076 56 12 coding code VERB fcis-1076 56 13 methods method NOUN fcis-1076 56 14 . . PUNCT fcis-1076 57 1 15 15 NUM fcis-1076 57 2 table table NOUN fcis-1076 57 3 2 2 NUM fcis-1076 57 4 . . X fcis-1076 57 5 visual visual ADJ fcis-1076 57 6 coding coding NOUN fcis-1076 57 7 method method NOUN fcis-1076 57 8 and and CCONJ fcis-1076 57 9 its its PRON fcis-1076 57 10 advantages advantage NOUN fcis-1076 57 11 and and CCONJ fcis-1076 57 12 disadvantages disadvantage VERB fcis-1076 57 13 visual visual ADJ fcis-1076 57 14 encoding encoding NOUN fcis-1076 57 15 method method PROPN fcis-1076 57 16 vggnet vggnet PROPN fcis-1076 57 17 googlenet googlenet PROPN fcis-1076 57 18 resnet resnet VERB fcis-1076 57 19 first first ADJ fcis-1076 57 20 appearance appearance NOUN fcis-1076 57 21 2014 2014 NUM fcis-1076 57 22 2014 2014 NUM fcis-1076 57 23 2015 2015 NUM fcis-1076 57 24 number number NOUN fcis-1076 57 25 of of ADP fcis-1076 57 26 layers layer NOUN fcis-1076 57 27 19 19 NUM fcis-1076 57 28 22 22 NUM fcis-1076 57 29 152 152 NUM fcis-1076 57 30 top-5 top-5 NOUN fcis-1076 57 31 error error NOUN fcis-1076 57 32 7.3 7.3 NUM fcis-1076 57 33 % % NOUN fcis-1076 57 34 6.7 6.7 NUM fcis-1076 57 35 % % NOUN fcis-1076 57 36 3.57 3.57 NUM fcis-1076 57 37 % % NOUN fcis-1076 57 38 convolution convolution NOUN fcis-1076 57 39 layers layer NOUN fcis-1076 57 40 16 16 NUM fcis-1076 57 41 21 21 NUM fcis-1076 57 42 151 151 NUM fcis-1076 57 43 convolution convolution NOUN fcis-1076 57 44 kernel kernel NOUN fcis-1076 57 45 size size NOUN fcis-1076 58 1 3 3 NUM fcis-1076 58 2 7,1,3,5 7,1,3,5 NUM fcis-1076 58 3 7,1,3,5 7,1,3,5 NUM fcis-1076 58 4 number number NOUN fcis-1076 58 5 of of ADP fcis-1076 58 6 fully fully ADV fcis-1076 58 7 connected connected ADJ fcis-1076 58 8 layers layer NOUN fcis-1076 58 9 3 3 NUM fcis-1076 58 10 1 1 NUM fcis-1076 58 11 1 1 NUM fcis-1076 58 12 full full ADJ fcis-1076 58 13 connection connection NOUN fcis-1076 58 14 layer layer NOUN fcis-1076 58 15 size size NOUN fcis-1076 58 16 4096,4096,1000 4096,4096,1000 NOUN fcis-1076 58 17 1000 1000 NUM fcis-1076 58 18 1000 1000 NUM fcis-1076 58 19 advantages advantage NOUN fcis-1076 58 20 good good ADJ fcis-1076 58 21 generalization generalization NOUN fcis-1076 58 22 performance performance NOUN fcis-1076 58 23 with with ADP fcis-1076 58 24 small small ADJ fcis-1076 58 25 parameters parameter NOUN fcis-1076 58 26 and and CCONJ fcis-1076 58 27 good good ADJ fcis-1076 58 28 performance performance NOUN fcis-1076 58 29 , , PUNCT fcis-1076 58 30 it it PRON fcis-1076 58 31 can can AUX fcis-1076 58 32 make make VERB fcis-1076 58 33 more more ADV fcis-1076 58 34 efficient efficient ADJ fcis-1076 58 35 use use NOUN fcis-1076 58 36 of of ADP fcis-1076 58 37 computing compute VERB fcis-1076 58 38 resources resource NOUN fcis-1076 58 39 and and CCONJ fcis-1076 58 40 extract extract VERB fcis-1076 58 41 more more ADJ fcis-1076 58 42 features feature NOUN fcis-1076 58 43 under under ADP fcis-1076 58 44 the the DET fcis-1076 58 45 same same ADJ fcis-1076 58 46 amount amount NOUN fcis-1076 58 47 of of ADP fcis-1076 58 48 computing compute VERB fcis-1076 58 49 deeper deep ADJ fcis-1076 58 50 level level NOUN fcis-1076 58 51 and and CCONJ fcis-1076 58 52 higher high ADJ fcis-1076 58 53 accuracy accuracy NOUN fcis-1076 58 54 disadvantages disadvantage VERB fcis-1076 58 55 large large ADJ fcis-1076 58 56 number number NOUN fcis-1076 58 57 of of ADP fcis-1076 58 58 parameters parameter NOUN fcis-1076 58 59 , , PUNCT fcis-1076 58 60 easy easy ADJ fcis-1076 58 61 over over ADP fcis-1076 58 62 fitting fitting ADJ fcis-1076 58 63 and and CCONJ fcis-1076 58 64 low low ADJ fcis-1076 58 65 efficiency efficiency NOUN fcis-1076 58 66 efficiency efficiency NOUN fcis-1076 58 67 depends depend VERB fcis-1076 58 68 on on ADP fcis-1076 58 69 model model NOUN fcis-1076 58 70 although although SCONJ fcis-1076 58 71 most most ADJ fcis-1076 58 72 works work NOUN fcis-1076 58 73 use use VERB fcis-1076 58 74 the the DET fcis-1076 58 75 information information NOUN fcis-1076 58 76 of of ADP fcis-1076 58 77 the the DET fcis-1076 58 78 target target NOUN fcis-1076 58 79 data datum NOUN fcis-1076 58 80 set set VERB fcis-1076 58 81 to to PART fcis-1076 58 82 train train VERB fcis-1076 58 83 their their PRON fcis-1076 58 84 own own ADJ fcis-1076 58 85 shared share VERB fcis-1076 58 86 embedding embed VERB fcis-1076 58 87 space space NOUN fcis-1076 58 88 for for ADP fcis-1076 58 89 multiple multiple ADJ fcis-1076 58 90 modes mode NOUN fcis-1076 58 91 , , PUNCT fcis-1076 58 92 a a DET fcis-1076 58 93 considerable considerable ADJ fcis-1076 58 94 number number NOUN fcis-1076 58 95 of of ADP fcis-1076 58 96 works work NOUN fcis-1076 58 97 [ [ X fcis-1076 58 98 1 1 NUM fcis-1076 58 99 , , PUNCT fcis-1076 58 100 2 2 NUM fcis-1076 58 101 ] ] PUNCT fcis-1076 58 102 also also ADV fcis-1076 58 103 tend tend VERB fcis-1076 58 104 to to PART fcis-1076 58 105 use use VERB fcis-1076 58 106 the the DET fcis-1076 58 107 pre pre NOUN fcis-1076 58 108 training training NOUN fcis-1076 58 109 neural neural ADJ fcis-1076 58 110 network network NOUN fcis-1076 58 111 model model NOUN fcis-1076 58 112 trained train VERB fcis-1076 58 113 on on ADP fcis-1076 58 114 the the DET fcis-1076 58 115 image image NOUN fcis-1076 58 116 caption caption NOUN fcis-1076 58 117 dataset dataset VERB fcis-1076 59 1 [ [ X fcis-1076 59 2 17,18 17,18 NUM fcis-1076 59 3 ] ] X fcis-1076 59 4 , , PUNCT fcis-1076 59 5 such such ADJ fcis-1076 59 6 as as ADP fcis-1076 59 7 pascal1k pascal1k PROPN fcis-1076 59 8 , , PUNCT fcis-1076 59 9 flickr8k flickr8k NOUN fcis-1076 59 10 , , PUNCT fcis-1076 59 11 flick30k flick30k ADJ fcis-1076 59 12 , , PUNCT fcis-1076 59 13 so so SCONJ fcis-1076 59 14 as as SCONJ fcis-1076 59 15 to to PART fcis-1076 59 16 take take VERB fcis-1076 59 17 advantage advantage NOUN fcis-1076 59 18 of of ADP fcis-1076 59 19 the the DET fcis-1076 59 20 information information NOUN fcis-1076 59 21 overlap overlap NOUN fcis-1076 59 22 between between ADP fcis-1076 59 23 different different ADJ fcis-1076 59 24 modes mode NOUN fcis-1076 59 25 . . PUNCT fcis-1076 60 1 this this PRON fcis-1076 60 2 is be AUX fcis-1076 60 3 necessary necessary ADJ fcis-1076 60 4 for for ADP fcis-1076 60 5 small small ADJ fcis-1076 60 6 datasets dataset NOUN fcis-1076 60 7 that that PRON fcis-1076 60 8 are be AUX fcis-1076 60 9 mainly mainly ADV fcis-1076 60 10 used use VERB fcis-1076 60 11 to to PART fcis-1076 60 12 extract extract VERB fcis-1076 60 13 summaries summary NOUN fcis-1076 60 14 . . PUNCT fcis-1076 61 1 however however ADV fcis-1076 61 2 , , PUNCT fcis-1076 61 3 even even ADV fcis-1076 61 4 these these DET fcis-1076 61 5 pre pre ADJ fcis-1076 61 6 - - ADJ fcis-1076 61 7 trained train VERB fcis-1076 61 8 models model NOUN fcis-1076 61 9 can can AUX fcis-1076 61 10 not not PART fcis-1076 61 11 process process VERB fcis-1076 61 12 the the DET fcis-1076 61 13 original original ADJ fcis-1076 61 14 data datum NOUN fcis-1076 61 15 . . PUNCT fcis-1076 62 1 therefore therefore ADV fcis-1076 62 2 , , PUNCT fcis-1076 62 3 text text NOUN fcis-1076 62 4 and and CCONJ fcis-1076 62 5 image image NOUN fcis-1076 62 6 input input NOUN fcis-1076 62 7 are be AUX fcis-1076 62 8 first first ADV fcis-1076 62 9 preprocessed preprocesse VERB fcis-1076 62 10 into into ADP fcis-1076 62 11 the the DET fcis-1076 62 12 required require VERB fcis-1076 62 13 embedded embed VERB fcis-1076 62 14 format format NOUN fcis-1076 62 15 , , PUNCT fcis-1076 62 16 and and CCONJ fcis-1076 62 17 then then ADV fcis-1076 62 18 fed feed VERB fcis-1076 62 19 to to ADP fcis-1076 62 20 these these DET fcis-1076 62 21 models model NOUN fcis-1076 62 22 with with ADP fcis-1076 62 23 pre pre ADJ fcis-1076 62 24 - - ADJ fcis-1076 62 25 trained train VERB fcis-1076 62 26 weights weight NOUN fcis-1076 62 27 . . PUNCT fcis-1076 63 1 for for ADP fcis-1076 63 2 example example NOUN fcis-1076 63 3 , , PUNCT fcis-1076 63 4 wang wang PROPN fcis-1076 63 5 et et PROPN fcis-1076 63 6 al al PROPN fcis-1076 63 7 . . PROPN fcis-1076 63 8 obtained obtain VERB fcis-1076 63 9 a a DET fcis-1076 63 10 6000 6000 NUM fcis-1076 63 11 - - PUNCT fcis-1076 63 12 dimensional dimensional ADJ fcis-1076 63 13 sentence sentence NOUN fcis-1076 63 14 vector vector NOUN fcis-1076 63 15 by by ADP fcis-1076 63 16 applying apply VERB fcis-1076 63 17 principal principal ADJ fcis-1076 63 18 component component NOUN fcis-1076 63 19 analysis analysis NOUN fcis-1076 63 20 on on ADP fcis-1076 63 21 the the DET fcis-1076 63 22 18000 18000 NUM fcis-1076 63 23 - - PUNCT fcis-1076 63 24 dimensional dimensional ADJ fcis-1076 63 25 vector vector NOUN fcis-1076 63 26 obtained obtain VERB fcis-1076 63 27 by by ADP fcis-1076 63 28 the the DET fcis-1076 63 29 gaussian gaussian ADJ fcis-1076 63 30 laplace laplace NOUN fcis-1076 63 31 mixture mixture NOUN fcis-1076 63 32 model model NOUN fcis-1076 63 33 , , PUNCT fcis-1076 63 34 and and CCONJ fcis-1076 63 35 a a DET fcis-1076 63 36 4096dimensional 4096dimensional ADJ fcis-1076 63 37 image image NOUN fcis-1076 63 38 vector vector NOUN fcis-1076 63 39 by by ADP fcis-1076 63 40 extracting extract VERB fcis-1076 63 41 weights weight NOUN fcis-1076 63 42 from from ADP fcis-1076 63 43 the the DET fcis-1076 63 44 full full ADJ fcis-1076 63 45 connection connection NOUN fcis-1076 63 46 layer layer NOUN fcis-1076 63 47 of of ADP fcis-1076 63 48 vggnet vggnet NOUN fcis-1076 63 49 . . PUNCT fcis-1076 64 1 2.2 2.2 NUM fcis-1076 64 2 . . PUNCT fcis-1076 64 3 multimodal multimodal ADJ fcis-1076 64 4 fusion fusion NOUN fcis-1076 64 5 module module NOUN fcis-1076 64 6 with with ADP fcis-1076 64 7 the the DET fcis-1076 64 8 introduction introduction NOUN fcis-1076 64 9 of of ADP fcis-1076 64 10 attention attention NOUN fcis-1076 64 11 mechanism mechanism NOUN fcis-1076 64 12 , , PUNCT fcis-1076 64 13 most most ADJ fcis-1076 64 14 tasks task NOUN fcis-1076 64 15 based base VERB fcis-1076 64 16 on on ADP fcis-1076 64 17 text text NOUN fcis-1076 64 18 image image NOUN fcis-1076 64 19 input input NOUN fcis-1076 64 20 focus focus NOUN fcis-1076 64 21 on on ADP fcis-1076 64 22 the the DET fcis-1076 64 23 use use NOUN fcis-1076 64 24 of of ADP fcis-1076 64 25 multimodal multimodal ADJ fcis-1076 64 26 attention attention NOUN fcis-1076 64 27 mechanism mechanism NOUN fcis-1076 64 28 . . PUNCT fcis-1076 65 1 attention attention NOUN fcis-1076 65 2 strategies strategy NOUN fcis-1076 65 3 have have AUX fcis-1076 65 4 proved prove VERB fcis-1076 65 5 to to PART fcis-1076 65 6 be be AUX fcis-1076 65 7 a a DET fcis-1076 65 8 very very ADV fcis-1076 65 9 useful useful ADJ fcis-1076 65 10 technique technique NOUN fcis-1076 65 11 to to PART fcis-1076 65 12 eliminate eliminate VERB fcis-1076 65 13 noise noise NOUN fcis-1076 65 14 and and CCONJ fcis-1076 65 15 focus focus VERB fcis-1076 65 16 on on ADP fcis-1076 65 17 relevant relevant ADJ fcis-1076 65 18 information information NOUN fcis-1076 65 19 . . PUNCT fcis-1076 66 1 it it PRON fcis-1076 66 2 has have AUX fcis-1076 66 3 been be AUX fcis-1076 66 4 applied apply VERB fcis-1076 66 5 to to ADP fcis-1076 66 6 specific specific ADJ fcis-1076 66 7 modal modal ADJ fcis-1076 66 8 information information NOUN fcis-1076 66 9 and and CCONJ fcis-1076 66 10 information information NOUN fcis-1076 66 11 sharing sharing NOUN fcis-1076 66 12 steps step NOUN fcis-1076 66 13 in in ADP fcis-1076 66 14 the the DET fcis-1076 66 15 form form NOUN fcis-1076 66 16 of of ADP fcis-1076 66 17 multimodal multimodal ADJ fcis-1076 66 18 attention attention NOUN fcis-1076 66 19 to to PART fcis-1076 66 20 determine determine VERB fcis-1076 66 21 the the DET fcis-1076 66 22 participation participation NOUN fcis-1076 66 23 of of ADP fcis-1076 66 24 specific specific ADJ fcis-1076 66 25 modes mode NOUN fcis-1076 66 26 of of ADP fcis-1076 66 27 each each DET fcis-1076 66 28 input input NOUN fcis-1076 66 29 . . PUNCT fcis-1076 67 1 li li PROPN fcis-1076 67 2 et et PROPN fcis-1076 67 3 al al PROPN fcis-1076 67 4 . . PROPN fcis-1076 68 1 first first PROPN fcis-1076 68 2 proposed propose VERB fcis-1076 68 3 hierarchical hierarchical ADJ fcis-1076 68 4 multimodal multimodal NOUN fcis-1076 68 5 attention attention NOUN fcis-1076 68 6 to to PART fcis-1076 68 7 solve solve VERB fcis-1076 68 8 the the DET fcis-1076 68 9 multimodal multimodal ADJ fcis-1076 68 10 summary summary NOUN fcis-1076 68 11 task task NOUN fcis-1076 68 12 of of ADP fcis-1076 68 13 long long ADJ fcis-1076 68 14 sentences sentence NOUN fcis-1076 68 15 . . PUNCT fcis-1076 69 1 the the DET fcis-1076 69 2 attention attention NOUN fcis-1076 69 3 module module NOUN fcis-1076 69 4 includes include VERB fcis-1076 69 5 a a DET fcis-1076 69 6 separate separate ADJ fcis-1076 69 7 text text NOUN fcis-1076 69 8 attention attention NOUN fcis-1076 69 9 layer layer NOUN fcis-1076 69 10 and and CCONJ fcis-1076 69 11 image image NOUN fcis-1076 69 12 attention attention NOUN fcis-1076 69 13 layer layer NOUN fcis-1076 69 14 , , PUNCT fcis-1076 69 15 and and CCONJ fcis-1076 69 16 then then ADV fcis-1076 69 17 multimodal multimodal ADJ fcis-1076 69 18 attention attention NOUN fcis-1076 69 19 layer layer NOUN fcis-1076 69 20 . . PUNCT fcis-1076 70 1 although although SCONJ fcis-1076 70 2 multimodal multimodal ADJ fcis-1076 70 3 attention attention NOUN fcis-1076 70 4 shows show VERB fcis-1076 70 5 great great ADJ fcis-1076 70 6 potential potential NOUN fcis-1076 70 7 in in ADP fcis-1076 70 8 the the DET fcis-1076 70 9 task task NOUN fcis-1076 70 10 of of ADP fcis-1076 70 11 text text NOUN fcis-1076 70 12 image image NOUN fcis-1076 70 13 summarization summarization NOUN fcis-1076 70 14 , , PUNCT fcis-1076 70 15 it it PRON fcis-1076 70 16 is be AUX fcis-1076 70 17 not not PART fcis-1076 70 18 enough enough ADJ fcis-1076 70 19 to to PART fcis-1076 70 20 complete complete VERB fcis-1076 70 21 the the DET fcis-1076 70 22 task task NOUN fcis-1076 70 23 of of ADP fcis-1076 70 24 text text NOUN fcis-1076 70 25 video video NOUN fcis-1076 70 26 audio audio NOUN fcis-1076 70 27 summarization summarization NOUN fcis-1076 70 28 by by ADP fcis-1076 70 29 itself itself PRON fcis-1076 70 30 . . PUNCT fcis-1076 71 1 therefore therefore ADV fcis-1076 71 2 , , PUNCT fcis-1076 71 3 in in ADP fcis-1076 71 4 order order NOUN fcis-1076 71 5 to to PART fcis-1076 71 6 overcome overcome VERB fcis-1076 71 7 this this DET fcis-1076 71 8 weakness weakness NOUN fcis-1076 71 9 , , PUNCT fcis-1076 71 10 fu fu PROPN fcis-1076 71 11 et et PROPN fcis-1076 71 12 al al PROPN fcis-1076 71 13 . . PROPN fcis-1076 71 14 proposed propose VERB fcis-1076 71 15 the the DET fcis-1076 71 16 double double ADJ fcis-1076 71 17 jump jump NOUN fcis-1076 71 18 attention attention NOUN fcis-1076 71 19 mechanism mechanism NOUN fcis-1076 71 20 as as ADP fcis-1076 71 21 an an DET fcis-1076 71 22 extension extension NOUN fcis-1076 71 23 of of ADP fcis-1076 71 24 bilinear bilinear ADJ fcis-1076 71 25 attention attention NOUN fcis-1076 71 26 , , PUNCT fcis-1076 71 27 while while SCONJ fcis-1076 71 28 li li PROPN fcis-1076 71 29 et et PROPN fcis-1076 71 30 al al PROPN fcis-1076 71 31 . . PROPN fcis-1076 71 32 developed develop VERB fcis-1076 71 33 a a DET fcis-1076 71 34 new new ADJ fcis-1076 71 35 conditional conditional ADJ fcis-1076 71 36 self self NOUN fcis-1076 71 37 - - PUNCT fcis-1076 71 38 attention attention NOUN fcis-1076 71 39 mechanism mechanism NOUN fcis-1076 71 40 module module NOUN fcis-1076 71 41 to to PART fcis-1076 71 42 capture capture VERB fcis-1076 71 43 video video NOUN fcis-1076 71 44 local local ADJ fcis-1076 71 45 semantic semantic ADJ fcis-1076 71 46 information information NOUN fcis-1076 71 47 based base VERB fcis-1076 71 48 on on ADP fcis-1076 71 49 input input NOUN fcis-1076 71 50 text text NOUN fcis-1076 71 51 information information NOUN fcis-1076 71 52 . . PUNCT fcis-1076 72 1 2.3 2.3 NUM fcis-1076 72 2 . . PUNCT fcis-1076 72 3 decoder decoder NOUN fcis-1076 72 4 according accord VERB fcis-1076 72 5 to to ADP fcis-1076 72 6 the the DET fcis-1076 72 7 coding code VERB fcis-1076 72 8 strategy strategy NOUN fcis-1076 72 9 used use VERB fcis-1076 72 10 , , PUNCT fcis-1076 72 11 the the DET fcis-1076 72 12 text text NOUN fcis-1076 72 13 decoder decoder NOUN fcis-1076 72 14 is be AUX fcis-1076 72 15 also also ADV fcis-1076 72 16 different different ADJ fcis-1076 72 17 , , PUNCT fcis-1076 72 18 from from ADP fcis-1076 72 19 the the DET fcis-1076 72 20 ordinary ordinary ADJ fcis-1076 72 21 one one NUM fcis-1076 72 22 - - PUNCT fcis-1076 72 23 way way NOUN fcis-1076 72 24 rnn rnn NOUN fcis-1076 72 25 that that PRON fcis-1076 72 26 generates generate VERB fcis-1076 72 27 one one NUM fcis-1076 72 28 word word NOUN fcis-1076 72 29 at at ADP fcis-1076 72 30 a a DET fcis-1076 72 31 time time NOUN fcis-1076 72 32 to to ADP fcis-1076 72 33 the the DET fcis-1076 72 34 hierarchical hierarchical ADJ fcis-1076 72 35 rnn rnn NOUN fcis-1076 72 36 decoder decoder NOUN fcis-1076 72 37 that that PRON fcis-1076 72 38 performs perform VERB fcis-1076 72 39 this this DET fcis-1076 72 40 step step NOUN fcis-1076 72 41 at at ADP fcis-1076 72 42 multiple multiple ADJ fcis-1076 72 43 granularity granularity NOUN fcis-1076 72 44 levels level NOUN fcis-1076 72 45 . . PUNCT fcis-1076 73 1 although although SCONJ fcis-1076 73 2 most most ADJ fcis-1076 73 3 neural neural ADJ fcis-1076 73 4 models model NOUN fcis-1076 73 5 only only ADV fcis-1076 73 6 focus focus VERB fcis-1076 73 7 on on ADP fcis-1076 73 8 using use VERB fcis-1076 73 9 multimodal multimodal ADJ fcis-1076 73 10 information information NOUN fcis-1076 73 11 as as ADP fcis-1076 73 12 input input NOUN fcis-1076 73 13 to to PART fcis-1076 73 14 generate generate VERB fcis-1076 73 15 text text NOUN fcis-1076 73 16 summaries summary NOUN fcis-1076 73 17 , , PUNCT fcis-1076 73 18 some some DET fcis-1076 73 19 work work NOUN fcis-1076 73 20 also also ADV fcis-1076 73 21 outputs output VERB fcis-1076 73 22 images image NOUN fcis-1076 73 23 as as ADP fcis-1076 73 24 a a DET fcis-1076 73 25 supplement supplement NOUN fcis-1076 73 26 to to ADP fcis-1076 73 27 the the DET fcis-1076 73 28 generated generate VERB fcis-1076 73 29 summaries summary NOUN fcis-1076 73 30 [ [ X fcis-1076 73 31 9 9 NUM fcis-1076 73 32 , , PUNCT fcis-1076 73 33 10 10 NUM fcis-1076 73 34 ] ] PUNCT fcis-1076 73 35 to to PART fcis-1076 73 36 enhance enhance VERB fcis-1076 73 37 text text NOUN fcis-1076 73 38 information information NOUN fcis-1076 73 39 and and CCONJ fcis-1076 73 40 improve improve VERB fcis-1076 73 41 user user NOUN fcis-1076 73 42 satisfaction satisfaction NOUN fcis-1076 73 43 . . PUNCT fcis-1076 74 1 there there PRON fcis-1076 74 2 are be VERB fcis-1076 74 3 mainly mainly ADV fcis-1076 74 4 two two NUM fcis-1076 74 5 methods method NOUN fcis-1076 74 6 to to PART fcis-1076 74 7 select select VERB fcis-1076 74 8 the the DET fcis-1076 74 9 output output NOUN fcis-1076 74 10 image image NOUN fcis-1076 74 11 . . PUNCT fcis-1076 75 1 one one NUM fcis-1076 75 2 is be AUX fcis-1076 75 3 to to PART fcis-1076 75 4 use use VERB fcis-1076 75 5 the the DET fcis-1076 75 6 multimodal multimodal ADJ fcis-1076 75 7 attention attention NOUN fcis-1076 75 8 mechanism mechanism NOUN fcis-1076 75 9 to to PART fcis-1076 75 10 determine determine VERB fcis-1076 75 11 the the DET fcis-1076 75 12 correlation correlation NOUN fcis-1076 75 13 of of ADP fcis-1076 75 14 different different ADJ fcis-1076 75 15 modes mode NOUN fcis-1076 75 16 , , PUNCT fcis-1076 75 17 so so SCONJ fcis-1076 75 18 as as SCONJ fcis-1076 75 19 to to PART fcis-1076 75 20 select select VERB fcis-1076 75 21 the the DET fcis-1076 75 22 most most ADV fcis-1076 75 23 relevant relevant ADJ fcis-1076 75 24 image image NOUN fcis-1076 75 25 . . PUNCT fcis-1076 76 1 more more ADV fcis-1076 76 2 precisely precisely ADV fcis-1076 76 3 , , PUNCT fcis-1076 76 4 the the DET fcis-1076 76 5 visual visual ADJ fcis-1076 76 6 coverage coverage NOUN fcis-1076 76 7 score score NOUN fcis-1076 76 8 after after SCONJ fcis-1076 76 9 the the DET fcis-1076 76 10 last last ADJ fcis-1076 76 11 decoding decode VERB fcis-1076 76 12 step step NOUN fcis-1076 76 13 is be AUX fcis-1076 76 14 used use VERB fcis-1076 76 15 to to PART fcis-1076 76 16 select select VERB fcis-1076 76 17 the the DET fcis-1076 76 18 most most ADV fcis-1076 76 19 relevant relevant ADJ fcis-1076 76 20 image image NOUN fcis-1076 76 21 , , PUNCT fcis-1076 76 22 such such ADJ fcis-1076 76 23 as as ADP fcis-1076 76 24 the the DET fcis-1076 76 25 sum sum NOUN fcis-1076 76 26 of of ADP fcis-1076 76 27 attention attention NOUN fcis-1076 76 28 values value NOUN fcis-1076 76 29 when when SCONJ fcis-1076 76 30 generating generate VERB fcis-1076 76 31 text text NOUN fcis-1076 76 32 summaries summary NOUN fcis-1076 76 33 . . PUNCT fcis-1076 77 1 the the DET fcis-1076 77 2 second second ADJ fcis-1076 77 3 method method NOUN fcis-1076 77 4 is be AUX fcis-1076 77 5 to to PART fcis-1076 77 6 classify classify VERB fcis-1076 77 7 the the DET fcis-1076 77 8 image image NOUN fcis-1076 77 9 selection selection NOUN fcis-1076 77 10 task task NOUN fcis-1076 77 11 into into ADP fcis-1076 77 12 the the DET fcis-1076 77 13 model model NOUN fcis-1076 77 14 . . PUNCT fcis-1076 78 1 for for ADP fcis-1076 78 2 example example NOUN fcis-1076 78 3 , , PUNCT fcis-1076 78 4 fu fu PROPN fcis-1076 78 5 et et PROPN fcis-1076 78 6 al al PROPN fcis-1076 78 7 . . PROPN fcis-1076 78 8 used use VERB fcis-1076 78 9 unsupervised unsupervised ADJ fcis-1076 78 10 learning learning NOUN fcis-1076 78 11 technology technology NOUN fcis-1076 78 12 using use VERB fcis-1076 78 13 reinforcement reinforcement NOUN fcis-1076 78 14 learning learning NOUN fcis-1076 78 15 ( ( PUNCT fcis-1076 78 16 rl rl NOUN fcis-1076 78 17 ) ) PUNCT fcis-1076 78 18 method method NOUN fcis-1076 78 19 to to PART fcis-1076 78 20 select select VERB fcis-1076 78 21 images image NOUN fcis-1076 78 22 , , PUNCT fcis-1076 78 23 which which PRON fcis-1076 78 24 takes take VERB fcis-1076 78 25 representativeness representativeness ADJ fcis-1076 78 26 and and CCONJ fcis-1076 78 27 diversity diversity NOUN fcis-1076 78 28 as as ADP fcis-1076 78 29 two two NUM fcis-1076 78 30 reward reward NOUN fcis-1076 78 31 functions function NOUN fcis-1076 78 32 of of ADP fcis-1076 78 33 rl rl ADP fcis-1076 78 34 learning learning NOUN fcis-1076 78 35 . . PUNCT fcis-1076 79 1 li li PROPN fcis-1076 79 2 et et PROPN fcis-1076 79 3 al al PROPN fcis-1076 79 4 . . PROPN fcis-1076 79 5 proposed propose VERB fcis-1076 79 6 a a DET fcis-1076 79 7 cover cover NOUN fcis-1076 79 8 box box NOUN fcis-1076 79 9 selector selector NOUN fcis-1076 79 10 , , PUNCT fcis-1076 79 11 which which PRON fcis-1076 79 12 is be AUX fcis-1076 79 13 based base VERB fcis-1076 79 14 on on ADP fcis-1076 79 15 cnn cnn PROPN fcis-1076 79 16 - - PUNCT fcis-1076 79 17 rnn rnn VERB fcis-1076 79 18 layered layered ADJ fcis-1076 79 19 video video NOUN fcis-1076 79 20 coding coding NOUN fcis-1076 79 21 , , PUNCT fcis-1076 79 22 and and CCONJ fcis-1076 79 23 uses use VERB fcis-1076 79 24 the the DET fcis-1076 79 25 conditional conditional ADJ fcis-1076 79 26 self self NOUN fcis-1076 79 27 - - PUNCT fcis-1076 79 28 attention attention NOUN fcis-1076 79 29 module module NOUN fcis-1076 79 30 to to PART fcis-1076 79 31 select select VERB fcis-1076 79 32 an an DET fcis-1076 79 33 image image NOUN fcis-1076 79 34 based base VERB fcis-1076 79 35 on on ADP fcis-1076 79 36 the the DET fcis-1076 79 37 semantics semantic NOUN fcis-1076 79 38 of of ADP fcis-1076 79 39 the the DET fcis-1076 79 40 article article NOUN fcis-1076 79 41 . . PUNCT fcis-1076 80 1 the the DET fcis-1076 80 2 three three NUM fcis-1076 80 3 frameworks framework NOUN fcis-1076 80 4 with with ADP fcis-1076 80 5 implicit implicit ADJ fcis-1076 80 6 text text NOUN fcis-1076 80 7 image image NOUN fcis-1076 80 8 summarization summarization NOUN fcis-1076 80 9 generation generation NOUN fcis-1076 80 10 features feature NOUN fcis-1076 80 11 cited cite VERB fcis-1076 80 12 above above ADP fcis-1076 80 13 adjust adjust VERB fcis-1076 80 14 the the DET fcis-1076 80 15 final final ADJ fcis-1076 80 16 loss loss NOUN fcis-1076 80 17 together together ADV fcis-1076 80 18 with with ADP fcis-1076 80 19 the the DET fcis-1076 80 20 image image NOUN fcis-1076 80 21 selection selection NOUN fcis-1076 80 22 loss loss NOUN fcis-1076 80 23 to to ADP fcis-1076 80 24 the the DET fcis-1076 80 25 weighted weighted ADJ fcis-1076 80 26 average average NOUN fcis-1076 80 27 of of ADP fcis-1076 80 28 the the DET fcis-1076 80 29 text text NOUN fcis-1076 80 30 generation generation NOUN fcis-1076 80 31 loss loss NOUN fcis-1076 80 32 . . PUNCT fcis-1076 81 1 3 3 X fcis-1076 81 2 . . X fcis-1076 81 3 evaluation evaluation NOUN fcis-1076 81 4 method method NOUN fcis-1076 81 5 in in ADP fcis-1076 81 6 order order NOUN fcis-1076 81 7 to to PART fcis-1076 81 8 evaluate evaluate VERB fcis-1076 81 9 multimodal multimodal NOUN fcis-1076 81 10 abstracts abstract NOUN fcis-1076 81 11 , , PUNCT fcis-1076 81 12 zhu zhu PROPN fcis-1076 81 13 et et PROPN fcis-1076 81 14 al al PROPN fcis-1076 81 15 . . PROPN fcis-1076 81 16 proposed propose VERB fcis-1076 81 17 a a DET fcis-1076 81 18 multimodal multimodal ADJ fcis-1076 81 19 automatic automatic ADJ fcis-1076 81 20 evaluation evaluation NOUN fcis-1076 81 21 ( ( PUNCT fcis-1076 81 22 mmae mmae NOUN fcis-1076 81 23 ) ) PUNCT fcis-1076 81 24 technology technology NOUN fcis-1076 81 25 , , PUNCT fcis-1076 81 26 which which PRON fcis-1076 81 27 comprehensively comprehensively ADV fcis-1076 81 28 considers consider VERB fcis-1076 81 29 the the DET fcis-1076 81 30 unimodal unimodal ADJ fcis-1076 81 31 significance significance NOUN fcis-1076 81 32 and and CCONJ fcis-1076 81 33 cross cross VERB fcis-1076 81 34 modal modal ADJ fcis-1076 81 35 correlation correlation NOUN fcis-1076 81 36 . . PUNCT fcis-1076 82 1 the the DET fcis-1076 82 2 final final ADJ fcis-1076 82 3 summary summary NOUN fcis-1076 82 4 is be AUX fcis-1076 82 5 composed compose VERB fcis-1076 82 6 of of ADP fcis-1076 82 7 text text NOUN fcis-1076 82 8 and and CCONJ fcis-1076 82 9 image image NOUN fcis-1076 82 10 , , PUNCT fcis-1076 82 11 and and CCONJ fcis-1076 82 12 the the DET fcis-1076 82 13 final final ADJ fcis-1076 82 14 objective objective ADJ fcis-1076 82 15 function function NOUN fcis-1076 82 16 is be AUX fcis-1076 82 17 composed compose VERB fcis-1076 82 18 of of ADP fcis-1076 82 19 three three NUM fcis-1076 82 20 objective objective ADJ fcis-1076 82 21 functions function NOUN fcis-1076 82 22 : : PUNCT fcis-1076 82 23 1 1 X fcis-1076 82 24 ) ) PUNCT fcis-1076 82 25 text text NOUN fcis-1076 82 26 significance significance NOUN fcis-1076 82 27 , , PUNCT fcis-1076 82 28 2 2 X fcis-1076 82 29 ) ) PUNCT fcis-1076 82 30 image image NOUN fcis-1076 82 31 significance significance NOUN fcis-1076 82 32 , , PUNCT fcis-1076 82 33 and and CCONJ fcis-1076 82 34 3 3 X fcis-1076 82 35 ) ) PUNCT fcis-1076 82 36 text text NOUN fcis-1076 82 37 image image NOUN fcis-1076 82 38 correlation correlation NOUN fcis-1076 82 39 . . PUNCT fcis-1076 83 1 monitoring monitor VERB fcis-1076 83 2 techniques technique NOUN fcis-1076 83 3 ( ( PUNCT fcis-1076 83 4 linear linear ADJ fcis-1076 83 5 regression regression NOUN fcis-1076 83 6 , , PUNCT fcis-1076 83 7 logistic logistic ADJ fcis-1076 83 8 regression regression NOUN fcis-1076 83 9 and and CCONJ fcis-1076 83 10 multilayer multilayer PROPN fcis-1076 83 11 perceptron perceptron PROPN fcis-1076 83 12 ) ) PUNCT fcis-1076 83 13 are be AUX fcis-1076 83 14 used use VERB fcis-1076 83 15 to to PART fcis-1076 83 16 learn learn VERB fcis-1076 83 17 the the DET fcis-1076 83 18 mapping mapping NOUN fcis-1076 83 19 function function NOUN fcis-1076 83 20 to to PART fcis-1076 83 21 minimize minimize VERB fcis-1076 83 22 the the DET fcis-1076 83 23 training training NOUN fcis-1076 83 24 loss loss NOUN fcis-1076 83 25 of of ADP fcis-1076 83 26 human human ADJ fcis-1076 83 27 judgment judgment NOUN fcis-1076 83 28 scores score NOUN fcis-1076 83 29 . . PUNCT fcis-1076 84 1 although although SCONJ fcis-1076 84 2 this this DET fcis-1076 84 3 indicator indicator NOUN fcis-1076 84 4 seems seem VERB fcis-1076 84 5 promising promise VERB fcis-1076 84 6 , , PUNCT fcis-1076 84 7 many many ADJ fcis-1076 84 8 conditions condition NOUN fcis-1076 84 9 must must AUX fcis-1076 84 10 be be AUX fcis-1076 84 11 met meet VERB fcis-1076 84 12 for for ADP fcis-1076 84 13 evaluation evaluation NOUN fcis-1076 84 14 . . PUNCT fcis-1076 85 1 because because SCONJ fcis-1076 85 2 the the DET fcis-1076 85 3 mmae mmae NOUN fcis-1076 85 4 index index NOUN fcis-1076 85 5 takes take VERB fcis-1076 85 6 the the DET fcis-1076 85 7 unimodal unimodal ADJ fcis-1076 85 8 significance significance NOUN fcis-1076 85 9 score score NOUN fcis-1076 85 10 as as ADP fcis-1076 85 11 a a DET fcis-1076 85 12 feature feature NOUN fcis-1076 85 13 of of ADP fcis-1076 85 14 the the DET fcis-1076 85 15 overall overall ADJ fcis-1076 85 16 judgment judgment NOUN fcis-1076 85 17 process process NOUN fcis-1076 85 18 , , PUNCT fcis-1076 85 19 resulting result VERB fcis-1076 85 20 in in ADP fcis-1076 85 21 cognitive cognitive ADJ fcis-1076 85 22 bias bias NOUN fcis-1076 85 23 , , PUNCT fcis-1076 85 24 it it PRON fcis-1076 85 25 can can AUX fcis-1076 85 26 not not PART fcis-1076 85 27 effectively effectively ADV fcis-1076 85 28 evaluate evaluate VERB fcis-1076 85 29 the the DET fcis-1076 85 30 information information NOUN fcis-1076 85 31 integrity integrity NOUN fcis-1076 85 32 of of ADP fcis-1076 85 33 multimodal multimodal NOUN fcis-1076 85 34 summaries summary NOUN fcis-1076 85 35 . . PUNCT fcis-1076 86 1 subsequently subsequently ADV fcis-1076 86 2 , , PUNCT fcis-1076 86 3 zhu zhu PROPN fcis-1076 86 4 et et PROPN fcis-1076 86 5 al al PROPN fcis-1076 86 6 . . PROPN fcis-1076 86 7 improved improve VERB fcis-1076 86 8 this this PRON fcis-1076 86 9 and and CCONJ fcis-1076 86 10 proposed propose VERB fcis-1076 86 11 an an DET fcis-1076 86 12 evaluation evaluation NOUN fcis-1076 86 13 index index NOUN fcis-1076 86 14 to to PART fcis-1076 86 15 summarize summarize VERB fcis-1076 86 16 the the DET fcis-1076 86 17 generated generate VERB fcis-1076 86 18 summary summary NOUN fcis-1076 86 19 and and CCONJ fcis-1076 86 20 basic basic ADJ fcis-1076 86 21 facts fact NOUN fcis-1076 86 22 into into ADP fcis-1076 86 23 a a DET fcis-1076 86 24 joint joint ADJ fcis-1076 86 25 multimodal multimodal NOUN fcis-1076 86 26 16 16 NUM fcis-1076 86 27 representation representation NOUN fcis-1076 86 28 of of ADP fcis-1076 86 29 a a DET fcis-1076 86 30 common common ADJ fcis-1076 86 31 semantic semantic ADJ fcis-1076 86 32 space space NOUN fcis-1076 86 33 . . PUNCT fcis-1076 87 1 compared compare VERB fcis-1076 87 2 with with ADP fcis-1076 87 3 other other ADJ fcis-1076 87 4 multimodal multimodal NOUN fcis-1076 87 5 evaluation evaluation NOUN fcis-1076 87 6 indicators indicator NOUN fcis-1076 87 7 , , PUNCT fcis-1076 87 8 we we PRON fcis-1076 87 9 try try VERB fcis-1076 87 10 to to PART fcis-1076 87 11 treat treat VERB fcis-1076 87 12 the the DET fcis-1076 87 13 multimodal multimodal ADJ fcis-1076 87 14 summary summary NOUN fcis-1076 87 15 as as ADP fcis-1076 87 16 a a DET fcis-1076 87 17 whole whole NOUN fcis-1076 87 18 , , PUNCT fcis-1076 87 19 rather rather ADV fcis-1076 87 20 than than ADP fcis-1076 87 21 a a DET fcis-1076 87 22 combination combination NOUN fcis-1076 87 23 of of ADP fcis-1076 87 24 segmented segment VERB fcis-1076 87 25 important important ADJ fcis-1076 87 26 elements element NOUN fcis-1076 87 27 . . PUNCT fcis-1076 88 1 the the DET fcis-1076 88 2 model model NOUN fcis-1076 88 3 based base VERB fcis-1076 88 4 on on ADP fcis-1076 88 5 neural neural ADJ fcis-1076 88 6 network network NOUN fcis-1076 88 7 is be AUX fcis-1076 88 8 used use VERB fcis-1076 88 9 to to PART fcis-1076 88 10 train train VERB fcis-1076 88 11 this this DET fcis-1076 88 12 joint joint ADJ fcis-1076 88 13 representation representation NOUN fcis-1076 88 14 . . PUNCT fcis-1076 89 1 in in ADP fcis-1076 89 2 order order NOUN fcis-1076 89 3 to to PART fcis-1076 89 4 automatically automatically ADV fcis-1076 89 5 obtain obtain VERB fcis-1076 89 6 the the DET fcis-1076 89 7 training training NOUN fcis-1076 89 8 data datum NOUN fcis-1076 89 9 of of ADP fcis-1076 89 10 joint joint ADJ fcis-1076 89 11 representation representation NOUN fcis-1076 89 12 , , PUNCT fcis-1076 89 13 the the DET fcis-1076 89 14 images image NOUN fcis-1076 89 15 of of ADP fcis-1076 89 16 two two NUM fcis-1076 89 17 image image NOUN fcis-1076 89 18 title title NOUN fcis-1076 89 19 pairs pair NOUN fcis-1076 89 20 are be AUX fcis-1076 89 21 exchanged exchange VERB fcis-1076 89 22 to to PART fcis-1076 89 23 obtain obtain VERB fcis-1076 89 24 two two NUM fcis-1076 89 25 image image NOUN fcis-1076 89 26 text text NOUN fcis-1076 89 27 pairs pair NOUN fcis-1076 89 28 with with ADP fcis-1076 89 29 similar similar ADJ fcis-1076 89 30 semantics semantic NOUN fcis-1076 89 31 . . PUNCT fcis-1076 90 1 then then ADV fcis-1076 90 2 , , PUNCT fcis-1076 90 3 the the DET fcis-1076 90 4 evaluation evaluation NOUN fcis-1076 90 5 model model NOUN fcis-1076 90 6 is be AUX fcis-1076 90 7 trained train VERB fcis-1076 90 8 using use VERB fcis-1076 90 9 the the DET fcis-1076 90 10 multimodal multimodal ADJ fcis-1076 90 11 attention attention NOUN fcis-1076 90 12 mechanism mechanism NOUN fcis-1076 90 13 to to PART fcis-1076 90 14 fuse fuse VERB fcis-1076 90 15 text text NOUN fcis-1076 90 16 and and CCONJ fcis-1076 90 17 image image NOUN fcis-1076 90 18 vectors vector NOUN fcis-1076 90 19 , , PUNCT fcis-1076 90 20 and and CCONJ fcis-1076 90 21 the the DET fcis-1076 90 22 maximum maximum ADJ fcis-1076 90 23 edge edge NOUN fcis-1076 90 24 loss loss NOUN fcis-1076 90 25 is be AUX fcis-1076 90 26 taken take VERB fcis-1076 90 27 as as ADP fcis-1076 90 28 the the DET fcis-1076 90 29 loss loss NOUN fcis-1076 90 30 function function NOUN fcis-1076 90 31 . . PUNCT fcis-1076 91 1 table table NOUN fcis-1076 91 2 3 3 NUM fcis-1076 91 3 shows show VERB fcis-1076 91 4 the the DET fcis-1076 91 5 evaluation evaluation NOUN fcis-1076 91 6 methods method NOUN fcis-1076 91 7 and and CCONJ fcis-1076 91 8 advantages advantage NOUN fcis-1076 91 9 and and CCONJ fcis-1076 91 10 disadvantage disadvantage NOUN fcis-1076 91 11 . . PUNCT fcis-1076 92 1 table table NOUN fcis-1076 92 2 3 3 NUM fcis-1076 92 3 . . PUNCT fcis-1076 92 4 evaluation evaluation NOUN fcis-1076 92 5 methods method NOUN fcis-1076 92 6 and and CCONJ fcis-1076 92 7 advantages advantage NOUN fcis-1076 92 8 and and CCONJ fcis-1076 92 9 disadvantages disadvantage VERB fcis-1076 92 10 evaluation evaluation NOUN fcis-1076 92 11 method method NOUN fcis-1076 92 12 advantages advantage NOUN fcis-1076 92 13 disadvantages disadvantage VERB fcis-1076 92 14 mmae mmae VERB fcis-1076 92 15 it it PRON fcis-1076 92 16 is be AUX fcis-1076 92 17 highly highly ADV fcis-1076 92 18 similar similar ADJ fcis-1076 92 19 to to ADP fcis-1076 92 20 the the DET fcis-1076 92 21 score score NOUN fcis-1076 92 22 of of ADP fcis-1076 92 23 manual manual ADJ fcis-1076 92 24 discrimination discrimination NOUN fcis-1076 92 25 datasets dataset NOUN fcis-1076 92 26 that that PRON fcis-1076 92 27 require require VERB fcis-1076 92 28 a a DET fcis-1076 92 29 large large ADJ fcis-1076 92 30 number number NOUN fcis-1076 92 31 of of ADP fcis-1076 92 32 manual manual ADJ fcis-1076 92 33 annotations annotation NOUN fcis-1076 92 34 ; ; PUNCT fcis-1076 92 35 abstracts abstract NOUN fcis-1076 92 36 from from ADP fcis-1076 92 37 other other ADJ fcis-1076 92 38 areas area NOUN fcis-1076 92 39 may may AUX fcis-1076 92 40 be be AUX fcis-1076 92 41 ambiguous ambiguous ADJ fcis-1076 92 42 when when SCONJ fcis-1076 92 43 evaluated evaluate VERB fcis-1076 92 44 mmae++ mmae++ ADV fcis-1076 92 45 the the DET fcis-1076 92 46 joint joint ADJ fcis-1076 92 47 multimodal multimodal NOUN fcis-1076 92 48 representation representation NOUN fcis-1076 92 49 of of ADP fcis-1076 92 50 sentence sentence NOUN fcis-1076 92 51 image image NOUN fcis-1076 92 52 pairs pair NOUN fcis-1076 92 53 is be AUX fcis-1076 92 54 used use VERB fcis-1076 92 55 to to PART fcis-1076 92 56 improve improve VERB fcis-1076 92 57 the the DET fcis-1076 92 58 correlation correlation NOUN fcis-1076 92 59 score score NOUN fcis-1076 92 60 of of ADP fcis-1076 92 61 mmae mmae NOUN fcis-1076 92 62 datasets dataset NOUN fcis-1076 92 63 that that PRON fcis-1076 92 64 require require VERB fcis-1076 92 65 a a DET fcis-1076 92 66 large large ADJ fcis-1076 92 67 number number NOUN fcis-1076 92 68 of of ADP fcis-1076 92 69 manual manual ADJ fcis-1076 92 70 annotations annotation NOUN fcis-1076 92 71 ; ; PUNCT fcis-1076 92 72 abstracts abstract NOUN fcis-1076 92 73 from from ADP fcis-1076 92 74 other other ADJ fcis-1076 92 75 areas area NOUN fcis-1076 92 76 may may AUX fcis-1076 92 77 be be AUX fcis-1076 92 78 ambiguous ambiguous ADJ fcis-1076 92 79 when when SCONJ fcis-1076 92 80 evaluated evaluate VERB fcis-1076 92 81 to to PART fcis-1076 92 82 sum sum VERB fcis-1076 92 83 up up ADP fcis-1076 92 84 , , PUNCT fcis-1076 92 85 only only ADV fcis-1076 92 86 a a DET fcis-1076 92 87 few few ADJ fcis-1076 92 88 works work NOUN fcis-1076 92 89 focus focus VERB fcis-1076 92 90 on on ADP fcis-1076 92 91 the the DET fcis-1076 92 92 evaluation evaluation NOUN fcis-1076 92 93 of of ADP fcis-1076 92 94 multimodal multimodal NOUN fcis-1076 92 95 abstracts abstract NOUN fcis-1076 92 96 , , PUNCT fcis-1076 92 97 and and CCONJ fcis-1076 92 98 the the DET fcis-1076 92 99 proposed propose VERB fcis-1076 92 100 evaluation evaluation NOUN fcis-1076 92 101 indicators indicator NOUN fcis-1076 92 102 also also ADV fcis-1076 92 103 have have VERB fcis-1076 92 104 many many ADJ fcis-1076 92 105 shortcomings shortcoming NOUN fcis-1076 92 106 . . PUNCT fcis-1076 93 1 the the DET fcis-1076 93 2 evaluation evaluation NOUN fcis-1076 93 3 index index NOUN fcis-1076 93 4 proposed propose VERB fcis-1076 93 5 by by ADP fcis-1076 93 6 zhu zhu PROPN fcis-1076 93 7 et et PROPN fcis-1076 93 8 al al PROPN fcis-1076 93 9 . . PROPN fcis-1076 93 10 and and CCONJ fcis-1076 93 11 zhu zhu PROPN fcis-1076 93 12 et et PROPN fcis-1076 93 13 al al PROPN fcis-1076 93 14 . . PROPN fcis-1076 93 15 requires require VERB fcis-1076 93 16 a a DET fcis-1076 93 17 large large ADJ fcis-1076 93 18 amount amount NOUN fcis-1076 93 19 of of ADP fcis-1076 93 20 training training NOUN fcis-1076 93 21 data datum NOUN fcis-1076 93 22 based base VERB fcis-1076 93 23 on on ADP fcis-1076 93 24 manual manual ADJ fcis-1076 93 25 evaluation evaluation NOUN fcis-1076 93 26 scores score NOUN fcis-1076 93 27 to to PART fcis-1076 93 28 learn learn VERB fcis-1076 93 29 parameter parameter NOUN fcis-1076 93 30 weights weight NOUN fcis-1076 93 31 . . PUNCT fcis-1076 94 1 in in ADP fcis-1076 94 2 addition addition NOUN fcis-1076 94 3 , , PUNCT fcis-1076 94 4 the the DET fcis-1076 94 5 evaluation evaluation NOUN fcis-1076 94 6 indicators indicator NOUN fcis-1076 94 7 depend depend VERB fcis-1076 94 8 on on ADP fcis-1076 94 9 the the DET fcis-1076 94 10 field field NOUN fcis-1076 94 11 of of ADP fcis-1076 94 12 training training NOUN fcis-1076 94 13 data datum NOUN fcis-1076 94 14 , , PUNCT fcis-1076 94 15 which which PRON fcis-1076 94 16 limits limit VERB fcis-1076 94 17 the the DET fcis-1076 94 18 scope scope NOUN fcis-1076 94 19 of of ADP fcis-1076 94 20 application application NOUN fcis-1076 94 21 of of ADP fcis-1076 94 22 these these DET fcis-1076 94 23 indicators indicator NOUN fcis-1076 94 24 . . PUNCT fcis-1076 95 1 4 4 X fcis-1076 95 2 . . X fcis-1076 95 3 conclusion conclusion NOUN fcis-1076 95 4 although although SCONJ fcis-1076 95 5 multimodal multimodal NOUN fcis-1076 95 6 summarization summarization NOUN fcis-1076 95 7 has have AUX fcis-1076 95 8 made make VERB fcis-1076 95 9 some some DET fcis-1076 95 10 progress progress NOUN fcis-1076 95 11 , , PUNCT fcis-1076 95 12 there there PRON fcis-1076 95 13 are be VERB fcis-1076 95 14 still still ADV fcis-1076 95 15 several several ADJ fcis-1076 95 16 key key ADJ fcis-1076 95 17 points point NOUN fcis-1076 95 18 worthy worthy ADJ fcis-1076 95 19 of of ADP fcis-1076 95 20 serious serious ADJ fcis-1076 95 21 consideration consideration NOUN fcis-1076 95 22 : : PUNCT fcis-1076 95 23 ( ( PUNCT fcis-1076 95 24 1 1 X fcis-1076 95 25 ) ) PUNCT fcis-1076 95 26 the the DET fcis-1076 95 27 existing exist VERB fcis-1076 95 28 model model NOUN fcis-1076 95 29 is be AUX fcis-1076 95 30 simple simple ADJ fcis-1076 95 31 in in ADP fcis-1076 95 32 structure structure NOUN fcis-1076 95 33 . . PUNCT fcis-1076 96 1 the the DET fcis-1076 96 2 existing exist VERB fcis-1076 96 3 model model NOUN fcis-1076 96 4 architecture architecture NOUN fcis-1076 96 5 is be AUX fcis-1076 96 6 basically basically ADV fcis-1076 96 7 sequence sequence NOUN fcis-1076 96 8 to to ADP fcis-1076 96 9 sequence sequence NOUN fcis-1076 96 10 model model NOUN fcis-1076 96 11 combined combine VERB fcis-1076 96 12 with with ADP fcis-1076 96 13 hierarchical hierarchical ADJ fcis-1076 96 14 attention attention NOUN fcis-1076 96 15 mechanism mechanism NOUN fcis-1076 96 16 . . PUNCT fcis-1076 97 1 different different ADJ fcis-1076 97 2 tasks task NOUN fcis-1076 97 3 will will AUX fcis-1076 97 4 be be AUX fcis-1076 97 5 improved improve VERB fcis-1076 97 6 according accord VERB fcis-1076 97 7 to to ADP fcis-1076 97 8 the the DET fcis-1076 97 9 characteristics characteristic NOUN fcis-1076 97 10 of of ADP fcis-1076 97 11 tasks task NOUN fcis-1076 97 12 . . PUNCT fcis-1076 98 1 in in ADP fcis-1076 98 2 order order NOUN fcis-1076 98 3 to to PART fcis-1076 98 4 integrate integrate VERB fcis-1076 98 5 multimodal multimodal NOUN fcis-1076 98 6 information information NOUN fcis-1076 98 7 more more ADV fcis-1076 98 8 effectively effectively ADV fcis-1076 98 9 and and CCONJ fcis-1076 98 10 give give VERB fcis-1076 98 11 full full ADJ fcis-1076 98 12 play play NOUN fcis-1076 98 13 to to ADP fcis-1076 98 14 the the DET fcis-1076 98 15 interaction interaction NOUN fcis-1076 98 16 and and CCONJ fcis-1076 98 17 complementarity complementarity NOUN fcis-1076 98 18 of of ADP fcis-1076 98 19 modal modal ADJ fcis-1076 98 20 information information NOUN fcis-1076 98 21 , , PUNCT fcis-1076 98 22 we we PRON fcis-1076 98 23 should should AUX fcis-1076 98 24 consider consider VERB fcis-1076 98 25 a a DET fcis-1076 98 26 more more ADV fcis-1076 98 27 appropriate appropriate ADJ fcis-1076 98 28 architecture architecture NOUN fcis-1076 98 29 based base VERB fcis-1076 98 30 on on ADP fcis-1076 98 31 the the DET fcis-1076 98 32 current current ADJ fcis-1076 98 33 architecture architecture NOUN fcis-1076 98 34 . . PUNCT fcis-1076 99 1 ( ( PUNCT fcis-1076 99 2 2 2 X fcis-1076 99 3 ) ) PUNCT fcis-1076 99 4 there there PRON fcis-1076 99 5 is be VERB fcis-1076 99 6 less less ADJ fcis-1076 99 7 information information NOUN fcis-1076 99 8 interaction interaction NOUN fcis-1076 99 9 between between ADP fcis-1076 99 10 different different ADJ fcis-1076 99 11 modes mode NOUN fcis-1076 99 12 . . PUNCT fcis-1076 100 1 the the DET fcis-1076 100 2 core core NOUN fcis-1076 100 3 of of ADP fcis-1076 100 4 the the DET fcis-1076 100 5 existing exist VERB fcis-1076 100 6 work work NOUN fcis-1076 100 7 mode mode NOUN fcis-1076 100 8 fusion fusion NOUN fcis-1076 100 9 lies lie VERB fcis-1076 100 10 in in ADP fcis-1076 100 11 the the DET fcis-1076 100 12 hierarchical hierarchical ADJ fcis-1076 100 13 attention attention NOUN fcis-1076 100 14 mechanism mechanism NOUN fcis-1076 100 15 . . PUNCT fcis-1076 101 1 in in ADP fcis-1076 101 2 addition addition NOUN fcis-1076 101 3 , , PUNCT fcis-1076 101 4 different different ADJ fcis-1076 101 5 modal modal ADJ fcis-1076 101 6 information information NOUN fcis-1076 101 7 lacks lack VERB fcis-1076 101 8 an an DET fcis-1076 101 9 explicit explicit ADJ fcis-1076 101 10 interaction interaction NOUN fcis-1076 101 11 mode mode NOUN fcis-1076 101 12 , , PUNCT fcis-1076 101 13 which which PRON fcis-1076 101 14 can can AUX fcis-1076 101 15 not not PART fcis-1076 101 16 give give VERB fcis-1076 101 17 full full ADJ fcis-1076 101 18 play play NOUN fcis-1076 101 19 to to ADP fcis-1076 101 20 the the DET fcis-1076 101 21 complementary complementary ADJ fcis-1076 101 22 relationship relationship NOUN fcis-1076 101 23 between between ADP fcis-1076 101 24 modal modal ADJ fcis-1076 101 25 information information NOUN fcis-1076 101 26 . . PUNCT fcis-1076 102 1 ( ( PUNCT fcis-1076 102 2 3 3 X fcis-1076 102 3 ) ) PUNCT fcis-1076 102 4 little little ADJ fcis-1076 102 5 consideration consideration NOUN fcis-1076 102 6 is be AUX fcis-1076 102 7 given give VERB fcis-1076 102 8 to to ADP fcis-1076 102 9 data datum NOUN fcis-1076 102 10 privacy privacy NOUN fcis-1076 102 11 . . PUNCT fcis-1076 103 1 multimodal multimodal NOUN fcis-1076 103 2 data datum NOUN fcis-1076 103 3 not not PART fcis-1076 103 4 only only ADV fcis-1076 103 5 provides provide VERB fcis-1076 103 6 more more ADJ fcis-1076 103 7 information information NOUN fcis-1076 103 8 , , PUNCT fcis-1076 103 9 but but CCONJ fcis-1076 103 10 also also ADV fcis-1076 103 11 brings bring VERB fcis-1076 103 12 some some DET fcis-1076 103 13 challenges challenge NOUN fcis-1076 103 14 to to ADP fcis-1076 103 15 data datum NOUN fcis-1076 103 16 confidentiality confidentiality NOUN fcis-1076 103 17 . . PUNCT fcis-1076 104 1 for for ADP fcis-1076 104 2 example example NOUN fcis-1076 104 3 , , PUNCT fcis-1076 104 4 in in ADP fcis-1076 104 5 multimodal multimodal NOUN fcis-1076 104 6 conference conference NOUN fcis-1076 104 7 data datum NOUN fcis-1076 104 8 , , PUNCT fcis-1076 104 9 voiceprint voiceprint NOUN fcis-1076 104 10 features feature NOUN fcis-1076 104 11 and and CCONJ fcis-1076 104 12 facial facial ADJ fcis-1076 104 13 features feature NOUN fcis-1076 104 14 are be AUX fcis-1076 104 15 very very ADV fcis-1076 104 16 important important ADJ fcis-1076 104 17 personal personal ADJ fcis-1076 104 18 privacy privacy NOUN fcis-1076 104 19 information information NOUN fcis-1076 104 20 . . PUNCT fcis-1076 105 1 therefore therefore ADV fcis-1076 105 2 , , PUNCT fcis-1076 105 3 data datum NOUN fcis-1076 105 4 privacy privacy NOUN fcis-1076 105 5 needs need VERB fcis-1076 105 6 to to PART fcis-1076 105 7 be be AUX fcis-1076 105 8 fully fully ADV fcis-1076 105 9 considered consider VERB fcis-1076 105 10 in in ADP fcis-1076 105 11 the the DET fcis-1076 105 12 actual actual ADJ fcis-1076 105 13 implementation implementation NOUN fcis-1076 105 14 . . PUNCT fcis-1076 106 1 in in ADP fcis-1076 106 2 general general ADJ fcis-1076 106 3 , , PUNCT fcis-1076 106 4 under under ADP fcis-1076 106 5 the the DET fcis-1076 106 6 background background NOUN fcis-1076 106 7 of of ADP fcis-1076 106 8 the the DET fcis-1076 106 9 hot hot ADJ fcis-1076 106 10 development development NOUN fcis-1076 106 11 of of ADP fcis-1076 106 12 multimodality multimodality NOUN fcis-1076 106 13 , , PUNCT fcis-1076 106 14 multimodal multimodal NOUN fcis-1076 106 15 summarization summarization NOUN fcis-1076 106 16 , , PUNCT fcis-1076 106 17 as as ADP fcis-1076 106 18 a a DET fcis-1076 106 19 branch branch NOUN fcis-1076 106 20 of of ADP fcis-1076 106 21 multimodal multimodal NOUN fcis-1076 106 22 learning learning NOUN fcis-1076 106 23 , , PUNCT fcis-1076 106 24 has have VERB fcis-1076 106 25 both both CCONJ fcis-1076 106 26 the the DET fcis-1076 106 27 common common ADJ fcis-1076 106 28 problems problem NOUN fcis-1076 106 29 of of ADP fcis-1076 106 30 multimodal multimodal NOUN fcis-1076 106 31 learning learning NOUN fcis-1076 106 32 and and CCONJ fcis-1076 106 33 the the DET fcis-1076 106 34 personality personality NOUN fcis-1076 106 35 problems problem NOUN fcis-1076 106 36 of of ADP fcis-1076 106 37 the the DET fcis-1076 106 38 summarization summarization NOUN fcis-1076 106 39 task task NOUN fcis-1076 106 40 itself itself PRON fcis-1076 106 41 . . PUNCT fcis-1076 107 1 this this DET fcis-1076 107 2 task task NOUN fcis-1076 107 3 has have AUX fcis-1076 107 4 begun begin VERB fcis-1076 107 5 to to PART fcis-1076 107 6 flourish flourish VERB fcis-1076 107 7 in in ADP fcis-1076 107 8 recent recent ADJ fcis-1076 107 9 years year NOUN fcis-1076 107 10 and and CCONJ fcis-1076 107 11 will will AUX fcis-1076 107 12 become become VERB fcis-1076 107 13 an an DET fcis-1076 107 14 important important ADJ fcis-1076 107 15 research research NOUN fcis-1076 107 16 direction direction NOUN fcis-1076 107 17 in in ADP fcis-1076 107 18 the the DET fcis-1076 107 19 future future NOUN fcis-1076 107 20 . . PUNCT fcis-1076 108 1 references reference NOUN fcis-1076 108 2 [ [ X fcis-1076 108 3 1 1 NUM fcis-1076 108 4 ] ] X fcis-1076 108 5 jangra jangra PROPN fcis-1076 108 6 , , PUNCT fcis-1076 108 7 a. a. PROPN fcis-1076 108 8 , , PUNCT fcis-1076 108 9 et et PROPN fcis-1076 108 10 al al PROPN fcis-1076 108 11 . . PROPN fcis-1076 108 12 , , PUNCT fcis-1076 108 13 text text NOUN fcis-1076 108 14 - - PUNCT fcis-1076 108 15 image image NOUN fcis-1076 108 16 - - PUNCT fcis-1076 108 17 video video NOUN fcis-1076 108 18 summary summary NOUN fcis-1076 108 19 generation generation NOUN fcis-1076 108 20 using use VERB fcis-1076 108 21 joint joint ADJ fcis-1076 108 22 integer integer NOUN fcis-1076 108 23 linear linear PROPN fcis-1076 108 24 programming programming NOUN fcis-1076 108 25 , , PUNCT fcis-1076 108 26 in in ADP fcis-1076 108 27 advances advance NOUN fcis-1076 108 28 in in ADP fcis-1076 108 29 information information NOUN fcis-1076 108 30 retrieval retrieval NOUN fcis-1076 108 31 . . PUNCT fcis-1076 109 1 2020 2020 NUM fcis-1076 109 2 . . PUNCT fcis-1076 110 1 p. p. NOUN fcis-1076 110 2 190 190 NUM fcis-1076 110 3 - - SYM fcis-1076 110 4 198 198 NUM fcis-1076 110 5 . . PUNCT fcis-1076 111 1 [ [ X fcis-1076 111 2 2 2 NUM fcis-1076 111 3 ] ] SYM fcis-1076 111 4 li li PROPN fcis-1076 111 5 , , PUNCT fcis-1076 111 6 h. h. PROPN fcis-1076 111 7 , , PUNCT fcis-1076 111 8 et et PROPN fcis-1076 111 9 al al PROPN fcis-1076 111 10 . . PUNCT fcis-1076 111 11 multi multi ADJ fcis-1076 111 12 - - ADJ fcis-1076 111 13 modal modal ADJ fcis-1076 111 14 summarization summarization NOUN fcis-1076 111 15 for for ADP fcis-1076 111 16 asynchronous asynchronous ADJ fcis-1076 111 17 collection collection NOUN fcis-1076 111 18 of of ADP fcis-1076 111 19 text text NOUN fcis-1076 111 20 , , PUNCT fcis-1076 111 21 image image NOUN fcis-1076 111 22 , , PUNCT fcis-1076 111 23 audio audio NOUN fcis-1076 111 24 and and CCONJ fcis-1076 111 25 video video NOUN fcis-1076 111 26 . . PUNCT fcis-1076 112 1 in in ADP fcis-1076 112 2 proceedings proceeding NOUN fcis-1076 112 3 of of ADP fcis-1076 112 4 the the DET fcis-1076 112 5 2017 2017 NUM fcis-1076 112 6 conference conference NOUN fcis-1076 112 7 on on ADP fcis-1076 112 8 empirical empirical ADJ fcis-1076 112 9 methods method NOUN fcis-1076 112 10 in in ADP fcis-1076 112 11 natural natural ADJ fcis-1076 112 12 language language NOUN fcis-1076 112 13 processing processing NOUN fcis-1076 112 14 . . PUNCT fcis-1076 113 1 2017 2017 NUM fcis-1076 113 2 . . PUNCT fcis-1076 114 1 [ [ X fcis-1076 114 2 3 3 X fcis-1076 114 3 ] ] X fcis-1076 115 1 [ [ X fcis-1076 115 2 cho cho NOUN fcis-1076 115 3 , , PUNCT fcis-1076 115 4 k. k. PROPN fcis-1076 115 5 , , PUNCT fcis-1076 115 6 et et PROPN fcis-1076 115 7 al al PROPN fcis-1076 115 8 . . PROPN fcis-1076 115 9 , , PUNCT fcis-1076 115 10 learning learn VERB fcis-1076 115 11 phrase phrase NOUN fcis-1076 115 12 representations representation NOUN fcis-1076 115 13 using use VERB fcis-1076 115 14 rnn rnn NOUN fcis-1076 115 15 encoder encoder NOUN fcis-1076 115 16 - - PUNCT fcis-1076 115 17 decoder decoder NOUN fcis-1076 115 18 for for ADP fcis-1076 115 19 statistical statistical ADJ fcis-1076 115 20 machine machine NOUN fcis-1076 115 21 translation translation NOUN fcis-1076 115 22 . . PUNCT fcis-1076 116 1 arxiv arxiv PROPN fcis-1076 116 2 preprint preprint NOUN fcis-1076 116 3 arxiv:1406.1078 arxiv:1406.1078 NOUN fcis-1076 116 4 , , PUNCT fcis-1076 116 5 2014 2014 NUM fcis-1076 116 6 . . PUNCT fcis-1076 117 1 [ [ X fcis-1076 117 2 4 4 X fcis-1076 117 3 ] ] PUNCT fcis-1076 117 4 sutskever sutskever PROPN fcis-1076 117 5 , , PUNCT fcis-1076 117 6 i. i. PROPN fcis-1076 117 7 , , PUNCT fcis-1076 117 8 o. o. PROPN fcis-1076 117 9 vinyals vinyal NOUN fcis-1076 117 10 , , PUNCT fcis-1076 117 11 and and CCONJ fcis-1076 117 12 q.v q.v PROPN fcis-1076 117 13 . . PROPN fcis-1076 117 14 le le PROPN fcis-1076 117 15 . . PUNCT fcis-1076 117 16 sequence sequence NOUN fcis-1076 117 17 to to ADP fcis-1076 117 18 sequence sequence NOUN fcis-1076 117 19 learning learn VERB fcis-1076 117 20 with with ADP fcis-1076 117 21 neural neural ADJ fcis-1076 117 22 networks network NOUN fcis-1076 117 23 . . PUNCT fcis-1076 118 1 in in ADP fcis-1076 118 2 advances advance NOUN fcis-1076 118 3 in in ADP fcis-1076 118 4 neural neural ADJ fcis-1076 118 5 information information NOUN fcis-1076 118 6 processing processing NOUN fcis-1076 118 7 systems system NOUN fcis-1076 118 8 . . PUNCT fcis-1076 118 9 2014 2014 NUM fcis-1076 118 10 . . PUNCT fcis-1076 119 1 [ [ X fcis-1076 119 2 5 5 NUM fcis-1076 119 3 ] ] SYM fcis-1076 119 4 bahdanau bahdanau NOUN fcis-1076 119 5 , , PUNCT fcis-1076 119 6 d. d. PROPN fcis-1076 119 7 , , PUNCT fcis-1076 119 8 k. k. PROPN fcis-1076 119 9 cho cho PROPN fcis-1076 119 10 , , PUNCT fcis-1076 119 11 and and CCONJ fcis-1076 119 12 y. y. PROPN fcis-1076 119 13 bengio bengio PROPN fcis-1076 119 14 , , PUNCT fcis-1076 119 15 neural neural ADJ fcis-1076 119 16 machine machine NOUN fcis-1076 119 17 translation translation NOUN fcis-1076 119 18 by by ADP fcis-1076 119 19 jointly jointly ADV fcis-1076 119 20 learning learn VERB fcis-1076 119 21 to to PART fcis-1076 119 22 align align VERB fcis-1076 119 23 and and CCONJ fcis-1076 119 24 translate translate VERB fcis-1076 119 25 . . PUNCT fcis-1076 120 1 arxiv arxiv PROPN fcis-1076 120 2 preprint preprint NOUN fcis-1076 120 3 arxiv:1409.0473 arxiv:1409.0473 PROPN fcis-1076 120 4 , , PUNCT fcis-1076 120 5 2014 2014 NUM fcis-1076 120 6 . . PUNCT fcis-1076 121 1 [ [ X fcis-1076 121 2 6 6 NUM fcis-1076 121 3 ] ] PUNCT fcis-1076 121 4 rush rush NOUN fcis-1076 121 5 , , PUNCT fcis-1076 121 6 a.m. a.m. ADV fcis-1076 121 7 , , PUNCT fcis-1076 121 8 s. s. PROPN fcis-1076 121 9 chopra chopra PROPN fcis-1076 121 10 , , PUNCT fcis-1076 121 11 and and CCONJ fcis-1076 121 12 j. j. PROPN fcis-1076 121 13 weston weston PROPN fcis-1076 121 14 , , PUNCT fcis-1076 121 15 a a DET fcis-1076 121 16 neural neural ADJ fcis-1076 121 17 attention attention NOUN fcis-1076 121 18 model model NOUN fcis-1076 121 19 for for ADP fcis-1076 121 20 abstractive abstractive ADJ fcis-1076 121 21 sentence sentence NOUN fcis-1076 121 22 summarization summarization NOUN fcis-1076 121 23 . . PUNCT fcis-1076 122 1 arxiv arxiv PROPN fcis-1076 122 2 preprint preprint PROPN fcis-1076 122 3 arxiv:1509.00685 arxiv:1509.00685 ADJ fcis-1076 122 4 , , PUNCT fcis-1076 122 5 2015 2015 NUM fcis-1076 122 6 . . PUNCT fcis-1076 123 1 [ [ X fcis-1076 123 2 7 7 NUM fcis-1076 123 3 ] ] X fcis-1076 123 4 shi shi PROPN fcis-1076 123 5 , , PUNCT fcis-1076 123 6 t. t. PROPN fcis-1076 123 7 , , PUNCT fcis-1076 123 8 et et PROPN fcis-1076 123 9 al al PROPN fcis-1076 123 10 . . PROPN fcis-1076 123 11 , , PUNCT fcis-1076 123 12 neural neural ADJ fcis-1076 123 13 abstractive abstractive ADJ fcis-1076 123 14 text text NOUN fcis-1076 123 15 summarization summarization NOUN fcis-1076 123 16 with with ADP fcis-1076 123 17 sequence sequence NOUN fcis-1076 123 18 - - PUNCT fcis-1076 123 19 to to ADP fcis-1076 123 20 - - PUNCT fcis-1076 123 21 sequence sequence NOUN fcis-1076 123 22 models model NOUN fcis-1076 123 23 . . PUNCT fcis-1076 124 1 acm acm NOUN fcis-1076 124 2 transactions transaction NOUN fcis-1076 124 3 on on ADP fcis-1076 124 4 data datum NOUN fcis-1076 124 5 science science NOUN fcis-1076 124 6 , , PUNCT fcis-1076 124 7 2021 2021 NUM fcis-1076 124 8 . . PUNCT fcis-1076 125 1 2(1 2(1 NUM fcis-1076 125 2 ): ): PUNCT fcis-1076 126 1 p. p. NOUN fcis-1076 126 2 1 1 NUM fcis-1076 126 3 - - SYM fcis-1076 126 4 37 37 NUM fcis-1076 127 1 [ [ X fcis-1076 127 2 8 8 NUM fcis-1076 127 3 ] ] X fcis-1076 127 4 hochreiter hochreiter PROPN fcis-1076 127 5 , , PUNCT fcis-1076 127 6 s. s. PROPN fcis-1076 127 7 and and CCONJ fcis-1076 127 8 j. j. PROPN fcis-1076 127 9 schmidhuber schmidhuber PROPN fcis-1076 127 10 , , PUNCT fcis-1076 127 11 long long ADJ fcis-1076 127 12 short short ADJ fcis-1076 127 13 - - PUNCT fcis-1076 127 14 term term NOUN fcis-1076 127 15 memory memory NOUN fcis-1076 127 16 . . PUNCT fcis-1076 128 1 neural neural ADJ fcis-1076 128 2 computation computation NOUN fcis-1076 128 3 , , PUNCT fcis-1076 128 4 1997 1997 NUM fcis-1076 128 5 . . PUNCT fcis-1076 129 1 9(8 9(8 NUM fcis-1076 129 2 ): ): PUNCT fcis-1076 129 3 p. p. NOUN fcis-1076 129 4 1735 1735 NUM fcis-1076 129 5 - - SYM fcis-1076 129 6 1780 1780 NUM fcis-1076 129 7 . . PUNCT fcis-1076 130 1 [ [ X fcis-1076 130 2 9 9 NUM fcis-1076 130 3 ] ] X fcis-1076 130 4 zhu zhu PROPN fcis-1076 130 5 , , PUNCT fcis-1076 130 6 j. j. PROPN fcis-1076 130 7 , , PUNCT fcis-1076 130 8 et et PROPN fcis-1076 130 9 al al PROPN fcis-1076 130 10 . . PROPN fcis-1076 130 11 msmo msmo PROPN fcis-1076 130 12 : : PUNCT fcis-1076 130 13 multimodal multimodal NOUN fcis-1076 130 14 summarization summarization NOUN fcis-1076 130 15 with with ADP fcis-1076 130 16 multimodal multimodal ADJ fcis-1076 130 17 output output NOUN fcis-1076 130 18 . . PUNCT fcis-1076 131 1 in in ADP fcis-1076 131 2 proceedings proceeding NOUN fcis-1076 131 3 of of ADP fcis-1076 131 4 the the DET fcis-1076 131 5 2018 2018 NUM fcis-1076 131 6 conference conference NOUN fcis-1076 131 7 on on ADP fcis-1076 131 8 empirical empirical ADJ fcis-1076 131 9 methods method NOUN fcis-1076 131 10 in in ADP fcis-1076 131 11 natural natural ADJ fcis-1076 131 12 language language NOUN fcis-1076 131 13 processing processing NOUN fcis-1076 131 14 . . PUNCT fcis-1076 132 1 2018 2018 NUM fcis-1076 132 2 . . PUNCT fcis-1076 133 1 [ [ X fcis-1076 133 2 10 10 NUM fcis-1076 133 3 ] ] X fcis-1076 133 4 zhu zhu PROPN fcis-1076 133 5 , , PUNCT fcis-1076 133 6 j. j. PROPN fcis-1076 133 7 , , PUNCT fcis-1076 133 8 et et PROPN fcis-1076 133 9 al al PROPN fcis-1076 133 10 . . PROPN fcis-1076 133 11 , , PUNCT fcis-1076 133 12 multimodal multimodal NOUN fcis-1076 133 13 summarization summarization NOUN fcis-1076 133 14 with with ADP fcis-1076 133 15 guidance guidance NOUN fcis-1076 133 16 of of ADP fcis-1076 133 17 multimodal multimodal NOUN fcis-1076 133 18 reference reference NOUN fcis-1076 133 19 . . PUNCT fcis-1076 134 1 proceedings proceeding NOUN fcis-1076 134 2 of of ADP fcis-1076 134 3 the the DET fcis-1076 134 4 aaai aaai PROPN fcis-1076 134 5 conference conference NOUN fcis-1076 134 6 on on ADP fcis-1076 134 7 artificial artificial ADJ fcis-1076 134 8 intelligence intelligence NOUN fcis-1076 134 9 , , PUNCT fcis-1076 134 10 2020 2020 NUM fcis-1076 134 11 . . PUNCT fcis-1076 135 1 34(05 34(05 NUM fcis-1076 135 2 ): ): PUNCT fcis-1076 135 3 p. p. NOUN fcis-1076 135 4 9749 9749 NUM fcis-1076 135 5 - - SYM fcis-1076 135 6 9756 9756 NUM fcis-1076 135 7 . . PUNCT fcis-1076 136 1 [ [ X fcis-1076 136 2 11 11 NUM fcis-1076 136 3 ] ] X fcis-1076 136 4 khullar khullar ADJ fcis-1076 136 5 , , PUNCT fcis-1076 136 6 a. a. NOUN fcis-1076 136 7 and and CCONJ fcis-1076 136 8 u. u. PROPN fcis-1076 136 9 arora arora PROPN fcis-1076 136 10 , , PUNCT fcis-1076 136 11 mast mast NOUN fcis-1076 136 12 : : PUNCT fcis-1076 136 13 multimodal multimodal ADJ fcis-1076 136 14 abstractive abstractive ADJ fcis-1076 136 15 summarization summarization NOUN fcis-1076 136 16 with with ADP fcis-1076 136 17 trimodal trimodal NOUN fcis-1076 136 18 hierarchical hierarchical ADJ fcis-1076 136 19 attention attention NOUN fcis-1076 136 20 . . PUNCT fcis-1076 137 1 arxiv arxiv PROPN fcis-1076 137 2 preprint preprint VERB fcis-1076 137 3 arxiv:2010.08021 arxiv:2010.08021 NOUN fcis-1076 137 4 , , PUNCT fcis-1076 137 5 2020 2020 NUM fcis-1076 137 6 . . PUNCT fcis-1076 138 1 [ [ X fcis-1076 138 2 12 12 NUM fcis-1076 138 3 ] ] X fcis-1076 138 4 liu liu PROPN fcis-1076 138 5 , , PUNCT fcis-1076 138 6 n. n. PROPN fcis-1076 138 7 , , PUNCT fcis-1076 138 8 et et PROPN fcis-1076 138 9 al al PROPN fcis-1076 138 10 . . PUNCT fcis-1076 138 11 multistage multistage NOUN fcis-1076 138 12 fusion fusion NOUN fcis-1076 138 13 with with ADP fcis-1076 138 14 forget forget NOUN fcis-1076 138 15 gate gate NOUN fcis-1076 138 16 for for ADP fcis-1076 138 17 multimodal multimodal NOUN fcis-1076 138 18 summarization summarization NOUN fcis-1076 138 19 in in ADP fcis-1076 138 20 open open ADJ fcis-1076 138 21 - - PUNCT fcis-1076 138 22 domain domain NOUN fcis-1076 138 23 videos video NOUN fcis-1076 138 24 . . PUNCT fcis-1076 139 1 in in ADP fcis-1076 139 2 proceedings proceeding NOUN fcis-1076 139 3 of of ADP fcis-1076 139 4 the the DET fcis-1076 139 5 2020 2020 NUM fcis-1076 139 6 conference conference NOUN fcis-1076 139 7 on on ADP fcis-1076 139 8 empirical empirical ADJ fcis-1076 139 9 methods method NOUN fcis-1076 139 10 in in ADP fcis-1076 139 11 natural natural ADJ fcis-1076 139 12 language language NOUN fcis-1076 139 13 processing processing NOUN fcis-1076 139 14 ( ( PUNCT fcis-1076 139 15 emnlp emnlp ADJ fcis-1076 139 16 ) ) PUNCT fcis-1076 139 17 . . PUNCT fcis-1076 140 1 2020 2020 NUM fcis-1076 140 2 . . PUNCT fcis-1076 141 1 [ [ X fcis-1076 141 2 13 13 NUM fcis-1076 141 3 ] ] PUNCT fcis-1076 141 4 he he PRON fcis-1076 141 5 , , PUNCT fcis-1076 141 6 k. k. PROPN fcis-1076 141 7 , , PUNCT fcis-1076 141 8 et et PROPN fcis-1076 141 9 al al PROPN fcis-1076 141 10 . . PUNCT fcis-1076 142 1 deep deep ADJ fcis-1076 142 2 residual residual ADJ fcis-1076 142 3 learning learning NOUN fcis-1076 142 4 for for ADP fcis-1076 142 5 image image NOUN fcis-1076 142 6 recognition recognition NOUN fcis-1076 142 7 . . PUNCT fcis-1076 143 1 in in ADP fcis-1076 143 2 proceedings proceeding NOUN fcis-1076 143 3 of of ADP fcis-1076 143 4 the the DET fcis-1076 143 5 ieee ieee NOUN fcis-1076 143 6 conference conference NOUN fcis-1076 143 7 on on ADP fcis-1076 143 8 computer computer NOUN fcis-1076 143 9 vision vision NOUN fcis-1076 143 10 and and CCONJ fcis-1076 143 11 pattern pattern NOUN fcis-1076 143 12 recognition recognition NOUN fcis-1076 143 13 . . PUNCT fcis-1076 144 1 2016 2016 NUM fcis-1076 144 2 . . PUNCT fcis-1076 145 1 [ [ X fcis-1076 145 2 14 14 NUM fcis-1076 145 3 ] ] SYM fcis-1076 145 4 simonyan simonyan PROPN fcis-1076 145 5 , , PUNCT fcis-1076 145 6 k. k. PROPN fcis-1076 145 7 and and CCONJ fcis-1076 145 8 a. a. NOUN fcis-1076 145 9 zisserman zisserman PROPN fcis-1076 145 10 , , PUNCT fcis-1076 145 11 very very ADV fcis-1076 145 12 deep deep ADJ fcis-1076 145 13 convolutional convolutional ADJ fcis-1076 145 14 networks network NOUN fcis-1076 145 15 for for ADP fcis-1076 145 16 large large ADJ fcis-1076 145 17 - - PUNCT fcis-1076 145 18 scale scale NOUN fcis-1076 145 19 image image NOUN fcis-1076 145 20 recognition recognition NOUN fcis-1076 145 21 . . PUNCT fcis-1076 146 1 arxiv arxiv PROPN fcis-1076 146 2 preprint preprint PROPN fcis-1076 146 3 arxiv:1409.1556 arxiv:1409.1556 NOUN fcis-1076 146 4 , , PUNCT fcis-1076 146 5 2014 2014 NUM fcis-1076 146 6 . . PUNCT fcis-1076 147 1 17 17 NUM fcis-1076 148 1 [ [ SYM fcis-1076 148 2 15 15 NUM fcis-1076 148 3 ] ] X fcis-1076 148 4 szegedy szegedy PROPN fcis-1076 148 5 , , PUNCT fcis-1076 148 6 c. c. PROPN fcis-1076 148 7 , , PUNCT fcis-1076 148 8 et et PROPN fcis-1076 148 9 al al PROPN fcis-1076 148 10 . . PUNCT fcis-1076 148 11 going go VERB fcis-1076 148 12 deeper deeply ADV fcis-1076 148 13 with with ADP fcis-1076 148 14 convolutions convolution NOUN fcis-1076 148 15 . . PUNCT fcis-1076 149 1 in in ADP fcis-1076 149 2 proceedings proceeding NOUN fcis-1076 149 3 of of ADP fcis-1076 149 4 the the DET fcis-1076 149 5 ieee ieee NOUN fcis-1076 149 6 conference conference NOUN fcis-1076 149 7 on on ADP fcis-1076 149 8 computer computer NOUN fcis-1076 149 9 vision vision NOUN fcis-1076 149 10 and and CCONJ fcis-1076 149 11 pattern pattern NOUN fcis-1076 149 12 recognition recognition NOUN fcis-1076 149 13 . . PUNCT fcis-1076 150 1 2015 2015 NUM fcis-1076 150 2 . . PUNCT fcis-1076 151 1 [ [ X fcis-1076 151 2 16 16 NUM fcis-1076 151 3 ] ] X fcis-1076 151 4 deng deng PROPN fcis-1076 151 5 , , PUNCT fcis-1076 151 6 j. j. PROPN fcis-1076 151 7 , , PUNCT fcis-1076 151 8 et et PROPN fcis-1076 151 9 al al PROPN fcis-1076 151 10 . . PROPN fcis-1076 151 11 imagenet imagenet PROPN fcis-1076 151 12 : : PUNCT fcis-1076 151 13 a a DET fcis-1076 151 14 large large ADJ fcis-1076 151 15 - - PUNCT fcis-1076 151 16 scale scale NOUN fcis-1076 151 17 hierarchical hierarchical ADJ fcis-1076 151 18 image image NOUN fcis-1076 151 19 database database NOUN fcis-1076 151 20 . . PUNCT fcis-1076 152 1 in in ADP fcis-1076 152 2 2009 2009 NUM fcis-1076 152 3 ieee ieee NOUN fcis-1076 152 4 conference conference NOUN fcis-1076 152 5 on on ADP fcis-1076 152 6 computer computer NOUN fcis-1076 152 7 vision vision NOUN fcis-1076 152 8 and and CCONJ fcis-1076 152 9 pattern pattern NOUN fcis-1076 152 10 recognition recognition NOUN fcis-1076 152 11 . . PUNCT fcis-1076 153 1 2009 2009 NUM fcis-1076 153 2 . . PUNCT fcis-1076 154 1 ieee ieee NOUN fcis-1076 154 2 . . PUNCT fcis-1076 155 1 [ [ X fcis-1076 155 2 17 17 NUM fcis-1076 155 3 ] ] SYM fcis-1076 155 4 karpathy karpathy PROPN fcis-1076 155 5 , , PUNCT fcis-1076 155 6 a. a. NOUN fcis-1076 155 7 , , PUNCT fcis-1076 155 8 a. a. PROPN fcis-1076 155 9 joulin joulin PROPN fcis-1076 155 10 , , PUNCT fcis-1076 155 11 and and CCONJ fcis-1076 155 12 l. l. PROPN fcis-1076 155 13 fei fei PROPN fcis-1076 155 14 - - PUNCT fcis-1076 155 15 fei fei PROPN fcis-1076 155 16 , , PUNCT fcis-1076 155 17 deep deep ADJ fcis-1076 155 18 fragment fragment NOUN fcis-1076 155 19 embeddings embedding NOUN fcis-1076 155 20 for for ADP fcis-1076 155 21 bidirectional bidirectional ADJ fcis-1076 155 22 image image NOUN fcis-1076 155 23 sentence sentence NOUN fcis-1076 155 24 mapping mapping NOUN fcis-1076 155 25 . . PUNCT fcis-1076 156 1 arxiv arxiv PROPN fcis-1076 156 2 preprint preprint VERB fcis-1076 156 3 arxiv:1406.5679 arxiv:1406.5679 NOUN fcis-1076 156 4 , , PUNCT fcis-1076 156 5 2014 2014 NUM fcis-1076 156 6 . . PUNCT fcis-1076 157 1 [ [ X fcis-1076 157 2 18 18 NUM fcis-1076 157 3 ] ] X fcis-1076 157 4 wang wang PROPN fcis-1076 157 5 , , PUNCT fcis-1076 157 6 l. l. PROPN fcis-1076 157 7 , , PUNCT fcis-1076 157 8 y. y. PROPN fcis-1076 157 9 li li PROPN fcis-1076 157 10 , , PUNCT fcis-1076 157 11 and and CCONJ fcis-1076 157 12 s. s. PROPN fcis-1076 157 13 lazebnik lazebnik PROPN fcis-1076 157 14 . . PUNCT fcis-1076 158 1 learning learn VERB fcis-1076 158 2 deep deep ADJ fcis-1076 158 3 structurepreserving structurepreserving NOUN fcis-1076 158 4 image image NOUN fcis-1076 158 5 - - PUNCT fcis-1076 158 6 text text NOUN fcis-1076 158 7 embeddings embedding NOUN fcis-1076 158 8 . . PUNCT fcis-1076 159 1 in in ADP fcis-1076 159 2 proceedings proceeding NOUN fcis-1076 159 3 of of ADP fcis-1076 159 4 the the DET fcis-1076 159 5 ieee ieee NOUN fcis-1076 159 6 conference conference NOUN fcis-1076 159 7 on on ADP fcis-1076 159 8 computer computer NOUN fcis-1076 159 9 vision vision NOUN fcis-1076 159 10 and and CCONJ fcis-1076 159 11 pattern pattern NOUN fcis-1076 159 12 recognition recognition NOUN fcis-1076 159 13 . . PUNCT fcis-1076 160 1 2016 2016 NUM fcis-1076 160 2 . . PUNCT fcis-1076 161 1 [ [ X fcis-1076 161 2 19 19 NUM fcis-1076 161 3 ] ] X fcis-1076 161 4 rashtchian rashtchian ADJ fcis-1076 161 5 , , PUNCT fcis-1076 161 6 c. c. PROPN fcis-1076 161 7 , , PUNCT fcis-1076 161 8 et et PROPN fcis-1076 161 9 al al PROPN fcis-1076 161 10 . . PUNCT fcis-1076 161 11 collecting collect VERB fcis-1076 161 12 image image NOUN fcis-1076 161 13 annotations annotation NOUN fcis-1076 161 14 using use VERB fcis-1076 161 15 amazon amazon NOUN fcis-1076 161 16 ’s ’s PART fcis-1076 161 17 mechanical mechanical ADJ fcis-1076 161 18 turk turk PROPN fcis-1076 161 19 . . PUNCT fcis-1076 162 1 in in ADP fcis-1076 162 2 proceedings proceeding NOUN fcis-1076 162 3 of of ADP fcis-1076 162 4 the the DET fcis-1076 162 5 naacl naacl PROPN fcis-1076 162 6 hlt hlt PROPN fcis-1076 162 7 2010 2010 NUM fcis-1076 162 8 workshop workshop NOUN fcis-1076 162 9 on on ADP fcis-1076 162 10 creating create VERB fcis-1076 162 11 speech speech NOUN fcis-1076 162 12 and and CCONJ fcis-1076 162 13 language language NOUN fcis-1076 162 14 data datum NOUN fcis-1076 162 15 with with ADP fcis-1076 162 16 amazon amazon NOUN fcis-1076 162 17 ’s ’s PART fcis-1076 162 18 mechanical mechanical ADJ fcis-1076 162 19 turk turk PROPN fcis-1076 162 20 . . PUNCT fcis-1076 163 1 2010 2010 NUM fcis-1076 163 2 . . PUNCT fcis-1076 164 1 [ [ X fcis-1076 164 2 20 20 NUM fcis-1076 164 3 ] ] X fcis-1076 164 4 hodosh hodosh NOUN fcis-1076 164 5 , , PUNCT fcis-1076 164 6 m. m. NOUN fcis-1076 164 7 , , PUNCT fcis-1076 164 8 p. p. PROPN fcis-1076 164 9 young young PROPN fcis-1076 164 10 , , PUNCT fcis-1076 164 11 and and CCONJ fcis-1076 164 12 j. j. PROPN fcis-1076 164 13 hockenmaier hockenmaier PROPN fcis-1076 164 14 , , PUNCT fcis-1076 164 15 framing frame VERB fcis-1076 164 16 image image NOUN fcis-1076 164 17 description description NOUN fcis-1076 164 18 as as ADP fcis-1076 164 19 a a DET fcis-1076 164 20 ranking ranking ADJ fcis-1076 164 21 task task NOUN fcis-1076 164 22 : : PUNCT fcis-1076 164 23 data datum NOUN fcis-1076 164 24 , , PUNCT fcis-1076 164 25 models model NOUN fcis-1076 164 26 and and CCONJ fcis-1076 164 27 evaluation evaluation NOUN fcis-1076 164 28 metrics metric NOUN fcis-1076 164 29 . . PUNCT fcis-1076 165 1 journal journal NOUN fcis-1076 165 2 of of ADP fcis-1076 165 3 artificial artificial ADJ fcis-1076 165 4 intelligence intelligence NOUN fcis-1076 165 5 research research NOUN fcis-1076 165 6 , , PUNCT fcis-1076 165 7 2013 2013 NUM fcis-1076 165 8 . . PUNCT fcis-1076 166 1 47 47 NUM fcis-1076 166 2 : : PUNCT fcis-1076 167 1 p. p. NOUN fcis-1076 167 2 853 853 NUM fcis-1076 167 3 - - SYM fcis-1076 167 4 899 899 NUM fcis-1076 167 5 . . PUNCT fcis-1076 168 1 [ [ X fcis-1076 168 2 21 21 NUM fcis-1076 168 3 ] ] X fcis-1076 168 4 young young ADJ fcis-1076 168 5 , , PUNCT fcis-1076 168 6 p. p. PROPN fcis-1076 168 7 , , PUNCT fcis-1076 168 8 et et PROPN fcis-1076 168 9 al al PROPN fcis-1076 168 10 . . PROPN fcis-1076 168 11 , , PUNCT fcis-1076 168 12 from from ADP fcis-1076 168 13 image image NOUN fcis-1076 168 14 descriptions description NOUN fcis-1076 168 15 to to ADP fcis-1076 168 16 visual visual ADJ fcis-1076 168 17 denotations denotation NOUN fcis-1076 168 18 : : PUNCT fcis-1076 168 19 new new ADJ fcis-1076 168 20 similarity similarity NOUN fcis-1076 168 21 metrics metric NOUN fcis-1076 168 22 for for ADP fcis-1076 168 23 semantic semantic ADJ fcis-1076 168 24 inference inference NOUN fcis-1076 168 25 over over ADP fcis-1076 168 26 event event NOUN fcis-1076 168 27 descriptions description NOUN fcis-1076 168 28 . . PUNCT fcis-1076 169 1 transactions transaction NOUN fcis-1076 169 2 of of ADP fcis-1076 169 3 the the DET fcis-1076 169 4 association association NOUN fcis-1076 169 5 for for ADP fcis-1076 169 6 computational computational ADJ fcis-1076 169 7 linguistics linguistic NOUN fcis-1076 169 8 , , PUNCT fcis-1076 169 9 2014 2014 NUM fcis-1076 169 10 . . PUNCT fcis-1076 170 1 2 2 NUM fcis-1076 170 2 : : PUNCT fcis-1076 170 3 p. p. NOUN fcis-1076 170 4 67 67 NUM fcis-1076 170 5 - - SYM fcis-1076 170 6 78 78 NUM fcis-1076 170 7 . . PUNCT fcis-1076 171 1 [ [ X fcis-1076 171 2 22 22 NUM fcis-1076 171 3 ] ] X fcis-1076 171 4 wang wang PROPN fcis-1076 171 5 , , PUNCT fcis-1076 171 6 l. l. PROPN fcis-1076 171 7 , , PUNCT fcis-1076 171 8 y. y. PROPN fcis-1076 171 9 li li PROPN fcis-1076 171 10 , , PUNCT fcis-1076 171 11 and and CCONJ fcis-1076 171 12 s. s. PROPN fcis-1076 171 13 lazebnik lazebnik PROPN fcis-1076 171 14 . . PUNCT fcis-1076 172 1 learning learn VERB fcis-1076 172 2 deep deep ADJ fcis-1076 172 3 structurepreserving structurepreserving NOUN fcis-1076 172 4 image image NOUN fcis-1076 172 5 - - PUNCT fcis-1076 172 6 text text NOUN fcis-1076 172 7 embeddings embedding NOUN fcis-1076 172 8 . . PUNCT fcis-1076 173 1 in in ADP fcis-1076 173 2 proceedings proceeding NOUN fcis-1076 173 3 of of ADP fcis-1076 173 4 the the DET fcis-1076 173 5 ieee ieee NOUN fcis-1076 173 6 conference conference NOUN fcis-1076 173 7 on on ADP fcis-1076 173 8 computer computer NOUN fcis-1076 173 9 vision vision NOUN fcis-1076 173 10 and and CCONJ fcis-1076 173 11 pattern pattern NOUN fcis-1076 173 12 recognition recognition NOUN fcis-1076 173 13 . . PUNCT fcis-1076 174 1 2016 2016 NUM fcis-1076 174 2 . . PUNCT fcis-1076 175 1 [ [ X fcis-1076 175 2 23 23 NUM fcis-1076 175 3 ] ] X fcis-1076 175 4 vaswani vaswani NOUN fcis-1076 175 5 , , PUNCT fcis-1076 175 6 a. a. NOUN fcis-1076 175 7 , , PUNCT fcis-1076 175 8 et et PROPN fcis-1076 175 9 al al PROPN fcis-1076 175 10 . . PUNCT fcis-1076 175 11 attention attention NOUN fcis-1076 175 12 is be AUX fcis-1076 175 13 all all PRON fcis-1076 175 14 you you PRON fcis-1076 175 15 need need VERB fcis-1076 175 16 . . PUNCT fcis-1076 176 1 in in ADP fcis-1076 176 2 advances advance NOUN fcis-1076 176 3 in in ADP fcis-1076 176 4 neural neural ADJ fcis-1076 176 5 information information NOUN fcis-1076 176 6 processing processing NOUN fcis-1076 176 7 systems system NOUN fcis-1076 176 8 . . PUNCT fcis-1076 177 1 2017 2017 NUM fcis-1076 177 2 . . PUNCT fcis-1076 178 1 [ [ X fcis-1076 178 2 24 24 NUM fcis-1076 178 3 ] ] SYM fcis-1076 178 4 li li PROPN fcis-1076 178 5 , , PUNCT fcis-1076 178 6 h. h. PROPN fcis-1076 178 7 , , PUNCT fcis-1076 178 8 et et PROPN fcis-1076 178 9 al al PROPN fcis-1076 178 10 . . PUNCT fcis-1076 178 11 multi multi ADJ fcis-1076 178 12 - - ADJ fcis-1076 178 13 modal modal ADJ fcis-1076 178 14 sentence sentence NOUN fcis-1076 178 15 summarization summarization NOUN fcis-1076 178 16 with with ADP fcis-1076 178 17 modality modality NOUN fcis-1076 178 18 attention attention NOUN fcis-1076 178 19 and and CCONJ fcis-1076 178 20 image image NOUN fcis-1076 178 21 filtering filtering NOUN fcis-1076 178 22 . . PUNCT fcis-1076 179 1 in in ADP fcis-1076 179 2 ijcai ijcai PROPN fcis-1076 179 3 . . PUNCT fcis-1076 179 4 2018 2018 NUM fcis-1076 180 1 [ [ X fcis-1076 180 2 25 25 NUM fcis-1076 180 3 ] ] SYM fcis-1076 180 4 fu fu NOUN fcis-1076 180 5 , , PUNCT fcis-1076 180 6 x. x. PROPN fcis-1076 180 7 , , PUNCT fcis-1076 180 8 j. j. PROPN fcis-1076 180 9 wang wang PROPN fcis-1076 180 10 , , PUNCT fcis-1076 180 11 and and CCONJ fcis-1076 180 12 z. z. PROPN fcis-1076 180 13 yang yang PROPN fcis-1076 180 14 , , PUNCT fcis-1076 180 15 multi multi ADJ fcis-1076 180 16 - - ADJ fcis-1076 180 17 modal modal ADJ fcis-1076 180 18 summarization summarization NOUN fcis-1076 180 19 for for ADP fcis-1076 180 20 video video NOUN fcis-1076 180 21 - - PUNCT fcis-1076 180 22 containing contain VERB fcis-1076 180 23 documents document NOUN fcis-1076 180 24 . . PUNCT fcis-1076 181 1 arxiv arxiv PROPN fcis-1076 181 2 preprint preprint VERB fcis-1076 181 3 arxiv:2009.08018 arxiv:2009.08018 NOUN fcis-1076 181 4 , , PUNCT fcis-1076 181 5 2020 2020 NUM fcis-1076 182 1 [ [ X fcis-1076 182 2 26 26 NUM fcis-1076 182 3 ] ] X fcis-1076 182 4 kim kim PROPN fcis-1076 182 5 , , PUNCT fcis-1076 182 6 j.-h j.-h PROPN fcis-1076 182 7 . . PUNCT fcis-1076 182 8 , , PUNCT fcis-1076 182 9 et et PROPN fcis-1076 182 10 al al PROPN fcis-1076 182 11 . . PROPN fcis-1076 182 12 , , PUNCT fcis-1076 182 13 hadamard hadamard ADJ fcis-1076 182 14 product product NOUN fcis-1076 182 15 for for ADP fcis-1076 182 16 low low ADJ fcis-1076 182 17 - - PUNCT fcis-1076 182 18 rank rank NOUN fcis-1076 182 19 bilinear bilinear NOUN fcis-1076 182 20 pooling pooling NOUN fcis-1076 182 21 . . PUNCT fcis-1076 183 1 arxiv arxiv PROPN fcis-1076 183 2 preprint preprint VERB fcis-1076 183 3 arxiv:1610.04325 arxiv:1610.04325 PROPN fcis-1076 183 4 , , PUNCT fcis-1076 183 5 2016 2016 NUM fcis-1076 183 6 . . PUNCT fcis-1076 184 1 [ [ X fcis-1076 184 2 27 27 NUM fcis-1076 184 3 ] ] SYM fcis-1076 184 4 li li PROPN fcis-1076 184 5 , , PUNCT fcis-1076 184 6 m. m. NOUN fcis-1076 184 7 , , PUNCT fcis-1076 184 8 et et PROPN fcis-1076 184 9 al al PROPN fcis-1076 184 10 . . PROPN fcis-1076 184 11 , , PUNCT fcis-1076 184 12 vmsmo vmsmo PROPN fcis-1076 184 13 : : PUNCT fcis-1076 184 14 learning learn VERB fcis-1076 184 15 to to PART fcis-1076 184 16 generate generate VERB fcis-1076 184 17 multimodal multimodal ADJ fcis-1076 184 18 summary summary NOUN fcis-1076 184 19 for for ADP fcis-1076 184 20 video video NOUN fcis-1076 184 21 - - PUNCT fcis-1076 184 22 based base VERB fcis-1076 184 23 news news NOUN fcis-1076 184 24 articles article NOUN fcis-1076 184 25 . . PUNCT fcis-1076 185 1 arxiv arxiv PROPN fcis-1076 185 2 preprint preprint NOUN fcis-1076 185 3 arxiv:2010.05406 arxiv:2010.05406 NOUN fcis-1076 185 4 , , PUNCT fcis-1076 185 5 2020 2020 NUM