id sid tid token lemma pos bracis-33592 1 1 investigating investigate VERB bracis-33592 1 2 universal universal ADJ bracis-33592 1 3 adversarial adversarial ADJ bracis-33592 1 4 attacks attack NOUN bracis-33592 1 5 against against ADP bracis-33592 1 6 transformers transformer NOUN bracis-33592 1 7 - - PUNCT bracis-33592 1 8 based base VERB bracis-33592 1 9 automatic automatic ADJ bracis-33592 1 10 essay essay NOUN bracis-33592 1 11 scoring scoring NOUN bracis-33592 1 12 systems system NOUN bracis-33592 1 13 | | ADV bracis-33592 1 14 springer springer NOUN bracis-33592 1 15 nature nature PROPN bracis-33592 1 16 link link PROPN bracis-33592 1 17 ( ( PUNCT bracis-33592 1 18 formerly formerly ADV bracis-33592 1 19 springerlink springerlink NOUN bracis-33592 1 20 ) ) PUNCT bracis-33592 1 21 skip skip VERB bracis-33592 1 22 to to ADP bracis-33592 1 23 main main ADJ bracis-33592 1 24 content content NOUN bracis-33592 1 25 advertisement advertisement NOUN bracis-33592 1 26 log log NOUN bracis-33592 1 27 in in ADP bracis-33592 1 28 menu menu NOUN bracis-33592 1 29 find find VERB bracis-33592 1 30 a a DET bracis-33592 1 31 journal journal NOUN bracis-33592 1 32 publish publish VERB bracis-33592 1 33 with with ADP bracis-33592 1 34 us we PRON bracis-33592 1 35 track track VERB bracis-33592 1 36 your your PRON bracis-33592 1 37 research research NOUN bracis-33592 1 38 search search NOUN bracis-33592 1 39 cart cart NOUN bracis-33592 1 40 home home NOUN bracis-33592 1 41 intelligent intelligent ADJ bracis-33592 1 42 systems system NOUN bracis-33592 1 43 conference conference NOUN bracis-33592 1 44 paper paper NOUN bracis-33592 1 45 investigating investigate VERB bracis-33592 1 46 universal universal ADJ bracis-33592 1 47 adversarial adversarial ADJ bracis-33592 1 48 attacks attack NOUN bracis-33592 1 49 against against ADP bracis-33592 1 50 transformers transformer NOUN bracis-33592 1 51 - - PUNCT bracis-33592 1 52 based base VERB bracis-33592 1 53 automatic automatic ADJ bracis-33592 1 54 essay essay NOUN bracis-33592 1 55 scoring scoring NOUN bracis-33592 1 56 systems system NOUN bracis-33592 1 57 conference conference NOUN bracis-33592 1 58 paper paper NOUN bracis-33592 1 59 first first ADV bracis-33592 1 60 online online ADV bracis-33592 1 61 : : PUNCT bracis-33592 1 62 30 30 NUM bracis-33592 1 63 january january NOUN bracis-33592 1 64 2025 2025 NUM bracis-33592 1 65 pp pp PRON bracis-33592 1 66 169–183 169–183 NUM bracis-33592 1 67 cite cite VERB bracis-33592 1 68 this this DET bracis-33592 1 69 conference conference NOUN bracis-33592 1 70 paper paper NOUN bracis-33592 1 71 access access NOUN bracis-33592 1 72 provided provide VERB bracis-33592 1 73 by by ADP bracis-33592 1 74 university university PROPN bracis-33592 1 75 of of ADP bracis-33592 1 76 notre notre PROPN bracis-33592 1 77 dame dame PROPN bracis-33592 1 78 hesburgh hesburgh PROPN bracis-33592 1 79 library library PROPN bracis-33592 1 80 download download PROPN bracis-33592 1 81 book book NOUN bracis-33592 1 82 pdf pdf PROPN bracis-33592 1 83 download download NOUN bracis-33592 1 84 book book NOUN bracis-33592 1 85 epub epub PROPN bracis-33592 1 86 intelligent intelligent ADJ bracis-33592 1 87 systems system NOUN bracis-33592 1 88 ( ( PUNCT bracis-33592 1 89 bracis bracis NOUN bracis-33592 1 90 2024 2024 NUM bracis-33592 1 91 ) ) PUNCT bracis-33592 1 92 investigating investigate VERB bracis-33592 1 93 universal universal ADJ bracis-33592 1 94 adversarial adversarial ADJ bracis-33592 1 95 attacks attack NOUN bracis-33592 1 96 against against ADP bracis-33592 1 97 transformers transformer NOUN bracis-33592 1 98 - - PUNCT bracis-33592 1 99 based base VERB bracis-33592 1 100 automatic automatic ADJ bracis-33592 1 101 essay essay NOUN bracis-33592 1 102 scoring scoring NOUN bracis-33592 1 103 systems system NOUN bracis-33592 1 104 download download NOUN bracis-33592 1 105 book book NOUN bracis-33592 1 106 pdf pdf PROPN bracis-33592 1 107 download download NOUN bracis-33592 1 108 book book PROPN bracis-33592 1 109 epub epub PROPN bracis-33592 1 110 igor igor PROPN bracis-33592 1 111 cataneo cataneo PROPN bracis-33592 1 112 silveira silveira PROPN bracis-33592 1 113     SPACE bracis-33592 1 114 orcid orcid PROPN bracis-33592 1 115 : : PUNCT bracis-33592 1 116 orcid.org/0000-0003-1599-83479 orcid.org/0000-0003-1599-83479 PROPN bracis-33592 1 117 , , PUNCT bracis-33592 1 118 andré andré VERB bracis-33592 1 119 barbosa barbosa PROPN bracis-33592 1 120     SPACE bracis-33592 1 121 orcid orcid PROPN bracis-33592 1 122 : : PUNCT bracis-33592 1 123 orcid.org/0009-0005-9526-28409 orcid.org/0009-0005-9526-28409 PROPN bracis-33592 1 124 , , PUNCT bracis-33592 1 125 daniel daniel PROPN bracis-33592 1 126 silva silva PROPN bracis-33592 1 127 lopes lopes PROPN bracis-33592 1 128 da da PROPN bracis-33592 1 129 costa costa PROPN bracis-33592 1 130     SPACE bracis-33592 1 131 orcid orcid PROPN bracis-33592 1 132 : : PUNCT bracis-33592 1 133 orcid.org/0009-0008-8358-131x9 orcid.org/0009-0008-8358-131x9 PUNCT bracis-33592 1 134 & & CCONJ bracis-33592 1 135 … … PUNCT bracis-33592 1 136 denis denis PROPN bracis-33592 1 137 deratani deratani PROPN bracis-33592 1 138 mauá mauá NOUN bracis-33592 1 139     SPACE bracis-33592 1 140 orcid orcid NOUN bracis-33592 1 141 : : PUNCT bracis-33592 1 142 orcid.org/0000-0003-2297-63499 orcid.org/0000-0003-2297-63499 ADJ bracis-33592 1 143     SPACE bracis-33592 1 144 show show VERB bracis-33592 1 145 authors author NOUN bracis-33592 1 146 part part NOUN bracis-33592 1 147 of of ADP bracis-33592 1 148 the the DET bracis-33592 1 149 book book NOUN bracis-33592 1 150 series series NOUN bracis-33592 1 151 : : PUNCT bracis-33592 1 152 lecture lecture NOUN bracis-33592 1 153 notes note NOUN bracis-33592 1 154 in in ADP bracis-33592 1 155 computer computer NOUN bracis-33592 1 156 science science NOUN bracis-33592 1 157 ( ( PUNCT bracis-33592 1 158 ( ( PUNCT bracis-33592 1 159 lnai lnai ADJ bracis-33592 1 160 , , PUNCT bracis-33592 1 161 volume volume NOUN bracis-33592 1 162 15413 15413 NUM bracis-33592 1 163 ) ) PUNCT bracis-33592 1 164 ) ) PUNCT bracis-33592 1 165 included include VERB bracis-33592 1 166 in in ADP bracis-33592 1 167 the the DET bracis-33592 1 168 following follow VERB bracis-33592 1 169 conference conference NOUN bracis-33592 1 170 series series NOUN bracis-33592 1 171 : : PUNCT bracis-33592 1 172 brazilian brazilian ADJ bracis-33592 1 173 conference conference NOUN bracis-33592 1 174 on on ADP bracis-33592 1 175 intelligent intelligent ADJ bracis-33592 1 176 systems system NOUN bracis-33592 1 177 387 387 NUM bracis-33592 1 178 accesses access VERB bracis-33592 1 179 3 3 NUM bracis-33592 1 180 citations citation NOUN bracis-33592 1 181 abstract abstract ADJ bracis-33592 1 182 automatic automatic ADJ bracis-33592 1 183 essay essay NOUN bracis-33592 1 184 scoring scoring NOUN bracis-33592 1 185 promises promise NOUN bracis-33592 1 186 to to PART bracis-33592 1 187 scale scale VERB bracis-33592 1 188 up up ADP bracis-33592 1 189 student student NOUN bracis-33592 1 190 feedback feedback NOUN bracis-33592 1 191 on on ADP bracis-33592 1 192 written write VERB bracis-33592 1 193 input input NOUN bracis-33592 1 194 , , PUNCT bracis-33592 1 195 addressing address VERB bracis-33592 1 196 the the DET bracis-33592 1 197 excessive excessive ADJ bracis-33592 1 198 cost cost NOUN bracis-33592 1 199 and and CCONJ bracis-33592 1 200 time time NOUN bracis-33592 1 201 demand demand NOUN bracis-33592 1 202 associated associate VERB bracis-33592 1 203 with with ADP bracis-33592 1 204 human human ADJ bracis-33592 1 205 grading grading NOUN bracis-33592 1 206 . . PUNCT bracis-33592 2 1 state state NOUN bracis-33592 2 2 - - PUNCT bracis-33592 2 3 of of ADP bracis-33592 2 4 - - PUNCT bracis-33592 2 5 the the DET bracis-33592 2 6 - - PUNCT bracis-33592 2 7 art art NOUN bracis-33592 2 8 automatic automatic ADJ bracis-33592 2 9 scorers scorer NOUN bracis-33592 2 10 are be AUX bracis-33592 2 11 based base VERB bracis-33592 2 12 on on ADP bracis-33592 2 13 transformers transformer NOUN bracis-33592 2 14 - - PUNCT bracis-33592 2 15 based base VERB bracis-33592 2 16 neural neural ADJ bracis-33592 2 17 networks network NOUN bracis-33592 2 18 . . PUNCT bracis-33592 3 1 while while SCONJ bracis-33592 3 2 such such ADJ bracis-33592 3 3 models model NOUN bracis-33592 3 4 have have AUX bracis-33592 3 5 shown show VERB bracis-33592 3 6 impressive impressive ADJ bracis-33592 3 7 results result NOUN bracis-33592 3 8 in in ADP bracis-33592 3 9 reasoning reasoning NOUN bracis-33592 3 10 tasks task NOUN bracis-33592 3 11 , , PUNCT bracis-33592 3 12 learned learn VERB bracis-33592 3 13 models model NOUN bracis-33592 3 14 often often ADV bracis-33592 3 15 produce produce VERB bracis-33592 3 16 answers answer NOUN bracis-33592 3 17 that that PRON bracis-33592 3 18 arise arise VERB bracis-33592 3 19 from from ADP bracis-33592 3 20 statistical statistical ADJ bracis-33592 3 21 clues clue NOUN bracis-33592 3 22 in in ADP bracis-33592 3 23 datasets dataset NOUN bracis-33592 3 24 and and CCONJ bracis-33592 3 25 are be AUX bracis-33592 3 26 misaligned misalign VERB bracis-33592 3 27 with with ADP bracis-33592 3 28 human human ADJ bracis-33592 3 29 objectives objective NOUN bracis-33592 3 30 . . PUNCT bracis-33592 4 1 such such ADJ bracis-33592 4 2 systems system NOUN bracis-33592 4 3 are be AUX bracis-33592 4 4 thus thus ADV bracis-33592 4 5 potentially potentially ADV bracis-33592 4 6 fragile fragile ADJ bracis-33592 4 7 for for ADP bracis-33592 4 8 scenarios scenario NOUN bracis-33592 4 9 where where SCONJ bracis-33592 4 10 users user NOUN bracis-33592 4 11 are be AUX bracis-33592 4 12 incentivized incentivize VERB bracis-33592 4 13 to to PART bracis-33592 4 14 deceive deceive VERB bracis-33592 4 15 the the DET bracis-33592 4 16 system system NOUN bracis-33592 4 17 , , PUNCT bracis-33592 4 18 as as ADP bracis-33592 4 19 in in ADP bracis-33592 4 20 a a DET bracis-33592 4 21 classroom classroom NOUN bracis-33592 4 22 setting set VERB bracis-33592 4 23 . . PUNCT bracis-33592 5 1 in in ADP bracis-33592 5 2 this this DET bracis-33592 5 3 work work NOUN bracis-33592 5 4 , , PUNCT bracis-33592 5 5 we we PRON bracis-33592 5 6 evaluate evaluate VERB bracis-33592 5 7 the the DET bracis-33592 5 8 susceptibility susceptibility NOUN bracis-33592 5 9 of of ADP bracis-33592 5 10 state state NOUN bracis-33592 5 11 - - PUNCT bracis-33592 5 12 of of ADP bracis-33592 5 13 - - PUNCT bracis-33592 5 14 the the DET bracis-33592 5 15 - - PUNCT bracis-33592 5 16 art art NOUN bracis-33592 5 17 automatic automatic ADJ bracis-33592 5 18 scorers scorer NOUN bracis-33592 5 19 to to ADP bracis-33592 5 20 attacks attack NOUN bracis-33592 5 21 made make VERB bracis-33592 5 22 by by ADP bracis-33592 5 23 non non ADJ bracis-33592 5 24 - - ADJ bracis-33592 5 25 expert expert ADJ bracis-33592 5 26 users user NOUN bracis-33592 5 27 , , PUNCT bracis-33592 5 28 such such ADJ bracis-33592 5 29 as as ADP bracis-33592 5 30 students student NOUN bracis-33592 5 31 interacting interact VERB bracis-33592 5 32 with with ADP bracis-33592 5 33 an an DET bracis-33592 5 34 automatic automatic ADJ bracis-33592 5 35 grader grader NOUN bracis-33592 5 36 . . PUNCT bracis-33592 6 1 we we PRON bracis-33592 6 2 develop develop VERB bracis-33592 6 3 a a DET bracis-33592 6 4 methodology methodology NOUN bracis-33592 6 5 to to PART bracis-33592 6 6 simulate simulate VERB bracis-33592 6 7 such such ADJ bracis-33592 6 8 student student NOUN bracis-33592 6 9 attacks attack NOUN bracis-33592 6 10 and and CCONJ bracis-33592 6 11 test test VERB bracis-33592 6 12 them they PRON bracis-33592 6 13 against against ADP bracis-33592 6 14 scorers scorer NOUN bracis-33592 6 15 based base VERB bracis-33592 6 16 on on ADP bracis-33592 6 17 bert bert PROPN bracis-33592 6 18 , , PUNCT bracis-33592 6 19 phi-3 phi-3 PUNCT bracis-33592 6 20 and and CCONJ bracis-33592 6 21 gemini gemini NOUN bracis-33592 6 22 models model NOUN bracis-33592 6 23 . . PUNCT bracis-33592 7 1 our our PRON bracis-33592 7 2 findings finding NOUN bracis-33592 7 3 suggest suggest VERB bracis-33592 7 4 that that SCONJ bracis-33592 7 5 ( ( PUNCT bracis-33592 7 6 i i NOUN bracis-33592 7 7 ) ) PUNCT bracis-33592 7 8 a a DET bracis-33592 7 9 bert bert NOUN bracis-33592 7 10 - - PUNCT bracis-33592 7 11 based base VERB bracis-33592 7 12 grader grader NOUN bracis-33592 7 13 can can AUX bracis-33592 7 14 be be AUX bracis-33592 7 15 deceived deceive VERB bracis-33592 7 16 using use VERB bracis-33592 7 17 simple simple ADJ bracis-33592 7 18 feature feature NOUN bracis-33592 7 19 - - PUNCT bracis-33592 7 20 based base VERB bracis-33592 7 21 attacks attack NOUN bracis-33592 7 22 ; ; PUNCT bracis-33592 7 23 ( ( PUNCT bracis-33592 7 24 ii ii NOUN bracis-33592 7 25 ) ) PUNCT bracis-33592 7 26 although although SCONJ bracis-33592 7 27 google google PROPN bracis-33592 7 28 ’s ’s PART bracis-33592 7 29 gemini gemini PROPN bracis-33592 7 30 has have VERB bracis-33592 7 31 a a DET bracis-33592 7 32 solid solid ADJ bracis-33592 7 33 agreement agreement NOUN bracis-33592 7 34 with with ADP bracis-33592 7 35 graders grader NOUN bracis-33592 7 36 , , PUNCT bracis-33592 7 37 it it PRON bracis-33592 7 38 can can AUX bracis-33592 7 39 assign assign VERB bracis-33592 7 40 undeservedly undeservedly ADV bracis-33592 7 41 high high ADJ bracis-33592 7 42 grades grade NOUN bracis-33592 7 43 for for ADP bracis-33592 7 44 small small ADJ bracis-33592 7 45 sentences sentence NOUN bracis-33592 7 46 ; ; PUNCT bracis-33592 7 47 ( ( PUNCT bracis-33592 7 48 iii iii X bracis-33592 7 49 ) ) PUNCT bracis-33592 7 50 a a DET bracis-33592 7 51 phi-3 phi-3 ADV bracis-33592 7 52 - - PUNCT bracis-33592 7 53 based base VERB bracis-33592 7 54 grader grader NOUN bracis-33592 7 55 was be AUX bracis-33592 7 56 less less ADV bracis-33592 7 57 susceptible susceptible ADJ bracis-33592 7 58 than than ADP bracis-33592 7 59 bert bert NOUN bracis-33592 7 60 , , PUNCT bracis-33592 7 61 but but CCONJ bracis-33592 7 62 it it PRON bracis-33592 7 63 still still ADV bracis-33592 7 64 assigned assign VERB bracis-33592 7 65 relatively relatively ADV bracis-33592 7 66 high high ADJ bracis-33592 7 67 grades grade NOUN bracis-33592 7 68 to to ADP bracis-33592 7 69 some some PRON bracis-33592 7 70 of of ADP bracis-33592 7 71 our our PRON bracis-33592 7 72 attacks attack NOUN bracis-33592 7 73 . . PUNCT bracis-33592 8 1 access access NOUN bracis-33592 8 2 provided provide VERB bracis-33592 8 3 by by ADP bracis-33592 8 4 university university PROPN bracis-33592 8 5 of of ADP bracis-33592 8 6 notre notre PROPN bracis-33592 8 7 dame dame PROPN bracis-33592 8 8 hesburgh hesburgh PROPN bracis-33592 8 9 library library PROPN bracis-33592 8 10 . . PUNCT bracis-33592 9 1 download download PROPN bracis-33592 9 2 conference conference NOUN bracis-33592 9 3 paper paper NOUN bracis-33592 9 4 pdf pdf NOUN bracis-33592 9 5 similar similar ADJ bracis-33592 9 6 content content NOUN bracis-33592 9 7 being be AUX bracis-33592 9 8 viewed view VERB bracis-33592 9 9 by by ADP bracis-33592 9 10 others other NOUN bracis-33592 9 11 cheating cheat VERB bracis-33592 9 12 automatic automatic ADJ bracis-33592 9 13 short short ADJ bracis-33592 9 14 answer answer NOUN bracis-33592 9 15 grading grade VERB bracis-33592 9 16 with with ADP bracis-33592 9 17 the the DET bracis-33592 9 18 adversarial adversarial ADJ bracis-33592 9 19 usage usage NOUN bracis-33592 9 20 of of ADP bracis-33592 9 21 adjectives adjective NOUN bracis-33592 9 22 and and CCONJ bracis-33592 9 23 adverbs adverb NOUN bracis-33592 9 24 article article NOUN bracis-33592 9 25 open open ADJ bracis-33592 9 26 access access NOUN bracis-33592 9 27 26 26 NUM bracis-33592 9 28 july july PROPN bracis-33592 9 29 2023 2023 NUM bracis-33592 9 30 automated automate VERB bracis-33592 9 31 grading grading NOUN bracis-33592 9 32 of of ADP bracis-33592 9 33 open open ADV bracis-33592 9 34 - - PUNCT bracis-33592 9 35 ended end VERB bracis-33592 9 36 questions question NOUN bracis-33592 9 37 in in ADP bracis-33592 9 38 higher high ADJ bracis-33592 9 39 education education NOUN bracis-33592 9 40 using use VERB bracis-33592 9 41 genai genai ADJ bracis-33592 9 42 models model NOUN bracis-33592 9 43 article article NOUN bracis-33592 9 44 open open ADJ bracis-33592 9 45 access access NOUN bracis-33592 9 46 19 19 NUM bracis-33592 9 47 october october PROPN bracis-33592 9 48 2025 2025 NUM bracis-33592 9 49 automatic automatic ADJ bracis-33592 9 50 grading grade VERB bracis-33592 9 51 system system NOUN bracis-33592 9 52 using use VERB bracis-33592 9 53 sentence sentence NOUN bracis-33592 9 54 - - PUNCT bracis-33592 9 55 bert bert NOUN bracis-33592 9 56 network network NOUN bracis-33592 9 57 chapter chapter NOUN bracis-33592 9 58 © © PROPN bracis-33592 9 59 2020 2020 NUM bracis-33592 10 1 1 1 NUM bracis-33592 10 2 introduction introduction NOUN bracis-33592 10 3 automatic automatic ADJ bracis-33592 10 4 essay essay NOUN bracis-33592 10 5 scoring scoring NOUN bracis-33592 10 6 ( ( PUNCT bracis-33592 10 7 aes aes NOUN bracis-33592 10 8 ) ) PUNCT bracis-33592 10 9 systems system NOUN bracis-33592 10 10 ease ease VERB bracis-33592 10 11 the the DET bracis-33592 10 12 burden burden NOUN bracis-33592 10 13 on on ADP bracis-33592 10 14 teachers teacher NOUN bracis-33592 10 15 while while SCONJ bracis-33592 10 16 enhancing enhance VERB bracis-33592 10 17 student student NOUN bracis-33592 10 18 learning learning NOUN bracis-33592 10 19 by by ADP bracis-33592 10 20 providing provide VERB bracis-33592 10 21 meaningful meaningful ADJ bracis-33592 10 22 , , PUNCT bracis-33592 10 23 timely timely ADJ bracis-33592 10 24 and and CCONJ bracis-33592 10 25 personalized personalized ADJ bracis-33592 10 26 feedback feedback NOUN bracis-33592 10 27 [ [ X bracis-33592 10 28 32 32 NUM bracis-33592 10 29 ] ] PUNCT bracis-33592 10 30 . . PUNCT bracis-33592 11 1 the the DET bracis-33592 11 2 technology technology NOUN bracis-33592 11 3 is be AUX bracis-33592 11 4 currently currently ADV bracis-33592 11 5 mature mature ADJ bracis-33592 11 6 enough enough ADV bracis-33592 11 7 to to PART bracis-33592 11 8 be be AUX bracis-33592 11 9 employed employ VERB bracis-33592 11 10 in in ADP bracis-33592 11 11 high high ADJ bracis-33592 11 12 - - PUNCT bracis-33592 11 13 stakes stake NOUN bracis-33592 11 14 standardized standardized ADJ bracis-33592 11 15 exams exam NOUN bracis-33592 11 16 such such ADJ bracis-33592 11 17 as as ADP bracis-33592 11 18 the the DET bracis-33592 11 19 gre gre NOUN bracis-33592 11 20 and and CCONJ bracis-33592 11 21 toefl toefl NOUN bracis-33592 12 1 [ [ X bracis-33592 12 2 3 3 NUM bracis-33592 12 3 , , PUNCT bracis-33592 12 4 5 5 NUM bracis-33592 12 5 ] ] PUNCT bracis-33592 12 6 . . PUNCT bracis-33592 13 1 like like ADP bracis-33592 13 2 many many ADJ bracis-33592 13 3 machine machine NOUN bracis-33592 13 4 learning learn VERB bracis-33592 13 5 based base VERB bracis-33592 13 6 solutions solution NOUN bracis-33592 13 7 , , PUNCT bracis-33592 13 8 aes aes NOUN bracis-33592 13 9 systems system NOUN bracis-33592 13 10 are be AUX bracis-33592 13 11 prone prone ADJ bracis-33592 13 12 to to ADP bracis-33592 13 13 malicious malicious ADJ bracis-33592 13 14 usage usage NOUN bracis-33592 13 15 and and CCONJ bracis-33592 13 16 are be AUX bracis-33592 13 17 sensitive sensitive ADJ bracis-33592 13 18 to to ADP bracis-33592 13 19 spurious spurious ADJ bracis-33592 13 20 correlations correlation NOUN bracis-33592 13 21 in in ADP bracis-33592 13 22 the the DET bracis-33592 13 23 training training NOUN bracis-33592 13 24 data datum NOUN bracis-33592 13 25 [ [ X bracis-33592 13 26 22 22 NUM bracis-33592 13 27 ] ] PUNCT bracis-33592 13 28 . . PUNCT bracis-33592 14 1 this this PRON bracis-33592 14 2 is be AUX bracis-33592 14 3 particularly particularly ADV bracis-33592 14 4 relevant relevant ADJ bracis-33592 14 5 in in ADP bracis-33592 14 6 educational educational ADJ bracis-33592 14 7 and and CCONJ bracis-33592 14 8 scoring scoring NOUN bracis-33592 14 9 settings setting NOUN bracis-33592 14 10 , , PUNCT bracis-33592 14 11 where where SCONJ bracis-33592 14 12 users user NOUN bracis-33592 14 13 might might AUX bracis-33592 14 14 be be AUX bracis-33592 14 15 motivated motivate VERB bracis-33592 14 16 to to PART bracis-33592 14 17 exploit exploit VERB bracis-33592 14 18 the the DET bracis-33592 14 19 system system NOUN bracis-33592 14 20 ’s ’s PART bracis-33592 14 21 vulnerability vulnerability NOUN bracis-33592 14 22 , , PUNCT bracis-33592 14 23 and and CCONJ bracis-33592 14 24 where where SCONJ bracis-33592 14 25 a a DET bracis-33592 14 26 correct correct ADJ bracis-33592 14 27 prediction prediction NOUN bracis-33592 14 28 for for ADP bracis-33592 14 29 the the DET bracis-33592 14 30 wrong wrong ADJ bracis-33592 14 31 reason reason NOUN bracis-33592 14 32 might might AUX bracis-33592 14 33 be be AUX bracis-33592 14 34 as as ADV bracis-33592 14 35 harmful harmful ADJ bracis-33592 14 36 as as ADP bracis-33592 14 37 an an DET bracis-33592 14 38 incorrect incorrect ADJ bracis-33592 14 39 prediction prediction NOUN bracis-33592 14 40 . . PUNCT bracis-33592 15 1 unintended unintended ADJ bracis-33592 15 2 uses use NOUN bracis-33592 15 3 of of ADP bracis-33592 15 4 predictive predictive ADJ bracis-33592 15 5 systems system NOUN bracis-33592 15 6 are be AUX bracis-33592 15 7 known know VERB bracis-33592 15 8 in in ADP bracis-33592 15 9 the the DET bracis-33592 15 10 machine machine NOUN bracis-33592 15 11 learning learn VERB bracis-33592 15 12 literature literature NOUN bracis-33592 15 13 as as ADP bracis-33592 15 14 adversarial adversarial ADJ bracis-33592 15 15 attacks attack NOUN bracis-33592 15 16 [ [ X bracis-33592 15 17 21 21 NUM bracis-33592 15 18 ] ] PUNCT bracis-33592 15 19 . . PUNCT bracis-33592 16 1 the the DET bracis-33592 16 2 more more ADV bracis-33592 16 3 recent recent ADJ bracis-33592 16 4 literature literature NOUN bracis-33592 16 5 cares care VERB bracis-33592 16 6 specifically specifically ADV bracis-33592 16 7 about about ADP bracis-33592 16 8 attacks attack NOUN bracis-33592 16 9 that that PRON bracis-33592 16 10 are be AUX bracis-33592 16 11 carried carry VERB bracis-33592 16 12 out out ADP bracis-33592 16 13 by by ADP bracis-33592 16 14 changing change VERB bracis-33592 16 15 an an DET bracis-33592 16 16 existing exist VERB bracis-33592 16 17 input input NOUN bracis-33592 16 18 in in ADP bracis-33592 16 19 a a DET bracis-33592 16 20 human human ADJ bracis-33592 16 21 imperceptible imperceptible ADJ bracis-33592 16 22 way way NOUN bracis-33592 16 23 and and CCONJ bracis-33592 16 24 such such ADJ bracis-33592 16 25 that that SCONJ bracis-33592 16 26 the the DET bracis-33592 16 27 prediction prediction NOUN bracis-33592 16 28 shifts shift NOUN bracis-33592 16 29 towards towards ADP bracis-33592 16 30 or or CCONJ bracis-33592 16 31 away away ADV bracis-33592 16 32 from from ADP bracis-33592 16 33 some some DET bracis-33592 16 34 desired desire VERB bracis-33592 16 35 target target NOUN bracis-33592 16 36 value value NOUN bracis-33592 16 37 [ [ X bracis-33592 16 38 13 13 NUM bracis-33592 16 39 , , PUNCT bracis-33592 16 40 44 44 NUM bracis-33592 16 41 ] ] PUNCT bracis-33592 16 42 . . PUNCT bracis-33592 17 1 for for ADP bracis-33592 17 2 example example NOUN bracis-33592 17 3 , , PUNCT bracis-33592 17 4 in in ADP bracis-33592 17 5 aes aes NOUN bracis-33592 17 6 that that PRON bracis-33592 17 7 might might AUX bracis-33592 17 8 amount amount VERB bracis-33592 17 9 to to ADP bracis-33592 17 10 determining determine VERB bracis-33592 17 11 words word NOUN bracis-33592 17 12 or or CCONJ bracis-33592 17 13 expressions expression NOUN bracis-33592 17 14 in in ADP bracis-33592 17 15 an an DET bracis-33592 17 16 essay essay NOUN bracis-33592 17 17 that that PRON bracis-33592 17 18 , , PUNCT bracis-33592 17 19 if if SCONJ bracis-33592 17 20 changed change VERB bracis-33592 17 21 , , PUNCT bracis-33592 17 22 maintain maintain VERB bracis-33592 17 23 the the DET bracis-33592 17 24 text text NOUN bracis-33592 17 25 ’s ’s PART bracis-33592 17 26 overall overall ADJ bracis-33592 17 27 characteristics characteristic NOUN bracis-33592 17 28 as as SCONJ bracis-33592 17 29 perceived perceive VERB bracis-33592 17 30 by by ADP bracis-33592 17 31 a a DET bracis-33592 17 32 human human ADJ bracis-33592 17 33 while while NOUN bracis-33592 17 34 leading lead VERB bracis-33592 17 35 the the DET bracis-33592 17 36 predictive predictive ADJ bracis-33592 17 37 model model NOUN bracis-33592 17 38 to to PART bracis-33592 17 39 raise raise VERB bracis-33592 17 40 the the DET bracis-33592 17 41 score score NOUN bracis-33592 17 42 significantly significantly ADV bracis-33592 17 43 . . PUNCT bracis-33592 18 1 such such ADJ bracis-33592 18 2 attacks attack NOUN bracis-33592 18 3 usually usually ADV bracis-33592 18 4 require require VERB bracis-33592 18 5 knowledge knowledge NOUN bracis-33592 18 6 and and CCONJ bracis-33592 18 7 access access NOUN bracis-33592 18 8 to to ADP bracis-33592 18 9 the the DET bracis-33592 18 10 system system NOUN bracis-33592 18 11 ’s ’s PART bracis-33592 18 12 characteristics characteristic NOUN bracis-33592 18 13 . . PUNCT bracis-33592 19 1 thus thus ADV bracis-33592 19 2 , , PUNCT bracis-33592 19 3 they they PRON bracis-33592 19 4 are be AUX bracis-33592 19 5 usually usually ADV bracis-33592 19 6 difficult difficult ADJ bracis-33592 19 7 to to PART bracis-33592 19 8 be be AUX bracis-33592 19 9 performed perform VERB bracis-33592 19 10 by by ADP bracis-33592 19 11 regular regular ADJ bracis-33592 19 12 users user NOUN bracis-33592 19 13 of of ADP bracis-33592 19 14 aes aes PROPN bracis-33592 19 15 systems system NOUN bracis-33592 19 16 . . PUNCT bracis-33592 20 1 a a DET bracis-33592 20 2 more more ADV bracis-33592 20 3 straightforward straightforward ADJ bracis-33592 20 4 approach approach NOUN bracis-33592 20 5 is be AUX bracis-33592 20 6 the the DET bracis-33592 20 7 universal universal ADJ bracis-33592 20 8 attack attack NOUN bracis-33592 20 9 , , PUNCT bracis-33592 20 10 which which PRON bracis-33592 20 11 consists consist VERB bracis-33592 20 12 of of ADP bracis-33592 20 13 finding find VERB bracis-33592 20 14 input input NOUN bracis-33592 20 15 - - PUNCT bracis-33592 20 16 unrelated unrelated ADJ bracis-33592 20 17 rules rule NOUN bracis-33592 20 18 that that PRON bracis-33592 20 19 cause cause VERB bracis-33592 20 20 the the DET bracis-33592 20 21 predictive predictive ADJ bracis-33592 20 22 model model NOUN bracis-33592 20 23 to to PART bracis-33592 20 24 increase increase VERB bracis-33592 20 25 scores score NOUN bracis-33592 20 26 . . PUNCT bracis-33592 21 1 for for ADP bracis-33592 21 2 example example NOUN bracis-33592 21 3 , , PUNCT bracis-33592 21 4 it it PRON bracis-33592 21 5 has have AUX bracis-33592 21 6 been be AUX bracis-33592 21 7 noticed notice VERB bracis-33592 21 8 that that SCONJ bracis-33592 21 9 essay essay ADJ bracis-33592 21 10 length length NOUN bracis-33592 21 11 is be AUX bracis-33592 21 12 a a DET bracis-33592 21 13 good good ADJ bracis-33592 21 14 predictor predictor NOUN bracis-33592 21 15 of of ADP bracis-33592 21 16 essay essay ADJ bracis-33592 21 17 quality quality NOUN bracis-33592 22 1 [ [ X bracis-33592 22 2 26 26 NUM bracis-33592 22 3 ] ] PUNCT bracis-33592 22 4 . . PUNCT bracis-33592 23 1 a a DET bracis-33592 23 2 malicious malicious ADJ bracis-33592 23 3 user user NOUN bracis-33592 23 4 can can AUX bracis-33592 23 5 exploit exploit VERB bracis-33592 23 6 this this DET bracis-33592 23 7 fact fact NOUN bracis-33592 23 8 by by ADP bracis-33592 23 9 appending append VERB bracis-33592 23 10 unrelated unrelated ADJ bracis-33592 23 11 text text NOUN bracis-33592 23 12 to to ADP bracis-33592 23 13 an an DET bracis-33592 23 14 essay essay NOUN bracis-33592 23 15 to to PART bracis-33592 23 16 increase increase VERB bracis-33592 23 17 its its PRON bracis-33592 23 18 score score NOUN bracis-33592 23 19 . . PUNCT bracis-33592 24 1 such such DET bracis-33592 24 2 an an DET bracis-33592 24 3 attack attack NOUN bracis-33592 24 4 does do AUX bracis-33592 24 5 not not PART bracis-33592 24 6 require require VERB bracis-33592 24 7 deep deep ADJ bracis-33592 24 8 knowledge knowledge NOUN bracis-33592 24 9 of of ADP bracis-33592 24 10 the the DET bracis-33592 24 11 system system NOUN bracis-33592 24 12 and and CCONJ bracis-33592 24 13 can can AUX bracis-33592 24 14 be be AUX bracis-33592 24 15 formulated formulate VERB bracis-33592 24 16 by by ADP bracis-33592 24 17 repeated repeat VERB bracis-33592 24 18 use use NOUN bracis-33592 24 19 . . PUNCT bracis-33592 25 1 designers designer NOUN bracis-33592 25 2 of of ADP bracis-33592 25 3 aes aes PROPN bracis-33592 25 4 systems system NOUN bracis-33592 25 5 have have AUX bracis-33592 25 6 long long ADV bracis-33592 25 7 been be AUX bracis-33592 25 8 aware aware ADJ bracis-33592 25 9 of of ADP bracis-33592 25 10 the the DET bracis-33592 25 11 pitfalls pitfall NOUN bracis-33592 25 12 of of ADP bracis-33592 25 13 malicious malicious ADJ bracis-33592 25 14 usage usage NOUN bracis-33592 25 15 . . PUNCT bracis-33592 26 1 for for ADP bracis-33592 26 2 instance instance NOUN bracis-33592 26 3 , , PUNCT bracis-33592 26 4 authors author NOUN bracis-33592 26 5 have have AUX bracis-33592 26 6 discussed discuss VERB bracis-33592 26 7 whether whether SCONJ bracis-33592 26 8 to to PART bracis-33592 26 9 remove remove VERB bracis-33592 26 10 certain certain ADJ bracis-33592 26 11 predictive predictive ADJ bracis-33592 26 12 features feature NOUN bracis-33592 26 13 such such ADJ bracis-33592 26 14 as as ADP bracis-33592 26 15 text text NOUN bracis-33592 26 16 length length NOUN bracis-33592 26 17 in in ADP bracis-33592 26 18 order order NOUN bracis-33592 26 19 to to PART bracis-33592 26 20 improve improve VERB bracis-33592 26 21 the the DET bracis-33592 26 22 system system NOUN bracis-33592 26 23 ’s ’s PART bracis-33592 26 24 robustness robustness NOUN bracis-33592 26 25 against against ADP bracis-33592 26 26 attacks attack NOUN bracis-33592 26 27 [ [ X bracis-33592 26 28 42 42 NUM bracis-33592 26 29 ] ] PUNCT bracis-33592 26 30 . . PUNCT bracis-33592 27 1 the the DET bracis-33592 27 2 matter matter NOUN bracis-33592 27 3 is be AUX bracis-33592 27 4 complicated complicate VERB bracis-33592 27 5 by by ADP bracis-33592 27 6 the the DET bracis-33592 27 7 fact fact NOUN bracis-33592 27 8 that that SCONJ bracis-33592 27 9 textual textual ADJ bracis-33592 27 10 features feature NOUN bracis-33592 27 11 are be AUX bracis-33592 27 12 often often ADV bracis-33592 27 13 correlated correlate VERB bracis-33592 27 14 , , PUNCT bracis-33592 27 15 so so SCONJ bracis-33592 27 16 that that SCONJ bracis-33592 27 17 removing remove VERB bracis-33592 27 18 a a DET bracis-33592 27 19 feature feature NOUN bracis-33592 27 20 might might AUX bracis-33592 27 21 hurt hurt VERB bracis-33592 27 22 performance performance NOUN bracis-33592 27 23 without without ADP bracis-33592 27 24 actually actually ADV bracis-33592 27 25 making make VERB bracis-33592 27 26 the the DET bracis-33592 27 27 system system NOUN bracis-33592 27 28 less less ADV bracis-33592 27 29 susceptible susceptible ADJ bracis-33592 27 30 to to ADP bracis-33592 27 31 attacks attack NOUN bracis-33592 27 32 . . PUNCT bracis-33592 28 1 following follow VERB bracis-33592 28 2 the the DET bracis-33592 28 3 trend trend NOUN bracis-33592 28 4 in in ADP bracis-33592 28 5 natural natural ADJ bracis-33592 28 6 language language NOUN bracis-33592 28 7 processing processing NOUN bracis-33592 28 8 , , PUNCT bracis-33592 28 9 state state NOUN bracis-33592 28 10 - - PUNCT bracis-33592 28 11 of of ADP bracis-33592 28 12 - - PUNCT bracis-33592 28 13 the the DET bracis-33592 28 14 - - PUNCT bracis-33592 28 15 art art NOUN bracis-33592 28 16 aes aes NOUN bracis-33592 28 17 systems system NOUN bracis-33592 28 18 are be AUX bracis-33592 28 19 based base VERB bracis-33592 28 20 on on ADP bracis-33592 28 21 transformer transformer ADJ bracis-33592 28 22 deep deep ADJ bracis-33592 28 23 neural neural ADJ bracis-33592 28 24 networks network NOUN bracis-33592 28 25 [ [ X bracis-33592 28 26 27 27 NUM bracis-33592 28 27 ] ] PUNCT bracis-33592 28 28 . . PUNCT bracis-33592 29 1 while while SCONJ bracis-33592 29 2 in in ADP bracis-33592 29 3 principle principle NOUN bracis-33592 29 4 such such ADJ bracis-33592 29 5 models model NOUN bracis-33592 29 6 can can AUX bracis-33592 29 7 learn learn VERB bracis-33592 29 8 rich rich ADJ bracis-33592 29 9 representations representation NOUN bracis-33592 29 10 and and CCONJ bracis-33592 29 11 enable enable VERB bracis-33592 29 12 logical logical ADJ bracis-33592 29 13 and and CCONJ bracis-33592 29 14 commonsense commonsense ADJ bracis-33592 29 15 reasoning reasoning NOUN bracis-33592 29 16 , , PUNCT bracis-33592 29 17 there there PRON bracis-33592 29 18 is be VERB bracis-33592 29 19 evidence evidence NOUN bracis-33592 29 20 that that SCONJ bracis-33592 29 21 learning learn VERB bracis-33592 29 22 algorithms algorithm NOUN bracis-33592 29 23 and and CCONJ bracis-33592 29 24 spurious spurious ADJ bracis-33592 29 25 correlations correlation NOUN bracis-33592 29 26 in in ADP bracis-33592 29 27 data datum NOUN bracis-33592 29 28 often often ADV bracis-33592 29 29 lead lead VERB bracis-33592 29 30 to to ADP bracis-33592 29 31 predictions prediction NOUN bracis-33592 29 32 being be AUX bracis-33592 29 33 based base VERB bracis-33592 29 34 on on ADP bracis-33592 29 35 simple simple ADJ bracis-33592 29 36 statistical statistical ADJ bracis-33592 29 37 cues cue NOUN bracis-33592 30 1 [ [ X bracis-33592 30 2 43 43 NUM bracis-33592 30 3 ] ] X bracis-33592 30 4 ; ; PUNCT bracis-33592 30 5 hence hence ADV bracis-33592 30 6 such such ADJ bracis-33592 30 7 systems system NOUN bracis-33592 30 8 might might AUX bracis-33592 30 9 be be AUX bracis-33592 30 10 as as ADP bracis-33592 30 11 exposed expose VERB bracis-33592 30 12 to to ADP bracis-33592 30 13 attacks attack NOUN bracis-33592 30 14 as as ADP bracis-33592 30 15 traditional traditional ADJ bracis-33592 30 16 feature feature NOUN bracis-33592 30 17 - - PUNCT bracis-33592 30 18 based base VERB bracis-33592 30 19 systems system NOUN bracis-33592 30 20 . . PUNCT bracis-33592 31 1 in in ADP bracis-33592 31 2 this this DET bracis-33592 31 3 work work NOUN bracis-33592 31 4 , , PUNCT bracis-33592 31 5 we we PRON bracis-33592 31 6 investigate investigate VERB bracis-33592 31 7 whether whether SCONJ bracis-33592 31 8 transformers transformer NOUN bracis-33592 31 9 - - PUNCT bracis-33592 31 10 based base VERB bracis-33592 31 11 aes aes NOUN bracis-33592 31 12 systems system NOUN bracis-33592 31 13 are be AUX bracis-33592 31 14 susceptible susceptible ADJ bracis-33592 31 15 to to ADP bracis-33592 31 16 universal universal ADJ bracis-33592 31 17 adversarial adversarial ADJ bracis-33592 31 18 attacks attack NOUN bracis-33592 31 19 that that PRON bracis-33592 31 20 might might AUX bracis-33592 31 21 occur occur VERB bracis-33592 31 22 in in ADP bracis-33592 31 23 a a DET bracis-33592 31 24 classroom classroom NOUN bracis-33592 31 25 setting setting NOUN bracis-33592 31 26 . . PUNCT bracis-33592 32 1 in in ADP bracis-33592 32 2 particular particular ADJ bracis-33592 32 3 , , PUNCT bracis-33592 32 4 we we PRON bracis-33592 32 5 consider consider VERB bracis-33592 32 6 a a DET bracis-33592 32 7 fictitious fictitious ADJ bracis-33592 32 8 ( ( PUNCT bracis-33592 32 9 but but CCONJ bracis-33592 32 10 realistic realistic ADJ bracis-33592 32 11 ) ) PUNCT bracis-33592 32 12 case case NOUN bracis-33592 32 13 where where SCONJ bracis-33592 32 14 users user NOUN bracis-33592 32 15 ( ( PUNCT bracis-33592 32 16 i.e. i.e. X bracis-33592 32 17 , , PUNCT bracis-33592 32 18 students student NOUN bracis-33592 32 19 ) ) PUNCT bracis-33592 32 20 interact interact VERB bracis-33592 32 21 with with ADP bracis-33592 32 22 the the DET bracis-33592 32 23 system system NOUN bracis-33592 32 24 by by ADP bracis-33592 32 25 submitting submit VERB bracis-33592 32 26 an an DET bracis-33592 32 27 essay essay NOUN bracis-33592 32 28 ( ( PUNCT bracis-33592 32 29 input input NOUN bracis-33592 32 30 ) ) PUNCT bracis-33592 32 31 and and CCONJ bracis-33592 32 32 receiving receive VERB bracis-33592 32 33 a a DET bracis-33592 32 34 score score NOUN bracis-33592 32 35 ( ( PUNCT bracis-33592 32 36 the the DET bracis-33592 32 37 output output NOUN bracis-33592 32 38 ) ) PUNCT bracis-33592 32 39 , , PUNCT bracis-33592 32 40 while while SCONJ bracis-33592 32 41 having have VERB bracis-33592 32 42 no no DET bracis-33592 32 43 knowledge knowledge NOUN bracis-33592 32 44 of of ADP bracis-33592 32 45 the the DET bracis-33592 32 46 predictive predictive ADJ bracis-33592 32 47 model model NOUN bracis-33592 32 48 being be AUX bracis-33592 32 49 used use VERB bracis-33592 32 50 ( ( PUNCT bracis-33592 32 51 including include VERB bracis-33592 32 52 parameters parameter NOUN bracis-33592 32 53 , , PUNCT bracis-33592 32 54 derivatives derivative NOUN bracis-33592 32 55 , , PUNCT bracis-33592 32 56 etc etc X bracis-33592 32 57 . . X bracis-33592 32 58 ) ) PUNCT bracis-33592 32 59 . . PUNCT bracis-33592 33 1 the the DET bracis-33592 33 2 hypothetical hypothetical ADJ bracis-33592 33 3 system system NOUN bracis-33592 33 4 is be AUX bracis-33592 33 5 non non ADJ bracis-33592 33 6 - - ADJ bracis-33592 33 7 adaptive adaptive ADJ bracis-33592 33 8 ( ( PUNCT bracis-33592 33 9 i.e. i.e. X bracis-33592 33 10 , , PUNCT bracis-33592 33 11 no no DET bracis-33592 33 12 learning learning NOUN bracis-33592 33 13 after after ADP bracis-33592 33 14 deployment deployment NOUN bracis-33592 33 15 ) ) PUNCT bracis-33592 33 16 and and CCONJ bracis-33592 33 17 students student NOUN bracis-33592 33 18 use use VERB bracis-33592 33 19 the the DET bracis-33592 33 20 system system NOUN bracis-33592 33 21 repeatedly repeatedly ADV bracis-33592 33 22 ( ( PUNCT bracis-33592 33 23 e.g. e.g. ADV bracis-33592 33 24 grading grade VERB bracis-33592 33 25 different different ADJ bracis-33592 33 26 essays essay NOUN bracis-33592 33 27 along along ADP bracis-33592 33 28 the the DET bracis-33592 33 29 academic academic ADJ bracis-33592 33 30 term term NOUN bracis-33592 33 31 or or CCONJ bracis-33592 33 32 being be AUX bracis-33592 33 33 able able ADJ bracis-33592 33 34 to to PART bracis-33592 33 35 submit submit VERB bracis-33592 33 36 an an DET bracis-33592 33 37 improved improved ADJ bracis-33592 33 38 version version NOUN bracis-33592 33 39 ) ) PUNCT bracis-33592 33 40 and and CCONJ bracis-33592 33 41 share share VERB bracis-33592 33 42 information information NOUN bracis-33592 33 43 about about ADP bracis-33592 33 44 usage usage NOUN bracis-33592 33 45 among among ADP bracis-33592 33 46 themselves themselves PRON bracis-33592 33 47 . . PUNCT bracis-33592 34 1 given give VERB bracis-33592 34 2 that that PRON bracis-33592 34 3 cheating cheat VERB bracis-33592 34 4 in in ADP bracis-33592 34 5 the the DET bracis-33592 34 6 classroom classroom NOUN bracis-33592 34 7 is be AUX bracis-33592 34 8 prevalent prevalent ADJ bracis-33592 34 9 [ [ X bracis-33592 34 10 20 20 NUM bracis-33592 34 11 , , PUNCT bracis-33592 34 12 41 41 NUM bracis-33592 34 13 ] ] PUNCT bracis-33592 34 14 , , PUNCT bracis-33592 34 15 some some DET bracis-33592 34 16 students student NOUN bracis-33592 34 17 are be AUX bracis-33592 34 18 expected expect VERB bracis-33592 34 19 to to PART bracis-33592 34 20 exploit exploit VERB bracis-33592 34 21 any any DET bracis-33592 34 22 vulnerability vulnerability NOUN bracis-33592 34 23 to to PART bracis-33592 34 24 improve improve VERB bracis-33592 34 25 their their PRON bracis-33592 34 26 grade grade NOUN bracis-33592 34 27 without without ADP bracis-33592 34 28 necessarily necessarily ADV bracis-33592 34 29 improving improve VERB bracis-33592 34 30 the the DET bracis-33592 34 31 quality quality NOUN bracis-33592 34 32 of of ADP bracis-33592 34 33 their their PRON bracis-33592 34 34 texts text NOUN bracis-33592 34 35 . . PUNCT bracis-33592 35 1 to to PART bracis-33592 35 2 simulate simulate VERB bracis-33592 35 3 how how SCONJ bracis-33592 35 4 such such ADJ bracis-33592 35 5 non non ADJ bracis-33592 35 6 - - ADJ bracis-33592 35 7 expert expert ADJ bracis-33592 35 8 users user NOUN bracis-33592 35 9 might might AUX bracis-33592 35 10 learn learn VERB bracis-33592 35 11 vulnerabilities vulnerability NOUN bracis-33592 35 12 , , PUNCT bracis-33592 35 13 we we PRON bracis-33592 35 14 train train VERB bracis-33592 35 15 an an DET bracis-33592 35 16 interpretable interpretable ADJ bracis-33592 35 17 predictive predictive ADJ bracis-33592 35 18 model model NOUN bracis-33592 35 19 ( ( PUNCT bracis-33592 35 20 a a DET bracis-33592 35 21 linear linear ADJ bracis-33592 35 22 regressor regressor NOUN bracis-33592 35 23 ) ) PUNCT bracis-33592 35 24 based base VERB bracis-33592 35 25 on on ADP bracis-33592 35 26 handcrafted handcraft VERB bracis-33592 35 27 features feature NOUN bracis-33592 35 28 and and CCONJ bracis-33592 35 29 select select VERB bracis-33592 35 30 the the DET bracis-33592 35 31 most most ADV bracis-33592 35 32 predictive predictive ADJ bracis-33592 35 33 features feature NOUN bracis-33592 35 34 to to PART bracis-33592 35 35 compose compose VERB bracis-33592 35 36 attacks attack NOUN bracis-33592 35 37 , , PUNCT bracis-33592 35 38 such such ADJ bracis-33592 35 39 as as ADP bracis-33592 35 40 inserting insert VERB bracis-33592 35 41 lists list NOUN bracis-33592 35 42 of of ADP bracis-33592 35 43 adjectives adjective NOUN bracis-33592 35 44 of of ADP bracis-33592 35 45 adverbs adverb NOUN bracis-33592 35 46 , , PUNCT bracis-33592 35 47 employing employ VERB bracis-33592 35 48 more more ADJ bracis-33592 35 49 adjectives adjective NOUN bracis-33592 35 50 and and CCONJ bracis-33592 35 51 adverbs adverb NOUN bracis-33592 35 52 than than ADP bracis-33592 35 53 usual usual ADJ bracis-33592 35 54 , , PUNCT bracis-33592 35 55 and and CCONJ bracis-33592 35 56 repeating repeat VERB bracis-33592 35 57 the the DET bracis-33592 35 58 same same ADJ bracis-33592 35 59 sentence sentence NOUN bracis-33592 35 60 multiple multiple ADJ bracis-33592 35 61 times time NOUN bracis-33592 35 62 . . PUNCT bracis-33592 36 1 the the DET bracis-33592 36 2 rationale rationale NOUN bracis-33592 36 3 is be AUX bracis-33592 36 4 that that SCONJ bracis-33592 36 5 by by ADP bracis-33592 36 6 repeated repeat VERB bracis-33592 36 7 interaction interaction NOUN bracis-33592 36 8 and and CCONJ bracis-33592 36 9 information information NOUN bracis-33592 36 10 sharing sharing NOUN bracis-33592 36 11 , , PUNCT bracis-33592 36 12 users user NOUN bracis-33592 36 13 might might AUX bracis-33592 36 14 develop develop VERB bracis-33592 36 15 a a DET bracis-33592 36 16 simplified simplified ADJ bracis-33592 36 17 model model NOUN bracis-33592 36 18 of of ADP bracis-33592 36 19 the the DET bracis-33592 36 20 system system NOUN bracis-33592 36 21 ’s ’s PART bracis-33592 36 22 inner inner ADJ bracis-33592 36 23 work work NOUN bracis-33592 36 24 and and CCONJ bracis-33592 36 25 use use VERB bracis-33592 36 26 such such DET bracis-33592 36 27 a a DET bracis-33592 36 28 model model NOUN bracis-33592 36 29 to to PART bracis-33592 36 30 come come VERB bracis-33592 36 31 up up ADP bracis-33592 36 32 with with ADP bracis-33592 36 33 such such ADJ bracis-33592 36 34 simple simple ADJ bracis-33592 36 35 attacking attacking NOUN bracis-33592 36 36 strategies strategy NOUN bracis-33592 36 37 . . PUNCT bracis-33592 37 1 the the DET bracis-33592 37 2 effectiveness effectiveness NOUN bracis-33592 37 3 of of ADP bracis-33592 37 4 the the DET bracis-33592 37 5 attacks attack NOUN bracis-33592 37 6 is be AUX bracis-33592 37 7 evaluated evaluate VERB bracis-33592 37 8 with with ADP bracis-33592 37 9 respect respect NOUN bracis-33592 37 10 to to ADP bracis-33592 37 11 bert bert PROPN bracis-33592 37 12 , , PUNCT bracis-33592 37 13 phi-3 phi-3 PUNCT bracis-33592 37 14 and and CCONJ bracis-33592 37 15 gemini gemini NOUN bracis-33592 37 16 - - PUNCT bracis-33592 37 17 based base VERB bracis-33592 37 18 aes aes NOUN bracis-33592 37 19 systems system NOUN bracis-33592 37 20 on on ADP bracis-33592 37 21 a a DET bracis-33592 37 22 benchmark benchmark NOUN bracis-33592 37 23 of of ADP bracis-33592 37 24 human human ADJ bracis-33592 37 25 annotated annotate VERB bracis-33592 37 26 ( ( PUNCT bracis-33592 37 27 brazilian brazilian ADJ bracis-33592 37 28 ) ) PUNCT bracis-33592 37 29 portuguese portuguese ADJ bracis-33592 37 30 essays essay NOUN bracis-33592 37 31 [ [ X bracis-33592 37 32 34 34 NUM bracis-33592 37 33 ] ] PUNCT bracis-33592 37 34 . . PUNCT bracis-33592 38 1 we we PRON bracis-33592 38 2 show show VERB bracis-33592 38 3 that that SCONJ bracis-33592 38 4 all all DET bracis-33592 38 5 models model NOUN bracis-33592 38 6 are be AUX bracis-33592 38 7 vulnerable vulnerable ADJ bracis-33592 38 8 to to ADP bracis-33592 38 9 simple simple ADJ bracis-33592 38 10 attacks.footnote attacks.footnote PROPN bracis-33592 38 11 1 1 NUM bracis-33592 38 12 for for ADP bracis-33592 38 13 example example NOUN bracis-33592 38 14 , , PUNCT bracis-33592 38 15 on on ADP bracis-33592 38 16 a a DET bracis-33592 38 17 scale scale NOUN bracis-33592 38 18 of of ADP bracis-33592 38 19 0 0 NUM bracis-33592 38 20 to to ADP bracis-33592 38 21 1000 1000 NUM bracis-33592 38 22 , , PUNCT bracis-33592 38 23 where where SCONJ bracis-33592 38 24 higher high ADJ bracis-33592 38 25 numbers number NOUN bracis-33592 38 26 are be AUX bracis-33592 38 27 better well ADJ bracis-33592 38 28 , , PUNCT bracis-33592 38 29 the the DET bracis-33592 38 30 bert bert NOUN bracis-33592 38 31 and and CCONJ bracis-33592 38 32 phi-3 phi-3 PUNCT bracis-33592 38 33 based base VERB bracis-33592 38 34 models model NOUN bracis-33592 38 35 assigned assign VERB bracis-33592 38 36 , , PUNCT bracis-33592 38 37 respectively respectively ADV bracis-33592 38 38 , , PUNCT bracis-33592 38 39 800 800 NUM bracis-33592 38 40 and and CCONJ bracis-33592 38 41 560 560 NUM bracis-33592 38 42 points point NOUN bracis-33592 38 43 to to ADP bracis-33592 38 44 a a DET bracis-33592 38 45 universal universal ADJ bracis-33592 38 46 attack attack NOUN bracis-33592 38 47 ( ( PUNCT bracis-33592 38 48 i.e. i.e. X bracis-33592 38 49 , , PUNCT bracis-33592 38 50 a a DET bracis-33592 38 51 fabricated fabricate VERB bracis-33592 38 52 essay essay NOUN bracis-33592 38 53 ) ) PUNCT bracis-33592 38 54 based base VERB bracis-33592 38 55 on on ADP bracis-33592 38 56 repeating repeat VERB bracis-33592 38 57 a a DET bracis-33592 38 58 generic generic ADJ bracis-33592 38 59 conclusion conclusion NOUN bracis-33592 38 60 employing employ VERB bracis-33592 38 61 many many ADJ bracis-33592 38 62 adverbs adverb NOUN bracis-33592 38 63 and and CCONJ bracis-33592 38 64 adjectives adjective NOUN bracis-33592 38 65 . . PUNCT bracis-33592 39 1 the the DET bracis-33592 39 2 gemini gemini NOUN bracis-33592 39 3 - - PUNCT bracis-33592 39 4 based base VERB bracis-33592 39 5 model model NOUN bracis-33592 39 6 showed show VERB bracis-33592 39 7 resilience resilience NOUN bracis-33592 39 8 against against ADP bracis-33592 39 9 attacks attack NOUN bracis-33592 39 10 based base VERB bracis-33592 39 11 on on ADP bracis-33592 39 12 repetition repetition NOUN bracis-33592 39 13 , , PUNCT bracis-33592 39 14 but but CCONJ bracis-33592 39 15 for for ADP bracis-33592 39 16 instance instance NOUN bracis-33592 39 17 assigned assign VERB bracis-33592 39 18 a a DET bracis-33592 39 19 relatively relatively ADV bracis-33592 39 20 high high ADJ bracis-33592 39 21 score score NOUN bracis-33592 39 22 of of ADP bracis-33592 39 23 640 640 NUM bracis-33592 39 24 points point NOUN bracis-33592 39 25 to to ADP bracis-33592 39 26 an an DET bracis-33592 39 27 essay essay NOUN bracis-33592 39 28 consisting consist VERB bracis-33592 39 29 of of ADP bracis-33592 39 30 only only ADV bracis-33592 39 31 a a DET bracis-33592 39 32 small small ADJ bracis-33592 39 33 list list NOUN bracis-33592 39 34 of of ADP bracis-33592 39 35 adjectives adjective NOUN bracis-33592 39 36 . . PUNCT bracis-33592 40 1 in in ADP bracis-33592 40 2 the the DET bracis-33592 40 3 rest rest NOUN bracis-33592 40 4 of of ADP bracis-33592 40 5 the the DET bracis-33592 40 6 document document NOUN bracis-33592 40 7 , , PUNCT bracis-33592 40 8 we we PRON bracis-33592 40 9 present present VERB bracis-33592 40 10 related related ADJ bracis-33592 40 11 work work NOUN bracis-33592 40 12 in in ADP bracis-33592 40 13 sect sect NOUN bracis-33592 40 14 . . PUNCT bracis-33592 40 15     SPACE bracis-33592 41 1 2 2 NUM bracis-33592 41 2 , , PUNCT bracis-33592 41 3 discuss discuss VERB bracis-33592 41 4 our our PRON bracis-33592 41 5 methodology methodology NOUN bracis-33592 41 6 in in ADP bracis-33592 41 7 sect sect NOUN bracis-33592 41 8 . . PUNCT bracis-33592 41 9     SPACE bracis-33592 41 10 3 3 NUM bracis-33592 41 11 , , PUNCT bracis-33592 41 12 and and CCONJ bracis-33592 41 13 show show VERB bracis-33592 41 14 the the DET bracis-33592 41 15 experimental experimental ADJ bracis-33592 41 16 results result NOUN bracis-33592 41 17 in in ADP bracis-33592 41 18 sect sect NOUN bracis-33592 41 19 . . PUNCT bracis-33592 41 20     SPACE bracis-33592 42 1 4 4 X bracis-33592 42 2 . . PUNCT bracis-33592 43 1 we we PRON bracis-33592 43 2 conclude conclude VERB bracis-33592 43 3 with with ADP bracis-33592 43 4 a a DET bracis-33592 43 5 summary summary NOUN bracis-33592 43 6 and and CCONJ bracis-33592 43 7 final final ADJ bracis-33592 43 8 remarks remark NOUN bracis-33592 43 9 in in ADP bracis-33592 43 10 sect sect NOUN bracis-33592 43 11 . . PUNCT bracis-33592 43 12     SPACE bracis-33592 44 1 5 5 NUM bracis-33592 44 2 . . SYM bracis-33592 44 3 2 2 NUM bracis-33592 44 4 related related ADJ bracis-33592 44 5 work work NOUN bracis-33592 44 6 the the DET bracis-33592 44 7 evolution evolution NOUN bracis-33592 44 8 of of ADP bracis-33592 44 9 aes aes PROPN bracis-33592 44 10 systems system NOUN bracis-33592 44 11 mimics mimic VERB bracis-33592 44 12 that that PRON bracis-33592 44 13 of of ADP bracis-33592 44 14 other other ADJ bracis-33592 44 15 nlp nlp NOUN bracis-33592 44 16 tasks task NOUN bracis-33592 44 17 . . PUNCT bracis-33592 45 1 the the DET bracis-33592 45 2 initial initial ADJ bracis-33592 45 3 systems system NOUN bracis-33592 45 4 were be AUX bracis-33592 45 5 based base VERB bracis-33592 45 6 on on ADP bracis-33592 45 7 handcrafted handcrafted ADJ bracis-33592 45 8 features feature NOUN bracis-33592 45 9 , , PUNCT bracis-33592 45 10 which which PRON bracis-33592 45 11 had have AUX bracis-33592 45 12 limited limit VERB bracis-33592 45 13 predictive predictive ADJ bracis-33592 45 14 power power NOUN bracis-33592 45 15 and and CCONJ bracis-33592 45 16 require require VERB bracis-33592 45 17 extensive extensive ADJ bracis-33592 45 18 expertise expertise NOUN bracis-33592 45 19 to to PART bracis-33592 45 20 be be AUX bracis-33592 45 21 designed design VERB bracis-33592 45 22 [ [ PUNCT bracis-33592 45 23 32 32 NUM bracis-33592 45 24 ] ] PUNCT bracis-33592 45 25 . . PUNCT bracis-33592 46 1 those those DET bracis-33592 46 2 systems system NOUN bracis-33592 46 3 were be AUX bracis-33592 46 4 incrementally incrementally ADV bracis-33592 46 5 superseded supersede VERB bracis-33592 46 6 by by ADP bracis-33592 46 7 systems system NOUN bracis-33592 46 8 that that PRON bracis-33592 46 9 learned learn VERB bracis-33592 46 10 representations representation NOUN bracis-33592 46 11 directly directly ADV bracis-33592 46 12 from from ADP bracis-33592 46 13 data datum NOUN bracis-33592 46 14 . . PUNCT bracis-33592 47 1 first first ADV bracis-33592 47 2 , , PUNCT bracis-33592 47 3 by by ADP bracis-33592 47 4 approaches approach NOUN bracis-33592 47 5 that that PRON bracis-33592 47 6 extracted extract VERB bracis-33592 47 7 shallow shallow ADJ bracis-33592 47 8 representations representation NOUN bracis-33592 47 9 such such ADJ bracis-33592 47 10 as as ADP bracis-33592 47 11 word word NOUN bracis-33592 47 12 embeddings embedding NOUN bracis-33592 47 13 [ [ X bracis-33592 47 14 37 37 NUM bracis-33592 47 15 ] ] PUNCT bracis-33592 47 16 . . PUNCT bracis-33592 48 1 more more ADV bracis-33592 48 2 recently recently ADV bracis-33592 48 3 , , PUNCT bracis-33592 48 4 by by ADP bracis-33592 48 5 approaches approach NOUN bracis-33592 48 6 that that PRON bracis-33592 48 7 learn learn VERB bracis-33592 48 8 representation representation NOUN bracis-33592 48 9 by by ADP bracis-33592 48 10 deep deep ADJ bracis-33592 48 11 models model NOUN bracis-33592 48 12 such such ADJ bracis-33592 48 13 as as ADP bracis-33592 48 14 transformer transformer ADJ bracis-33592 48 15 neural neural ADJ bracis-33592 48 16 networks network NOUN bracis-33592 48 17 [ [ X bracis-33592 48 18 27 27 NUM bracis-33592 48 19 ] ] PUNCT bracis-33592 48 20 . . PUNCT bracis-33592 49 1 several several ADJ bracis-33592 49 2 aes aes PROPN bracis-33592 49 3 systems system NOUN bracis-33592 49 4 for for ADP bracis-33592 49 5 ( ( PUNCT bracis-33592 49 6 brazilian brazilian ADJ bracis-33592 49 7 ) ) PUNCT bracis-33592 49 8 portuguese portuguese NOUN bracis-33592 49 9 have have AUX bracis-33592 49 10 been be AUX bracis-33592 49 11 proposed propose VERB bracis-33592 49 12 , , PUNCT bracis-33592 49 13 mostly mostly ADV bracis-33592 49 14 focusing focus VERB bracis-33592 49 15 on on ADP bracis-33592 49 16 grading grade VERB bracis-33592 49 17 essays essay NOUN bracis-33592 49 18 in in ADP bracis-33592 49 19 the the DET bracis-33592 49 20 format format NOUN bracis-33592 49 21 required require VERB bracis-33592 49 22 for for ADP bracis-33592 49 23 the the DET bracis-33592 49 24 brazilian brazilian ADJ bracis-33592 49 25 national national ADJ bracis-33592 49 26 entrance entrance NOUN bracis-33592 49 27 exam exam NOUN bracis-33592 49 28 ( ( PUNCT bracis-33592 49 29 enem enem PROPN bracis-33592 49 30 ) ) PUNCT bracis-33592 50 1 [ [ X bracis-33592 50 2 2 2 NUM bracis-33592 50 3 , , PUNCT bracis-33592 50 4 4 4 NUM bracis-33592 50 5 , , PUNCT bracis-33592 50 6 12 12 NUM bracis-33592 50 7 , , PUNCT bracis-33592 50 8 15 15 NUM bracis-33592 50 9 , , PUNCT bracis-33592 50 10 28 28 NUM bracis-33592 50 11 , , PUNCT bracis-33592 50 12 34 34 NUM bracis-33592 50 13 ] ] PUNCT bracis-33592 50 14 . . PUNCT bracis-33592 51 1 an an DET bracis-33592 51 2 early early ADJ bracis-33592 51 3 work work NOUN bracis-33592 51 4 used use VERB bracis-33592 51 5 words word NOUN bracis-33592 51 6 as as ADP bracis-33592 51 7 features feature NOUN bracis-33592 51 8 of of ADP bracis-33592 51 9 a a DET bracis-33592 51 10 naïve naïve ADJ bracis-33592 51 11 bayes bayes NOUN bracis-33592 51 12 [ [ X bracis-33592 51 13 4 4 NUM bracis-33592 51 14 ] ] PUNCT bracis-33592 51 15 . . PUNCT bracis-33592 52 1 subsequent subsequent ADJ bracis-33592 52 2 approaches approach NOUN bracis-33592 52 3 ranged range VERB bracis-33592 52 4 from from ADP bracis-33592 52 5 defining define VERB bracis-33592 52 6 handcrafted handcraft VERB bracis-33592 52 7 features feature NOUN bracis-33592 52 8 to to ADP bracis-33592 52 9 combining combine VERB bracis-33592 52 10 word word NOUN bracis-33592 52 11 - - PUNCT bracis-33592 52 12 embedding embed VERB bracis-33592 52 13 with with ADP bracis-33592 52 14 recurrent recurrent ADJ bracis-33592 52 15 neural neural ADJ bracis-33592 52 16 networks network NOUN bracis-33592 52 17 [ [ X bracis-33592 52 18 2 2 NUM bracis-33592 52 19 , , PUNCT bracis-33592 52 20 15 15 NUM bracis-33592 52 21 , , PUNCT bracis-33592 52 22 29 29 NUM bracis-33592 52 23 ] ] PUNCT bracis-33592 52 24 . . PUNCT bracis-33592 53 1 our our PRON bracis-33592 53 2 study study NOUN bracis-33592 53 3 intersects intersect NOUN bracis-33592 53 4 with with ADP bracis-33592 53 5 the the DET bracis-33592 53 6 existing exist VERB bracis-33592 53 7 literature literature NOUN bracis-33592 53 8 by by ADP bracis-33592 53 9 evaluating evaluate VERB bracis-33592 53 10 the the DET bracis-33592 53 11 performance performance NOUN bracis-33592 53 12 of of ADP bracis-33592 53 13 previously previously ADV bracis-33592 53 14 unused unused ADJ bracis-33592 53 15 models model NOUN bracis-33592 53 16 , , PUNCT bracis-33592 53 17 specifically specifically ADV bracis-33592 53 18 the the DET bracis-33592 53 19 phi-3 phi-3 ADJ bracis-33592 53 20 and and CCONJ bracis-33592 53 21 gemini gemini NOUN bracis-33592 53 22 - - PUNCT bracis-33592 53 23 based base VERB bracis-33592 53 24 graders grader NOUN bracis-33592 53 25 . . PUNCT bracis-33592 54 1 our our PRON bracis-33592 54 2 goal goal NOUN bracis-33592 54 3 is be AUX bracis-33592 54 4 not not PART bracis-33592 54 5 pursuing pursue VERB bracis-33592 54 6 high high ADJ bracis-33592 54 7 agreement agreement NOUN bracis-33592 54 8 , , PUNCT bracis-33592 54 9 but but CCONJ bracis-33592 54 10 presenting present VERB bracis-33592 54 11 how how SCONJ bracis-33592 54 12 susceptible susceptible ADJ bracis-33592 54 13 these these DET bracis-33592 54 14 novel novel ADJ bracis-33592 54 15 models model NOUN bracis-33592 54 16 are be AUX bracis-33592 54 17 to to ADP bracis-33592 54 18 adversarial adversarial ADJ bracis-33592 54 19 attacks attack NOUN bracis-33592 54 20 . . PUNCT bracis-33592 55 1 a a DET bracis-33592 55 2 recent recent ADJ bracis-33592 55 3 work work NOUN bracis-33592 55 4 compared compare VERB bracis-33592 55 5 several several ADJ bracis-33592 55 6 ( ( PUNCT bracis-33592 55 7 brazilian brazilian ADJ bracis-33592 55 8 ) ) PUNCT bracis-33592 55 9 bert bert NOUN bracis-33592 55 10 - - PUNCT bracis-33592 55 11 based base VERB bracis-33592 55 12 graders grader NOUN bracis-33592 55 13 [ [ X bracis-33592 55 14 34 34 NUM bracis-33592 55 15 ] ] PUNCT bracis-33592 55 16 . . PUNCT bracis-33592 56 1 we we PRON bracis-33592 56 2 incorporate incorporate VERB bracis-33592 56 3 these these DET bracis-33592 56 4 models model NOUN bracis-33592 56 5 into into ADP bracis-33592 56 6 our our PRON bracis-33592 56 7 study study NOUN bracis-33592 56 8 , , PUNCT bracis-33592 56 9 alongside alongside ADP bracis-33592 56 10 new new ADJ bracis-33592 56 11 state state NOUN bracis-33592 56 12 - - PUNCT bracis-33592 56 13 of of ADP bracis-33592 56 14 - - PUNCT bracis-33592 56 15 the the DET bracis-33592 56 16 - - PUNCT bracis-33592 56 17 art art NOUN bracis-33592 56 18 models model NOUN bracis-33592 56 19 , , PUNCT bracis-33592 56 20 for for ADP bracis-33592 56 21 comparative comparative ADJ bracis-33592 56 22 analysis analysis NOUN bracis-33592 56 23 . . PUNCT bracis-33592 57 1 different different ADJ bracis-33592 57 2 aes aes PROPN bracis-33592 57 3 systems system NOUN bracis-33592 57 4 were be AUX bracis-33592 57 5 assessed assess VERB bracis-33592 57 6 based base VERB bracis-33592 57 7 not not PART bracis-33592 57 8 on on ADP bracis-33592 57 9 their their PRON bracis-33592 57 10 qwk qwk NOUN bracis-33592 57 11 performance performance NOUN bracis-33592 57 12 , , PUNCT bracis-33592 57 13 but but CCONJ bracis-33592 57 14 rather rather ADV bracis-33592 57 15 on on ADP bracis-33592 57 16 their their PRON bracis-33592 57 17 oversensibility oversensibility NOUN bracis-33592 57 18 and and CCONJ bracis-33592 57 19 overstability overstability NOUN bracis-33592 58 1 [ [ X bracis-33592 58 2 21 21 NUM bracis-33592 58 3 ] ] PUNCT bracis-33592 58 4 . . PUNCT bracis-33592 59 1 the the DET bracis-33592 59 2 authors author NOUN bracis-33592 59 3 investigated investigate VERB bracis-33592 59 4 how how SCONJ bracis-33592 59 5 perturbations perturbation NOUN bracis-33592 59 6 in in ADP bracis-33592 59 7 the the DET bracis-33592 59 8 input input NOUN bracis-33592 59 9 affected affect VERB bracis-33592 59 10 the the DET bracis-33592 59 11 models model NOUN bracis-33592 59 12 ’ ' PUNCT bracis-33592 59 13 output output NOUN bracis-33592 59 14 . . PUNCT bracis-33592 60 1 the the DET bracis-33592 60 2 asap asap PROPN bracis-33592 60 3 challenge challenge PROPN bracis-33592 60 4 dataset dataset VERB bracis-33592 60 5 [ [ X bracis-33592 60 6 18 18 NUM bracis-33592 60 7 ] ] PUNCT bracis-33592 60 8 was be AUX bracis-33592 60 9 used use VERB bracis-33592 60 10 to to PART bracis-33592 60 11 test test VERB bracis-33592 60 12 five five NUM bracis-33592 60 13 models model NOUN bracis-33592 60 14 , , PUNCT bracis-33592 60 15 ranging range VERB bracis-33592 60 16 from from ADP bracis-33592 60 17 a a DET bracis-33592 60 18 feature feature NOUN bracis-33592 60 19 - - PUNCT bracis-33592 60 20 based base VERB bracis-33592 60 21 to to ADP bracis-33592 60 22 a a DET bracis-33592 60 23 bert bert NOUN bracis-33592 60 24 - - PUNCT bracis-33592 60 25 based base VERB bracis-33592 60 26 grader grader NOUN bracis-33592 60 27 . . PUNCT bracis-33592 61 1 they they PRON bracis-33592 61 2 found find VERB bracis-33592 61 3 that that SCONJ bracis-33592 61 4 , , PUNCT bracis-33592 61 5 except except SCONJ bracis-33592 61 6 for for ADP bracis-33592 61 7 the the DET bracis-33592 61 8 feature feature NOUN bracis-33592 61 9 - - PUNCT bracis-33592 61 10 based base VERB bracis-33592 61 11 and and CCONJ bracis-33592 61 12 one one NUM bracis-33592 61 13 of of ADP bracis-33592 61 14 the the DET bracis-33592 61 15 word word NOUN bracis-33592 61 16 - - PUNCT bracis-33592 61 17 embedding embed VERB bracis-33592 61 18 - - PUNCT bracis-33592 61 19 based base VERB bracis-33592 61 20 , , PUNCT bracis-33592 61 21 all all DET bracis-33592 61 22 models model NOUN bracis-33592 61 23 were be AUX bracis-33592 61 24 overstable overstable ADJ bracis-33592 61 25 . . PUNCT bracis-33592 62 1 moreover moreover ADV bracis-33592 62 2 , , PUNCT bracis-33592 62 3 increasing increase VERB bracis-33592 62 4 the the DET bracis-33592 62 5 disturbance disturbance NOUN bracis-33592 62 6 percentage percentage NOUN bracis-33592 62 7 lead lead NOUN bracis-33592 62 8 to to ADP bracis-33592 62 9 increased increase VERB bracis-33592 62 10 standard standard ADJ bracis-33592 62 11 deviation deviation NOUN bracis-33592 62 12 in in ADP bracis-33592 62 13 the the DET bracis-33592 62 14 grades grade NOUN bracis-33592 62 15 . . PUNCT bracis-33592 63 1 their their PRON bracis-33592 63 2 work work NOUN bracis-33592 63 3 investigated investigate VERB bracis-33592 63 4 whether whether SCONJ bracis-33592 63 5 adding add VERB bracis-33592 63 6 , , PUNCT bracis-33592 63 7 removing remove VERB bracis-33592 63 8 or or CCONJ bracis-33592 63 9 changing change VERB bracis-33592 63 10 content content NOUN bracis-33592 63 11 of of ADP bracis-33592 63 12 an an DET bracis-33592 63 13 essay essay NOUN bracis-33592 63 14 caused cause VERB bracis-33592 63 15 it it PRON bracis-33592 63 16 to to PART bracis-33592 63 17 be be AUX bracis-33592 63 18 graded grade VERB bracis-33592 63 19 differently differently ADV bracis-33592 63 20 . . PUNCT bracis-33592 64 1 this this PRON bracis-33592 64 2 can can AUX bracis-33592 64 3 be be AUX bracis-33592 64 4 seen see VERB bracis-33592 64 5 as as ADP bracis-33592 64 6 students student NOUN bracis-33592 64 7 rewriting rewrite VERB bracis-33592 64 8 their their PRON bracis-33592 64 9 essays essay NOUN bracis-33592 64 10 after after ADP bracis-33592 64 11 grading grade VERB bracis-33592 64 12 . . PUNCT bracis-33592 65 1 our our PRON bracis-33592 65 2 scope scope NOUN bracis-33592 65 3 is be AUX bracis-33592 65 4 different different ADJ bracis-33592 65 5 , , PUNCT bracis-33592 65 6 for for SCONJ bracis-33592 65 7 they they PRON bracis-33592 65 8 do do AUX bracis-33592 65 9 not not PART bracis-33592 65 10 consider consider VERB bracis-33592 65 11 universal universal ADJ bracis-33592 65 12 attacks attack NOUN bracis-33592 65 13 , , PUNCT bracis-33592 65 14 and and CCONJ bracis-33592 65 15 we we PRON bracis-33592 65 16 consider consider VERB bracis-33592 65 17 ill ill ADV bracis-33592 65 18 - - PUNCT bracis-33592 65 19 intended intend VERB bracis-33592 65 20 attacks attack NOUN bracis-33592 65 21 that that PRON bracis-33592 65 22 could could AUX bracis-33592 65 23 be be AUX bracis-33592 65 24 done do VERB bracis-33592 65 25 by by ADP bracis-33592 65 26 students student NOUN bracis-33592 65 27 with with ADP bracis-33592 65 28 minimal minimal ADJ bracis-33592 65 29 effort effort NOUN bracis-33592 65 30 . . PUNCT bracis-33592 66 1 without without ADP bracis-33592 66 2 any any DET bracis-33592 66 3 scenario scenario NOUN bracis-33592 66 4 restrictions restriction NOUN bracis-33592 66 5 , , PUNCT bracis-33592 66 6 there there PRON bracis-33592 66 7 are be VERB bracis-33592 66 8 possibly possibly ADV bracis-33592 66 9 infinite infinite ADJ bracis-33592 66 10 adversarial adversarial ADJ bracis-33592 66 11 attack attack NOUN bracis-33592 66 12 setups setup NOUN bracis-33592 66 13 . . PUNCT bracis-33592 67 1 we we PRON bracis-33592 67 2 rule rule VERB bracis-33592 67 3 out out ADP bracis-33592 67 4 possible possible ADJ bracis-33592 67 5 attacks attack NOUN bracis-33592 67 6 by by ADP bracis-33592 67 7 restricting restrict VERB bracis-33592 67 8 ourselves ourselves PRON bracis-33592 67 9 to to ADP bracis-33592 67 10 a a DET bracis-33592 67 11 very very ADV bracis-33592 67 12 specific specific ADJ bracis-33592 67 13 scenario scenario NOUN bracis-33592 67 14 . . PUNCT bracis-33592 68 1 firstly firstly ADV bracis-33592 68 2 , , PUNCT bracis-33592 68 3 students student NOUN bracis-33592 68 4 do do AUX bracis-33592 68 5 not not PART bracis-33592 68 6 have have VERB bracis-33592 68 7 access access NOUN bracis-33592 68 8 to to ADP bracis-33592 68 9 any any DET bracis-33592 68 10 parts part NOUN bracis-33592 68 11 of of ADP bracis-33592 68 12 the the DET bracis-33592 68 13 model model NOUN bracis-33592 68 14 , , PUNCT bracis-33592 68 15 so so SCONJ bracis-33592 68 16 they they PRON bracis-33592 68 17 are be AUX bracis-33592 68 18 not not PART bracis-33592 68 19 allowed allow VERB bracis-33592 68 20 to to PART bracis-33592 68 21 attack attack VERB bracis-33592 68 22 the the DET bracis-33592 68 23 embedding embed VERB bracis-33592 68 24 space space NOUN bracis-33592 68 25 [ [ X bracis-33592 68 26 16 16 NUM bracis-33592 68 27 ] ] PUNCT bracis-33592 68 28 , , PUNCT bracis-33592 68 29 the the DET bracis-33592 68 30 gradient gradient NOUN bracis-33592 69 1 [ [ X bracis-33592 69 2 11 11 NUM bracis-33592 69 3 ] ] PUNCT bracis-33592 69 4 , , PUNCT bracis-33592 69 5 nor nor CCONJ bracis-33592 69 6 the the DET bracis-33592 69 7 calculated calculate VERB bracis-33592 69 8 features feature NOUN bracis-33592 69 9 , , PUNCT bracis-33592 69 10 to to PART bracis-33592 69 11 name name VERB bracis-33592 69 12 a a DET bracis-33592 69 13 few few ADJ bracis-33592 69 14 . . PUNCT bracis-33592 70 1 secondly secondly ADV bracis-33592 70 2 , , PUNCT bracis-33592 70 3 they they PRON bracis-33592 70 4 do do AUX bracis-33592 70 5 not not PART bracis-33592 70 6 know know VERB bracis-33592 70 7 the the DET bracis-33592 70 8 underlying underlie VERB bracis-33592 70 9 model model NOUN bracis-33592 70 10 , , PUNCT bracis-33592 70 11 which which PRON bracis-33592 70 12 prevents prevent VERB bracis-33592 70 13 them they PRON bracis-33592 70 14 from from ADP bracis-33592 70 15 using use VERB bracis-33592 70 16 the the DET bracis-33592 70 17 pre pre ADJ bracis-33592 70 18 - - ADJ bracis-33592 70 19 trained train VERB bracis-33592 70 20 version version NOUN bracis-33592 70 21 of of ADP bracis-33592 70 22 the the DET bracis-33592 70 23 model model NOUN bracis-33592 70 24 to to PART bracis-33592 70 25 generate generate VERB bracis-33592 70 26 attacks attack NOUN bracis-33592 70 27 , , PUNCT bracis-33592 70 28 such such ADJ bracis-33592 70 29 as as ADP bracis-33592 70 30 [ [ X bracis-33592 70 31 25 25 NUM bracis-33592 70 32 ] ] PUNCT bracis-33592 70 33 . . PUNCT bracis-33592 71 1 lastly lastly ADV bracis-33592 71 2 , , PUNCT bracis-33592 71 3 the the DET bracis-33592 71 4 students student NOUN bracis-33592 71 5 are be AUX bracis-33592 71 6 not not PART bracis-33592 71 7 too too ADV bracis-33592 71 8 invested invest VERB bracis-33592 71 9 in in ADP bracis-33592 71 10 breaching breach VERB bracis-33592 71 11 the the DET bracis-33592 71 12 system system NOUN bracis-33592 71 13 , , PUNCT bracis-33592 71 14 so so SCONJ bracis-33592 71 15 attacks attack NOUN bracis-33592 71 16 such such ADJ bracis-33592 71 17 as as ADP bracis-33592 71 18 [ [ X bracis-33592 71 19 14 14 NUM bracis-33592 71 20 , , PUNCT bracis-33592 71 21 40 40 NUM bracis-33592 71 22 ] ] PUNCT bracis-33592 71 23 are be AUX bracis-33592 71 24 not not PART bracis-33592 71 25 applicable applicable ADJ bracis-33592 71 26 . . PUNCT bracis-33592 72 1 the the DET bracis-33592 72 2 first first ADJ bracis-33592 72 3 requires require VERB bracis-33592 72 4 ( ( PUNCT bracis-33592 72 5 possibly possibly ADV bracis-33592 72 6 ) ) PUNCT bracis-33592 72 7 testing test VERB bracis-33592 72 8 combinations combination NOUN bracis-33592 72 9 of of ADP bracis-33592 72 10 the the DET bracis-33592 72 11 whole whole ADJ bracis-33592 72 12 vocabulary vocabulary NOUN bracis-33592 72 13 used use VERB bracis-33592 72 14 by by ADP bracis-33592 72 15 the the DET bracis-33592 72 16 tokenizer tokenizer NOUN bracis-33592 72 17 of of ADP bracis-33592 72 18 the the DET bracis-33592 72 19 underlying underlie VERB bracis-33592 72 20 model model NOUN bracis-33592 72 21 . . PUNCT bracis-33592 73 1 this this PRON bracis-33592 73 2 is be AUX bracis-33592 73 3 ruled rule VERB bracis-33592 73 4 out out ADP bracis-33592 73 5 of of ADP bracis-33592 73 6 our our PRON bracis-33592 73 7 scenario scenario NOUN bracis-33592 73 8 as as SCONJ bracis-33592 73 9 they they PRON bracis-33592 73 10 do do AUX bracis-33592 73 11 not not PART bracis-33592 73 12 have have VERB bracis-33592 73 13 access access NOUN bracis-33592 73 14 to to ADP bracis-33592 73 15 this this DET bracis-33592 73 16 information information NOUN bracis-33592 73 17 , , PUNCT bracis-33592 73 18 some some DET bracis-33592 73 19 models model NOUN bracis-33592 73 20 do do AUX bracis-33592 73 21 not not PART bracis-33592 73 22 have have VERB bracis-33592 73 23 a a DET bracis-33592 73 24 vocabulary vocabulary NOUN bracis-33592 73 25 , , PUNCT bracis-33592 73 26 and and CCONJ bracis-33592 73 27 gemini gemini PROPN bracis-33592 73 28 ’s ’s PART bracis-33592 73 29 vocabulary vocabulary NOUN bracis-33592 73 30 has have VERB bracis-33592 73 31 256k 256k VERB bracis-33592 73 32 tokens token NOUN bracis-33592 73 33 , , PUNCT bracis-33592 73 34 so so SCONJ bracis-33592 73 35 it it PRON bracis-33592 73 36 would would AUX bracis-33592 73 37 require require VERB bracis-33592 73 38 at at ADP bracis-33592 73 39 least least ADJ bracis-33592 73 40 this this DET bracis-33592 73 41 many many ADJ bracis-33592 73 42 attacks attack NOUN bracis-33592 73 43 . . PUNCT bracis-33592 74 1 similarly similarly ADV bracis-33592 74 2 , , PUNCT bracis-33592 74 3 the the DET bracis-33592 74 4 second second ADJ bracis-33592 74 5 attack attack NOUN bracis-33592 74 6 requires require VERB bracis-33592 74 7 changing change VERB bracis-33592 74 8 the the DET bracis-33592 74 9 whole whole ADJ bracis-33592 74 10 essay essay NOUN bracis-33592 74 11 to to PART bracis-33592 74 12 have have VERB bracis-33592 74 13 the the DET bracis-33592 74 14 same same ADJ bracis-33592 74 15 word word NOUN bracis-33592 74 16 , , PUNCT bracis-33592 74 17 which which PRON bracis-33592 74 18 results result VERB bracis-33592 74 19 in in ADP bracis-33592 74 20 a a DET bracis-33592 74 21 massive massive ADJ bracis-33592 74 22 number number NOUN bracis-33592 74 23 of of ADP bracis-33592 74 24 needed need VERB bracis-33592 74 25 attacks attack NOUN bracis-33592 74 26 . . PUNCT bracis-33592 75 1 thus thus ADV bracis-33592 75 2 , , PUNCT bracis-33592 75 3 the the DET bracis-33592 75 4 attack attack NOUN bracis-33592 75 5 we we PRON bracis-33592 75 6 devise devise VERB bracis-33592 75 7 here here ADV bracis-33592 75 8 is be AUX bracis-33592 75 9 much much ADV bracis-33592 75 10 closer close ADJ bracis-33592 75 11 to to ADP bracis-33592 75 12 what what PRON bracis-33592 75 13 one one PRON bracis-33592 75 14 could could AUX bracis-33592 75 15 expect expect VERB bracis-33592 75 16 to to PART bracis-33592 75 17 happen happen VERB bracis-33592 75 18 at at ADP bracis-33592 75 19 a a DET bracis-33592 75 20 classroom classroom NOUN bracis-33592 75 21 level level NOUN bracis-33592 75 22 and and CCONJ bracis-33592 75 23 has have AUX bracis-33592 75 24 not not PART bracis-33592 75 25 been be AUX bracis-33592 75 26 used use VERB bracis-33592 75 27 yet yet ADV bracis-33592 75 28 in in ADP bracis-33592 75 29 the the DET bracis-33592 75 30 literature literature NOUN bracis-33592 75 31 . . PUNCT bracis-33592 76 1 the the DET bracis-33592 76 2 task task NOUN bracis-33592 76 3 of of ADP bracis-33592 76 4 automatic automatic ADJ bracis-33592 76 5 short short ADJ bracis-33592 76 6 answer answer NOUN bracis-33592 76 7 grading grade VERB bracis-33592 76 8 ( ( PUNCT bracis-33592 76 9 asag asag ADJ bracis-33592 76 10 ) ) PUNCT bracis-33592 76 11 is be AUX bracis-33592 76 12 very very ADV bracis-33592 76 13 similar similar ADJ bracis-33592 76 14 to to ADP bracis-33592 76 15 aes aes PROPN bracis-33592 76 16 , , PUNCT bracis-33592 76 17 the the DET bracis-33592 76 18 difference difference NOUN bracis-33592 76 19 is be AUX bracis-33592 76 20 that that SCONJ bracis-33592 76 21 , , PUNCT bracis-33592 76 22 instead instead ADV bracis-33592 76 23 of of ADP bracis-33592 76 24 ( ( PUNCT bracis-33592 76 25 long long ADJ bracis-33592 76 26 ) ) PUNCT bracis-33592 76 27 essays essay NOUN bracis-33592 76 28 , , PUNCT bracis-33592 76 29 its its PRON bracis-33592 76 30 input input NOUN bracis-33592 76 31 is be AUX bracis-33592 76 32 a a DET bracis-33592 76 33 relatively relatively ADV bracis-33592 76 34 small small ADJ bracis-33592 76 35 text text NOUN bracis-33592 76 36 . . PUNCT bracis-33592 77 1 adjectives adjective NOUN bracis-33592 77 2 and and CCONJ bracis-33592 77 3 adverbs adverb NOUN bracis-33592 77 4 have have AUX bracis-33592 77 5 already already ADV bracis-33592 77 6 shown show VERB bracis-33592 77 7 to to PART bracis-33592 77 8 be be AUX bracis-33592 77 9 useful useful ADJ bracis-33592 77 10 to to PART bracis-33592 77 11 cheat cheat VERB bracis-33592 77 12 asag asag ADJ bracis-33592 77 13 systems system NOUN bracis-33592 77 14 based base VERB bracis-33592 77 15 on on ADP bracis-33592 77 16 transformers transformer NOUN bracis-33592 77 17 , , PUNCT bracis-33592 77 18 such such ADJ bracis-33592 77 19 as as ADP bracis-33592 77 20 bert bert PROPN bracis-33592 77 21 and and CCONJ bracis-33592 77 22 t5 t5 PROPN bracis-33592 78 1 [ [ X bracis-33592 78 2 13 13 NUM bracis-33592 78 3 ] ] PUNCT bracis-33592 78 4 . . PUNCT bracis-33592 79 1 the the DET bracis-33592 79 2 scenario scenario NOUN bracis-33592 79 3 of of ADP bracis-33592 79 4 their their PRON bracis-33592 79 5 study study NOUN bracis-33592 79 6 is be AUX bracis-33592 79 7 similar similar ADJ bracis-33592 79 8 to to ADP bracis-33592 79 9 ours ours PRON bracis-33592 79 10 . . PUNCT bracis-33592 80 1 the the DET bracis-33592 80 2 differences difference NOUN bracis-33592 80 3 are be AUX bracis-33592 80 4 : : PUNCT bracis-33592 80 5 they they PRON bracis-33592 80 6 are be AUX bracis-33592 80 7 concerned concern VERB bracis-33592 80 8 with with ADP bracis-33592 80 9 generating generate VERB bracis-33592 80 10 answers answer NOUN bracis-33592 80 11 that that PRON bracis-33592 80 12 can can AUX bracis-33592 80 13 not not PART bracis-33592 80 14 be be AUX bracis-33592 80 15 identified identify VERB bracis-33592 80 16 by by ADP bracis-33592 80 17 humans human NOUN bracis-33592 80 18 , , PUNCT bracis-33592 80 19 and and CCONJ bracis-33592 80 20 that that SCONJ bracis-33592 80 21 their their PRON bracis-33592 80 22 labels label NOUN bracis-33592 80 23 are be AUX bracis-33592 80 24 : : PUNCT bracis-33592 80 25 correct correct ADJ bracis-33592 80 26 , , PUNCT bracis-33592 80 27 incorrect incorrect ADJ bracis-33592 80 28 or or CCONJ bracis-33592 80 29 contradictory contradictory ADJ bracis-33592 80 30 . . PUNCT bracis-33592 81 1 by by ADP bracis-33592 81 2 inserting insert VERB bracis-33592 81 3 adjectives adjective NOUN bracis-33592 81 4 and and CCONJ bracis-33592 81 5 adverbs adverb NOUN bracis-33592 81 6 in in ADP bracis-33592 81 7 the the DET bracis-33592 81 8 sentences sentence NOUN bracis-33592 81 9 , , PUNCT bracis-33592 81 10 they they PRON bracis-33592 81 11 could could AUX bracis-33592 81 12 fool fool VERB bracis-33592 81 13 the the DET bracis-33592 81 14 models model NOUN bracis-33592 81 15 to to PART bracis-33592 81 16 change change VERB bracis-33592 81 17 the the DET bracis-33592 81 18 answer answer NOUN bracis-33592 81 19 from from ADP bracis-33592 81 20 incorrect incorrect ADJ bracis-33592 81 21 to to PART bracis-33592 81 22 correct correct VERB bracis-33592 81 23 between between ADP bracis-33592 81 24 8 8 NUM bracis-33592 81 25 and and CCONJ bracis-33592 81 26 22 22 NUM bracis-33592 81 27 % % NOUN bracis-33592 81 28 of of ADP bracis-33592 81 29 the the DET bracis-33592 81 30 cases case NOUN bracis-33592 81 31 . . PUNCT bracis-33592 82 1 showing show VERB bracis-33592 82 2 that that SCONJ bracis-33592 82 3 they they PRON bracis-33592 82 4 are be AUX bracis-33592 82 5 strong strong ADJ bracis-33592 82 6 candidates candidate NOUN bracis-33592 82 7 to to PART bracis-33592 82 8 fool fool VERB bracis-33592 82 9 the the DET bracis-33592 82 10 model model NOUN bracis-33592 82 11 . . PUNCT bracis-33592 83 1 additionally additionally ADV bracis-33592 83 2 , , PUNCT bracis-33592 83 3 their their PRON bracis-33592 83 4 investigation investigation NOUN bracis-33592 83 5 found find VERB bracis-33592 83 6 that that SCONJ bracis-33592 83 7 humans human NOUN bracis-33592 83 8 did do AUX bracis-33592 83 9 not not PART bracis-33592 83 10 find find VERB bracis-33592 83 11 the the DET bracis-33592 83 12 adversarial adversarial ADJ bracis-33592 83 13 examples example NOUN bracis-33592 83 14 suspicious suspicious ADJ bracis-33592 83 15 , , PUNCT bracis-33592 83 16 but but CCONJ bracis-33592 83 17 they they PRON bracis-33592 83 18 were be AUX bracis-33592 83 19 not not PART bracis-33592 83 20 natural natural ADJ bracis-33592 83 21 either either ADV bracis-33592 83 22 . . PUNCT bracis-33592 84 1 to to PART bracis-33592 84 2 evaluate evaluate VERB bracis-33592 84 3 the the DET bracis-33592 84 4 robustness robustness NOUN bracis-33592 84 5 of of ADP bracis-33592 84 6 available available ADJ bracis-33592 84 7 models model NOUN bracis-33592 84 8 against against ADP bracis-33592 84 9 such such ADJ bracis-33592 84 10 adversarial adversarial ADJ bracis-33592 84 11 attacks attack NOUN bracis-33592 84 12 , , PUNCT bracis-33592 84 13 we we PRON bracis-33592 84 14 rely rely VERB bracis-33592 84 15 on on ADP bracis-33592 84 16 the the DET bracis-33592 84 17 lmsys lmsys ADJ bracis-33592 84 18 benchmark benchmark NOUN bracis-33592 84 19 [ [ X bracis-33592 84 20 9 9 NUM bracis-33592 84 21 ] ] PUNCT bracis-33592 84 22 to to PART bracis-33592 84 23 choose choose VERB bracis-33592 84 24 candidates candidate NOUN bracis-33592 84 25 . . PUNCT bracis-33592 85 1 lmsys lmsys PROPN bracis-33592 85 2 is be AUX bracis-33592 85 3 a a DET bracis-33592 85 4 comprehensive comprehensive ADJ bracis-33592 85 5 benchmarking benchmarke VERB bracis-33592 85 6 platform platform NOUN bracis-33592 85 7 designed design VERB bracis-33592 85 8 to to PART bracis-33592 85 9 evaluate evaluate VERB bracis-33592 85 10 the the DET bracis-33592 85 11 performance performance NOUN bracis-33592 85 12 of of ADP bracis-33592 85 13 large large ADJ bracis-33592 85 14 language language NOUN bracis-33592 85 15 models model NOUN bracis-33592 85 16 across across ADP bracis-33592 85 17 a a DET bracis-33592 85 18 wide wide ADJ bracis-33592 85 19 array array NOUN bracis-33592 85 20 of of ADP bracis-33592 85 21 tasks task NOUN bracis-33592 85 22 , , PUNCT bracis-33592 85 23 including include VERB bracis-33592 85 24 natural natural ADJ bracis-33592 85 25 language language NOUN bracis-33592 85 26 understanding understanding NOUN bracis-33592 85 27 , , PUNCT bracis-33592 85 28 generation generation NOUN bracis-33592 85 29 , , PUNCT bracis-33592 85 30 and and CCONJ bracis-33592 85 31 interaction interaction NOUN bracis-33592 85 32 . . PUNCT bracis-33592 86 1 by by ADP bracis-33592 86 2 providing provide VERB bracis-33592 86 3 standardized standardized ADJ bracis-33592 86 4 evaluation evaluation NOUN bracis-33592 86 5 metrics metric NOUN bracis-33592 86 6 and and CCONJ bracis-33592 86 7 diverse diverse ADJ bracis-33592 86 8 benchmarks benchmark NOUN bracis-33592 86 9 , , PUNCT bracis-33592 86 10 it it PRON bracis-33592 86 11 enables enable VERB bracis-33592 86 12 a a DET bracis-33592 86 13 consistent consistent ADJ bracis-33592 86 14 comparison comparison NOUN bracis-33592 86 15 of of ADP bracis-33592 86 16 model model NOUN bracis-33592 86 17 capabilities capability NOUN bracis-33592 86 18 , , PUNCT bracis-33592 86 19 thus thus ADV bracis-33592 86 20 serving serve VERB bracis-33592 86 21 as as ADP bracis-33592 86 22 an an DET bracis-33592 86 23 invaluable invaluable ADJ bracis-33592 86 24 resource resource NOUN bracis-33592 86 25 for for ADP bracis-33592 86 26 researchers researcher NOUN bracis-33592 86 27 and and CCONJ bracis-33592 86 28 developers developer NOUN bracis-33592 86 29 . . PUNCT bracis-33592 87 1 as as ADP bracis-33592 87 2 of of ADP bracis-33592 87 3 the the DET bracis-33592 87 4 current current ADJ bracis-33592 87 5 date date NOUN bracis-33592 87 6 , , PUNCT bracis-33592 87 7 the the DET bracis-33592 87 8 10 10 NUM bracis-33592 87 9 best well ADV bracis-33592 87 10 - - PUNCT bracis-33592 87 11 ranked rank VERB bracis-33592 87 12 models model NOUN bracis-33592 87 13 available available ADJ bracis-33592 87 14 there there PRON bracis-33592 87 15 are be VERB bracis-33592 87 16 proprietary proprietary ADJ bracis-33592 87 17 and and CCONJ bracis-33592 87 18 can can AUX bracis-33592 87 19 not not PART bracis-33592 87 20 be be AUX bracis-33592 87 21 easily easily ADV bracis-33592 87 22 fine fine ADV bracis-33592 87 23 - - PUNCT bracis-33592 87 24 tuned tune VERB bracis-33592 87 25 due due ADP bracis-33592 87 26 to to ADP bracis-33592 87 27 their their PRON bracis-33592 87 28 restricted restricted ADJ bracis-33592 87 29 access access NOUN bracis-33592 87 30 . . PUNCT bracis-33592 88 1 among among ADP bracis-33592 88 2 the the DET bracis-33592 88 3 available available ADJ bracis-33592 88 4 ones one NOUN bracis-33592 88 5 , , PUNCT bracis-33592 88 6 nemotron nemotron NOUN bracis-33592 88 7 [ [ X bracis-33592 88 8 31 31 NUM bracis-33592 88 9 ] ] PUNCT bracis-33592 88 10 and and CCONJ bracis-33592 88 11 the the DET bracis-33592 88 12 top top ADV bracis-33592 88 13 - - PUNCT bracis-33592 88 14 performing perform VERB bracis-33592 88 15 models model NOUN bracis-33592 88 16 from from ADP bracis-33592 88 17 the the DET bracis-33592 88 18 llama llama NOUN bracis-33592 88 19 family family NOUN bracis-33592 89 1 [ [ X bracis-33592 89 2 39 39 NUM bracis-33592 89 3 ] ] PUNCT bracis-33592 89 4 are be AUX bracis-33592 89 5 notably notably ADV bracis-33592 89 6 expensive expensive ADJ bracis-33592 89 7 to to PART bracis-33592 89 8 run run VERB bracis-33592 89 9 , , PUNCT bracis-33592 89 10 limiting limit VERB bracis-33592 89 11 their their PRON bracis-33592 89 12 practicality practicality NOUN bracis-33592 89 13 for for ADP bracis-33592 89 14 widespread widespread ADJ bracis-33592 89 15 use use NOUN bracis-33592 89 16 . . PUNCT bracis-33592 90 1 in in ADP bracis-33592 90 2 contrast contrast NOUN bracis-33592 90 3 to to ADP bracis-33592 90 4 this this DET bracis-33592 90 5 trend trend NOUN bracis-33592 90 6 , , PUNCT bracis-33592 90 7 there there PRON bracis-33592 90 8 is be VERB bracis-33592 90 9 a a DET bracis-33592 90 10 growing grow VERB bracis-33592 90 11 interest interest NOUN bracis-33592 90 12 in in ADP bracis-33592 90 13 “ " PUNCT bracis-33592 90 14 small small ADJ bracis-33592 90 15 " " PUNCT bracis-33592 90 16 language language NOUN bracis-33592 90 17 models model NOUN bracis-33592 90 18 , , PUNCT bracis-33592 90 19 such such ADJ bracis-33592 90 20 as as ADP bracis-33592 90 21 llama3 llama3 ADJ bracis-33592 90 22 - - PUNCT bracis-33592 90 23 8b 8b NUM bracis-33592 90 24 and and CCONJ bracis-33592 90 25 the the DET bracis-33592 90 26 phi phi NOUN bracis-33592 90 27 - - PUNCT bracis-33592 90 28 family family NOUN bracis-33592 90 29 [ [ X bracis-33592 90 30 17 17 NUM bracis-33592 90 31 ] ] PUNCT bracis-33592 90 32 . . PUNCT bracis-33592 91 1 the the DET bracis-33592 91 2 phi phi NOUN bracis-33592 91 3 - - PUNCT bracis-33592 91 4 family family NOUN bracis-33592 91 5 has have AUX bracis-33592 91 6 recently recently ADV bracis-33592 91 7 released release VERB bracis-33592 91 8 the the DET bracis-33592 91 9 phi-3 phi-3 ADJ bracis-33592 91 10 models model NOUN bracis-33592 91 11 [ [ X bracis-33592 91 12 1 1 NUM bracis-33592 91 13 ] ] PUNCT bracis-33592 91 14 , , PUNCT bracis-33592 91 15 which which PRON bracis-33592 91 16 include include VERB bracis-33592 91 17 three three NUM bracis-33592 91 18 distinct distinct ADJ bracis-33592 91 19 model model NOUN bracis-33592 91 20 sizes size NOUN bracis-33592 91 21 : : PUNCT bracis-33592 91 22 3.8b 3.8b NUM bracis-33592 91 23 ( ( PUNCT bracis-33592 91 24 phi-3 phi-3 NOUN bracis-33592 91 25 - - PUNCT bracis-33592 91 26 mini mini NOUN bracis-33592 91 27 ) ) PUNCT bracis-33592 91 28 , , PUNCT bracis-33592 91 29 7b 7b NOUN bracis-33592 91 30 ( ( PUNCT bracis-33592 91 31 phi-3 phi-3 NOUN bracis-33592 91 32 - - PUNCT bracis-33592 91 33 small small ADJ bracis-33592 91 34 ) ) PUNCT bracis-33592 91 35 , , PUNCT bracis-33592 91 36 and and CCONJ bracis-33592 91 37 14b 14b NUM bracis-33592 91 38 ( ( PUNCT bracis-33592 91 39 phi-3 phi-3 NOUN bracis-33592 91 40 - - PUNCT bracis-33592 91 41 medium medium NOUN bracis-33592 91 42 ) ) PUNCT bracis-33592 91 43 . . PUNCT bracis-33592 92 1 these these DET bracis-33592 92 2 models model NOUN bracis-33592 92 3 , , PUNCT bracis-33592 92 4 despite despite SCONJ bracis-33592 92 5 their their PRON bracis-33592 92 6 smaller small ADJ bracis-33592 92 7 size size NOUN bracis-33592 92 8 , , PUNCT bracis-33592 92 9 are be AUX bracis-33592 92 10 claimed claim VERB bracis-33592 92 11 by by ADP bracis-33592 92 12 the the DET bracis-33592 92 13 authors author NOUN bracis-33592 92 14 to to PART bracis-33592 92 15 outperform outperform VERB bracis-33592 92 16 some some DET bracis-33592 92 17 proprietary proprietary ADJ bracis-33592 92 18 models model NOUN bracis-33592 92 19 , , PUNCT bracis-33592 92 20 such such ADJ bracis-33592 92 21 as as ADP bracis-33592 92 22 gpt-3.5 gpt-3.5 ADJ bracis-33592 92 23 , , PUNCT bracis-33592 92 24 in in ADP bracis-33592 92 25 various various ADJ bracis-33592 92 26 tasks task NOUN bracis-33592 92 27 . . PUNCT bracis-33592 93 1 we we PRON bracis-33592 93 2 chose choose VERB bracis-33592 93 3 the the DET bracis-33592 93 4 phi-3 phi-3 ADJ bracis-33592 93 5 model model NOUN bracis-33592 93 6 for for ADP bracis-33592 93 7 our our PRON bracis-33592 93 8 research research NOUN bracis-33592 93 9 due due ADP bracis-33592 93 10 to to ADP bracis-33592 93 11 its its PRON bracis-33592 93 12 open open ADJ bracis-33592 93 13 - - PUNCT bracis-33592 93 14 source source NOUN bracis-33592 93 15 nature nature NOUN bracis-33592 93 16 , , PUNCT bracis-33592 93 17 which which PRON bracis-33592 93 18 allows allow VERB bracis-33592 93 19 for for ADP bracis-33592 93 20 easier easy ADJ bracis-33592 93 21 access access NOUN bracis-33592 93 22 and and CCONJ bracis-33592 93 23 adaptability adaptability NOUN bracis-33592 93 24 . . PUNCT bracis-33592 94 1 additionally additionally ADV bracis-33592 94 2 , , PUNCT bracis-33592 94 3 the the DET bracis-33592 94 4 phi-3 phi-3 ADJ bracis-33592 94 5 model model NOUN bracis-33592 94 6 was be AUX bracis-33592 94 7 trained train VERB bracis-33592 94 8 exclusively exclusively ADV bracis-33592 94 9 on on ADP bracis-33592 94 10 english english ADJ bracis-33592 94 11 data datum NOUN bracis-33592 94 12 and and CCONJ bracis-33592 94 13 has have AUX bracis-33592 94 14 not not PART bracis-33592 94 15 been be AUX bracis-33592 94 16 extensively extensively ADV bracis-33592 94 17 tested test VERB bracis-33592 94 18 in in ADP bracis-33592 94 19 other other ADJ bracis-33592 94 20 languages language NOUN bracis-33592 94 21 . . PUNCT bracis-33592 95 1 our our PRON bracis-33592 95 2 study study NOUN bracis-33592 95 3 aims aim VERB bracis-33592 95 4 to to PART bracis-33592 95 5 evaluate evaluate VERB bracis-33592 95 6 the the DET bracis-33592 95 7 performance performance NOUN bracis-33592 95 8 of of ADP bracis-33592 95 9 the the DET bracis-33592 95 10 phi-3 phi-3 ADJ bracis-33592 95 11 model model NOUN bracis-33592 95 12 using use VERB bracis-33592 95 13 portuguese portuguese PROPN bracis-33592 95 14 - - PUNCT bracis-33592 95 15 only only ADV bracis-33592 95 16 data datum NOUN bracis-33592 95 17 , , PUNCT bracis-33592 95 18 contributing contribute VERB bracis-33592 95 19 to to ADP bracis-33592 95 20 the the DET bracis-33592 95 21 understanding understanding NOUN bracis-33592 95 22 of of ADP bracis-33592 95 23 its its PRON bracis-33592 95 24 capabilities capability NOUN bracis-33592 95 25 and and CCONJ bracis-33592 95 26 limitations limitation NOUN bracis-33592 95 27 in in ADP bracis-33592 95 28 multilingual multilingual ADJ bracis-33592 95 29 contexts contexts NOUN bracis-33592 95 30 . . PUNCT bracis-33592 96 1 the the DET bracis-33592 96 2 gpt gpt PROPN bracis-33592 96 3 family family NOUN bracis-33592 96 4 ( ( PUNCT bracis-33592 96 5 davinci davinci PROPN bracis-33592 96 6 , , PUNCT bracis-33592 96 7 3.5 3.5 NUM bracis-33592 96 8 turbo turbo NOUN bracis-33592 96 9 and and CCONJ bracis-33592 96 10 4.0 4.0 NUM bracis-33592 96 11 ) ) PUNCT bracis-33592 96 12 have have AUX bracis-33592 96 13 been be AUX bracis-33592 96 14 tested test VERB bracis-33592 96 15 in in ADP bracis-33592 96 16 aes aes PROPN bracis-33592 96 17 and and CCONJ bracis-33592 96 18 asag asag ADJ bracis-33592 96 19 . . PUNCT bracis-33592 97 1 the the DET bracis-33592 97 2 authors author NOUN bracis-33592 97 3 of of ADP bracis-33592 97 4 [ [ X bracis-33592 97 5 30 30 NUM bracis-33592 97 6 ] ] PUNCT bracis-33592 97 7 showed show VERB bracis-33592 97 8 that that SCONJ bracis-33592 97 9 the the DET bracis-33592 97 10 davinci davinci PROPN bracis-33592 97 11 model model NOUN bracis-33592 97 12 could could AUX bracis-33592 97 13 achieve achieve VERB bracis-33592 97 14 0.38 0.38 NUM bracis-33592 97 15 qwk qwk NOUN bracis-33592 97 16 in in ADP bracis-33592 97 17 the the DET bracis-33592 97 18 toefl toefl NOUN bracis-33592 97 19 dataset dataset NOUN bracis-33592 97 20 , , PUNCT bracis-33592 97 21 while while SCONJ bracis-33592 97 22 linguistic linguistic ADJ bracis-33592 97 23 features feature NOUN bracis-33592 97 24 could could AUX bracis-33592 97 25 achieve achieve VERB bracis-33592 97 26 almost almost ADV bracis-33592 97 27 0.6 0.6 NUM bracis-33592 97 28 qwk qwk NOUN bracis-33592 97 29 . . PUNCT bracis-33592 98 1 gpt gpt PROPN bracis-33592 98 2 3.5 3.5 NUM bracis-33592 98 3 and and CCONJ bracis-33592 98 4 4.0 4.0 NUM bracis-33592 98 5 were be AUX bracis-33592 98 6 used use VERB bracis-33592 98 7 to to PART bracis-33592 98 8 grade grade VERB bracis-33592 98 9 short short ADJ bracis-33592 98 10 answers answer NOUN bracis-33592 98 11 in in ADP bracis-33592 98 12 finnish finnish ADJ bracis-33592 98 13 [ [ X bracis-33592 98 14 7 7 NUM bracis-33592 98 15 ] ] PUNCT bracis-33592 98 16 . . PUNCT bracis-33592 99 1 the the DET bracis-33592 99 2 conclusion conclusion NOUN bracis-33592 99 3 was be AUX bracis-33592 99 4 that that SCONJ bracis-33592 99 5 the the DET bracis-33592 99 6 models model NOUN bracis-33592 99 7 could could AUX bracis-33592 99 8 not not PART bracis-33592 99 9 be be AUX bracis-33592 99 10 directly directly ADV bracis-33592 99 11 employed employ VERB bracis-33592 99 12 and and CCONJ bracis-33592 99 13 that that SCONJ bracis-33592 99 14 they they PRON bracis-33592 99 15 are be AUX bracis-33592 99 16 more more ADV bracis-33592 99 17 lenient lenient ADJ bracis-33592 99 18 than than ADP bracis-33592 99 19 human human ADJ bracis-33592 99 20 graders grader NOUN bracis-33592 99 21 , , PUNCT bracis-33592 99 22 assigning assign VERB bracis-33592 99 23 fewer few ADJ bracis-33592 99 24 failing fail VERB bracis-33592 99 25 grades grade NOUN bracis-33592 99 26 . . PUNCT bracis-33592 100 1 while while SCONJ bracis-33592 100 2 openai openai NOUN bracis-33592 100 3 models model NOUN bracis-33592 100 4 are be AUX bracis-33592 100 5 the the DET bracis-33592 100 6 most most ADV bracis-33592 100 7 famous famous ADJ bracis-33592 100 8 , , PUNCT bracis-33592 100 9 their their PRON bracis-33592 100 10 cost cost NOUN bracis-33592 100 11 is be AUX bracis-33592 100 12 prohibitive prohibitive ADJ bracis-33592 100 13 , , PUNCT bracis-33592 100 14 thus thus ADV bracis-33592 100 15 we we PRON bracis-33592 100 16 explore explore VERB bracis-33592 100 17 here here ADV bracis-33592 100 18 the the DET bracis-33592 100 19 usage usage NOUN bracis-33592 100 20 of of ADP bracis-33592 100 21 gemini gemini PROPN bracis-33592 100 22 ’s ’s PART bracis-33592 100 23 free free ADJ bracis-33592 100 24 api api NOUN bracis-33592 100 25 and and CCONJ bracis-33592 100 26 how how SCONJ bracis-33592 100 27 resilient resilient ADJ bracis-33592 100 28 it it PRON bracis-33592 100 29 is be AUX bracis-33592 100 30 against against ADP bracis-33592 100 31 adversarial adversarial ADJ bracis-33592 100 32 attacks attack NOUN bracis-33592 100 33 . . PUNCT bracis-33592 101 1 3 3 NUM bracis-33592 101 2 methodology methodology NOUN bracis-33592 101 3 we we PRON bracis-33592 101 4 start start VERB bracis-33592 101 5 by by ADP bracis-33592 101 6 selecting select VERB bracis-33592 101 7 the the DET bracis-33592 101 8 enem enem PROPN bracis-33592 101 9 - - PUNCT bracis-33592 101 10 aes aes NOUN bracis-33592 101 11 dataset dataset NOUN bracis-33592 101 12 and and CCONJ bracis-33592 101 13 pre pre ADJ bracis-33592 101 14 - - ADJ bracis-33592 101 15 trained train VERB bracis-33592 101 16 models model NOUN bracis-33592 101 17 [ [ X bracis-33592 101 18 34 34 NUM bracis-33592 101 19 ] ] PUNCT bracis-33592 101 20 , , PUNCT bracis-33592 101 21 which which PRON bracis-33592 101 22 are be AUX bracis-33592 101 23 easily easily ADV bracis-33592 101 24 accessible accessible ADJ bracis-33592 101 25 through through ADP bracis-33592 101 26 hugging hug VERB bracis-33592 101 27 face face NOUN bracis-33592 101 28 api api NOUN bracis-33592 101 29 . . PUNCT bracis-33592 102 1 essays essay NOUN bracis-33592 102 2 in in ADP bracis-33592 102 3 this this DET bracis-33592 102 4 dataset dataset NOUN bracis-33592 102 5 are be AUX bracis-33592 102 6 evaluated evaluate VERB bracis-33592 102 7 according accord VERB bracis-33592 102 8 to to ADP bracis-33592 102 9 five five NUM bracis-33592 102 10 difference difference NOUN bracis-33592 102 11 criteria criterion NOUN bracis-33592 102 12 , , PUNCT bracis-33592 102 13 or or CCONJ bracis-33592 102 14 competencies competency NOUN bracis-33592 102 15 , , PUNCT bracis-33592 102 16 and and CCONJ bracis-33592 102 17 separated separate VERB bracis-33592 102 18 predictors predictor NOUN bracis-33592 102 19 have have AUX bracis-33592 102 20 been be AUX bracis-33592 102 21 trained train VERB bracis-33592 102 22 for for ADP bracis-33592 102 23 each each DET bracis-33592 102 24 specific specific ADJ bracis-33592 102 25 criterion criterion NOUN bracis-33592 102 26 . . PUNCT bracis-33592 103 1 we we PRON bracis-33592 103 2 use use VERB bracis-33592 103 3 the the DET bracis-33592 103 4 bert bert NOUN bracis-33592 103 5 - - PUNCT bracis-33592 103 6 base base NOUN bracis-33592 103 7 ordinal ordinal ADJ bracis-33592 103 8 model model NOUN bracis-33592 103 9 available available ADJ bracis-33592 103 10 for for ADP bracis-33592 103 11 each each DET bracis-33592 103 12 competence competence NOUN bracis-33592 103 13 , , PUNCT bracis-33592 103 14 as as SCONJ bracis-33592 103 15 they they PRON bracis-33592 103 16 are be AUX bracis-33592 103 17 relatively relatively ADV bracis-33592 103 18 small small ADJ bracis-33592 103 19 and and CCONJ bracis-33592 103 20 achieve achieve VERB bracis-33592 103 21 high high ADJ bracis-33592 103 22 performances performance NOUN bracis-33592 103 23 . . PUNCT bracis-33592 104 1 additionally additionally ADV bracis-33592 104 2 , , PUNCT bracis-33592 104 3 we we PRON bracis-33592 104 4 used use VERB bracis-33592 104 5 the the DET bracis-33592 104 6 splits split NOUN bracis-33592 104 7 available available ADJ bracis-33592 104 8 using use VERB bracis-33592 104 9 the the DET bracis-33592 104 10 “ " PUNCT bracis-33592 104 11 propor2024 propor2024 PROPN bracis-33592 104 12 ” " PUNCT bracis-33592 104 13 parameter parameter NOUN bracis-33592 104 14 . . PUNCT bracis-33592 105 1 we we PRON bracis-33592 105 2 also also ADV bracis-33592 105 3 used use VERB bracis-33592 105 4 the the DET bracis-33592 105 5 scikit scikit NOUN bracis-33592 105 6 - - PUNCT bracis-33592 105 7 learn learn VERB bracis-33592 105 8 package package NOUN bracis-33592 105 9 [ [ X bracis-33592 105 10 33 33 NUM bracis-33592 105 11 ] ] PUNCT bracis-33592 105 12 to to PART bracis-33592 105 13 train train VERB bracis-33592 105 14 , , PUNCT bracis-33592 105 15 for for ADP bracis-33592 105 16 each each DET bracis-33592 105 17 individual individual ADJ bracis-33592 105 18 competence competence NOUN bracis-33592 105 19 , , PUNCT bracis-33592 105 20 a a DET bracis-33592 105 21 linear linear ADJ bracis-33592 105 22 regressor regressor NOUN bracis-33592 105 23 that that PRON bracis-33592 105 24 uses use VERB bracis-33592 105 25 72 72 NUM bracis-33592 105 26 features feature NOUN bracis-33592 105 27 calculated calculate VERB bracis-33592 105 28 by by ADP bracis-33592 105 29 the the DET bracis-33592 105 30 open open ADJ bracis-33592 105 31 api api NOUN bracis-33592 105 32 of of ADP bracis-33592 105 33 nilc nilc NOUN bracis-33592 105 34 - - PUNCT bracis-33592 105 35 metrix metrix NOUN bracis-33592 105 36 [ [ X bracis-33592 105 37 24 24 NUM bracis-33592 105 38 ] ] PUNCT bracis-33592 105 39 . . PUNCT bracis-33592 106 1 these these DET bracis-33592 106 2 features feature NOUN bracis-33592 106 3 are be AUX bracis-33592 106 4 interesting interesting ADJ bracis-33592 106 5 because because SCONJ bracis-33592 106 6 they they PRON bracis-33592 106 7 calculate calculate VERB bracis-33592 106 8 various various ADJ bracis-33592 106 9 linguistic linguistic ADJ bracis-33592 106 10 levels level NOUN bracis-33592 106 11 of of ADP bracis-33592 106 12 written write VERB bracis-33592 106 13 language language NOUN bracis-33592 106 14 , , PUNCT bracis-33592 106 15 such such ADJ bracis-33592 106 16 as as ADP bracis-33592 106 17 the the DET bracis-33592 106 18 ratio ratio NOUN bracis-33592 106 19 of of ADP bracis-33592 106 20 a a DET bracis-33592 106 21 certain certain ADJ bracis-33592 106 22 part part NOUN bracis-33592 106 23 of of ADP bracis-33592 106 24 speech speech NOUN bracis-33592 106 25 , , PUNCT bracis-33592 106 26 how how SCONJ bracis-33592 106 27 ambiguous ambiguous ADJ bracis-33592 106 28 the the DET bracis-33592 106 29 words word NOUN bracis-33592 106 30 are be AUX bracis-33592 106 31 , , PUNCT bracis-33592 106 32 text text NOUN bracis-33592 106 33 length length NOUN bracis-33592 106 34 , , PUNCT bracis-33592 106 35 flesch flesch PROPN bracis-33592 106 36 score score NOUN bracis-33592 106 37 and and CCONJ bracis-33592 106 38 others other NOUN bracis-33592 106 39 . . PUNCT bracis-33592 107 1 although although SCONJ bracis-33592 107 2 they they PRON bracis-33592 107 3 might might AUX bracis-33592 107 4 be be AUX bracis-33592 107 5 far far ADV bracis-33592 107 6 from from ADP bracis-33592 107 7 what what PRON bracis-33592 107 8 the the DET bracis-33592 107 9 enem enem PROPN bracis-33592 107 10 guiding guide VERB bracis-33592 107 11 book book NOUN bracis-33592 107 12 states state NOUN bracis-33592 107 13 that that PRON bracis-33592 107 14 are be AUX bracis-33592 107 15 being be AUX bracis-33592 107 16 evaluated evaluate VERB bracis-33592 107 17 , , PUNCT bracis-33592 107 18 they they PRON bracis-33592 107 19 might might AUX bracis-33592 107 20 relate relate VERB bracis-33592 107 21 closer close ADV bracis-33592 107 22 to to ADP bracis-33592 107 23 what what PRON bracis-33592 107 24 the the DET bracis-33592 107 25 neural neural ADJ bracis-33592 107 26 models model NOUN bracis-33592 107 27 are be AUX bracis-33592 107 28 evaluating evaluate VERB bracis-33592 107 29 . . PUNCT bracis-33592 108 1 for for ADP bracis-33592 108 2 this this DET bracis-33592 108 3 grader grader NOUN bracis-33592 108 4 , , PUNCT bracis-33592 108 5 as as SCONJ bracis-33592 108 6 we we PRON bracis-33592 108 7 did do AUX bracis-33592 108 8 not not PART bracis-33592 108 9 optimize optimize VERB bracis-33592 108 10 any any DET bracis-33592 108 11 hyperparameters hyperparameter NOUN bracis-33592 108 12 , , PUNCT bracis-33592 108 13 we we PRON bracis-33592 108 14 joined join VERB bracis-33592 108 15 the the DET bracis-33592 108 16 training training NOUN bracis-33592 108 17 and and CCONJ bracis-33592 108 18 validation validation NOUN bracis-33592 108 19 split split NOUN bracis-33592 108 20 to to PART bracis-33592 108 21 increase increase VERB bracis-33592 108 22 the the DET bracis-33592 108 23 number number NOUN bracis-33592 108 24 of of ADP bracis-33592 108 25 training training NOUN bracis-33592 108 26 samples sample NOUN bracis-33592 108 27 . . PUNCT bracis-33592 109 1 as as SCONJ bracis-33592 109 2 this this DET bracis-33592 109 3 model model NOUN bracis-33592 109 4 outputs output VERB bracis-33592 109 5 a a DET bracis-33592 109 6 real real ADJ bracis-33592 109 7 number number NOUN bracis-33592 109 8 and and CCONJ bracis-33592 109 9 the the DET bracis-33592 109 10 enem enem PROPN bracis-33592 109 11 grades grade NOUN bracis-33592 109 12 are be AUX bracis-33592 109 13 in in ADP bracis-33592 109 14 steps step NOUN bracis-33592 109 15 of of ADP bracis-33592 109 16 40 40 NUM bracis-33592 109 17 , , PUNCT bracis-33592 109 18 we we PRON bracis-33592 109 19 round round VERB bracis-33592 109 20 it it PRON bracis-33592 109 21 to to ADP bracis-33592 109 22 the the DET bracis-33592 109 23 closest close ADJ bracis-33592 109 24 allowed allow VERB bracis-33592 109 25 grade grade NOUN bracis-33592 109 26 . . PUNCT bracis-33592 110 1 for for ADP bracis-33592 110 2 fine fine ADV bracis-33592 110 3 - - PUNCT bracis-33592 110 4 tuning tuning VERB bracis-33592 110 5 a a DET bracis-33592 110 6 large large ADJ bracis-33592 110 7 language language NOUN bracis-33592 110 8 model model NOUN bracis-33592 110 9 , , PUNCT bracis-33592 110 10 we we PRON bracis-33592 110 11 chose choose VERB bracis-33592 110 12 to to ADP bracis-33592 110 13 fine fine ADJ bracis-33592 110 14 - - PUNCT bracis-33592 110 15 tune tune NOUN bracis-33592 110 16 a a DET bracis-33592 110 17 phi-3 phi-3 PRON bracis-33592 110 18 - - PUNCT bracis-33592 110 19 medium medium NOUN bracis-33592 110 20 model model NOUN bracis-33592 110 21 , , PUNCT bracis-33592 110 22 balancing balance VERB bracis-33592 110 23 model model NOUN bracis-33592 110 24 size size NOUN bracis-33592 110 25 with with ADP bracis-33592 110 26 our our PRON bracis-33592 110 27 available available ADJ bracis-33592 110 28 hardware hardware NOUN bracis-33592 110 29 . . PUNCT bracis-33592 111 1 although although SCONJ bracis-33592 111 2 smaller small ADJ bracis-33592 111 3 than than ADP bracis-33592 111 4 some some DET bracis-33592 111 5 other other ADJ bracis-33592 111 6 models model NOUN bracis-33592 111 7 , , PUNCT bracis-33592 111 8 it it PRON bracis-33592 111 9 still still ADV bracis-33592 111 10 demands demand VERB bracis-33592 111 11 significant significant ADJ bracis-33592 111 12 computational computational ADJ bracis-33592 111 13 power power NOUN bracis-33592 111 14 for for ADP bracis-33592 111 15 training training NOUN bracis-33592 111 16 . . PUNCT bracis-33592 112 1 we we PRON bracis-33592 112 2 had have VERB bracis-33592 112 3 access access NOUN bracis-33592 112 4 to to ADP bracis-33592 112 5 an an DET bracis-33592 112 6 nvidia nvidia PROPN bracis-33592 112 7 gpu gpu PROPN bracis-33592 112 8 rtx rtx PROPN bracis-33592 112 9 a6000 a6000 PROPN bracis-33592 112 10 , , PUNCT bracis-33592 112 11 and and CCONJ bracis-33592 112 12 we we PRON bracis-33592 112 13 applied apply VERB bracis-33592 112 14 several several ADJ bracis-33592 112 15 optimizations optimization NOUN bracis-33592 112 16 for for ADP bracis-33592 112 17 fine fine ADV bracis-33592 112 18 - - PUNCT bracis-33592 112 19 tuning tuning NOUN bracis-33592 112 20 [ [ X bracis-33592 112 21 35 35 NUM bracis-33592 112 22 ] ] PUNCT bracis-33592 112 23 , , PUNCT bracis-33592 112 24 including include VERB bracis-33592 112 25 gradient gradient NOUN bracis-33592 112 26 checkpointing checkpointing NOUN bracis-33592 112 27 , , PUNCT bracis-33592 112 28 flash flash VERB bracis-33592 112 29 attention attention NOUN bracis-33592 112 30 v2 v2 PROPN bracis-33592 113 1 [ [ X bracis-33592 113 2 10 10 NUM bracis-33592 113 3 ] ] PUNCT bracis-33592 113 4 , , PUNCT bracis-33592 113 5 and and CCONJ bracis-33592 113 6 lora lora X bracis-33592 114 1 [ [ X bracis-33592 114 2 19 19 NUM bracis-33592 114 3 ] ] PUNCT bracis-33592 114 4 . . PUNCT bracis-33592 115 1 the the DET bracis-33592 115 2 available available ADJ bracis-33592 115 3 model model NOUN bracis-33592 115 4 is be AUX bracis-33592 115 5 instruction instruction NOUN bracis-33592 115 6 fine-tuned.footnote fine-tuned.footnote CCONJ bracis-33592 115 7 2 2 NUM bracis-33592 115 8 during during ADP bracis-33592 115 9 our our PRON bracis-33592 115 10 tests test NOUN bracis-33592 115 11 , , PUNCT bracis-33592 115 12 we we PRON bracis-33592 115 13 found find VERB bracis-33592 115 14 it it PRON bracis-33592 115 15 necessary necessary ADJ bracis-33592 115 16 to to PART bracis-33592 115 17 pass pass VERB bracis-33592 115 18 an an DET bracis-33592 115 19 instruction instruction NOUN bracis-33592 115 20 to to ADP bracis-33592 115 21 the the DET bracis-33592 115 22 system system NOUN bracis-33592 115 23 role role NOUN bracis-33592 115 24 before before ADP bracis-33592 115 25 prompting prompt VERB bracis-33592 115 26 a a DET bracis-33592 115 27 given give VERB bracis-33592 115 28 essay essay NOUN bracis-33592 115 29 . . PUNCT bracis-33592 116 1 the the DET bracis-33592 116 2 prompt prompt ADJ bracis-33592 116 3 structure structure NOUN bracis-33592 116 4 is be AUX bracis-33592 116 5 shown show VERB bracis-33592 116 6 in in ADP bracis-33592 116 7 fig fig NOUN bracis-33592 116 8 . . PUNCT bracis-33592 116 9     SPACE bracis-33592 117 1 1 1 NUM bracis-33592 117 2 . . PUNCT bracis-33592 117 3 fig fig NOUN bracis-33592 117 4 . . PUNCT bracis-33592 118 1 1 1 X bracis-33592 118 2 . . X bracis-33592 118 3 prompt prompt ADJ bracis-33592 118 4 - - PUNCT bracis-33592 118 5 structure structure NOUN bracis-33592 118 6 used use VERB bracis-33592 118 7 for for ADP bracis-33592 118 8 each each DET bracis-33592 118 9 concept concept NOUN bracis-33592 118 10 fine fine ADV bracis-33592 118 11 - - PUNCT bracis-33592 118 12 tuning tune VERB bracis-33592 118 13 through through ADP bracis-33592 118 14 phi-3 phi-3 NOUN bracis-33592 118 15 . . PUNCT bracis-33592 119 1 full full ADJ bracis-33592 119 2 size size NOUN bracis-33592 119 3 image image NOUN bracis-33592 119 4 in in ADP bracis-33592 119 5 this this DET bracis-33592 119 6 structure structure NOUN bracis-33592 119 7 , , PUNCT bracis-33592 119 8 concept_system concept_system PROPN bracis-33592 119 9 is be AUX bracis-33592 119 10 a a DET bracis-33592 119 11 placeholder placeholder NOUN bracis-33592 119 12 for for ADP bracis-33592 119 13 the the DET bracis-33592 119 14 competence competence NOUN bracis-33592 119 15 being be AUX bracis-33592 119 16 fine fine ADV bracis-33592 119 17 - - PUNCT bracis-33592 119 18 tuned tune VERB bracis-33592 119 19 ( ( PUNCT bracis-33592 119 20 e.g. e.g. ADV bracis-33592 119 21 , , PUNCT bracis-33592 119 22 enem enem PROPN bracis-33592 119 23 competences competence NOUN bracis-33592 119 24 that that PRON bracis-33592 119 25 vary vary VERB bracis-33592 119 26 from from ADP bracis-33592 119 27 1 1 NUM bracis-33592 119 28 to to ADP bracis-33592 119 29 5 5 NUM bracis-33592 119 30 ) ) PUNCT bracis-33592 119 31 , , PUNCT bracis-33592 119 32 and and CCONJ bracis-33592 119 33 essay_example essay_example PROPN bracis-33592 119 34 represents represent VERB bracis-33592 119 35 the the DET bracis-33592 119 36 essay essay NOUN bracis-33592 119 37 we we PRON bracis-33592 119 38 are be AUX bracis-33592 119 39 interested interested ADJ bracis-33592 119 40 in in ADP bracis-33592 119 41 fine fine ADV bracis-33592 119 42 - - PUNCT bracis-33592 119 43 tuning tuning NOUN bracis-33592 119 44 . . PUNCT bracis-33592 120 1 to to PART bracis-33592 120 2 define define VERB bracis-33592 120 3 the the DET bracis-33592 120 4 concepts concept NOUN bracis-33592 120 5 , , PUNCT bracis-33592 120 6 we we PRON bracis-33592 120 7 copied copy VERB bracis-33592 120 8 the the DET bracis-33592 120 9 definitions definition NOUN bracis-33592 120 10 directly directly ADV bracis-33592 120 11 from from ADP bracis-33592 120 12 the the DET bracis-33592 120 13 enem enem PROPN bracis-33592 120 14 2023 2023 NUM bracis-33592 120 15 student student NOUN bracis-33592 120 16 ’s ’s PART bracis-33592 120 17 reference reference NOUN bracis-33592 120 18 manual manual NOUN bracis-33592 120 19 , , PUNCT bracis-33592 120 20 which which PRON bracis-33592 120 21 provides provide VERB bracis-33592 120 22 clear clear ADJ bracis-33592 120 23 instructions instruction NOUN bracis-33592 120 24 on on ADP bracis-33592 120 25 how how SCONJ bracis-33592 120 26 each each DET bracis-33592 120 27 grade grade NOUN bracis-33592 120 28 is be AUX bracis-33592 120 29 applied apply VERB bracis-33592 120 30 . . PUNCT bracis-33592 121 1 following follow VERB bracis-33592 121 2 these these DET bracis-33592 121 3 optimizations optimization NOUN bracis-33592 121 4 , , PUNCT bracis-33592 121 5 we we PRON bracis-33592 121 6 added add VERB bracis-33592 121 7 a a DET bracis-33592 121 8 sequence sequence NOUN bracis-33592 121 9 classification classification NOUN bracis-33592 121 10 head head NOUN bracis-33592 121 11 to to ADP bracis-33592 121 12 the the DET bracis-33592 121 13 existing exist VERB bracis-33592 121 14 model model NOUN bracis-33592 121 15 and and CCONJ bracis-33592 121 16 fine fine ADV bracis-33592 121 17 - - PUNCT bracis-33592 121 18 tuned tune VERB bracis-33592 121 19 it it PRON bracis-33592 121 20 according accord VERB bracis-33592 121 21 to to ADP bracis-33592 121 22 the the DET bracis-33592 121 23 hyperparameters hyperparameter NOUN bracis-33592 121 24 outlined outline VERB bracis-33592 121 25 in in ADP bracis-33592 121 26 table table NOUN bracis-33592 121 27     SPACE bracis-33592 121 28 1 1 NUM bracis-33592 121 29 . . PUNCT bracis-33592 122 1 it it PRON bracis-33592 122 2 is be AUX bracis-33592 122 3 noteworthy noteworthy ADJ bracis-33592 122 4 that that SCONJ bracis-33592 122 5 these these DET bracis-33592 122 6 models model NOUN bracis-33592 122 7 were be AUX bracis-33592 122 8 instruction instruction NOUN bracis-33592 122 9 fine fine ADV bracis-33592 122 10 - - PUNCT bracis-33592 122 11 tuned tune VERB bracis-33592 122 12 through through ADP bracis-33592 122 13 supervised supervised ADJ bracis-33592 122 14 fine fine ADV bracis-33592 122 15 - - PUNCT bracis-33592 122 16 tuning tuning NOUN bracis-33592 122 17 and and CCONJ bracis-33592 122 18 direct direct ADJ bracis-33592 122 19 preference preference NOUN bracis-33592 122 20 optimization optimization NOUN bracis-33592 122 21 for for ADP bracis-33592 122 22 english english ADJ bracis-33592 122 23 conversations conversation NOUN bracis-33592 122 24 . . PUNCT bracis-33592 123 1 fine fine ADJ bracis-33592 123 2 - - PUNCT bracis-33592 123 3 tuning tune VERB bracis-33592 123 4 capabilities capability NOUN bracis-33592 123 5 for for ADP bracis-33592 123 6 languages language NOUN bracis-33592 123 7 other other ADJ bracis-33592 123 8 than than ADP bracis-33592 123 9 english english PROPN bracis-33592 123 10 were be AUX bracis-33592 123 11 not not PART bracis-33592 123 12 investigated investigate VERB bracis-33592 123 13 prior prior ADV bracis-33592 123 14 to to ADP bracis-33592 123 15 the the DET bracis-33592 123 16 completion completion NOUN bracis-33592 123 17 of of ADP bracis-33592 123 18 this this DET bracis-33592 123 19 study study NOUN bracis-33592 123 20 [ [ X bracis-33592 123 21 1 1 NUM bracis-33592 123 22 ] ] PUNCT bracis-33592 123 23 . . PUNCT bracis-33592 124 1 we we PRON bracis-33592 124 2 found find VERB bracis-33592 124 3 it it PRON bracis-33592 124 4 interesting interesting ADJ bracis-33592 124 5 to to PART bracis-33592 124 6 notice notice VERB bracis-33592 124 7 an an DET bracis-33592 124 8 ability ability NOUN bracis-33592 124 9 of of ADP bracis-33592 124 10 the the DET bracis-33592 124 11 final final ADJ bracis-33592 124 12 phi-3 phi-3 PUNCT bracis-33592 124 13 to to PART bracis-33592 124 14 generalize generalize VERB bracis-33592 124 15 and and CCONJ bracis-33592 124 16 outperform outperform NOUN bracis-33592 124 17 models model NOUN bracis-33592 124 18 that that PRON bracis-33592 124 19 were be AUX bracis-33592 124 20 trained train VERB bracis-33592 124 21 solely solely ADV bracis-33592 124 22 on on ADP bracis-33592 124 23 portuguese portuguese ADJ bracis-33592 124 24 data datum NOUN bracis-33592 124 25 — — PUNCT bracis-33592 124 26 the the DET bracis-33592 124 27 prior prior ADJ bracis-33592 124 28 model model NOUN bracis-33592 124 29 based base VERB bracis-33592 124 30 on on ADP bracis-33592 124 31 bertimbau bertimbau NOUN bracis-33592 124 32 [ [ X bracis-33592 124 33 36 36 NUM bracis-33592 124 34 ] ] PUNCT bracis-33592 124 35 . . PUNCT bracis-33592 125 1 table table NOUN bracis-33592 125 2 1 1 NUM bracis-33592 125 3 . . PUNCT bracis-33592 125 4 phi-3 phi-3 PUNCT bracis-33592 125 5 fine fine ADJ bracis-33592 125 6 tuning tune VERB bracis-33592 125 7 hyperparametersfull hyperparametersfull ADJ bracis-33592 125 8 size size NOUN bracis-33592 125 9 table table NOUN bracis-33592 125 10 finally finally ADV bracis-33592 125 11 , , PUNCT bracis-33592 125 12 we we PRON bracis-33592 125 13 employ employ VERB bracis-33592 125 14 google google PROPN bracis-33592 125 15 ’s ’s PART bracis-33592 125 16 gemini gemini PROPN bracis-33592 125 17 chatbot chatbot NOUN bracis-33592 125 18 . . PUNCT bracis-33592 126 1 we we PRON bracis-33592 126 2 chose choose VERB bracis-33592 126 3 gemini gemini PROPN bracis-33592 126 4 over over ADP bracis-33592 126 5 chatgpt chatgpt NOUN bracis-33592 126 6 because because SCONJ bracis-33592 126 7 gemini gemini NOUN bracis-33592 126 8 offers offer VERB bracis-33592 126 9 a a DET bracis-33592 126 10 free free ADJ bracis-33592 126 11 api api NOUN bracis-33592 126 12 . . PUNCT bracis-33592 127 1 the the DET bracis-33592 127 2 current current ADJ bracis-33592 127 3 model model NOUN bracis-33592 127 4 available available ADJ bracis-33592 127 5 is be AUX bracis-33592 127 6 the the DET bracis-33592 127 7 gemini gemini PROPN bracis-33592 127 8 1.5 1.5 NUM bracis-33592 127 9 flash flash NOUN bracis-33592 128 1 [ [ X bracis-33592 128 2 38 38 NUM bracis-33592 128 3 ] ] PUNCT bracis-33592 128 4 . . PUNCT bracis-33592 129 1 its its PRON bracis-33592 129 2 size size NOUN bracis-33592 129 3 and and CCONJ bracis-33592 129 4 training training NOUN bracis-33592 129 5 set set NOUN bracis-33592 129 6 are be AUX bracis-33592 129 7 not not PART bracis-33592 129 8 publicly publicly ADV bracis-33592 129 9 disclosed disclose VERB bracis-33592 129 10 , , PUNCT bracis-33592 129 11 so so CCONJ bracis-33592 129 12 we we PRON bracis-33592 129 13 can can AUX bracis-33592 129 14 only only ADV bracis-33592 129 15 speculate speculate VERB bracis-33592 129 16 about about ADP bracis-33592 129 17 its its PRON bracis-33592 129 18 dimensions dimension NOUN bracis-33592 129 19 . . PUNCT bracis-33592 130 1 according accord VERB bracis-33592 130 2 to to ADP bracis-33592 130 3 the the DET bracis-33592 130 4 lmsys lmsys ADJ bracis-33592 130 5 leaderboard leaderboard NOUN bracis-33592 130 6 [ [ X bracis-33592 130 7 9 9 NUM bracis-33592 130 8 ] ] PUNCT bracis-33592 130 9 , , PUNCT bracis-33592 130 10 it it PRON bracis-33592 130 11 is be AUX bracis-33592 130 12 ranked rank VERB bracis-33592 130 13 near near ADP bracis-33592 130 14 the the DET bracis-33592 130 15 llama3 llama3 ADJ bracis-33592 130 16 70b 70b NOUN bracis-33592 130 17 model model PROPN bracis-33592 130 18 , , PUNCT bracis-33592 130 19 suggesting suggest VERB bracis-33592 130 20 it it PRON bracis-33592 130 21 has have VERB bracis-33592 130 22 at at ADV bracis-33592 130 23 least least ADJ bracis-33592 130 24 70 70 NUM bracis-33592 130 25 billion billion NUM bracis-33592 130 26 parameters parameter NOUN bracis-33592 130 27 . . PUNCT bracis-33592 131 1 we we PRON bracis-33592 131 2 do do AUX bracis-33592 131 3 not not PART bracis-33592 131 4 train train VERB bracis-33592 131 5 this this DET bracis-33592 131 6 model model NOUN bracis-33592 131 7 ; ; PUNCT bracis-33592 131 8 we we PRON bracis-33592 131 9 use use VERB bracis-33592 131 10 it it PRON bracis-33592 131 11 exclusively exclusively ADV bracis-33592 131 12 through through ADP bracis-33592 131 13 its its PRON bracis-33592 131 14 api api NOUN bracis-33592 131 15 . . PUNCT bracis-33592 132 1 additionally additionally ADV bracis-33592 132 2 , , PUNCT bracis-33592 132 3 we we PRON bracis-33592 132 4 did do AUX bracis-33592 132 5 not not PART bracis-33592 132 6 employ employ VERB bracis-33592 132 7 any any DET bracis-33592 132 8 prompt prompt ADJ bracis-33592 132 9 engineering engineering NOUN bracis-33592 132 10 techniques technique NOUN bracis-33592 132 11 , , PUNCT bracis-33592 132 12 instead instead ADV bracis-33592 132 13 applying apply VERB bracis-33592 132 14 a a DET bracis-33592 132 15 zero zero NUM bracis-33592 132 16 - - PUNCT bracis-33592 132 17 shot shot NOUN bracis-33592 132 18 learning learning NOUN bracis-33592 132 19 process process NOUN bracis-33592 132 20 [ [ X bracis-33592 132 21 6 6 NUM bracis-33592 132 22 , , PUNCT bracis-33592 132 23 8 8 NUM bracis-33592 132 24 , , PUNCT bracis-33592 132 25 23 23 NUM bracis-33592 132 26 ] ] PUNCT bracis-33592 132 27 , , PUNCT bracis-33592 132 28 as as SCONJ bracis-33592 132 29 we we PRON bracis-33592 132 30 only only ADV bracis-33592 132 31 asked ask VERB bracis-33592 132 32 it it PRON bracis-33592 132 33 to to PART bracis-33592 132 34 grade grade VERB bracis-33592 132 35 the the DET bracis-33592 132 36 essays essay NOUN bracis-33592 132 37 using use VERB bracis-33592 132 38 the the DET bracis-33592 132 39 enem enem PROPN bracis-33592 132 40 criteria criterion NOUN bracis-33592 132 41 . . PUNCT bracis-33592 133 1 however however ADV bracis-33592 133 2 , , PUNCT bracis-33592 133 3 the the DET bracis-33592 133 4 model model NOUN bracis-33592 133 5 produces produce VERB bracis-33592 133 6 grades grade NOUN bracis-33592 133 7 that that PRON bracis-33592 133 8 do do AUX bracis-33592 133 9 not not PART bracis-33592 133 10 follow follow VERB bracis-33592 133 11 the the DET bracis-33592 133 12 40 40 NUM bracis-33592 133 13 - - PUNCT bracis-33592 133 14 point point NOUN bracis-33592 133 15 step step NOUN bracis-33592 133 16 , , PUNCT bracis-33592 133 17 so so ADV bracis-33592 133 18 we we PRON bracis-33592 133 19 round round VERB bracis-33592 133 20 the the DET bracis-33592 133 21 numbers number NOUN bracis-33592 133 22 according accord VERB bracis-33592 133 23 to to ADP bracis-33592 133 24 the the DET bracis-33592 133 25 same same ADJ bracis-33592 133 26 criteria criterion NOUN bracis-33592 133 27 as as ADP bracis-33592 133 28 the the DET bracis-33592 133 29 linear linear ADJ bracis-33592 133 30 regressor regressor NOUN bracis-33592 133 31 . . PUNCT bracis-33592 134 1 using use VERB bracis-33592 134 2 these these DET bracis-33592 134 3 graders grader NOUN bracis-33592 134 4 , , PUNCT bracis-33592 134 5 we we PRON bracis-33592 134 6 test test VERB bracis-33592 134 7 them they PRON bracis-33592 134 8 — — PUNCT bracis-33592 134 9 except except SCONJ bracis-33592 134 10 for for ADP bracis-33592 134 11 the the DET bracis-33592 134 12 bert bert PROPN bracis-33592 134 13 one one NUM bracis-33592 134 14 , , PUNCT bracis-33592 134 15 as as SCONJ bracis-33592 134 16 its its PRON bracis-33592 134 17 performance performance NOUN bracis-33592 134 18 has have AUX bracis-33592 134 19 already already ADV bracis-33592 134 20 been be AUX bracis-33592 134 21 published publish VERB bracis-33592 134 22 in in ADP bracis-33592 134 23 [ [ X bracis-33592 134 24 34]—and 34]—and NUM bracis-33592 134 25 compare compare VERB bracis-33592 134 26 them they PRON bracis-33592 134 27 against against ADP bracis-33592 134 28 the the DET bracis-33592 134 29 baselines baseline NOUN bracis-33592 134 30 . . PUNCT bracis-33592 135 1 then then ADV bracis-33592 135 2 , , PUNCT bracis-33592 135 3 we we PRON bracis-33592 135 4 use use VERB bracis-33592 135 5 the the DET bracis-33592 135 6 weights weight NOUN bracis-33592 135 7 of of ADP bracis-33592 135 8 the the DET bracis-33592 135 9 linear linear ADJ bracis-33592 135 10 regression regression NOUN bracis-33592 135 11 to to PART bracis-33592 135 12 identify identify VERB bracis-33592 135 13 the the DET bracis-33592 135 14 relevant relevant ADJ bracis-33592 135 15 features feature NOUN bracis-33592 135 16 . . PUNCT bracis-33592 136 1 finally finally ADV bracis-33592 136 2 , , PUNCT bracis-33592 136 3 we we PRON bracis-33592 136 4 use use VERB bracis-33592 136 5 them they PRON bracis-33592 136 6 to to PART bracis-33592 136 7 create create VERB bracis-33592 136 8 possible possible ADJ bracis-33592 136 9 universal universal ADJ bracis-33592 136 10 adversarial adversarial ADJ bracis-33592 136 11 attacks attack NOUN bracis-33592 136 12 that that SCONJ bracis-33592 136 13 students student NOUN bracis-33592 136 14 could could AUX bracis-33592 136 15 create create VERB bracis-33592 136 16 . . PUNCT bracis-33592 137 1 there there PRON bracis-33592 137 2 are be VERB bracis-33592 137 3 no no DET bracis-33592 137 4 limits limit NOUN bracis-33592 137 5 to to ADP bracis-33592 137 6 the the DET bracis-33592 137 7 number number NOUN bracis-33592 137 8 of of ADP bracis-33592 137 9 adversarial adversarial ADJ bracis-33592 137 10 attacks attack NOUN bracis-33592 137 11 that that PRON bracis-33592 137 12 can can AUX bracis-33592 137 13 be be AUX bracis-33592 137 14 created create VERB bracis-33592 137 15 . . PUNCT bracis-33592 138 1 in in ADP bracis-33592 138 2 our our PRON bracis-33592 138 3 investigation investigation NOUN bracis-33592 138 4 , , PUNCT bracis-33592 138 5 we we PRON bracis-33592 138 6 restricted restrict VERB bracis-33592 138 7 ourselves ourselves PRON bracis-33592 138 8 to to ADP bracis-33592 138 9 a a DET bracis-33592 138 10 scenario scenario NOUN bracis-33592 138 11 where where SCONJ bracis-33592 138 12 ( ( PUNCT bracis-33592 138 13 1 1 X bracis-33592 138 14 ) ) PUNCT bracis-33592 138 15 the the DET bracis-33592 138 16 students student NOUN bracis-33592 138 17 do do AUX bracis-33592 138 18 not not PART bracis-33592 138 19 know know VERB bracis-33592 138 20 the the DET bracis-33592 138 21 underlying underlie VERB bracis-33592 138 22 aes aes NOUN bracis-33592 138 23 system system NOUN bracis-33592 138 24 ; ; PUNCT bracis-33592 138 25 ( ( PUNCT bracis-33592 138 26 2 2 X bracis-33592 138 27 ) ) PUNCT bracis-33592 138 28 they they PRON bracis-33592 138 29 do do AUX bracis-33592 138 30 not not PART bracis-33592 138 31 have have VERB bracis-33592 138 32 access access NOUN bracis-33592 138 33 to to ADP bracis-33592 138 34 any any DET bracis-33592 138 35 part part NOUN bracis-33592 138 36 of of ADP bracis-33592 138 37 the the DET bracis-33592 138 38 system system NOUN bracis-33592 138 39 except except SCONJ bracis-33592 138 40 for for ADP bracis-33592 138 41 an an DET bracis-33592 138 42 interface interface NOUN bracis-33592 138 43 where where SCONJ bracis-33592 138 44 they they PRON bracis-33592 138 45 can can AUX bracis-33592 138 46 type type VERB bracis-33592 138 47 the the DET bracis-33592 138 48 essay essay NOUN bracis-33592 138 49 and and CCONJ bracis-33592 138 50 see see VERB bracis-33592 138 51 their their PRON bracis-33592 138 52 final final ADJ bracis-33592 138 53 score score NOUN bracis-33592 138 54 ; ; PUNCT bracis-33592 138 55 ( ( PUNCT bracis-33592 138 56 3 3 X bracis-33592 138 57 ) ) PUNCT bracis-33592 138 58 students student NOUN bracis-33592 138 59 can can AUX bracis-33592 138 60 share share VERB bracis-33592 138 61 their their PRON bracis-33592 138 62 intuitions intuition NOUN bracis-33592 138 63 with with ADP bracis-33592 138 64 each each DET bracis-33592 138 65 other other ADJ bracis-33592 138 66 ; ; PUNCT bracis-33592 138 67 ( ( PUNCT bracis-33592 138 68 4 4 X bracis-33592 138 69 ) ) PUNCT bracis-33592 138 70 they they PRON bracis-33592 138 71 do do AUX bracis-33592 138 72 not not PART bracis-33592 138 73 want want VERB bracis-33592 138 74 to to PART bracis-33592 138 75 properly properly ADV bracis-33592 138 76 write write VERB bracis-33592 138 77 the the DET bracis-33592 138 78 essay essay NOUN bracis-33592 138 79 , , PUNCT bracis-33592 138 80 but but CCONJ bracis-33592 138 81 still still ADV bracis-33592 138 82 they they PRON bracis-33592 138 83 want want VERB bracis-33592 138 84 the the DET bracis-33592 138 85 best good ADJ bracis-33592 138 86 possible possible ADJ bracis-33592 138 87 grade grade NOUN bracis-33592 138 88 . . PUNCT bracis-33592 139 1 having have VERB bracis-33592 139 2 these these DET bracis-33592 139 3 restrictions restriction NOUN bracis-33592 139 4 , , PUNCT bracis-33592 139 5 we we PRON bracis-33592 139 6 devise devise VERB bracis-33592 139 7 some some DET bracis-33592 139 8 attacks attack NOUN bracis-33592 139 9 and and CCONJ bracis-33592 139 10 test test VERB bracis-33592 139 11 them they PRON bracis-33592 139 12 against against ADP bracis-33592 139 13 all all DET bracis-33592 139 14 graders grader NOUN bracis-33592 139 15 . . PUNCT bracis-33592 140 1 4 4 NUM bracis-33592 140 2 experiments experiment NOUN bracis-33592 140 3 we we PRON bracis-33592 140 4 now now ADV bracis-33592 140 5 discuss discuss VERB bracis-33592 140 6 the the DET bracis-33592 140 7 empirical empirical ADJ bracis-33592 140 8 findings finding NOUN bracis-33592 140 9 of of ADP bracis-33592 140 10 our our PRON bracis-33592 140 11 investigation investigation NOUN bracis-33592 140 12 . . PUNCT bracis-33592 141 1 4.1 4.1 NUM bracis-33592 141 2 testing test VERB bracis-33592 141 3 the the DET bracis-33592 141 4     SPACE bracis-33592 141 5 new new ADJ bracis-33592 141 6 models model NOUN bracis-33592 141 7 the the DET bracis-33592 141 8 first first ADJ bracis-33592 141 9 step step NOUN bracis-33592 141 10 before before ADP bracis-33592 141 11 devising devise VERB bracis-33592 141 12 our our PRON bracis-33592 141 13 adversarial adversarial ADJ bracis-33592 141 14 attacks attack NOUN bracis-33592 141 15 is be AUX bracis-33592 141 16 comparing compare VERB bracis-33592 141 17 the the DET bracis-33592 141 18 performance performance NOUN bracis-33592 141 19 of of ADP bracis-33592 141 20 an an DET bracis-33592 141 21 interpretable interpretable ADJ bracis-33592 141 22 model model NOUN bracis-33592 141 23 against against ADP bracis-33592 141 24 the the DET bracis-33592 141 25 neural neural ADJ bracis-33592 141 26 ones one NOUN bracis-33592 141 27 . . PUNCT bracis-33592 142 1 we we PRON bracis-33592 142 2 train train VERB bracis-33592 142 3 the the DET bracis-33592 142 4 models model NOUN bracis-33592 142 5 presented present VERB bracis-33592 142 6 in in ADP bracis-33592 142 7 the the DET bracis-33592 142 8 previous previous ADJ bracis-33592 142 9 section section NOUN bracis-33592 142 10 , , PUNCT bracis-33592 142 11 namely namely ADV bracis-33592 142 12 a a DET bracis-33592 142 13 linear linear ADJ bracis-33592 142 14 regressor regressor NOUN bracis-33592 142 15 ( ( PUNCT bracis-33592 142 16 lr lr PROPN bracis-33592 142 17 ) ) PUNCT bracis-33592 142 18 , , PUNCT bracis-33592 142 19 a a DET bracis-33592 142 20 phi-3 phi-3 ADJ bracis-33592 142 21 and and CCONJ bracis-33592 142 22 a a DET bracis-33592 142 23 gemini gemini NOUN bracis-33592 142 24 - - PUNCT bracis-33592 142 25 based base VERB bracis-33592 142 26 grader grader NOUN bracis-33592 142 27 . . PUNCT bracis-33592 143 1 their their PRON bracis-33592 143 2 performances performance NOUN bracis-33592 143 3 , , PUNCT bracis-33592 143 4 measured measure VERB bracis-33592 143 5 in in ADP bracis-33592 143 6 quadratic quadratic ADJ bracis-33592 143 7 weighted weight VERB bracis-33592 143 8 kappa kappa PROPN bracis-33592 143 9 ( ( PUNCT bracis-33592 143 10 qwk qwk PROPN bracis-33592 143 11 ) ) PUNCT bracis-33592 143 12 , , PUNCT bracis-33592 143 13 are be AUX bracis-33592 143 14 presented present VERB bracis-33592 143 15 in in ADP bracis-33592 143 16 table table NOUN bracis-33592 143 17     SPACE bracis-33592 143 18 2 2 NUM bracis-33592 143 19 and and CCONJ bracis-33592 143 20 compared compare VERB bracis-33592 143 21 against against ADP bracis-33592 143 22 the the DET bracis-33592 143 23 berts bert NOUN bracis-33592 143 24 available available ADJ bracis-33592 143 25 and and CCONJ bracis-33592 143 26 presented present VERB bracis-33592 143 27 in in ADP bracis-33592 143 28 [ [ X bracis-33592 143 29 34 34 NUM bracis-33592 143 30 ] ] PUNCT bracis-33592 143 31 . . PUNCT bracis-33592 144 1 table table NOUN bracis-33592 144 2 2 2 NUM bracis-33592 144 3 . . PUNCT bracis-33592 144 4 performance performance NOUN bracis-33592 144 5 comparison comparison NOUN bracis-33592 144 6 of of ADP bracis-33592 144 7 different different ADJ bracis-33592 144 8 automatic automatic ADJ bracis-33592 144 9 graders grader NOUN bracis-33592 144 10 , , PUNCT bracis-33592 144 11 as as SCONJ bracis-33592 144 12 measured measure VERB bracis-33592 144 13 by by ADP bracis-33592 144 14 qwk.full qwk.full ADJ bracis-33592 144 15 size size NOUN bracis-33592 144 16 table table NOUN bracis-33592 144 17 the the DET bracis-33592 144 18 first first ADJ bracis-33592 144 19 thing thing NOUN bracis-33592 144 20 to to PART bracis-33592 144 21 notice notice VERB bracis-33592 144 22 is be AUX bracis-33592 144 23 that that SCONJ bracis-33592 144 24 their their PRON bracis-33592 144 25 sizes size NOUN bracis-33592 144 26 have have VERB bracis-33592 144 27 different different ADJ bracis-33592 144 28 orders order NOUN bracis-33592 144 29 of of ADP bracis-33592 144 30 magnitude magnitude NOUN bracis-33592 144 31 , , PUNCT bracis-33592 144 32 but but CCONJ bracis-33592 144 33 the the DET bracis-33592 144 34 performances performance NOUN bracis-33592 144 35 are be AUX bracis-33592 144 36 still still ADV bracis-33592 144 37 not not PART bracis-33592 144 38 that that ADV bracis-33592 144 39 distant distant ADJ bracis-33592 144 40 . . PUNCT bracis-33592 145 1 the the DET bracis-33592 145 2 linear linear ADJ bracis-33592 145 3 regressor regressor NOUN bracis-33592 145 4 was be AUX bracis-33592 145 5 expected expect VERB bracis-33592 145 6 to to PART bracis-33592 145 7 have have VERB bracis-33592 145 8 the the DET bracis-33592 145 9 worst bad ADJ bracis-33592 145 10 performance performance NOUN bracis-33592 145 11 across across ADP bracis-33592 145 12 all all DET bracis-33592 145 13 competences competence NOUN bracis-33592 145 14 , , PUNCT bracis-33592 145 15 but but CCONJ bracis-33592 145 16 it it PRON bracis-33592 145 17 happened happen VERB bracis-33592 145 18 only only ADV bracis-33592 145 19 in in ADP bracis-33592 145 20 two two NUM bracis-33592 145 21 out out ADP bracis-33592 145 22 of of ADP bracis-33592 145 23 the the DET bracis-33592 145 24 five five NUM bracis-33592 145 25 . . PUNCT bracis-33592 146 1 when when SCONJ bracis-33592 146 2 it it PRON bracis-33592 146 3 had have VERB bracis-33592 146 4 the the DET bracis-33592 146 5 lowest low ADJ bracis-33592 146 6 performance performance NOUN bracis-33592 146 7 , , PUNCT bracis-33592 146 8 it it PRON bracis-33592 146 9 was be AUX bracis-33592 146 10 only only ADV bracis-33592 146 11 by by ADP bracis-33592 146 12 a a DET bracis-33592 146 13 margin margin NOUN bracis-33592 146 14 of of ADP bracis-33592 146 15 0.07 0.07 NUM bracis-33592 146 16 and and CCONJ bracis-33592 146 17 0.04 0.04 NUM bracis-33592 146 18 compared compare VERB bracis-33592 146 19 to to ADP bracis-33592 146 20 the the DET bracis-33592 146 21 bert bert PROPN bracis-33592 146 22 baseline baseline PROPN bracis-33592 146 23 . . PUNCT bracis-33592 147 1 the the DET bracis-33592 147 2 similar similar ADJ bracis-33592 147 3 performance performance NOUN bracis-33592 147 4 to to ADP bracis-33592 147 5 the the DET bracis-33592 147 6 others other NOUN bracis-33592 147 7 may may AUX bracis-33592 147 8 point point VERB bracis-33592 147 9 out out ADP bracis-33592 147 10 that that SCONJ bracis-33592 147 11 the the DET bracis-33592 147 12 neural neural ADJ bracis-33592 147 13 models model NOUN bracis-33592 147 14 are be AUX bracis-33592 147 15 exploring explore VERB bracis-33592 147 16 simple simple ADJ bracis-33592 147 17 features feature NOUN bracis-33592 147 18 of of ADP bracis-33592 147 19 the the DET bracis-33592 147 20 essays essay NOUN bracis-33592 147 21 . . PUNCT bracis-33592 148 1 the the DET bracis-33592 148 2 phi-3 phi-3 ADJ bracis-33592 148 3 model model NOUN bracis-33592 148 4 performs perform VERB bracis-33592 148 5 best good ADJ bracis-33592 148 6 in in ADP bracis-33592 148 7 three three NUM bracis-33592 148 8 competences competence NOUN bracis-33592 148 9 , , PUNCT bracis-33592 148 10 with with ADP bracis-33592 148 11 a a DET bracis-33592 148 12 surprising surprising ADJ bracis-33592 148 13 0.61 0.61 NUM bracis-33592 148 14 in in ADP bracis-33592 148 15 the the DET bracis-33592 148 16 fifth fifth ADJ bracis-33592 148 17 competence competence NOUN bracis-33592 148 18 . . PUNCT bracis-33592 149 1 notably notably ADV bracis-33592 149 2 , , PUNCT bracis-33592 149 3 this this DET bracis-33592 149 4 model model NOUN bracis-33592 149 5 loses lose VERB bracis-33592 149 6 twice twice ADV bracis-33592 149 7 to to ADP bracis-33592 149 8 the the DET bracis-33592 149 9 linear linear ADJ bracis-33592 149 10 regression regression NOUN bracis-33592 149 11 model model NOUN bracis-33592 149 12 , , PUNCT bracis-33592 149 13 but but CCONJ bracis-33592 149 14 it it PRON bracis-33592 149 15 is be AUX bracis-33592 149 16 always always ADV bracis-33592 149 17 at at ADP bracis-33592 149 18 least least ADJ bracis-33592 149 19 competitive competitive ADJ bracis-33592 149 20 with with ADP bracis-33592 149 21 the the DET bracis-33592 149 22 bert bert PROPN bracis-33592 149 23 baseline baseline PROPN bracis-33592 149 24 . . PUNCT bracis-33592 150 1 finally finally ADV bracis-33592 150 2 , , PUNCT bracis-33592 150 3 gemini gemini PROPN bracis-33592 150 4 is be AUX bracis-33592 150 5 the the DET bracis-33592 150 6 largest large ADJ bracis-33592 150 7 model model NOUN bracis-33592 150 8 , , PUNCT bracis-33592 150 9 with with ADP bracis-33592 150 10 an an DET bracis-33592 150 11 unknown unknown ADJ bracis-33592 150 12 size size NOUN bracis-33592 150 13 and and CCONJ bracis-33592 150 14 training training NOUN bracis-33592 150 15 set set NOUN bracis-33592 150 16 . . PUNCT bracis-33592 151 1 it it PRON bracis-33592 151 2 drew draw VERB bracis-33592 151 3 with with ADP bracis-33592 151 4 the the DET bracis-33592 151 5 linear linear ADJ bracis-33592 151 6 regressor regressor NOUN bracis-33592 151 7 for for ADP bracis-33592 151 8 the the DET bracis-33592 151 9 best good ADJ bracis-33592 151 10 performance performance NOUN bracis-33592 151 11 in in ADP bracis-33592 151 12 competence competence NOUN bracis-33592 151 13 2 2 NUM bracis-33592 151 14 and and CCONJ bracis-33592 151 15 had have VERB bracis-33592 151 16 the the DET bracis-33592 151 17 worst bad ADJ bracis-33592 151 18 performance performance NOUN bracis-33592 151 19 in in ADP bracis-33592 151 20 competence competence NOUN bracis-33592 151 21 3 3 NUM bracis-33592 151 22 . . PUNCT bracis-33592 152 1 in in ADP bracis-33592 152 2 general general ADJ bracis-33592 152 3 , , PUNCT bracis-33592 152 4 this this DET bracis-33592 152 5 model model NOUN bracis-33592 152 6 is be AUX bracis-33592 152 7 the the DET bracis-33592 152 8 most most ADV bracis-33592 152 9 stable stable ADJ bracis-33592 152 10 one one NOUN bracis-33592 152 11 , , PUNCT bracis-33592 152 12 with with ADP bracis-33592 152 13 performances performance NOUN bracis-33592 152 14 between between ADP bracis-33592 152 15 0.41 0.41 NUM bracis-33592 152 16 and and CCONJ bracis-33592 152 17 0.35 0.35 NUM bracis-33592 152 18 . . PUNCT bracis-33592 153 1 its its PRON bracis-33592 153 2 relatively relatively ADV bracis-33592 153 3 poor poor ADJ bracis-33592 153 4 performance performance NOUN bracis-33592 153 5 must must AUX bracis-33592 153 6 be be AUX bracis-33592 153 7 seen see VERB bracis-33592 153 8 with with ADP bracis-33592 153 9 a a DET bracis-33592 153 10 grain grain NOUN bracis-33592 153 11 of of ADP bracis-33592 153 12 salt salt NOUN bracis-33592 153 13 , , PUNCT bracis-33592 153 14 as as SCONJ bracis-33592 153 15 it it PRON bracis-33592 153 16 is be AUX bracis-33592 153 17 the the DET bracis-33592 153 18 only only ADJ bracis-33592 153 19 model model NOUN bracis-33592 153 20 that that PRON bracis-33592 153 21 was be AUX bracis-33592 153 22 not not PART bracis-33592 153 23 fine fine ADV bracis-33592 153 24 - - PUNCT bracis-33592 153 25 tuned tune VERB bracis-33592 153 26 to to PART bracis-33592 153 27 actually actually ADV bracis-33592 153 28 grade grade VERB bracis-33592 153 29 the the DET bracis-33592 153 30 essays essay NOUN bracis-33592 153 31 . . PUNCT bracis-33592 154 1 having having AUX bracis-33592 154 2 shown show VERB bracis-33592 154 3 that that SCONJ bracis-33592 154 4 the the DET bracis-33592 154 5 performances performance NOUN bracis-33592 154 6 between between ADP bracis-33592 154 7 the the DET bracis-33592 154 8 different different ADJ bracis-33592 154 9 transformers transformer NOUN bracis-33592 154 10 - - PUNCT bracis-33592 154 11 based base VERB bracis-33592 154 12 graders grader NOUN bracis-33592 154 13 are be AUX bracis-33592 154 14 not not PART bracis-33592 154 15 that that ADV bracis-33592 154 16 different different ADJ bracis-33592 154 17 from from ADP bracis-33592 154 18 the the DET bracis-33592 154 19 linear linear PROPN bracis-33592 154 20 regressor regressor NOUN bracis-33592 154 21 , , PUNCT bracis-33592 154 22 we we PRON bracis-33592 154 23 check check VERB bracis-33592 154 24 which which PRON bracis-33592 154 25 features feature VERB bracis-33592 154 26 receive receive VERB bracis-33592 154 27 the the DET bracis-33592 154 28 largest large ADJ bracis-33592 154 29 weights weight NOUN bracis-33592 154 30 . . PUNCT bracis-33592 155 1 the the DET bracis-33592 155 2 names name NOUN bracis-33592 155 3 of of ADP bracis-33592 155 4 the the DET bracis-33592 155 5 five five NUM bracis-33592 155 6 more more ADV bracis-33592 155 7 important important ADJ bracis-33592 155 8 ( ( PUNCT bracis-33592 155 9 positive positive ADJ bracis-33592 155 10 ) ) PUNCT bracis-33592 155 11 features feature NOUN bracis-33592 155 12 are be AUX bracis-33592 155 13 presented present VERB bracis-33592 155 14 in in ADP bracis-33592 155 15 table table NOUN bracis-33592 155 16     SPACE bracis-33592 155 17 3 3 NUM bracis-33592 155 18 . . PUNCT bracis-33592 156 1 although although SCONJ bracis-33592 156 2 named name VERB bracis-33592 156 3 with with ADP bracis-33592 156 4 a a DET bracis-33592 156 5 different different ADJ bracis-33592 156 6 pattern pattern NOUN bracis-33592 156 7 , , PUNCT bracis-33592 156 8 adverbs adverb NOUN bracis-33592 156 9 and and CCONJ bracis-33592 156 10 verbs verbs PROPN bracis-33592 156 11 are be AUX bracis-33592 156 12 features feature NOUN bracis-33592 156 13 that that PRON bracis-33592 156 14 measure measure VERB bracis-33592 156 15 the the DET bracis-33592 156 16 ratio ratio NOUN bracis-33592 156 17 of of ADP bracis-33592 156 18 such such ADJ bracis-33592 156 19 parts part NOUN bracis-33592 156 20 of of ADP bracis-33592 156 21 speech speech NOUN bracis-33592 156 22 . . PUNCT bracis-33592 157 1 content content NOUN bracis-33592 157 2 words word NOUN bracis-33592 157 3 and and CCONJ bracis-33592 157 4 function function NOUN bracis-33592 157 5 words word NOUN bracis-33592 157 6 also also ADV bracis-33592 157 7 account account VERB bracis-33592 157 8 for for ADP bracis-33592 157 9 the the DET bracis-33592 157 10 ratio ratio NOUN bracis-33592 157 11 of of ADP bracis-33592 157 12 words word NOUN bracis-33592 157 13 . . PUNCT bracis-33592 158 1 the the DET bracis-33592 158 2 former former ADJ bracis-33592 158 3 aggregates aggregate NOUN bracis-33592 158 4 nouns noun NOUN bracis-33592 158 5 , , PUNCT bracis-33592 158 6 verbs verbs PROPN bracis-33592 158 7 , , PUNCT bracis-33592 158 8 adjectives adjective NOUN bracis-33592 158 9 , , PUNCT bracis-33592 158 10 adverbs adverb NOUN bracis-33592 158 11 , , PUNCT bracis-33592 158 12 and and CCONJ bracis-33592 158 13 discourse discourse NOUN bracis-33592 158 14 markers marker NOUN bracis-33592 158 15 . . PUNCT bracis-33592 159 1 the the DET bracis-33592 159 2 latter latter ADJ bracis-33592 159 3 accounts account VERB bracis-33592 159 4 for for ADP bracis-33592 159 5 articles article NOUN bracis-33592 159 6 , , PUNCT bracis-33592 159 7 conjunctions conjunction NOUN bracis-33592 159 8 , , PUNCT bracis-33592 159 9 interjections interjection NOUN bracis-33592 159 10 , , PUNCT bracis-33592 159 11 numerals numeral NOUN bracis-33592 159 12 , , PUNCT bracis-33592 159 13 pronouns pronoun NOUN bracis-33592 159 14 , , PUNCT bracis-33592 159 15 prepositions preposition NOUN bracis-33592 159 16 , , PUNCT bracis-33592 159 17 conjunctive conjunctive ADJ bracis-33592 159 18 and and CCONJ bracis-33592 159 19 subordinating subordinate VERB bracis-33592 159 20 adverbs adverb NOUN bracis-33592 159 21 . . PUNCT bracis-33592 160 1 related relate VERB bracis-33592 160 2 to to ADP bracis-33592 160 3 the the DET bracis-33592 160 4 previous previous ADJ bracis-33592 160 5 two two NUM bracis-33592 160 6 , , PUNCT bracis-33592 160 7 content content NOUN bracis-33592 160 8 density density NOUN bracis-33592 160 9 is be AUX bracis-33592 160 10 the the DET bracis-33592 160 11 ratio ratio NOUN bracis-33592 160 12 of of ADP bracis-33592 160 13 content content ADJ bracis-33592 160 14 words word NOUN bracis-33592 160 15 to to ADP bracis-33592 160 16 functional functional ADJ bracis-33592 160 17 words word NOUN bracis-33592 160 18 . . PUNCT bracis-33592 161 1 finally finally ADV bracis-33592 161 2 , , PUNCT bracis-33592 161 3 cau cau PROPN bracis-33592 161 4 neg neg PROPN bracis-33592 161 5 conn conn PROPN bracis-33592 161 6 ratio ratio PROPN bracis-33592 161 7 is be AUX bracis-33592 161 8 the the DET bracis-33592 161 9 ratio ratio NOUN bracis-33592 161 10 of of ADP bracis-33592 161 11 negative negative ADJ bracis-33592 161 12 causal causal ADJ bracis-33592 161 13 connectives connective NOUN bracis-33592 161 14 . . PUNCT bracis-33592 162 1 competences competence NOUN bracis-33592 162 2 1 1 NUM bracis-33592 162 3 to to PART bracis-33592 162 4 3 3 NUM bracis-33592 162 5 have have VERB bracis-33592 162 6 the the DET bracis-33592 162 7 same same ADJ bracis-33592 162 8 set set NOUN bracis-33592 162 9 in in ADP bracis-33592 162 10 the the DET bracis-33592 162 11 same same ADJ bracis-33592 162 12 order order NOUN bracis-33592 162 13 , , PUNCT bracis-33592 162 14 which which PRON bracis-33592 162 15 does do AUX bracis-33592 162 16 not not PART bracis-33592 162 17 match match VERB bracis-33592 162 18 what what PRON bracis-33592 162 19 is be AUX bracis-33592 162 20 expected expect VERB bracis-33592 162 21 from from ADP bracis-33592 162 22 the the DET bracis-33592 162 23 official official ADJ bracis-33592 162 24 grading grading NOUN bracis-33592 162 25 , , PUNCT bracis-33592 162 26 as as SCONJ bracis-33592 162 27 each each DET bracis-33592 162 28 competence competence NOUN bracis-33592 162 29 is be AUX bracis-33592 162 30 supposed suppose VERB bracis-33592 162 31 to to PART bracis-33592 162 32 evaluate evaluate VERB bracis-33592 162 33 a a DET bracis-33592 162 34 different different ADJ bracis-33592 162 35 aspect aspect NOUN bracis-33592 162 36 of of ADP bracis-33592 162 37 the the DET bracis-33592 162 38 text text NOUN bracis-33592 162 39 . . PUNCT bracis-33592 163 1 competence competence NOUN bracis-33592 163 2 5 5 NUM bracis-33592 163 3 has have VERB bracis-33592 163 4 the the DET bracis-33592 163 5 same same ADJ bracis-33592 163 6 set set NOUN bracis-33592 163 7 as as ADP bracis-33592 163 8 the the DET bracis-33592 163 9 previous previous ADJ bracis-33592 163 10 three three NUM bracis-33592 163 11 but but CCONJ bracis-33592 163 12 in in ADP bracis-33592 163 13 a a DET bracis-33592 163 14 different different ADJ bracis-33592 163 15 order order NOUN bracis-33592 163 16 . . PUNCT bracis-33592 164 1 competence competence NOUN bracis-33592 164 2 4 4 NUM bracis-33592 164 3 is be AUX bracis-33592 164 4 supposed suppose VERB bracis-33592 164 5 to to PART bracis-33592 164 6 evaluate evaluate VERB bracis-33592 164 7 the the DET bracis-33592 164 8 proper proper ADJ bracis-33592 164 9 usage usage NOUN bracis-33592 164 10 of of ADP bracis-33592 164 11 connectives connective NOUN bracis-33592 164 12 , , PUNCT bracis-33592 164 13 so so ADV bracis-33592 164 14 having have VERB bracis-33592 164 15 function function NOUN bracis-33592 164 16 words word NOUN bracis-33592 164 17 and and CCONJ bracis-33592 164 18 cau cau PROPN bracis-33592 164 19 neg neg PROPN bracis-33592 164 20 conn conn PROPN bracis-33592 164 21 ratio ratio NOUN bracis-33592 164 22 between between ADP bracis-33592 164 23 the the DET bracis-33592 164 24 top top ADJ bracis-33592 164 25 5 5 NUM bracis-33592 164 26 makes make VERB bracis-33592 164 27 sense sense NOUN bracis-33592 164 28 . . PUNCT bracis-33592 165 1 table table NOUN bracis-33592 165 2 3 3 NUM bracis-33592 165 3 . . PUNCT bracis-33592 166 1 the the DET bracis-33592 166 2 four four NUM bracis-33592 166 3 most most ADV bracis-33592 166 4 important important ADJ bracis-33592 166 5 features feature NOUN bracis-33592 166 6 for for ADP bracis-33592 166 7 each each DET bracis-33592 166 8 competence competence NOUN bracis-33592 166 9 according accord VERB bracis-33592 166 10 to to ADP bracis-33592 166 11 the the DET bracis-33592 166 12 linear linear ADJ bracis-33592 166 13 regressor.full regressor.full NOUN bracis-33592 166 14 size size NOUN bracis-33592 166 15 table table NOUN bracis-33592 166 16 with with ADP bracis-33592 166 17 this this DET bracis-33592 166 18 information information NOUN bracis-33592 166 19 , , PUNCT bracis-33592 166 20 we we PRON bracis-33592 166 21 can can AUX bracis-33592 166 22 assume assume VERB bracis-33592 166 23 that that SCONJ bracis-33592 166 24 students student NOUN bracis-33592 166 25 will will AUX bracis-33592 166 26 come come VERB bracis-33592 166 27 up up ADP bracis-33592 166 28 with with ADP bracis-33592 166 29 intuitions intuition NOUN bracis-33592 166 30 like like ADP bracis-33592 166 31 ( ( PUNCT bracis-33592 166 32 1 1 X bracis-33592 166 33 ) ) PUNCT bracis-33592 166 34 it it PRON bracis-33592 166 35 is be AUX bracis-33592 166 36 important important ADJ bracis-33592 166 37 to to PART bracis-33592 166 38 have have VERB bracis-33592 166 39 a a DET bracis-33592 166 40 lot lot NOUN bracis-33592 166 41 of of ADP bracis-33592 166 42 adverbs adverb NOUN bracis-33592 166 43 , , PUNCT bracis-33592 166 44 as as SCONJ bracis-33592 166 45 it it PRON bracis-33592 166 46 is be AUX bracis-33592 166 47 the the DET bracis-33592 166 48 most most ADV bracis-33592 166 49 important important ADJ bracis-33592 166 50 feature feature NOUN bracis-33592 166 51 in in ADP bracis-33592 166 52 3 3 NUM bracis-33592 166 53 competencies competency NOUN bracis-33592 166 54 , , PUNCT bracis-33592 166 55 it it PRON bracis-33592 166 56 is be AUX bracis-33592 166 57 part part NOUN bracis-33592 166 58 of of ADP bracis-33592 166 59 the the DET bracis-33592 166 60 top top ADJ bracis-33592 166 61 feature feature NOUN bracis-33592 166 62 in in ADP bracis-33592 166 63 competence competence NOUN bracis-33592 166 64 4 4 NUM bracis-33592 166 65 and and CCONJ bracis-33592 166 66 is be AUX bracis-33592 166 67 the the DET bracis-33592 166 68 third third ADJ bracis-33592 166 69 most most ADV bracis-33592 166 70 important important ADJ bracis-33592 166 71 in in ADP bracis-33592 166 72 competence competence NOUN bracis-33592 166 73 5 5 NUM bracis-33592 166 74 ; ; PUNCT bracis-33592 166 75 ( ( PUNCT bracis-33592 166 76 2 2 X bracis-33592 166 77 ) ) PUNCT bracis-33592 166 78 it it PRON bracis-33592 166 79 is be AUX bracis-33592 166 80 important important ADJ bracis-33592 166 81 to to PART bracis-33592 166 82 have have VERB bracis-33592 166 83 a a DET bracis-33592 166 84 lot lot NOUN bracis-33592 166 85 of of ADP bracis-33592 166 86 adjectives adjective NOUN bracis-33592 166 87 , , PUNCT bracis-33592 166 88 as as SCONJ bracis-33592 166 89 it it PRON bracis-33592 166 90 is be AUX bracis-33592 166 91 the the DET bracis-33592 166 92 most most ADV bracis-33592 166 93 important important ADJ bracis-33592 166 94 in in ADP bracis-33592 166 95 one one NUM bracis-33592 166 96 competence competence NOUN bracis-33592 166 97 , , PUNCT bracis-33592 166 98 the the DET bracis-33592 166 99 second second ADJ bracis-33592 166 100 most most ADV bracis-33592 166 101 important important ADJ bracis-33592 166 102 in in ADP bracis-33592 166 103 three three NUM bracis-33592 166 104 , , PUNCT bracis-33592 166 105 and and CCONJ bracis-33592 166 106 part part NOUN bracis-33592 166 107 of of ADP bracis-33592 166 108 the the DET bracis-33592 166 109 top top ADJ bracis-33592 166 110 feature feature NOUN bracis-33592 166 111 in in ADP bracis-33592 166 112 the the DET bracis-33592 166 113 other other ADJ bracis-33592 166 114 . . PUNCT bracis-33592 167 1 4.2 4.2 NUM bracis-33592 167 2 defining define VERB bracis-33592 167 3 the the DET bracis-33592 167 4     SPACE bracis-33592 167 5 ( ( PUNCT bracis-33592 167 6 universal universal ADJ bracis-33592 167 7 ) ) PUNCT bracis-33592 167 8 adversarial adversarial ADJ bracis-33592 167 9 attacks attack NOUN bracis-33592 167 10 bearing bear VERB bracis-33592 167 11 the the DET bracis-33592 167 12 previous previous ADJ bracis-33592 167 13 intuitions intuition NOUN bracis-33592 167 14 in in ADP bracis-33592 167 15 mind mind NOUN bracis-33592 167 16 , , PUNCT bracis-33592 167 17 we we PRON bracis-33592 167 18 can can AUX bracis-33592 167 19 devise devise VERB bracis-33592 167 20 three three NUM bracis-33592 167 21 goals goal NOUN bracis-33592 167 22 : : PUNCT bracis-33592 167 23 increase increase VERB bracis-33592 167 24 the the DET bracis-33592 167 25 number number NOUN bracis-33592 167 26 of of ADP bracis-33592 167 27 adverbs adverb NOUN bracis-33592 167 28 , , PUNCT bracis-33592 167 29 increase increase VERB bracis-33592 167 30 the the DET bracis-33592 167 31 number number NOUN bracis-33592 167 32 of of ADP bracis-33592 167 33 adjectives adjective NOUN bracis-33592 167 34 , , PUNCT bracis-33592 167 35 and and CCONJ bracis-33592 167 36 increase increase VERB bracis-33592 167 37 both both PRON bracis-33592 167 38 . . PUNCT bracis-33592 168 1 we we PRON bracis-33592 168 2 consider consider VERB bracis-33592 168 3 that that SCONJ bracis-33592 168 4 although although SCONJ bracis-33592 168 5 the the DET bracis-33592 168 6 feature feature NOUN bracis-33592 168 7 is be AUX bracis-33592 168 8 the the DET bracis-33592 168 9 ratio ratio NOUN bracis-33592 168 10 of of ADP bracis-33592 168 11 such such ADJ bracis-33592 168 12 classes class NOUN bracis-33592 168 13 , , PUNCT bracis-33592 168 14 it it PRON bracis-33592 168 15 is be AUX bracis-33592 168 16 not not PART bracis-33592 168 17 trivial trivial ADJ bracis-33592 168 18 to to PART bracis-33592 168 19 identify identify VERB bracis-33592 168 20 that that SCONJ bracis-33592 168 21 the the DET bracis-33592 168 22 importance importance NOUN bracis-33592 168 23 is be AUX bracis-33592 168 24 the the DET bracis-33592 168 25 ratio ratio NOUN bracis-33592 168 26 and and CCONJ bracis-33592 168 27 not not PART bracis-33592 168 28 the the DET bracis-33592 168 29 total total ADJ bracis-33592 168 30 number number NOUN bracis-33592 168 31 . . PUNCT bracis-33592 169 1 to to PART bracis-33592 169 2 increase increase VERB bracis-33592 169 3 the the DET bracis-33592 169 4 number number NOUN bracis-33592 169 5 of of ADP bracis-33592 169 6 one one NUM bracis-33592 169 7 of of ADP bracis-33592 169 8 the the DET bracis-33592 169 9 classes class NOUN bracis-33592 169 10 , , PUNCT bracis-33592 169 11 we we PRON bracis-33592 169 12 can can AUX bracis-33592 169 13 consider consider VERB bracis-33592 169 14 some some DET bracis-33592 169 15 strategies strategy NOUN bracis-33592 169 16 , , PUNCT bracis-33592 169 17 such such ADJ bracis-33592 169 18 as as ADP bracis-33592 169 19 ( ( PUNCT bracis-33592 169 20 a a X bracis-33592 169 21 ) ) PUNCT bracis-33592 169 22 writing write VERB bracis-33592 169 23 a a DET bracis-33592 169 24 list list NOUN bracis-33592 169 25 of of ADP bracis-33592 169 26 words word NOUN bracis-33592 169 27 belonging belong VERB bracis-33592 169 28 to to ADP bracis-33592 169 29 this this DET bracis-33592 169 30 class class NOUN bracis-33592 169 31 , , PUNCT bracis-33592 169 32 ( ( PUNCT bracis-33592 169 33 b b NOUN bracis-33592 169 34 ) ) PUNCT bracis-33592 169 35 copying copying NOUN bracis-33592 169 36 and and CCONJ bracis-33592 169 37 pasting paste VERB bracis-33592 169 38 the the DET bracis-33592 169 39 previous previous ADJ bracis-33592 169 40 list list NOUN bracis-33592 169 41 so so SCONJ bracis-33592 169 42 that that SCONJ bracis-33592 169 43 it it PRON bracis-33592 169 44 becomes become VERB bracis-33592 169 45 similar similar ADJ bracis-33592 169 46 to to ADP bracis-33592 169 47 the the DET bracis-33592 169 48 four four NUM bracis-33592 169 49 paragraphs paragraph NOUN bracis-33592 169 50 structure structure NOUN bracis-33592 169 51 ; ; PUNCT bracis-33592 169 52 ( ( PUNCT bracis-33592 169 53 c c X bracis-33592 169 54 ) ) PUNCT bracis-33592 169 55 similar similar ADJ bracis-33592 169 56 to to ADP bracis-33592 169 57 the the DET bracis-33592 169 58 previous previous ADJ bracis-33592 169 59 , , PUNCT bracis-33592 169 60 but but CCONJ bracis-33592 169 61 with with ADP bracis-33592 169 62 more more ADJ bracis-33592 169 63 copying copying NOUN bracis-33592 169 64 and and CCONJ bracis-33592 169 65 pasting pasting NOUN bracis-33592 169 66 and and CCONJ bracis-33592 169 67 using use VERB bracis-33592 169 68 a a DET bracis-33592 169 69 memorized memorized ADJ bracis-33592 169 70 sentence sentence NOUN bracis-33592 169 71 that that PRON bracis-33592 169 72 uses use VERB bracis-33592 169 73 many many ADJ bracis-33592 169 74 of of ADP bracis-33592 169 75 the the DET bracis-33592 169 76 desired desire VERB bracis-33592 169 77 part part NOUN bracis-33592 169 78 of of ADP bracis-33592 169 79 speech speech NOUN bracis-33592 169 80 . . PUNCT bracis-33592 170 1 we we PRON bracis-33592 170 2 have have VERB bracis-33592 170 3 3 3 NUM bracis-33592 170 4 goals goal NOUN bracis-33592 170 5 , , PUNCT bracis-33592 170 6 and and CCONJ bracis-33592 170 7 each each DET bracis-33592 170 8 goal goal NOUN bracis-33592 170 9 can can AUX bracis-33592 170 10 be be AUX bracis-33592 170 11 reached reach VERB bracis-33592 170 12 by by ADP bracis-33592 170 13 3 3 NUM bracis-33592 170 14 strategies strategy NOUN bracis-33592 170 15 , , PUNCT bracis-33592 170 16 so so SCONJ bracis-33592 170 17 we we PRON bracis-33592 170 18 can can AUX bracis-33592 170 19 define define VERB bracis-33592 170 20 9 9 NUM bracis-33592 170 21 adversarial adversarial ADJ bracis-33592 170 22 attacks attack NOUN bracis-33592 170 23 : : PUNCT bracis-33592 170 24 attack attack NOUN bracis-33592 170 25 1a 1a NOUN bracis-33592 170 26 : : PUNCT bracis-33592 170 27 : : PUNCT bracis-33592 170 28 we we PRON bracis-33592 170 29 devise devise VERB bracis-33592 170 30 a a DET bracis-33592 170 31 list list NOUN bracis-33592 170 32 of of ADP bracis-33592 170 33 adverbs adverb NOUN bracis-33592 170 34 , , PUNCT bracis-33592 170 35 such such ADJ bracis-33592 170 36 as as ADP bracis-33592 170 37 : : PUNCT bracis-33592 170 38 “ " PUNCT bracis-33592 170 39 well well INTJ bracis-33592 170 40 , , PUNCT bracis-33592 170 41 badly badly ADV bracis-33592 170 42 , , PUNCT bracis-33592 170 43 enormously enormously ADV bracis-33592 170 44 , , PUNCT bracis-33592 170 45 smally smally ADV bracis-33592 170 46 , , PUNCT bracis-33592 170 47 certainly certainly ADV bracis-33592 170 48 , , PUNCT bracis-33592 170 49 wrongly wrongly ADV bracis-33592 170 50 , , PUNCT bracis-33592 170 51 rapidly rapidly ADV bracis-33592 170 52 , , PUNCT bracis-33592 170 53 slowly slowly ADV bracis-33592 170 54 , , PUNCT bracis-33592 170 55 fairly fairly ADV bracis-33592 170 56 , , PUNCT bracis-33592 170 57 unfairly unfairly ADV bracis-33592 170 58 " " PUNCT bracis-33592 170 59 . . PUNCT bracis-33592 171 1 attack attack NOUN bracis-33592 171 2 1b 1b NUM bracis-33592 171 3 : : PUNCT bracis-33592 171 4 : : PUNCT bracis-33592 171 5 we we PRON bracis-33592 171 6 repeat repeat VERB bracis-33592 171 7 the the DET bracis-33592 171 8 previous previous ADJ bracis-33592 171 9 list list NOUN bracis-33592 171 10 four four NUM bracis-33592 171 11 times time NOUN bracis-33592 171 12 , , PUNCT bracis-33592 171 13 one one NUM bracis-33592 171 14 in in ADP bracis-33592 171 15 each each DET bracis-33592 171 16 paragraph paragraph NOUN bracis-33592 171 17 . . PUNCT bracis-33592 172 1 attack attack NOUN bracis-33592 172 2 1c 1c NOUN bracis-33592 172 3 : : PUNCT bracis-33592 172 4 : : PUNCT bracis-33592 172 5 we we PRON bracis-33592 172 6 create create VERB bracis-33592 172 7 a a DET bracis-33592 172 8 sentence sentence NOUN bracis-33592 172 9 that that PRON bracis-33592 172 10 uses use VERB bracis-33592 172 11 more more ADJ bracis-33592 172 12 adverbs adverb NOUN bracis-33592 172 13 than than ADP bracis-33592 172 14 usual usual ADJ bracis-33592 172 15 , , PUNCT bracis-33592 172 16 such such ADJ bracis-33592 172 17 as as ADP bracis-33592 172 18 : : PUNCT bracis-33592 172 19 “ " PUNCT bracis-33592 172 20 undeniably undeniably ADV bracis-33592 172 21 , , PUNCT bracis-33592 172 22 progressing progress VERB bracis-33592 172 23 slowly slowly ADV bracis-33592 172 24 , , PUNCT bracis-33592 172 25 leisurely leisurely ADV bracis-33592 172 26 , , PUNCT bracis-33592 172 27 carefully carefully ADV bracis-33592 172 28 , , PUNCT bracis-33592 172 29 silently silently ADV bracis-33592 172 30 while while SCONJ bracis-33592 172 31 deeply deeply ADV bracis-33592 172 32 breathing breathe VERB bracis-33592 172 33 and and CCONJ bracis-33592 172 34 thinking think VERB bracis-33592 172 35 intensively intensively ADV bracis-33592 172 36 about about ADP bracis-33592 172 37 the the DET bracis-33592 172 38 given give VERB bracis-33592 172 39 problem problem NOUN bracis-33592 172 40 " " PUNCT bracis-33592 172 41 , , PUNCT bracis-33592 172 42 copy copy VERB bracis-33592 172 43 it it PRON bracis-33592 172 44 10 10 NUM bracis-33592 172 45 times time NOUN bracis-33592 172 46 in in ADP bracis-33592 172 47 each each DET bracis-33592 172 48 paragraph paragraph NOUN bracis-33592 172 49 , , PUNCT bracis-33592 172 50 in in ADP bracis-33592 172 51 4 4 NUM bracis-33592 172 52 paragraphs paragraph NOUN bracis-33592 172 53 . . PUNCT bracis-33592 173 1 attack attack NOUN bracis-33592 173 2 2a 2a NUM bracis-33592 173 3 : : PUNCT bracis-33592 173 4 : : PUNCT bracis-33592 173 5 we we PRON bracis-33592 173 6 devise devise VERB bracis-33592 173 7 a a DET bracis-33592 173 8 list list NOUN bracis-33592 173 9 of of ADP bracis-33592 173 10 adjectives adjective NOUN bracis-33592 173 11 , , PUNCT bracis-33592 173 12 such such ADJ bracis-33592 173 13 as as ADP bracis-33592 173 14 : : PUNCT bracis-33592 173 15 “ " PUNCT bracis-33592 173 16 good good ADJ bracis-33592 173 17 , , PUNCT bracis-33592 173 18 bad bad ADJ bracis-33592 173 19 , , PUNCT bracis-33592 173 20 big big ADJ bracis-33592 173 21 , , PUNCT bracis-33592 173 22 small small ADJ bracis-33592 173 23 , , PUNCT bracis-33592 173 24 best good ADJ bracis-33592 173 25 , , PUNCT bracis-33592 173 26 worst bad ADJ bracis-33592 173 27 , , PUNCT bracis-33592 173 28 right right ADJ bracis-33592 173 29 , , PUNCT bracis-33592 173 30 wrong wrong ADJ bracis-33592 173 31 , , PUNCT bracis-33592 173 32 last last ADJ bracis-33592 173 33 , , PUNCT bracis-33592 173 34 first first ADJ bracis-33592 173 35 , , PUNCT bracis-33592 173 36 fair fair ADJ bracis-33592 173 37 , , PUNCT bracis-33592 173 38 unfair unfair ADJ bracis-33592 173 39 " " PUNCT bracis-33592 173 40 . . PUNCT bracis-33592 174 1 attack attack NOUN bracis-33592 174 2 2b 2b NOUN bracis-33592 174 3 : : PUNCT bracis-33592 174 4 : : PUNCT bracis-33592 174 5 we we PRON bracis-33592 174 6 repeat repeat VERB bracis-33592 174 7 the the DET bracis-33592 174 8 previous previous ADJ bracis-33592 174 9 list list NOUN bracis-33592 174 10 four four NUM bracis-33592 174 11 times time NOUN bracis-33592 174 12 , , PUNCT bracis-33592 174 13 one one NUM bracis-33592 174 14 in in ADP bracis-33592 174 15 each each DET bracis-33592 174 16 paragraph paragraph NOUN bracis-33592 174 17 . . PUNCT bracis-33592 175 1 attack attack NOUN bracis-33592 175 2 2c 2c NUM bracis-33592 175 3 : : PUNCT bracis-33592 175 4 : : PUNCT bracis-33592 175 5 we we PRON bracis-33592 175 6 create create VERB bracis-33592 175 7 a a DET bracis-33592 175 8 sentence sentence NOUN bracis-33592 175 9 that that PRON bracis-33592 175 10 uses use VERB bracis-33592 175 11 more more ADJ bracis-33592 175 12 adjectives adjective NOUN bracis-33592 175 13 than than ADP bracis-33592 175 14 usual usual ADJ bracis-33592 175 15 , , PUNCT bracis-33592 175 16 such such ADJ bracis-33592 175 17 as as ADP bracis-33592 175 18 : : PUNCT bracis-33592 175 19 “ " PUNCT bracis-33592 175 20 the the DET bracis-33592 175 21 constant constant ADJ bracis-33592 175 22 and and CCONJ bracis-33592 175 23 innovative innovative ADJ bracis-33592 175 24 development development NOUN bracis-33592 175 25 of of ADP bracis-33592 175 26 modern modern ADJ bracis-33592 175 27 technology technology NOUN bracis-33592 175 28 has have AUX bracis-33592 175 29 brought bring VERB bracis-33592 175 30 significant significant ADJ bracis-33592 175 31 and and CCONJ bracis-33592 175 32 deep deep ADJ bracis-33592 175 33 changes change NOUN bracis-33592 175 34 to to ADP bracis-33592 175 35 this this DET bracis-33592 175 36 issue issue NOUN bracis-33592 175 37 , , PUNCT bracis-33592 175 38 creating create VERB bracis-33592 175 39 diverse diverse ADJ bracis-33592 175 40 and and CCONJ bracis-33592 175 41 thrilling thrilling ADJ bracis-33592 175 42 opportunities opportunity NOUN bracis-33592 175 43 for for ADP bracis-33592 175 44 a a DET bracis-33592 175 45 more more ADV bracis-33592 175 46 promising promising ADJ bracis-33592 175 47 and and CCONJ bracis-33592 175 48 sustainable sustainable ADJ bracis-33592 175 49 future future NOUN bracis-33592 175 50 . . PUNCT bracis-33592 175 51 " " PUNCT bracis-33592 176 1 then then ADV bracis-33592 176 2 we we PRON bracis-33592 176 3 copy copy VERB bracis-33592 176 4 it it PRON bracis-33592 176 5 10 10 NUM bracis-33592 176 6 times time NOUN bracis-33592 176 7 in in ADP bracis-33592 176 8 each each DET bracis-33592 176 9 paragraph paragraph NOUN bracis-33592 176 10 , , PUNCT bracis-33592 176 11 in in ADP bracis-33592 176 12 4 4 NUM bracis-33592 176 13 paragraphs paragraph NOUN bracis-33592 176 14 . . PUNCT bracis-33592 177 1 attack attack NOUN bracis-33592 177 2 3a 3a NUM bracis-33592 177 3 : : PUNCT bracis-33592 177 4 : : PUNCT bracis-33592 177 5 we we PRON bracis-33592 177 6 concatenate concatenate VERB bracis-33592 177 7 the the DET bracis-33592 177 8 sentences sentence NOUN bracis-33592 177 9 of of ADP bracis-33592 177 10 1a 1a NOUN bracis-33592 177 11 and and CCONJ bracis-33592 177 12 2a 2a NUM bracis-33592 177 13 . . PUNCT bracis-33592 178 1 attack attack NOUN bracis-33592 178 2 3b 3b NUM bracis-33592 178 3 : : PUNCT bracis-33592 178 4 : : PUNCT bracis-33592 178 5 we we PRON bracis-33592 178 6 repeat repeat VERB bracis-33592 178 7 the the DET bracis-33592 178 8 previous previous ADJ bracis-33592 178 9 list list NOUN bracis-33592 178 10 four four NUM bracis-33592 178 11 times time NOUN bracis-33592 178 12 , , PUNCT bracis-33592 178 13 one one NUM bracis-33592 178 14 in in ADP bracis-33592 178 15 each each DET bracis-33592 178 16 paragraph paragraph NOUN bracis-33592 178 17 . . PUNCT bracis-33592 179 1 attack attack NOUN bracis-33592 179 2 3c 3c NUM bracis-33592 179 3 : : PUNCT bracis-33592 179 4 : : PUNCT bracis-33592 179 5 we we PRON bracis-33592 179 6 create create VERB bracis-33592 179 7 a a DET bracis-33592 179 8 sentence sentence NOUN bracis-33592 179 9 that that PRON bracis-33592 179 10 uses use VERB bracis-33592 179 11 more more ADJ bracis-33592 179 12 adjectives adjective NOUN bracis-33592 179 13 and and CCONJ bracis-33592 179 14 adverbs adverb NOUN bracis-33592 179 15 than than ADP bracis-33592 179 16 usual usual ADJ bracis-33592 179 17 , , PUNCT bracis-33592 179 18 such such ADJ bracis-33592 179 19 as as ADP bracis-33592 179 20 : : PUNCT bracis-33592 179 21 “ " PUNCT bracis-33592 179 22 consequently consequently ADV bracis-33592 179 23 and and CCONJ bracis-33592 179 24 undeniably undeniably ADV bracis-33592 179 25 , , PUNCT bracis-33592 179 26 the the DET bracis-33592 179 27 constant constant ADJ bracis-33592 179 28 innovative innovative ADJ bracis-33592 179 29 development development NOUN bracis-33592 179 30 slowly slowly ADV bracis-33592 179 31 and and CCONJ bracis-33592 179 32 progressively progressively ADV bracis-33592 179 33 brought bring VERB bracis-33592 179 34 some some DET bracis-33592 179 35 significant significant ADJ bracis-33592 179 36 and and CCONJ bracis-33592 179 37 deep deep ADJ bracis-33592 179 38 changes change NOUN bracis-33592 179 39 to to PART bracis-33592 179 40 constantly constantly ADV bracis-33592 179 41 needed need VERB bracis-33592 179 42 problems problem NOUN bracis-33592 179 43 " " PUNCT bracis-33592 179 44 , , PUNCT bracis-33592 179 45 then then ADV bracis-33592 179 46 we we PRON bracis-33592 179 47 copy copy VERB bracis-33592 179 48 it it PRON bracis-33592 179 49 10 10 NUM bracis-33592 179 50 times time NOUN bracis-33592 179 51 in in ADP bracis-33592 179 52 each each DET bracis-33592 179 53 paragraph paragraph NOUN bracis-33592 179 54 , , PUNCT bracis-33592 179 55 in in ADP bracis-33592 179 56 4 4 NUM bracis-33592 179 57 paragraphs paragraph NOUN bracis-33592 179 58 . . PUNCT bracis-33592 180 1 4.3 4.3 NUM bracis-33592 180 2 testing test VERB bracis-33592 180 3 the the DET bracis-33592 180 4     SPACE bracis-33592 180 5 attacks attack NOUN bracis-33592 180 6 we we PRON bracis-33592 180 7 now now ADV bracis-33592 180 8 use use VERB bracis-33592 180 9 the the DET bracis-33592 180 10 sentences sentence NOUN bracis-33592 180 11 created create VERB bracis-33592 180 12 for for ADP bracis-33592 180 13 each each DET bracis-33592 180 14 attack attack NOUN bracis-33592 180 15 and and CCONJ bracis-33592 180 16 submit submit VERB bracis-33592 180 17 them they PRON bracis-33592 180 18 to to ADP bracis-33592 180 19 the the DET bracis-33592 180 20 models model NOUN bracis-33592 180 21 . . PUNCT bracis-33592 181 1 the the DET bracis-33592 181 2 grades grade NOUN bracis-33592 181 3 assigned assign VERB bracis-33592 181 4 by by ADP bracis-33592 181 5 each each DET bracis-33592 181 6 model model NOUN bracis-33592 181 7 are be AUX bracis-33592 181 8 displayed display VERB bracis-33592 181 9 in in ADP bracis-33592 181 10 table table NOUN bracis-33592 181 11     SPACE bracis-33592 181 12 4 4 NUM bracis-33592 181 13 . . PUNCT bracis-33592 182 1 schools school NOUN bracis-33592 182 2 in in ADP bracis-33592 182 3 brazil brazil PROPN bracis-33592 182 4 usually usually ADV bracis-33592 182 5 require require VERB bracis-33592 182 6 students student NOUN bracis-33592 182 7 to to PART bracis-33592 182 8 get get VERB bracis-33592 182 9 at at ADV bracis-33592 182 10 least least ADV bracis-33592 182 11 6 6 NUM bracis-33592 182 12 or or CCONJ bracis-33592 182 13 7 7 NUM bracis-33592 182 14 to to PART bracis-33592 182 15 be be AUX bracis-33592 182 16 approved approve VERB bracis-33592 182 17 , , PUNCT bracis-33592 182 18 so so SCONJ bracis-33592 182 19 we we PRON bracis-33592 182 20 can can AUX bracis-33592 182 21 assume assume VERB bracis-33592 182 22 that that SCONJ bracis-33592 182 23 an an DET bracis-33592 182 24 adversarial adversarial ADJ bracis-33592 182 25 student student NOUN bracis-33592 182 26 would would AUX bracis-33592 182 27 be be AUX bracis-33592 182 28 happy happy ADJ bracis-33592 182 29 with with ADP bracis-33592 182 30 getting get VERB bracis-33592 182 31 a a DET bracis-33592 182 32 600 600 NUM bracis-33592 182 33 or or CCONJ bracis-33592 182 34 higher high ADJ bracis-33592 182 35 with with ADP bracis-33592 182 36 the the DET bracis-33592 182 37 lowest low ADJ bracis-33592 182 38 possible possible ADJ bracis-33592 182 39 effort effort NOUN bracis-33592 182 40 , , PUNCT bracis-33592 182 41 and and CCONJ bracis-33592 182 42 this this PRON bracis-33592 182 43 will will AUX bracis-33592 182 44 count count VERB bracis-33592 182 45 as as ADP bracis-33592 182 46 fooling fool VERB bracis-33592 182 47 the the DET bracis-33592 182 48 model model NOUN bracis-33592 182 49 . . PUNCT bracis-33592 183 1 table table NOUN bracis-33592 183 2 4 4 NUM bracis-33592 183 3 . . PUNCT bracis-33592 184 1 grades grade NOUN bracis-33592 184 2 assigned assign VERB bracis-33592 184 3 by by ADP bracis-33592 184 4 all all DET bracis-33592 184 5 models model NOUN bracis-33592 184 6 , , PUNCT bracis-33592 184 7 in in ADP bracis-33592 184 8 all all DET bracis-33592 184 9 competences competence NOUN bracis-33592 184 10 , , PUNCT bracis-33592 184 11 to to ADP bracis-33592 184 12 each each DET bracis-33592 184 13 attack.full attack.full NUM bracis-33592 184 14 size size NOUN bracis-33592 184 15 table table NOUN bracis-33592 184 16 we we PRON bracis-33592 184 17 can can AUX bracis-33592 184 18 see see VERB bracis-33592 184 19 that that SCONJ bracis-33592 184 20 our our PRON bracis-33592 184 21 simplest simple ADJ bracis-33592 184 22 attacks attack NOUN bracis-33592 184 23 , , PUNCT bracis-33592 184 24 the the DET bracis-33592 184 25 ones one NOUN bracis-33592 184 26 with with ADP bracis-33592 184 27 suffix suffix PROPN bracis-33592 184 28 a a PRON bracis-33592 184 29 , , PUNCT bracis-33592 184 30 are be AUX bracis-33592 184 31 sufficient sufficient ADJ bracis-33592 184 32 to to PART bracis-33592 184 33 fool fool VERB bracis-33592 184 34 a a DET bracis-33592 184 35 simple simple ADJ bracis-33592 184 36 linear linear ADJ bracis-33592 184 37 regressor regressor NOUN bracis-33592 184 38 on on ADP bracis-33592 184 39 all all PRON bracis-33592 184 40 but but SCONJ bracis-33592 184 41 the the DET bracis-33592 184 42 competence competence NOUN bracis-33592 184 43 5 5 NUM bracis-33592 184 44 . . PUNCT bracis-33592 185 1 this this PRON bracis-33592 185 2 is be AUX bracis-33592 185 3 not not PART bracis-33592 185 4 surprising surprising ADJ bracis-33592 185 5 , , PUNCT bracis-33592 185 6 as as SCONJ bracis-33592 185 7 the the DET bracis-33592 185 8 attacks attack NOUN bracis-33592 185 9 were be AUX bracis-33592 185 10 defined define VERB bracis-33592 185 11 according accord VERB bracis-33592 185 12 to to ADP bracis-33592 185 13 most most ADJ bracis-33592 185 14 correlated correlate VERB bracis-33592 185 15 predictor predictor NOUN bracis-33592 185 16 variables variable NOUN bracis-33592 185 17 . . PUNCT bracis-33592 186 1 changing change VERB bracis-33592 186 2 from from ADP bracis-33592 186 3 one one NUM bracis-33592 186 4 sentence sentence NOUN bracis-33592 186 5 to to ADP bracis-33592 186 6 four four NUM bracis-33592 186 7 paragraphs paragraph NOUN bracis-33592 186 8 ( ( PUNCT bracis-33592 186 9 i.e. i.e. X bracis-33592 186 10 , , PUNCT bracis-33592 186 11 changing change VERB bracis-33592 186 12 an an DET bracis-33592 186 13 attack attack NOUN bracis-33592 186 14 from from ADP bracis-33592 186 15 type type NOUN bracis-33592 186 16 a a DET bracis-33592 186 17 to to ADP bracis-33592 186 18 type type NOUN bracis-33592 186 19 b b NOUN bracis-33592 186 20 ) ) PUNCT bracis-33592 186 21 did do AUX bracis-33592 186 22 not not PART bracis-33592 186 23 increase increase VERB bracis-33592 186 24 the the DET bracis-33592 186 25 score score NOUN bracis-33592 186 26 , , PUNCT bracis-33592 186 27 while while SCONJ bracis-33592 186 28 changing change VERB bracis-33592 186 29 from from ADP bracis-33592 186 30 type type NOUN bracis-33592 186 31 b b NOUN bracis-33592 186 32 to to PART bracis-33592 186 33 type type NOUN bracis-33592 186 34 c c NOUN bracis-33592 186 35 did do AUX bracis-33592 186 36 improve improve VERB bracis-33592 186 37 the the DET bracis-33592 186 38 score score NOUN bracis-33592 186 39 . . PUNCT bracis-33592 187 1 we we PRON bracis-33592 187 2 note note VERB bracis-33592 187 3 that that SCONJ bracis-33592 187 4 either either CCONJ bracis-33592 187 5 changing change VERB bracis-33592 187 6 from from ADP bracis-33592 187 7 type type NOUN bracis-33592 187 8 a a PRON bracis-33592 187 9 to to ADP bracis-33592 187 10 type type NOUN bracis-33592 187 11 b b NOUN bracis-33592 187 12 or or CCONJ bracis-33592 187 13 from from ADP bracis-33592 187 14 type type NOUN bracis-33592 187 15 b b NOUN bracis-33592 187 16 to to ADP bracis-33592 187 17 c c PROPN bracis-33592 187 18 leads lead VERB bracis-33592 187 19 to to ADP bracis-33592 187 20 a a DET bracis-33592 187 21 significant significant ADJ bracis-33592 187 22 increase increase NOUN bracis-33592 187 23 of of ADP bracis-33592 187 24 the the DET bracis-33592 187 25 text text NOUN bracis-33592 187 26 length length NOUN bracis-33592 187 27 . . PUNCT bracis-33592 188 1 the the DET bracis-33592 188 2 number number NOUN bracis-33592 188 3 of of ADP bracis-33592 188 4 words word NOUN bracis-33592 188 5 and and CCONJ bracis-33592 188 6 paragraphs paragraph NOUN bracis-33592 188 7 are be AUX bracis-33592 188 8 features feature NOUN bracis-33592 188 9 available available ADJ bracis-33592 188 10 to to ADP bracis-33592 188 11 the the DET bracis-33592 188 12 model model NOUN bracis-33592 188 13 and and CCONJ bracis-33592 188 14 have have VERB bracis-33592 188 15 a a DET bracis-33592 188 16 positive positive ADJ bracis-33592 188 17 coefficient coefficient NOUN bracis-33592 188 18 , , PUNCT bracis-33592 188 19 so so ADV bracis-33592 188 20 increasing increase VERB bracis-33592 188 21 them they PRON bracis-33592 188 22 naturally naturally ADV bracis-33592 188 23 increases increase VERB bracis-33592 188 24 the the DET bracis-33592 188 25 grade grade NOUN bracis-33592 188 26 . . PUNCT bracis-33592 189 1 additionally additionally ADV bracis-33592 189 2 , , PUNCT bracis-33592 189 3 as as SCONJ bracis-33592 189 4 this this DET bracis-33592 189 5 linear linear ADJ bracis-33592 189 6 regressor regressor NOUN bracis-33592 189 7 has have VERB bracis-33592 189 8 a a DET bracis-33592 189 9 limit limit NOUN bracis-33592 189 10 of of ADP bracis-33592 189 11 2k 2k NUM bracis-33592 189 12 words word NOUN bracis-33592 189 13 , , PUNCT bracis-33592 189 14 it it PRON bracis-33592 189 15 is be AUX bracis-33592 189 16 easy easy ADJ bracis-33592 189 17 to to PART bracis-33592 189 18 get get VERB bracis-33592 189 19 a a DET bracis-33592 189 20 full full ADJ bracis-33592 189 21 grade grade NOUN bracis-33592 189 22 by by ADP bracis-33592 189 23 increasing increase VERB bracis-33592 189 24 certain certain ADJ bracis-33592 189 25 word word NOUN bracis-33592 189 26 frequencies frequency NOUN bracis-33592 189 27 . . PUNCT bracis-33592 190 1 more more ADV bracis-33592 190 2 interestingly interestingly ADV bracis-33592 190 3 , , PUNCT bracis-33592 190 4 the the DET bracis-33592 190 5 bert bert PROPN bracis-33592 190 6 grader grader NOUN bracis-33592 190 7 was be AUX bracis-33592 190 8 not not PART bracis-33592 190 9 completely completely ADV bracis-33592 190 10 fooled fool VERB bracis-33592 190 11 by by ADP bracis-33592 190 12 the the DET bracis-33592 190 13 simplest simple ADJ bracis-33592 190 14 attacks attack NOUN bracis-33592 190 15 ; ; PUNCT bracis-33592 190 16 it it PRON bracis-33592 190 17 assigned assign VERB bracis-33592 190 18 attacks attack NOUN bracis-33592 190 19 1a 1a NOUN bracis-33592 190 20 and and CCONJ bracis-33592 190 21 2a 2a NUM bracis-33592 190 22 with with ADP bracis-33592 190 23 360 360 NUM bracis-33592 190 24 points point NOUN bracis-33592 190 25 , , PUNCT bracis-33592 190 26 and and CCONJ bracis-33592 190 27 attack attack NOUN bracis-33592 190 28 3a 3a NUM bracis-33592 190 29 with with ADP bracis-33592 190 30 400 400 NUM bracis-33592 190 31 points point NOUN bracis-33592 190 32 . . PUNCT bracis-33592 191 1 this this PRON bracis-33592 191 2 suggests suggest VERB bracis-33592 191 3 that that SCONJ bracis-33592 191 4 bert bert NOUN bracis-33592 191 5 - - PUNCT bracis-33592 191 6 based base VERB bracis-33592 191 7 predictors predictor NOUN bracis-33592 191 8 are be AUX bracis-33592 191 9 either either ADV bracis-33592 191 10 sensitive sensitive ADJ bracis-33592 191 11 to to ADP bracis-33592 191 12 diversity diversity NOUN bracis-33592 191 13 in in ADP bracis-33592 191 14 parts part NOUN bracis-33592 191 15 of of ADP bracis-33592 191 16 speech speech NOUN bracis-33592 191 17 or or CCONJ bracis-33592 191 18 that that SCONJ bracis-33592 191 19 their their PRON bracis-33592 191 20 inner inner ADJ bracis-33592 191 21 representation representation NOUN bracis-33592 191 22 considers consider VERB bracis-33592 191 23 word word NOUN bracis-33592 191 24 frequencies frequency NOUN bracis-33592 191 25 . . PUNCT bracis-33592 192 1 changing change VERB bracis-33592 192 2 from from ADP bracis-33592 192 3 attack attack NOUN bracis-33592 192 4 type type NOUN bracis-33592 192 5 a a DET bracis-33592 192 6 to to ADP bracis-33592 192 7 b b NOUN bracis-33592 192 8 increased increase VERB bracis-33592 192 9 scores score NOUN bracis-33592 192 10 , , PUNCT bracis-33592 192 11 which which PRON bracis-33592 192 12 could could AUX bracis-33592 192 13 point point VERB bracis-33592 192 14 to to ADP bracis-33592 192 15 the the DET bracis-33592 192 16 fact fact NOUN bracis-33592 192 17 that that SCONJ bracis-33592 192 18 bert bert PROPN bracis-33592 192 19 is be AUX bracis-33592 192 20 sensitive sensitive ADJ bracis-33592 192 21 to to ADP bracis-33592 192 22 the the DET bracis-33592 192 23 four four NUM bracis-33592 192 24 paragraph paragraph NOUN bracis-33592 192 25 structure structure NOUN bracis-33592 192 26 — — PUNCT bracis-33592 192 27 this this PRON bracis-33592 192 28 could could AUX bracis-33592 192 29 be be AUX bracis-33592 192 30 further far ADV bracis-33592 192 31 emphasized emphasize VERB bracis-33592 192 32 by by ADP bracis-33592 192 33 competence competence NOUN bracis-33592 192 34 3 3 NUM bracis-33592 192 35 ’s ’s PART bracis-33592 192 36 scores score NOUN bracis-33592 192 37 going go VERB bracis-33592 192 38 up up ADP bracis-33592 192 39 . . PUNCT bracis-33592 193 1 but but CCONJ bracis-33592 193 2 since since SCONJ bracis-33592 193 3 bert bert PROPN bracis-33592 193 4 does do AUX bracis-33592 193 5 not not PART bracis-33592 193 6 have have VERB bracis-33592 193 7 a a DET bracis-33592 193 8 token token NOUN bracis-33592 193 9 for for ADP bracis-33592 193 10 line line NOUN bracis-33592 193 11 break break NOUN bracis-33592 193 12 , , PUNCT bracis-33592 193 13 it it PRON bracis-33592 193 14 is be AUX bracis-33592 193 15 not not PART bracis-33592 193 16 clear clear ADJ bracis-33592 193 17 whether whether SCONJ bracis-33592 193 18 that that DET bracis-33592 193 19 effect effect NOUN bracis-33592 193 20 is be AUX bracis-33592 193 21 due due ADJ bracis-33592 193 22 to to ADP bracis-33592 193 23 the the DET bracis-33592 193 24 presence presence NOUN bracis-33592 193 25 of of ADP bracis-33592 193 26 longer long ADJ bracis-33592 193 27 sentences sentence NOUN bracis-33592 193 28 or or CCONJ bracis-33592 193 29 due due ADP bracis-33592 193 30 to to ADP bracis-33592 193 31 the the DET bracis-33592 193 32 increased increase VERB bracis-33592 193 33 word word NOUN bracis-33592 193 34 frequency frequency NOUN bracis-33592 193 35 . . PUNCT bracis-33592 194 1 finally finally ADV bracis-33592 194 2 , , PUNCT bracis-33592 194 3 we we PRON bracis-33592 194 4 see see VERB bracis-33592 194 5 that that SCONJ bracis-33592 194 6 going go VERB bracis-33592 194 7 from from ADP bracis-33592 194 8 attacks attack NOUN bracis-33592 194 9 of of ADP bracis-33592 194 10 type type NOUN bracis-33592 194 11 b b NOUN bracis-33592 194 12 to to ADP bracis-33592 194 13 c c NOUN bracis-33592 194 14 resulted result VERB bracis-33592 194 15 in in ADP bracis-33592 194 16 an an DET bracis-33592 194 17 increase increase NOUN bracis-33592 194 18 of of ADP bracis-33592 194 19 200 200 NUM bracis-33592 194 20 points point NOUN bracis-33592 194 21 . . PUNCT bracis-33592 195 1 this this PRON bracis-33592 195 2 was be AUX bracis-33592 195 3 the the DET bracis-33592 195 4 only only ADJ bracis-33592 195 5 attack attack NOUN bracis-33592 195 6 that that PRON bracis-33592 195 7 received receive VERB bracis-33592 195 8 a a DET bracis-33592 195 9 score score NOUN bracis-33592 195 10 above above ADP bracis-33592 195 11 zero zero NUM bracis-33592 195 12 in in ADP bracis-33592 195 13 competence competence NOUN bracis-33592 195 14 5 5 NUM bracis-33592 195 15 with with ADP bracis-33592 195 16 this this DET bracis-33592 195 17 model model NOUN bracis-33592 195 18 . . PUNCT bracis-33592 196 1 gemini gemini PROPN bracis-33592 196 2 , , PUNCT bracis-33592 196 3 on on ADP bracis-33592 196 4 the the DET bracis-33592 196 5 other other ADJ bracis-33592 196 6 hand hand NOUN bracis-33592 196 7 , , PUNCT bracis-33592 196 8 seems seem VERB bracis-33592 196 9 to to PART bracis-33592 196 10 recognize recognize VERB bracis-33592 196 11 attacks attack NOUN bracis-33592 196 12 that that PRON bracis-33592 196 13 increase increase VERB bracis-33592 196 14 essay essay ADJ bracis-33592 196 15 length length NOUN bracis-33592 196 16 . . PUNCT bracis-33592 197 1 the the DET bracis-33592 197 2 simplest simple ADJ bracis-33592 197 3 attacks attack NOUN bracis-33592 197 4 1a 1a NOUN bracis-33592 197 5 and and CCONJ bracis-33592 197 6 2a 2a NUM bracis-33592 197 7 were be AUX bracis-33592 197 8 assigned assign VERB bracis-33592 197 9 scores score NOUN bracis-33592 197 10 of of ADP bracis-33592 197 11 560 560 NUM bracis-33592 197 12 and and CCONJ bracis-33592 197 13 640 640 NUM bracis-33592 197 14 points point NOUN bracis-33592 197 15 , , PUNCT bracis-33592 197 16 respectively respectively ADV bracis-33592 197 17 , , PUNCT bracis-33592 197 18 by by ADP bracis-33592 197 19 this this DET bracis-33592 197 20 model model NOUN bracis-33592 197 21 , , PUNCT bracis-33592 197 22 which which PRON bracis-33592 197 23 are be AUX bracis-33592 197 24 relatively relatively ADV bracis-33592 197 25 high high ADJ bracis-33592 197 26 and and CCONJ bracis-33592 197 27 way way ADV bracis-33592 197 28 higher high ADJ bracis-33592 197 29 than than SCONJ bracis-33592 197 30 expected expect VERB bracis-33592 197 31 . . PUNCT bracis-33592 198 1 however however ADV bracis-33592 198 2 , , PUNCT bracis-33592 198 3 the the DET bracis-33592 198 4 relatively relatively ADV bracis-33592 198 5 longer long ADJ bracis-33592 198 6 essays essay NOUN bracis-33592 198 7 produced produce VERB bracis-33592 198 8 by by ADP bracis-33592 198 9 attacks attack NOUN bracis-33592 198 10 of of ADP bracis-33592 198 11 type type NOUN bracis-33592 198 12 b b NOUN bracis-33592 198 13 received receive VERB bracis-33592 198 14 lower low ADJ bracis-33592 198 15 scores score NOUN bracis-33592 198 16 , , PUNCT bracis-33592 198 17 showing show VERB bracis-33592 198 18 that that SCONJ bracis-33592 198 19 the the DET bracis-33592 198 20 gemini gemini NOUN bracis-33592 198 21 is be AUX bracis-33592 198 22 sensitive sensitive ADJ bracis-33592 198 23 to to PART bracis-33592 198 24 repetition repetition VERB bracis-33592 198 25 . . PUNCT bracis-33592 199 1 this this PRON bracis-33592 199 2 is be AUX bracis-33592 199 3 further far ADV bracis-33592 199 4 evidenced evidence VERB bracis-33592 199 5 by by ADP bracis-33592 199 6 the the DET bracis-33592 199 7 fact fact NOUN bracis-33592 199 8 that that PRON bracis-33592 199 9 is be AUX bracis-33592 199 10 assigns assign NOUN bracis-33592 199 11 only only ADV bracis-33592 199 12 40 40 NUM bracis-33592 199 13 points point NOUN bracis-33592 199 14 to to PART bracis-33592 199 15 attack attack VERB bracis-33592 199 16 3c 3c NUM bracis-33592 199 17 , , PUNCT bracis-33592 199 18 which which PRON bracis-33592 199 19 is be AUX bracis-33592 199 20 the the DET bracis-33592 199 21 one one NOUN bracis-33592 199 22 with with ADP bracis-33592 199 23 most most ADJ bracis-33592 199 24 repetitions repetition NOUN bracis-33592 199 25 . . PUNCT bracis-33592 200 1 this this DET bracis-33592 200 2 model model NOUN bracis-33592 200 3 was be AUX bracis-33592 200 4 the the DET bracis-33592 200 5 only only ADJ bracis-33592 200 6 one one NUM bracis-33592 200 7 to to PART bracis-33592 200 8 identify identify VERB bracis-33592 200 9 one one NUM bracis-33592 200 10 attack attack NOUN bracis-33592 200 11 ( ( PUNCT bracis-33592 200 12 2c 2c NOUN bracis-33592 200 13 ) ) PUNCT bracis-33592 200 14 and and CCONJ bracis-33592 200 15 it it PRON bracis-33592 200 16 almost almost ADV bracis-33592 200 17 identified identify VERB bracis-33592 200 18 attack attack NOUN bracis-33592 200 19 3c 3c NUM bracis-33592 200 20 . . PUNCT bracis-33592 201 1 finally finally ADV bracis-33592 201 2 , , PUNCT bracis-33592 201 3 the the DET bracis-33592 201 4 phi-3 phi-3 ADV bracis-33592 201 5 - - PUNCT bracis-33592 201 6 based base VERB bracis-33592 201 7 model model NOUN bracis-33592 201 8 was be AUX bracis-33592 201 9 the the DET bracis-33592 201 10 most most ADV bracis-33592 201 11 resilient resilient ADJ bracis-33592 201 12 to to ADP bracis-33592 201 13 the the DET bracis-33592 201 14 simplest simple ADJ bracis-33592 201 15 attacks attack NOUN bracis-33592 201 16 od od ADP bracis-33592 201 17 type type NOUN bracis-33592 201 18 a. a. NOUN bracis-33592 201 19 even even ADV bracis-33592 201 20 though though SCONJ bracis-33592 201 21 attacks attack NOUN bracis-33592 201 22 1a 1a NOUN bracis-33592 201 23 and and CCONJ bracis-33592 201 24 2a 2a NUM bracis-33592 201 25 are be AUX bracis-33592 201 26 very very ADV bracis-33592 201 27 similar similar ADJ bracis-33592 201 28 , , PUNCT bracis-33592 201 29 the the DET bracis-33592 201 30 model model NOUN bracis-33592 201 31 was be AUX bracis-33592 201 32 more more ADV bracis-33592 201 33 sensitive sensitive ADJ bracis-33592 201 34 to to ADP bracis-33592 201 35 the the DET bracis-33592 201 36 latter latter ADJ bracis-33592 201 37 one one NUM bracis-33592 201 38 . . PUNCT bracis-33592 202 1 attack attack NOUN bracis-33592 202 2 3a 3a NUM bracis-33592 202 3 , , PUNCT bracis-33592 202 4 which which PRON bracis-33592 202 5 somehow somehow ADV bracis-33592 202 6 combines combine VERB bracis-33592 202 7 attack attack NOUN bracis-33592 202 8 1a 1a NOUN bracis-33592 202 9 and and CCONJ bracis-33592 202 10 2a 2a NUM bracis-33592 202 11 ) ) PUNCT bracis-33592 202 12 received receive VERB bracis-33592 202 13 an an DET bracis-33592 202 14 intermediary intermediary ADJ bracis-33592 202 15 score score NOUN bracis-33592 202 16 , , PUNCT bracis-33592 202 17 suggesting suggest VERB bracis-33592 202 18 that that SCONJ bracis-33592 202 19 the the DET bracis-33592 202 20 model model NOUN bracis-33592 202 21 is be AUX bracis-33592 202 22 not not PART bracis-33592 202 23 composing compose VERB bracis-33592 202 24 the the DET bracis-33592 202 25 representation representation NOUN bracis-33592 202 26 to to PART bracis-33592 202 27 produce produce VERB bracis-33592 202 28 scores score NOUN bracis-33592 202 29 . . PUNCT bracis-33592 203 1 phi-3 phi-3 NOUN bracis-33592 203 2 ’s ’s PART bracis-33592 203 3 tokenizer tokenizer NOUN bracis-33592 203 4 has have VERB bracis-33592 203 5 a a DET bracis-33592 203 6 token token NOUN bracis-33592 203 7 for for ADP bracis-33592 203 8 the the DET bracis-33592 203 9 line line NOUN bracis-33592 203 10 break break NOUN bracis-33592 203 11 , , PUNCT bracis-33592 203 12 which which PRON bracis-33592 203 13 can can AUX bracis-33592 203 14 make make VERB bracis-33592 203 15 it it PRON bracis-33592 203 16 sensitive sensitive ADJ bracis-33592 203 17 to to ADP bracis-33592 203 18 the the DET bracis-33592 203 19 four four NUM bracis-33592 203 20 paragraphs paragraph NOUN bracis-33592 203 21 structure structure NOUN bracis-33592 203 22 . . PUNCT bracis-33592 204 1 this this PRON bracis-33592 204 2 appears appear VERB bracis-33592 204 3 to to PART bracis-33592 204 4 be be AUX bracis-33592 204 5 the the DET bracis-33592 204 6 case case NOUN bracis-33592 204 7 , , PUNCT bracis-33592 204 8 since since SCONJ bracis-33592 204 9 attacks attack NOUN bracis-33592 204 10 of of ADP bracis-33592 204 11 type type NOUN bracis-33592 204 12 a a DET bracis-33592 204 13 received receive VERB bracis-33592 204 14 smaller small ADJ bracis-33592 204 15 scores score NOUN bracis-33592 204 16 than than ADP bracis-33592 204 17 attacks attack NOUN bracis-33592 204 18 of of ADP bracis-33592 204 19 type type NOUN bracis-33592 204 20 b. b. PROPN bracis-33592 204 21 finally finally ADV bracis-33592 204 22 , , PUNCT bracis-33592 204 23 the the DET bracis-33592 204 24 attacks attack NOUN bracis-33592 204 25 of of ADP bracis-33592 204 26 type type NOUN bracis-33592 204 27 c c NOUN bracis-33592 204 28 , , PUNCT bracis-33592 204 29 which which PRON bracis-33592 204 30 are be AUX bracis-33592 204 31 based base VERB bracis-33592 204 32 on on ADP bracis-33592 204 33 repetition repetition NOUN bracis-33592 204 34 of of ADP bracis-33592 204 35 sentences sentence NOUN bracis-33592 204 36 were be AUX bracis-33592 204 37 not not PART bracis-33592 204 38 as as ADV bracis-33592 204 39 effective effective ADJ bracis-33592 204 40 as as ADP bracis-33592 204 41 the the DET bracis-33592 204 42 attacks attack NOUN bracis-33592 204 43 of of ADP bracis-33592 204 44 type type NOUN bracis-33592 204 45 b b PROPN bracis-33592 204 46 which which PRON bracis-33592 204 47 are be AUX bracis-33592 204 48 based base VERB bracis-33592 204 49 of of ADP bracis-33592 204 50 repetition repetition NOUN bracis-33592 204 51 of of ADP bracis-33592 204 52 lists list NOUN bracis-33592 204 53 . . PUNCT bracis-33592 205 1 we we PRON bracis-33592 205 2 highlight highlight VERB bracis-33592 205 3 that that PRON bracis-33592 205 4 phi-3 phi-3 PUNCT bracis-33592 205 5 achieved achieve VERB bracis-33592 205 6 the the DET bracis-33592 205 7 highest high ADJ bracis-33592 205 8 qwk qwk NOUN bracis-33592 205 9 score score NOUN bracis-33592 205 10 for for ADP bracis-33592 205 11 competence competence NOUN bracis-33592 205 12 5 5 NUM bracis-33592 205 13 , , PUNCT bracis-33592 205 14 across across ADP bracis-33592 205 15 all all DET bracis-33592 205 16 models model NOUN bracis-33592 205 17 and and CCONJ bracis-33592 205 18 competences competence NOUN bracis-33592 205 19 , , PUNCT bracis-33592 205 20 and and CCONJ bracis-33592 205 21 was be AUX bracis-33592 205 22 not not PART bracis-33592 205 23 fooled fool VERB bracis-33592 205 24 by by ADP bracis-33592 205 25 any any DET bracis-33592 205 26 attack attack NOUN bracis-33592 205 27 — — PUNCT bracis-33592 205 28 that that ADV bracis-33592 205 29 is is ADV bracis-33592 205 30 , , PUNCT bracis-33592 205 31 it it PRON bracis-33592 205 32 scored score VERB bracis-33592 205 33 all all DET bracis-33592 205 34 attacks attack NOUN bracis-33592 205 35 with with ADP bracis-33592 205 36 a a DET bracis-33592 205 37 zero zero NUM bracis-33592 205 38 . . PUNCT bracis-33592 206 1 seeing see VERB bracis-33592 206 2 that that DET bracis-33592 206 3 competence competence NOUN bracis-33592 206 4 5 5 NUM bracis-33592 206 5 is be AUX bracis-33592 206 6 the the DET bracis-33592 206 7 one one NOUN bracis-33592 206 8 that that PRON bracis-33592 206 9 most most ADV bracis-33592 206 10 often often ADV bracis-33592 206 11 receives receive VERB bracis-33592 206 12 a a DET bracis-33592 206 13 zero zero NUM bracis-33592 206 14 , , PUNCT bracis-33592 206 15 it it PRON bracis-33592 206 16 is be AUX bracis-33592 206 17 possible possible ADJ bracis-33592 206 18 to to PART bracis-33592 206 19 imagine imagine VERB bracis-33592 206 20 that that SCONJ bracis-33592 206 21 students student NOUN bracis-33592 206 22 would would AUX bracis-33592 206 23 try try VERB bracis-33592 206 24 to to PART bracis-33592 206 25 target target VERB bracis-33592 206 26 it it PRON bracis-33592 206 27 . . PUNCT bracis-33592 207 1 the the DET bracis-33592 207 2 fifth fifth ADJ bracis-33592 207 3 competence competence NOUN bracis-33592 207 4 is be AUX bracis-33592 207 5 responsible responsible ADJ bracis-33592 207 6 for for ADP bracis-33592 207 7 evaluating evaluate VERB bracis-33592 207 8 the the DET bracis-33592 207 9 conclusion conclusion NOUN bracis-33592 207 10 . . PUNCT bracis-33592 208 1 accordingly accordingly ADV bracis-33592 208 2 , , PUNCT bracis-33592 208 3 we we PRON bracis-33592 208 4 devised devise VERB bracis-33592 208 5 the the DET bracis-33592 208 6 attack attack NOUN bracis-33592 208 7 4 4 NUM bracis-33592 208 8 based base VERB bracis-33592 208 9 on on ADP bracis-33592 208 10 the the DET bracis-33592 208 11 previous previous ADJ bracis-33592 208 12 results result NOUN bracis-33592 208 13 : : PUNCT bracis-33592 208 14 a a DET bracis-33592 208 15 sentence sentence NOUN bracis-33592 208 16 that that PRON bracis-33592 208 17 resembles resemble VERB bracis-33592 208 18 a a DET bracis-33592 208 19 conclusion conclusion NOUN bracis-33592 208 20 using use VERB bracis-33592 208 21 adjectives adjective NOUN bracis-33592 208 22 and and CCONJ bracis-33592 208 23 adverbs adverb NOUN bracis-33592 208 24 , , PUNCT bracis-33592 208 25 namely namely ADV bracis-33592 208 26 : : PUNCT bracis-33592 208 27 “ " PUNCT bracis-33592 208 28 consequently consequently ADV bracis-33592 208 29 , , PUNCT bracis-33592 208 30 it it PRON bracis-33592 208 31 is be AUX bracis-33592 208 32 up up ADP bracis-33592 208 33 to to ADP bracis-33592 208 34 the the DET bracis-33592 208 35 fair fair ADJ bracis-33592 208 36 and and CCONJ bracis-33592 208 37 democratic democratic ADJ bracis-33592 208 38 federal federal ADJ bracis-33592 208 39 government government NOUN bracis-33592 208 40 to to PART bracis-33592 208 41 rapidly rapidly ADV bracis-33592 208 42 approve approve VERB bracis-33592 208 43 laws law NOUN bracis-33592 208 44 that that PRON bracis-33592 208 45 rapidly rapidly ADV bracis-33592 208 46 reduce reduce VERB bracis-33592 208 47 the the DET bracis-33592 208 48 occurrence occurrence NOUN bracis-33592 208 49 of of ADP bracis-33592 208 50 these these DET bracis-33592 208 51 horrendous horrendous ADJ bracis-33592 208 52 problems problem NOUN bracis-33592 208 53 . . PUNCT bracis-33592 209 1 following follow VERB bracis-33592 209 2 , , PUNCT bracis-33592 209 3 the the DET bracis-33592 209 4 dear dear ADJ bracis-33592 209 5 brazilian brazilian ADJ bracis-33592 209 6 population population NOUN bracis-33592 209 7 must must AUX bracis-33592 209 8 abide abide VERB bracis-33592 209 9 by by ADP bracis-33592 209 10 the the DET bracis-33592 209 11 undeniable undeniable ADJ bracis-33592 209 12 laws law NOUN bracis-33592 209 13 , , PUNCT bracis-33592 209 14 and and CCONJ bracis-33592 209 15 the the DET bracis-33592 209 16 fast fast ADJ bracis-33592 209 17 police police NOUN bracis-33592 209 18 must must AUX bracis-33592 209 19 arrest arrest VERB bracis-33592 209 20 those those PRON bracis-33592 209 21 that that PRON bracis-33592 209 22 committed commit VERB bracis-33592 209 23 any any DET bracis-33592 209 24 inhuman inhuman ADJ bracis-33592 209 25 crime crime NOUN bracis-33592 209 26 ” " PUNCT bracis-33592 209 27 , , PUNCT bracis-33592 209 28 copied copy VERB bracis-33592 209 29 in in ADP bracis-33592 209 30 7 7 NUM bracis-33592 209 31 paragraphs paragraph NOUN bracis-33592 209 32 . . PUNCT bracis-33592 210 1 the the DET bracis-33592 210 2 grades grade NOUN bracis-33592 210 3 assigned assign VERB bracis-33592 210 4 by by ADP bracis-33592 210 5 each each DET bracis-33592 210 6 model model NOUN bracis-33592 210 7 for for ADP bracis-33592 210 8 this this DET bracis-33592 210 9 attack attack NOUN bracis-33592 210 10 are be AUX bracis-33592 210 11 presented present VERB bracis-33592 210 12 in in ADP bracis-33592 210 13 table table NOUN bracis-33592 210 14     SPACE bracis-33592 210 15 5 5 NUM bracis-33592 210 16 . . PUNCT bracis-33592 210 17 table table NOUN bracis-33592 210 18 5 5 NUM bracis-33592 210 19 . . PUNCT bracis-33592 211 1 grades grade NOUN bracis-33592 211 2 assigned assign VERB bracis-33592 211 3 by by ADP bracis-33592 211 4 each each DET bracis-33592 211 5 model model NOUN bracis-33592 211 6 when when SCONJ bracis-33592 211 7 grading grade VERB bracis-33592 211 8 the the DET bracis-33592 211 9 attack attack NOUN bracis-33592 211 10 4.full 4.full NUM bracis-33592 211 11 size size NOUN bracis-33592 211 12 table table NOUN bracis-33592 211 13 surprisingly surprisingly ADV bracis-33592 211 14 , , PUNCT bracis-33592 211 15 the the DET bracis-33592 211 16 linear linear ADJ bracis-33592 211 17 regressor regressor NOUN bracis-33592 211 18 was be AUX bracis-33592 211 19 the the DET bracis-33592 211 20 only only ADJ bracis-33592 211 21 grader grader NOUN bracis-33592 211 22 not not PART bracis-33592 211 23 deceived deceive VERB bracis-33592 211 24 in in ADP bracis-33592 211 25 the the DET bracis-33592 211 26 fifth fifth ADJ bracis-33592 211 27 competence competence NOUN bracis-33592 211 28 . . PUNCT bracis-33592 212 1 nonetheless nonetheless ADV bracis-33592 212 2 , , PUNCT bracis-33592 212 3 the the DET bracis-33592 212 4 model model NOUN bracis-33592 212 5 as as ADP bracis-33592 212 6 a a DET bracis-33592 212 7 whole whole ADJ bracis-33592 212 8 assigned assign VERB bracis-33592 212 9 800 800 NUM bracis-33592 212 10 points point NOUN bracis-33592 212 11 , , PUNCT bracis-33592 212 12 a a DET bracis-33592 212 13 high high ADJ bracis-33592 212 14 score score NOUN bracis-33592 212 15 . . PUNCT bracis-33592 213 1 the the DET bracis-33592 213 2 grade grade NOUN bracis-33592 213 3 can can AUX bracis-33592 213 4 be be AUX bracis-33592 213 5 further far ADV bracis-33592 213 6 boosted boost VERB bracis-33592 213 7 by by ADP bracis-33592 213 8 adding add VERB bracis-33592 213 9 more more ADJ bracis-33592 213 10 repetitions repetition NOUN bracis-33592 213 11 and and CCONJ bracis-33592 213 12 thus thus ADV bracis-33592 213 13 achieving achieve VERB bracis-33592 213 14 a a DET bracis-33592 213 15 perfect perfect ADJ bracis-33592 213 16 score score NOUN bracis-33592 213 17 . . PUNCT bracis-33592 214 1 as as SCONJ bracis-33592 214 2 intended intend VERB bracis-33592 214 3 , , PUNCT bracis-33592 214 4 the the DET bracis-33592 214 5 bert bert PROPN bracis-33592 214 6 grader grader NOUN bracis-33592 214 7 assigned assign VERB bracis-33592 214 8 a a DET bracis-33592 214 9 high high ADJ bracis-33592 214 10 grade grade NOUN bracis-33592 214 11 in in ADP bracis-33592 214 12 competence competence NOUN bracis-33592 214 13 5 5 NUM bracis-33592 214 14 , , PUNCT bracis-33592 214 15 demonstrating demonstrate VERB bracis-33592 214 16 that that SCONJ bracis-33592 214 17 it it PRON bracis-33592 214 18 is be AUX bracis-33592 214 19 not not PART bracis-33592 214 20 only only ADV bracis-33592 214 21 using use VERB bracis-33592 214 22 word word NOUN bracis-33592 214 23 frequency frequency NOUN bracis-33592 214 24 . . PUNCT bracis-33592 215 1 still still ADV bracis-33592 215 2 , , PUNCT bracis-33592 215 3 this this DET bracis-33592 215 4 attack attack NOUN bracis-33592 215 5 uses use VERB bracis-33592 215 6 a a DET bracis-33592 215 7 lot lot NOUN bracis-33592 215 8 of of ADP bracis-33592 215 9 words word NOUN bracis-33592 215 10 , , PUNCT bracis-33592 215 11 which which PRON bracis-33592 215 12 could could AUX bracis-33592 215 13 cause cause VERB bracis-33592 215 14 lead lead NOUN bracis-33592 215 15 to to ADP bracis-33592 215 16 other other ADJ bracis-33592 215 17 competences competence NOUN bracis-33592 215 18 receiving receive VERB bracis-33592 215 19 high high ADJ bracis-33592 215 20 scores score NOUN bracis-33592 215 21 . . PUNCT bracis-33592 216 1 it it PRON bracis-33592 216 2 is be AUX bracis-33592 216 3 noteworthy noteworthy ADJ bracis-33592 216 4 that that SCONJ bracis-33592 216 5 when when SCONJ bracis-33592 216 6 trained train VERB bracis-33592 216 7 , , PUNCT bracis-33592 216 8 this this DET bracis-33592 216 9 grader grader NOUN bracis-33592 216 10 did do AUX bracis-33592 216 11 not not PART bracis-33592 216 12 have have VERB bracis-33592 216 13 access access NOUN bracis-33592 216 14 to to ADP bracis-33592 216 15 many many ADJ bracis-33592 216 16 training training NOUN bracis-33592 216 17 instances instance NOUN bracis-33592 216 18 of of ADP bracis-33592 216 19 perfect perfect ADJ bracis-33592 216 20 grade grade NOUN bracis-33592 216 21 , , PUNCT bracis-33592 216 22 which which PRON bracis-33592 216 23 causes cause VERB bracis-33592 216 24 it it PRON bracis-33592 216 25 to to PART bracis-33592 216 26 rarely rarely ADV bracis-33592 216 27 ( ( PUNCT bracis-33592 216 28 if if SCONJ bracis-33592 216 29 ever ever ADV bracis-33592 216 30 ) ) PUNCT bracis-33592 216 31 assign assign VERB bracis-33592 216 32 200 200 NUM bracis-33592 216 33 points point NOUN bracis-33592 216 34 in in ADP bracis-33592 216 35 a a DET bracis-33592 216 36 competence competence NOUN bracis-33592 216 37 . . PUNCT bracis-33592 217 1 thus thus ADV bracis-33592 217 2 , , PUNCT bracis-33592 217 3 800 800 NUM bracis-33592 217 4 points point NOUN bracis-33592 217 5 is be AUX bracis-33592 217 6 the the DET bracis-33592 217 7 highest high ADJ bracis-33592 217 8 possible possible ADJ bracis-33592 217 9 grade grade NOUN bracis-33592 217 10 in in ADP bracis-33592 217 11 this this DET bracis-33592 217 12 method method NOUN bracis-33592 217 13 , , PUNCT bracis-33592 217 14 which which PRON bracis-33592 217 15 means mean VERB bracis-33592 217 16 that that SCONJ bracis-33592 217 17 it it PRON bracis-33592 217 18 was be AUX bracis-33592 217 19 completely completely ADV bracis-33592 217 20 deceived deceive VERB bracis-33592 217 21 by by ADP bracis-33592 217 22 this this DET bracis-33592 217 23 attack attack NOUN bracis-33592 217 24 . . PUNCT bracis-33592 218 1 if if SCONJ bracis-33592 218 2 gemini gemini PROPN bracis-33592 218 3 is be AUX bracis-33592 218 4 sensitive sensitive ADJ bracis-33592 218 5 to to PART bracis-33592 218 6 repetition repetition VERB bracis-33592 218 7 and and CCONJ bracis-33592 218 8 we we PRON bracis-33592 218 9 repeat repeat VERB bracis-33592 218 10 the the DET bracis-33592 218 11 same same ADJ bracis-33592 218 12 sentence sentence NOUN bracis-33592 218 13 7 7 NUM bracis-33592 218 14 times time NOUN bracis-33592 218 15 , , PUNCT bracis-33592 218 16 then then ADV bracis-33592 218 17 we we PRON bracis-33592 218 18 would would AUX bracis-33592 218 19 expect expect VERB bracis-33592 218 20 a a DET bracis-33592 218 21 low low ADJ bracis-33592 218 22 grade grade NOUN bracis-33592 218 23 assigned assign VERB bracis-33592 218 24 by by ADP bracis-33592 218 25 this this DET bracis-33592 218 26 model model NOUN bracis-33592 218 27 . . PUNCT bracis-33592 219 1 that that PRON bracis-33592 219 2 is be AUX bracis-33592 219 3 exactly exactly ADV bracis-33592 219 4 what what PRON bracis-33592 219 5 happens happen VERB bracis-33592 219 6 . . PUNCT bracis-33592 220 1 the the DET bracis-33592 220 2 zero zero NUM bracis-33592 220 3 assigned assign VERB bracis-33592 220 4 in in ADP bracis-33592 220 5 competence competence NOUN bracis-33592 220 6 3 3 NUM bracis-33592 220 7 is be AUX bracis-33592 220 8 rounded round VERB bracis-33592 220 9 from from ADP bracis-33592 220 10 20 20 NUM bracis-33592 220 11 . . PUNCT bracis-33592 221 1 in in ADP bracis-33592 221 2 our our PRON bracis-33592 221 3 rounding round VERB bracis-33592 221 4 we we PRON bracis-33592 221 5 always always ADV bracis-33592 221 6 round round VERB bracis-33592 221 7 down down ADP bracis-33592 221 8 , , PUNCT bracis-33592 221 9 but but CCONJ bracis-33592 221 10 maybe maybe ADV bracis-33592 221 11 systems system NOUN bracis-33592 221 12 used use VERB bracis-33592 221 13 in in ADP bracis-33592 221 14 classrooms classroom NOUN bracis-33592 221 15 would would AUX bracis-33592 221 16 round round VERB bracis-33592 221 17 up up ADP bracis-33592 221 18 to to PART bracis-33592 221 19 motivate motivate VERB bracis-33592 221 20 students student NOUN bracis-33592 221 21 . . PUNCT bracis-33592 222 1 if if SCONJ bracis-33592 222 2 we we PRON bracis-33592 222 3 had have AUX bracis-33592 222 4 rounded round VERB bracis-33592 222 5 it it PRON bracis-33592 222 6 up up ADP bracis-33592 222 7 , , PUNCT bracis-33592 222 8 this this DET bracis-33592 222 9 attack attack NOUN bracis-33592 222 10 would would AUX bracis-33592 222 11 have have AUX bracis-33592 222 12 been be AUX bracis-33592 222 13 the the DET bracis-33592 222 14 attack attack NOUN bracis-33592 222 15 based base VERB bracis-33592 222 16 on on ADP bracis-33592 222 17 repetition repetition NOUN bracis-33592 222 18 that that PRON bracis-33592 222 19 fooled fool VERB bracis-33592 222 20 gemini gemini NOUN bracis-33592 222 21 the the DET bracis-33592 222 22 most most ADJ bracis-33592 222 23 . . PUNCT bracis-33592 223 1 as as ADP bracis-33592 223 2 an an DET bracis-33592 223 3 additional additional ADJ bracis-33592 223 4 test test NOUN bracis-33592 223 5 made make VERB bracis-33592 223 6 only only ADV bracis-33592 223 7 for for ADP bracis-33592 223 8 this this DET bracis-33592 223 9 grader grader NOUN bracis-33592 223 10 , , PUNCT bracis-33592 223 11 we we PRON bracis-33592 223 12 tested test VERB bracis-33592 223 13 asking ask VERB bracis-33592 223 14 it it PRON bracis-33592 223 15 to to ADP bracis-33592 223 16 grade grade NOUN bracis-33592 223 17 only only ADV bracis-33592 223 18 the the DET bracis-33592 223 19 first first ADJ bracis-33592 223 20 paragraph paragraph NOUN bracis-33592 223 21 of of ADP bracis-33592 223 22 attack attack NOUN bracis-33592 223 23 4 4 NUM bracis-33592 223 24 ; ; PUNCT bracis-33592 223 25 it it PRON bracis-33592 223 26 assigned assign VERB bracis-33592 223 27 a a DET bracis-33592 223 28 total total NOUN bracis-33592 223 29 of of ADP bracis-33592 223 30 440 440 NUM bracis-33592 223 31 points point NOUN bracis-33592 223 32 . . PUNCT bracis-33592 224 1 finally finally ADV bracis-33592 224 2 , , PUNCT bracis-33592 224 3 this this DET bracis-33592 224 4 attack attack NOUN bracis-33592 224 5 was be AUX bracis-33592 224 6 the the DET bracis-33592 224 7 one one NUM bracis-33592 224 8 in in ADP bracis-33592 224 9 which which PRON bracis-33592 224 10 phi-3 phi-3 CCONJ bracis-33592 224 11 assigned assign VERB bracis-33592 224 12 the the DET bracis-33592 224 13 highest high ADJ bracis-33592 224 14 grades grade NOUN bracis-33592 224 15 , , PUNCT bracis-33592 224 16 for for ADP bracis-33592 224 17 the the DET bracis-33592 224 18 first first ADJ bracis-33592 224 19 time time NOUN bracis-33592 224 20 assigning assign VERB bracis-33592 224 21 points point NOUN bracis-33592 224 22 in in ADP bracis-33592 224 23 the the DET bracis-33592 224 24 fifth fifth ADJ bracis-33592 224 25 competence competence NOUN bracis-33592 224 26 . . PUNCT bracis-33592 225 1 overall overall ADV bracis-33592 225 2 , , PUNCT bracis-33592 225 3 phi-3 phi-3 PUNCT bracis-33592 225 4 was be AUX bracis-33592 225 5 more more ADV bracis-33592 225 6 resilient resilient ADJ bracis-33592 225 7 than than ADP bracis-33592 225 8 bert bert PROPN bracis-33592 225 9 and and CCONJ bracis-33592 225 10 the the DET bracis-33592 225 11 linear linear ADJ bracis-33592 225 12 regressor regressor NOUN bracis-33592 225 13 but but CCONJ bracis-33592 225 14 less less ADJ bracis-33592 225 15 than than ADP bracis-33592 225 16 gemini gemini PROPN bracis-33592 225 17 . . PUNCT bracis-33592 226 1 this this PRON bracis-33592 226 2 might might AUX bracis-33592 226 3 suggest suggest VERB bracis-33592 226 4 that that SCONJ bracis-33592 226 5 being be AUX bracis-33592 226 6 an an DET bracis-33592 226 7 order order NOUN bracis-33592 226 8 of of ADP bracis-33592 226 9 magnitude magnitude NOUN bracis-33592 226 10 pays pay VERB bracis-33592 226 11 off off ADP bracis-33592 226 12 by by ADP bracis-33592 226 13 having have VERB bracis-33592 226 14 higher high ADJ bracis-33592 226 15 - - PUNCT bracis-33592 226 16 level level NOUN bracis-33592 226 17 features feature NOUN bracis-33592 226 18 . . PUNCT bracis-33592 227 1 nonetheless nonetheless ADV bracis-33592 227 2 , , PUNCT bracis-33592 227 3 the the DET bracis-33592 227 4 grade grade NOUN bracis-33592 227 5 assigned assign VERB bracis-33592 227 6 by by ADP bracis-33592 227 7 this this DET bracis-33592 227 8 model model NOUN bracis-33592 227 9 was be AUX bracis-33592 227 10 surprisingly surprisingly ADV bracis-33592 227 11 high high ADJ bracis-33592 227 12 . . PUNCT bracis-33592 228 1 5 5 NUM bracis-33592 228 2 conclusion conclusion NOUN bracis-33592 228 3 in in ADP bracis-33592 228 4 this this DET bracis-33592 228 5 work work NOUN bracis-33592 228 6 we we PRON bracis-33592 228 7 investigated investigate VERB bracis-33592 228 8 whether whether SCONJ bracis-33592 228 9 transformers transformer NOUN bracis-33592 228 10 - - PUNCT bracis-33592 228 11 based base VERB bracis-33592 228 12 automatic automatic ADJ bracis-33592 228 13 essay essay NOUN bracis-33592 228 14 scorers scorer NOUN bracis-33592 228 15 are be AUX bracis-33592 228 16 susceptible susceptible ADJ bracis-33592 228 17 to to ADP bracis-33592 228 18 universal universal ADJ bracis-33592 228 19 adversarial adversarial ADJ bracis-33592 228 20 attacks attack NOUN bracis-33592 228 21 that that PRON bracis-33592 228 22 might might AUX bracis-33592 228 23 occur occur VERB bracis-33592 228 24 in in ADP bracis-33592 228 25 a a DET bracis-33592 228 26 classroom classroom NOUN bracis-33592 228 27 setting set VERB bracis-33592 228 28 . . PUNCT bracis-33592 229 1 to to PART bracis-33592 229 2 simulate simulate VERB bracis-33592 229 3 realistic realistic ADJ bracis-33592 229 4 student student NOUN bracis-33592 229 5 behavior behavior NOUN bracis-33592 229 6 , , PUNCT bracis-33592 229 7 we we PRON bracis-33592 229 8 trained train VERB bracis-33592 229 9 a a DET bracis-33592 229 10 linear linear ADJ bracis-33592 229 11 regressor regressor NOUN bracis-33592 229 12 based base VERB bracis-33592 229 13 on on ADP bracis-33592 229 14 standard standard ADJ bracis-33592 229 15 nlp nlp NOUN bracis-33592 229 16 features feature NOUN bracis-33592 229 17 to to PART bracis-33592 229 18 predict predict VERB bracis-33592 229 19 scores score NOUN bracis-33592 229 20 , , PUNCT bracis-33592 229 21 then then ADV bracis-33592 229 22 used use VERB bracis-33592 229 23 the the DET bracis-33592 229 24 most most ADV bracis-33592 229 25 important important ADJ bracis-33592 229 26 features feature NOUN bracis-33592 229 27 to to PART bracis-33592 229 28 craft craft VERB bracis-33592 229 29 simple simple ADJ bracis-33592 229 30 essays essay NOUN bracis-33592 229 31 that that PRON bracis-33592 229 32 should should AUX bracis-33592 229 33 receive receive VERB bracis-33592 229 34 low low ADJ bracis-33592 229 35 scores score NOUN bracis-33592 229 36 according accord VERB bracis-33592 229 37 to to ADP bracis-33592 229 38 grading grade VERB bracis-33592 229 39 guidelines guideline NOUN bracis-33592 229 40 yet yet ADV bracis-33592 229 41 might might AUX bracis-33592 229 42 fool fool VERB bracis-33592 229 43 statistical statistical ADJ bracis-33592 229 44 aes aes NOUN bracis-33592 229 45 systems system NOUN bracis-33592 229 46 . . PUNCT bracis-33592 230 1 such such ADJ bracis-33592 230 2 essays essay NOUN bracis-33592 230 3 varied varied ADJ bracis-33592 230 4 from from ADP bracis-33592 230 5 simple simple ADJ bracis-33592 230 6 list list NOUN bracis-33592 230 7 of of ADP bracis-33592 230 8 adjectives adjective NOUN bracis-33592 230 9 and and CCONJ bracis-33592 230 10 adverbs adverb NOUN bracis-33592 230 11 to to ADP bracis-33592 230 12 repetitions repetition NOUN bracis-33592 230 13 of of ADP bracis-33592 230 14 template template NOUN bracis-33592 230 15 texts text NOUN bracis-33592 230 16 unrelated unrelated ADJ bracis-33592 230 17 to to ADP bracis-33592 230 18 the the DET bracis-33592 230 19 required require VERB bracis-33592 230 20 prompt prompt NOUN bracis-33592 230 21 . . PUNCT bracis-33592 231 1 we we PRON bracis-33592 231 2 found find VERB bracis-33592 231 3 that that SCONJ bracis-33592 231 4 the the DET bracis-33592 231 5 bert bert NOUN bracis-33592 231 6 - - PUNCT bracis-33592 231 7 based base VERB bracis-33592 231 8 grader grader NOUN bracis-33592 231 9 is be AUX bracis-33592 231 10 susceptible susceptible ADJ bracis-33592 231 11 to to ADP bracis-33592 231 12 both both DET bracis-33592 231 13 long long ADJ bracis-33592 231 14 sentences sentence NOUN bracis-33592 231 15 and and CCONJ bracis-33592 231 16 to to ADP bracis-33592 231 17 the the DET bracis-33592 231 18 presence presence NOUN bracis-33592 231 19 of of ADP bracis-33592 231 20 adjectives adjective NOUN bracis-33592 231 21 and and CCONJ bracis-33592 231 22 adverbs adverb NOUN bracis-33592 231 23 , , PUNCT bracis-33592 231 24 similarly similarly ADV bracis-33592 231 25 to to PART bracis-33592 231 26 reported report VERB bracis-33592 231 27 in in ADP bracis-33592 231 28 the the DET bracis-33592 231 29 literature literature NOUN bracis-33592 231 30 [ [ X bracis-33592 231 31 13 13 NUM bracis-33592 231 32 ] ] PUNCT bracis-33592 231 33 . . PUNCT bracis-33592 232 1 the the DET bracis-33592 232 2 gemini gemini NOUN bracis-33592 232 3 - - PUNCT bracis-33592 232 4 based base VERB bracis-33592 232 5 grader grader NOUN bracis-33592 232 6 was be AUX bracis-33592 232 7 the the DET bracis-33592 232 8 only only ADJ bracis-33592 232 9 model model NOUN bracis-33592 232 10 to to PART bracis-33592 232 11 completely completely ADV bracis-33592 232 12 identify identify VERB bracis-33592 232 13 an an DET bracis-33592 232 14 attack attack NOUN bracis-33592 232 15 , , PUNCT bracis-33592 232 16 meaning mean VERB bracis-33592 232 17 that that SCONJ bracis-33592 232 18 it it PRON bracis-33592 232 19 assigned assign VERB bracis-33592 232 20 0 0 NUM bracis-33592 232 21 to to ADP bracis-33592 232 22 such such ADJ bracis-33592 232 23 essays essay NOUN bracis-33592 232 24 ( ( PUNCT bracis-33592 232 25 as as ADP bracis-33592 232 26 a a DET bracis-33592 232 27 human human ADJ bracis-33592 232 28 grader grader NOUN bracis-33592 232 29 would would AUX bracis-33592 232 30 ) ) PUNCT bracis-33592 232 31 . . PUNCT bracis-33592 233 1 the the DET bracis-33592 233 2 more more ADJ bracis-33592 233 3 repetition repetition NOUN bracis-33592 233 4 an an DET bracis-33592 233 5 essay essay NOUN bracis-33592 233 6 had have VERB bracis-33592 233 7 , , PUNCT bracis-33592 233 8 the the DET bracis-33592 233 9 lower low ADJ bracis-33592 233 10 was be AUX bracis-33592 233 11 the the DET bracis-33592 233 12 score score NOUN bracis-33592 233 13 assigned assign VERB bracis-33592 233 14 by by ADP bracis-33592 233 15 that that DET bracis-33592 233 16 model model NOUN bracis-33592 233 17 . . PUNCT bracis-33592 234 1 this this PRON bracis-33592 234 2 contrasts contrast VERB bracis-33592 234 3 with with ADP bracis-33592 234 4 the the DET bracis-33592 234 5 bert bert NOUN bracis-33592 234 6 - - PUNCT bracis-33592 234 7 based base VERB bracis-33592 234 8 model model NOUN bracis-33592 234 9 , , PUNCT bracis-33592 234 10 which which PRON bracis-33592 234 11 assigned assign VERB bracis-33592 234 12 higher high ADJ bracis-33592 234 13 scores score NOUN bracis-33592 234 14 to to ADP bracis-33592 234 15 longer long ADV bracis-33592 234 16 ( ( PUNCT bracis-33592 234 17 and and CCONJ bracis-33592 234 18 more more ADV bracis-33592 234 19 repetitive repetitive ADJ bracis-33592 234 20 ) ) PUNCT bracis-33592 234 21 texts text NOUN bracis-33592 234 22 . . PUNCT bracis-33592 235 1 despite despite SCONJ bracis-33592 235 2 that that DET bracis-33592 235 3 relative relative ADJ bracis-33592 235 4 improvement improvement NOUN bracis-33592 235 5 , , PUNCT bracis-33592 235 6 this this DET bracis-33592 235 7 type type NOUN bracis-33592 235 8 of of ADP bracis-33592 235 9 model model NOUN bracis-33592 235 10 still still ADV bracis-33592 235 11 assigned assign VERB bracis-33592 235 12 undeserving undeserve VERB bracis-33592 235 13 high high ADJ bracis-33592 235 14 grades grade NOUN bracis-33592 235 15 to to ADP bracis-33592 235 16 most most ADJ bracis-33592 235 17 of of ADP bracis-33592 235 18 the the DET bracis-33592 235 19 attacks attack NOUN bracis-33592 235 20 . . PUNCT bracis-33592 236 1 this this PRON bracis-33592 236 2 is be AUX bracis-33592 236 3 aligned align VERB bracis-33592 236 4 with with ADP bracis-33592 236 5 the the DET bracis-33592 236 6 findings finding NOUN bracis-33592 236 7 of of ADP bracis-33592 236 8 the the DET bracis-33592 236 9 chatgpt chatgpt NOUN bracis-33592 236 10 - - PUNCT bracis-33592 236 11 based base VERB bracis-33592 236 12 grader grader NOUN bracis-33592 236 13 [ [ X bracis-33592 236 14 7 7 NUM bracis-33592 236 15 ] ] PUNCT bracis-33592 236 16 , , PUNCT bracis-33592 236 17 which which PRON bracis-33592 236 18 found find VERB bracis-33592 236 19 that that SCONJ bracis-33592 236 20 the the DET bracis-33592 236 21 model model NOUN bracis-33592 236 22 was be AUX bracis-33592 236 23 more more ADV bracis-33592 236 24 lenient lenient ADJ bracis-33592 236 25 than than ADP bracis-33592 236 26 a a DET bracis-33592 236 27 human human NOUN bracis-33592 236 28 - - PUNCT bracis-33592 236 29 grader grader NOUN bracis-33592 236 30 . . PUNCT bracis-33592 237 1 it it PRON bracis-33592 237 2 is be AUX bracis-33592 237 3 however however ADV bracis-33592 237 4 interesting interesting ADJ bracis-33592 237 5 to to PART bracis-33592 237 6 note note VERB bracis-33592 237 7 that that SCONJ bracis-33592 237 8 this this DET bracis-33592 237 9 model model NOUN bracis-33592 237 10 assigned assign VERB bracis-33592 237 11 the the DET bracis-33592 237 12 highest high ADJ bracis-33592 237 13 score score NOUN bracis-33592 237 14 to to ADP bracis-33592 237 15 essays essay NOUN bracis-33592 237 16 consisting consist VERB bracis-33592 237 17 only only ADV bracis-33592 237 18 of of ADP bracis-33592 237 19 a a DET bracis-33592 237 20 list list NOUN bracis-33592 237 21 of of ADP bracis-33592 237 22 adjectives adjective NOUN bracis-33592 237 23 or or CCONJ bracis-33592 237 24 adverbs adverb NOUN bracis-33592 237 25 . . PUNCT bracis-33592 238 1 the the DET bracis-33592 238 2 phi-3 phi-3 ADV bracis-33592 238 3 - - PUNCT bracis-33592 238 4 based base VERB bracis-33592 238 5 model model NOUN bracis-33592 238 6 , , PUNCT bracis-33592 238 7 which which PRON bracis-33592 238 8 is be AUX bracis-33592 238 9 the the DET bracis-33592 238 10 best good ADJ bracis-33592 238 11 performing performing NOUN bracis-33592 238 12 model model NOUN bracis-33592 238 13 for for ADP bracis-33592 238 14 most most ADJ bracis-33592 238 15 competences competence NOUN bracis-33592 238 16 , , PUNCT bracis-33592 238 17 performed perform VERB bracis-33592 238 18 similarly similarly ADV bracis-33592 238 19 to to ADP bracis-33592 238 20 the the DET bracis-33592 238 21 other other ADJ bracis-33592 238 22 models model NOUN bracis-33592 238 23 . . PUNCT bracis-33592 239 1 in in ADP bracis-33592 239 2 particular particular ADJ bracis-33592 239 3 , , PUNCT bracis-33592 239 4 it it PRON bracis-33592 239 5 was be AUX bracis-33592 239 6 more more ADV bracis-33592 239 7 susceptible susceptible ADJ bracis-33592 239 8 than than ADP bracis-33592 239 9 gemini gemini NOUN bracis-33592 239 10 to to ADP bracis-33592 239 11 an an DET bracis-33592 239 12 attack attack NOUN bracis-33592 239 13 that that PRON bracis-33592 239 14 included include VERB bracis-33592 239 15 a a DET bracis-33592 239 16 unrelated unrelated ADJ bracis-33592 239 17 but but CCONJ bracis-33592 239 18 sensible sensible ADJ bracis-33592 239 19 template template NOUN bracis-33592 239 20 conclusion conclusion NOUN bracis-33592 239 21 to to PART bracis-33592 239 22 essay essay VERB bracis-33592 239 23 . . PUNCT bracis-33592 240 1 this this PRON bracis-33592 240 2 is be AUX bracis-33592 240 3 in in ADP bracis-33592 240 4 line line NOUN bracis-33592 240 5 with with ADP bracis-33592 240 6 findings finding NOUN bracis-33592 240 7 that that SCONJ bracis-33592 240 8 fine fine ADV bracis-33592 240 9 - - PUNCT bracis-33592 240 10 tuned tune VERB bracis-33592 240 11 models model NOUN bracis-33592 240 12 improve improve VERB bracis-33592 240 13 performance performance NOUN bracis-33592 240 14 at at ADP bracis-33592 240 15 the the DET bracis-33592 240 16 expense expense NOUN bracis-33592 240 17 of of ADP bracis-33592 240 18 being be AUX bracis-33592 240 19 more more ADV bracis-33592 240 20 susceptible susceptible ADJ bracis-33592 240 21 to to ADP bracis-33592 240 22 attacks attack NOUN bracis-33592 240 23 [ [ X bracis-33592 240 24 40 40 NUM bracis-33592 240 25 ] ] PUNCT bracis-33592 240 26 . . PUNCT bracis-33592 241 1 interestingly interestingly ADV bracis-33592 241 2 , , PUNCT bracis-33592 241 3 although although SCONJ bracis-33592 241 4 the the DET bracis-33592 241 5 model model NOUN bracis-33592 241 6 was be AUX bracis-33592 241 7 originally originally ADV bracis-33592 241 8 trained train VERB bracis-33592 241 9 only only ADJ bracis-33592 241 10 texts text NOUN bracis-33592 241 11 in in ADP bracis-33592 241 12 english english PROPN bracis-33592 241 13 and and CCONJ bracis-33592 241 14 fine fine ADV bracis-33592 241 15 - - PUNCT bracis-33592 241 16 tuned tune VERB bracis-33592 241 17 with with ADP bracis-33592 241 18 texts text NOUN bracis-33592 241 19 in in ADP bracis-33592 241 20 portuguese portuguese PROPN bracis-33592 241 21 , , PUNCT bracis-33592 241 22 it it PRON bracis-33592 241 23 achieved achieve VERB bracis-33592 241 24 the the DET bracis-33592 241 25 best good ADJ bracis-33592 241 26 performances performance NOUN bracis-33592 241 27 , , PUNCT bracis-33592 241 28 even even ADV bracis-33592 241 29 when when SCONJ bracis-33592 241 30 compared compare VERB bracis-33592 241 31 to to ADP bracis-33592 241 32 models model NOUN bracis-33592 241 33 trained train VERB bracis-33592 241 34 exclusively exclusively ADV bracis-33592 241 35 with with ADP bracis-33592 241 36 portuguese portuguese ADJ bracis-33592 241 37 texts text NOUN bracis-33592 241 38 . . PUNCT bracis-33592 242 1 this this DET bracis-33592 242 2 work work NOUN bracis-33592 242 3 shows show VERB bracis-33592 242 4 that that SCONJ bracis-33592 242 5 even even ADV bracis-33592 242 6 sophisticated sophisticated ADJ bracis-33592 242 7 aes aes NOUN bracis-33592 242 8 systems system NOUN bracis-33592 242 9 can can AUX bracis-33592 242 10 be be AUX bracis-33592 242 11 fooled fool VERB bracis-33592 242 12 by by ADP bracis-33592 242 13 simple simple ADJ bracis-33592 242 14 attacks attack NOUN bracis-33592 242 15 , , PUNCT bracis-33592 242 16 which which PRON bracis-33592 242 17 should should AUX bracis-33592 242 18 be be AUX bracis-33592 242 19 of of ADP bracis-33592 242 20 great great ADJ bracis-33592 242 21 concern concern NOUN bracis-33592 242 22 to to ADP bracis-33592 242 23 real real ADJ bracis-33592 242 24 - - PUNCT bracis-33592 242 25 world world NOUN bracis-33592 242 26 systems system NOUN bracis-33592 242 27 deployed deploy VERB bracis-33592 242 28 . . PUNCT bracis-33592 243 1 future future ADJ bracis-33592 243 2 work work NOUN bracis-33592 243 3 should should AUX bracis-33592 243 4 better well ADV bracis-33592 243 5 investigate investigate VERB bracis-33592 243 6 whether whether SCONJ bracis-33592 243 7 standard standard ADJ bracis-33592 243 8 techniques technique NOUN bracis-33592 243 9 such such ADJ bracis-33592 243 10 as as ADP bracis-33592 243 11 data datum NOUN bracis-33592 243 12 augmentation augmentation NOUN bracis-33592 243 13 and and CCONJ bracis-33592 243 14 loss loss NOUN bracis-33592 243 15 function function NOUN bracis-33592 243 16 shaping shaping NOUN bracis-33592 243 17 can can AUX bracis-33592 243 18 mitigate mitigate VERB bracis-33592 243 19 such such DET bracis-33592 243 20 an an DET bracis-33592 243 21 effect effect NOUN bracis-33592 243 22 . . PUNCT bracis-33592 244 1 notes note VERB bracis-33592 244 2 1.the 1.the DET bracis-33592 244 3 models model NOUN bracis-33592 244 4 are be AUX bracis-33592 244 5 available available ADJ bracis-33592 244 6 at at ADP bracis-33592 244 7 : : PUNCT bracis-33592 244 8 https://github.com/kamel-usp/aes_enem/. https://github.com/kamel-usp/aes_enem/. ADJ bracis-33592 244 9 2.specifically 2.specifically NUM bracis-33592 244 10 , , PUNCT bracis-33592 244 11 microsoft microsoft PROPN bracis-33592 244 12 / / SYM bracis-33592 244 13 phi-3 phi-3 NOUN bracis-33592 244 14 - - PUNCT bracis-33592 244 15 medium-128k medium-128k NOUN bracis-33592 244 16 - - PUNCT bracis-33592 244 17 instruct instruct NOUN bracis-33592 244 18 on on ADP bracis-33592 244 19 the the DET bracis-33592 244 20 hugging hugging NOUN bracis-33592 244 21 face face NOUN bracis-33592 244 22 hub hub NOUN bracis-33592 244 23 : : PUNCT bracis-33592 244 24 https://huggingface.co/microsoft/phi-3-medium-128k-instruct https://huggingface.co/microsoft/phi-3-medium-128k-instruct PROPN bracis-33592 244 25 . . PUNCT bracis-33592 245 1 references reference NOUN bracis-33592 245 2 abdin abdin NOUN bracis-33592 245 3 , , PUNCT bracis-33592 245 4 m. m. NOUN bracis-33592 245 5 , , PUNCT bracis-33592 245 6 jacobs jacobs PROPN bracis-33592 245 7 , , PUNCT bracis-33592 245 8 s.a s.a PROPN bracis-33592 245 9 . . PROPN bracis-33592 245 10 , , PUNCT bracis-33592 245 11 awan awan PROPN bracis-33592 245 12 , , PUNCT bracis-33592 245 13 a.a a.a PROPN bracis-33592 245 14 . . PROPN bracis-33592 245 15 , , PUNCT bracis-33592 245 16 aneja aneja PROPN bracis-33592 245 17 , , PUNCT bracis-33592 245 18 j. j. PROPN bracis-33592 245 19 , , PUNCT bracis-33592 245 20 awadallah awadallah PROPN bracis-33592 245 21 , , PUNCT bracis-33592 245 22 a. a. NOUN bracis-33592 245 23 , , PUNCT bracis-33592 245 24 et et PROPN bracis-33592 245 25     SPACE bracis-33592 245 26 al al PROPN bracis-33592 245 27 . . PROPN bracis-33592 245 28 : : PUNCT bracis-33592 245 29 phi-3 phi-3 PUNCT bracis-33592 245 30 technical technical ADJ bracis-33592 245 31 report report NOUN bracis-33592 245 32 : : PUNCT bracis-33592 245 33 a a DET bracis-33592 245 34 highly highly ADV bracis-33592 245 35 capable capable ADJ bracis-33592 245 36 language language NOUN bracis-33592 245 37 model model NOUN bracis-33592 245 38 locally locally ADV bracis-33592 245 39 on on ADP bracis-33592 245 40 your your PRON bracis-33592 245 41 phone phone NOUN bracis-33592 245 42 ( ( PUNCT bracis-33592 245 43 2024 2024 NUM bracis-33592 245 44 ) ) PUNCT bracis-33592 245 45 google google NOUN bracis-33592 245 46 scholar scholar NOUN bracis-33592 245 47     SPACE bracis-33592 245 48 amorim amorim NOUN bracis-33592 245 49 , , PUNCT bracis-33592 245 50 e. e. PROPN bracis-33592 245 51 , , PUNCT bracis-33592 245 52 veloso veloso PROPN bracis-33592 245 53 , , PUNCT bracis-33592 245 54 a. a. NOUN bracis-33592 245 55 : : PUNCT bracis-33592 245 56 a a DET bracis-33592 245 57 multi multi ADJ bracis-33592 245 58 - - ADJ bracis-33592 245 59 aspect aspect ADJ bracis-33592 245 60 analysis analysis NOUN bracis-33592 245 61 of of ADP bracis-33592 245 62 automatic automatic ADJ bracis-33592 245 63 essay essay NOUN bracis-33592 245 64 scoring scoring NOUN bracis-33592 245 65 for for ADP bracis-33592 245 66 brazilian brazilian ADJ bracis-33592 245 67 portuguese portuguese NOUN bracis-33592 245 68 . . PUNCT bracis-33592 246 1 in in ADP bracis-33592 246 2 : : PUNCT bracis-33592 246 3 proceedings proceeding NOUN bracis-33592 246 4 of of ADP bracis-33592 246 5 the the DET bracis-33592 246 6 student student NOUN bracis-33592 246 7 research research NOUN bracis-33592 246 8 workshop workshop NOUN bracis-33592 246 9 at at ADP bracis-33592 246 10 the the DET bracis-33592 246 11 15th 15th ADJ bracis-33592 246 12 conference conference NOUN bracis-33592 246 13 of of ADP bracis-33592 246 14 the the DET bracis-33592 246 15 european european ADJ bracis-33592 246 16 chapter chapter NOUN bracis-33592 246 17 of of ADP bracis-33592 246 18 the the DET bracis-33592 246 19 association association NOUN bracis-33592 246 20 for for ADP bracis-33592 246 21 computational computational ADJ bracis-33592 246 22 linguistics linguistic NOUN bracis-33592 246 23 , , PUNCT bracis-33592 246 24 pp pp ADJ bracis-33592 246 25 . . PUNCT bracis-33592 247 1 94–102 94–102 NUM bracis-33592 247 2 ( ( PUNCT bracis-33592 247 3 2017 2017 NUM bracis-33592 247 4 ) ) PUNCT bracis-33592 247 5 google google PROPN bracis-33592 247 6 scholar scholar NOUN bracis-33592 247 7     SPACE bracis-33592 247 8 attali attali PROPN bracis-33592 247 9 , , PUNCT bracis-33592 247 10 y. y. PROPN bracis-33592 247 11 , , PUNCT bracis-33592 247 12 burstein burstein PROPN bracis-33592 247 13 , , PUNCT bracis-33592 247 14 j. j. PROPN bracis-33592 247 15 : : PUNCT bracis-33592 247 16 automated automate VERB bracis-33592 247 17 essay essay NOUN bracis-33592 247 18 scoring score VERB bracis-33592 247 19 with with ADP bracis-33592 247 20 e e NOUN bracis-33592 247 21 - - NOUN bracis-33592 247 22 rater rater ADJ bracis-33592 247 23 v.2 v.2 NOUN bracis-33592 247 24 . . PUNCT bracis-33592 248 1 j. j. PROPN bracis-33592 248 2 technol technol PROPN bracis-33592 248 3 . . PUNCT bracis-33592 248 4 learn learn PROPN bracis-33592 248 5 . . PUNCT bracis-33592 249 1 assess assess VERB bracis-33592 249 2 . . PUNCT bracis-33592 250 1 4(3 4(3 NUM bracis-33592 250 2 ) ) PUNCT bracis-33592 250 3 ( ( PUNCT bracis-33592 250 4 2006 2006 NUM bracis-33592 250 5 ) ) PUNCT bracis-33592 250 6 google google PROPN bracis-33592 250 7 scholar scholar NOUN bracis-33592 250 8     SPACE bracis-33592 250 9 bazelato bazelato NOUN bracis-33592 250 10 , , PUNCT bracis-33592 250 11 b.s b.s PROPN bracis-33592 250 12 . . PROPN bracis-33592 250 13 , , PUNCT bracis-33592 250 14 amorim amorim PROPN bracis-33592 250 15 , , PUNCT bracis-33592 250 16 e. e. PROPN bracis-33592 250 17 : : PUNCT bracis-33592 250 18 a a DET bracis-33592 250 19 bayesian bayesian NOUN bracis-33592 250 20 classifier classifier NOUN bracis-33592 250 21 to to ADP bracis-33592 250 22 automatic automatic ADJ bracis-33592 250 23 correction correction NOUN bracis-33592 250 24 of of ADP bracis-33592 250 25 portuguese portuguese ADJ bracis-33592 250 26 essays essay NOUN bracis-33592 250 27 . . PUNCT bracis-33592 251 1 in in ADP bracis-33592 251 2 : : PUNCT bracis-33592 251 3 conferência conferência PROPN bracis-33592 251 4 internacional internacional PROPN bracis-33592 251 5 sobre sobre PROPN bracis-33592 251 6 informática informática PROPN bracis-33592 251 7 na na PROPN bracis-33592 251 8 educação educação PROPN bracis-33592 251 9 ( ( PUNCT bracis-33592 251 10 tise tise NOUN bracis-33592 251 11 ) ) PUNCT bracis-33592 251 12 , , PUNCT bracis-33592 251 13 vol vol NOUN bracis-33592 251 14 . . PUNCT bracis-33592 251 15     SPACE bracis-33592 251 16 18 18 NUM bracis-33592 251 17 , , PUNCT bracis-33592 251 18 pp pp ADJ bracis-33592 251 19 . . PUNCT bracis-33592 252 1 779–782 779–782 NUM bracis-33592 252 2 ( ( PUNCT bracis-33592 252 3 2013 2013 NUM bracis-33592 252 4 ) ) PUNCT bracis-33592 252 5 google google PROPN bracis-33592 252 6 scholar scholar NOUN bracis-33592 252 7     SPACE bracis-33592 252 8 beigman beigman NOUN bracis-33592 252 9     SPACE bracis-33592 252 10 klebanov klebanov PROPN bracis-33592 252 11 , , PUNCT bracis-33592 252 12 b. b. PROPN bracis-33592 252 13 , , PUNCT bracis-33592 252 14 madnani madnani PROPN bracis-33592 252 15 , , PUNCT bracis-33592 252 16 n. n. NOUN bracis-33592 252 17 : : PUNCT bracis-33592 252 18 automated automate VERB bracis-33592 252 19 evaluation evaluation NOUN bracis-33592 252 20 of of ADP bracis-33592 252 21 writing writing NOUN bracis-33592 252 22 – – PUNCT bracis-33592 252 23 50 50 NUM bracis-33592 252 24 years year NOUN bracis-33592 252 25 and and CCONJ bracis-33592 252 26 counting counting NOUN bracis-33592 252 27 . . PUNCT bracis-33592 253 1 in in ADP bracis-33592 253 2 : : PUNCT bracis-33592 253 3 proceedings proceeding NOUN bracis-33592 253 4 of of ADP bracis-33592 253 5 the the DET bracis-33592 253 6 58th 58th ADJ bracis-33592 253 7 annual annual ADJ bracis-33592 253 8 meeting meeting NOUN bracis-33592 253 9 of of ADP bracis-33592 253 10 the the DET bracis-33592 253 11 association association NOUN bracis-33592 253 12 for for ADP bracis-33592 253 13 computational computational ADJ bracis-33592 253 14 linguistics linguistic NOUN bracis-33592 253 15 . . PUNCT bracis-33592 254 1 pp pp ADJ bracis-33592 254 2 . . PUNCT bracis-33592 255 1 7796–7810 7796–7810 NUM bracis-33592 255 2 ( ( PUNCT bracis-33592 255 3 2020 2020 NUM bracis-33592 255 4 ) ) PUNCT bracis-33592 255 5 google google PROPN bracis-33592 255 6 scholar scholar NOUN bracis-33592 255 7     SPACE bracis-33592 255 8 brown brown NOUN bracis-33592 255 9 , , PUNCT bracis-33592 255 10 t. t. PROPN bracis-33592 255 11 , , PUNCT bracis-33592 255 12 mann mann PROPN bracis-33592 255 13 , , PUNCT bracis-33592 255 14 b. b. PROPN bracis-33592 255 15 , , PUNCT bracis-33592 255 16 ryder ryder PROPN bracis-33592 255 17 , , PUNCT bracis-33592 255 18 n. n. NOUN bracis-33592 255 19 , , PUNCT bracis-33592 255 20 subbiah subbiah PROPN bracis-33592 255 21 , , PUNCT bracis-33592 255 22 m. m. NOUN bracis-33592 255 23 , , PUNCT bracis-33592 255 24 kaplan kaplan PROPN bracis-33592 255 25 , , PUNCT bracis-33592 255 26 j.d j.d PROPN bracis-33592 255 27 . . PROPN bracis-33592 255 28 , , PUNCT bracis-33592 255 29 et et PROPN bracis-33592 255 30     SPACE bracis-33592 255 31 al al PROPN bracis-33592 255 32 . . PROPN bracis-33592 255 33 : : PUNCT bracis-33592 256 1 language language NOUN bracis-33592 256 2 models model NOUN bracis-33592 256 3 are be AUX bracis-33592 256 4 few few ADJ bracis-33592 256 5 - - PUNCT bracis-33592 256 6 shot shot NOUN bracis-33592 256 7 learners learner NOUN bracis-33592 256 8 . . PUNCT bracis-33592 257 1 in in ADP bracis-33592 257 2 : : PUNCT bracis-33592 257 3 advances advance NOUN bracis-33592 257 4 in in ADP bracis-33592 257 5 neural neural ADJ bracis-33592 257 6 information information NOUN bracis-33592 257 7 processing processing NOUN bracis-33592 257 8 systems system NOUN bracis-33592 257 9 , , PUNCT bracis-33592 257 10 vol vol NOUN bracis-33592 257 11 . . PUNCT bracis-33592 257 12     SPACE bracis-33592 258 1 33 33 NUM bracis-33592 258 2 , , PUNCT bracis-33592 258 3 pp pp ADJ bracis-33592 258 4 . . PUNCT bracis-33592 258 5 1877–1901 1877–1901 NUM bracis-33592 258 6 . . PUNCT bracis-33592 259 1 curran curran PROPN bracis-33592 259 2 associates associates PROPN bracis-33592 259 3 , , PUNCT bracis-33592 259 4 inc inc PROPN bracis-33592 259 5 . . PROPN bracis-33592 259 6 ( ( PUNCT bracis-33592 259 7 2020 2020 NUM bracis-33592 259 8 ) ) PUNCT bracis-33592 259 9 google google PROPN bracis-33592 259 10 scholar scholar NOUN bracis-33592 259 11     SPACE bracis-33592 259 12 chang chang PROPN bracis-33592 259 13 , , PUNCT bracis-33592 259 14 l.h l.h PROPN bracis-33592 259 15 . . PROPN bracis-33592 259 16 , , PUNCT bracis-33592 259 17 ginter ginter PROPN bracis-33592 259 18 , , PUNCT bracis-33592 259 19 f. f. PROPN bracis-33592 259 20 : : PUNCT bracis-33592 259 21 automatic automatic ADJ bracis-33592 259 22 short short ADJ bracis-33592 259 23 answer answer NOUN bracis-33592 259 24 grading grade VERB bracis-33592 259 25 for for ADP bracis-33592 259 26 finnish finnish ADJ bracis-33592 259 27 with with ADP bracis-33592 259 28 chatgpt chatgpt NOUN bracis-33592 259 29 . . PUNCT bracis-33592 260 1 in in ADP bracis-33592 260 2 : : PUNCT bracis-33592 260 3 proceedings proceeding NOUN bracis-33592 260 4 of of ADP bracis-33592 260 5 the the DET bracis-33592 260 6 aaai aaai PROPN bracis-33592 260 7 conference conference NOUN bracis-33592 260 8 on on ADP bracis-33592 260 9 artificial artificial ADJ bracis-33592 260 10 intelligence intelligence NOUN bracis-33592 260 11 , , PUNCT bracis-33592 260 12 vol vol NOUN bracis-33592 260 13 . . PUNCT bracis-33592 260 14     SPACE bracis-33592 260 15 38 38 NUM bracis-33592 260 16 , , PUNCT bracis-33592 260 17 pp pp ADJ bracis-33592 260 18 . . PUNCT bracis-33592 261 1 23173–23181 23173–23181 NUM bracis-33592 261 2 ( ( PUNCT bracis-33592 261 3 2024 2024 NUM bracis-33592 261 4 ) ) PUNCT bracis-33592 261 5 google google PROPN bracis-33592 261 6 scholar scholar NOUN bracis-33592 261 7     SPACE bracis-33592 261 8 chang chang PROPN bracis-33592 261 9 , , PUNCT bracis-33592 261 10 m.w m.w PROPN bracis-33592 261 11 . . PROPN bracis-33592 261 12 , , PUNCT bracis-33592 261 13 ratinov ratinov PROPN bracis-33592 261 14 , , PUNCT bracis-33592 261 15 l. l. PROPN bracis-33592 261 16 , , PUNCT bracis-33592 261 17 roth roth PROPN bracis-33592 261 18 , , PUNCT bracis-33592 261 19 d. d. PROPN bracis-33592 261 20 , , PUNCT bracis-33592 261 21 srikumar srikumar PROPN bracis-33592 261 22 , , PUNCT bracis-33592 261 23 v. v. ADJ bracis-33592 261 24 : : PUNCT bracis-33592 261 25 importance importance NOUN bracis-33592 261 26 of of ADP bracis-33592 261 27 semantic semantic ADJ bracis-33592 261 28 representation representation NOUN bracis-33592 261 29 : : PUNCT bracis-33592 261 30 dataless dataless ADJ bracis-33592 261 31 classification classification NOUN bracis-33592 261 32 . . PUNCT bracis-33592 262 1 in in ADP bracis-33592 262 2 : : PUNCT bracis-33592 262 3 proceedings proceeding NOUN bracis-33592 262 4 of of ADP bracis-33592 262 5 the the DET bracis-33592 262 6 23rd 23rd ADJ bracis-33592 262 7 national national ADJ bracis-33592 262 8 conference conference NOUN bracis-33592 262 9 on on ADP bracis-33592 262 10 artificial artificial ADJ bracis-33592 262 11 intelligence intelligence NOUN bracis-33592 262 12 , , PUNCT bracis-33592 262 13 vol vol NOUN bracis-33592 262 14 . . PROPN bracis-33592 262 15 2 2 NUM bracis-33592 262 16 , , PUNCT bracis-33592 262 17 pp pp ADJ bracis-33592 262 18 . . PUNCT bracis-33592 263 1 830–835 830–835 NUM bracis-33592 263 2 ( ( PUNCT bracis-33592 263 3 2008 2008 NUM bracis-33592 263 4 ) ) PUNCT bracis-33592 263 5 google google PROPN bracis-33592 263 6 scholar scholar NOUN bracis-33592 263 7     SPACE bracis-33592 263 8 chiang chiang PROPN bracis-33592 263 9 , , PUNCT bracis-33592 263 10 w.l w.l PROPN bracis-33592 263 11 . . PROPN bracis-33592 263 12 , , PUNCT bracis-33592 263 13 et et PROPN bracis-33592 263 14 al al PROPN bracis-33592 263 15 . . PUNCT bracis-33592 263 16 : : PUNCT bracis-33592 263 17 chatbot chatbot NOUN bracis-33592 263 18 arena arena NOUN bracis-33592 263 19 : : PUNCT bracis-33592 263 20 an an DET bracis-33592 263 21 open open ADJ bracis-33592 263 22 platform platform NOUN bracis-33592 263 23 for for ADP bracis-33592 263 24 evaluating evaluate VERB bracis-33592 263 25 llms llm NOUN bracis-33592 263 26 by by ADP bracis-33592 263 27 human human ADJ bracis-33592 263 28 preference preference NOUN bracis-33592 263 29 ( ( PUNCT bracis-33592 263 30 2024 2024 NUM bracis-33592 263 31 ) ) PUNCT bracis-33592 263 32 google google NOUN bracis-33592 263 33 scholar scholar NOUN bracis-33592 263 34     SPACE bracis-33592 263 35 dao dao NOUN bracis-33592 263 36 , , PUNCT bracis-33592 263 37 t. t. PROPN bracis-33592 263 38 : : PUNCT bracis-33592 263 39 flashattention-2 flashattention-2 NOUN bracis-33592 263 40 : : PUNCT bracis-33592 263 41 faster fast ADJ bracis-33592 263 42 attention attention NOUN bracis-33592 263 43 with with ADP bracis-33592 263 44 better well ADJ bracis-33592 263 45 parallelism parallelism NOUN bracis-33592 263 46 and and CCONJ bracis-33592 263 47 work work NOUN bracis-33592 263 48 partitioning partitioning NOUN bracis-33592 263 49 ( ( PUNCT bracis-33592 263 50 2023 2023 NUM bracis-33592 263 51 ) ) PUNCT bracis-33592 263 52 google google NOUN bracis-33592 263 53 scholar scholar NOUN bracis-33592 263 54     SPACE bracis-33592 263 55 ebrahimi ebrahimi PROPN bracis-33592 263 56 , , PUNCT bracis-33592 263 57 j. j. PROPN bracis-33592 263 58 , , PUNCT bracis-33592 263 59 rao rao PROPN bracis-33592 263 60 , , PUNCT bracis-33592 263 61 a. a. NOUN bracis-33592 263 62 , , PUNCT bracis-33592 263 63 lowd lowd PROPN bracis-33592 263 64 , , PUNCT bracis-33592 263 65 d. d. PROPN bracis-33592 263 66 , , PUNCT bracis-33592 263 67 dou dou PROPN bracis-33592 263 68 , , PUNCT bracis-33592 263 69 d. d. PROPN bracis-33592 263 70 : : PUNCT bracis-33592 263 71 hotflip hotflip PROPN bracis-33592 263 72 : : PUNCT bracis-33592 263 73 white white ADJ bracis-33592 263 74 - - PUNCT bracis-33592 263 75 box box NOUN bracis-33592 263 76 adversarial adversarial ADJ bracis-33592 263 77 examples example NOUN bracis-33592 263 78 for for ADP bracis-33592 263 79 text text NOUN bracis-33592 263 80 classification classification NOUN bracis-33592 263 81 . . PUNCT bracis-33592 264 1 in in ADP bracis-33592 264 2 : : PUNCT bracis-33592 264 3 proceedings proceeding NOUN bracis-33592 264 4 of of ADP bracis-33592 264 5 the the DET bracis-33592 264 6 56th 56th ADJ bracis-33592 264 7 annual annual ADJ bracis-33592 264 8 meeting meeting NOUN bracis-33592 264 9 of of ADP bracis-33592 264 10 the the DET bracis-33592 264 11 association association NOUN bracis-33592 264 12 for for ADP bracis-33592 264 13 computational computational ADJ bracis-33592 264 14 linguistics linguistic NOUN bracis-33592 264 15 , , PUNCT bracis-33592 264 16 vol vol NOUN bracis-33592 264 17 . . PROPN bracis-33592 265 1 2 2 NUM bracis-33592 265 2 : : PUNCT bracis-33592 265 3 short short ADJ bracis-33592 265 4 papers paper NOUN bracis-33592 265 5 , , PUNCT bracis-33592 265 6 pp pp ADV bracis-33592 265 7 . . PUNCT bracis-33592 266 1 31–36 31–36 NUM bracis-33592 266 2 ( ( PUNCT bracis-33592 266 3 2018 2018 NUM bracis-33592 266 4 ) ) PUNCT bracis-33592 266 5 google google NOUN bracis-33592 266 6 scholar scholar NOUN bracis-33592 266 7     SPACE bracis-33592 266 8 filho filho NOUN bracis-33592 266 9 , , PUNCT bracis-33592 266 10 a.h a.h PROPN bracis-33592 266 11 . . PROPN bracis-33592 266 12 , , PUNCT bracis-33592 266 13 concatto concatto PROPN bracis-33592 266 14 , , PUNCT bracis-33592 266 15 f. f. PROPN bracis-33592 266 16 , , PUNCT bracis-33592 266 17 do do AUX bracis-33592 266 18     SPACE bracis-33592 266 19 prado prado VERB bracis-33592 266 20 , , PUNCT bracis-33592 266 21 h.a h.a PROPN bracis-33592 266 22 . . PROPN bracis-33592 266 23 , , PUNCT bracis-33592 266 24 ferneda ferneda PROPN bracis-33592 266 25 , , PUNCT bracis-33592 266 26 e. e. PROPN bracis-33592 266 27 : : PUNCT bracis-33592 266 28 comparing compare VERB bracis-33592 266 29 feature feature NOUN bracis-33592 266 30 engineering engineering NOUN bracis-33592 266 31 and and CCONJ bracis-33592 266 32 deep deep ADJ bracis-33592 266 33 learning learning NOUN bracis-33592 266 34 methods method NOUN bracis-33592 266 35 for for ADP bracis-33592 266 36 automated automate VERB bracis-33592 266 37 essay essay NOUN bracis-33592 266 38 scoring scoring NOUN bracis-33592 266 39 of of ADP bracis-33592 266 40 brazilian brazilian ADJ bracis-33592 266 41 national national PROPN bracis-33592 266 42 high high ADJ bracis-33592 266 43 school school NOUN bracis-33592 266 44 examination examination NOUN bracis-33592 266 45 . . PUNCT bracis-33592 267 1 in in ADP bracis-33592 267 2 : : PUNCT bracis-33592 267 3 international international ADJ bracis-33592 267 4 conference conference NOUN bracis-33592 267 5 on on ADP bracis-33592 267 6 enterprise enterprise NOUN bracis-33592 267 7 information information NOUN bracis-33592 267 8 systems system NOUN bracis-33592 267 9 ( ( PUNCT bracis-33592 267 10 2021 2021 NUM bracis-33592 267 11 ) ) PUNCT bracis-33592 267 12 google google NOUN bracis-33592 267 13 scholar scholar NOUN bracis-33592 267 14     SPACE bracis-33592 267 15 filighera filighera NOUN bracis-33592 267 16 , , PUNCT bracis-33592 267 17 a. a. NOUN bracis-33592 267 18 , , PUNCT bracis-33592 267 19 ochs och NOUN bracis-33592 267 20 , , PUNCT bracis-33592 267 21 s. s. PROPN bracis-33592 267 22 , , PUNCT bracis-33592 267 23 steuer steuer PROPN bracis-33592 267 24 , , PUNCT bracis-33592 267 25 t. t. PROPN bracis-33592 267 26 , , PUNCT bracis-33592 267 27 tregel tregel NOUN bracis-33592 267 28 , , PUNCT bracis-33592 267 29 t. t. PROPN bracis-33592 267 30 : : PUNCT bracis-33592 267 31 cheating cheat VERB bracis-33592 267 32 automatic automatic ADJ bracis-33592 267 33 short short ADJ bracis-33592 267 34 answer answer NOUN bracis-33592 267 35 grading grade VERB bracis-33592 267 36 with with ADP bracis-33592 267 37 the the DET bracis-33592 267 38 adversarial adversarial ADJ bracis-33592 267 39 usage usage NOUN bracis-33592 267 40 of of ADP bracis-33592 267 41 adjectives adjective NOUN bracis-33592 267 42 and and CCONJ bracis-33592 267 43 adverbs adverb NOUN bracis-33592 267 44 . . PROPN bracis-33592 267 45 int int PROPN bracis-33592 267 46 . . PUNCT bracis-33592 268 1 j. j. PROPN bracis-33592 268 2 artif artif PROPN bracis-33592 268 3 . . PUNCT bracis-33592 269 1 intell intell PROPN bracis-33592 269 2 . . PUNCT bracis-33592 270 1 educ educ PROPN bracis-33592 270 2 . . PUNCT bracis-33592 271 1 34(2 34(2 NUM bracis-33592 271 2 ) ) PUNCT bracis-33592 271 3 , , PUNCT bracis-33592 271 4 616–646 616–646 NUM bracis-33592 271 5 ( ( PUNCT bracis-33592 271 6 2024 2024 NUM bracis-33592 271 7 ) ) PUNCT bracis-33592 271 8 article article NOUN bracis-33592 271 9     SPACE bracis-33592 271 10 google google PROPN bracis-33592 271 11 scholar scholar NOUN bracis-33592 271 12     SPACE bracis-33592 271 13 filighera filighera NOUN bracis-33592 271 14 , , PUNCT bracis-33592 271 15 a. a. NOUN bracis-33592 271 16 , , PUNCT bracis-33592 271 17 steuer steuer PROPN bracis-33592 271 18 , , PUNCT bracis-33592 271 19 t. t. PROPN bracis-33592 271 20 , , PUNCT bracis-33592 271 21 rensing rense VERB bracis-33592 271 22 , , PUNCT bracis-33592 271 23 c. c. NOUN bracis-33592 271 24 : : PUNCT bracis-33592 271 25 fooling fool VERB bracis-33592 271 26 automatic automatic ADJ bracis-33592 271 27 short short ADJ bracis-33592 271 28 answer answer NOUN bracis-33592 271 29 grading grade VERB bracis-33592 271 30 systems system NOUN bracis-33592 271 31 . . PUNCT bracis-33592 272 1 in in ADP bracis-33592 272 2 : : PUNCT bracis-33592 272 3 bittencourt bittencourt PROPN bracis-33592 272 4 , , PUNCT bracis-33592 272 5 i.i i.i PROPN bracis-33592 272 6 . . PROPN bracis-33592 272 7 , , PUNCT bracis-33592 272 8 cukurova cukurova PROPN bracis-33592 272 9 , , PUNCT bracis-33592 272 10 m. m. NOUN bracis-33592 272 11 , , PUNCT bracis-33592 272 12 muldner muldner NOUN bracis-33592 272 13 , , PUNCT bracis-33592 272 14 k. k. PROPN bracis-33592 272 15 , , PUNCT bracis-33592 272 16 luckin luckin PROPN bracis-33592 272 17 , , PUNCT bracis-33592 272 18 r. r. PROPN bracis-33592 272 19 , , PUNCT bracis-33592 272 20 millán millán PROPN bracis-33592 272 21 , , PUNCT bracis-33592 272 22 e. e. PROPN bracis-33592 272 23 ( ( PUNCT bracis-33592 272 24 eds eds PROPN bracis-33592 272 25 . . PUNCT bracis-33592 272 26 ) ) PUNCT bracis-33592 272 27 aied aie VERB bracis-33592 272 28 2020 2020 NUM bracis-33592 272 29 . . PUNCT bracis-33592 273 1 lncs lncs PROPN bracis-33592 273 2 ( ( PUNCT bracis-33592 273 3 lnai lnai ADJ bracis-33592 273 4 ) ) PUNCT bracis-33592 273 5 , , PUNCT bracis-33592 273 6 vol vol NOUN bracis-33592 273 7 . . PROPN bracis-33592 273 8 12163 12163 NUM bracis-33592 273 9 , , PUNCT bracis-33592 273 10 pp pp ADJ bracis-33592 273 11 . . PUNCT bracis-33592 274 1 177–190 177–190 NUM bracis-33592 274 2 . . PUNCT bracis-33592 274 3 springer springer NOUN bracis-33592 274 4 , , PUNCT bracis-33592 274 5 cham cham PROPN bracis-33592 274 6 ( ( PUNCT bracis-33592 274 7 2020 2020 NUM bracis-33592 274 8 ) ) PUNCT bracis-33592 274 9 . . PUNCT bracis-33592 275 1 https://doi.org/10.1007/978-3-030-52237-7_15 https://doi.org/10.1007/978-3-030-52237-7_15 PROPN bracis-33592 275 2 chapter chapter NOUN bracis-33592 275 3     SPACE bracis-33592 275 4 math math NOUN bracis-33592 275 5     SPACE bracis-33592 275 6 google google PROPN bracis-33592 275 7 scholar scholar NOUN bracis-33592 275 8     SPACE bracis-33592 275 9 fonseca fonseca PROPN bracis-33592 275 10 , , PUNCT bracis-33592 275 11 e.r e.r PROPN bracis-33592 275 12 . . PROPN bracis-33592 275 13 , , PUNCT bracis-33592 275 14 medeiros medeiros PROPN bracis-33592 275 15 , , PUNCT bracis-33592 275 16 i. i. PROPN bracis-33592 275 17 , , PUNCT bracis-33592 275 18 kamikawachi kamikawachi PROPN bracis-33592 275 19 , , PUNCT bracis-33592 275 20 d. d. PROPN bracis-33592 275 21 , , PUNCT bracis-33592 275 22 bokan bokan PROPN bracis-33592 275 23 , , PUNCT bracis-33592 275 24 a. a. NOUN bracis-33592 275 25 : : PUNCT bracis-33592 275 26 automatically automatically ADV bracis-33592 275 27 grading grade VERB bracis-33592 275 28 brazilian brazilian ADJ bracis-33592 275 29 student student NOUN bracis-33592 275 30 essays essay NOUN bracis-33592 275 31 . . PUNCT bracis-33592 276 1 in in ADP bracis-33592 276 2 : : PUNCT bracis-33592 276 3 proceedings proceeding NOUN bracis-33592 276 4 of of ADP bracis-33592 276 5 international international ADJ bracis-33592 276 6 conference conference NOUN bracis-33592 276 7 on on ADP bracis-33592 276 8 computational computational ADJ bracis-33592 276 9 processing processing NOUN bracis-33592 276 10 of of ADP bracis-33592 276 11 the the DET bracis-33592 276 12 portuguese portuguese ADJ bracis-33592 276 13 language language NOUN bracis-33592 276 14 , , PUNCT bracis-33592 276 15 pp pp ADJ bracis-33592 276 16 . . PUNCT bracis-33592 277 1 170–179 170–179 NUM bracis-33592 277 2 ( ( PUNCT bracis-33592 277 3 2018 2018 NUM bracis-33592 277 4 ) ) PUNCT bracis-33592 277 5 google google PROPN bracis-33592 277 6 scholar scholar NOUN bracis-33592 277 7     SPACE bracis-33592 277 8 gao gao PROPN bracis-33592 277 9 , , PUNCT bracis-33592 277 10 h. h. PROPN bracis-33592 277 11 , , PUNCT bracis-33592 277 12 oates oates PROPN bracis-33592 277 13 , , PUNCT bracis-33592 277 14 t. t. PROPN bracis-33592 277 15 : : PUNCT bracis-33592 277 16 universal universal ADJ bracis-33592 277 17 adversarial adversarial ADJ bracis-33592 277 18 perturbation perturbation NOUN bracis-33592 277 19 for for ADP bracis-33592 277 20 text text NOUN bracis-33592 277 21 classification classification NOUN bracis-33592 277 22 . . PUNCT bracis-33592 278 1 arxiv arxiv PROPN bracis-33592 278 2 preprint preprint VERB bracis-33592 278 3 arxiv:1910.04618 arxiv:1910.04618 NOUN bracis-33592 278 4 ( ( PUNCT bracis-33592 278 5 2019 2019 NUM bracis-33592 278 6 ) ) PUNCT bracis-33592 278 7 gunasekar gunasekar PROPN bracis-33592 278 8 , , PUNCT bracis-33592 278 9 s. s. PROPN bracis-33592 278 10 , , PUNCT bracis-33592 278 11 et et PROPN bracis-33592 278 12 al al PROPN bracis-33592 278 13 . . PROPN bracis-33592 278 14 : : PUNCT bracis-33592 279 1 textbooks textbook NOUN bracis-33592 279 2 are be AUX bracis-33592 279 3 all all PRON bracis-33592 279 4 you you PRON bracis-33592 279 5 need need VERB bracis-33592 279 6 ( ( PUNCT bracis-33592 279 7 2023 2023 NUM bracis-33592 279 8 ) ) PUNCT bracis-33592 279 9 google google PROPN bracis-33592 279 10 scholar scholar NOUN bracis-33592 279 11     SPACE bracis-33592 279 12 hamner hamner NOUN bracis-33592 279 13 , , PUNCT bracis-33592 279 14 b. b. PROPN bracis-33592 279 15 , , PUNCT bracis-33592 279 16 morgan morgan PROPN bracis-33592 279 17 , , PUNCT bracis-33592 279 18 j. j. PROPN bracis-33592 279 19 , , PUNCT bracis-33592 279 20 lynnvandev lynnvandev PROPN bracis-33592 279 21 , , PUNCT bracis-33592 279 22 m. m. NOUN bracis-33592 279 23 , , PUNCT bracis-33592 279 24 ark ark PROPN bracis-33592 279 25 , , PUNCT bracis-33592 279 26 t. t. PROPN bracis-33592 279 27 : : PUNCT bracis-33592 279 28 the the DET bracis-33592 279 29 hewlett hewlett PROPN bracis-33592 279 30 foundation foundation PROPN bracis-33592 279 31 : : PUNCT bracis-33592 279 32 automated automate VERB bracis-33592 279 33 essay essay ADJ bracis-33592 279 34 scoring scoring NOUN bracis-33592 279 35 . . PUNCT bracis-33592 280 1 kaggle kaggle PROPN bracis-33592 280 2 ( ( PUNCT bracis-33592 280 3 2012 2012 NUM bracis-33592 280 4 ) ) PUNCT bracis-33592 280 5 google google PROPN bracis-33592 280 6 scholar scholar NOUN bracis-33592 280 7     SPACE bracis-33592 280 8 hu hu PROPN bracis-33592 280 9 , , PUNCT bracis-33592 280 10 e.j e.j PROPN bracis-33592 280 11 . . PROPN bracis-33592 280 12 , , PUNCT bracis-33592 280 13 et et PROPN bracis-33592 280 14 al al PROPN bracis-33592 280 15 . . PUNCT bracis-33592 280 16 : : PUNCT bracis-33592 281 1 lora lora NOUN bracis-33592 281 2 : : PUNCT bracis-33592 281 3 low low ADJ bracis-33592 281 4 - - PUNCT bracis-33592 281 5 rank rank NOUN bracis-33592 281 6 adaptation adaptation NOUN bracis-33592 281 7 of of ADP bracis-33592 281 8 large large ADJ bracis-33592 281 9 language language NOUN bracis-33592 281 10 models model NOUN bracis-33592 281 11 ( ( PUNCT bracis-33592 281 12 2021 2021 NUM bracis-33592 281 13 ) ) PUNCT bracis-33592 281 14 google google PROPN bracis-33592 281 15 scholar scholar NOUN bracis-33592 281 16     SPACE bracis-33592 281 17 klein klein PROPN bracis-33592 281 18 , , PUNCT bracis-33592 281 19 h. h. PROPN bracis-33592 281 20 , , PUNCT bracis-33592 281 21 levenburg levenburg NOUN bracis-33592 281 22 , , PUNCT bracis-33592 281 23 n. n. NOUN bracis-33592 281 24 , , PUNCT bracis-33592 281 25 mckendall mckendall NOUN bracis-33592 281 26 , , PUNCT bracis-33592 281 27 m. m. NOUN bracis-33592 281 28 , , PUNCT bracis-33592 281 29 mothersell mothersell PROPN bracis-33592 281 30 , , PUNCT bracis-33592 281 31 w. w. NOUN bracis-33592 281 32 : : PUNCT bracis-33592 281 33 cheating cheat VERB bracis-33592 281 34 during during ADP bracis-33592 281 35 the the DET bracis-33592 281 36 college college NOUN bracis-33592 281 37 years year NOUN bracis-33592 281 38 : : PUNCT bracis-33592 281 39 how how SCONJ bracis-33592 281 40 do do AUX bracis-33592 281 41 business business NOUN bracis-33592 281 42 students student NOUN bracis-33592 281 43 compare compare VERB bracis-33592 281 44 ? ? PUNCT bracis-33592 282 1 j. j. PROPN bracis-33592 282 2 bus bus PROPN bracis-33592 282 3 . . PUNCT bracis-33592 283 1 ethics ethic NOUN bracis-33592 283 2 72 72 NUM bracis-33592 283 3 , , PUNCT bracis-33592 283 4 197–206 197–206 NUM bracis-33592 283 5 ( ( PUNCT bracis-33592 283 6 2007 2007 NUM bracis-33592 283 7 ) ) PUNCT bracis-33592 283 8 article article NOUN bracis-33592 283 9     SPACE bracis-33592 283 10 google google PROPN bracis-33592 283 11 scholar scholar NOUN bracis-33592 283 12     SPACE bracis-33592 283 13 kumar kumar PROPN bracis-33592 283 14 , , PUNCT bracis-33592 283 15 y. y. PROPN bracis-33592 283 16 , , PUNCT bracis-33592 283 17 bhatia bhatia PROPN bracis-33592 283 18 , , PUNCT bracis-33592 283 19 m. m. NOUN bracis-33592 283 20 , , PUNCT bracis-33592 283 21 kabra kabra PROPN bracis-33592 283 22 , , PUNCT bracis-33592 283 23 a. a. PROPN bracis-33592 283 24 , , PUNCT bracis-33592 283 25 li li PROPN bracis-33592 283 26 , , PUNCT bracis-33592 283 27 j.j j.j PROPN bracis-33592 283 28 . . PROPN bracis-33592 283 29 , , PUNCT bracis-33592 283 30 jin jin PROPN bracis-33592 283 31 , , PUNCT bracis-33592 283 32 d. d. PROPN bracis-33592 283 33 , , PUNCT bracis-33592 283 34 shah shah PROPN bracis-33592 283 35 , , PUNCT bracis-33592 283 36 r.r r.r PROPN bracis-33592 283 37 . . PUNCT bracis-33592 283 38 : : PUNCT bracis-33592 284 1 calling call VERB bracis-33592 284 2 out out ADP bracis-33592 284 3 bluff bluff NOUN bracis-33592 284 4 : : PUNCT bracis-33592 284 5 attacking attack VERB bracis-33592 284 6 the the DET bracis-33592 284 7 robustness robustness NOUN bracis-33592 284 8 of of ADP bracis-33592 284 9 automatic automatic ADJ bracis-33592 284 10 scoring scoring NOUN bracis-33592 284 11 systems system NOUN bracis-33592 284 12 with with ADP bracis-33592 284 13 simple simple ADJ bracis-33592 284 14 adversarial adversarial ADJ bracis-33592 284 15 testing testing NOUN bracis-33592 284 16 . . PUNCT bracis-33592 285 1 corr corr PROPN bracis-33592 285 2 arxiv:2007.06796 arxiv:2007.06796 ADV bracis-33592 285 3 ( ( PUNCT bracis-33592 285 4 2020 2020 NUM bracis-33592 285 5 ) ) PUNCT bracis-33592 285 6 kumar kumar PROPN bracis-33592 285 7 , , PUNCT bracis-33592 285 8 y. y. PROPN bracis-33592 285 9 , , PUNCT bracis-33592 285 10 parekh parekh PROPN bracis-33592 285 11 , , PUNCT bracis-33592 285 12 s. s. PROPN bracis-33592 285 13 , , PUNCT bracis-33592 285 14 singh singh PROPN bracis-33592 285 15 , , PUNCT bracis-33592 285 16 s. s. PROPN bracis-33592 285 17 , , PUNCT bracis-33592 285 18 li li PROPN bracis-33592 285 19 , , PUNCT bracis-33592 285 20 j.j j.j PROPN bracis-33592 285 21 . . PROPN bracis-33592 285 22 , , PUNCT bracis-33592 285 23 shah shah PROPN bracis-33592 285 24 , , PUNCT bracis-33592 285 25 r.r r.r PROPN bracis-33592 285 26 . . PROPN bracis-33592 285 27 , , PUNCT bracis-33592 285 28 chen chen PROPN bracis-33592 285 29 , , PUNCT bracis-33592 285 30 c. c. PROPN bracis-33592 285 31 : : PUNCT bracis-33592 285 32 automatic automatic ADJ bracis-33592 285 33 essay essay NOUN bracis-33592 285 34 scoring scoring NOUN bracis-33592 285 35 systems system NOUN bracis-33592 285 36 are be AUX bracis-33592 285 37 both both CCONJ bracis-33592 285 38 overstable overstable ADJ bracis-33592 285 39 and and CCONJ bracis-33592 285 40 oversensitive oversensitive ADJ bracis-33592 285 41 : : PUNCT bracis-33592 285 42 explaining explain VERB bracis-33592 285 43 why why SCONJ bracis-33592 285 44 and and CCONJ bracis-33592 285 45 proposing propose VERB bracis-33592 285 46 defenses defense NOUN bracis-33592 285 47 . . PUNCT bracis-33592 286 1 dial dial NOUN bracis-33592 286 2 . . PUNCT bracis-33592 287 1 discour discour ADJ bracis-33592 287 2 . . PUNCT bracis-33592 288 1 14(1 14(1 NUM bracis-33592 288 2 ) ) PUNCT bracis-33592 288 3 , , PUNCT bracis-33592 288 4 1–33 1–33 NUM bracis-33592 288 5 ( ( PUNCT bracis-33592 288 6 2023 2023 NUM bracis-33592 288 7 ) ) PUNCT bracis-33592 288 8 article article NOUN bracis-33592 288 9     SPACE bracis-33592 288 10 google google PROPN bracis-33592 288 11 scholar scholar NOUN bracis-33592 288 12     SPACE bracis-33592 288 13 larochelle larochelle PROPN bracis-33592 288 14 , , PUNCT bracis-33592 288 15 h. h. PROPN bracis-33592 288 16 , , PUNCT bracis-33592 288 17 erhan erhan PROPN bracis-33592 288 18 , , PUNCT bracis-33592 288 19 d. d. PROPN bracis-33592 288 20 , , PUNCT bracis-33592 288 21 bengio bengio PROPN bracis-33592 288 22 , , PUNCT bracis-33592 288 23 y. y. NOUN bracis-33592 288 24 : : PUNCT bracis-33592 288 25 zero zero NUM bracis-33592 288 26 - - PUNCT bracis-33592 288 27 data datum NOUN bracis-33592 288 28 learning learning NOUN bracis-33592 288 29 of of ADP bracis-33592 288 30 new new ADJ bracis-33592 288 31 tasks task NOUN bracis-33592 288 32 . . PUNCT bracis-33592 289 1 in in ADP bracis-33592 289 2 : : PUNCT bracis-33592 289 3 proceedings proceeding NOUN bracis-33592 289 4 of of ADP bracis-33592 289 5 the the DET bracis-33592 289 6 23rd 23rd ADJ bracis-33592 289 7 national national ADJ bracis-33592 289 8 conference conference NOUN bracis-33592 289 9 on on ADP bracis-33592 289 10 artificial artificial ADJ bracis-33592 289 11 intelligence intelligence NOUN bracis-33592 289 12 , , PUNCT bracis-33592 289 13 vol vol NOUN bracis-33592 289 14 . . PROPN bracis-33592 289 15 2 2 NUM bracis-33592 289 16 , , PUNCT bracis-33592 289 17 pp pp ADJ bracis-33592 289 18 . . PUNCT bracis-33592 290 1 646–651 646–651 NUM bracis-33592 290 2 ( ( PUNCT bracis-33592 290 3 2008 2008 NUM bracis-33592 290 4 ) ) PUNCT bracis-33592 290 5 google google PROPN bracis-33592 290 6 scholar scholar NOUN bracis-33592 290 7     SPACE bracis-33592 290 8 leal leal NOUN bracis-33592 290 9 , , PUNCT bracis-33592 290 10 s.e s.e PROPN bracis-33592 290 11 . . PROPN bracis-33592 290 12 , , PUNCT bracis-33592 290 13 duran duran PROPN bracis-33592 290 14 , , PUNCT bracis-33592 290 15 m.s m.s PROPN bracis-33592 290 16 . . PROPN bracis-33592 290 17 , , PUNCT bracis-33592 290 18 scarton scarton PROPN bracis-33592 290 19 , , PUNCT bracis-33592 290 20 c.e c.e PROPN bracis-33592 290 21 . . PROPN bracis-33592 290 22 , , PUNCT bracis-33592 290 23 hartmann hartmann PROPN bracis-33592 290 24 , , PUNCT bracis-33592 290 25 n.s n.s PROPN bracis-33592 290 26 . . PROPN bracis-33592 290 27 , , PUNCT bracis-33592 290 28 aluísio aluísio PROPN bracis-33592 290 29 , , PUNCT bracis-33592 290 30 s.m s.m PROPN bracis-33592 290 31 . . PROPN bracis-33592 290 32 : : PUNCT bracis-33592 290 33 nilc nilc NOUN bracis-33592 290 34 - - PUNCT bracis-33592 290 35 metrix metrix NOUN bracis-33592 290 36 : : PUNCT bracis-33592 290 37 assessing assess VERB bracis-33592 290 38 the the DET bracis-33592 290 39 complexity complexity NOUN bracis-33592 290 40 of of ADP bracis-33592 290 41 written write VERB bracis-33592 290 42 and and CCONJ bracis-33592 290 43 spoken spoken ADJ bracis-33592 290 44 language language NOUN bracis-33592 290 45 in in ADP bracis-33592 290 46 brazilian brazilian ADJ bracis-33592 290 47 portuguese portuguese PROPN bracis-33592 290 48 . . PUNCT bracis-33592 291 1 lang lang PROPN bracis-33592 291 2 . . PUNCT bracis-33592 292 1 res re NOUN bracis-33592 292 2 . . PUNCT bracis-33592 293 1 eval eval PROPN bracis-33592 293 2 . . PUNCT bracis-33592 294 1 58(1 58(1 NOUN bracis-33592 294 2 ) ) PUNCT bracis-33592 294 3 , , PUNCT bracis-33592 294 4 73–110 73–110 PROPN bracis-33592 294 5 ( ( PUNCT bracis-33592 294 6 2024 2024 NUM bracis-33592 294 7 ) ) PUNCT bracis-33592 294 8 article article NOUN bracis-33592 294 9     SPACE bracis-33592 294 10 google google PROPN bracis-33592 294 11 scholar scholar NOUN bracis-33592 294 12     SPACE bracis-33592 294 13 li li PROPN bracis-33592 294 14 , , PUNCT bracis-33592 294 15 l. l. PROPN bracis-33592 294 16 , , PUNCT bracis-33592 294 17 ma ma PROPN bracis-33592 294 18 , , PUNCT bracis-33592 294 19 r. r. PROPN bracis-33592 294 20 , , PUNCT bracis-33592 294 21 guo guo PROPN bracis-33592 294 22 , , PUNCT bracis-33592 294 23 q. q. PROPN bracis-33592 294 24 , , PUNCT bracis-33592 294 25 xue xue PROPN bracis-33592 294 26 , , PUNCT bracis-33592 294 27 x. x. PROPN bracis-33592 294 28 , , PUNCT bracis-33592 294 29 qiu qiu PROPN bracis-33592 294 30 , , PUNCT bracis-33592 294 31 x. x. NOUN bracis-33592 294 32 : : PUNCT bracis-33592 294 33 bert bert NOUN bracis-33592 294 34 - - PUNCT bracis-33592 294 35 attack attack NOUN bracis-33592 294 36 : : PUNCT bracis-33592 294 37 adversarial adversarial ADJ bracis-33592 294 38 attack attack NOUN bracis-33592 294 39 against against ADP bracis-33592 294 40 bert bert NOUN bracis-33592 294 41 using use VERB bracis-33592 294 42 bert bert NOUN bracis-33592 294 43 . . PUNCT bracis-33592 295 1 in in ADP bracis-33592 295 2 : : PUNCT bracis-33592 295 3 proceedings proceeding NOUN bracis-33592 295 4 of of ADP bracis-33592 295 5 the the DET bracis-33592 295 6 2020 2020 NUM bracis-33592 295 7 conference conference NOUN bracis-33592 295 8 on on ADP bracis-33592 295 9 empirical empirical ADJ bracis-33592 295 10 methods method NOUN bracis-33592 295 11 in in ADP bracis-33592 295 12 natural natural ADJ bracis-33592 295 13 language language NOUN bracis-33592 295 14 processing processing NOUN bracis-33592 295 15 , , PUNCT bracis-33592 295 16 pp pp ADP bracis-33592 295 17 . . PUNCT bracis-33592 296 1 6193–6202 6193–6202 NUM bracis-33592 296 2 ( ( PUNCT bracis-33592 296 3 2020 2020 NUM bracis-33592 296 4 ) ) PUNCT bracis-33592 296 5 google google PROPN bracis-33592 296 6 scholar scholar NOUN bracis-33592 296 7     SPACE bracis-33592 296 8 lim lim PROPN bracis-33592 296 9 , , PUNCT bracis-33592 296 10 c.t c.t PROPN bracis-33592 296 11 . . PROPN bracis-33592 296 12 , , PUNCT bracis-33592 296 13 bong bong PROPN bracis-33592 296 14 , , PUNCT bracis-33592 296 15 c.h c.h PROPN bracis-33592 296 16 . . PROPN bracis-33592 296 17 , , PUNCT bracis-33592 296 18 wong wong PROPN bracis-33592 296 19 , , PUNCT bracis-33592 296 20 w.s w.s PROPN bracis-33592 296 21 . . PROPN bracis-33592 296 22 , , PUNCT bracis-33592 296 23 lee lee PROPN bracis-33592 296 24 , , PUNCT bracis-33592 296 25 n.k n.k PROPN bracis-33592 296 26 . . PROPN bracis-33592 296 27 : : PUNCT bracis-33592 296 28 a a DET bracis-33592 296 29 comprehensive comprehensive ADJ bracis-33592 296 30 review review NOUN bracis-33592 296 31 of of ADP bracis-33592 296 32 automated automate VERB bracis-33592 296 33 essay essay NOUN bracis-33592 296 34 scoring scoring NOUN bracis-33592 296 35 ( ( PUNCT bracis-33592 296 36 aes aes NOUN bracis-33592 296 37 ) ) PUNCT bracis-33592 296 38 research research NOUN bracis-33592 296 39 and and CCONJ bracis-33592 296 40 development development NOUN bracis-33592 296 41 . . PUNCT bracis-33592 297 1 pertanika pertanika PROPN bracis-33592 297 2 j. j. PROPN bracis-33592 297 3 sci sci PROPN bracis-33592 297 4 . . PROPN bracis-33592 297 5 technol technol PROPN bracis-33592 297 6 . . PUNCT bracis-33592 298 1 29(3 29(3 NUM bracis-33592 298 2 ) ) PUNCT bracis-33592 298 3 , , PUNCT bracis-33592 298 4 1875–1899 1875–1899 NUM bracis-33592 298 5 ( ( PUNCT bracis-33592 298 6 2021 2021 NUM bracis-33592 298 7 ) ) PUNCT bracis-33592 298 8 article article NOUN bracis-33592 298 9     SPACE bracis-33592 298 10 math math NOUN bracis-33592 298 11     SPACE bracis-33592 298 12 google google PROPN bracis-33592 298 13 scholar scholar NOUN bracis-33592 298 14     SPACE bracis-33592 298 15 liu liu PROPN bracis-33592 298 16 , , PUNCT bracis-33592 298 17 j. j. PROPN bracis-33592 298 18 , , PUNCT bracis-33592 298 19 xu xu PROPN bracis-33592 298 20 , , PUNCT bracis-33592 298 21 y. y. PROPN bracis-33592 298 22 , , PUNCT bracis-33592 298 23 zhu zhu PROPN bracis-33592 298 24 , , PUNCT bracis-33592 298 25 y. y. PROPN bracis-33592 298 26 : : PUNCT bracis-33592 298 27 automated automate VERB bracis-33592 298 28 essay essay NOUN bracis-33592 298 29 scoring scoring NOUN bracis-33592 298 30 based base VERB bracis-33592 298 31 on on ADP bracis-33592 298 32 two two NUM bracis-33592 298 33 - - PUNCT bracis-33592 298 34 stage stage NOUN bracis-33592 298 35 learning learning NOUN bracis-33592 298 36 ( ( PUNCT bracis-33592 298 37 2019 2019 NUM bracis-33592 298 38 ) ) PUNCT bracis-33592 298 39 google google NOUN bracis-33592 298 40 scholar scholar NOUN bracis-33592 298 41     SPACE bracis-33592 298 42 marinho marinho NOUN bracis-33592 298 43 , , PUNCT bracis-33592 298 44 j. j. PROPN bracis-33592 298 45 , , PUNCT bracis-33592 298 46 anchiêta anchiêta NOUN bracis-33592 298 47 , , PUNCT bracis-33592 298 48 r. r. PROPN bracis-33592 298 49 , , PUNCT bracis-33592 298 50 moura moura PROPN bracis-33592 298 51 , , PUNCT bracis-33592 298 52 r. r. PROPN bracis-33592 298 53 : : PUNCT bracis-33592 298 54 essay essay PROPN bracis-33592 298 55 - - PUNCT bracis-33592 298 56 br br NOUN bracis-33592 298 57 : : PUNCT bracis-33592 298 58 a a DET bracis-33592 298 59 brazilian brazilian ADJ bracis-33592 298 60 corpus corpus NOUN bracis-33592 298 61 of of ADP bracis-33592 298 62 essays essay NOUN bracis-33592 298 63 . . PUNCT bracis-33592 299 1 in in ADP bracis-33592 299 2 : : PUNCT bracis-33592 299 3 anais anais PROPN bracis-33592 299 4 do do VERB bracis-33592 299 5 iii iii NUM bracis-33592 299 6 dataset dataset VERB bracis-33592 299 7 showcase showcase NOUN bracis-33592 299 8 workshop workshop NOUN bracis-33592 299 9 , , PUNCT bracis-33592 299 10 pp pp ADJ bracis-33592 299 11 . . PUNCT bracis-33592 300 1 53–64 53–64 NUM bracis-33592 300 2 ( ( PUNCT bracis-33592 300 3 2021 2021 NUM bracis-33592 300 4 ) ) PUNCT bracis-33592 300 5 google google NOUN bracis-33592 300 6 scholar scholar NOUN bracis-33592 300 7     SPACE bracis-33592 300 8 marinho marinho NOUN bracis-33592 300 9 , , PUNCT bracis-33592 300 10 j. j. PROPN bracis-33592 300 11 , , PUNCT bracis-33592 300 12 cordeiro cordeiro PROPN bracis-33592 300 13 , , PUNCT bracis-33592 300 14 f. f. PROPN bracis-33592 300 15 , , PUNCT bracis-33592 300 16 anchiêta anchiêta NOUN bracis-33592 300 17 , , PUNCT bracis-33592 300 18 r. r. PROPN bracis-33592 300 19 , , PUNCT bracis-33592 300 20 moura moura PROPN bracis-33592 300 21 , , PUNCT bracis-33592 300 22 r. r. PROPN bracis-33592 300 23 : : PUNCT bracis-33592 300 24 automated automate VERB bracis-33592 300 25 essay essay ADJ bracis-33592 300 26 scoring scoring NOUN bracis-33592 300 27 : : PUNCT bracis-33592 300 28 an an DET bracis-33592 300 29 approach approach NOUN bracis-33592 300 30 based base VERB bracis-33592 300 31 on on ADP bracis-33592 300 32 enem enem PROPN bracis-33592 300 33 competencies competency NOUN bracis-33592 300 34 . . PUNCT bracis-33592 301 1 in in ADP bracis-33592 301 2 : : PUNCT bracis-33592 301 3 anais anais PROPN bracis-33592 301 4 do do AUX bracis-33592 301 5 xix xix NOUN bracis-33592 301 6 encontro encontro VERB bracis-33592 301 7 nacional nacional ADJ bracis-33592 301 8 de de X bracis-33592 301 9 inteligência inteligência PROPN bracis-33592 301 10 artificial artificial PROPN bracis-33592 301 11 e e NOUN bracis-33592 301 12 computacional computacional NOUN bracis-33592 301 13 , , PUNCT bracis-33592 301 14 pp pp ADJ bracis-33592 301 15 . . PUNCT bracis-33592 302 1 49–60 49–60 NUM bracis-33592 302 2 ( ( PUNCT bracis-33592 302 3 2022 2022 NUM bracis-33592 302 4 ) ) PUNCT bracis-33592 302 5 google google NOUN bracis-33592 302 6 scholar scholar NOUN bracis-33592 302 7     SPACE bracis-33592 302 8 mizumoto mizumoto NOUN bracis-33592 302 9 , , PUNCT bracis-33592 302 10 a. a. NOUN bracis-33592 302 11 , , PUNCT bracis-33592 302 12 eguchi eguchi NOUN bracis-33592 302 13 , , PUNCT bracis-33592 302 14 m. m. NOUN bracis-33592 302 15 : : PUNCT bracis-33592 302 16 exploring explore VERB bracis-33592 302 17 the the DET bracis-33592 302 18 potential potential NOUN bracis-33592 302 19 of of ADP bracis-33592 302 20 using use VERB bracis-33592 302 21 an an DET bracis-33592 302 22 ai ai NOUN bracis-33592 302 23 language language NOUN bracis-33592 302 24 model model NOUN bracis-33592 302 25 for for ADP bracis-33592 302 26 automated automate VERB bracis-33592 302 27 essay essay ADJ bracis-33592 302 28 scoring scoring NOUN bracis-33592 302 29 . . PUNCT bracis-33592 303 1 res re NOUN bracis-33592 303 2 . . PUNCT bracis-33592 304 1 methods method NOUN bracis-33592 304 2 appl appl PROPN bracis-33592 304 3 . . PUNCT bracis-33592 305 1 linguist linguist NOUN bracis-33592 305 2 . . PUNCT bracis-33592 306 1 2(2 2(2 NUM bracis-33592 306 2 ) ) PUNCT bracis-33592 306 3 , , PUNCT bracis-33592 306 4 100050 100050 NUM bracis-33592 306 5 ( ( PUNCT bracis-33592 306 6 2023 2023 NUM bracis-33592 306 7 ) ) PUNCT bracis-33592 306 8 article article NOUN bracis-33592 306 9     SPACE bracis-33592 306 10 math math NOUN bracis-33592 306 11     SPACE bracis-33592 306 12 google google PROPN bracis-33592 306 13 scholar scholar NOUN bracis-33592 306 14     SPACE bracis-33592 306 15 nvidia nvidia PROPN bracis-33592 306 16 , , PUNCT bracis-33592 306 17 adler adler PROPN bracis-33592 306 18 , , PUNCT bracis-33592 306 19 b. b. PROPN bracis-33592 306 20 , , PUNCT bracis-33592 306 21 agarwal agarwal PROPN bracis-33592 306 22 , , PUNCT bracis-33592 306 23 n. n. NOUN bracis-33592 306 24 , , PUNCT bracis-33592 306 25 aithal aithal NOUN bracis-33592 306 26 , , PUNCT bracis-33592 306 27 a. a. NOUN bracis-33592 306 28 , , PUNCT bracis-33592 306 29 anh anh PROPN bracis-33592 306 30 , , PUNCT bracis-33592 306 31 d.h d.h PROPN bracis-33592 306 32 . . PROPN bracis-33592 306 33 , , PUNCT bracis-33592 307 1 et et PROPN bracis-33592 307 2     SPACE bracis-33592 307 3 al al PROPN bracis-33592 307 4 . . PROPN bracis-33592 307 5 : : PUNCT bracis-33592 308 1 nemotron-4 nemotron-4 PROPN bracis-33592 308 2 340b 340b NUM bracis-33592 308 3 technical technical ADJ bracis-33592 308 4 report report NOUN bracis-33592 308 5 ( ( PUNCT bracis-33592 308 6 2024 2024 NUM bracis-33592 308 7 ) ) PUNCT bracis-33592 308 8 google google NOUN bracis-33592 308 9 scholar scholar NOUN bracis-33592 308 10     SPACE bracis-33592 308 11 page page NOUN bracis-33592 308 12 , , PUNCT bracis-33592 308 13 e.b e.b PROPN bracis-33592 308 14 . . PUNCT bracis-33592 308 15 : : PUNCT bracis-33592 309 1 the the DET bracis-33592 309 2 imminence imminence NOUN bracis-33592 309 3 of of ADP bracis-33592 309 4 ... ... PUNCT bracis-33592 309 5 grading grade VERB bracis-33592 309 6 essays essay NOUN bracis-33592 309 7 by by ADP bracis-33592 309 8 computer computer NOUN bracis-33592 309 9 . . PUNCT bracis-33592 310 1 in in ADP bracis-33592 310 2 : : PUNCT bracis-33592 310 3 the the DET bracis-33592 310 4 phi phi NOUN bracis-33592 310 5 delta delta NOUN bracis-33592 310 6 kappan kappan NOUN bracis-33592 310 7 , , PUNCT bracis-33592 310 8 pp pp ADJ bracis-33592 310 9 . . PUNCT bracis-33592 311 1 238–243 238–243 NUM bracis-33592 311 2 ( ( PUNCT bracis-33592 311 3 1966 1966 NUM bracis-33592 311 4 ) ) PUNCT bracis-33592 311 5 google google NOUN bracis-33592 311 6 scholar scholar NOUN bracis-33592 311 7     SPACE bracis-33592 311 8 pedregosa pedregosa PROPN bracis-33592 311 9 , , PUNCT bracis-33592 311 10 f. f. PROPN bracis-33592 311 11 , , PUNCT bracis-33592 311 12 et et PROPN bracis-33592 311 13 al al PROPN bracis-33592 311 14 . . PROPN bracis-33592 311 15 : : PUNCT bracis-33592 312 1 scikit scikit NOUN bracis-33592 312 2 - - PUNCT bracis-33592 312 3 learn learn VERB bracis-33592 312 4 : : PUNCT bracis-33592 312 5 machine machine NOUN bracis-33592 312 6 learning learning NOUN bracis-33592 312 7 in in ADP bracis-33592 312 8 python python PROPN bracis-33592 312 9 . . PUNCT bracis-33592 313 1 j. j. PROPN bracis-33592 313 2 mach mach PROPN bracis-33592 313 3 . . PUNCT bracis-33592 314 1 learn learn VERB bracis-33592 314 2 . . PUNCT bracis-33592 315 1 res re NOUN bracis-33592 315 2 . . PROPN bracis-33592 316 1 12 12 NUM bracis-33592 316 2 , , PUNCT bracis-33592 316 3 2825–2830 2825–2830 NUM bracis-33592 316 4 ( ( PUNCT bracis-33592 316 5 2011 2011 NUM bracis-33592 316 6 ) ) PUNCT bracis-33592 316 7 mathscinet mathscinet NOUN bracis-33592 316 8     SPACE bracis-33592 316 9 math math NOUN bracis-33592 316 10     SPACE bracis-33592 316 11 google google PROPN bracis-33592 316 12 scholar scholar NOUN bracis-33592 316 13     SPACE bracis-33592 316 14 silveira silveira PROPN bracis-33592 316 15 , , PUNCT bracis-33592 316 16 i.c i.c PROPN bracis-33592 316 17 . . PROPN bracis-33592 316 18 , , PUNCT bracis-33592 316 19 barbosa barbosa PROPN bracis-33592 316 20 , , PUNCT bracis-33592 316 21 a. a. PROPN bracis-33592 316 22 , , PUNCT bracis-33592 316 23 mauá mauá NOUN bracis-33592 316 24 , , PUNCT bracis-33592 316 25 d.d d.d PROPN bracis-33592 316 26 . . PROPN bracis-33592 316 27 : : PUNCT bracis-33592 317 1 a a DET bracis-33592 317 2 new new ADJ bracis-33592 317 3 benchmark benchmark NOUN bracis-33592 317 4 for for ADP bracis-33592 317 5 automatic automatic ADJ bracis-33592 317 6 essay essay NOUN bracis-33592 317 7 scoring scoring NOUN bracis-33592 317 8 in in ADP bracis-33592 317 9 portuguese portuguese PROPN bracis-33592 317 10 . . PUNCT bracis-33592 318 1 in in ADP bracis-33592 318 2 : : PUNCT bracis-33592 318 3 proceedings proceeding NOUN bracis-33592 318 4 of of ADP bracis-33592 318 5 the the DET bracis-33592 318 6 16th 16th ADJ bracis-33592 318 7 international international ADJ bracis-33592 318 8 conference conference NOUN bracis-33592 318 9 on on ADP bracis-33592 318 10 computational computational ADJ bracis-33592 318 11 processing processing NOUN bracis-33592 318 12 of of ADP bracis-33592 318 13 portuguese portuguese ADJ bracis-33592 318 14 , , PUNCT bracis-33592 318 15 vol vol NOUN bracis-33592 318 16 . . PROPN bracis-33592 319 1 1 1 NUM bracis-33592 319 2 . . X bracis-33592 320 1 pp pp ADJ bracis-33592 320 2 . . PUNCT bracis-33592 321 1 228–237 228–237 NUM bracis-33592 321 2 ( ( PUNCT bracis-33592 321 3 2024 2024 NUM bracis-33592 321 4 ) ) PUNCT bracis-33592 321 5 google google NOUN bracis-33592 321 6 scholar scholar NOUN bracis-33592 321 7     SPACE bracis-33592 321 8 singh singh PROPN bracis-33592 321 9 , , PUNCT bracis-33592 321 10 a. a. PROPN bracis-33592 321 11 , , PUNCT bracis-33592 321 12 pandey pandey PROPN bracis-33592 321 13 , , PUNCT bracis-33592 321 14 n. n. NOUN bracis-33592 321 15 , , PUNCT bracis-33592 321 16 shirgaonkar shirgaonkar NOUN bracis-33592 321 17 , , PUNCT bracis-33592 321 18 a. a. PROPN bracis-33592 321 19 , , PUNCT bracis-33592 321 20 manoj manoj PROPN bracis-33592 321 21 , , PUNCT bracis-33592 321 22 p. p. NOUN bracis-33592 321 23 , , PUNCT bracis-33592 321 24 aski aski PROPN bracis-33592 321 25 , , PUNCT bracis-33592 321 26 v. v. ADV bracis-33592 321 27 : : PUNCT bracis-33592 321 28 a a DET bracis-33592 321 29 study study NOUN bracis-33592 321 30 of of ADP bracis-33592 321 31 optimizations optimization NOUN bracis-33592 321 32 for for ADP bracis-33592 321 33 fine fine ADV bracis-33592 321 34 - - PUNCT bracis-33592 321 35 tuning tune VERB bracis-33592 321 36 large large ADJ bracis-33592 321 37 language language NOUN bracis-33592 321 38 models model NOUN bracis-33592 321 39 ( ( PUNCT bracis-33592 321 40 2024 2024 NUM bracis-33592 321 41 ) ) PUNCT bracis-33592 321 42 google google NOUN bracis-33592 321 43 scholar scholar NOUN bracis-33592 321 44     SPACE bracis-33592 321 45 souza souza PROPN bracis-33592 321 46 , , PUNCT bracis-33592 321 47 f. f. PROPN bracis-33592 321 48 , , PUNCT bracis-33592 321 49 nogueira nogueira PROPN bracis-33592 321 50 , , PUNCT bracis-33592 321 51 r. r. PROPN bracis-33592 321 52 , , PUNCT bracis-33592 321 53 lotufo lotufo PROPN bracis-33592 321 54 , , PUNCT bracis-33592 321 55 r. r. PROPN bracis-33592 321 56 : : PUNCT bracis-33592 321 57 bertimbau bertimbau PROPN bracis-33592 321 58 : : PUNCT bracis-33592 321 59 pretrained pretraine VERB bracis-33592 321 60 bert bert NOUN bracis-33592 321 61 models model NOUN bracis-33592 321 62 for for ADP bracis-33592 321 63 brazilian brazilian ADJ bracis-33592 321 64 portuguese portuguese NOUN bracis-33592 321 65 . . PUNCT bracis-33592 322 1 in in ADP bracis-33592 322 2 : : PUNCT bracis-33592 322 3 cerri cerri PROPN bracis-33592 322 4 , , PUNCT bracis-33592 322 5 r. r. PROPN bracis-33592 322 6 , , PUNCT bracis-33592 322 7 prati prati PROPN bracis-33592 322 8 , , PUNCT bracis-33592 322 9 r.c r.c PROPN bracis-33592 322 10 . . PROPN bracis-33592 322 11 ( ( PUNCT bracis-33592 322 12 eds ed NOUN bracis-33592 322 13 . . PUNCT bracis-33592 322 14 ) ) PUNCT bracis-33592 322 15 bracis bracis PROPN bracis-33592 322 16 2020 2020 NUM bracis-33592 322 17 . . PUNCT bracis-33592 323 1 lncs lncs PROPN bracis-33592 323 2 ( ( PUNCT bracis-33592 323 3 lnai lnai ADJ bracis-33592 323 4 ) ) PUNCT bracis-33592 323 5 , , PUNCT bracis-33592 323 6 vol vol NOUN bracis-33592 323 7 . . PUNCT bracis-33592 323 8 12319 12319 NUM bracis-33592 323 9 , , PUNCT bracis-33592 323 10 pp pp ADP bracis-33592 323 11 . . PUNCT bracis-33592 324 1 403–417 403–417 NUM bracis-33592 324 2 . . PUNCT bracis-33592 324 3 springer springer NOUN bracis-33592 324 4 , , PUNCT bracis-33592 324 5 cham cham PROPN bracis-33592 324 6 ( ( PUNCT bracis-33592 324 7 2020 2020 NUM bracis-33592 324 8 ) ) PUNCT bracis-33592 324 9 . . PUNCT bracis-33592 325 1 https://doi.org/10.1007/978-3-030-61377-8_28 https://doi.org/10.1007/978-3-030-61377-8_28 PROPN bracis-33592 325 2 chapter chapter NOUN bracis-33592 325 3     SPACE bracis-33592 325 4 google google PROPN bracis-33592 325 5 scholar scholar NOUN bracis-33592 325 6     SPACE bracis-33592 325 7 taghipour taghipour NOUN bracis-33592 325 8 , , PUNCT bracis-33592 325 9 k. k. PROPN bracis-33592 325 10 , , PUNCT bracis-33592 325 11 ng ng PROPN bracis-33592 325 12 , , PUNCT bracis-33592 325 13 h.t h.t PROPN bracis-33592 325 14 . . PROPN bracis-33592 325 15 : : PUNCT bracis-33592 326 1 a a DET bracis-33592 326 2 neural neural ADJ bracis-33592 326 3 approach approach NOUN bracis-33592 326 4 to to ADP bracis-33592 326 5 automated automate VERB bracis-33592 326 6 essay essay ADJ bracis-33592 326 7 scoring scoring NOUN bracis-33592 326 8 . . PUNCT bracis-33592 327 1 in in ADP bracis-33592 327 2 : : PUNCT bracis-33592 327 3 proceedings proceeding NOUN bracis-33592 327 4 of of ADP bracis-33592 327 5 the the DET bracis-33592 327 6 2016 2016 NUM bracis-33592 327 7 conference conference NOUN bracis-33592 327 8 on on ADP bracis-33592 327 9 empirical empirical ADJ bracis-33592 327 10 methods method NOUN bracis-33592 327 11 in in ADP bracis-33592 327 12 natural natural ADJ bracis-33592 327 13 language language NOUN bracis-33592 327 14 processing processing NOUN bracis-33592 327 15 , , PUNCT bracis-33592 327 16 pp pp ADJ bracis-33592 327 17 . . PUNCT bracis-33592 327 18 1882–1891 1882–1891 NUM bracis-33592 327 19 . . PUNCT bracis-33592 328 1 association association NOUN bracis-33592 328 2 for for ADP bracis-33592 328 3 computational computational ADJ bracis-33592 328 4 linguistics linguistic NOUN bracis-33592 328 5 ( ( PUNCT bracis-33592 328 6 2016 2016 NUM bracis-33592 328 7 ) ) PUNCT bracis-33592 328 8 google google PROPN bracis-33592 328 9 scholar scholar NOUN bracis-33592 328 10     SPACE bracis-33592 328 11 team team NOUN bracis-33592 328 12 , , PUNCT bracis-33592 328 13 g. g. PROPN bracis-33592 328 14 , , PUNCT bracis-33592 328 15 georgiev georgiev NOUN bracis-33592 328 16 , , PUNCT bracis-33592 328 17 p. p. PROPN bracis-33592 328 18 , , PUNCT bracis-33592 328 19 lei lei PROPN bracis-33592 328 20 , , PUNCT bracis-33592 328 21 v.i v.i PROPN bracis-33592 328 22 . . PROPN bracis-33592 328 23 , , PUNCT bracis-33592 328 24 burnell burnell PROPN bracis-33592 328 25 , , PUNCT bracis-33592 328 26 r. r. PROPN bracis-33592 328 27 , , PUNCT bracis-33592 328 28 bai bai PROPN bracis-33592 328 29 , , PUNCT bracis-33592 328 30 l. l. PROPN bracis-33592 328 31 , , PUNCT bracis-33592 328 32 et et PROPN bracis-33592 328 33     SPACE bracis-33592 328 34 al al PROPN bracis-33592 328 35 . . PROPN bracis-33592 328 36 : : PUNCT bracis-33592 328 37 gemini gemini PROPN bracis-33592 328 38 1.5 1.5 NUM bracis-33592 328 39 : : PUNCT bracis-33592 328 40 unlocking unlock VERB bracis-33592 328 41 multimodal multimodal ADJ bracis-33592 328 42 understanding understanding NOUN bracis-33592 328 43 across across ADP bracis-33592 328 44 millions million NOUN bracis-33592 328 45 of of ADP bracis-33592 328 46 tokens token NOUN bracis-33592 328 47 of of ADP bracis-33592 328 48 context context NOUN bracis-33592 328 49 ( ( PUNCT bracis-33592 328 50 2024 2024 NUM bracis-33592 328 51 ) ) PUNCT bracis-33592 328 52 google google NOUN bracis-33592 328 53 scholar scholar NOUN bracis-33592 328 54     SPACE bracis-33592 328 55 touvron touvron ADJ bracis-33592 328 56 , , PUNCT bracis-33592 328 57 h. h. PROPN bracis-33592 328 58 , , PUNCT bracis-33592 328 59 et et PROPN bracis-33592 328 60 al al PROPN bracis-33592 328 61 . . PUNCT bracis-33592 328 62 : : PUNCT bracis-33592 329 1 llama llama NOUN bracis-33592 329 2 : : PUNCT bracis-33592 329 3 open open ADJ bracis-33592 329 4 and and CCONJ bracis-33592 329 5 efficient efficient ADJ bracis-33592 329 6 foundation foundation NOUN bracis-33592 329 7 language language NOUN bracis-33592 329 8 models model NOUN bracis-33592 329 9 ( ( PUNCT bracis-33592 329 10 2023 2023 NUM bracis-33592 329 11 ) ) PUNCT bracis-33592 329 12 google google NOUN bracis-33592 329 13 scholar scholar NOUN bracis-33592 329 14     SPACE bracis-33592 329 15 wangkriangkri wangkriangkri NOUN bracis-33592 329 16 , , PUNCT bracis-33592 329 17 p. p. NOUN bracis-33592 329 18 , , PUNCT bracis-33592 329 19 viboonlarp viboonlarp NOUN bracis-33592 329 20 , , PUNCT bracis-33592 329 21 c. c. PROPN bracis-33592 329 22 , , PUNCT bracis-33592 329 23 rutherford rutherford PROPN bracis-33592 329 24 , , PUNCT bracis-33592 329 25 a.t a.t PROPN bracis-33592 329 26 . . PROPN bracis-33592 329 27 , , PUNCT bracis-33592 329 28 chuangsuwanich chuangsuwanich PROPN bracis-33592 329 29 , , PUNCT bracis-33592 329 30 e. e. PROPN bracis-33592 329 31 : : PUNCT bracis-33592 329 32 a a DET bracis-33592 329 33 comparative comparative ADJ bracis-33592 329 34 study study NOUN bracis-33592 329 35 of of ADP bracis-33592 329 36 pretrained pretraine VERB bracis-33592 329 37 language language NOUN bracis-33592 329 38 models model NOUN bracis-33592 329 39 for for ADP bracis-33592 329 40 automated automate VERB bracis-33592 329 41 essay essay NOUN bracis-33592 329 42 scoring score VERB bracis-33592 329 43 with with ADP bracis-33592 329 44 adversarial adversarial ADJ bracis-33592 329 45 inputs input NOUN bracis-33592 329 46 . . PUNCT bracis-33592 330 1 in in ADP bracis-33592 330 2 : : PUNCT bracis-33592 330 3 ieee ieee NOUN bracis-33592 330 4 region region NOUN bracis-33592 330 5 10 10 NUM bracis-33592 330 6 international international ADJ bracis-33592 330 7 conference conference NOUN bracis-33592 330 8 tencon tencon NOUN bracis-33592 330 9 , , PUNCT bracis-33592 330 10 pp pp ADV bracis-33592 330 11 . . PUNCT bracis-33592 331 1 875–880 875–880 NUM bracis-33592 331 2 ( ( PUNCT bracis-33592 331 3 2020 2020 NUM bracis-33592 331 4 ) ) PUNCT bracis-33592 331 5 google google PROPN bracis-33592 331 6 scholar scholar NOUN bracis-33592 331 7     SPACE bracis-33592 331 8 whitley whitley PROPN bracis-33592 331 9 , , PUNCT bracis-33592 331 10 b.e b.e PROPN bracis-33592 331 11 . . PROPN bracis-33592 331 12 : : PUNCT bracis-33592 331 13 factors factor NOUN bracis-33592 331 14 associated associate VERB bracis-33592 331 15 with with ADP bracis-33592 331 16 cheating cheat VERB bracis-33592 331 17 among among ADP bracis-33592 331 18 college college NOUN bracis-33592 331 19 students student NOUN bracis-33592 331 20 : : PUNCT bracis-33592 331 21 a a DET bracis-33592 331 22 review review NOUN bracis-33592 331 23 . . PUNCT bracis-33592 332 1 res re NOUN bracis-33592 332 2 . . PROPN bracis-33592 333 1 high high PROPN bracis-33592 333 2 . . PUNCT bracis-33592 334 1 educ educ PROPN bracis-33592 334 2 . . PUNCT bracis-33592 335 1 39(3 39(3 NUM bracis-33592 335 2 ) ) PUNCT bracis-33592 335 3 , , PUNCT bracis-33592 335 4 235–274 235–274 NUM bracis-33592 335 5 ( ( PUNCT bracis-33592 335 6 1998 1998 NUM bracis-33592 335 7 ) ) PUNCT bracis-33592 335 8 article article NOUN bracis-33592 335 9     SPACE bracis-33592 335 10 math math NOUN bracis-33592 335 11     SPACE bracis-33592 335 12 google google PROPN bracis-33592 335 13 scholar scholar NOUN bracis-33592 335 14     SPACE bracis-33592 335 15 woods wood NOUN bracis-33592 335 16 , , PUNCT bracis-33592 335 17 b. b. PROPN bracis-33592 335 18 , , PUNCT bracis-33592 335 19 adamson adamson PROPN bracis-33592 335 20 , , PUNCT bracis-33592 335 21 d. d. PROPN bracis-33592 335 22 , , PUNCT bracis-33592 335 23 miel miel PROPN bracis-33592 335 24 , , PUNCT bracis-33592 335 25 s. s. PROPN bracis-33592 335 26 , , PUNCT bracis-33592 335 27 mayfield mayfield PROPN bracis-33592 335 28 , , PUNCT bracis-33592 335 29 e. e. PROPN bracis-33592 335 30 : : PUNCT bracis-33592 335 31 formative formative ADJ bracis-33592 335 32 essay essay ADJ bracis-33592 335 33 feedback feedback NOUN bracis-33592 335 34 using use VERB bracis-33592 335 35 predictive predictive ADJ bracis-33592 335 36 scoring scoring NOUN bracis-33592 335 37 models model NOUN bracis-33592 335 38 . . PUNCT bracis-33592 336 1 in in ADP bracis-33592 336 2 : : PUNCT bracis-33592 336 3 proceedings proceeding NOUN bracis-33592 336 4 of of ADP bracis-33592 336 5 the the DET bracis-33592 336 6 23rd 23rd ADJ bracis-33592 336 7 acm acm PROPN bracis-33592 336 8 sigkdd sigkdd NOUN bracis-33592 336 9 international international ADJ bracis-33592 336 10 conference conference NOUN bracis-33592 336 11 on on ADP bracis-33592 336 12 knowledge knowledge NOUN bracis-33592 336 13 discovery discovery PROPN bracis-33592 336 14 and and CCONJ bracis-33592 336 15 data datum NOUN bracis-33592 336 16 mining mining NOUN bracis-33592 336 17 , , PUNCT bracis-33592 336 18 pp pp ADP bracis-33592 336 19 . . PUNCT bracis-33592 336 20 2071–2080 2071–2080 NUM bracis-33592 336 21 ( ( PUNCT bracis-33592 336 22 2017 2017 NUM bracis-33592 336 23 ) ) PUNCT bracis-33592 336 24 google google PROPN bracis-33592 336 25 scholar scholar NOUN bracis-33592 336 26     SPACE bracis-33592 336 27 zhang zhang PROPN bracis-33592 336 28 , , PUNCT bracis-33592 336 29 h. h. PROPN bracis-33592 336 30 , , PUNCT bracis-33592 336 31 li li PROPN bracis-33592 336 32 , , PUNCT bracis-33592 336 33 l.h l.h PROPN bracis-33592 336 34 . . PROPN bracis-33592 336 35 , , PUNCT bracis-33592 336 36 meng meng PROPN bracis-33592 336 37 , , PUNCT bracis-33592 336 38 t. t. PROPN bracis-33592 336 39 , , PUNCT bracis-33592 336 40 chang chang PROPN bracis-33592 336 41 , , PUNCT bracis-33592 336 42 k.w k.w PROPN bracis-33592 336 43 . . PROPN bracis-33592 336 44 , , PUNCT bracis-33592 336 45 van van PROPN bracis-33592 336 46     SPACE bracis-33592 336 47 den den PROPN bracis-33592 336 48 broeck broeck NOUN bracis-33592 336 49 , , PUNCT bracis-33592 336 50 g. g. PROPN bracis-33592 336 51 : : PUNCT bracis-33592 336 52 on on ADP bracis-33592 336 53 the the DET bracis-33592 336 54 paradox paradox NOUN bracis-33592 336 55 of of ADP bracis-33592 336 56 learning learn VERB bracis-33592 336 57 to to PART bracis-33592 336 58 reason reason VERB bracis-33592 336 59 from from ADP bracis-33592 336 60 data datum NOUN bracis-33592 336 61 . . PUNCT bracis-33592 337 1 in in ADP bracis-33592 337 2 : : PUNCT bracis-33592 337 3 proceedings proceeding NOUN bracis-33592 337 4 of of ADP bracis-33592 337 5 the the DET bracis-33592 337 6 thirty thirty NUM bracis-33592 337 7 - - PUNCT bracis-33592 337 8 second second NOUN bracis-33592 337 9 international international ADJ bracis-33592 337 10 joint joint ADJ bracis-33592 337 11 conference conference NOUN bracis-33592 337 12 on on ADP bracis-33592 337 13 artificial artificial ADJ bracis-33592 337 14 intelligence intelligence NOUN bracis-33592 337 15 , , PUNCT bracis-33592 337 16 pp pp X bracis-33592 337 17 . . PUNCT bracis-33592 338 1 3365–3373 3365–3373 NUM bracis-33592 338 2 ( ( PUNCT bracis-33592 338 3 2023 2023 NUM bracis-33592 338 4 ) ) PUNCT bracis-33592 338 5 google google PROPN bracis-33592 338 6 scholar scholar NOUN bracis-33592 338 7     SPACE bracis-33592 338 8 zhang zhang PROPN bracis-33592 338 9 , , PUNCT bracis-33592 338 10 j. j. PROPN bracis-33592 338 11 , , PUNCT bracis-33592 338 12 wang wang PROPN bracis-33592 338 13 , , PUNCT bracis-33592 338 14 j. j. PROPN bracis-33592 338 15 , , PUNCT bracis-33592 338 16 luo luo PROPN bracis-33592 338 17 , , PUNCT bracis-33592 338 18 x. x. PROPN bracis-33592 338 19 , , PUNCT bracis-33592 338 20 ma ma PROPN bracis-33592 338 21 , , PUNCT bracis-33592 338 22 b. b. PROPN bracis-33592 338 23 , , PUNCT bracis-33592 338 24 xiong xiong PROPN bracis-33592 338 25 , , PUNCT bracis-33592 338 26 n. n. NOUN bracis-33592 338 27 : : PUNCT bracis-33592 338 28 imperceptible imperceptible ADJ bracis-33592 338 29 and and CCONJ bracis-33592 338 30 reliable reliable ADJ bracis-33592 338 31 adversarial adversarial ADJ bracis-33592 338 32 attack attack NOUN bracis-33592 338 33 . . PUNCT bracis-33592 339 1 in in ADP bracis-33592 339 2 : : PUNCT bracis-33592 339 3 cao cao PROPN bracis-33592 339 4 , , PUNCT bracis-33592 339 5 c. c. PROPN bracis-33592 339 6 , , PUNCT bracis-33592 339 7 zhang zhang PROPN bracis-33592 339 8 , , PUNCT bracis-33592 339 9 y. y. PROPN bracis-33592 339 10 , , PUNCT bracis-33592 339 11 hong hong PROPN bracis-33592 339 12 , , PUNCT bracis-33592 339 13 y. y. PROPN bracis-33592 339 14 , , PUNCT bracis-33592 339 15 wang wang PROPN bracis-33592 339 16 , , PUNCT bracis-33592 339 17 d. d. PROPN bracis-33592 339 18 ( ( PUNCT bracis-33592 339 19 eds eds PROPN bracis-33592 339 20 . . PUNCT bracis-33592 339 21 ) ) PUNCT bracis-33592 339 22 fcs fc NOUN bracis-33592 339 23 2021 2021 NUM bracis-33592 339 24 . . PUNCT bracis-33592 340 1 ccis ccis PROPN bracis-33592 340 2 , , PUNCT bracis-33592 340 3 vol vol NOUN bracis-33592 340 4 . . PROPN bracis-33592 340 5 1558 1558 NUM bracis-33592 340 6 , , PUNCT bracis-33592 340 7 pp pp ADJ bracis-33592 340 8 . . PUNCT bracis-33592 341 1 49–62 49–62 NUM bracis-33592 341 2 . . PUNCT bracis-33592 341 3 springer springer NOUN bracis-33592 341 4 , , PUNCT bracis-33592 341 5 singapore singapore PROPN bracis-33592 341 6 ( ( PUNCT bracis-33592 341 7 2022 2022 NUM bracis-33592 341 8 ) ) PUNCT bracis-33592 341 9 . . PUNCT bracis-33592 342 1 https://doi.org/10.1007/978-981-19-0523-0_4 https://doi.org/10.1007/978-981-19-0523-0_4 NOUN bracis-33592 342 2 chapter chapter NOUN bracis-33592 342 3     SPACE bracis-33592 342 4 math math NOUN bracis-33592 342 5     SPACE bracis-33592 342 6 google google PROPN bracis-33592 342 7 scholar scholar NOUN bracis-33592 342 8     SPACE bracis-33592 342 9 download download NOUN bracis-33592 342 10 references reference NOUN bracis-33592 342 11 acknowledgments acknowledgment VERB bracis-33592 342 12 the the DET bracis-33592 342 13 authors author NOUN bracis-33592 342 14 of of ADP bracis-33592 342 15 this this DET bracis-33592 342 16 work work NOUN bracis-33592 342 17 would would AUX bracis-33592 342 18 like like VERB bracis-33592 342 19 to to PART bracis-33592 342 20 thank thank VERB bracis-33592 342 21 the the DET bracis-33592 342 22 center center NOUN bracis-33592 342 23 for for ADP bracis-33592 342 24 artificial artificial ADJ bracis-33592 342 25 intelligence intelligence NOUN bracis-33592 342 26 ( ( PUNCT bracis-33592 342 27 c4ai c4ai NOUN bracis-33592 342 28 - - PUNCT bracis-33592 342 29 usp usp NOUN bracis-33592 342 30 ) ) PUNCT bracis-33592 342 31 and and CCONJ bracis-33592 342 32 the the DET bracis-33592 342 33 support support NOUN bracis-33592 342 34 from from ADP bracis-33592 342 35 the the DET bracis-33592 342 36 são são PROPN bracis-33592 342 37 paulo paulo PROPN bracis-33592 342 38 research research PROPN bracis-33592 342 39 foundation foundation PROPN bracis-33592 342 40 ( ( PUNCT bracis-33592 342 41 fapesp fapesp NOUN bracis-33592 342 42 grants grant VERB bracis-33592 342 43 # # SYM bracis-33592 342 44 2019/07665 2019/07665 NUM bracis-33592 342 45 - - SYM bracis-33592 342 46 4 4 NUM bracis-33592 342 47 and and CCONJ bracis-33592 342 48 2022/02937 2022/02937 NUM bracis-33592 342 49 - - SYM bracis-33592 342 50 9 9 NUM bracis-33592 342 51 ) ) PUNCT bracis-33592 342 52 and and CCONJ bracis-33592 342 53 from from ADP bracis-33592 342 54 the the DET bracis-33592 342 55 ibm ibm PROPN bracis-33592 342 56 corporation corporation NOUN bracis-33592 342 57 . . PUNCT bracis-33592 343 1 this this DET bracis-33592 343 2 work work NOUN bracis-33592 343 3 was be AUX bracis-33592 343 4 also also ADV bracis-33592 343 5 supported support VERB bracis-33592 343 6 by by ADP bracis-33592 343 7 cnpq cnpq PROPN bracis-33592 343 8 grant grant VERB bracis-33592 343 9 no no INTJ bracis-33592 343 10 . . PUNCT bracis-33592 344 1 305136/2022 305136/2022 NUM bracis-33592 344 2 - - SYM bracis-33592 344 3 4 4 NUM bracis-33592 344 4 and and CCONJ bracis-33592 344 5 capes cape NOUN bracis-33592 344 6 finance finance PROPN bracis-33592 344 7 code code NOUN bracis-33592 344 8 001 001 NUM bracis-33592 344 9 . . PUNCT bracis-33592 345 1 author author NOUN bracis-33592 345 2 information information NOUN bracis-33592 345 3 authors author NOUN bracis-33592 345 4 and and CCONJ bracis-33592 345 5 affiliations affiliation NOUN bracis-33592 345 6 universidade universidade PROPN bracis-33592 345 7 de de PROPN bracis-33592 345 8 são são PROPN bracis-33592 345 9 paulo paulo PROPN bracis-33592 345 10 , , PUNCT bracis-33592 345 11 são são PROPN bracis-33592 345 12 paulo paulo PROPN bracis-33592 345 13 , , PUNCT bracis-33592 345 14 brazil brazil PROPN bracis-33592 345 15 igor igor PROPN bracis-33592 345 16 cataneo cataneo PROPN bracis-33592 345 17 silveira silveira PROPN bracis-33592 345 18 , , PUNCT bracis-33592 345 19     SPACE bracis-33592 345 20 andré andré PROPN bracis-33592 345 21 barbosa barbosa PROPN bracis-33592 345 22 , , PUNCT bracis-33592 345 23     SPACE bracis-33592 345 24 daniel daniel PROPN bracis-33592 345 25 silva silva PROPN bracis-33592 345 26 lopes lopes PROPN bracis-33592 345 27 da da PROPN bracis-33592 345 28 costa costa PROPN bracis-33592 345 29     SPACE bracis-33592 345 30 & & CCONJ bracis-33592 345 31     SPACE bracis-33592 345 32 denis denis PROPN bracis-33592 345 33 deratani deratani PROPN bracis-33592 345 34 mauá mauá PROPN bracis-33592 345 35 authors author NOUN bracis-33592 345 36 igor igor PROPN bracis-33592 345 37 cataneo cataneo NOUN bracis-33592 345 38 silveiraview silveiraview PROPN bracis-33592 345 39 author author NOUN bracis-33592 345 40 publications publication NOUN bracis-33592 345 41 search search NOUN bracis-33592 345 42 author author NOUN bracis-33592 345 43 on on ADP bracis-33592 345 44 : : PUNCT bracis-33592 345 45 pubmed pubmed PROPN bracis-33592 345 46     SPACE bracis-33592 345 47 google google PROPN bracis-33592 345 48 scholar scholar NOUN bracis-33592 345 49 andré andré VERB bracis-33592 345 50 barbosaview barbosaview NOUN bracis-33592 345 51 author author NOUN bracis-33592 345 52 publications publication NOUN bracis-33592 345 53 search search NOUN bracis-33592 345 54 author author NOUN bracis-33592 345 55 on on ADP bracis-33592 345 56 : : PUNCT bracis-33592 345 57 pubmed pubmed PROPN bracis-33592 345 58     SPACE bracis-33592 345 59 google google PROPN bracis-33592 345 60 scholar scholar PROPN bracis-33592 345 61 daniel daniel PROPN bracis-33592 345 62 silva silva PROPN bracis-33592 345 63 lopes lopes PROPN bracis-33592 345 64 da da PROPN bracis-33592 345 65 costaview costaview PROPN bracis-33592 345 66 author author NOUN bracis-33592 345 67 publications publication NOUN bracis-33592 345 68 search search NOUN bracis-33592 345 69 author author NOUN bracis-33592 345 70 on on ADP bracis-33592 345 71 : : PUNCT bracis-33592 345 72 pubmed pubmed PROPN bracis-33592 345 73     SPACE bracis-33592 345 74 google google PROPN bracis-33592 345 75 scholar scholar NOUN bracis-33592 345 76 denis denis PROPN bracis-33592 345 77 deratani deratani PROPN bracis-33592 345 78 mauáview mauáview PROPN bracis-33592 345 79 author author NOUN bracis-33592 345 80 publications publication NOUN bracis-33592 345 81 search search NOUN bracis-33592 345 82 author author NOUN bracis-33592 345 83 on on ADP bracis-33592 345 84 : : PUNCT bracis-33592 345 85 pubmed pubmed PROPN bracis-33592 345 86     SPACE bracis-33592 345 87 google google PROPN bracis-33592 345 88 scholar scholar NOUN bracis-33592 345 89 corresponding correspond VERB bracis-33592 345 90 author author NOUN bracis-33592 345 91 correspondence correspondence NOUN bracis-33592 345 92 to to ADP bracis-33592 345 93 igor igor PROPN bracis-33592 345 94 cataneo cataneo PROPN bracis-33592 345 95 silveira silveira PROPN bracis-33592 345 96 . . PUNCT bracis-33592 346 1 editor editor NOUN bracis-33592 346 2 information information NOUN bracis-33592 346 3 editors editor NOUN bracis-33592 346 4 and and CCONJ bracis-33592 346 5 affiliations affiliation NOUN bracis-33592 346 6 universidade universidade VERB bracis-33592 346 7 federal federal PROPN bracis-33592 346 8 fluminense fluminense PROPN bracis-33592 346 9 , , PUNCT bracis-33592 346 10 niterói niterói PROPN bracis-33592 346 11 , , PUNCT bracis-33592 346 12 brazil brazil PROPN bracis-33592 346 13 aline aline PROPN bracis-33592 346 14 paes paes PROPN bracis-33592 346 15 instituto instituto PROPN bracis-33592 346 16 tecnológico tecnológico PROPN bracis-33592 346 17 de de PROPN bracis-33592 346 18 aeronáutica aeronáutica PROPN bracis-33592 346 19 , , PUNCT bracis-33592 346 20 são são PROPN bracis-33592 346 21 josé josé PROPN bracis-33592 346 22 dos dos PROPN bracis-33592 346 23 campos campos PROPN bracis-33592 346 24 , , PUNCT bracis-33592 346 25 brazil brazil PROPN bracis-33592 346 26 filipe filipe PROPN bracis-33592 346 27 a. a. PROPN bracis-33592 346 28 n. n. PROPN bracis-33592 346 29 verri verri PROPN bracis-33592 346 30 rights right NOUN bracis-33592 346 31 and and CCONJ bracis-33592 346 32 permissions permission NOUN bracis-33592 346 33 reprints reprint NOUN bracis-33592 346 34 and and CCONJ bracis-33592 346 35 permissions permission VERB bracis-33592 346 36 copyright copyright NOUN bracis-33592 346 37 information information NOUN bracis-33592 346 38 © © ADP bracis-33592 346 39 2025 2025 NUM bracis-33592 346 40 the the DET bracis-33592 346 41 author(s author(s NOUN bracis-33592 346 42 ) ) PUNCT bracis-33592 346 43 , , PUNCT bracis-33592 346 44 under under ADP bracis-33592 346 45 exclusive exclusive ADJ bracis-33592 346 46 license license NOUN bracis-33592 346 47 to to ADP bracis-33592 346 48 springer springer NOUN bracis-33592 346 49 nature nature PROPN bracis-33592 346 50 switzerland switzerland PROPN bracis-33592 346 51 ag ag PROPN bracis-33592 346 52 about about ADP bracis-33592 346 53 this this DET bracis-33592 346 54 paper paper NOUN bracis-33592 346 55 cite cite VERB bracis-33592 346 56 this this DET bracis-33592 346 57 paper paper NOUN bracis-33592 346 58 silveira silveira PROPN bracis-33592 346 59 , , PUNCT bracis-33592 346 60 i.c i.c PROPN bracis-33592 346 61 . . PROPN bracis-33592 346 62 , , PUNCT bracis-33592 346 63 barbosa barbosa PROPN bracis-33592 346 64 , , PUNCT bracis-33592 346 65 a. a. PROPN bracis-33592 346 66 , , PUNCT bracis-33592 346 67 da da PROPN bracis-33592 346 68 costa costa PROPN bracis-33592 346 69 , , PUNCT bracis-33592 346 70 d.s.l d.s.l NOUN bracis-33592 346 71 . . PROPN bracis-33592 346 72 , , PUNCT bracis-33592 346 73 mauá mauá NOUN bracis-33592 346 74 , , PUNCT bracis-33592 346 75 d.d d.d PROPN bracis-33592 346 76 . . PROPN bracis-33592 346 77 ( ( PUNCT bracis-33592 346 78 2025 2025 NUM bracis-33592 346 79 ) ) PUNCT bracis-33592 346 80 . . PUNCT bracis-33592 347 1 investigating investigate VERB bracis-33592 347 2 universal universal ADJ bracis-33592 347 3 adversarial adversarial ADJ bracis-33592 347 4 attacks attack NOUN bracis-33592 347 5 against against ADP bracis-33592 347 6 transformers transformer NOUN bracis-33592 347 7 - - PUNCT bracis-33592 347 8 based base VERB bracis-33592 347 9 automatic automatic ADJ bracis-33592 347 10 essay essay NOUN bracis-33592 347 11 scoring scoring NOUN bracis-33592 347 12 systems system NOUN bracis-33592 347 13 . . PUNCT bracis-33592 348 1 in in ADP bracis-33592 348 2 : : PUNCT bracis-33592 348 3 paes paes PROPN bracis-33592 348 4 , , PUNCT bracis-33592 348 5 a. a. PROPN bracis-33592 348 6 , , PUNCT bracis-33592 348 7 verri verri NOUN bracis-33592 348 8 , , PUNCT bracis-33592 348 9 f.a.n f.a.n ADJ bracis-33592 348 10 . . PUNCT bracis-33592 348 11 ( ( PUNCT bracis-33592 348 12 eds ed NOUN bracis-33592 348 13 ) ) PUNCT bracis-33592 348 14 intelligent intelligent ADJ bracis-33592 348 15 systems system NOUN bracis-33592 348 16 . . PUNCT bracis-33592 348 17 bracis bracis PROPN bracis-33592 348 18 2024 2024 NUM bracis-33592 348 19 . . PUNCT bracis-33592 349 1 lecture lecture NOUN bracis-33592 349 2 notes note NOUN bracis-33592 349 3 in in ADP bracis-33592 349 4 computer computer NOUN bracis-33592 349 5 science science NOUN bracis-33592 349 6 ( ( PUNCT bracis-33592 349 7 ) ) PUNCT bracis-33592 349 8 , , PUNCT bracis-33592 349 9 vol vol NOUN bracis-33592 349 10 15413 15413 NUM bracis-33592 349 11 . . PUNCT bracis-33592 350 1 springer springer NOUN bracis-33592 350 2 , , PUNCT bracis-33592 350 3 cham cham PROPN bracis-33592 350 4 . . PUNCT bracis-33592 351 1 https://doi.org/10.1007/978-3-031-79032-4_12 https://doi.org/10.1007/978-3-031-79032-4_12 PART bracis-33592 351 2 download download NOUN bracis-33592 351 3 citation citation NOUN bracis-33592 351 4 .ris .ris PUNCT bracis-33592 352 1 .enw .enw PROPN bracis-33592 352 2 .bib .bib PUNCT bracis-33592 353 1 doi doi PROPN bracis-33592 353 2 : : PUNCT bracis-33592 353 3 https://doi.org/10.1007/978-3-031-79032-4_12 https://doi.org/10.1007/978-3-031-79032-4_12 X bracis-33592 353 4 published publish VERB bracis-33592 353 5 : : PUNCT bracis-33592 353 6 30 30 NUM bracis-33592 353 7 january january PROPN bracis-33592 353 8 2025 2025 NUM bracis-33592 353 9 publisher publisher NOUN bracis-33592 353 10 name name NOUN bracis-33592 353 11 : : PUNCT bracis-33592 353 12 springer springer NOUN bracis-33592 353 13 , , PUNCT bracis-33592 353 14 cham cham PROPN bracis-33592 353 15 print print PROPN bracis-33592 353 16 isbn isbn PROPN bracis-33592 353 17 : : PUNCT bracis-33592 353 18 978 978 NUM bracis-33592 353 19 - - SYM bracis-33592 353 20 3 3 NUM bracis-33592 353 21 - - PUNCT bracis-33592 353 22 031 031 NUM bracis-33592 353 23 - - PUNCT bracis-33592 353 24 79031 79031 NUM bracis-33592 353 25 - - SYM bracis-33592 353 26 7 7 NUM bracis-33592 353 27 online online ADJ bracis-33592 353 28 isbn isbn NOUN bracis-33592 353 29 : : PUNCT bracis-33592 353 30 978 978 NUM bracis-33592 353 31 - - SYM bracis-33592 353 32 3 3 NUM bracis-33592 353 33 - - PUNCT bracis-33592 353 34 031 031 NUM bracis-33592 353 35 - - PUNCT bracis-33592 353 36 79032 79032 NUM bracis-33592 353 37 - - SYM bracis-33592 353 38 4 4 NUM bracis-33592 353 39 ebook ebook NOUN bracis-33592 353 40 packages package NOUN bracis-33592 353 41 : : PUNCT bracis-33592 353 42 computer computer NOUN bracis-33592 353 43 sciencecomputer sciencecomputer NOUN bracis-33592 353 44 science science NOUN bracis-33592 353 45 ( ( PUNCT bracis-33592 353 46 r0 r0 NOUN bracis-33592 353 47 ) ) PUNCT bracis-33592 353 48 share share VERB bracis-33592 353 49 this this DET bracis-33592 353 50 paper paper NOUN bracis-33592 353 51 anyone anyone PRON bracis-33592 353 52 you you PRON bracis-33592 353 53 share share VERB bracis-33592 353 54 the the DET bracis-33592 353 55 following follow VERB bracis-33592 353 56 link link NOUN bracis-33592 353 57 with with ADP bracis-33592 353 58 will will AUX bracis-33592 353 59 be be AUX bracis-33592 353 60 able able ADJ bracis-33592 353 61 to to PART bracis-33592 353 62 read read VERB bracis-33592 353 63 this this DET bracis-33592 353 64 content content NOUN bracis-33592 353 65 : : PUNCT bracis-33592 353 66 get get VERB bracis-33592 353 67 shareable shareable ADJ bracis-33592 353 68 linksorry linksorry NOUN bracis-33592 353 69 , , PUNCT bracis-33592 353 70 a a DET bracis-33592 353 71 shareable shareable ADJ bracis-33592 353 72 link link NOUN bracis-33592 353 73 is be AUX bracis-33592 353 74 not not PART bracis-33592 353 75 currently currently ADV bracis-33592 353 76 available available ADJ bracis-33592 353 77 for for ADP bracis-33592 353 78 this this DET bracis-33592 353 79 article article NOUN bracis-33592 353 80 . . PUNCT bracis-33592 354 1 copy copy VERB bracis-33592 354 2 shareable shareable ADJ bracis-33592 354 3 link link NOUN bracis-33592 354 4 to to PART bracis-33592 354 5 clipboard clipboard NOUN bracis-33592 354 6 provided provide VERB bracis-33592 354 7 by by ADP bracis-33592 354 8 the the DET bracis-33592 354 9 springer springer NOUN bracis-33592 354 10 nature nature PROPN bracis-33592 354 11 sharedit sharedit PROPN bracis-33592 354 12 content content NOUN bracis-33592 354 13 - - PUNCT bracis-33592 354 14 sharing share VERB bracis-33592 354 15 initiative initiative NOUN bracis-33592 354 16 keywords keyword VERB bracis-33592 354 17 automatic automatic ADJ bracis-33592 354 18 essay essay NOUN bracis-33592 354 19 scoring score VERB bracis-33592 354 20 adversarial adversarial ADJ bracis-33592 354 21 attack attack NOUN bracis-33592 354 22 natural natural ADJ bracis-33592 354 23 language language NOUN bracis-33592 354 24 processing processing NOUN bracis-33592 354 25 publish publish NOUN bracis-33592 354 26 with with ADP bracis-33592 354 27 us us PROPN bracis-33592 354 28 policies policy NOUN bracis-33592 354 29 and and CCONJ bracis-33592 354 30 ethics ethic NOUN bracis-33592 354 31 search search NOUN bracis-33592 354 32 search search NOUN bracis-33592 354 33 by by ADP bracis-33592 354 34 keyword keyword NOUN bracis-33592 354 35 or or CCONJ bracis-33592 354 36 author author NOUN bracis-33592 354 37 search search NOUN bracis-33592 354 38 navigation navigation NOUN bracis-33592 354 39 find find VERB bracis-33592 354 40 a a DET bracis-33592 354 41 journal journal NOUN bracis-33592 354 42 publish publish VERB bracis-33592 354 43 with with ADP bracis-33592 354 44 us we PRON bracis-33592 354 45 track track VERB bracis-33592 354 46 your your PRON bracis-33592 354 47 research research NOUN bracis-33592 354 48 discover discover VERB bracis-33592 354 49 content content NOUN bracis-33592 354 50 journals journal NOUN bracis-33592 354 51 a a DET bracis-33592 354 52 - - PUNCT bracis-33592 354 53 z z NOUN bracis-33592 354 54 books book NOUN bracis-33592 354 55 a a DET bracis-33592 354 56 - - PUNCT bracis-33592 354 57 z z NOUN bracis-33592 354 58 publish publish NOUN bracis-33592 354 59 with with ADP bracis-33592 354 60 us us PROPN bracis-33592 354 61 journal journal PROPN bracis-33592 354 62 finder finder PROPN bracis-33592 354 63 publish publish VERB bracis-33592 354 64 your your PRON bracis-33592 354 65 research research NOUN bracis-33592 354 66 language language NOUN bracis-33592 354 67 editing edit VERB bracis-33592 354 68 open open ADJ bracis-33592 354 69 access access NOUN bracis-33592 354 70 publishing publishing NOUN bracis-33592 354 71 products product NOUN bracis-33592 354 72 and and CCONJ bracis-33592 354 73 services service NOUN bracis-33592 354 74 our our PRON bracis-33592 354 75 products product NOUN bracis-33592 354 76 librarians librarian VERB bracis-33592 354 77 societies society NOUN bracis-33592 354 78 partners partner NOUN bracis-33592 354 79 and and CCONJ bracis-33592 354 80 advertisers advertiser NOUN bracis-33592 354 81 our our PRON bracis-33592 354 82 brands brand NOUN bracis-33592 354 83 springer springer NOUN bracis-33592 354 84 nature nature PROPN bracis-33592 354 85 portfolio portfolio PROPN bracis-33592 354 86 bmc bmc PROPN bracis-33592 354 87 palgrave palgrave PROPN bracis-33592 354 88 macmillan macmillan PROPN bracis-33592 354 89 apress apress PROPN bracis-33592 354 90 discover discover VERB bracis-33592 354 91 your your PRON bracis-33592 354 92 privacy privacy NOUN bracis-33592 354 93 choices choice NOUN bracis-33592 354 94 / / SYM bracis-33592 354 95 manage manage NOUN bracis-33592 354 96 cookies cookie NOUN bracis-33592 354 97 your your PRON bracis-33592 354 98 us us PROPN bracis-33592 355 1 state state NOUN bracis-33592 355 2 privacy privacy NOUN bracis-33592 355 3 rights right NOUN bracis-33592 355 4 accessibility accessibility NOUN bracis-33592 355 5 statement statement NOUN bracis-33592 355 6 terms term NOUN bracis-33592 355 7 and and CCONJ bracis-33592 355 8 conditions condition NOUN bracis-33592 355 9 privacy privacy NOUN bracis-33592 355 10 policy policy NOUN bracis-33592 355 11 help help NOUN bracis-33592 355 12 and and CCONJ bracis-33592 355 13 support support VERB bracis-33592 355 14 legal legal ADJ bracis-33592 355 15 notice notice NOUN bracis-33592 355 16 cancel cancel VERB bracis-33592 355 17 contracts contract NOUN bracis-33592 355 18 here here ADV bracis-33592 355 19 129.74.145.123 129.74.145.123 NUM bracis-33592 355 20 hesburgh hesburgh PROPN bracis-33592 355 21 library library PROPN bracis-33592 355 22 er er INTJ bracis-33592 355 23 unit unit NOUN bracis-33592 355 24 ( ( PUNCT bracis-33592 355 25 3005732405 3005732405 NUM bracis-33592 355 26 ) ) PUNCT bracis-33592 355 27 northeast northeast ADJ bracis-33592 355 28 research research NOUN bracis-33592 355 29 libraries library NOUN bracis-33592 355 30 ( ( PUNCT bracis-33592 355 31 nerl nerl PROPN bracis-33592 355 32 ) ) PUNCT bracis-33592 355 33 ( ( PUNCT bracis-33592 355 34 8200828607 8200828607 NUM bracis-33592 355 35 ) ) PUNCT bracis-33592 355 36 nerl nerl VERB bracis-33592 355 37 ta ta X bracis-33592 355 38 account account NOUN bracis-33592 355 39 ( ( PUNCT bracis-33592 355 40 3006206169 3006206169 NUM bracis-33592 355 41 ) ) PUNCT bracis-33592 355 42 university university NOUN bracis-33592 355 43 of of ADP bracis-33592 355 44 notre notre PROPN bracis-33592 355 45 dame dame PROPN bracis-33592 355 46 hesburgh hesburgh PROPN bracis-33592 355 47 library library NOUN bracis-33592 355 48 ( ( PUNCT bracis-33592 355 49 3000184373 3000184373 NUM bracis-33592 355 50 ) ) PUNCT bracis-33592 356 1 © © ADP bracis-33592 356 2 2025 2025 NUM bracis-33592 356 3 springer springer NOUN bracis-33592 356 4 nature nature NOUN