id sid tid token lemma pos eblip-19158 1 1 article article NOUN eblip-19158 1 2     SPACE eblip-19158 1 3 developing develop VERB eblip-19158 1 4 and and CCONJ eblip-19158 1 5 applying apply VERB eblip-19158 1 6 an an DET eblip-19158 1 7 information information NOUN eblip-19158 1 8 literacy literacy NOUN eblip-19158 1 9 rubric rubric NOUN eblip-19158 1 10 to to ADP eblip-19158 1 11 student student NOUN eblip-19158 1 12 annotated annotate VERB eblip-19158 1 13 bibliographies bibliography NOUN eblip-19158 1 14     SPACE eblip-19158 1 15 erin erin PROPN eblip-19158 1 16 e. e. PROPN eblip-19158 1 17 rinto rinto PROPN eblip-19158 1 18 undergraduate undergraduate NOUN eblip-19158 1 19 learning learn VERB eblip-19158 1 20 librarian librarian ADJ eblip-19158 1 21 and and CCONJ eblip-19158 1 22 assistant assistant NOUN eblip-19158 1 23 professor professor PROPN eblip-19158 1 24 university university PROPN eblip-19158 1 25 of of ADP eblip-19158 1 26 nevada nevada PROPN eblip-19158 1 27 las las PROPN eblip-19158 1 28 vegas vegas PROPN eblip-19158 1 29 las las PROPN eblip-19158 1 30 vegas vegas PROPN eblip-19158 1 31 , , PUNCT eblip-19158 1 32 nevada nevada PROPN eblip-19158 1 33 , , PUNCT eblip-19158 1 34 united united PROPN eblip-19158 1 35 states states PROPN eblip-19158 1 36 of of ADP eblip-19158 1 37 america america PROPN eblip-19158 1 38 email email NOUN eblip-19158 1 39 : : PUNCT eblip-19158 1 40 erin.rinto@unlv.edu erin.rinto@unlv.edu X eblip-19158 1 41     SPACE eblip-19158 1 42 received receive VERB eblip-19158 1 43 : : PUNCT eblip-19158 1 44 20 20 NUM eblip-19158 1 45 mar mar PROPN eblip-19158 1 46 . . PROPN eblip-19158 1 47 2013 2013 NUM eblip-19158 1 48       SPACE eblip-19158 1 49 accepted accept VERB eblip-19158 1 50 : : PUNCT eblip-19158 1 51 23 23 NUM eblip-19158 1 52 may may AUX eblip-19158 1 53 2013 2013 NUM eblip-19158 1 54             SPACE eblip-19158 1 55 2013 2013 NUM eblip-19158 1 56 rinto rinto NOUN eblip-19158 1 57 . . PUNCT eblip-19158 2 1 this this PRON eblip-19158 2 2 is be AUX eblip-19158 2 3 an an DET eblip-19158 2 4 open open ADJ eblip-19158 2 5 access access NOUN eblip-19158 2 6 article article NOUN eblip-19158 2 7 distributed distribute VERB eblip-19158 2 8 under under ADP eblip-19158 2 9 the the DET eblip-19158 2 10 terms term NOUN eblip-19158 2 11 of of ADP eblip-19158 2 12 the the DET eblip-19158 2 13 creative creative ADJ eblip-19158 2 14 commons‐attribution‐noncommercial‐share commons‐attribution‐noncommercial‐share PROPN eblip-19158 2 15 alike alike ADJ eblip-19158 2 16 license license VERB eblip-19158 2 17 2.5 2.5 NUM eblip-19158 2 18 canada canada PROPN eblip-19158 2 19 ( ( PUNCT eblip-19158 2 20 http://creativecommons.org/licenses/by-nc-sa/2.5/ca/ http://creativecommons.org/licenses/by-nc-sa/2.5/ca/ PROPN eblip-19158 2 21 ) ) PUNCT eblip-19158 2 22 , , PUNCT eblip-19158 2 23 which which PRON eblip-19158 2 24 permits permit VERB eblip-19158 2 25 unrestricted unrestricted ADJ eblip-19158 2 26 use use NOUN eblip-19158 2 27 , , PUNCT eblip-19158 2 28 distribution distribution NOUN eblip-19158 2 29 , , PUNCT eblip-19158 2 30 and and CCONJ eblip-19158 2 31 reproduction reproduction NOUN eblip-19158 2 32 in in ADP eblip-19158 2 33 any any DET eblip-19158 2 34 medium medium NOUN eblip-19158 2 35 , , PUNCT eblip-19158 2 36 provided provide VERB eblip-19158 2 37 the the DET eblip-19158 2 38 original original ADJ eblip-19158 2 39 work work NOUN eblip-19158 2 40 is be AUX eblip-19158 2 41 properly properly ADV eblip-19158 2 42 attributed attribute VERB eblip-19158 2 43 , , PUNCT eblip-19158 2 44 not not PART eblip-19158 2 45 used use VERB eblip-19158 2 46 for for ADP eblip-19158 2 47 commercial commercial ADJ eblip-19158 2 48 purposes purpose NOUN eblip-19158 2 49 , , PUNCT eblip-19158 2 50 and and CCONJ eblip-19158 2 51 , , PUNCT eblip-19158 2 52 if if SCONJ eblip-19158 2 53 transformed transform VERB eblip-19158 2 54 , , PUNCT eblip-19158 2 55 the the DET eblip-19158 2 56 resulting result VERB eblip-19158 2 57 work work NOUN eblip-19158 2 58 is be AUX eblip-19158 2 59 redistributed redistribute VERB eblip-19158 2 60 under under ADP eblip-19158 2 61 the the DET eblip-19158 2 62 same same ADJ eblip-19158 2 63 or or CCONJ eblip-19158 2 64 similar similar ADJ eblip-19158 2 65 license license NOUN eblip-19158 2 66 to to ADP eblip-19158 2 67 this this DET eblip-19158 2 68 one one NUM eblip-19158 2 69 . . PUNCT eblip-19158 2 70     SPACE eblip-19158 3 1 abstract abstract ADJ eblip-19158 3 2     SPACE eblip-19158 3 3 objective objective NOUN eblip-19158 3 4 – – PUNCT eblip-19158 3 5 this this DET eblip-19158 3 6 study study NOUN eblip-19158 3 7 demonstrates demonstrate VERB eblip-19158 3 8 one one NUM eblip-19158 3 9 method method NOUN eblip-19158 3 10 of of ADP eblip-19158 3 11 developing develop VERB eblip-19158 3 12 and and CCONJ eblip-19158 3 13 applying apply VERB eblip-19158 3 14 rubrics rubric NOUN eblip-19158 3 15 to to PART eblip-19158 3 16 student student VERB eblip-19158 3 17 writing write VERB eblip-19158 3 18 in in ADP eblip-19158 3 19 order order NOUN eblip-19158 3 20 to to PART eblip-19158 3 21 gather gather VERB eblip-19158 3 22 evidence evidence NOUN eblip-19158 3 23 of of ADP eblip-19158 3 24 how how SCONJ eblip-19158 3 25 students student NOUN eblip-19158 3 26 utilize utilize VERB eblip-19158 3 27 information information NOUN eblip-19158 3 28 literacy literacy NOUN eblip-19158 3 29 skills skill NOUN eblip-19158 3 30 in in ADP eblip-19158 3 31 the the DET eblip-19158 3 32 context context NOUN eblip-19158 3 33 of of ADP eblip-19158 3 34 an an DET eblip-19158 3 35 authentic authentic ADJ eblip-19158 3 36 assessment assessment NOUN eblip-19158 3 37 activity activity NOUN eblip-19158 3 38 . . PUNCT eblip-19158 4 1 the the DET eblip-19158 4 2 process process NOUN eblip-19158 4 3 of of ADP eblip-19158 4 4 creating create VERB eblip-19158 4 5 a a DET eblip-19158 4 6 rubric rubric NOUN eblip-19158 4 7 , , PUNCT eblip-19158 4 8 training training NOUN eblip-19158 4 9 scorers scorer NOUN eblip-19158 4 10 to to PART eblip-19158 4 11 use use VERB eblip-19158 4 12 the the DET eblip-19158 4 13 rubric rubric NOUN eblip-19158 4 14 , , PUNCT eblip-19158 4 15 collecting collect VERB eblip-19158 4 16 annotated annotate VERB eblip-19158 4 17 bibliographies bibliography NOUN eblip-19158 4 18 , , PUNCT eblip-19158 4 19 applying apply VERB eblip-19158 4 20 the the DET eblip-19158 4 21 rubric rubric NOUN eblip-19158 4 22 to to ADP eblip-19158 4 23 student student NOUN eblip-19158 4 24 work work NOUN eblip-19158 4 25 , , PUNCT eblip-19158 4 26 and and CCONJ eblip-19158 4 27 the the DET eblip-19158 4 28 results result NOUN eblip-19158 4 29 of of ADP eblip-19158 4 30 the the DET eblip-19158 4 31 rubric rubric ADJ eblip-19158 4 32 assessment assessment NOUN eblip-19158 4 33 are be AUX eblip-19158 4 34 described describe VERB eblip-19158 4 35 . . PUNCT eblip-19158 5 1 implications implication NOUN eblip-19158 5 2 for for ADP eblip-19158 5 3 information information NOUN eblip-19158 5 4 literacy literacy NOUN eblip-19158 5 5 instruction instruction NOUN eblip-19158 5 6 are be AUX eblip-19158 5 7 also also ADV eblip-19158 5 8 discussed discuss VERB eblip-19158 5 9 . . PUNCT eblip-19158 5 10     SPACE eblip-19158 6 1 methods method NOUN eblip-19158 6 2 – – PUNCT eblip-19158 6 3 the the DET eblip-19158 6 4 focus focus NOUN eblip-19158 6 5 of of ADP eblip-19158 6 6 this this DET eblip-19158 6 7 study study NOUN eblip-19158 6 8 was be AUX eblip-19158 6 9 the the DET eblip-19158 6 10 english english ADJ eblip-19158 6 11 102 102 NUM eblip-19158 6 12 ( ( PUNCT eblip-19158 6 13 eng eng NOUN eblip-19158 6 14 102 102 NUM eblip-19158 6 15 ) ) PUNCT eblip-19158 6 16 course course NOUN eblip-19158 6 17 , , PUNCT eblip-19158 6 18 a a DET eblip-19158 6 19 required require VERB eblip-19158 6 20 research research NOUN eblip-19158 6 21 - - PUNCT eblip-19158 6 22 based base VERB eblip-19158 6 23 writing writing NOUN eblip-19158 6 24 course course NOUN eblip-19158 6 25 that that PRON eblip-19158 6 26 partners partner VERB eblip-19158 6 27 the the DET eblip-19158 6 28 instructors instructor NOUN eblip-19158 6 29 with with ADP eblip-19158 6 30 the the DET eblip-19158 6 31 university university NOUN eblip-19158 6 32 librarians librarian NOUN eblip-19158 6 33 for for ADP eblip-19158 6 34 information information NOUN eblip-19158 6 35 literacy literacy NOUN eblip-19158 6 36 instruction instruction NOUN eblip-19158 6 37 . . PUNCT eblip-19158 7 1 the the DET eblip-19158 7 2 author author NOUN eblip-19158 7 3 developed develop VERB eblip-19158 7 4 an an DET eblip-19158 7 5 information information NOUN eblip-19158 7 6 literacy literacy NOUN eblip-19158 7 7 rubric rubric NOUN eblip-19158 7 8 to to PART eblip-19158 7 9 assess assess VERB eblip-19158 7 10 student student NOUN eblip-19158 7 11 evaluation evaluation NOUN eblip-19158 7 12 of of ADP eblip-19158 7 13 information information NOUN eblip-19158 7 14 resources resource NOUN eblip-19158 7 15 in in ADP eblip-19158 7 16 the the DET eblip-19158 7 17 eng eng PROPN eblip-19158 7 18 102 102 NUM eblip-19158 7 19 annotated annotate VERB eblip-19158 7 20 bibliography bibliography NOUN eblip-19158 7 21 assignment assignment NOUN eblip-19158 7 22 and and CCONJ eblip-19158 7 23 trained train VERB eblip-19158 7 24 three three NUM eblip-19158 7 25 other other ADJ eblip-19158 7 26 librarians librarian NOUN eblip-19158 7 27 how how SCONJ eblip-19158 7 28 to to PART eblip-19158 7 29 apply apply VERB eblip-19158 7 30 the the DET eblip-19158 7 31 rubric rubric NOUN eblip-19158 7 32 to to ADP eblip-19158 7 33 student student NOUN eblip-19158 7 34 work work NOUN eblip-19158 7 35 . . PUNCT eblip-19158 8 1 the the DET eblip-19158 8 2 rubric rubric NOUN eblip-19158 8 3 assessed assess VERB eblip-19158 8 4 the the DET eblip-19158 8 5 extent extent NOUN eblip-19158 8 6 to to PART eblip-19158 8 7 which which PRON eblip-19158 8 8 students student NOUN eblip-19158 8 9 critically critically ADV eblip-19158 8 10 applied apply VERB eblip-19158 8 11 the the DET eblip-19158 8 12 evaluative evaluative ADJ eblip-19158 8 13 criteria criterion NOUN eblip-19158 8 14 currency currency NOUN eblip-19158 8 15 , , PUNCT eblip-19158 8 16 relevance relevance NOUN eblip-19158 8 17 , , PUNCT eblip-19158 8 18 accuracy accuracy NOUN eblip-19158 8 19 , , PUNCT eblip-19158 8 20 authority authority NOUN eblip-19158 8 21 , , PUNCT eblip-19158 8 22 and and CCONJ eblip-19158 8 23 purpose purpose NOUN eblip-19158 8 24 to to ADP eblip-19158 8 25 the the DET eblip-19158 8 26 information information NOUN eblip-19158 8 27 sources source NOUN eblip-19158 8 28 in in ADP eblip-19158 8 29 their their PRON eblip-19158 8 30 annotations annotation NOUN eblip-19158 8 31 . . PUNCT eblip-19158 9 1 at at ADP eblip-19158 9 2 the the DET eblip-19158 9 3 end end NOUN eblip-19158 9 4 of of ADP eblip-19158 9 5 the the DET eblip-19158 9 6 semester semester NOUN eblip-19158 9 7 , , PUNCT eblip-19158 9 8 the the DET eblip-19158 9 9 author author NOUN eblip-19158 9 10 collected collect VERB eblip-19158 9 11 up up ADP eblip-19158 9 12 to to PART eblip-19158 9 13 three three NUM eblip-19158 9 14 de de ADJ eblip-19158 9 15 - - ADJ eblip-19158 9 16 identified identify VERB eblip-19158 9 17 annotated annotate VERB eblip-19158 9 18 bibliographies bibliography NOUN eblip-19158 9 19 from from ADP eblip-19158 9 20 each each PRON eblip-19158 9 21 of of ADP eblip-19158 9 22 the the DET eblip-19158 9 23 58 58 NUM eblip-19158 9 24 sections section NOUN eblip-19158 9 25 of of ADP eblip-19158 9 26 eng eng PROPN eblip-19158 9 27 102 102 NUM eblip-19158 9 28 . . PUNCT eblip-19158 10 1 the the DET eblip-19158 10 2 rubric rubric NOUN eblip-19158 10 3 was be AUX eblip-19158 10 4 applied apply VERB eblip-19158 10 5 to to ADP eblip-19158 10 6 up up ADP eblip-19158 10 7 to to PART eblip-19158 10 8 five five NUM eblip-19158 10 9 annotations annotation NOUN eblip-19158 10 10 in in ADP eblip-19158 10 11 each each DET eblip-19158 10 12 bibliography bibliography NOUN eblip-19158 10 13 , , PUNCT eblip-19158 10 14 resulting result VERB eblip-19158 10 15 in in ADP eblip-19158 10 16 a a DET eblip-19158 10 17 total total ADJ eblip-19158 10 18 examination examination NOUN eblip-19158 10 19 of of ADP eblip-19158 10 20 773 773 NUM eblip-19158 10 21 annotations annotation NOUN eblip-19158 10 22 ( ( PUNCT eblip-19158 10 23 some some DET eblip-19158 10 24 sections section NOUN eblip-19158 10 25 turned turn VERB eblip-19158 10 26 in in ADP eblip-19158 10 27 fewer few ADJ eblip-19158 10 28 than than ADP eblip-19158 10 29 3 3 NUM eblip-19158 10 30 samples sample NOUN eblip-19158 10 31 , , PUNCT eblip-19158 10 32 and and CCONJ eblip-19158 10 33 some some DET eblip-19158 10 34 bibliographies bibliography NOUN eblip-19158 10 35 had have VERB eblip-19158 10 36 fewer few ADJ eblip-19158 10 37 than than ADP eblip-19158 10 38 5 5 NUM eblip-19158 10 39 annotations annotation NOUN eblip-19158 10 40 ) ) PUNCT eblip-19158 10 41 . . PUNCT eblip-19158 11 1 results result NOUN eblip-19158 11 2 – – PUNCT eblip-19158 11 3 the the DET eblip-19158 11 4 results result NOUN eblip-19158 11 5 of of ADP eblip-19158 11 6 the the DET eblip-19158 11 7 study study NOUN eblip-19158 11 8 showed show VERB eblip-19158 11 9 that that SCONJ eblip-19158 11 10 students student NOUN eblip-19158 11 11 struggle struggle VERB eblip-19158 11 12 with with ADP eblip-19158 11 13 critically critically ADV eblip-19158 11 14 evaluating evaluate VERB eblip-19158 11 15 information information NOUN eblip-19158 11 16 resources resource NOUN eblip-19158 11 17 , , PUNCT eblip-19158 11 18 a a DET eblip-19158 11 19 finding finding NOUN eblip-19158 11 20 that that PRON eblip-19158 11 21 supports support VERB eblip-19158 11 22 the the DET eblip-19158 11 23 existing exist VERB eblip-19158 11 24 information information NOUN eblip-19158 11 25 literacy literacy NOUN eblip-19158 11 26 assessment assessment NOUN eblip-19158 11 27 literature literature NOUN eblip-19158 11 28 . . PUNCT eblip-19158 12 1 the the DET eblip-19158 12 2 overwhelming overwhelming ADJ eblip-19158 12 3 majority majority NOUN eblip-19158 12 4 of of ADP eblip-19158 12 5 annotations annotation NOUN eblip-19158 12 6 consisted consist VERB eblip-19158 12 7 of of ADP eblip-19158 12 8 summative summative ADJ eblip-19158 12 9 information information NOUN eblip-19158 12 10 with with ADP eblip-19158 12 11 little little ADJ eblip-19158 12 12 evidence evidence NOUN eblip-19158 12 13 that that SCONJ eblip-19158 12 14 students student NOUN eblip-19158 12 15 used use VERB eblip-19158 12 16 any any DET eblip-19158 12 17 evaluative evaluative ADJ eblip-19158 12 18 criteria criterion NOUN eblip-19158 12 19 when when SCONJ eblip-19158 12 20 they they PRON eblip-19158 12 21 selected select VERB eblip-19158 12 22 an an DET eblip-19158 12 23 information information NOUN eblip-19158 12 24 source source NOUN eblip-19158 12 25 . . PUNCT eblip-19158 13 1 of of ADP eblip-19158 13 2 the the DET eblip-19158 13 3 five five NUM eblip-19158 13 4 criteria criterion NOUN eblip-19158 13 5 examined examine VERB eblip-19158 13 6 , , PUNCT eblip-19158 13 7 relevance relevance NOUN eblip-19158 13 8 to to ADP eblip-19158 13 9 the the DET eblip-19158 13 10 student student NOUN eblip-19158 13 11 ’s ’s PART eblip-19158 13 12 research research NOUN eblip-19158 13 13 topic topic NOUN eblip-19158 13 14 and and CCONJ eblip-19158 13 15 authority authority NOUN eblip-19158 13 16 were be AUX eblip-19158 13 17 the the DET eblip-19158 13 18 most most ADV eblip-19158 13 19 commonly commonly ADV eblip-19158 13 20 used use VERB eblip-19158 13 21 methods method NOUN eblip-19158 13 22 of of ADP eblip-19158 13 23 resource resource NOUN eblip-19158 13 24 evaluation evaluation NOUN eblip-19158 13 25 , , PUNCT eblip-19158 13 26 while while SCONJ eblip-19158 13 27 currency currency NOUN eblip-19158 13 28 , , PUNCT eblip-19158 13 29 accuracy accuracy NOUN eblip-19158 13 30 , , PUNCT eblip-19158 13 31 and and CCONJ eblip-19158 13 32 purpose purpose NOUN eblip-19158 13 33 were be AUX eblip-19158 13 34 criteria criterion NOUN eblip-19158 13 35 least least ADV eblip-19158 13 36 - - PUNCT eblip-19158 13 37 often often ADV eblip-19158 13 38 used use VERB eblip-19158 13 39 . . PUNCT eblip-19158 14 1 the the DET eblip-19158 14 2 low low ADJ eblip-19158 14 3 average average ADJ eblip-19158 14 4 scores score NOUN eblip-19158 14 5 on on ADP eblip-19158 14 6 the the DET eblip-19158 14 7 rubric rubric ADJ eblip-19158 14 8 assessment assessment NOUN eblip-19158 14 9 indicate indicate VERB eblip-19158 14 10 that that SCONJ eblip-19158 14 11 students student NOUN eblip-19158 14 12 are be AUX eblip-19158 14 13 not not PART eblip-19158 14 14 adequately adequately ADV eblip-19158 14 15 learning learn VERB eblip-19158 14 16 how how SCONJ eblip-19158 14 17 to to PART eblip-19158 14 18 apply apply VERB eblip-19158 14 19 this this DET eblip-19158 14 20 set set NOUN eblip-19158 14 21 of of ADP eblip-19158 14 22 information information NOUN eblip-19158 14 23 literacy literacy NOUN eblip-19158 14 24 skills skill NOUN eblip-19158 14 25 . . PUNCT eblip-19158 14 26     SPACE eblip-19158 15 1 conclusions conclusion NOUN eblip-19158 15 2 – – PUNCT eblip-19158 15 3 the the DET eblip-19158 15 4 library library NOUN eblip-19158 15 5 instruction instruction NOUN eblip-19158 15 6 sessions session NOUN eblip-19158 15 7 for for ADP eblip-19158 15 8 eng eng PROPN eblip-19158 15 9 102 102 NUM eblip-19158 15 10 need need VERB eblip-19158 15 11 to to PART eblip-19158 15 12 move move VERB eblip-19158 15 13 beyond beyond ADP eblip-19158 15 14 the the DET eblip-19158 15 15 skills skill NOUN eblip-19158 15 16 of of ADP eblip-19158 15 17 choosing choose VERB eblip-19158 15 18 and and CCONJ eblip-19158 15 19 narrowing narrow VERB eblip-19158 15 20 a a DET eblip-19158 15 21 topic topic NOUN eblip-19158 15 22 , , PUNCT eblip-19158 15 23 selecting select VERB eblip-19158 15 24 keywords keyword NOUN eblip-19158 15 25 , , PUNCT eblip-19158 15 26 and and CCONJ eblip-19158 15 27 searching search VERB eblip-19158 15 28 in in ADP eblip-19158 15 29 a a DET eblip-19158 15 30 library library NOUN eblip-19158 15 31 database database NOUN eblip-19158 15 32 . . PUNCT eblip-19158 16 1 students student NOUN eblip-19158 16 2 also also ADV eblip-19158 16 3 need need VERB eblip-19158 16 4 more more ADV eblip-19158 16 5 targeted target VERB eblip-19158 16 6 instruction instruction NOUN eblip-19158 16 7 on on ADP eblip-19158 16 8 higher high ADJ eblip-19158 16 9 - - PUNCT eblip-19158 16 10 order order NOUN eblip-19158 16 11 skills skill NOUN eblip-19158 16 12 , , PUNCT eblip-19158 16 13 particularly particularly ADV eblip-19158 16 14 how how SCONJ eblip-19158 16 15 to to PART eblip-19158 16 16 critically critically ADV eblip-19158 16 17 evaluate evaluate VERB eblip-19158 16 18 and and CCONJ eblip-19158 16 19 question question VERB eblip-19158 16 20 the the DET eblip-19158 16 21 sources source NOUN eblip-19158 16 22 they they PRON eblip-19158 16 23 find find VERB eblip-19158 16 24 . . PUNCT eblip-19158 17 1 the the DET eblip-19158 17 2 results result NOUN eblip-19158 17 3 of of ADP eblip-19158 17 4 this this DET eblip-19158 17 5 assessment assessment NOUN eblip-19158 17 6 are be AUX eblip-19158 17 7 being be AUX eblip-19158 17 8 used use VERB eblip-19158 17 9 to to PART eblip-19158 17 10 refocus refocus VERB eblip-19158 17 11 the the DET eblip-19158 17 12 learning learn VERB eblip-19158 17 13 outcomes outcome NOUN eblip-19158 17 14 of of ADP eblip-19158 17 15 eng eng PROPN eblip-19158 17 16 102 102 NUM eblip-19158 17 17 library library NOUN eblip-19158 17 18 sessions session NOUN eblip-19158 17 19 so so SCONJ eblip-19158 17 20 that that DET eblip-19158 17 21 instruction instruction NOUN eblip-19158 17 22 can can AUX eblip-19158 17 23 better well ADV eblip-19158 17 24 meet meet VERB eblip-19158 17 25 student student NOUN eblip-19158 17 26 needs need NOUN eblip-19158 17 27 . . PUNCT eblip-19158 18 1 the the DET eblip-19158 18 2 results result NOUN eblip-19158 18 3 are be AUX eblip-19158 18 4 also also ADV eblip-19158 18 5 being be AUX eblip-19158 18 6 used use VERB eblip-19158 18 7 to to PART eblip-19158 18 8 make make VERB eblip-19158 18 9 the the DET eblip-19158 18 10 case case NOUN eblip-19158 18 11 for for ADP eblip-19158 18 12 further further ADJ eblip-19158 18 13 collaboration collaboration NOUN eblip-19158 18 14 between between ADP eblip-19158 18 15 eng eng PROPN eblip-19158 18 16 102 102 NUM eblip-19158 18 17 and and CCONJ eblip-19158 18 18 the the DET eblip-19158 18 19 university university NOUN eblip-19158 18 20 library library NOUN eblip-19158 18 21 . . PUNCT eblip-19158 18 22         SPACE eblip-19158 19 1 introduction introduction NOUN eblip-19158 19 2     SPACE eblip-19158 19 3 it it PRON eblip-19158 19 4 has have AUX eblip-19158 19 5 been be AUX eblip-19158 19 6 well well ADV eblip-19158 19 7 - - PUNCT eblip-19158 19 8 documented document VERB eblip-19158 19 9 in in ADP eblip-19158 19 10 the the DET eblip-19158 19 11 library library NOUN eblip-19158 19 12 literature literature NOUN eblip-19158 19 13 that that SCONJ eblip-19158 19 14 academic academic ADJ eblip-19158 19 15 libraries library NOUN eblip-19158 19 16 are be AUX eblip-19158 19 17 responsible responsible ADJ eblip-19158 19 18 for for ADP eblip-19158 19 19 assessing assess VERB eblip-19158 19 20 their their PRON eblip-19158 19 21 services service NOUN eblip-19158 19 22 , , PUNCT eblip-19158 19 23 especially especially ADV eblip-19158 19 24 library library ADJ eblip-19158 19 25 instruction instruction NOUN eblip-19158 19 26 , , PUNCT eblip-19158 19 27 in in ADP eblip-19158 19 28 order order NOUN eblip-19158 19 29 to to PART eblip-19158 19 30 communicate communicate VERB eblip-19158 19 31 impact impact NOUN eblip-19158 19 32 and and CCONJ eblip-19158 19 33 better well ADJ eblip-19158 19 34 meet meet VERB eblip-19158 19 35 student student NOUN eblip-19158 19 36 needs need NOUN eblip-19158 19 37 ( ( PUNCT eblip-19158 19 38 rockman rockman NOUN eblip-19158 19 39 , , PUNCT eblip-19158 19 40 2002 2002 NUM eblip-19158 19 41 ; ; PUNCT eblip-19158 19 42 avery avery NOUN eblip-19158 19 43 , , PUNCT eblip-19158 19 44 2003 2003 NUM eblip-19158 19 45 ; ; PUNCT eblip-19158 19 46 choinski choinski ADJ eblip-19158 19 47 , , PUNCT eblip-19158 19 48 mark mark NOUN eblip-19158 19 49 , , PUNCT eblip-19158 19 50 & & CCONJ eblip-19158 19 51 murphy murphy PROPN eblip-19158 19 52 , , PUNCT eblip-19158 19 53 2003 2003 NUM eblip-19158 19 54 ; ; PUNCT eblip-19158 19 55 oakleaf oakleaf NOUN eblip-19158 19 56 & & CCONJ eblip-19158 19 57 kaske kaske PROPN eblip-19158 19 58 , , PUNCT eblip-19158 19 59 2009 2009 NUM eblip-19158 19 60 ) ) PUNCT eblip-19158 19 61 . . PUNCT eblip-19158 20 1 in in ADP eblip-19158 20 2 order order NOUN eblip-19158 20 3 to to PART eblip-19158 20 4 intentionally intentionally ADV eblip-19158 20 5 design design VERB eblip-19158 20 6 , , PUNCT eblip-19158 20 7 implement implement VERB eblip-19158 20 8 , , PUNCT eblip-19158 20 9 and and CCONJ eblip-19158 20 10 assess assess VERB eblip-19158 20 11 information information NOUN eblip-19158 20 12 literacy literacy NOUN eblip-19158 20 13 instruction instruction NOUN eblip-19158 20 14 , , PUNCT eblip-19158 20 15 it it PRON eblip-19158 20 16 is be AUX eblip-19158 20 17 helpful helpful ADJ eblip-19158 20 18 to to PART eblip-19158 20 19 have have VERB eblip-19158 20 20 information information NOUN eblip-19158 20 21 about about ADP eblip-19158 20 22 how how SCONJ eblip-19158 20 23 students student NOUN eblip-19158 20 24 apply apply VERB eblip-19158 20 25 information information NOUN eblip-19158 20 26 literacy literacy NOUN eblip-19158 20 27 skills skill NOUN eblip-19158 20 28 in in ADP eblip-19158 20 29 practice practice NOUN eblip-19158 20 30 . . PUNCT eblip-19158 21 1 in in ADP eblip-19158 21 2 particular particular ADJ eblip-19158 21 3 , , PUNCT eblip-19158 21 4 how how SCONJ eblip-19158 21 5 do do AUX eblip-19158 21 6 students student NOUN eblip-19158 21 7 understand understand VERB eblip-19158 21 8 and and CCONJ eblip-19158 21 9 articulate articulate VERB eblip-19158 21 10 the the DET eblip-19158 21 11 concept concept NOUN eblip-19158 21 12 of of ADP eblip-19158 21 13 evaluating evaluate VERB eblip-19158 21 14 information information NOUN eblip-19158 21 15 resources resource NOUN eblip-19158 21 16 ? ? PUNCT eblip-19158 22 1 does do VERB eblip-19158 22 2 library library NOUN eblip-19158 22 3 instruction instruction NOUN eblip-19158 22 4 influence influence NOUN eblip-19158 22 5 the the DET eblip-19158 22 6 decisions decision NOUN eblip-19158 22 7 students student NOUN eblip-19158 22 8 make make VERB eblip-19158 22 9 during during ADP eblip-19158 22 10 their their PRON eblip-19158 22 11 research research NOUN eblip-19158 22 12 process process NOUN eblip-19158 22 13 ? ? PUNCT eblip-19158 23 1 how how SCONJ eblip-19158 23 2 can can AUX eblip-19158 23 3 the the DET eblip-19158 23 4 assessment assessment NOUN eblip-19158 23 5 of of ADP eblip-19158 23 6 student student NOUN eblip-19158 23 7 work work NOUN eblip-19158 23 8 help help NOUN eblip-19158 23 9 practicing practice VERB eblip-19158 23 10 librarians librarian NOUN eblip-19158 23 11 make make VERB eblip-19158 23 12 the the DET eblip-19158 23 13 most most ADJ eblip-19158 23 14 of of ADP eblip-19158 23 15 the the DET eblip-19158 23 16 ubiquitous ubiquitous ADJ eblip-19158 23 17 single single ADJ eblip-19158 23 18 course course NOUN eblip-19158 23 19 period period NOUN eblip-19158 23 20 instruction instruction NOUN eblip-19158 23 21 session session NOUN eblip-19158 23 22 ? ? PUNCT eblip-19158 23 23                                     SPACE eblip-19158 24 1 this this DET eblip-19158 24 2 research research NOUN eblip-19158 24 3 project project NOUN eblip-19158 24 4 is be AUX eblip-19158 24 5 informed inform VERB eblip-19158 24 6 by by ADP eblip-19158 24 7 the the DET eblip-19158 24 8 assessment assessment NOUN eblip-19158 24 9 component component NOUN eblip-19158 24 10 of of ADP eblip-19158 24 11 a a DET eblip-19158 24 12 new new ADJ eblip-19158 24 13 collaboration collaboration NOUN eblip-19158 24 14 between between ADP eblip-19158 24 15 the the DET eblip-19158 24 16 lied lie VERB eblip-19158 24 17 library library NOUN eblip-19158 24 18 and and CCONJ eblip-19158 24 19 the the DET eblip-19158 24 20 english english ADJ eblip-19158 24 21 composition composition NOUN eblip-19158 24 22 program program NOUN eblip-19158 24 23 at at ADP eblip-19158 24 24 the the DET eblip-19158 24 25 university university PROPN eblip-19158 24 26 of of ADP eblip-19158 24 27 nevada nevada PROPN eblip-19158 24 28 las las PROPN eblip-19158 24 29 vegas vegas PROPN eblip-19158 24 30 ( ( PUNCT eblip-19158 24 31 unlv unlv PROPN eblip-19158 24 32 ) ) PUNCT eblip-19158 24 33 . . PUNCT eblip-19158 25 1 the the DET eblip-19158 25 2 focus focus NOUN eblip-19158 25 3 of of ADP eblip-19158 25 4 the the DET eblip-19158 25 5 partnership partnership NOUN eblip-19158 25 6 was be AUX eblip-19158 25 7 the the DET eblip-19158 25 8 english english ADJ eblip-19158 25 9 102 102 NUM eblip-19158 25 10 ( ( PUNCT eblip-19158 25 11 eng eng NOUN eblip-19158 25 12 102 102 NUM eblip-19158 25 13 ) ) PUNCT eblip-19158 25 14 course course NOUN eblip-19158 25 15 , , PUNCT eblip-19158 25 16 a a DET eblip-19158 25 17 required require VERB eblip-19158 25 18 research research NOUN eblip-19158 25 19 - - PUNCT eblip-19158 25 20 based base VERB eblip-19158 25 21 writing writing NOUN eblip-19158 25 22 class class NOUN eblip-19158 25 23 . . PUNCT eblip-19158 26 1 in in ADP eblip-19158 26 2 prior prior ADJ eblip-19158 26 3 years year NOUN eblip-19158 26 4 , , PUNCT eblip-19158 26 5 the the DET eblip-19158 26 6 relationship relationship NOUN eblip-19158 26 7 between between ADP eblip-19158 26 8 the the DET eblip-19158 26 9 library library NOUN eblip-19158 26 10 and and CCONJ eblip-19158 26 11 the the DET eblip-19158 26 12 eng eng PROPN eblip-19158 26 13 102 102 NUM eblip-19158 26 14 course course NOUN eblip-19158 26 15 was be AUX eblip-19158 26 16 informal informal ADJ eblip-19158 26 17 in in ADP eblip-19158 26 18 nature nature NOUN eblip-19158 26 19 ; ; PUNCT eblip-19158 26 20 there there PRON eblip-19158 26 21 was be VERB eblip-19158 26 22 no no DET eblip-19158 26 23 established establish VERB eblip-19158 26 24 set set NOUN eblip-19158 26 25 of of ADP eblip-19158 26 26 learning learn VERB eblip-19158 26 27 goals goal NOUN eblip-19158 26 28 for for ADP eblip-19158 26 29 each each DET eblip-19158 26 30 library library ADJ eblip-19158 26 31 session session NOUN eblip-19158 26 32 that that PRON eblip-19158 26 33 applied apply VERB eblip-19158 26 34 directly directly ADV eblip-19158 26 35 to to ADP eblip-19158 26 36 the the DET eblip-19158 26 37 learning learn VERB eblip-19158 26 38 outcomes outcome NOUN eblip-19158 26 39 of of ADP eblip-19158 26 40 eng eng PROPN eblip-19158 26 41 102 102 NUM eblip-19158 26 42 nor nor CCONJ eblip-19158 26 43 a a DET eblip-19158 26 44 shared share VERB eblip-19158 26 45 understanding understanding NOUN eblip-19158 26 46 of of ADP eblip-19158 26 47 how how SCONJ eblip-19158 26 48 the the DET eblip-19158 26 49 library library NOUN eblip-19158 26 50 session session NOUN eblip-19158 26 51 related relate VERB eblip-19158 26 52 to to ADP eblip-19158 26 53 the the DET eblip-19158 26 54 larger large ADJ eblip-19158 26 55 goals goal NOUN eblip-19158 26 56 of of ADP eblip-19158 26 57 the the DET eblip-19158 26 58 eng eng PROPN eblip-19158 26 59 102 102 NUM eblip-19158 26 60 curriculum curriculum NOUN eblip-19158 26 61 . . PUNCT eblip-19158 27 1 no no DET eblip-19158 27 2 regular regular ADJ eblip-19158 27 3 assessment assessment NOUN eblip-19158 27 4 program program NOUN eblip-19158 27 5 showed show VERB eblip-19158 27 6 how how SCONJ eblip-19158 27 7 the the DET eblip-19158 27 8 library library NOUN eblip-19158 27 9 instruction instruction NOUN eblip-19158 27 10 sessions session NOUN eblip-19158 27 11 contributed contribute VERB eblip-19158 27 12 to to ADP eblip-19158 27 13 the the DET eblip-19158 27 14 information information NOUN eblip-19158 27 15 literacy literacy NOUN eblip-19158 27 16 needs need NOUN eblip-19158 27 17 of of ADP eblip-19158 27 18 eng eng PROPN eblip-19158 27 19 102 102 NUM eblip-19158 27 20 students student NOUN eblip-19158 27 21 . . PUNCT eblip-19158 28 1 one one NUM eblip-19158 28 2 goal goal NOUN eblip-19158 28 3 of of ADP eblip-19158 28 4 this this DET eblip-19158 28 5 new new ADJ eblip-19158 28 6 partnership partnership NOUN eblip-19158 28 7 was be AUX eblip-19158 28 8 to to PART eblip-19158 28 9 introduce introduce VERB eblip-19158 28 10 and and CCONJ eblip-19158 28 11 execute execute VERB eblip-19158 28 12 an an DET eblip-19158 28 13 assessment assessment NOUN eblip-19158 28 14 plan plan NOUN eblip-19158 28 15 for for ADP eblip-19158 28 16 the the DET eblip-19158 28 17 eng eng PROPN eblip-19158 28 18 102 102 NUM eblip-19158 28 19 information information NOUN eblip-19158 28 20 literacy literacy NOUN eblip-19158 28 21 instruction instruction NOUN eblip-19158 28 22 program program NOUN eblip-19158 28 23 . . PUNCT eblip-19158 29 1 the the DET eblip-19158 29 2 assessment assessment NOUN eblip-19158 29 3 plan plan NOUN eblip-19158 29 4 culminated culminate VERB eblip-19158 29 5 with with ADP eblip-19158 29 6 the the DET eblip-19158 29 7 collection collection NOUN eblip-19158 29 8 and and CCONJ eblip-19158 29 9 analysis analysis NOUN eblip-19158 29 10 of of ADP eblip-19158 29 11 annotated annotate VERB eblip-19158 29 12 bibliographies bibliography NOUN eblip-19158 29 13 using use VERB eblip-19158 29 14 a a DET eblip-19158 29 15 rubric rubric NOUN eblip-19158 29 16 designed design VERB eblip-19158 29 17 by by ADP eblip-19158 29 18 the the DET eblip-19158 29 19 author author NOUN eblip-19158 29 20 to to PART eblip-19158 29 21 assess assess VERB eblip-19158 29 22 students student NOUN eblip-19158 29 23 ’ ’ PART eblip-19158 29 24 skills skill NOUN eblip-19158 29 25 with with ADP eblip-19158 29 26 evaluating evaluate VERB eblip-19158 29 27 information information NOUN eblip-19158 29 28 resources resource NOUN eblip-19158 29 29 . . PUNCT eblip-19158 29 30     SPACE eblip-19158 30 1 the the DET eblip-19158 30 2 purpose purpose NOUN eblip-19158 30 3 of of ADP eblip-19158 30 4 this this DET eblip-19158 30 5 case case NOUN eblip-19158 30 6 study study NOUN eblip-19158 30 7 is be AUX eblip-19158 30 8 to to PART eblip-19158 30 9 demonstrate demonstrate VERB eblip-19158 30 10 one one NUM eblip-19158 30 11 way way NOUN eblip-19158 30 12 that that PRON eblip-19158 30 13 rubrics rubric NOUN eblip-19158 30 14 can can AUX eblip-19158 30 15 be be AUX eblip-19158 30 16 developed develop VERB eblip-19158 30 17 and and CCONJ eblip-19158 30 18 applied apply VERB eblip-19158 30 19 to to ADP eblip-19158 30 20 student student NOUN eblip-19158 30 21 writing writing NOUN eblip-19158 30 22 to to PART eblip-19158 30 23 show show VERB eblip-19158 30 24 how how SCONJ eblip-19158 30 25 students student NOUN eblip-19158 30 26 apply apply VERB eblip-19158 30 27 information information NOUN eblip-19158 30 28 literacy literacy NOUN eblip-19158 30 29 skills skill NOUN eblip-19158 30 30 in in ADP eblip-19158 30 31 the the DET eblip-19158 30 32 context context NOUN eblip-19158 30 33 of of ADP eblip-19158 30 34 an an DET eblip-19158 30 35 authentic authentic ADJ eblip-19158 30 36 assessment assessment NOUN eblip-19158 30 37 activity activity NOUN eblip-19158 30 38 . . PUNCT eblip-19158 31 1 this this DET eblip-19158 31 2 study study NOUN eblip-19158 31 3 contributes contribute VERB eblip-19158 31 4 to to ADP eblip-19158 31 5 the the DET eblip-19158 31 6 information information NOUN eblip-19158 31 7 literacy literacy NOUN eblip-19158 31 8 assessment assessment NOUN eblip-19158 31 9 literature literature NOUN eblip-19158 31 10 by by ADP eblip-19158 31 11 using use VERB eblip-19158 31 12 a a DET eblip-19158 31 13 rubric rubric NOUN eblip-19158 31 14 to to PART eblip-19158 31 15 assess assess VERB eblip-19158 31 16 the the DET eblip-19158 31 17 information information NOUN eblip-19158 31 18 literacy literacy NOUN eblip-19158 31 19 skills skill NOUN eblip-19158 31 20 evidenced evidence VERB eblip-19158 31 21 by by ADP eblip-19158 31 22 a a DET eblip-19158 31 23 sample sample NOUN eblip-19158 31 24 of of ADP eblip-19158 31 25 student student NOUN eblip-19158 31 26 work work NOUN eblip-19158 31 27 from from ADP eblip-19158 31 28 a a DET eblip-19158 31 29 large large ADJ eblip-19158 31 30 , , PUNCT eblip-19158 31 31 high high ADJ eblip-19158 31 32 - - PUNCT eblip-19158 31 33 impact impact NOUN eblip-19158 31 34 undergraduate undergraduate NOUN eblip-19158 31 35 composition composition NOUN eblip-19158 31 36 course course NOUN eblip-19158 31 37 . . PUNCT eblip-19158 32 1 the the DET eblip-19158 32 2 results result NOUN eblip-19158 32 3 of of ADP eblip-19158 32 4 this this DET eblip-19158 32 5 research research NOUN eblip-19158 32 6 project project NOUN eblip-19158 32 7 will will AUX eblip-19158 32 8 allow allow VERB eblip-19158 32 9 librarians librarian NOUN eblip-19158 32 10 to to ADP eblip-19158 32 11 fine fine ADJ eblip-19158 32 12 - - PUNCT eblip-19158 32 13 tune tune NOUN eblip-19158 32 14 the the DET eblip-19158 32 15 single single ADJ eblip-19158 32 16 course course NOUN eblip-19158 32 17 period period NOUN eblip-19158 32 18 library library NOUN eblip-19158 32 19 instruction instruction NOUN eblip-19158 32 20 sessions session NOUN eblip-19158 32 21 that that PRON eblip-19158 32 22 accompany accompany VERB eblip-19158 32 23 the the DET eblip-19158 32 24 research research NOUN eblip-19158 32 25 component component NOUN eblip-19158 32 26 of of ADP eblip-19158 32 27 the the DET eblip-19158 32 28 eng eng PROPN eblip-19158 32 29 102 102 NUM eblip-19158 32 30 course course NOUN eblip-19158 32 31 . . PUNCT eblip-19158 32 32           SPACE eblip-19158 33 1 literature literature PROPN eblip-19158 33 2 review review VERB eblip-19158 33 3     SPACE eblip-19158 33 4 the the DET eblip-19158 33 5 assessment assessment NOUN eblip-19158 33 6 literature literature NOUN eblip-19158 33 7 indicates indicate VERB eblip-19158 33 8 that that SCONJ eblip-19158 33 9 an an DET eblip-19158 33 10 important important ADJ eblip-19158 33 11 and and CCONJ eblip-19158 33 12 ongoing ongoing ADJ eblip-19158 33 13 trend trend NOUN eblip-19158 33 14 is be AUX eblip-19158 33 15 authentic authentic ADJ eblip-19158 33 16 assessment assessment NOUN eblip-19158 33 17 ; ; PUNCT eblip-19158 33 18 that that PRON eblip-19158 33 19 is is ADV eblip-19158 33 20 , , PUNCT eblip-19158 33 21 using use VERB eblip-19158 33 22 meaningful meaningful ADJ eblip-19158 33 23 tasks task NOUN eblip-19158 33 24 to to PART eblip-19158 33 25 measure measure VERB eblip-19158 33 26 student student NOUN eblip-19158 33 27 learning learning NOUN eblip-19158 33 28 ( ( PUNCT eblip-19158 33 29 knight knight NOUN eblip-19158 33 30 , , PUNCT eblip-19158 33 31 2006 2006 NUM eblip-19158 33 32 ) ) PUNCT eblip-19158 33 33 . . PUNCT eblip-19158 34 1 performance performance NOUN eblip-19158 34 2 - - PUNCT eblip-19158 34 3 based base VERB eblip-19158 34 4 assignments assignment NOUN eblip-19158 34 5 are be AUX eblip-19158 34 6 key key ADJ eblip-19158 34 7 ways way NOUN eblip-19158 34 8 to to PART eblip-19158 34 9 gauge gauge VERB eblip-19158 34 10 how how SCONJ eblip-19158 34 11 students student NOUN eblip-19158 34 12 are be AUX eblip-19158 34 13 internalizing internalize VERB eblip-19158 34 14 what what PRON eblip-19158 34 15 they they PRON eblip-19158 34 16 are be AUX eblip-19158 34 17 taught teach VERB eblip-19158 34 18 in in ADP eblip-19158 34 19 class class NOUN eblip-19158 34 20 . . PUNCT eblip-19158 35 1 unfortunately unfortunately ADV eblip-19158 35 2 , , PUNCT eblip-19158 35 3 unless unless SCONJ eblip-19158 35 4 librarians librarian NOUN eblip-19158 35 5 are be AUX eblip-19158 35 6 teaching teach VERB eblip-19158 35 7 full full ADJ eblip-19158 35 8 - - PUNCT eblip-19158 35 9 semester semester NOUN eblip-19158 35 10 courses course NOUN eblip-19158 35 11 , , PUNCT eblip-19158 35 12 they they PRON eblip-19158 35 13 rarely rarely ADV eblip-19158 35 14 see see VERB eblip-19158 35 15 the the DET eblip-19158 35 16 outcome outcome NOUN eblip-19158 35 17 of of ADP eblip-19158 35 18 what what PRON eblip-19158 35 19 they they PRON eblip-19158 35 20 teach teach VERB eblip-19158 35 21 . . PUNCT eblip-19158 36 1 one one NUM eblip-19158 36 2 way way NOUN eblip-19158 36 3 that that PRON eblip-19158 36 4 librarians librarian NOUN eblip-19158 36 5 can can AUX eblip-19158 36 6 become become VERB eblip-19158 36 7 involved involve VERB eblip-19158 36 8 in in ADP eblip-19158 36 9 authentic authentic ADJ eblip-19158 36 10 assessment assessment NOUN eblip-19158 36 11 is be AUX eblip-19158 36 12 to to PART eblip-19158 36 13 collect collect VERB eblip-19158 36 14 work work NOUN eblip-19158 36 15 samples sample NOUN eblip-19158 36 16 from from ADP eblip-19158 36 17 the the DET eblip-19158 36 18 students student NOUN eblip-19158 36 19 who who PRON eblip-19158 36 20 come come VERB eblip-19158 36 21 to to ADP eblip-19158 36 22 the the DET eblip-19158 36 23 library library NOUN eblip-19158 36 24 for for ADP eblip-19158 36 25 instruction instruction NOUN eblip-19158 36 26 . . PUNCT eblip-19158 37 1 librarians librarian NOUN eblip-19158 37 2 can can AUX eblip-19158 37 3 then then ADV eblip-19158 37 4 evaluate evaluate VERB eblip-19158 37 5 the the DET eblip-19158 37 6 samples sample NOUN eblip-19158 37 7 based base VERB eblip-19158 37 8 on on ADP eblip-19158 37 9 the the DET eblip-19158 37 10 skills skill NOUN eblip-19158 37 11 that that SCONJ eblip-19158 37 12 they they PRON eblip-19158 37 13 would would AUX eblip-19158 37 14 expect expect VERB eblip-19158 37 15 to to PART eblip-19158 37 16 see see VERB eblip-19158 37 17 in in ADP eblip-19158 37 18 student student NOUN eblip-19158 37 19 work work NOUN eblip-19158 37 20 . . PUNCT eblip-19158 38 1 the the DET eblip-19158 38 2 results result NOUN eblip-19158 38 3 of of ADP eblip-19158 38 4 such such DET eblip-19158 38 5 an an DET eblip-19158 38 6 assessment assessment NOUN eblip-19158 38 7 can can AUX eblip-19158 38 8 inform inform VERB eblip-19158 38 9 future future ADJ eblip-19158 38 10 decisions decision NOUN eblip-19158 38 11 about about ADP eblip-19158 38 12 instruction instruction NOUN eblip-19158 38 13 , , PUNCT eblip-19158 38 14 identifying identify VERB eblip-19158 38 15 areas area NOUN eblip-19158 38 16 where where SCONJ eblip-19158 38 17 students student NOUN eblip-19158 38 18 excel excel VERB eblip-19158 38 19 or or CCONJ eblip-19158 38 20 struggle struggle VERB eblip-19158 38 21 and and CCONJ eblip-19158 38 22 designing designing NOUN eblip-19158 38 23 instruction instruction NOUN eblip-19158 38 24 programs program NOUN eblip-19158 38 25 that that PRON eblip-19158 38 26 better well ADV eblip-19158 38 27 support support VERB eblip-19158 38 28 student student NOUN eblip-19158 38 29 learning learning NOUN eblip-19158 38 30 . . PUNCT eblip-19158 38 31                                 SPACE eblip-19158 39 1 one one NUM eblip-19158 39 2 particular particular ADJ eblip-19158 39 3 method method NOUN eblip-19158 39 4 that that PRON eblip-19158 39 5 librarians librarian NOUN eblip-19158 39 6 have have AUX eblip-19158 39 7 used use VERB eblip-19158 39 8 to to PART eblip-19158 39 9 assess assess VERB eblip-19158 39 10 student student NOUN eblip-19158 39 11 information information NOUN eblip-19158 39 12 literacy literacy NOUN eblip-19158 39 13 skills skill NOUN eblip-19158 39 14 is be AUX eblip-19158 39 15 the the DET eblip-19158 39 16 rubric rubric NOUN eblip-19158 39 17 . . PUNCT eblip-19158 40 1 rubrics rubric NOUN eblip-19158 40 2 are be AUX eblip-19158 40 3 advantageous advantageous ADJ eblip-19158 40 4 assessment assessment NOUN eblip-19158 40 5 tools tool NOUN eblip-19158 40 6 because because SCONJ eblip-19158 40 7 they they PRON eblip-19158 40 8 can can AUX eblip-19158 40 9 be be AUX eblip-19158 40 10 used use VERB eblip-19158 40 11 to to PART eblip-19158 40 12 turn turn VERB eblip-19158 40 13 subjective subjective ADJ eblip-19158 40 14 data datum NOUN eblip-19158 40 15 into into ADP eblip-19158 40 16 objective objective ADJ eblip-19158 40 17 information information NOUN eblip-19158 40 18 that that PRON eblip-19158 40 19 can can AUX eblip-19158 40 20 help help VERB eblip-19158 40 21 librarians librarian NOUN eblip-19158 40 22 make make VERB eblip-19158 40 23 decisions decision NOUN eblip-19158 40 24 about about ADP eblip-19158 40 25 how how SCONJ eblip-19158 40 26 to to PART eblip-19158 40 27 best well ADV eblip-19158 40 28 support support VERB eblip-19158 40 29 student student NOUN eblip-19158 40 30 learning learning NOUN eblip-19158 40 31 ( ( PUNCT eblip-19158 40 32 oakleaf oakleaf NOUN eblip-19158 40 33 , , PUNCT eblip-19158 40 34 2007 2007 NUM eblip-19158 40 35 ; ; PUNCT eblip-19158 40 36 arter arter NOUN eblip-19158 40 37 & & CCONJ eblip-19158 40 38 mctighe mctighe PROPN eblip-19158 40 39 , , PUNCT eblip-19158 40 40 2001 2001 NUM eblip-19158 40 41 ) ) PUNCT eblip-19158 40 42 . . PUNCT eblip-19158 41 1 rubrics rubric NOUN eblip-19158 41 2 allow allow VERB eblip-19158 41 3 an an DET eblip-19158 41 4 evaluation evaluation NOUN eblip-19158 41 5 of of ADP eblip-19158 41 6 students student NOUN eblip-19158 41 7 ’ ’ PART eblip-19158 41 8 information information NOUN eblip-19158 41 9 literacy literacy NOUN eblip-19158 41 10 skills skill NOUN eblip-19158 41 11 within within ADP eblip-19158 41 12 the the DET eblip-19158 41 13 context context NOUN eblip-19158 41 14 of of ADP eblip-19158 41 15 an an DET eblip-19158 41 16 actual actual ADJ eblip-19158 41 17 writing writing NOUN eblip-19158 41 18 assignment assignment NOUN eblip-19158 41 19 , , PUNCT eblip-19158 41 20 supporting support VERB eblip-19158 41 21 the the DET eblip-19158 41 22 notion notion NOUN eblip-19158 41 23 of of ADP eblip-19158 41 24 authentic authentic ADJ eblip-19158 41 25 assessment assessment NOUN eblip-19158 41 26 . . PUNCT eblip-19158 41 27                                     SPACE eblip-19158 41 28 in in ADP eblip-19158 41 29 the the DET eblip-19158 41 30 last last ADJ eblip-19158 41 31 ten ten NUM eblip-19158 41 32 years year NOUN eblip-19158 41 33 several several ADJ eblip-19158 41 34 studies study NOUN eblip-19158 41 35 that that PRON eblip-19158 41 36 use use VERB eblip-19158 41 37 rubrics rubric NOUN eblip-19158 41 38 to to PART eblip-19158 41 39 assess assess VERB eblip-19158 41 40 student student NOUN eblip-19158 41 41 information information NOUN eblip-19158 41 42 literacy literacy NOUN eblip-19158 41 43 skills skill NOUN eblip-19158 41 44 have have AUX eblip-19158 41 45 been be AUX eblip-19158 41 46 conducted conduct VERB eblip-19158 41 47 . . PUNCT eblip-19158 42 1 in in ADP eblip-19158 42 2 2002 2002 NUM eblip-19158 42 3 , , PUNCT eblip-19158 42 4 emmons emmons PROPN eblip-19158 42 5 and and CCONJ eblip-19158 42 6 martin martin PROPN eblip-19158 42 7 used use VERB eblip-19158 42 8 rubrics rubric NOUN eblip-19158 42 9 to to PART eblip-19158 42 10 evaluate evaluate VERB eblip-19158 42 11 10 10 NUM eblip-19158 42 12 semesters semester NOUN eblip-19158 42 13 ’ ' PUNCT eblip-19158 42 14 worth worth NOUN eblip-19158 42 15 of of ADP eblip-19158 42 16 student student NOUN eblip-19158 42 17 portfolios portfolio NOUN eblip-19158 42 18 from from ADP eblip-19158 42 19 an an DET eblip-19158 42 20 english english ADJ eblip-19158 42 21 composition composition NOUN eblip-19158 42 22 course course NOUN eblip-19158 42 23 in in ADP eblip-19158 42 24 order order NOUN eblip-19158 42 25 to to PART eblip-19158 42 26 evaluate evaluate VERB eblip-19158 42 27 how how SCONJ eblip-19158 42 28 changes change NOUN eblip-19158 42 29 to to ADP eblip-19158 42 30 library library NOUN eblip-19158 42 31 instruction instruction NOUN eblip-19158 42 32 impacted impact VERB eblip-19158 42 33 the the DET eblip-19158 42 34 students student NOUN eblip-19158 42 35 ’ ’ PART eblip-19158 42 36 research research NOUN eblip-19158 42 37 processes process NOUN eblip-19158 42 38 . . PUNCT eblip-19158 43 1 this this DET eblip-19158 43 2 study study NOUN eblip-19158 43 3 showed show VERB eblip-19158 43 4 that that SCONJ eblip-19158 43 5 while while SCONJ eblip-19158 43 6 some some DET eblip-19158 43 7 small small ADJ eblip-19158 43 8 improvements improvement NOUN eblip-19158 43 9 were be AUX eblip-19158 43 10 made make VERB eblip-19158 43 11 in in ADP eblip-19158 43 12 the the DET eblip-19158 43 13 way way NOUN eblip-19158 43 14 students student NOUN eblip-19158 43 15 selected select VERB eblip-19158 43 16 information information NOUN eblip-19158 43 17 resources resource NOUN eblip-19158 43 18 , , PUNCT eblip-19158 43 19 closer close ADJ eblip-19158 43 20 collaboration collaboration NOUN eblip-19158 43 21 between between ADP eblip-19158 43 22 the the DET eblip-19158 43 23 composition composition NOUN eblip-19158 43 24 program program NOUN eblip-19158 43 25 and and CCONJ eblip-19158 43 26 the the DET eblip-19158 43 27 library library NOUN eblip-19158 43 28 was be AUX eblip-19158 43 29 needed need VERB eblip-19158 43 30 ( ( PUNCT eblip-19158 43 31 emmons emmons PROPN eblip-19158 43 32 & & CCONJ eblip-19158 43 33 martin martin PROPN eblip-19158 43 34 , , PUNCT eblip-19158 43 35 2002 2002 NUM eblip-19158 43 36 ) ) PUNCT eblip-19158 43 37 . . PUNCT eblip-19158 44 1 choinski choinski PROPN eblip-19158 44 2 , , PUNCT eblip-19158 44 3 mark mark NOUN eblip-19158 44 4 , , PUNCT eblip-19158 44 5 and and CCONJ eblip-19158 44 6 murphy murphy PROPN eblip-19158 44 7 ( ( PUNCT eblip-19158 44 8 2003 2003 NUM eblip-19158 44 9 ) ) PUNCT eblip-19158 44 10 developed develop VERB eblip-19158 44 11 a a DET eblip-19158 44 12 rubric rubric NOUN eblip-19158 44 13 to to ADP eblip-19158 44 14 score score VERB eblip-19158 44 15 papers paper NOUN eblip-19158 44 16 from from ADP eblip-19158 44 17 an an DET eblip-19158 44 18 information information NOUN eblip-19158 44 19 resources resource NOUN eblip-19158 44 20 course course NOUN eblip-19158 44 21 at at ADP eblip-19158 44 22 the the DET eblip-19158 44 23 university university PROPN eblip-19158 44 24 of of ADP eblip-19158 44 25 mississippi mississippi PROPN eblip-19158 44 26 . . PUNCT eblip-19158 45 1 they they PRON eblip-19158 45 2 found find VERB eblip-19158 45 3 that that SCONJ eblip-19158 45 4 while while SCONJ eblip-19158 45 5 students student NOUN eblip-19158 45 6 succeeded succeed VERB eblip-19158 45 7 in in ADP eblip-19158 45 8 narrowing narrow VERB eblip-19158 45 9 research research NOUN eblip-19158 45 10 topics topic NOUN eblip-19158 45 11 , , PUNCT eblip-19158 45 12 discussing discuss VERB eblip-19158 45 13 their their PRON eblip-19158 45 14 research research NOUN eblip-19158 45 15 process process NOUN eblip-19158 45 16 , , PUNCT eblip-19158 45 17 and and CCONJ eblip-19158 45 18 identifying identify VERB eblip-19158 45 19 source source NOUN eblip-19158 45 20 types type NOUN eblip-19158 45 21 , , PUNCT eblip-19158 45 22 they they PRON eblip-19158 45 23 struggled struggle VERB eblip-19158 45 24 with with ADP eblip-19158 45 25 higher high ADJ eblip-19158 45 26 - - PUNCT eblip-19158 45 27 order order NOUN eblip-19158 45 28 critical critical ADJ eblip-19158 45 29 thinking thinking NOUN eblip-19158 45 30 skills skill NOUN eblip-19158 45 31 . . PUNCT eblip-19158 46 1 knight knight NOUN eblip-19158 46 2 ( ( PUNCT eblip-19158 46 3 2006 2006 NUM eblip-19158 46 4 ) ) PUNCT eblip-19158 46 5 scored score VERB eblip-19158 46 6 annotated annotate VERB eblip-19158 46 7 bibliographies bibliography NOUN eblip-19158 46 8 in in ADP eblip-19158 46 9 order order NOUN eblip-19158 46 10 to to PART eblip-19158 46 11 evaluate evaluate VERB eblip-19158 46 12 information information NOUN eblip-19158 46 13 literacy literacy NOUN eblip-19158 46 14 skills skill NOUN eblip-19158 46 15 in in ADP eblip-19158 46 16 a a DET eblip-19158 46 17 freshman freshman NOUN eblip-19158 46 18 - - PUNCT eblip-19158 46 19 level level NOUN eblip-19158 46 20 writing writing NOUN eblip-19158 46 21 course course NOUN eblip-19158 46 22 . . PUNCT eblip-19158 47 1 the the DET eblip-19158 47 2 study study NOUN eblip-19158 47 3 uncovered uncover VERB eblip-19158 47 4 areas area NOUN eblip-19158 47 5 where where SCONJ eblip-19158 47 6 the the DET eblip-19158 47 7 library library NOUN eblip-19158 47 8 could could AUX eblip-19158 47 9 better well ADV eblip-19158 47 10 support support VERB eblip-19158 47 11 student student NOUN eblip-19158 47 12 learning learning NOUN eblip-19158 47 13 , , PUNCT eblip-19158 47 14 including include VERB eblip-19158 47 15 focusing focus VERB eblip-19158 47 16 more more ADV eblip-19158 47 17 on on ADP eblip-19158 47 18 mechanical mechanical ADJ eblip-19158 47 19 skills skill NOUN eblip-19158 47 20 ( ( PUNCT eblip-19158 47 21 database database NOUN eblip-19158 47 22 selection selection NOUN eblip-19158 47 23 and and CCONJ eblip-19158 47 24 use use NOUN eblip-19158 47 25 ) ) PUNCT eblip-19158 47 26 as as ADV eblip-19158 47 27 well well ADV eblip-19158 47 28 as as ADP eblip-19158 47 29 critical critical ADJ eblip-19158 47 30 - - PUNCT eblip-19158 47 31 thinking thinking NOUN eblip-19158 47 32 skills skill NOUN eblip-19158 47 33 ( ( PUNCT eblip-19158 47 34 evaluating evaluate VERB eblip-19158 47 35 the the DET eblip-19158 47 36 sources source NOUN eblip-19158 47 37 found find VERB eblip-19158 47 38 in in ADP eblip-19158 47 39 the the DET eblip-19158 47 40 databases database NOUN eblip-19158 47 41 ) ) PUNCT eblip-19158 47 42 . . PUNCT eblip-19158 48 1 these these DET eblip-19158 48 2 studies study NOUN eblip-19158 48 3 , , PUNCT eblip-19158 48 4 which which PRON eblip-19158 48 5 used use VERB eblip-19158 48 6 rubrics rubric NOUN eblip-19158 48 7 to to PART eblip-19158 48 8 evaluate evaluate VERB eblip-19158 48 9 student student NOUN eblip-19158 48 10 writing writing NOUN eblip-19158 48 11 , , PUNCT eblip-19158 48 12 all all PRON eblip-19158 48 13 share share VERB eblip-19158 48 14 similar similar ADJ eblip-19158 48 15 findings finding NOUN eblip-19158 48 16 — — PUNCT eblip-19158 48 17 students student NOUN eblip-19158 48 18 succeed succeed VERB eblip-19158 48 19 in in ADP eblip-19158 48 20 identifying identify VERB eblip-19158 48 21 basic basic ADJ eblip-19158 48 22 information information NOUN eblip-19158 48 23 if if SCONJ eblip-19158 48 24 they they PRON eblip-19158 48 25 are be AUX eblip-19158 48 26 directly directly ADV eblip-19158 48 27 asked ask VERB eblip-19158 48 28 to to PART eblip-19158 48 29 do do VERB eblip-19158 48 30 so so ADV eblip-19158 48 31 but but CCONJ eblip-19158 48 32 have have VERB eblip-19158 48 33 difficulty difficulty NOUN eblip-19158 48 34 critically critically ADV eblip-19158 48 35 evaluating evaluate VERB eblip-19158 48 36 and and CCONJ eblip-19158 48 37 using use VERB eblip-19158 48 38 academic academic ADJ eblip-19158 48 39 - - PUNCT eblip-19158 48 40 level level NOUN eblip-19158 48 41 sources source NOUN eblip-19158 48 42 . . PUNCT eblip-19158 48 43                                     SPACE eblip-19158 49 1 while while SCONJ eblip-19158 49 2 these these DET eblip-19158 49 3 articles article NOUN eblip-19158 49 4 help help VERB eblip-19158 49 5 inform inform VERB eblip-19158 49 6 how how SCONJ eblip-19158 49 7 students student NOUN eblip-19158 49 8 apply apply VERB eblip-19158 49 9 information information NOUN eblip-19158 49 10 literacy literacy NOUN eblip-19158 49 11 skills skill NOUN eblip-19158 49 12 in in ADP eblip-19158 49 13 authentic authentic ADJ eblip-19158 49 14 assessment assessment NOUN eblip-19158 49 15 tasks task NOUN eblip-19158 49 16 , , PUNCT eblip-19158 49 17 they they PRON eblip-19158 49 18 do do AUX eblip-19158 49 19 not not PART eblip-19158 49 20 provide provide VERB eblip-19158 49 21 very very ADV eblip-19158 49 22 detailed detailed ADJ eblip-19158 49 23 information information NOUN eblip-19158 49 24 on on ADP eblip-19158 49 25 how how SCONJ eblip-19158 49 26 information information NOUN eblip-19158 49 27 literacy literacy NOUN eblip-19158 49 28 rubrics rubric NOUN eblip-19158 49 29 were be AUX eblip-19158 49 30 developed develop VERB eblip-19158 49 31 and and CCONJ eblip-19158 49 32 applied apply VERB eblip-19158 49 33 to to ADP eblip-19158 49 34 student student NOUN eblip-19158 49 35 work work NOUN eblip-19158 49 36 . . PUNCT eblip-19158 50 1 studies study NOUN eblip-19158 50 2 that that PRON eblip-19158 50 3 delve delve VERB eblip-19158 50 4 deeper deeply ADV eblip-19158 50 5 into into ADP eblip-19158 50 6 the the DET eblip-19158 50 7 rubric rubric ADJ eblip-19158 50 8 creation creation NOUN eblip-19158 50 9 process process NOUN eblip-19158 50 10 rectify rectify VERB eblip-19158 50 11 some some PRON eblip-19158 50 12 of of ADP eblip-19158 50 13 these these DET eblip-19158 50 14 issues issue NOUN eblip-19158 50 15 . . PUNCT eblip-19158 51 1 fagerheim fagerheim PROPN eblip-19158 51 2 and and CCONJ eblip-19158 51 3 shrode shrode PROPN eblip-19158 51 4 ( ( PUNCT eblip-19158 51 5 2009 2009 NUM eblip-19158 51 6 ) ) PUNCT eblip-19158 51 7 provide provide VERB eblip-19158 51 8 insight insight NOUN eblip-19158 51 9 into into ADP eblip-19158 51 10 the the DET eblip-19158 51 11 development development NOUN eblip-19158 51 12 of of ADP eblip-19158 51 13 an an DET eblip-19158 51 14 information information NOUN eblip-19158 51 15 literacy literacy NOUN eblip-19158 51 16 rubric rubric NOUN eblip-19158 51 17 for for ADP eblip-19158 51 18 upper upper ADJ eblip-19158 51 19 - - PUNCT eblip-19158 51 20 level level NOUN eblip-19158 51 21 science science NOUN eblip-19158 51 22 students student NOUN eblip-19158 51 23 , , PUNCT eblip-19158 51 24 such such ADJ eblip-19158 51 25 as as ADP eblip-19158 51 26 collecting collect VERB eblip-19158 51 27 benchmarks benchmark NOUN eblip-19158 51 28 for for ADP eblip-19158 51 29 graduates graduate NOUN eblip-19158 51 30 , , PUNCT eblip-19158 51 31 identifying identify VERB eblip-19158 51 32 measurable measurable ADJ eblip-19158 51 33 objectives objective NOUN eblip-19158 51 34 for for ADP eblip-19158 51 35 these these DET eblip-19158 51 36 benchmarks benchmark NOUN eblip-19158 51 37 , , PUNCT eblip-19158 51 38 and and CCONJ eblip-19158 51 39 consulting consult VERB eblip-19158 51 40 with with ADP eblip-19158 51 41 faculty faculty NOUN eblip-19158 51 42 members member NOUN eblip-19158 51 43 within within ADP eblip-19158 51 44 the the DET eblip-19158 51 45 discipline discipline NOUN eblip-19158 51 46 , , PUNCT eblip-19158 51 47 but but CCONJ eblip-19158 51 48 there there PRON eblip-19158 51 49 is be VERB eblip-19158 51 50 no no DET eblip-19158 51 51 discussion discussion NOUN eblip-19158 51 52 of of ADP eblip-19158 51 53 how how SCONJ eblip-19158 51 54 scorers scorer NOUN eblip-19158 51 55 were be AUX eblip-19158 51 56 trained train VERB eblip-19158 51 57 to to PART eblip-19158 51 58 use use VERB eblip-19158 51 59 the the DET eblip-19158 51 60 rubric rubric NOUN eblip-19158 51 61 . . PUNCT eblip-19158 52 1 hoffman hoffman PROPN eblip-19158 52 2 and and CCONJ eblip-19158 52 3 labonte labonte PROPN eblip-19158 52 4 ( ( PUNCT eblip-19158 52 5 2012 2012 NUM eblip-19158 52 6 ) ) PUNCT eblip-19158 52 7 explore explore VERB eblip-19158 52 8 the the DET eblip-19158 52 9 validity validity NOUN eblip-19158 52 10 of of ADP eblip-19158 52 11 using use VERB eblip-19158 52 12 an an DET eblip-19158 52 13 information information NOUN eblip-19158 52 14 literacy literacy NOUN eblip-19158 52 15 rubric rubric NOUN eblip-19158 52 16 to to ADP eblip-19158 52 17 score score NOUN eblip-19158 52 18 student student NOUN eblip-19158 52 19 writing writing NOUN eblip-19158 52 20 . . PUNCT eblip-19158 53 1 the the DET eblip-19158 53 2 authors author NOUN eblip-19158 53 3 discuss discuss VERB eblip-19158 53 4 the the DET eblip-19158 53 5 brainstorming brainstorming NOUN eblip-19158 53 6 of of ADP eblip-19158 53 7 performance performance NOUN eblip-19158 53 8 criteria criterion NOUN eblip-19158 53 9 and and CCONJ eblip-19158 53 10 the the DET eblip-19158 53 11 alignment alignment NOUN eblip-19158 53 12 of of ADP eblip-19158 53 13 the the DET eblip-19158 53 14 rubric rubric NOUN eblip-19158 53 15 to to ADP eblip-19158 53 16 institutional institutional ADJ eblip-19158 53 17 outcomes outcome NOUN eblip-19158 53 18 but but CCONJ eblip-19158 53 19 there there PRON eblip-19158 53 20 is be VERB eblip-19158 53 21 no no DET eblip-19158 53 22 description description NOUN eblip-19158 53 23 of of ADP eblip-19158 53 24 the the DET eblip-19158 53 25 training training NOUN eblip-19158 53 26 process process NOUN eblip-19158 53 27 for for ADP eblip-19158 53 28 raters rater NOUN eblip-19158 53 29 . . PUNCT eblip-19158 54 1 helvoort helvoort NOUN eblip-19158 54 2 ( ( PUNCT eblip-19158 54 3 2010 2010 NUM eblip-19158 54 4 ) ) PUNCT eblip-19158 54 5 explains explain VERB eblip-19158 54 6 how how SCONJ eblip-19158 54 7 a a DET eblip-19158 54 8 rubric rubric NOUN eblip-19158 54 9 was be AUX eblip-19158 54 10 created create VERB eblip-19158 54 11 to to PART eblip-19158 54 12 evaluate evaluate VERB eblip-19158 54 13 dutch dutch ADJ eblip-19158 54 14 students student NOUN eblip-19158 54 15 ’ ’ PART eblip-19158 54 16 information information NOUN eblip-19158 54 17 literacy literacy NOUN eblip-19158 54 18 skills skill NOUN eblip-19158 54 19 , , PUNCT eblip-19158 54 20 but but CCONJ eblip-19158 54 21 the the DET eblip-19158 54 22 rubric rubric NOUN eblip-19158 54 23 was be AUX eblip-19158 54 24 meant mean VERB eblip-19158 54 25 to to PART eblip-19158 54 26 be be AUX eblip-19158 54 27 generalizable generalizable ADJ eblip-19158 54 28 to to ADP eblip-19158 54 29 a a DET eblip-19158 54 30 variety variety NOUN eblip-19158 54 31 of of ADP eblip-19158 54 32 courses course NOUN eblip-19158 54 33 and and CCONJ eblip-19158 54 34 assignments assignment NOUN eblip-19158 54 35 , , PUNCT eblip-19158 54 36 making make VERB eblip-19158 54 37 it it PRON eblip-19158 54 38 difficult difficult ADJ eblip-19158 54 39 to to PART eblip-19158 54 40 transfer transfer VERB eblip-19158 54 41 the the DET eblip-19158 54 42 processes process NOUN eblip-19158 54 43 described describe VERB eblip-19158 54 44 to to ADP eblip-19158 54 45 a a DET eblip-19158 54 46 single single ADJ eblip-19158 54 47 course course NOUN eblip-19158 54 48 assignment assignment NOUN eblip-19158 54 49 . . PUNCT eblip-19158 54 50         SPACE eblip-19158 55 1 perhaps perhaps ADV eblip-19158 55 2 the the DET eblip-19158 55 3 most most ADJ eblip-19158 55 4 in in ADP eblip-19158 55 5 - - PUNCT eblip-19158 55 6 depth depth NOUN eblip-19158 55 7 descriptions description NOUN eblip-19158 55 8 of of ADP eblip-19158 55 9 the the DET eblip-19158 55 10 rubric rubric ADJ eblip-19158 55 11 development development NOUN eblip-19158 55 12 and and CCONJ eblip-19158 55 13 training training NOUN eblip-19158 55 14 process process NOUN eblip-19158 55 15 appear appear VERB eblip-19158 55 16 in in ADP eblip-19158 55 17 two two NUM eblip-19158 55 18 studies study NOUN eblip-19158 55 19 by by ADP eblip-19158 55 20 oakleaf oakleaf NOUN eblip-19158 55 21 ( ( PUNCT eblip-19158 55 22 2007 2007 NUM eblip-19158 55 23 ; ; PUNCT eblip-19158 55 24 2009 2009 NUM eblip-19158 55 25 ) ) PUNCT eblip-19158 55 26 in in ADP eblip-19158 55 27 which which PRON eblip-19158 55 28 rubrics rubric NOUN eblip-19158 55 29 were be AUX eblip-19158 55 30 used use VERB eblip-19158 55 31 to to PART eblip-19158 55 32 score score VERB eblip-19158 55 33 student student NOUN eblip-19158 55 34 responses response NOUN eblip-19158 55 35 on on ADP eblip-19158 55 36 an an DET eblip-19158 55 37 online online ADJ eblip-19158 55 38 information information NOUN eblip-19158 55 39 literacy literacy NOUN eblip-19158 55 40 tutorial tutorial NOUN eblip-19158 55 41 . . PUNCT eblip-19158 56 1 oakleaf oakleaf NOUN eblip-19158 56 2 describes describe VERB eblip-19158 56 3 the the DET eblip-19158 56 4 process process NOUN eblip-19158 56 5 for for ADP eblip-19158 56 6 training train VERB eblip-19158 56 7 the the DET eblip-19158 56 8 raters rater NOUN eblip-19158 56 9 on on ADP eblip-19158 56 10 rubric rubric ADJ eblip-19158 56 11 application application NOUN eblip-19158 56 12 and and CCONJ eblip-19158 56 13 the the DET eblip-19158 56 14 ways way NOUN eblip-19158 56 15 in in ADP eblip-19158 56 16 which which PRON eblip-19158 56 17 that that SCONJ eblip-19158 56 18 training training NOUN eblip-19158 56 19 impacted impact VERB eblip-19158 56 20 inter inter ADJ eblip-19158 56 21 - - ADJ eblip-19158 56 22 rater rater ADJ eblip-19158 56 23 reliability reliability NOUN eblip-19158 56 24 and and CCONJ eblip-19158 56 25 validity validity NOUN eblip-19158 56 26 ( ( PUNCT eblip-19158 56 27 oakleaf oakleaf NOUN eblip-19158 56 28 , , PUNCT eblip-19158 56 29 2007 2007 NUM eblip-19158 56 30 ) ) PUNCT eblip-19158 56 31 . . PUNCT eblip-19158 57 1 oakleaf oakleaf NOUN eblip-19158 57 2 ( ( PUNCT eblip-19158 57 3 2009 2009 NUM eblip-19158 57 4 ) ) PUNCT eblip-19158 57 5 gives give VERB eblip-19158 57 6 a a DET eblip-19158 57 7 description description NOUN eblip-19158 57 8 of of ADP eblip-19158 57 9 the the DET eblip-19158 57 10 mandatory mandatory ADJ eblip-19158 57 11 training training NOUN eblip-19158 57 12 session session NOUN eblip-19158 57 13 . . PUNCT eblip-19158 58 1 the the DET eblip-19158 58 2 raters rater NOUN eblip-19158 58 3 were be AUX eblip-19158 58 4 divided divide VERB eblip-19158 58 5 into into ADP eblip-19158 58 6 small small ADJ eblip-19158 58 7 groups group NOUN eblip-19158 58 8 ; ; PUNCT eblip-19158 58 9 the the DET eblip-19158 58 10 purpose purpose NOUN eblip-19158 58 11 of of ADP eblip-19158 58 12 the the DET eblip-19158 58 13 study study NOUN eblip-19158 58 14 , , PUNCT eblip-19158 58 15 the the DET eblip-19158 58 16 assignment assignment NOUN eblip-19158 58 17 , , PUNCT eblip-19158 58 18 and and CCONJ eblip-19158 58 19 the the DET eblip-19158 58 20 rubric rubric NOUN eblip-19158 58 21 were be AUX eblip-19158 58 22 introduced introduce VERB eblip-19158 58 23 and and CCONJ eblip-19158 58 24 discussed discuss VERB eblip-19158 58 25 ; ; PUNCT eblip-19158 58 26 and and CCONJ eblip-19158 58 27 five five NUM eblip-19158 58 28 sample sample NOUN eblip-19158 58 29 papers paper NOUN eblip-19158 58 30 were be AUX eblip-19158 58 31 used use VERB eblip-19158 58 32 as as ADP eblip-19158 58 33 “ " PUNCT eblip-19158 58 34 anchors anchor NOUN eblip-19158 58 35 ” " PUNCT eblip-19158 58 36 and and CCONJ eblip-19158 58 37 scored score VERB eblip-19158 58 38 during during ADP eblip-19158 58 39 a a DET eblip-19158 58 40 model model NOUN eblip-19158 58 41 read read NOUN eblip-19158 58 42 - - PUNCT eblip-19158 58 43 aloud aloud NOUN eblip-19158 58 44 of of ADP eblip-19158 58 45 how how SCONJ eblip-19158 58 46 to to PART eblip-19158 58 47 apply apply VERB eblip-19158 58 48 the the DET eblip-19158 58 49 rubric rubric NOUN eblip-19158 58 50 . . PUNCT eblip-19158 59 1 oakleaf oakleaf NOUN eblip-19158 59 2 used use VERB eblip-19158 59 3 maki maki PROPN eblip-19158 59 4 ’s ’s PART eblip-19158 59 5 6 6 NUM eblip-19158 59 6 step step NOUN eblip-19158 59 7 - - PUNCT eblip-19158 59 8 norming norme VERB eblip-19158 59 9 process process NOUN eblip-19158 59 10 to to PART eblip-19158 59 11 have have VERB eblip-19158 59 12 the the DET eblip-19158 59 13 raters rater NOUN eblip-19158 59 14 score score NOUN eblip-19158 59 15 sample sample NOUN eblip-19158 59 16 papers paper NOUN eblip-19158 59 17 and and CCONJ eblip-19158 59 18 then then ADV eblip-19158 59 19 discuss discuss VERB eblip-19158 59 20 and and CCONJ eblip-19158 59 21 reconcile reconcile ADJ eblip-19158 59 22 differences difference NOUN eblip-19158 59 23 in in ADP eblip-19158 59 24 their their PRON eblip-19158 59 25 scores score NOUN eblip-19158 59 26 ( ( PUNCT eblip-19158 59 27 oakleaf oakleaf NOUN eblip-19158 59 28 , , PUNCT eblip-19158 59 29 2009 2009 NUM eblip-19158 59 30 ; ; PUNCT eblip-19158 59 31 maki maki NOUN eblip-19158 59 32 , , PUNCT eblip-19158 59 33 2010 2010 NUM eblip-19158 59 34 ) ) PUNCT eblip-19158 59 35 . . PUNCT eblip-19158 60 1 this this DET eblip-19158 60 2 process process NOUN eblip-19158 60 3 was be AUX eblip-19158 60 4 repeated repeat VERB eblip-19158 60 5 twice twice ADV eblip-19158 60 6 on on ADP eblip-19158 60 7 sample sample NOUN eblip-19158 60 8 papers paper NOUN eblip-19158 60 9 before before SCONJ eblip-19158 60 10 the the DET eblip-19158 60 11 raters rater NOUN eblip-19158 60 12 were be AUX eblip-19158 60 13 ready ready ADJ eblip-19158 60 14 to to PART eblip-19158 60 15 score score VERB eblip-19158 60 16 sets set NOUN eblip-19158 60 17 of of ADP eblip-19158 60 18 student student NOUN eblip-19158 60 19 responses response NOUN eblip-19158 60 20 on on ADP eblip-19158 60 21 their their PRON eblip-19158 60 22 own own ADJ eblip-19158 60 23 . . PUNCT eblip-19158 61 1 oakleaf oakleaf NOUN eblip-19158 61 2 ’s ’s PART eblip-19158 61 3 explanation explanation NOUN eblip-19158 61 4 of of ADP eblip-19158 61 5 how how SCONJ eblip-19158 61 6 to to PART eblip-19158 61 7 train train VERB eblip-19158 61 8 raters rater NOUN eblip-19158 61 9 on on ADP eblip-19158 61 10 an an DET eblip-19158 61 11 information information NOUN eblip-19158 61 12 literacy literacy NOUN eblip-19158 61 13 rubric rubric NOUN eblip-19158 61 14 was be AUX eblip-19158 61 15 used use VERB eblip-19158 61 16 as as ADP eblip-19158 61 17 the the DET eblip-19158 61 18 model model NOUN eblip-19158 61 19 for for ADP eblip-19158 61 20 rubric rubric ADJ eblip-19158 61 21 training training NOUN eblip-19158 61 22 for for ADP eblip-19158 61 23 this this DET eblip-19158 61 24 study study NOUN eblip-19158 61 25 . . PUNCT eblip-19158 61 26                                 SPACE eblip-19158 62 1 though though SCONJ eblip-19158 62 2 the the DET eblip-19158 62 3 literature literature NOUN eblip-19158 62 4 on on ADP eblip-19158 62 5 using use VERB eblip-19158 62 6 rubrics rubric NOUN eblip-19158 62 7 to to PART eblip-19158 62 8 evaluate evaluate VERB eblip-19158 62 9 information information NOUN eblip-19158 62 10 literacy literacy NOUN eblip-19158 62 11 skills skill NOUN eblip-19158 62 12 has have AUX eblip-19158 62 13 grown grow VERB eblip-19158 62 14 over over ADP eblip-19158 62 15 the the DET eblip-19158 62 16 last last ADJ eblip-19158 62 17 decade decade NOUN eblip-19158 62 18 , , PUNCT eblip-19158 62 19 oakleaf oakleaf NOUN eblip-19158 62 20 ’s ’s PART eblip-19158 62 21 studies study NOUN eblip-19158 62 22 remain remain VERB eblip-19158 62 23 some some PRON eblip-19158 62 24 of of ADP eblip-19158 62 25 the the DET eblip-19158 62 26 only only ADJ eblip-19158 62 27 examples example NOUN eblip-19158 62 28 of of ADP eblip-19158 62 29 how how SCONJ eblip-19158 62 30 to to PART eblip-19158 62 31 actually actually ADV eblip-19158 62 32 apply apply VERB eblip-19158 62 33 the the DET eblip-19158 62 34 rubrics rubric NOUN eblip-19158 62 35 in in ADP eblip-19158 62 36 an an DET eblip-19158 62 37 academic academic ADJ eblip-19158 62 38 library library NOUN eblip-19158 62 39 setting setting NOUN eblip-19158 62 40 . . PUNCT eblip-19158 63 1 thus thus ADV eblip-19158 63 2 , , PUNCT eblip-19158 63 3 there there PRON eblip-19158 63 4 is be VERB eblip-19158 63 5 still still ADV eblip-19158 63 6 a a DET eblip-19158 63 7 need need NOUN eblip-19158 63 8 for for ADP eblip-19158 63 9 localized localized ADJ eblip-19158 63 10 studies study NOUN eblip-19158 63 11 that that PRON eblip-19158 63 12 describe describe VERB eblip-19158 63 13 the the DET eblip-19158 63 14 application application NOUN eblip-19158 63 15 of of ADP eblip-19158 63 16 information information NOUN eblip-19158 63 17 literacy literacy NOUN eblip-19158 63 18 rubrics rubric NOUN eblip-19158 63 19 . . PUNCT eblip-19158 64 1 this this DET eblip-19158 64 2 study study NOUN eblip-19158 64 3 contributes contribute VERB eblip-19158 64 4 to to ADP eblip-19158 64 5 the the DET eblip-19158 64 6 literature literature NOUN eblip-19158 64 7 by by ADP eblip-19158 64 8 providing provide VERB eblip-19158 64 9 a a DET eblip-19158 64 10 case case NOUN eblip-19158 64 11 study study NOUN eblip-19158 64 12 of of ADP eblip-19158 64 13 developing develop VERB eblip-19158 64 14 and and CCONJ eblip-19158 64 15 using use VERB eblip-19158 64 16 rubrics rubric NOUN eblip-19158 64 17 to to PART eblip-19158 64 18 evaluate evaluate VERB eblip-19158 64 19 how how SCONJ eblip-19158 64 20 students student NOUN eblip-19158 64 21 apply apply VERB eblip-19158 64 22 information information NOUN eblip-19158 64 23 literacy literacy NOUN eblip-19158 64 24 skills skill NOUN eblip-19158 64 25 in in ADP eblip-19158 64 26 their their PRON eblip-19158 64 27 class class NOUN eblip-19158 64 28 assignments assignment NOUN eblip-19158 64 29 . . PUNCT eblip-19158 64 30     SPACE eblip-19158 65 1 context context NOUN eblip-19158 65 2 and and CCONJ eblip-19158 65 3 aims aim VERB eblip-19158 65 4     SPACE eblip-19158 65 5 context context NOUN eblip-19158 65 6     SPACE eblip-19158 65 7 eng eng PROPN eblip-19158 65 8 102 102 NUM eblip-19158 65 9 is be AUX eblip-19158 65 10 the the DET eblip-19158 65 11 second second ADJ eblip-19158 65 12 in in ADP eblip-19158 65 13 a a DET eblip-19158 65 14 two two NUM eblip-19158 65 15 - - PUNCT eblip-19158 65 16 course course NOUN eblip-19158 65 17 sequence sequence NOUN eblip-19158 65 18 that that PRON eblip-19158 65 19 fulfills fulfill VERB eblip-19158 65 20 the the DET eblip-19158 65 21 english english ADJ eblip-19158 65 22 composition composition NOUN eblip-19158 65 23 requirement requirement NOUN eblip-19158 65 24 for for ADP eblip-19158 65 25 degree degree NOUN eblip-19158 65 26 completion completion NOUN eblip-19158 65 27 at at ADP eblip-19158 65 28 unlv unlv NOUN eblip-19158 65 29 . . PUNCT eblip-19158 66 1 eng eng PROPN eblip-19158 66 2 102 102 NUM eblip-19158 66 3 is be AUX eblip-19158 66 4 a a DET eblip-19158 66 5 high high ADJ eblip-19158 66 6 impact impact NOUN eblip-19158 66 7 course course NOUN eblip-19158 66 8 that that PRON eblip-19158 66 9 sees see VERB eblip-19158 66 10 a a DET eblip-19158 66 11 very very ADV eblip-19158 66 12 large large ADJ eblip-19158 66 13 enrollment enrollment NOUN eblip-19158 66 14 ; ; PUNCT eblip-19158 66 15 in in ADP eblip-19158 66 16 the the DET eblip-19158 66 17 fall fall NOUN eblip-19158 66 18 of of ADP eblip-19158 66 19 2012 2012 NUM eblip-19158 66 20 , , PUNCT eblip-19158 66 21 there there PRON eblip-19158 66 22 were be VERB eblip-19158 66 23 58 58 NUM eblip-19158 66 24 sections section NOUN eblip-19158 66 25 of of ADP eblip-19158 66 26 eng eng PROPN eblip-19158 66 27 102 102 NUM eblip-19158 66 28 , , PUNCT eblip-19158 66 29 with with ADP eblip-19158 66 30 25 25 NUM eblip-19158 66 31 students student NOUN eblip-19158 66 32 in in ADP eblip-19158 66 33 each each DET eblip-19158 66 34 section section NOUN eblip-19158 66 35 . . PUNCT eblip-19158 67 1 the the DET eblip-19158 67 2 course course NOUN eblip-19158 67 3 has have VERB eblip-19158 67 4 four four NUM eblip-19158 67 5 major major ADJ eblip-19158 67 6 assignments assignment NOUN eblip-19158 67 7 , , PUNCT eblip-19158 67 8 consisting consist VERB eblip-19158 67 9 of of ADP eblip-19158 67 10 a a DET eblip-19158 67 11 summary summary NOUN eblip-19158 67 12 and and CCONJ eblip-19158 67 13 synthesis synthesis NOUN eblip-19158 67 14 paper paper NOUN eblip-19158 67 15 , , PUNCT eblip-19158 67 16 an an DET eblip-19158 67 17 argument argument NOUN eblip-19158 67 18 analysis analysis NOUN eblip-19158 67 19 , , PUNCT eblip-19158 67 20 an an DET eblip-19158 67 21 annotated annotate VERB eblip-19158 67 22 bibliography bibliography NOUN eblip-19158 67 23 , , PUNCT eblip-19158 67 24 and and CCONJ eblip-19158 67 25 a a DET eblip-19158 67 26 researched research VERB eblip-19158 67 27 - - PUNCT eblip-19158 67 28 based base VERB eblip-19158 67 29 argument argument NOUN eblip-19158 67 30 essay essay NOUN eblip-19158 67 31 . . PUNCT eblip-19158 68 1 the the DET eblip-19158 68 2 third third ADJ eblip-19158 68 3 assignment assignment NOUN eblip-19158 68 4 , , PUNCT eblip-19158 68 5 the the DET eblip-19158 68 6 annotated annotate VERB eblip-19158 68 7 bibliography bibliography NOUN eblip-19158 68 8 , , PUNCT eblip-19158 68 9 was be AUX eblip-19158 68 10 the the DET eblip-19158 68 11 focus focus NOUN eblip-19158 68 12 of of ADP eblip-19158 68 13 this this DET eblip-19158 68 14 study study NOUN eblip-19158 68 15 since since SCONJ eblip-19158 68 16 the the DET eblip-19158 68 17 eng eng PROPN eblip-19158 68 18 102 102 NUM eblip-19158 68 19 library library NOUN eblip-19158 68 20 instruction instruction NOUN eblip-19158 68 21 sessions session NOUN eblip-19158 68 22 have have AUX eblip-19158 68 23 traditionally traditionally ADV eblip-19158 68 24 targeted target VERB eblip-19158 68 25 the the DET eblip-19158 68 26 learning learn VERB eblip-19158 68 27 outcomes outcome NOUN eblip-19158 68 28 of of ADP eblip-19158 68 29 the the DET eblip-19158 68 30 annotated annotate VERB eblip-19158 68 31 bibliography bibliography NOUN eblip-19158 68 32 project project NOUN eblip-19158 68 33 . . PUNCT eblip-19158 68 34     SPACE eblip-19158 69 1 aims aim VERB eblip-19158 69 2                                 SPACE eblip-19158 69 3 the the DET eblip-19158 69 4 author author NOUN eblip-19158 69 5 had have VERB eblip-19158 69 6 two two NUM eblip-19158 69 7 aims aim NOUN eblip-19158 69 8 for for ADP eblip-19158 69 9 this this DET eblip-19158 69 10 research research NOUN eblip-19158 69 11 project project NOUN eblip-19158 69 12 : : PUNCT eblip-19158 69 13 the the DET eblip-19158 69 14 first first ADJ eblip-19158 69 15 was be AUX eblip-19158 69 16 to to PART eblip-19158 69 17 gather gather VERB eblip-19158 69 18 evidence evidence NOUN eblip-19158 69 19 of of ADP eblip-19158 69 20 how how SCONJ eblip-19158 69 21 students student NOUN eblip-19158 69 22 apply apply VERB eblip-19158 69 23 information information NOUN eblip-19158 69 24 literacy literacy NOUN eblip-19158 69 25 skills skill NOUN eblip-19158 69 26 in in ADP eblip-19158 69 27 the the DET eblip-19158 69 28 context context NOUN eblip-19158 69 29 of of ADP eblip-19158 69 30 an an DET eblip-19158 69 31 authentic authentic ADJ eblip-19158 69 32 assessment assessment NOUN eblip-19158 69 33 activity activity NOUN eblip-19158 69 34 , , PUNCT eblip-19158 69 35 and and CCONJ eblip-19158 69 36 to to PART eblip-19158 69 37 use use VERB eblip-19158 69 38 that that DET eblip-19158 69 39 information information NOUN eblip-19158 69 40 to to ADP eblip-19158 69 41 fine fine ADJ eblip-19158 69 42 - - PUNCT eblip-19158 69 43 tune tune NOUN eblip-19158 69 44 information information NOUN eblip-19158 69 45 literacy literacy NOUN eblip-19158 69 46 instruction instruction NOUN eblip-19158 69 47 sessions session NOUN eblip-19158 69 48 for for ADP eblip-19158 69 49 the the DET eblip-19158 69 50 eng eng PROPN eblip-19158 69 51 102 102 NUM eblip-19158 69 52 course course NOUN eblip-19158 69 53 . . PUNCT eblip-19158 70 1 the the DET eblip-19158 70 2 second second ADJ eblip-19158 70 3 aim aim NOUN eblip-19158 70 4 was be AUX eblip-19158 70 5 to to PART eblip-19158 70 6 fill fill VERB eblip-19158 70 7 a a DET eblip-19158 70 8 gap gap NOUN eblip-19158 70 9 in in ADP eblip-19158 70 10 the the DET eblip-19158 70 11 literature literature NOUN eblip-19158 70 12 by by ADP eblip-19158 70 13 providing provide VERB eblip-19158 70 14 a a DET eblip-19158 70 15 case case NOUN eblip-19158 70 16 study study NOUN eblip-19158 70 17 of of ADP eblip-19158 70 18 rubric rubric ADJ eblip-19158 70 19 development development NOUN eblip-19158 70 20 and and CCONJ eblip-19158 70 21 application application NOUN eblip-19158 70 22 to to ADP eblip-19158 70 23 student student NOUN eblip-19158 70 24 work work NOUN eblip-19158 70 25 . . PUNCT eblip-19158 71 1 by by ADP eblip-19158 71 2 offering offer VERB eblip-19158 71 3 a a DET eblip-19158 71 4 transparent transparent ADJ eblip-19158 71 5 view view NOUN eblip-19158 71 6 of of ADP eblip-19158 71 7 how how SCONJ eblip-19158 71 8 the the DET eblip-19158 71 9 rubric rubric NOUN eblip-19158 71 10 was be AUX eblip-19158 71 11 created create VERB eblip-19158 71 12 and and CCONJ eblip-19158 71 13 how how SCONJ eblip-19158 71 14 raters rater NOUN eblip-19158 71 15 were be AUX eblip-19158 71 16 trained train VERB eblip-19158 71 17 , , PUNCT eblip-19158 71 18 the the DET eblip-19158 71 19 author author NOUN eblip-19158 71 20 hopes hope VERB eblip-19158 71 21 to to PART eblip-19158 71 22 provide provide VERB eblip-19158 71 23 a a DET eblip-19158 71 24 localized localized ADJ eblip-19158 71 25 case case NOUN eblip-19158 71 26 study study NOUN eblip-19158 71 27 of of ADP eblip-19158 71 28 the the DET eblip-19158 71 29 practicalities practicality NOUN eblip-19158 71 30 of of ADP eblip-19158 71 31 rubric rubric ADJ eblip-19158 71 32 usage usage NOUN eblip-19158 71 33 . . PUNCT eblip-19158 71 34     SPACE eblip-19158 71 35 methodology methodology NOUN eblip-19158 71 36     SPACE eblip-19158 71 37 developing develop VERB eblip-19158 71 38 rubrics rubric NOUN eblip-19158 71 39 for for ADP eblip-19158 71 40 information information NOUN eblip-19158 71 41 literacy literacy NOUN eblip-19158 71 42     SPACE eblip-19158 71 43 a a DET eblip-19158 71 44 rubric rubric NOUN eblip-19158 71 45 is be AUX eblip-19158 71 46 an an DET eblip-19158 71 47 assessment assessment NOUN eblip-19158 71 48 tool tool NOUN eblip-19158 71 49 that that PRON eblip-19158 71 50 establishes establish VERB eblip-19158 71 51 the the DET eblip-19158 71 52 criteria criterion NOUN eblip-19158 71 53 used use VERB eblip-19158 71 54 to to PART eblip-19158 71 55 judge judge VERB eblip-19158 71 56 a a DET eblip-19158 71 57 performance performance NOUN eblip-19158 71 58 , , PUNCT eblip-19158 71 59 the the DET eblip-19158 71 60 range range NOUN eblip-19158 71 61 in in ADP eblip-19158 71 62 quality quality NOUN eblip-19158 71 63 one one PRON eblip-19158 71 64 might might AUX eblip-19158 71 65 expect expect VERB eblip-19158 71 66 to to PART eblip-19158 71 67 see see VERB eblip-19158 71 68 for for ADP eblip-19158 71 69 a a DET eblip-19158 71 70 task task NOUN eblip-19158 71 71 , , PUNCT eblip-19158 71 72 what what DET eblip-19158 71 73 score score NOUN eblip-19158 71 74 should should AUX eblip-19158 71 75 be be AUX eblip-19158 71 76 given give VERB eblip-19158 71 77 , , PUNCT eblip-19158 71 78 and and CCONJ eblip-19158 71 79 what what PRON eblip-19158 71 80 that that DET eblip-19158 71 81 score score NOUN eblip-19158 71 82 means mean VERB eblip-19158 71 83 , , PUNCT eblip-19158 71 84 regardless regardless ADV eblip-19158 71 85 of of ADP eblip-19158 71 86 who who PRON eblip-19158 71 87 scores score VERB eblip-19158 71 88 the the DET eblip-19158 71 89 performance performance NOUN eblip-19158 71 90 or or CCONJ eblip-19158 71 91 when when SCONJ eblip-19158 71 92 that that DET eblip-19158 71 93 score score NOUN eblip-19158 71 94 is be AUX eblip-19158 71 95 given give VERB eblip-19158 71 96 ( ( PUNCT eblip-19158 71 97 callison callison PROPN eblip-19158 71 98 , , PUNCT eblip-19158 71 99 2000 2000 NUM eblip-19158 71 100 ; ; PUNCT eblip-19158 71 101 maki maki NOUN eblip-19158 71 102 , , PUNCT eblip-19158 71 103 2010 2010 NUM eblip-19158 71 104 ) ) PUNCT eblip-19158 71 105 . . PUNCT eblip-19158 72 1 a a DET eblip-19158 72 2 scoring scoring NOUN eblip-19158 72 3 rubric rubric ADJ eblip-19158 72 4 consists consist VERB eblip-19158 72 5 of of ADP eblip-19158 72 6 two two NUM eblip-19158 72 7 parts part NOUN eblip-19158 72 8 : : PUNCT eblip-19158 72 9 criteria criterion NOUN eblip-19158 72 10 , , PUNCT eblip-19158 72 11 which which PRON eblip-19158 72 12 describe describe VERB eblip-19158 72 13 the the DET eblip-19158 72 14 traits trait NOUN eblip-19158 72 15 that that PRON eblip-19158 72 16 will will AUX eblip-19158 72 17 be be AUX eblip-19158 72 18 evaluated evaluate VERB eblip-19158 72 19 by by ADP eblip-19158 72 20 the the DET eblip-19158 72 21 rubric rubric NOUN eblip-19158 72 22 , , PUNCT eblip-19158 72 23 and and CCONJ eblip-19158 72 24 performance performance NOUN eblip-19158 72 25 indicators indicator NOUN eblip-19158 72 26 , , PUNCT eblip-19158 72 27 which which PRON eblip-19158 72 28 describe describe VERB eblip-19158 72 29 the the DET eblip-19158 72 30 range range NOUN eblip-19158 72 31 of of ADP eblip-19158 72 32 possible possible ADJ eblip-19158 72 33 performances performance NOUN eblip-19158 72 34 “ " PUNCT eblip-19158 72 35 along along ADP eblip-19158 72 36 an an DET eblip-19158 72 37 achievement achievement NOUN eblip-19158 72 38 continuum continuum X eblip-19158 72 39 ” " PUNCT eblip-19158 72 40 ( ( PUNCT eblip-19158 72 41 maki maki NOUN eblip-19158 72 42 , , PUNCT eblip-19158 72 43 2010 2010 NUM eblip-19158 72 44 , , PUNCT eblip-19158 72 45 p. p. NOUN eblip-19158 72 46 219 219 NUM eblip-19158 72 47 ) ) PUNCT eblip-19158 72 48 . . PUNCT eblip-19158 72 49                                     SPACE eblip-19158 73 1 the the DET eblip-19158 73 2 benefits benefit NOUN eblip-19158 73 3 of of ADP eblip-19158 73 4 rubrics rubric NOUN eblip-19158 73 5 as as SCONJ eblip-19158 73 6 assessment assessment NOUN eblip-19158 73 7 tools tool NOUN eblip-19158 73 8 are be AUX eblip-19158 73 9 widely widely ADV eblip-19158 73 10 recognized recognize VERB eblip-19158 73 11 : : PUNCT eblip-19158 73 12 they they PRON eblip-19158 73 13 help help VERB eblip-19158 73 14 establish establish VERB eblip-19158 73 15 evaluation evaluation NOUN eblip-19158 73 16 standards standard NOUN eblip-19158 73 17 and and CCONJ eblip-19158 73 18 keep keep VERB eblip-19158 73 19 assessment assessment NOUN eblip-19158 73 20 consistent consistent ADJ eblip-19158 73 21 and and CCONJ eblip-19158 73 22 objective objective ADJ eblip-19158 73 23 ( ( PUNCT eblip-19158 73 24 huba huba PROPN eblip-19158 73 25 & & CCONJ eblip-19158 73 26 freed free VERB eblip-19158 73 27 2000 2000 NUM eblip-19158 73 28 ; ; PUNCT eblip-19158 73 29 callison callison PROPN eblip-19158 73 30 , , PUNCT eblip-19158 73 31 2000 2000 NUM eblip-19158 73 32 ) ) PUNCT eblip-19158 73 33 ; ; PUNCT eblip-19158 73 34 they they PRON eblip-19158 73 35 also also ADV eblip-19158 73 36 make make VERB eblip-19158 73 37 the the DET eblip-19158 73 38 evaluation evaluation NOUN eblip-19158 73 39 criteria criterion NOUN eblip-19158 73 40 explicit explicit ADJ eblip-19158 73 41 and and CCONJ eblip-19158 73 42 communicable communicable NOUN eblip-19158 73 43 to to ADP eblip-19158 73 44 other other ADJ eblip-19158 73 45 educators educator NOUN eblip-19158 73 46 , , PUNCT eblip-19158 73 47 stakeholders stakeholder NOUN eblip-19158 73 48 , , PUNCT eblip-19158 73 49 and and CCONJ eblip-19158 73 50 students student NOUN eblip-19158 73 51 ( ( PUNCT eblip-19158 73 52 montgomery montgomery PROPN eblip-19158 73 53 , , PUNCT eblip-19158 73 54 2002 2002 NUM eblip-19158 73 55 ) ) PUNCT eblip-19158 73 56 . . PUNCT eblip-19158 74 1 the the DET eblip-19158 74 2 most most ADV eblip-19158 74 3 commonly commonly ADV eblip-19158 74 4 cited cite VERB eblip-19158 74 5 disadvantage disadvantage NOUN eblip-19158 74 6 of of ADP eblip-19158 74 7 rubrics rubric NOUN eblip-19158 74 8 is be AUX eblip-19158 74 9 that that SCONJ eblip-19158 74 10 they they PRON eblip-19158 74 11 are be AUX eblip-19158 74 12 time time NOUN eblip-19158 74 13 consuming consume VERB eblip-19158 74 14 to to PART eblip-19158 74 15 develop develop VERB eblip-19158 74 16 and and CCONJ eblip-19158 74 17 apply apply VERB eblip-19158 74 18 ( ( PUNCT eblip-19158 74 19 callison callison PROPN eblip-19158 74 20 , , PUNCT eblip-19158 74 21 2000 2000 NUM eblip-19158 74 22 ; ; PUNCT eblip-19158 74 23 mertler mertler NOUN eblip-19158 74 24 , , PUNCT eblip-19158 74 25 2001 2001 NUM eblip-19158 74 26 ; ; PUNCT eblip-19158 74 27 montgomery montgomery PROPN eblip-19158 74 28 , , PUNCT eblip-19158 74 29 2002 2002 NUM eblip-19158 74 30 ) ) PUNCT eblip-19158 74 31 . . PUNCT eblip-19158 75 1 the the DET eblip-19158 75 2 advantages advantage NOUN eblip-19158 75 3 of of ADP eblip-19158 75 4 the the DET eblip-19158 75 5 descriptive descriptive ADJ eblip-19158 75 6 data datum NOUN eblip-19158 75 7 that that PRON eblip-19158 75 8 come come VERB eblip-19158 75 9 from from ADP eblip-19158 75 10 rubrics rubric NOUN eblip-19158 75 11 should should AUX eblip-19158 75 12 be be AUX eblip-19158 75 13 weighed weigh VERB eblip-19158 75 14 against against ADP eblip-19158 75 15 their their PRON eblip-19158 75 16 time time NOUN eblip-19158 75 17 - - PUNCT eblip-19158 75 18 consuming consume VERB eblip-19158 75 19 nature nature NOUN eblip-19158 75 20 , , PUNCT eblip-19158 75 21 and and CCONJ eblip-19158 75 22 proper proper ADJ eblip-19158 75 23 time time NOUN eblip-19158 75 24 should should AUX eblip-19158 75 25 be be AUX eblip-19158 75 26 allotted allot VERB eblip-19158 75 27 for for ADP eblip-19158 75 28 creating create VERB eblip-19158 75 29 , , PUNCT eblip-19158 75 30 teaching teaching NOUN eblip-19158 75 31 , , PUNCT eblip-19158 75 32 and and CCONJ eblip-19158 75 33 applying apply VERB eblip-19158 75 34 a a DET eblip-19158 75 35 rubric rubric NOUN eblip-19158 75 36 . . PUNCT eblip-19158 75 37                                 SPACE eblip-19158 76 1 there there PRON eblip-19158 76 2 is be VERB eblip-19158 76 3 much much ADJ eblip-19158 76 4 information information NOUN eblip-19158 76 5 in in ADP eblip-19158 76 6 the the DET eblip-19158 76 7 assessment assessment NOUN eblip-19158 76 8 literature literature NOUN eblip-19158 76 9 on on ADP eblip-19158 76 10 the the DET eblip-19158 76 11 general general ADJ eblip-19158 76 12 steps step NOUN eblip-19158 76 13 one one PRON eblip-19158 76 14 can can AUX eblip-19158 76 15 take take VERB eblip-19158 76 16 to to PART eblip-19158 76 17 develop develop VERB eblip-19158 76 18 a a DET eblip-19158 76 19 scoring scoring NOUN eblip-19158 76 20 rubric rubric NOUN eblip-19158 76 21 . . PUNCT eblip-19158 77 1 the the DET eblip-19158 77 2 model model NOUN eblip-19158 77 3 adapted adapt VERB eblip-19158 77 4 by by ADP eblip-19158 77 5 the the DET eblip-19158 77 6 author author NOUN eblip-19158 77 7 for for ADP eblip-19158 77 8 the the DET eblip-19158 77 9 study study NOUN eblip-19158 77 10 consists consist VERB eblip-19158 77 11 of of ADP eblip-19158 77 12 seven seven NUM eblip-19158 77 13 stages stage NOUN eblip-19158 77 14 and and CCONJ eblip-19158 77 15 was be AUX eblip-19158 77 16 developed develop VERB eblip-19158 77 17 by by ADP eblip-19158 77 18 mertler mertler NOUN eblip-19158 77 19 ( ( PUNCT eblip-19158 77 20 2001 2001 NUM eblip-19158 77 21 ) ) PUNCT eblip-19158 77 22 . . PUNCT eblip-19158 78 1 other other ADJ eblip-19158 78 2 examples example NOUN eblip-19158 78 3 of of ADP eblip-19158 78 4 rubric rubric ADJ eblip-19158 78 5 development development NOUN eblip-19158 78 6 models model NOUN eblip-19158 78 7 can can AUX eblip-19158 78 8 be be AUX eblip-19158 78 9 found find VERB eblip-19158 78 10 in in ADP eblip-19158 78 11 arter arter NOUN eblip-19158 78 12 and and CCONJ eblip-19158 78 13 mctighe mctighe NOUN eblip-19158 78 14 ( ( PUNCT eblip-19158 78 15 2001 2001 NUM eblip-19158 78 16 ) ) PUNCT eblip-19158 78 17 , , PUNCT eblip-19158 78 18 moskal moskal NOUN eblip-19158 78 19 ( ( PUNCT eblip-19158 78 20 2003 2003 NUM eblip-19158 78 21 ) ) PUNCT eblip-19158 78 22 , , PUNCT eblip-19158 78 23 stevens stevens PROPN eblip-19158 78 24 and and CCONJ eblip-19158 78 25 levi levi PROPN eblip-19158 78 26 ( ( PUNCT eblip-19158 78 27 2005 2005 NUM eblip-19158 78 28 ) ) PUNCT eblip-19158 78 29 , , PUNCT eblip-19158 78 30 and and CCONJ eblip-19158 78 31 maki maki NOUN eblip-19158 78 32 ( ( PUNCT eblip-19158 78 33 2010 2010 NUM eblip-19158 78 34 ) ) PUNCT eblip-19158 78 35 . . PUNCT eblip-19158 79 1 mertler mertler NOUN eblip-19158 79 2 ’s ’s PART eblip-19158 79 3 model model NOUN eblip-19158 79 4 ( ( PUNCT eblip-19158 79 5 mertler mertler NOUN eblip-19158 79 6 , , PUNCT eblip-19158 79 7 2001 2001 NUM eblip-19158 79 8 ) ) PUNCT eblip-19158 79 9 . . PUNCT eblip-19158 79 10                                 SPACE eblip-19158 80 1 1 1 X eblip-19158 80 2 . . X eblip-19158 80 3 reexamine reexamine NOUN eblip-19158 80 4 learning learn VERB eblip-19158 80 5 outcomes outcome NOUN eblip-19158 80 6 to to PART eblip-19158 80 7 be be AUX eblip-19158 80 8 addressed address VERB eblip-19158 80 9 . . PUNCT eblip-19158 81 1 2 2 X eblip-19158 81 2 . . X eblip-19158 81 3 identify identify VERB eblip-19158 81 4 specific specific ADJ eblip-19158 81 5 observable observable ADJ eblip-19158 81 6 attributes attribute NOUN eblip-19158 81 7 that that PRON eblip-19158 81 8 you you PRON eblip-19158 81 9 want want VERB eblip-19158 81 10 to to PART eblip-19158 81 11 see see VERB eblip-19158 81 12 or or CCONJ eblip-19158 81 13 do do AUX eblip-19158 81 14 not not PART eblip-19158 81 15 want want VERB eblip-19158 81 16 to to PART eblip-19158 81 17 see see VERB eblip-19158 81 18 students student NOUN eblip-19158 81 19 demonstrate demonstrate VERB eblip-19158 81 20 . . PUNCT eblip-19158 82 1 3 3 X eblip-19158 82 2 . . X eblip-19158 82 3 brainstorm brainstorm NOUN eblip-19158 82 4 characteristics characteristic NOUN eblip-19158 82 5 that that PRON eblip-19158 82 6 describe describe VERB eblip-19158 82 7 each each DET eblip-19158 82 8 attribute attribute NOUN eblip-19158 82 9 . . PUNCT eblip-19158 83 1 identify identify VERB eblip-19158 83 2 ways way NOUN eblip-19158 83 3 to to PART eblip-19158 83 4 describe describe VERB eblip-19158 83 5 above above ADP eblip-19158 83 6 average average ADJ eblip-19158 83 7 , , PUNCT eblip-19158 83 8 average average ADJ eblip-19158 83 9 , , PUNCT eblip-19158 83 10 and and CCONJ eblip-19158 83 11 below below ADP eblip-19158 83 12 average average ADJ eblip-19158 83 13 performance performance NOUN eblip-19158 83 14 for for ADP eblip-19158 83 15 each each DET eblip-19158 83 16 observable observable ADJ eblip-19158 83 17 attribute attribute NOUN eblip-19158 83 18 . . PUNCT eblip-19158 84 1 4 4 X eblip-19158 84 2 . . X eblip-19158 84 3 write write VERB eblip-19158 84 4 thorough thorough ADJ eblip-19158 84 5 narrative narrative NOUN eblip-19158 84 6 descriptions description NOUN eblip-19158 84 7 for for ADP eblip-19158 84 8 excellent excellent ADJ eblip-19158 84 9 and and CCONJ eblip-19158 84 10 poor poor ADJ eblip-19158 84 11 work work NOUN eblip-19158 84 12 for for ADP eblip-19158 84 13 each each DET eblip-19158 84 14 individual individual ADJ eblip-19158 84 15 attribute attribute NOUN eblip-19158 84 16 . . PUNCT eblip-19158 85 1 5 5 X eblip-19158 85 2 . . X eblip-19158 85 3 complete complete ADJ eblip-19158 85 4 rubric rubric NOUN eblip-19158 85 5 by by ADP eblip-19158 85 6 describing describe VERB eblip-19158 85 7 other other ADJ eblip-19158 85 8 levels level NOUN eblip-19158 85 9 on on ADP eblip-19158 85 10 continuum continuum PROPN eblip-19158 85 11 . . PUNCT eblip-19158 85 12                                 SPACE eblip-19158 86 1 6 6 NUM eblip-19158 86 2 . . PUNCT eblip-19158 86 3 collect collect VERB eblip-19158 86 4 student student NOUN eblip-19158 86 5 work work NOUN eblip-19158 86 6 samples sample NOUN eblip-19158 86 7 for for ADP eblip-19158 86 8 each each DET eblip-19158 86 9 level level NOUN eblip-19158 86 10 . . PUNCT eblip-19158 86 11                                 SPACE eblip-19158 87 1 7 7 X eblip-19158 87 2 . . X eblip-19158 87 3 revise revise VERB eblip-19158 87 4 and and CCONJ eblip-19158 87 5 reflect reflect VERB eblip-19158 87 6 . . PUNCT eblip-19158 87 7     SPACE eblip-19158 88 1 in in ADP eblip-19158 88 2 accordance accordance NOUN eblip-19158 88 3 with with ADP eblip-19158 88 4 mertler mertler NOUN eblip-19158 88 5 ’s ’s PART eblip-19158 88 6 model model NOUN eblip-19158 88 7 , , PUNCT eblip-19158 88 8 the the DET eblip-19158 88 9 author author NOUN eblip-19158 88 10 began begin VERB eblip-19158 88 11 the the DET eblip-19158 88 12 process process NOUN eblip-19158 88 13 of of ADP eblip-19158 88 14 designing design VERB eblip-19158 88 15 the the DET eblip-19158 88 16 eng eng PROPN eblip-19158 88 17 102 102 NUM eblip-19158 88 18 information information NOUN eblip-19158 88 19 literacy literacy NOUN eblip-19158 88 20 rubric rubric NOUN eblip-19158 88 21 by by ADP eblip-19158 88 22 defining define VERB eblip-19158 88 23 the the DET eblip-19158 88 24 learning learn VERB eblip-19158 88 25 outcomes outcome NOUN eblip-19158 88 26 that that PRON eblip-19158 88 27 needed need VERB eblip-19158 88 28 to to PART eblip-19158 88 29 be be AUX eblip-19158 88 30 addressed address VERB eblip-19158 88 31 . . PUNCT eblip-19158 89 1 the the DET eblip-19158 89 2 learning learn VERB eblip-19158 89 3 outcomes outcome NOUN eblip-19158 89 4 that that SCONJ eblip-19158 89 5 the the DET eblip-19158 89 6 composition composition NOUN eblip-19158 89 7 program program NOUN eblip-19158 89 8 identified identify VERB eblip-19158 89 9 for for ADP eblip-19158 89 10 the the DET eblip-19158 89 11 annotated annotate VERB eblip-19158 89 12 bibliography bibliography NOUN eblip-19158 89 13 assignment assignment NOUN eblip-19158 89 14 were be AUX eblip-19158 89 15 used use VERB eblip-19158 89 16 as as ADP eblip-19158 89 17 a a DET eblip-19158 89 18 starting starting NOUN eblip-19158 89 19 point point NOUN eblip-19158 89 20 for for ADP eblip-19158 89 21 developing develop VERB eblip-19158 89 22 the the DET eblip-19158 89 23 rubric rubric ADJ eblip-19158 89 24 criteria criterion NOUN eblip-19158 89 25 . . PUNCT eblip-19158 90 1 the the DET eblip-19158 90 2 annotated annotate VERB eblip-19158 90 3 bibliography bibliography NOUN eblip-19158 90 4 assignment assignment NOUN eblip-19158 90 5 has have VERB eblip-19158 90 6 six six NUM eblip-19158 90 7 information information NOUN eblip-19158 90 8 - - PUNCT eblip-19158 90 9 literacy literacy NOUN eblip-19158 90 10 - - PUNCT eblip-19158 90 11 centered center VERB eblip-19158 90 12 learning learning NOUN eblip-19158 90 13 outcomes outcome NOUN eblip-19158 90 14 , , PUNCT eblip-19158 90 15 including include VERB eblip-19158 90 16 choosing choose VERB eblip-19158 90 17 and and CCONJ eblip-19158 90 18 narrowing narrow VERB eblip-19158 90 19 a a DET eblip-19158 90 20 research research NOUN eblip-19158 90 21 topic topic NOUN eblip-19158 90 22 , , PUNCT eblip-19158 90 23 designing design VERB eblip-19158 90 24 search search NOUN eblip-19158 90 25 strategies strategy NOUN eblip-19158 90 26 , , PUNCT eblip-19158 90 27 conducting conduct VERB eblip-19158 90 28 academic academic ADJ eblip-19158 90 29 research research NOUN eblip-19158 90 30 , , PUNCT eblip-19158 90 31 evaluating evaluate VERB eblip-19158 90 32 sources source NOUN eblip-19158 90 33 , , PUNCT eblip-19158 90 34 writing write VERB eblip-19158 90 35 citations citation NOUN eblip-19158 90 36 , , PUNCT eblip-19158 90 37 and and CCONJ eblip-19158 90 38 planning plan VERB eblip-19158 90 39 a a DET eblip-19158 90 40 research research NOUN eblip-19158 90 41 - - PUNCT eblip-19158 90 42 based base VERB eblip-19158 90 43 argument argument NOUN eblip-19158 90 44 essay essay NOUN eblip-19158 90 45 . . PUNCT eblip-19158 91 1 many many ADJ eblip-19158 91 2 of of ADP eblip-19158 91 3 these these DET eblip-19158 91 4 outcomes outcome NOUN eblip-19158 91 5 require require VERB eblip-19158 91 6 students student NOUN eblip-19158 91 7 to to PART eblip-19158 91 8 use use VERB eblip-19158 91 9 higher high ADJ eblip-19158 91 10 - - PUNCT eblip-19158 91 11 order order NOUN eblip-19158 91 12 critical critical ADJ eblip-19158 91 13 thinking thinking NOUN eblip-19158 91 14 skills skill NOUN eblip-19158 91 15 , , PUNCT eblip-19158 91 16 which which PRON eblip-19158 91 17 were be AUX eblip-19158 91 18 identified identify VERB eblip-19158 91 19 as as ADP eblip-19158 91 20 areas area NOUN eblip-19158 91 21 of of ADP eblip-19158 91 22 difficulty difficulty NOUN eblip-19158 91 23 in in ADP eblip-19158 91 24 previous previous ADJ eblip-19158 91 25 studies study NOUN eblip-19158 91 26 that that PRON eblip-19158 91 27 used use VERB eblip-19158 91 28 information information NOUN eblip-19158 91 29 literacy literacy NOUN eblip-19158 91 30 rubrics rubric NOUN eblip-19158 91 31 , , PUNCT eblip-19158 91 32 so so SCONJ eblip-19158 91 33 the the DET eblip-19158 91 34 author author NOUN eblip-19158 91 35 was be AUX eblip-19158 91 36 particularly particularly ADV eblip-19158 91 37 interested interested ADJ eblip-19158 91 38 in in ADP eblip-19158 91 39 assessing assess VERB eblip-19158 91 40 those those DET eblip-19158 91 41 areas area NOUN eblip-19158 91 42 . . PUNCT eblip-19158 92 1 the the DET eblip-19158 92 2 author author NOUN eblip-19158 92 3 then then ADV eblip-19158 92 4 mapped map VERB eblip-19158 92 5 each each PRON eblip-19158 92 6 of of ADP eblip-19158 92 7 the the DET eblip-19158 92 8 six six NUM eblip-19158 92 9 outcomes outcome NOUN eblip-19158 92 10 to to ADP eblip-19158 92 11 the the DET eblip-19158 92 12 association association NOUN eblip-19158 92 13 of of ADP eblip-19158 92 14 college college PROPN eblip-19158 92 15 and and CCONJ eblip-19158 92 16 research research NOUN eblip-19158 92 17 libraries library NOUN eblip-19158 92 18 information information NOUN eblip-19158 92 19 literacy literacy NOUN eblip-19158 92 20 competency competency NOUN eblip-19158 92 21 standards standard NOUN eblip-19158 92 22 for for ADP eblip-19158 92 23 higher high ADJ eblip-19158 92 24 education education NOUN eblip-19158 92 25 and and CCONJ eblip-19158 92 26 used use VERB eblip-19158 92 27 a a DET eblip-19158 92 28 set set NOUN eblip-19158 92 29 of of ADP eblip-19158 92 30 sample sample NOUN eblip-19158 92 31 annotated annotate VERB eblip-19158 92 32 bibliographies bibliography NOUN eblip-19158 92 33 from from ADP eblip-19158 92 34 a a DET eblip-19158 92 35 previous previous ADJ eblip-19158 92 36 semester semester NOUN eblip-19158 92 37 to to PART eblip-19158 92 38 identify identify VERB eblip-19158 92 39 attributes attribute NOUN eblip-19158 92 40 in in ADP eblip-19158 92 41 student student NOUN eblip-19158 92 42 work work NOUN eblip-19158 92 43 that that PRON eblip-19158 92 44 represented represent VERB eblip-19158 92 45 a a DET eblip-19158 92 46 range range NOUN eblip-19158 92 47 of of ADP eblip-19158 92 48 good good ADJ eblip-19158 92 49 and and CCONJ eblip-19158 92 50 poor poor ADJ eblip-19158 92 51 performances performance NOUN eblip-19158 92 52 for for ADP eblip-19158 92 53 each each PRON eblip-19158 92 54 of of ADP eblip-19158 92 55 the the DET eblip-19158 92 56 six six NUM eblip-19158 92 57 criteria criterion NOUN eblip-19158 92 58 ( ( PUNCT eblip-19158 92 59 acrl acrl NOUN eblip-19158 92 60 , , PUNCT eblip-19158 92 61 2000 2000 NUM eblip-19158 92 62 ) ) PUNCT eblip-19158 92 63 . . PUNCT eblip-19158 93 1 next next ADV eblip-19158 93 2 , , PUNCT eblip-19158 93 3 the the DET eblip-19158 93 4 author author NOUN eblip-19158 93 5 created create VERB eblip-19158 93 6 written write VERB eblip-19158 93 7 descriptions description NOUN eblip-19158 93 8 of of ADP eblip-19158 93 9 the the DET eblip-19158 93 10 aspects aspect NOUN eblip-19158 93 11 of of ADP eblip-19158 93 12 performances performance NOUN eblip-19158 93 13 that that PRON eblip-19158 93 14 qualified qualify VERB eblip-19158 93 15 them they PRON eblip-19158 93 16 as as ADP eblip-19158 93 17 good good ADJ eblip-19158 93 18 or or CCONJ eblip-19158 93 19 poor poor ADJ eblip-19158 93 20 , , PUNCT eblip-19158 93 21 and and CCONJ eblip-19158 93 22 filled fill VERB eblip-19158 93 23 in in ADP eblip-19158 93 24 the the DET eblip-19158 93 25 rubric rubric NOUN eblip-19158 93 26 with with ADP eblip-19158 93 27 descriptions description NOUN eblip-19158 93 28 of of ADP eblip-19158 93 29 “ " PUNCT eblip-19158 93 30 middle middle ADJ eblip-19158 93 31 - - PUNCT eblip-19158 93 32 range range NOUN eblip-19158 93 33 ” " PUNCT eblip-19158 93 34 performances performance NOUN eblip-19158 93 35 . . PUNCT eblip-19158 94 1 this this DET eblip-19158 94 2 first first ADJ eblip-19158 94 3 draft draft NOUN eblip-19158 94 4 resulted result VERB eblip-19158 94 5 in in ADP eblip-19158 94 6 three three NUM eblip-19158 94 7 rubrics rubric NOUN eblip-19158 94 8 that that PRON eblip-19158 94 9 were be AUX eblip-19158 94 10 shared share VERB eblip-19158 94 11 with with ADP eblip-19158 94 12 other other ADJ eblip-19158 94 13 instruction instruction NOUN eblip-19158 94 14 librarians librarian NOUN eblip-19158 94 15 and and CCONJ eblip-19158 94 16 the the DET eblip-19158 94 17 eng eng PROPN eblip-19158 94 18 102 102 NUM eblip-19158 94 19 coordinator coordinator NOUN eblip-19158 94 20 during during ADP eblip-19158 94 21 a a DET eblip-19158 94 22 rubric rubric ADJ eblip-19158 94 23 workshop workshop NOUN eblip-19158 94 24 led lead VERB eblip-19158 94 25 by by ADP eblip-19158 94 26 an an DET eblip-19158 94 27 expert expert NOUN eblip-19158 94 28 in in ADP eblip-19158 94 29 the the DET eblip-19158 94 30 field field NOUN eblip-19158 94 31 who who PRON eblip-19158 94 32 came come VERB eblip-19158 94 33 to to ADP eblip-19158 94 34 unlv unlv VERB eblip-19158 94 35 ’s ’s PART eblip-19158 94 36 campus campus NOUN eblip-19158 94 37 to to PART eblip-19158 94 38 help help VERB eblip-19158 94 39 support support VERB eblip-19158 94 40 library library ADJ eblip-19158 94 41 assessment assessment NOUN eblip-19158 94 42 efforts effort NOUN eblip-19158 94 43 . . PUNCT eblip-19158 94 44     SPACE eblip-19158 95 1 the the DET eblip-19158 95 2 discussions discussion NOUN eblip-19158 95 3 during during ADP eblip-19158 95 4 the the DET eblip-19158 95 5 workshop workshop NOUN eblip-19158 95 6 led lead VERB eblip-19158 95 7 to to ADP eblip-19158 95 8 substantial substantial ADJ eblip-19158 95 9 revision revision NOUN eblip-19158 95 10 of of ADP eblip-19158 95 11 the the DET eblip-19158 95 12 rubrics rubric NOUN eblip-19158 95 13 ’ ' PUNCT eblip-19158 95 14 content content NOUN eblip-19158 95 15 and and CCONJ eblip-19158 95 16 format format NOUN eblip-19158 95 17 . . PUNCT eblip-19158 96 1 language language NOUN eblip-19158 96 2 was be AUX eblip-19158 96 3 standardized standardize VERB eblip-19158 96 4 and and CCONJ eblip-19158 96 5 clarified clarify VERB eblip-19158 96 6 , , PUNCT eblip-19158 96 7 with with ADP eblip-19158 96 8 careful careful ADJ eblip-19158 96 9 attention attention NOUN eblip-19158 96 10 paid pay VERB eblip-19158 96 11 to to ADP eblip-19158 96 12 using use VERB eblip-19158 96 13 parallel parallel ADJ eblip-19158 96 14 structure structure NOUN eblip-19158 96 15 . . PUNCT eblip-19158 97 1 in in ADP eblip-19158 97 2 addition addition NOUN eblip-19158 97 3 , , PUNCT eblip-19158 97 4 efforts effort NOUN eblip-19158 97 5 were be AUX eblip-19158 97 6 made make VERB eblip-19158 97 7 to to PART eblip-19158 97 8 ensure ensure VERB eblip-19158 97 9 that that SCONJ eblip-19158 97 10 only only ADV eblip-19158 97 11 one one NUM eblip-19158 97 12 element element NOUN eblip-19158 97 13 was be AUX eblip-19158 97 14 assessed assess VERB eblip-19158 97 15 in in ADP eblip-19158 97 16 each each DET eblip-19158 97 17 criterion criterion NOUN eblip-19158 97 18 and and CCONJ eblip-19158 97 19 that that SCONJ eblip-19158 97 20 the the DET eblip-19158 97 21 performance performance NOUN eblip-19158 97 22 indicators indicator NOUN eblip-19158 97 23 on on ADP eblip-19158 97 24 the the DET eblip-19158 97 25 rubric rubric NOUN eblip-19158 97 26 were be AUX eblip-19158 97 27 mutually mutually ADV eblip-19158 97 28 exclusive exclusive ADJ eblip-19158 97 29 . . PUNCT eblip-19158 98 1 maki maki NOUN eblip-19158 98 2 ’s ’s PART eblip-19158 98 3 checklist checklist NOUN eblip-19158 98 4 for for ADP eblip-19158 98 5 evaluating evaluate VERB eblip-19158 98 6 a a DET eblip-19158 98 7 rubric rubric NOUN eblip-19158 98 8 proved prove VERB eblip-19158 98 9 to to PART eblip-19158 98 10 be be AUX eblip-19158 98 11 a a DET eblip-19158 98 12 useful useful ADJ eblip-19158 98 13 tool tool NOUN eblip-19158 98 14 for for ADP eblip-19158 98 15 identifying identify VERB eblip-19158 98 16 areas area NOUN eblip-19158 98 17 of of ADP eblip-19158 98 18 ambiguity ambiguity NOUN eblip-19158 98 19 and and CCONJ eblip-19158 98 20 overlap overlap NOUN eblip-19158 98 21 ( ( PUNCT eblip-19158 98 22 maki maki NOUN eblip-19158 98 23 , , PUNCT eblip-19158 98 24 2010 2010 NUM eblip-19158 98 25 ) ) PUNCT eblip-19158 98 26 . . PUNCT eblip-19158 99 1 the the DET eblip-19158 99 2 author author NOUN eblip-19158 99 3 also also ADV eblip-19158 99 4 refocused refocus VERB eblip-19158 99 5 the the DET eblip-19158 99 6 scope scope NOUN eblip-19158 99 7 of of ADP eblip-19158 99 8 the the DET eblip-19158 99 9 project project NOUN eblip-19158 99 10 , , PUNCT eblip-19158 99 11 which which PRON eblip-19158 99 12 was be AUX eblip-19158 99 13 too too ADV eblip-19158 99 14 broad broad ADJ eblip-19158 99 15 for for ADP eblip-19158 99 16 the the DET eblip-19158 99 17 first first ADJ eblip-19158 99 18 stage stage NOUN eblip-19158 99 19 of of ADP eblip-19158 99 20 the the DET eblip-19158 99 21 assessment assessment NOUN eblip-19158 99 22 project project NOUN eblip-19158 99 23 . . PUNCT eblip-19158 100 1 instead instead ADV eblip-19158 100 2 of of ADP eblip-19158 100 3 addressing address VERB eblip-19158 100 4 all all DET eblip-19158 100 5 six six NUM eblip-19158 100 6 information information NOUN eblip-19158 100 7 literacy literacy NOUN eblip-19158 100 8 learning learn VERB eblip-19158 100 9 outcomes outcome NOUN eblip-19158 100 10 identified identify VERB eblip-19158 100 11 for for ADP eblip-19158 100 12 eng eng PROPN eblip-19158 100 13 102 102 NUM eblip-19158 100 14 , , PUNCT eblip-19158 100 15 the the DET eblip-19158 100 16 author author NOUN eblip-19158 100 17 decided decide VERB eblip-19158 100 18 to to PART eblip-19158 100 19 start start VERB eblip-19158 100 20 with with ADP eblip-19158 100 21 just just ADV eblip-19158 100 22 one one NUM eblip-19158 100 23 outcome outcome NOUN eblip-19158 100 24 : : PUNCT eblip-19158 100 25 source source NOUN eblip-19158 100 26 evaluation evaluation NOUN eblip-19158 100 27 . . PUNCT eblip-19158 100 28         SPACE eblip-19158 100 29 for for ADP eblip-19158 100 30 the the DET eblip-19158 100 31 source source NOUN eblip-19158 100 32 evaluation evaluation NOUN eblip-19158 100 33 rubric rubric NOUN eblip-19158 100 34 , , PUNCT eblip-19158 100 35 five five NUM eblip-19158 100 36 criteria criterion NOUN eblip-19158 100 37 were be AUX eblip-19158 100 38 selected select VERB eblip-19158 100 39 to to PART eblip-19158 100 40 assess assess VERB eblip-19158 100 41 : : PUNCT eblip-19158 100 42 currency currency NOUN eblip-19158 100 43 , , PUNCT eblip-19158 100 44 relevance relevance NOUN eblip-19158 100 45 , , PUNCT eblip-19158 100 46 accuracy accuracy NOUN eblip-19158 100 47 , , PUNCT eblip-19158 100 48 authority authority NOUN eblip-19158 100 49 , , PUNCT eblip-19158 100 50 and and CCONJ eblip-19158 100 51 purpose purpose NOUN eblip-19158 100 52 . . PUNCT eblip-19158 101 1 these these DET eblip-19158 101 2 criteria criterion NOUN eblip-19158 101 3 were be AUX eblip-19158 101 4 drawn draw VERB eblip-19158 101 5 from from ADP eblip-19158 101 6 a a DET eblip-19158 101 7 unlv unlv NOUN eblip-19158 101 8 libraries library NOUN eblip-19158 101 9 ’ ' PUNCT eblip-19158 101 10 handout handout NOUN eblip-19158 101 11 that that PRON eblip-19158 101 12 aids aid VERB eblip-19158 101 13 students student NOUN eblip-19158 101 14 in in ADP eblip-19158 101 15 evaluating evaluate VERB eblip-19158 101 16 the the DET eblip-19158 101 17 credibility credibility NOUN eblip-19158 101 18 of of ADP eblip-19158 101 19 a a DET eblip-19158 101 20 resource resource NOUN eblip-19158 101 21 and and CCONJ eblip-19158 101 22 walks walk VERB eblip-19158 101 23 them they PRON eblip-19158 101 24 through through ADP eblip-19158 101 25 how how SCONJ eblip-19158 101 26 to to PART eblip-19158 101 27 decide decide VERB eblip-19158 101 28 if if SCONJ eblip-19158 101 29 a a DET eblip-19158 101 30 source source NOUN eblip-19158 101 31 is be AUX eblip-19158 101 32 useful useful ADJ eblip-19158 101 33 for for ADP eblip-19158 101 34 their their PRON eblip-19158 101 35 project project NOUN eblip-19158 101 36 . . PUNCT eblip-19158 102 1 the the DET eblip-19158 102 2 rubric rubric NOUN eblip-19158 102 3 had have VERB eblip-19158 102 4 three three NUM eblip-19158 102 5 performance performance NOUN eblip-19158 102 6 indicators indicator NOUN eblip-19158 102 7 to to PART eblip-19158 102 8 represent represent VERB eblip-19158 102 9 the the DET eblip-19158 102 10 range range NOUN eblip-19158 102 11 of of ADP eblip-19158 102 12 student student NOUN eblip-19158 102 13 work work NOUN eblip-19158 102 14 in in ADP eblip-19158 102 15 terms term NOUN eblip-19158 102 16 of of ADP eblip-19158 102 17 how how SCONJ eblip-19158 102 18 the the DET eblip-19158 102 19 student student NOUN eblip-19158 102 20 applied apply VERB eblip-19158 102 21 the the DET eblip-19158 102 22 evaluative evaluative ADJ eblip-19158 102 23 criteria criterion NOUN eblip-19158 102 24 : : PUNCT eblip-19158 102 25 “ " PUNCT eblip-19158 102 26 level level NOUN eblip-19158 102 27 0 0 NUM eblip-19158 102 28 — — PUNCT eblip-19158 102 29 not not PART eblip-19158 102 30 evidenced evidence VERB eblip-19158 102 31 , , PUNCT eblip-19158 102 32 ” " PUNCT eblip-19158 102 33 “ " PUNCT eblip-19158 102 34 level level NOUN eblip-19158 102 35 1 1 NUM eblip-19158 102 36 — — PUNCT eblip-19158 102 37 developing develop VERB eblip-19158 102 38 ( ( PUNCT eblip-19158 102 39 using use VERB eblip-19158 102 40 evaluation evaluation NOUN eblip-19158 102 41 criteria criterion NOUN eblip-19158 102 42 at at ADP eblip-19158 102 43 face face NOUN eblip-19158 102 44 value value NOUN eblip-19158 102 45 ) ) PUNCT eblip-19158 102 46 , , PUNCT eblip-19158 102 47 ” " PUNCT eblip-19158 102 48 and and CCONJ eblip-19158 102 49 “ " PUNCT eblip-19158 102 50 level level NOUN eblip-19158 102 51 2 2 NUM eblip-19158 102 52 — — PUNCT eblip-19158 102 53 competent competent ADJ eblip-19158 102 54 ( ( PUNCT eblip-19158 102 55 using use VERB eblip-19158 102 56 evaluation evaluation NOUN eblip-19158 102 57 criteria criterion NOUN eblip-19158 102 58 critically critically ADV eblip-19158 102 59 ) ) PUNCT eblip-19158 102 60 ” " PUNCT eblip-19158 102 61 ( ( PUNCT eblip-19158 102 62 see see VERB eblip-19158 102 63 figure figure NOUN eblip-19158 102 64 1 1 NUM eblip-19158 102 65 ) ) PUNCT eblip-19158 102 66 . . PUNCT eblip-19158 102 67     SPACE eblip-19158 103 1 the the DET eblip-19158 103 2 goal goal NOUN eblip-19158 103 3 of of ADP eblip-19158 103 4 the the DET eblip-19158 103 5 rubric rubric NOUN eblip-19158 103 6 was be AUX eblip-19158 103 7 to to PART eblip-19158 103 8 identify identify VERB eblip-19158 103 9 which which PRON eblip-19158 103 10 evaluative evaluative ADJ eblip-19158 103 11 criteria criterion NOUN eblip-19158 103 12 students student NOUN eblip-19158 103 13 were be AUX eblip-19158 103 14 not not PART eblip-19158 103 15 using use VERB eblip-19158 103 16 at at ADV eblip-19158 103 17 all all ADV eblip-19158 103 18 , , PUNCT eblip-19158 103 19 which which PRON eblip-19158 103 20 they they PRON eblip-19158 103 21 were be AUX eblip-19158 103 22 using use VERB eblip-19158 103 23 in in ADP eblip-19158 103 24 only only ADV eblip-19158 103 25 a a DET eblip-19158 103 26 shallow shallow ADJ eblip-19158 103 27 way way NOUN eblip-19158 103 28 , , PUNCT eblip-19158 103 29 and and CCONJ eblip-19158 103 30 which which PRON eblip-19158 103 31 criteria criterion NOUN eblip-19158 103 32 students student NOUN eblip-19158 103 33 were be AUX eblip-19158 103 34 using use VERB eblip-19158 103 35 as as ADP eblip-19158 103 36 critical critical ADJ eblip-19158 103 37 consumers consumer NOUN eblip-19158 103 38 of of ADP eblip-19158 103 39 information information NOUN eblip-19158 103 40 . . PUNCT eblip-19158 103 41     SPACE eblip-19158 104 1 in in ADP eblip-19158 104 2 order order NOUN eblip-19158 104 3 to to PART eblip-19158 104 4 gather gather VERB eblip-19158 104 5 this this DET eblip-19158 104 6 level level NOUN eblip-19158 104 7 of of ADP eblip-19158 104 8 detail detail NOUN eblip-19158 104 9 , , PUNCT eblip-19158 104 10 the the DET eblip-19158 104 11 author author NOUN eblip-19158 104 12 decided decide VERB eblip-19158 104 13 that that SCONJ eblip-19158 104 14 the the DET eblip-19158 104 15 rubric rubric NOUN eblip-19158 104 16 would would AUX eblip-19158 104 17 be be AUX eblip-19158 104 18 used use VERB eblip-19158 104 19 to to PART eblip-19158 104 20 evaluate evaluate VERB eblip-19158 104 21 the the DET eblip-19158 104 22 individual individual ADJ eblip-19158 104 23 annotations annotation NOUN eblip-19158 104 24 in in ADP eblip-19158 104 25 each each DET eblip-19158 104 26 bibliography bibliography NOUN eblip-19158 104 27 , , PUNCT eblip-19158 104 28 not not PART eblip-19158 104 29 the the DET eblip-19158 104 30 bibliography bibliography NOUN eblip-19158 104 31 as as ADP eblip-19158 104 32 a a DET eblip-19158 104 33 whole whole NOUN eblip-19158 104 34 . . PUNCT eblip-19158 104 35     SPACE eblip-19158 105 1 this this PRON eblip-19158 105 2 meant mean VERB eblip-19158 105 3 that that SCONJ eblip-19158 105 4 the the DET eblip-19158 105 5 rubric rubric NOUN eblip-19158 105 6 would would AUX eblip-19158 105 7 be be AUX eblip-19158 105 8 applied apply VERB eblip-19158 105 9 up up ADP eblip-19158 105 10 to to PART eblip-19158 105 11 five five NUM eblip-19158 105 12 times time NOUN eblip-19158 105 13 for for ADP eblip-19158 105 14 each each DET eblip-19158 105 15 student student NOUN eblip-19158 105 16 ’s ’s PART eblip-19158 105 17 paper paper NOUN eblip-19158 105 18 , , PUNCT eblip-19158 105 19 since since SCONJ eblip-19158 105 20 students student NOUN eblip-19158 105 21 were be AUX eblip-19158 105 22 to to PART eblip-19158 105 23 turn turn VERB eblip-19158 105 24 in in ADP eblip-19158 105 25 at at ADV eblip-19158 105 26 least least ADJ eblip-19158 105 27 five five NUM eblip-19158 105 28 annotations annotation NOUN eblip-19158 105 29 . . PUNCT eblip-19158 105 30               SPACE eblip-19158 105 31 figure figure NOUN eblip-19158 105 32 1 1 NUM eblip-19158 105 33 source source NOUN eblip-19158 105 34 evaluation evaluation NOUN eblip-19158 105 35 rubric rubric NOUN eblip-19158 105 36         SPACE eblip-19158 105 37 applying apply VERB eblip-19158 105 38 the the DET eblip-19158 105 39 rubrics rubric NOUN eblip-19158 105 40                                 SPACE eblip-19158 105 41 collecting collect VERB eblip-19158 105 42 and and CCONJ eblip-19158 105 43 preparing prepare VERB eblip-19158 105 44 student student NOUN eblip-19158 105 45 samples sample NOUN eblip-19158 105 46         SPACE eblip-19158 105 47 the the DET eblip-19158 105 48 eng eng PROPN eblip-19158 105 49 102 102 NUM eblip-19158 105 50 coordinator coordinator NOUN eblip-19158 105 51 , , PUNCT eblip-19158 105 52 a a DET eblip-19158 105 53 faculty faculty NOUN eblip-19158 105 54 member member NOUN eblip-19158 105 55 in in ADP eblip-19158 105 56 the the DET eblip-19158 105 57 composition composition NOUN eblip-19158 105 58 department department NOUN eblip-19158 105 59 , , PUNCT eblip-19158 105 60 had have AUX eblip-19158 105 61 already already ADV eblip-19158 105 62 established establish VERB eblip-19158 105 63 a a DET eblip-19158 105 64 method method NOUN eblip-19158 105 65 for for ADP eblip-19158 105 66 collecting collect VERB eblip-19158 105 67 a a DET eblip-19158 105 68 sampling sampling NOUN eblip-19158 105 69 of of ADP eblip-19158 105 70 student student NOUN eblip-19158 105 71 work work NOUN eblip-19158 105 72 every every DET eblip-19158 105 73 semester semester NOUN eblip-19158 105 74 , , PUNCT eblip-19158 105 75 so so ADV eblip-19158 105 76 the the DET eblip-19158 105 77 author author NOUN eblip-19158 105 78 was be AUX eblip-19158 105 79 able able ADJ eblip-19158 105 80 to to PART eblip-19158 105 81 receive receive VERB eblip-19158 105 82 copies copy NOUN eblip-19158 105 83 of of ADP eblip-19158 105 84 the the DET eblip-19158 105 85 annotated annotate VERB eblip-19158 105 86 bibliography bibliography NOUN eblip-19158 105 87 assignment assignment NOUN eblip-19158 105 88 from from ADP eblip-19158 105 89 this this DET eblip-19158 105 90 sampling sampling NOUN eblip-19158 105 91 . . PUNCT eblip-19158 106 1 the the DET eblip-19158 106 2 eng eng PROPN eblip-19158 106 3 102 102 NUM eblip-19158 106 4 coordinator coordinator NOUN eblip-19158 106 5 uses use VERB eblip-19158 106 6 a a DET eblip-19158 106 7 form form NOUN eblip-19158 106 8 of of ADP eblip-19158 106 9 systematic systematic ADJ eblip-19158 106 10 sampling sampling NOUN eblip-19158 106 11 where where SCONJ eblip-19158 106 12 the the DET eblip-19158 106 13 work work NOUN eblip-19158 106 14 from from ADP eblip-19158 106 15 every every DET eblip-19158 106 16 5th 5th ADJ eblip-19158 106 17 , , PUNCT eblip-19158 106 18 10th 10th ADJ eblip-19158 106 19 , , PUNCT eblip-19158 106 20 and and CCONJ eblip-19158 106 21 15th 15th ADJ eblip-19158 106 22 student student NOUN eblip-19158 106 23 in in ADP eblip-19158 106 24 each each DET eblip-19158 106 25 section section NOUN eblip-19158 106 26 is be AUX eblip-19158 106 27 collected collect VERB eblip-19158 106 28 ( ( PUNCT eblip-19158 106 29 creswell creswell PROPN eblip-19158 106 30 , , PUNCT eblip-19158 106 31 2005 2005 NUM eblip-19158 106 32 ) ) PUNCT eblip-19158 106 33 . . PUNCT eblip-19158 107 1 this this PRON eblip-19158 107 2 means mean VERB eblip-19158 107 3 that that SCONJ eblip-19158 107 4 at at ADV eblip-19158 107 5 least least ADV eblip-19158 107 6 three three NUM eblip-19158 107 7 papers paper NOUN eblip-19158 107 8 were be AUX eblip-19158 107 9 to to PART eblip-19158 107 10 be be AUX eblip-19158 107 11 collected collect VERB eblip-19158 107 12 from from ADP eblip-19158 107 13 each each PRON eblip-19158 107 14 of of ADP eblip-19158 107 15 the the DET eblip-19158 107 16 58 58 NUM eblip-19158 107 17 sections section NOUN eblip-19158 107 18 . . PUNCT eblip-19158 108 1 in in ADP eblip-19158 108 2 all all PRON eblip-19158 108 3 , , PUNCT eblip-19158 108 4 the the DET eblip-19158 108 5 author author NOUN eblip-19158 108 6 received receive VERB eblip-19158 108 7 a a DET eblip-19158 108 8 total total NOUN eblip-19158 108 9 of of ADP eblip-19158 108 10 155 155 NUM eblip-19158 108 11 annotated annotate VERB eblip-19158 108 12 bibliographies bibliography NOUN eblip-19158 108 13 , , PUNCT eblip-19158 108 14 representing represent VERB eblip-19158 108 15 10 10 NUM eblip-19158 108 16 % % NOUN eblip-19158 108 17 of of ADP eblip-19158 108 18 the the DET eblip-19158 108 19 total total ADJ eblip-19158 108 20 eng eng PROPN eblip-19158 108 21 102 102 NUM eblip-19158 108 22 student student NOUN eblip-19158 108 23 population population NOUN eblip-19158 108 24 ( ( PUNCT eblip-19158 108 25 not not PART eblip-19158 108 26 every every DET eblip-19158 108 27 section section NOUN eblip-19158 108 28 turned turn VERB eblip-19158 108 29 in in ADP eblip-19158 108 30 the the DET eblip-19158 108 31 required require VERB eblip-19158 108 32 3 3 NUM eblip-19158 108 33 samples sample NOUN eblip-19158 108 34 ) ) PUNCT eblip-19158 108 35 . . PUNCT eblip-19158 109 1 in in ADP eblip-19158 109 2 accordance accordance NOUN eblip-19158 109 3 with with ADP eblip-19158 109 4 irb irb NOUN eblip-19158 109 5 protocol protocol NOUN eblip-19158 109 6 , , PUNCT eblip-19158 109 7 the the DET eblip-19158 109 8 eng eng PROPN eblip-19158 109 9 102 102 NUM eblip-19158 109 10 coordinator coordinator NOUN eblip-19158 109 11 de de VERB eblip-19158 109 12 - - VERB eblip-19158 109 13 identified identify VERB eblip-19158 109 14 all all DET eblip-19158 109 15 papers paper NOUN eblip-19158 109 16 before before SCONJ eblip-19158 109 17 the the DET eblip-19158 109 18 author author NOUN eblip-19158 109 19 received receive VERB eblip-19158 109 20 them they PRON eblip-19158 109 21 for for ADP eblip-19158 109 22 this this DET eblip-19158 109 23 study study NOUN eblip-19158 109 24 . . PUNCT eblip-19158 109 25     SPACE eblip-19158 110 1 the the DET eblip-19158 110 2 author author NOUN eblip-19158 110 3 read read VERB eblip-19158 110 4 through through ADP eblip-19158 110 5 the the DET eblip-19158 110 6 first first ADJ eblip-19158 110 7 50 50 NUM eblip-19158 110 8 samples sample NOUN eblip-19158 110 9 received receive VERB eblip-19158 110 10 in in ADP eblip-19158 110 11 order order NOUN eblip-19158 110 12 to to PART eblip-19158 110 13 find find VERB eblip-19158 110 14 sets set NOUN eblip-19158 110 15 of of ADP eblip-19158 110 16 anchor anchor NOUN eblip-19158 110 17 papers paper NOUN eblip-19158 110 18 to to PART eblip-19158 110 19 use use VERB eblip-19158 110 20 during during ADP eblip-19158 110 21 the the DET eblip-19158 110 22 rubric rubric ADJ eblip-19158 110 23 training training NOUN eblip-19158 110 24 sessions session NOUN eblip-19158 110 25 , , PUNCT eblip-19158 110 26 as as SCONJ eblip-19158 110 27 was be AUX eblip-19158 110 28 recommended recommend VERB eblip-19158 110 29 by by ADP eblip-19158 110 30 oakleaf oakleaf NOUN eblip-19158 110 31 ( ( PUNCT eblip-19158 110 32 2009 2009 NUM eblip-19158 110 33 ) ) PUNCT eblip-19158 110 34 . . PUNCT eblip-19158 110 35     SPACE eblip-19158 111 1 anchor anchor NOUN eblip-19158 111 2 , , PUNCT eblip-19158 111 3 or or CCONJ eblip-19158 111 4 model model NOUN eblip-19158 111 5 , , PUNCT eblip-19158 111 6 papers paper NOUN eblip-19158 111 7 were be AUX eblip-19158 111 8 selected select VERB eblip-19158 111 9 as as ADP eblip-19158 111 10 examples example NOUN eblip-19158 111 11 for for ADP eblip-19158 111 12 the the DET eblip-19158 111 13 training training NOUN eblip-19158 111 14 session session NOUN eblip-19158 111 15 because because SCONJ eblip-19158 111 16 they they PRON eblip-19158 111 17 reflected reflect VERB eblip-19158 111 18 a a DET eblip-19158 111 19 range range NOUN eblip-19158 111 20 of of ADP eblip-19158 111 21 high high ADJ eblip-19158 111 22 , , PUNCT eblip-19158 111 23 medium medium ADJ eblip-19158 111 24 , , PUNCT eblip-19158 111 25 and and CCONJ eblip-19158 111 26 low low ADJ eblip-19158 111 27 scoring scoring NOUN eblip-19158 111 28 student student NOUN eblip-19158 111 29 work work NOUN eblip-19158 111 30 . . PUNCT eblip-19158 111 31     SPACE eblip-19158 112 1 fifteen fifteen NUM eblip-19158 112 2 annotated annotated ADJ eblip-19158 112 3 bibliographies bibliography NOUN eblip-19158 112 4 were be AUX eblip-19158 112 5 selected select VERB eblip-19158 112 6 and and CCONJ eblip-19158 112 7 grouped group VERB eblip-19158 112 8 into into ADP eblip-19158 112 9 three three NUM eblip-19158 112 10 sets set NOUN eblip-19158 112 11 so so SCONJ eblip-19158 112 12 as as SCONJ eblip-19158 112 13 to to PART eblip-19158 112 14 reflect reflect VERB eblip-19158 112 15 a a DET eblip-19158 112 16 variety variety NOUN eblip-19158 112 17 of of ADP eblip-19158 112 18 student student NOUN eblip-19158 112 19 responses response NOUN eblip-19158 112 20 to to ADP eblip-19158 112 21 the the DET eblip-19158 112 22 assignment assignment NOUN eblip-19158 112 23 . . PUNCT eblip-19158 112 24         SPACE eblip-19158 113 1 preparing prepare VERB eblip-19158 113 2 for for ADP eblip-19158 113 3 the the DET eblip-19158 113 4 training training NOUN eblip-19158 113 5 session session NOUN eblip-19158 113 6 : : PUNCT eblip-19158 113 7 issues issue NOUN eblip-19158 113 8 of of ADP eblip-19158 113 9 inter inter ADJ eblip-19158 113 10 - - ADJ eblip-19158 113 11 rater rater ADJ eblip-19158 113 12 reliability reliability NOUN eblip-19158 113 13     SPACE eblip-19158 113 14 the the DET eblip-19158 113 15 author author NOUN eblip-19158 113 16 selected select VERB eblip-19158 113 17 three three NUM eblip-19158 113 18 other other ADJ eblip-19158 113 19 librarians librarian NOUN eblip-19158 113 20 to to PART eblip-19158 113 21 help help VERB eblip-19158 113 22 score score VERB eblip-19158 113 23 the the DET eblip-19158 113 24 student student NOUN eblip-19158 113 25 work work NOUN eblip-19158 113 26 samples sample NOUN eblip-19158 113 27 . . PUNCT eblip-19158 114 1 the the DET eblip-19158 114 2 other other ADJ eblip-19158 114 3 three three NUM eblip-19158 114 4 librarians librarian NOUN eblip-19158 114 5 were be AUX eblip-19158 114 6 trained train VERB eblip-19158 114 7 on on ADP eblip-19158 114 8 how how SCONJ eblip-19158 114 9 to to PART eblip-19158 114 10 apply apply VERB eblip-19158 114 11 the the DET eblip-19158 114 12 rubric rubric NOUN eblip-19158 114 13 in in ADP eblip-19158 114 14 a a DET eblip-19158 114 15 series series NOUN eblip-19158 114 16 of of ADP eblip-19158 114 17 two two NUM eblip-19158 114 18 2 2 NUM eblip-19158 114 19 hour hour NOUN eblip-19158 114 20 sessions session NOUN eblip-19158 114 21 . . PUNCT eblip-19158 114 22         SPACE eblip-19158 115 1 inter inter ADJ eblip-19158 115 2 - - ADJ eblip-19158 115 3 rater rater ADJ eblip-19158 115 4 reliability reliability NOUN eblip-19158 115 5 was be AUX eblip-19158 115 6 an an DET eblip-19158 115 7 issue issue NOUN eblip-19158 115 8 of of ADP eblip-19158 115 9 interest interest NOUN eblip-19158 115 10 for for ADP eblip-19158 115 11 this this DET eblip-19158 115 12 project project NOUN eblip-19158 115 13 because because SCONJ eblip-19158 115 14 four four NUM eblip-19158 115 15 librarians librarian NOUN eblip-19158 115 16 were be AUX eblip-19158 115 17 involved involve VERB eblip-19158 115 18 in in ADP eblip-19158 115 19 the the DET eblip-19158 115 20 rating rating NOUN eblip-19158 115 21 process process NOUN eblip-19158 115 22 . . PUNCT eblip-19158 116 1 inter inter ADJ eblip-19158 116 2 - - ADJ eblip-19158 116 3 rater rater ADJ eblip-19158 116 4 reliability reliability NOUN eblip-19158 116 5 is be AUX eblip-19158 116 6 the the DET eblip-19158 116 7 degree degree NOUN eblip-19158 116 8 that that SCONJ eblip-19158 116 9 “ " PUNCT eblip-19158 116 10 raters rater NOUN eblip-19158 116 11 ’ ' PUNCT eblip-19158 116 12 responses response NOUN eblip-19158 116 13 are be AUX eblip-19158 116 14 consistent consistent ADJ eblip-19158 116 15 across across ADP eblip-19158 116 16 representative representative ADJ eblip-19158 116 17 student student NOUN eblip-19158 116 18 populations population NOUN eblip-19158 116 19 ” " PUNCT eblip-19158 116 20 ( ( PUNCT eblip-19158 116 21 maki maki NOUN eblip-19158 116 22 , , PUNCT eblip-19158 116 23 2010 2010 NUM eblip-19158 116 24 , , PUNCT eblip-19158 116 25 p. p. NOUN eblip-19158 116 26 224 224 NUM eblip-19158 116 27 ) ) PUNCT eblip-19158 116 28 . . PUNCT eblip-19158 117 1 calculating calculate VERB eblip-19158 117 2 inter inter ADJ eblip-19158 117 3 - - ADJ eblip-19158 117 4 rater rater ADJ eblip-19158 117 5 reliability reliability NOUN eblip-19158 117 6 can can AUX eblip-19158 117 7 determine determine VERB eblip-19158 117 8 if if SCONJ eblip-19158 117 9 raters rater NOUN eblip-19158 117 10 are be AUX eblip-19158 117 11 applying apply VERB eblip-19158 117 12 a a DET eblip-19158 117 13 rubric rubric NOUN eblip-19158 117 14 in in ADP eblip-19158 117 15 the the DET eblip-19158 117 16 same same ADJ eblip-19158 117 17 way way NOUN eblip-19158 117 18 , , PUNCT eblip-19158 117 19 meaning mean VERB eblip-19158 117 20 the the DET eblip-19158 117 21 ratings rating NOUN eblip-19158 117 22 can can AUX eblip-19158 117 23 statistically statistically ADV eblip-19158 117 24 be be AUX eblip-19158 117 25 considered consider VERB eblip-19158 117 26 equivalent equivalent ADJ eblip-19158 117 27 to to ADP eblip-19158 117 28 one one NUM eblip-19158 117 29 another another DET eblip-19158 117 30 ( ( PUNCT eblip-19158 117 31 cohen cohen PROPN eblip-19158 117 32 , , PUNCT eblip-19158 117 33 1960 1960 NUM eblip-19158 117 34 ; ; PUNCT eblip-19158 117 35 moskal moskal NOUN eblip-19158 117 36 , , PUNCT eblip-19158 117 37 2003 2003 NUM eblip-19158 117 38 ; ; PUNCT eblip-19158 117 39 oakleaf oakleaf NOUN eblip-19158 117 40 , , PUNCT eblip-19158 117 41 2009 2009 NUM eblip-19158 117 42 ) ) PUNCT eblip-19158 117 43 . . PUNCT eblip-19158 118 1 because because SCONJ eblip-19158 118 2 the the DET eblip-19158 118 3 sample sample NOUN eblip-19158 118 4 of of ADP eblip-19158 118 5 student student NOUN eblip-19158 118 6 work work NOUN eblip-19158 118 7 resulted result VERB eblip-19158 118 8 in in ADP eblip-19158 118 9 over over ADP eblip-19158 118 10 700 700 NUM eblip-19158 118 11 individual individual ADJ eblip-19158 118 12 annotations annotation NOUN eblip-19158 118 13 , , PUNCT eblip-19158 118 14 the the DET eblip-19158 118 15 author author NOUN eblip-19158 118 16 wanted want VERB eblip-19158 118 17 to to PART eblip-19158 118 18 determine determine VERB eblip-19158 118 19 if if SCONJ eblip-19158 118 20 this this DET eblip-19158 118 21 total total NOUN eblip-19158 118 22 could could AUX eblip-19158 118 23 be be AUX eblip-19158 118 24 equally equally ADV eblip-19158 118 25 divided divide VERB eblip-19158 118 26 between between ADP eblip-19158 118 27 the the DET eblip-19158 118 28 four four NUM eblip-19158 118 29 raters rater NOUN eblip-19158 118 30 , , PUNCT eblip-19158 118 31 resulting result VERB eblip-19158 118 32 in in ADP eblip-19158 118 33 each each DET eblip-19158 118 34 person person NOUN eblip-19158 118 35 having have VERB eblip-19158 118 36 to to PART eblip-19158 118 37 score score VERB eblip-19158 118 38 only only ADV eblip-19158 118 39 a a DET eblip-19158 118 40 quarter quarter NOUN eblip-19158 118 41 of of ADP eblip-19158 118 42 the the DET eblip-19158 118 43 samples sample NOUN eblip-19158 118 44 . . PUNCT eblip-19158 119 1 if if SCONJ eblip-19158 119 2 , , PUNCT eblip-19158 119 3 during during ADP eblip-19158 119 4 the the DET eblip-19158 119 5 training training NOUN eblip-19158 119 6 sessions session NOUN eblip-19158 119 7 , , PUNCT eblip-19158 119 8 the the DET eblip-19158 119 9 four four NUM eblip-19158 119 10 raters rater NOUN eblip-19158 119 11 could could AUX eblip-19158 119 12 be be AUX eblip-19158 119 13 shown show VERB eblip-19158 119 14 to to PART eblip-19158 119 15 have have VERB eblip-19158 119 16 a a DET eblip-19158 119 17 shared share VERB eblip-19158 119 18 understanding understanding NOUN eblip-19158 119 19 of of ADP eblip-19158 119 20 the the DET eblip-19158 119 21 rubric rubric NOUN eblip-19158 119 22 , , PUNCT eblip-19158 119 23 as as SCONJ eblip-19158 119 24 evidenced evidence VERB eblip-19158 119 25 through through ADP eblip-19158 119 26 calculating calculate VERB eblip-19158 119 27 inter inter ADJ eblip-19158 119 28 - - ADJ eblip-19158 119 29 rater rater ADJ eblip-19158 119 30 reliability reliability NOUN eblip-19158 119 31 statistics statistic NOUN eblip-19158 119 32 , , PUNCT eblip-19158 119 33 then then ADV eblip-19158 119 34 only only ADV eblip-19158 119 35 the the DET eblip-19158 119 36 recommended recommend VERB eblip-19158 119 37 30 30 NUM eblip-19158 119 38 % % NOUN eblip-19158 119 39 overlap overlap NOUN eblip-19158 119 40 between between ADP eblip-19158 119 41 papers paper NOUN eblip-19158 119 42 would would AUX eblip-19158 119 43 be be AUX eblip-19158 119 44 needed need VERB eblip-19158 119 45 ( ( PUNCT eblip-19158 119 46 stemler stemler NOUN eblip-19158 119 47 , , PUNCT eblip-19158 119 48 2004 2004 NUM eblip-19158 119 49 ) ) PUNCT eblip-19158 119 50 . . PUNCT eblip-19158 119 51     SPACE eblip-19158 120 1 in in ADP eblip-19158 120 2 order order NOUN eblip-19158 120 3 to to PART eblip-19158 120 4 calculate calculate VERB eblip-19158 120 5 inter inter ADJ eblip-19158 120 6 - - ADJ eblip-19158 120 7 rater rater ADJ eblip-19158 120 8 reliability reliability NOUN eblip-19158 120 9 for for ADP eblip-19158 120 10 this this DET eblip-19158 120 11 study study NOUN eblip-19158 120 12 , , PUNCT eblip-19158 120 13 the the DET eblip-19158 120 14 author author NOUN eblip-19158 120 15 used use VERB eblip-19158 120 16 agreestat agreestat NOUN eblip-19158 120 17 , , PUNCT eblip-19158 120 18 a a DET eblip-19158 120 19 downloadable downloadable ADJ eblip-19158 120 20 microsoft microsoft PROPN eblip-19158 120 21 excel excel PROPN eblip-19158 120 22 workbook workbook NOUN eblip-19158 120 23 that that PRON eblip-19158 120 24 calculates calculate VERB eblip-19158 120 25 a a DET eblip-19158 120 26 variety variety NOUN eblip-19158 120 27 of of ADP eblip-19158 120 28 agreement agreement NOUN eblip-19158 120 29 statistics statistic NOUN eblip-19158 120 30 . . PUNCT eblip-19158 121 1 due due ADP eblip-19158 121 2 to to ADP eblip-19158 121 3 the the DET eblip-19158 121 4 fact fact NOUN eblip-19158 121 5 that that SCONJ eblip-19158 121 6 there there PRON eblip-19158 121 7 were be VERB eblip-19158 121 8 four four NUM eblip-19158 121 9 raters rater NOUN eblip-19158 121 10 , , PUNCT eblip-19158 121 11 fleiss fleiss PROPN eblip-19158 121 12 ’s ’s PART eblip-19158 121 13 kappa kappa NOUN eblip-19158 121 14 and and CCONJ eblip-19158 121 15 conger conger PROPN eblip-19158 121 16 ’s ’s PART eblip-19158 121 17 kappa kappa PROPN eblip-19158 121 18 were be AUX eblip-19158 121 19 used use VERB eblip-19158 121 20 as as ADP eblip-19158 121 21 the the DET eblip-19158 121 22 agreement agreement NOUN eblip-19158 121 23 statistics statistic NOUN eblip-19158 121 24 for for ADP eblip-19158 121 25 this this DET eblip-19158 121 26 study study NOUN eblip-19158 121 27 . . PUNCT eblip-19158 122 1 these these DET eblip-19158 122 2 statistics statistic NOUN eblip-19158 122 3 are be AUX eblip-19158 122 4 based base VERB eblip-19158 122 5 on on ADP eblip-19158 122 6 cohen cohen PROPN eblip-19158 122 7 ’s ’s PART eblip-19158 122 8 kappa kappa PROPN eblip-19158 122 9 , , PUNCT eblip-19158 122 10 a a DET eblip-19158 122 11 well well ADV eblip-19158 122 12 - - PUNCT eblip-19158 122 13 established establish VERB eblip-19158 122 14 statistic statistic NOUN eblip-19158 122 15 for for ADP eblip-19158 122 16 calculating calculate VERB eblip-19158 122 17 agreement agreement NOUN eblip-19158 122 18 between between ADP eblip-19158 122 19 two two NUM eblip-19158 122 20 raters rater NOUN eblip-19158 122 21 . . PUNCT eblip-19158 123 1 fleiss fleiss PROPN eblip-19158 123 2 ’s ’s PART eblip-19158 123 3 kappa kappa NOUN eblip-19158 123 4 and and CCONJ eblip-19158 123 5 conger conger PROPN eblip-19158 123 6 ’s ’s PART eblip-19158 123 7 kappa kappa PROPN eblip-19158 123 8 modify modify PROPN eblip-19158 123 9 cohen cohen PROPN eblip-19158 123 10 ’s ’s PART eblip-19158 123 11 kappa kappa PROPN eblip-19158 123 12 to to PART eblip-19158 123 13 allow allow VERB eblip-19158 123 14 for for ADP eblip-19158 123 15 agreement agreement NOUN eblip-19158 123 16 between between ADP eblip-19158 123 17 multiple multiple ADJ eblip-19158 123 18 raters rater NOUN eblip-19158 123 19 ( ( PUNCT eblip-19158 123 20 stemler stemler NOUN eblip-19158 123 21 , , PUNCT eblip-19158 123 22 2004 2004 NUM eblip-19158 123 23 ; ; PUNCT eblip-19158 123 24 oakleaf oakleaf NOUN eblip-19158 123 25 , , PUNCT eblip-19158 123 26 2009 2009 NUM eblip-19158 123 27 ; ; PUNCT eblip-19158 123 28 fleiss fleiss PROPN eblip-19158 123 29 , , PUNCT eblip-19158 123 30 1971 1971 NUM eblip-19158 123 31 ; ; PUNCT eblip-19158 123 32 conger conger NOUN eblip-19158 123 33 , , PUNCT eblip-19158 123 34 1980 1980 NUM eblip-19158 123 35 ; ; PUNCT eblip-19158 123 36 gwet gwet NOUN eblip-19158 123 37 , , PUNCT eblip-19158 123 38 2010 2010 NUM eblip-19158 123 39 ) ) PUNCT eblip-19158 123 40 . . PUNCT eblip-19158 124 1 the the PRON eblip-19158 124 2 landis landis PROPN eblip-19158 124 3 and and CCONJ eblip-19158 124 4 koch koch PROPN eblip-19158 124 5 index index NOUN eblip-19158 124 6 for for ADP eblip-19158 124 7 interpreting interpret VERB eblip-19158 124 8 kappa kappa PROPN eblip-19158 124 9 statistics statistic NOUN eblip-19158 124 10 was be AUX eblip-19158 124 11 used use VERB eblip-19158 124 12 to to PART eblip-19158 124 13 determine determine VERB eblip-19158 124 14 if if SCONJ eblip-19158 124 15 sufficient sufficient ADJ eblip-19158 124 16 agreement agreement NOUN eblip-19158 124 17 had have AUX eblip-19158 124 18 been be AUX eblip-19158 124 19 reached reach VERB eblip-19158 124 20 . . PUNCT eblip-19158 125 1 a a DET eblip-19158 125 2 score score NOUN eblip-19158 125 3 of of ADP eblip-19158 125 4 0.70 0.70 NUM eblip-19158 125 5 is be AUX eblip-19158 125 6 the the DET eblip-19158 125 7 minimum minimum ADJ eblip-19158 125 8 score score NOUN eblip-19158 125 9 needed need VERB eblip-19158 125 10 on on ADP eblip-19158 125 11 the the DET eblip-19158 125 12 index index NOUN eblip-19158 125 13 for for SCONJ eblip-19158 125 14 raters rater NOUN eblip-19158 125 15 to to PART eblip-19158 125 16 be be AUX eblip-19158 125 17 considered consider VERB eblip-19158 125 18 equivalent equivalent ADJ eblip-19158 125 19 ( ( PUNCT eblip-19158 125 20 landis landis PROPN eblip-19158 125 21 & & CCONJ eblip-19158 125 22 koch koch PROPN eblip-19158 125 23 , , PUNCT eblip-19158 125 24 1971 1971 NUM eblip-19158 125 25 ; ; PUNCT eblip-19158 125 26 stemler stemler NOUN eblip-19158 125 27 , , PUNCT eblip-19158 125 28 2004 2004 NUM eblip-19158 125 29 ) ) PUNCT eblip-19158 125 30 . . PUNCT eblip-19158 125 31         SPACE eblip-19158 126 1 table table NOUN eblip-19158 126 2 1 1 NUM eblip-19158 126 3 kappa kappa PROPN eblip-19158 126 4 index index PROPN eblip-19158 126 5 kappa kappa PROPN eblip-19158 126 6 statistic statistic PROPN eblip-19158 126 7 strength strength NOUN eblip-19158 126 8 of of ADP eblip-19158 126 9 agreement agreement NOUN eblip-19158 126 10 < < X eblip-19158 126 11 0.00 0.00 NUM eblip-19158 126 12 poor poor ADJ eblip-19158 126 13 0.00 0.00 NUM eblip-19158 126 14 - - SYM eblip-19158 126 15 0.20 0.20 NUM eblip-19158 126 16 slight slight ADJ eblip-19158 126 17 0.21 0.21 NUM eblip-19158 126 18 - - SYM eblip-19158 126 19 0.40 0.40 NUM eblip-19158 126 20 fair fair ADJ eblip-19158 126 21 0.41 0.41 NUM eblip-19158 126 22 - - PUNCT eblip-19158 126 23 0.60 0.60 NUM eblip-19158 126 24 moderate moderate ADJ eblip-19158 126 25 0.61 0.61 NUM eblip-19158 126 26 - - SYM eblip-19158 126 27 0.80 0.80 NUM eblip-19158 126 28 substantial substantial ADJ eblip-19158 126 29 0.81 0.81 NUM eblip-19158 126 30 - - SYM eblip-19158 126 31 1.00 1.00 NUM eblip-19158 126 32 almost almost ADV eblip-19158 126 33 perfect perfect ADJ eblip-19158 126 34         SPACE eblip-19158 126 35 the the DET eblip-19158 126 36 training training NOUN eblip-19158 126 37 session session NOUN eblip-19158 126 38         SPACE eblip-19158 126 39 the the DET eblip-19158 126 40 rubric rubric ADJ eblip-19158 126 41 training training NOUN eblip-19158 126 42 sessions session NOUN eblip-19158 126 43 followed follow VERB eblip-19158 126 44 maki maki PROPN eblip-19158 126 45 ’s ’s PART eblip-19158 126 46 6 6 NUM eblip-19158 126 47 - - PUNCT eblip-19158 126 48 step step NOUN eblip-19158 126 49 process process NOUN eblip-19158 126 50 : : PUNCT eblip-19158 126 51 first first ADV eblip-19158 126 52 , , PUNCT eblip-19158 126 53 the the DET eblip-19158 126 54 author author NOUN eblip-19158 126 55 introduced introduce VERB eblip-19158 126 56 the the DET eblip-19158 126 57 annotated annotate VERB eblip-19158 126 58 bibliography bibliography NOUN eblip-19158 126 59 assignment assignment NOUN eblip-19158 126 60 and and CCONJ eblip-19158 126 61 the the DET eblip-19158 126 62 learning learn VERB eblip-19158 126 63 outcomes outcome NOUN eblip-19158 126 64 that that PRON eblip-19158 126 65 were be AUX eblip-19158 126 66 to to PART eblip-19158 126 67 be be AUX eblip-19158 126 68 assessed assess VERB eblip-19158 126 69 . . PUNCT eblip-19158 127 1 she she PRON eblip-19158 127 2 handed hand VERB eblip-19158 127 3 out out ADP eblip-19158 127 4 copies copy NOUN eblip-19158 127 5 of of ADP eblip-19158 127 6 the the DET eblip-19158 127 7 rubric rubric NOUN eblip-19158 127 8 and and CCONJ eblip-19158 127 9 explained explain VERB eblip-19158 127 10 the the DET eblip-19158 127 11 criteria criterion NOUN eblip-19158 127 12 and and CCONJ eblip-19158 127 13 performance performance NOUN eblip-19158 127 14 indicators indicator NOUN eblip-19158 127 15 . . PUNCT eblip-19158 128 1 the the DET eblip-19158 128 2 author author NOUN eblip-19158 128 3 then then ADV eblip-19158 128 4 conducted conduct VERB eblip-19158 128 5 a a DET eblip-19158 128 6 “ " PUNCT eblip-19158 128 7 read read NOUN eblip-19158 128 8 - - PUNCT eblip-19158 128 9 aloud aloud NOUN eblip-19158 128 10 ” " PUNCT eblip-19158 128 11 with with ADP eblip-19158 128 12 two two NUM eblip-19158 128 13 of of ADP eblip-19158 128 14 the the DET eblip-19158 128 15 anchor anchor NOUN eblip-19158 128 16 annotated annotate VERB eblip-19158 128 17 bibliographies bibliography NOUN eblip-19158 128 18 , , PUNCT eblip-19158 128 19 reading read VERB eblip-19158 128 20 through through ADP eblip-19158 128 21 the the DET eblip-19158 128 22 annotations annotation NOUN eblip-19158 128 23 and and CCONJ eblip-19158 128 24 articulating articulating ADJ eblip-19158 128 25 how how SCONJ eblip-19158 128 26 she she PRON eblip-19158 128 27 would would AUX eblip-19158 128 28 score score VERB eblip-19158 128 29 them they PRON eblip-19158 128 30 based base VERB eblip-19158 128 31 on on ADP eblip-19158 128 32 the the DET eblip-19158 128 33 rubric rubric NOUN eblip-19158 128 34 . . PUNCT eblip-19158 129 1 once once SCONJ eblip-19158 129 2 the the DET eblip-19158 129 3 librarians librarian NOUN eblip-19158 129 4 felt feel VERB eblip-19158 129 5 comfortable comfortable ADJ eblip-19158 129 6 with with ADP eblip-19158 129 7 the the DET eblip-19158 129 8 application application NOUN eblip-19158 129 9 of of ADP eblip-19158 129 10 the the DET eblip-19158 129 11 rubric rubric NOUN eblip-19158 129 12 , , PUNCT eblip-19158 129 13 each each DET eblip-19158 129 14 librarian librarian NOUN eblip-19158 129 15 individually individually ADV eblip-19158 129 16 scored score VERB eblip-19158 129 17 a a DET eblip-19158 129 18 practice practice NOUN eblip-19158 129 19 set set VERB eblip-19158 129 20 of of ADP eblip-19158 129 21 three three NUM eblip-19158 129 22 bibliographies bibliography NOUN eblip-19158 129 23 . . PUNCT eblip-19158 130 1 differences difference NOUN eblip-19158 130 2 in in ADP eblip-19158 130 3 scoring scoring NOUN eblip-19158 130 4 were be AUX eblip-19158 130 5 identified identify VERB eblip-19158 130 6 , , PUNCT eblip-19158 130 7 discussed discuss VERB eblip-19158 130 8 , , PUNCT eblip-19158 130 9 and and CCONJ eblip-19158 130 10 reconciled reconcile VERB eblip-19158 130 11 ( ( PUNCT eblip-19158 130 12 maki maki NOUN eblip-19158 130 13 , , PUNCT eblip-19158 130 14 2010 2010 NUM eblip-19158 130 15 ; ; PUNCT eblip-19158 130 16 oakleaf oakleaf NOUN eblip-19158 130 17 , , PUNCT eblip-19158 130 18 2009 2009 NUM eblip-19158 130 19 ) ) PUNCT eblip-19158 130 20 . . PUNCT eblip-19158 130 21         SPACE eblip-19158 131 1 at at ADP eblip-19158 131 2 this this DET eblip-19158 131 3 point point NOUN eblip-19158 131 4 , , PUNCT eblip-19158 131 5 the the DET eblip-19158 131 6 author author NOUN eblip-19158 131 7 used use VERB eblip-19158 131 8 the the DET eblip-19158 131 9 statistical statistical ADJ eblip-19158 131 10 software software NOUN eblip-19158 131 11 agreestat agreestat NOUN eblip-19158 131 12 to to PART eblip-19158 131 13 calculate calculate VERB eblip-19158 131 14 the the DET eblip-19158 131 15 group group NOUN eblip-19158 131 16 ’s ’s PART eblip-19158 131 17 inter inter ADJ eblip-19158 131 18 - - ADJ eblip-19158 131 19 rater rater ADJ eblip-19158 131 20 reliability reliability NOUN eblip-19158 131 21 . . PUNCT eblip-19158 132 1 since since SCONJ eblip-19158 132 2 the the DET eblip-19158 132 3 initial initial ADJ eblip-19158 132 4 scores score NOUN eblip-19158 132 5 were be AUX eblip-19158 132 6 very very ADV eblip-19158 132 7 low low ADJ eblip-19158 132 8 on on ADP eblip-19158 132 9 the the DET eblip-19158 132 10 landis landis PROPN eblip-19158 132 11 and and CCONJ eblip-19158 132 12 koch koch PROPN eblip-19158 132 13 index index PROPN eblip-19158 132 14 , , PUNCT eblip-19158 132 15 the the DET eblip-19158 132 16 scoring scoring NOUN eblip-19158 132 17 and and CCONJ eblip-19158 132 18 discussing discuss VERB eblip-19158 132 19 process process NOUN eblip-19158 132 20 was be AUX eblip-19158 132 21 repeated repeat VERB eblip-19158 132 22 twice twice ADV eblip-19158 132 23 more more ADJ eblip-19158 132 24 . . PUNCT eblip-19158 133 1 the the DET eblip-19158 133 2 inter inter ADJ eblip-19158 133 3 - - ADJ eblip-19158 133 4 rater rater ADJ eblip-19158 133 5 reliability reliability NOUN eblip-19158 133 6 scores score NOUN eblip-19158 133 7 for for ADP eblip-19158 133 8 each each DET eblip-19158 133 9 round round NOUN eblip-19158 133 10 were be AUX eblip-19158 133 11 as as SCONJ eblip-19158 133 12 follows follow VERB eblip-19158 133 13 : : PUNCT eblip-19158 133 14     SPACE eblip-19158 133 15 inter inter ADJ eblip-19158 133 16 - - ADJ eblip-19158 133 17 rater rater ADJ eblip-19158 133 18 reliability reliability NOUN eblip-19158 133 19 greater great ADJ eblip-19158 133 20 than than ADP eblip-19158 133 21 “ " PUNCT eblip-19158 133 22 fair fair ADJ eblip-19158 133 23 ” " PUNCT eblip-19158 133 24 was be AUX eblip-19158 133 25 not not PART eblip-19158 133 26 reached reach VERB eblip-19158 133 27 during during ADP eblip-19158 133 28 three three NUM eblip-19158 133 29 rounds round NOUN eblip-19158 133 30 of of ADP eblip-19158 133 31 scoring scoring NOUN eblip-19158 133 32 ; ; PUNCT eblip-19158 133 33 the the DET eblip-19158 133 34 numbers number NOUN eblip-19158 133 35 were be AUX eblip-19158 133 36 well well ADV eblip-19158 133 37 below below ADP eblip-19158 133 38 the the DET eblip-19158 133 39 recommended recommend VERB eblip-19158 133 40 0.70 0.70 NUM eblip-19158 133 41 score score NOUN eblip-19158 133 42 . . PUNCT eblip-19158 134 1 the the DET eblip-19158 134 2 author author NOUN eblip-19158 134 3 decided decide VERB eblip-19158 134 4 that that SCONJ eblip-19158 134 5 , , PUNCT eblip-19158 134 6 since since SCONJ eblip-19158 134 7 the the DET eblip-19158 134 8 raters rater NOUN eblip-19158 134 9 could could AUX eblip-19158 134 10 not not PART eblip-19158 134 11 be be AUX eblip-19158 134 12 considered consider VERB eblip-19158 134 13 equivalent equivalent ADJ eblip-19158 134 14 in in ADP eblip-19158 134 15 their their PRON eblip-19158 134 16 application application NOUN eblip-19158 134 17 of of ADP eblip-19158 134 18 the the DET eblip-19158 134 19 rubric rubric NOUN eblip-19158 134 20 , , PUNCT eblip-19158 134 21 the the DET eblip-19158 134 22 papers paper NOUN eblip-19158 134 23 could could AUX eblip-19158 134 24 not not PART eblip-19158 134 25 be be AUX eblip-19158 134 26 divided divide VERB eblip-19158 134 27 evenly evenly ADV eblip-19158 134 28 between between ADP eblip-19158 134 29 them they PRON eblip-19158 134 30 . . PUNCT eblip-19158 135 1 instead instead ADV eblip-19158 135 2 , , PUNCT eblip-19158 135 3 each each DET eblip-19158 135 4 bibliography bibliography NOUN eblip-19158 135 5 would would AUX eblip-19158 135 6 be be AUX eblip-19158 135 7 scored score VERB eblip-19158 135 8 twice twice ADV eblip-19158 135 9 , , PUNCT eblip-19158 135 10 by by ADP eblip-19158 135 11 two two NUM eblip-19158 135 12 different different ADJ eblip-19158 135 13 raters rater NOUN eblip-19158 135 14 , , PUNCT eblip-19158 135 15 and and CCONJ eblip-19158 135 16 the the DET eblip-19158 135 17 author author NOUN eblip-19158 135 18 would would AUX eblip-19158 135 19 reconcile reconcile VERB eblip-19158 135 20 any any DET eblip-19158 135 21 differences difference NOUN eblip-19158 135 22 in in ADP eblip-19158 135 23 scores score NOUN eblip-19158 135 24 , , PUNCT eblip-19158 135 25 a a DET eblip-19158 135 26 suggested suggest VERB eblip-19158 135 27 method method NOUN eblip-19158 135 28 for for ADP eblip-19158 135 29 resolving resolve VERB eblip-19158 135 30 dissimilar dissimilar ADJ eblip-19158 135 31 ratings rating NOUN eblip-19158 135 32 known know VERB eblip-19158 135 33 as as ADP eblip-19158 135 34 “ " PUNCT eblip-19158 135 35 tertium tertium NOUN eblip-19158 135 36 quid quid NOUN eblip-19158 135 37 , , PUNCT eblip-19158 135 38 ” " PUNCT eblip-19158 135 39 in in ADP eblip-19158 135 40 which which PRON eblip-19158 135 41 the the DET eblip-19158 135 42 score score NOUN eblip-19158 135 43 of of ADP eblip-19158 135 44 an an DET eblip-19158 135 45 adjudicator adjudicator NOUN eblip-19158 135 46 is be AUX eblip-19158 135 47 combined combine VERB eblip-19158 135 48 with with ADP eblip-19158 135 49 the the DET eblip-19158 135 50 closest close ADJ eblip-19158 135 51 score score NOUN eblip-19158 135 52 of of ADP eblip-19158 135 53 the the DET eblip-19158 135 54 original original ADJ eblip-19158 135 55 raters rater NOUN eblip-19158 135 56 , , PUNCT eblip-19158 135 57 and and CCONJ eblip-19158 135 58 the the DET eblip-19158 135 59 dissimilar dissimilar ADJ eblip-19158 135 60 score score NOUN eblip-19158 135 61 is be AUX eblip-19158 135 62 not not PART eblip-19158 135 63 used use VERB eblip-19158 135 64 ( ( PUNCT eblip-19158 135 65 johnson johnson PROPN eblip-19158 135 66 , , PUNCT eblip-19158 135 67 penny penny NOUN eblip-19158 135 68 , , PUNCT eblip-19158 135 69 and and CCONJ eblip-19158 135 70 gordon gordon PROPN eblip-19158 135 71 , , PUNCT eblip-19158 135 72 2008 2008 NUM eblip-19158 135 73 , , PUNCT eblip-19158 135 74 p. p. NOUN eblip-19158 135 75 241 241 NUM eblip-19158 135 76 ) ) PUNCT eblip-19158 135 77 . . PUNCT eblip-19158 136 1 the the DET eblip-19158 136 2 librarians librarian NOUN eblip-19158 136 3 had have VERB eblip-19158 136 4 two two NUM eblip-19158 136 5 weeks week NOUN eblip-19158 136 6 to to PART eblip-19158 136 7 score score VERB eblip-19158 136 8 their their PRON eblip-19158 136 9 set set NOUN eblip-19158 136 10 of of ADP eblip-19158 136 11 bibliographies bibliography NOUN eblip-19158 136 12 , , PUNCT eblip-19158 136 13 at at ADP eblip-19158 136 14 the the DET eblip-19158 136 15 end end NOUN eblip-19158 136 16 of of ADP eblip-19158 136 17 which which DET eblip-19158 136 18 time time NOUN eblip-19158 136 19 the the DET eblip-19158 136 20 author author NOUN eblip-19158 136 21 recorded record VERB eblip-19158 136 22 all all DET eblip-19158 136 23 scores score NOUN eblip-19158 136 24 . . PUNCT eblip-19158 136 25         SPACE eblip-19158 136 26 results result VERB eblip-19158 136 27     SPACE eblip-19158 136 28 a a DET eblip-19158 136 29 total total NOUN eblip-19158 136 30 of of ADP eblip-19158 136 31 773 773 NUM eblip-19158 136 32 annotations annotation NOUN eblip-19158 136 33 were be AUX eblip-19158 136 34 scored score VERB eblip-19158 136 35 for for ADP eblip-19158 136 36 the the DET eblip-19158 136 37 study study NOUN eblip-19158 136 38 . . PUNCT eblip-19158 137 1 the the DET eblip-19158 137 2 following follow VERB eblip-19158 137 3 table table NOUN eblip-19158 137 4 shows show VERB eblip-19158 137 5 the the DET eblip-19158 137 6 number number NOUN eblip-19158 137 7 of of ADP eblip-19158 137 8 annotations annotation NOUN eblip-19158 137 9 that that PRON eblip-19158 137 10 applied apply VERB eblip-19158 137 11 each each PRON eblip-19158 137 12 of of ADP eblip-19158 137 13 the the DET eblip-19158 137 14 evaluative evaluative ADJ eblip-19158 137 15 criteria criterion NOUN eblip-19158 137 16 , , PUNCT eblip-19158 137 17 and and CCONJ eblip-19158 137 18 to to ADP eblip-19158 137 19 what what DET eblip-19158 137 20 degree degree NOUN eblip-19158 137 21 : : PUNCT eblip-19158 137 22     SPACE eblip-19158 137 23 for for ADP eblip-19158 137 24 three three NUM eblip-19158 137 25 of of ADP eblip-19158 137 26 the the DET eblip-19158 137 27 five five NUM eblip-19158 137 28 evaluative evaluative ADJ eblip-19158 137 29 criteria criterion NOUN eblip-19158 137 30 , , PUNCT eblip-19158 137 31 the the DET eblip-19158 137 32 majority majority NOUN eblip-19158 137 33 of of ADP eblip-19158 137 34 the the DET eblip-19158 137 35 annotations annotation NOUN eblip-19158 137 36 received receive VERB eblip-19158 137 37 a a DET eblip-19158 137 38 level level NOUN eblip-19158 137 39 0 0 NUM eblip-19158 137 40 score score NOUN eblip-19158 137 41 , , PUNCT eblip-19158 137 42 meaning mean VERB eblip-19158 137 43 they they PRON eblip-19158 137 44 did do AUX eblip-19158 137 45 not not PART eblip-19158 137 46 provide provide VERB eblip-19158 137 47 any any DET eblip-19158 137 48 evidence evidence NOUN eblip-19158 137 49 of of ADP eblip-19158 137 50 using use VERB eblip-19158 137 51 the the DET eblip-19158 137 52 criterion criterion NOUN eblip-19158 137 53 in in ADP eblip-19158 137 54 question question NOUN eblip-19158 137 55 . . PUNCT eblip-19158 138 1 for for ADP eblip-19158 138 2 currency currency NOUN eblip-19158 138 3 , , PUNCT eblip-19158 138 4 686 686 NUM eblip-19158 138 5 ( ( PUNCT eblip-19158 138 6 88 88 NUM eblip-19158 138 7 % % NOUN eblip-19158 138 8 ) ) PUNCT eblip-19158 138 9 of of ADP eblip-19158 138 10 the the DET eblip-19158 138 11 annotations annotation NOUN eblip-19158 138 12 received receive VERB eblip-19158 138 13 a a DET eblip-19158 138 14 level level NOUN eblip-19158 138 15 0 0 NUM eblip-19158 138 16 ; ; PUNCT eblip-19158 138 17 for for ADP eblip-19158 138 18 accuracy accuracy NOUN eblip-19158 138 19 , , PUNCT eblip-19158 138 20 575 575 NUM eblip-19158 138 21 ( ( PUNCT eblip-19158 138 22 74 74 NUM eblip-19158 138 23 % % NOUN eblip-19158 138 24 ) ) PUNCT eblip-19158 138 25 received receive VERB eblip-19158 138 26 a a DET eblip-19158 138 27 level level NOUN eblip-19158 138 28 0 0 NUM eblip-19158 138 29 ; ; PUNCT eblip-19158 138 30 and and CCONJ eblip-19158 138 31 for for ADP eblip-19158 138 32 purpose purpose NOUN eblip-19158 138 33 , , PUNCT eblip-19158 138 34 615 615 NUM eblip-19158 138 35 ( ( PUNCT eblip-19158 138 36 79.5 79.5 NUM eblip-19158 138 37 % % NOUN eblip-19158 138 38 ) ) PUNCT eblip-19158 138 39 of of ADP eblip-19158 138 40 the the DET eblip-19158 138 41 annotations annotation NOUN eblip-19158 138 42 were be AUX eblip-19158 138 43 scored score VERB eblip-19158 138 44 level level NOUN eblip-19158 138 45 0 0 NUM eblip-19158 138 46 . . PUNCT eblip-19158 139 1 though though SCONJ eblip-19158 139 2 students student NOUN eblip-19158 139 3 did do AUX eblip-19158 139 4 better well ADV eblip-19158 139 5 applying apply VERB eblip-19158 139 6 the the DET eblip-19158 139 7 criteria criterion NOUN eblip-19158 139 8 relevance relevance NOUN eblip-19158 139 9 and and CCONJ eblip-19158 139 10 authority authority NOUN eblip-19158 139 11 , , PUNCT eblip-19158 139 12 a a DET eblip-19158 139 13 substantial substantial ADJ eblip-19158 139 14 percentage percentage NOUN eblip-19158 139 15 of of ADP eblip-19158 139 16 students student NOUN eblip-19158 139 17 did do AUX eblip-19158 139 18 not not PART eblip-19158 139 19 use use VERB eblip-19158 139 20 these these DET eblip-19158 139 21 criteria criterion NOUN eblip-19158 139 22 either—330 either—330 X eblip-19158 139 23 ( ( PUNCT eblip-19158 139 24 42.6 42.6 NUM eblip-19158 139 25 % % NOUN eblip-19158 139 26 ) ) PUNCT eblip-19158 139 27 annotations annotation NOUN eblip-19158 139 28 received receive VERB eblip-19158 139 29 a a DET eblip-19158 139 30 level level NOUN eblip-19158 139 31 0 0 NUM eblip-19158 139 32 for for ADP eblip-19158 139 33 relevance relevance NOUN eblip-19158 139 34 and and CCONJ eblip-19158 139 35 268 268 NUM eblip-19158 139 36 ( ( PUNCT eblip-19158 139 37 34.6 34.6 NUM eblip-19158 139 38 % % NOUN eblip-19158 139 39 ) ) PUNCT eblip-19158 139 40 annotations annotation NOUN eblip-19158 139 41 received receive VERB eblip-19158 139 42 a a DET eblip-19158 139 43 level level NOUN eblip-19158 139 44 0 0 NUM eblip-19158 139 45 for for ADP eblip-19158 139 46 authority authority NOUN eblip-19158 139 47 . . PUNCT eblip-19158 140 1 in in ADP eblip-19158 140 2 fact fact NOUN eblip-19158 140 3 , , PUNCT eblip-19158 140 4 in in ADP eblip-19158 140 5 every every DET eblip-19158 140 6 instance instance NOUN eblip-19158 140 7 except except SCONJ eblip-19158 140 8 authority authority NOUN eblip-19158 140 9 , , PUNCT eblip-19158 140 10 level level NOUN eblip-19158 140 11 0 0 NUM eblip-19158 140 12 was be AUX eblip-19158 140 13 the the DET eblip-19158 140 14 most most ADV eblip-19158 140 15 frequent frequent ADJ eblip-19158 140 16 score score NOUN eblip-19158 140 17 out out ADP eblip-19158 140 18 of of ADP eblip-19158 140 19 the the DET eblip-19158 140 20 three three NUM eblip-19158 140 21 possible possible ADJ eblip-19158 140 22 performance performance NOUN eblip-19158 140 23 levels level NOUN eblip-19158 140 24 . . PUNCT eblip-19158 140 25         SPACE eblip-19158 141 1 the the DET eblip-19158 141 2 rubric rubric NOUN eblip-19158 141 3 was be AUX eblip-19158 141 4 designed design VERB eblip-19158 141 5 in in ADP eblip-19158 141 6 such such DET eblip-19158 141 7 a a DET eblip-19158 141 8 way way NOUN eblip-19158 141 9 that that PRON eblip-19158 141 10 a a DET eblip-19158 141 11 level level NOUN eblip-19158 141 12 1 1 NUM eblip-19158 141 13 score score NOUN eblip-19158 141 14 indicates indicate VERB eblip-19158 141 15 that that SCONJ eblip-19158 141 16 students student NOUN eblip-19158 141 17 are be AUX eblip-19158 141 18 aware aware ADJ eblip-19158 141 19 that that SCONJ eblip-19158 141 20 a a DET eblip-19158 141 21 particular particular ADJ eblip-19158 141 22 evaluative evaluative ADJ eblip-19158 141 23 criterion criterion NOUN eblip-19158 141 24 exists exist VERB eblip-19158 141 25 and and CCONJ eblip-19158 141 26 attempt attempt VERB eblip-19158 141 27 to to PART eblip-19158 141 28 use use VERB eblip-19158 141 29 it it PRON eblip-19158 141 30 to to PART eblip-19158 141 31 assess assess VERB eblip-19158 141 32 the the DET eblip-19158 141 33 usefulness usefulness NOUN eblip-19158 141 34 and and CCONJ eblip-19158 141 35 appropriateness appropriateness NOUN eblip-19158 141 36 of of ADP eblip-19158 141 37 a a DET eblip-19158 141 38 source source NOUN eblip-19158 141 39 , , PUNCT eblip-19158 141 40 and and CCONJ eblip-19158 141 41 a a DET eblip-19158 141 42 level level NOUN eblip-19158 141 43 2 2 NUM eblip-19158 141 44 score score NOUN eblip-19158 141 45 indicates indicate VERB eblip-19158 141 46 that that SCONJ eblip-19158 141 47 students student NOUN eblip-19158 141 48 apply apply VERB eblip-19158 141 49 these these DET eblip-19158 141 50 criteria criterion NOUN eblip-19158 141 51 in in ADP eblip-19158 141 52 a a DET eblip-19158 141 53 critical critical ADJ eblip-19158 141 54 way way NOUN eblip-19158 141 55 . . PUNCT eblip-19158 142 1 when when SCONJ eblip-19158 142 2 level level NOUN eblip-19158 142 3 1 1 NUM eblip-19158 142 4 and and CCONJ eblip-19158 142 5 2 2 NUM eblip-19158 142 6 scores score NOUN eblip-19158 142 7 are be AUX eblip-19158 142 8 combined combine VERB eblip-19158 142 9 , , PUNCT eblip-19158 142 10 it it PRON eblip-19158 142 11 is be AUX eblip-19158 142 12 apparent apparent ADJ eblip-19158 142 13 that that SCONJ eblip-19158 142 14 87 87 NUM eblip-19158 142 15 annotations annotation NOUN eblip-19158 142 16 ( ( PUNCT eblip-19158 142 17 11 11 NUM eblip-19158 142 18 % % NOUN eblip-19158 142 19 ) ) PUNCT eblip-19158 142 20 use use VERB eblip-19158 142 21 the the DET eblip-19158 142 22 criterion criterion NOUN eblip-19158 142 23 currency currency NOUN eblip-19158 142 24 to to PART eblip-19158 142 25 evaluate evaluate VERB eblip-19158 142 26 the the DET eblip-19158 142 27 source source NOUN eblip-19158 142 28 , , PUNCT eblip-19158 142 29 158 158 NUM eblip-19158 142 30 annotations annotation NOUN eblip-19158 142 31 ( ( PUNCT eblip-19158 142 32 19.7 19.7 NUM eblip-19158 142 33 % % NOUN eblip-19158 142 34 ) ) PUNCT eblip-19158 142 35 use use VERB eblip-19158 142 36 the the DET eblip-19158 142 37 criterion criterion NOUN eblip-19158 142 38 purpose purpose NOUN eblip-19158 142 39 to to PART eblip-19158 142 40 evaluate evaluate VERB eblip-19158 142 41 the the DET eblip-19158 142 42 source source NOUN eblip-19158 142 43 , , PUNCT eblip-19158 142 44 198 198 NUM eblip-19158 142 45 annotations annotation NOUN eblip-19158 142 46 ( ( PUNCT eblip-19158 142 47 25.6 25.6 NUM eblip-19158 142 48 % % NOUN eblip-19158 142 49 ) ) PUNCT eblip-19158 142 50 use use VERB eblip-19158 142 51 the the DET eblip-19158 142 52 criterion criterion NOUN eblip-19158 142 53 accuracy accuracy NOUN eblip-19158 142 54 to to PART eblip-19158 142 55 evaluate evaluate VERB eblip-19158 142 56 the the DET eblip-19158 142 57 source source NOUN eblip-19158 142 58 , , PUNCT eblip-19158 142 59 443 443 NUM eblip-19158 142 60 annotations annotation NOUN eblip-19158 142 61 ( ( PUNCT eblip-19158 142 62 57.2 57.2 NUM eblip-19158 142 63 % % NOUN eblip-19158 142 64 ) ) PUNCT eblip-19158 142 65 use use VERB eblip-19158 142 66 the the DET eblip-19158 142 67 criterion criterion NOUN eblip-19158 142 68 relevance relevance NOUN eblip-19158 142 69 to to PART eblip-19158 142 70 evaluate evaluate VERB eblip-19158 142 71 the the DET eblip-19158 142 72 source source NOUN eblip-19158 142 73 , , PUNCT eblip-19158 142 74 and and CCONJ eblip-19158 142 75 505 505 NUM eblip-19158 142 76 annotations annotation NOUN eblip-19158 142 77 ( ( PUNCT eblip-19158 142 78 65.2 65.2 NUM eblip-19158 142 79 % % NOUN eblip-19158 142 80 ) ) PUNCT eblip-19158 142 81 use use VERB eblip-19158 142 82 the the DET eblip-19158 142 83 criterion criterion NOUN eblip-19158 142 84         SPACE eblip-19158 142 85 table table NOUN eblip-19158 142 86 2 2 NUM eblip-19158 142 87 interrater interrater NOUN eblip-19158 142 88 reliability reliability NOUN eblip-19158 142 89 from from ADP eblip-19158 142 90 training training NOUN eblip-19158 142 91 rounds round NOUN eblip-19158 142 92         SPACE eblip-19158 142 93 table table NOUN eblip-19158 142 94 3 3 NUM eblip-19158 142 95 scores score NOUN eblip-19158 142 96 for for ADP eblip-19158 142 97 evaluative evaluative ADJ eblip-19158 142 98 criteria criterion NOUN eblip-19158 142 99         SPACE eblip-19158 142 100 authority authority NOUN eblip-19158 142 101 to to PART eblip-19158 142 102 evaluate evaluate VERB eblip-19158 142 103 the the DET eblip-19158 142 104 source source NOUN eblip-19158 142 105 . . PUNCT eblip-19158 143 1 in in ADP eblip-19158 143 2 the the DET eblip-19158 143 3 instances instance NOUN eblip-19158 143 4 of of ADP eblip-19158 143 5 relevance relevance NOUN eblip-19158 143 6 and and CCONJ eblip-19158 143 7 authority authority NOUN eblip-19158 143 8 , , PUNCT eblip-19158 143 9 it it PRON eblip-19158 143 10 is be AUX eblip-19158 143 11 evident evident ADJ eblip-19158 143 12 that that SCONJ eblip-19158 143 13 the the DET eblip-19158 143 14 majority majority NOUN eblip-19158 143 15 of of ADP eblip-19158 143 16 students student NOUN eblip-19158 143 17 are be AUX eblip-19158 143 18 at at ADV eblip-19158 143 19 least least ADJ eblip-19158 143 20 aware aware ADJ eblip-19158 143 21 that that SCONJ eblip-19158 143 22 these these DET eblip-19158 143 23 criteria criterion NOUN eblip-19158 143 24 should should AUX eblip-19158 143 25 be be AUX eblip-19158 143 26 considered consider VERB eblip-19158 143 27 when when SCONJ eblip-19158 143 28 selecting select VERB eblip-19158 143 29 an an DET eblip-19158 143 30 information information NOUN eblip-19158 143 31 source source NOUN eblip-19158 143 32 . . PUNCT eblip-19158 143 33     SPACE eblip-19158 144 1 however however ADV eblip-19158 144 2 , , PUNCT eblip-19158 144 3 there there PRON eblip-19158 144 4 were be VERB eblip-19158 144 5 no no DET eblip-19158 144 6 instances instance NOUN eblip-19158 144 7 in in ADP eblip-19158 144 8 which which PRON eblip-19158 144 9 the the DET eblip-19158 144 10 majority majority NOUN eblip-19158 144 11 of of ADP eblip-19158 144 12 students student NOUN eblip-19158 144 13 applied apply VERB eblip-19158 144 14 the the DET eblip-19158 144 15 evaluative evaluative ADJ eblip-19158 144 16 criteria criterion NOUN eblip-19158 144 17 in in ADP eblip-19158 144 18 a a DET eblip-19158 144 19 critical critical ADJ eblip-19158 144 20 way way NOUN eblip-19158 144 21 , , PUNCT eblip-19158 144 22 the the DET eblip-19158 144 23 performance performance NOUN eblip-19158 144 24 required require VERB eblip-19158 144 25 to to PART eblip-19158 144 26 receive receive VERB eblip-19158 144 27 a a DET eblip-19158 144 28 level level NOUN eblip-19158 144 29 2 2 NUM eblip-19158 144 30 score score NOUN eblip-19158 144 31 . . PUNCT eblip-19158 145 1 for for ADP eblip-19158 145 2 currency currency NOUN eblip-19158 145 3 , , PUNCT eblip-19158 145 4 17 17 NUM eblip-19158 145 5 ( ( PUNCT eblip-19158 145 6 2 2 NUM eblip-19158 145 7 % % NOUN eblip-19158 145 8 ) ) PUNCT eblip-19158 145 9 of of ADP eblip-19158 145 10 annotations annotation NOUN eblip-19158 145 11 were be AUX eblip-19158 145 12 scored score VERB eblip-19158 145 13 level level NOUN eblip-19158 145 14 2 2 NUM eblip-19158 145 15 ; ; PUNCT eblip-19158 145 16 for for ADP eblip-19158 145 17 accuracy accuracy NOUN eblip-19158 145 18 , , PUNCT eblip-19158 145 19 14 14 NUM eblip-19158 145 20 ( ( PUNCT eblip-19158 145 21 1.8 1.8 NUM eblip-19158 145 22 % % NOUN eblip-19158 145 23 ) ) PUNCT eblip-19158 145 24 annotations annotation NOUN eblip-19158 145 25 were be AUX eblip-19158 145 26 scored score VERB eblip-19158 145 27 level level NOUN eblip-19158 145 28 2 2 NUM eblip-19158 145 29 ; ; PUNCT eblip-19158 145 30 and and CCONJ eblip-19158 145 31 even even ADV eblip-19158 145 32 fewer few ADJ eblip-19158 145 33 for for ADP eblip-19158 145 34 purpose purpose NOUN eblip-19158 145 35 — — PUNCT eblip-19158 145 36 only only ADV eblip-19158 145 37 6 6 NUM eblip-19158 145 38 annotations annotation NOUN eblip-19158 145 39 , , PUNCT eblip-19158 145 40 less less ADJ eblip-19158 145 41 than than ADP eblip-19158 145 42 1 1 NUM eblip-19158 145 43 % % NOUN eblip-19158 145 44 , , PUNCT eblip-19158 145 45 used use VERB eblip-19158 145 46 the the DET eblip-19158 145 47 criteria criterion NOUN eblip-19158 145 48 critically critically ADV eblip-19158 145 49 . . PUNCT eblip-19158 146 1 however however ADV eblip-19158 146 2 , , PUNCT eblip-19158 146 3 in in ADP eblip-19158 146 4 the the DET eblip-19158 146 5 case case NOUN eblip-19158 146 6 of of ADP eblip-19158 146 7 relevance relevance NOUN eblip-19158 146 8 and and CCONJ eblip-19158 146 9 authority authority NOUN eblip-19158 146 10 nearly nearly ADV eblip-19158 146 11 a a PRON eblip-19158 146 12 quarter quarter NOUN eblip-19158 146 13 of of ADP eblip-19158 146 14 the the DET eblip-19158 146 15 annotations annotation NOUN eblip-19158 146 16 applied apply VERB eblip-19158 146 17 the the DET eblip-19158 146 18 criteria criterion NOUN eblip-19158 146 19 critically critically ADV eblip-19158 146 20 , , PUNCT eblip-19158 146 21 at at ADP eblip-19158 146 22 180 180 NUM eblip-19158 146 23 annotations annotation NOUN eblip-19158 146 24 ( ( PUNCT eblip-19158 146 25 23.2 23.2 NUM eblip-19158 146 26 % % NOUN eblip-19158 146 27 ) ) PUNCT eblip-19158 146 28 and and CCONJ eblip-19158 146 29 189 189 NUM eblip-19158 146 30 annotations annotation NOUN eblip-19158 146 31 ( ( PUNCT eblip-19158 146 32 24.4 24.4 NUM eblip-19158 146 33 % % NOUN eblip-19158 146 34 ) ) PUNCT eblip-19158 146 35 , , PUNCT eblip-19158 146 36 respectively respectively ADV eblip-19158 146 37 . . PUNCT eblip-19158 146 38         SPACE eblip-19158 146 39 examining examine VERB eblip-19158 146 40 the the DET eblip-19158 146 41 annotations annotation NOUN eblip-19158 146 42 within within ADP eblip-19158 146 43 the the DET eblip-19158 146 44 context context NOUN eblip-19158 146 45 of of ADP eblip-19158 146 46 the the DET eblip-19158 146 47 bibliographies bibliography NOUN eblip-19158 146 48 in in ADP eblip-19158 146 49 which which PRON eblip-19158 146 50 they they PRON eblip-19158 146 51 appeared appear VERB eblip-19158 146 52 provides provide VERB eblip-19158 146 53 information information NOUN eblip-19158 146 54 on on ADP eblip-19158 146 55 how how SCONJ eblip-19158 146 56 students student NOUN eblip-19158 146 57 applied apply VERB eblip-19158 146 58 these these DET eblip-19158 146 59 evaluative evaluative ADJ eblip-19158 146 60 criteria criterion NOUN eblip-19158 146 61 across across ADP eblip-19158 146 62 their their PRON eblip-19158 146 63 entire entire ADJ eblip-19158 146 64 papers paper NOUN eblip-19158 146 65 . . PUNCT eblip-19158 147 1 each each PRON eblip-19158 147 2 of of ADP eblip-19158 147 3 the the DET eblip-19158 147 4 155 155 NUM eblip-19158 147 5 papers paper NOUN eblip-19158 147 6 examined examine VERB eblip-19158 147 7 contained contain VERB eblip-19158 147 8 up up ADP eblip-19158 147 9 to to PART eblip-19158 147 10 five five NUM eblip-19158 147 11 annotations annotation NOUN eblip-19158 147 12 , , PUNCT eblip-19158 147 13 meaning mean VERB eblip-19158 147 14 each each PRON eblip-19158 147 15 of of ADP eblip-19158 147 16 the the DET eblip-19158 147 17 evaluative evaluative ADJ eblip-19158 147 18 criteria criterion NOUN eblip-19158 147 19 could could AUX eblip-19158 147 20 receive receive VERB eblip-19158 147 21 a a DET eblip-19158 147 22 score score NOUN eblip-19158 147 23 of of ADP eblip-19158 147 24 up up ADP eblip-19158 147 25 to to PART eblip-19158 147 26 ten ten NUM eblip-19158 147 27 points point NOUN eblip-19158 147 28 — — PUNCT eblip-19158 147 29 two two NUM eblip-19158 147 30 points point NOUN eblip-19158 147 31 per per ADP eblip-19158 147 32 annotation annotation NOUN eblip-19158 147 33 . . PUNCT eblip-19158 148 1 the the DET eblip-19158 148 2 author author NOUN eblip-19158 148 3 totaled total VERB eblip-19158 148 4 the the DET eblip-19158 148 5 scores score NOUN eblip-19158 148 6 each each DET eblip-19158 148 7 paper paper NOUN eblip-19158 148 8 received receive VERB eblip-19158 148 9 for for ADP eblip-19158 148 10 the the DET eblip-19158 148 11 application application NOUN eblip-19158 148 12 of of ADP eblip-19158 148 13 the the DET eblip-19158 148 14 criteria criterion NOUN eblip-19158 148 15 currency currency NOUN eblip-19158 148 16 , , PUNCT eblip-19158 148 17 relevance relevance NOUN eblip-19158 148 18 , , PUNCT eblip-19158 148 19 accuracy accuracy NOUN eblip-19158 148 20 , , PUNCT eblip-19158 148 21 authority authority NOUN eblip-19158 148 22 , , PUNCT eblip-19158 148 23 and and CCONJ eblip-19158 148 24 purpose purpose NOUN eblip-19158 148 25 . . PUNCT eblip-19158 149 1 the the DET eblip-19158 149 2 following follow VERB eblip-19158 149 3 table table NOUN eblip-19158 149 4 contains contain VERB eblip-19158 149 5 the the DET eblip-19158 149 6 average average ADJ eblip-19158 149 7 total total ADJ eblip-19158 149 8 score score NOUN eblip-19158 149 9 for for ADP eblip-19158 149 10 each each DET eblip-19158 149 11 evaluative evaluative ADJ eblip-19158 149 12 criterion criterion NOUN eblip-19158 149 13 , , PUNCT eblip-19158 149 14 the the DET eblip-19158 149 15 maximum maximum ADJ eblip-19158 149 16 score score NOUN eblip-19158 149 17 for for ADP eblip-19158 149 18 each each DET eblip-19158 149 19 criterion criterion NOUN eblip-19158 149 20 , , PUNCT eblip-19158 149 21 and and CCONJ eblip-19158 149 22 the the DET eblip-19158 149 23 standard standard ADJ eblip-19158 149 24 deviation deviation NOUN eblip-19158 149 25 — — PUNCT eblip-19158 149 26 how how SCONJ eblip-19158 149 27 closely closely ADV eblip-19158 149 28 students student NOUN eblip-19158 149 29 clustered cluster VERB eblip-19158 149 30 to to ADP eblip-19158 149 31 the the DET eblip-19158 149 32 average average ADJ eblip-19158 149 33 score score NOUN eblip-19158 149 34 . . PUNCT eblip-19158 149 35     SPACE eblip-19158 150 1 it it PRON eblip-19158 150 2 is be AUX eblip-19158 150 3 interesting interesting ADJ eblip-19158 150 4 to to PART eblip-19158 150 5 note note VERB eblip-19158 150 6 that that SCONJ eblip-19158 150 7 the the DET eblip-19158 150 8 mean mean ADJ eblip-19158 150 9 scores score NOUN eblip-19158 150 10 for for ADP eblip-19158 150 11 currency currency NOUN eblip-19158 150 12 , , PUNCT eblip-19158 150 13 accuracy accuracy NOUN eblip-19158 150 14 , , PUNCT eblip-19158 150 15 and and CCONJ eblip-19158 150 16 purpose purpose NOUN eblip-19158 150 17 were be AUX eblip-19158 150 18 quite quite ADV eblip-19158 150 19 low low ADJ eblip-19158 150 20 ( ( PUNCT eblip-19158 150 21 0.66 0.66 NUM eblip-19158 150 22 , , PUNCT eblip-19158 150 23 1.35 1.35 NUM eblip-19158 150 24 , , PUNCT eblip-19158 150 25 and and CCONJ eblip-19158 150 26 1.15 1.15 NUM eblip-19158 150 27 out out ADP eblip-19158 150 28 of of ADP eblip-19158 150 29 10 10 NUM eblip-19158 150 30 , , PUNCT eblip-19158 150 31 respectively respectively ADV eblip-19158 150 32 ) ) PUNCT eblip-19158 150 33 , , PUNCT eblip-19158 150 34 and and CCONJ eblip-19158 150 35 that that SCONJ eblip-19158 150 36 the the DET eblip-19158 150 37 standard standard ADJ eblip-19158 150 38 deviations deviation NOUN eblip-19158 150 39 for for ADP eblip-19158 150 40 these these DET eblip-19158 150 41 criteria criterion NOUN eblip-19158 150 42 were be AUX eblip-19158 150 43 also also ADV eblip-19158 150 44 low low ADJ eblip-19158 150 45 ( ( PUNCT eblip-19158 150 46 1.47 1.47 NUM eblip-19158 150 47 , , PUNCT eblip-19158 150 48 1.61 1.61 NUM eblip-19158 150 49 . . PUNCT eblip-19158 150 50 , , PUNCT eblip-19158 150 51 and and CCONJ eblip-19158 150 52 1.67 1.67 NUM eblip-19158 150 53 , , PUNCT eblip-19158 150 54 respectively respectively ADV eblip-19158 150 55 ) ) PUNCT eblip-19158 150 56 . . PUNCT eblip-19158 151 1 a a DET eblip-19158 151 2 small small ADJ eblip-19158 151 3 standard standard ADJ eblip-19158 151 4 deviation deviation NOUN eblip-19158 151 5 means mean VERB eblip-19158 151 6 that that SCONJ eblip-19158 151 7 most most ADJ eblip-19158 151 8 of of ADP eblip-19158 151 9 the the DET eblip-19158 151 10 scores score NOUN eblip-19158 151 11 are be AUX eblip-19158 151 12 grouped group VERB eblip-19158 151 13 very very ADV eblip-19158 151 14 close close ADV eblip-19158 151 15 to to ADP eblip-19158 151 16 the the DET eblip-19158 151 17 average average ADJ eblip-19158 151 18 score score NOUN eblip-19158 151 19 , , PUNCT eblip-19158 151 20 with with ADP eblip-19158 151 21 only only ADV eblip-19158 151 22 a a DET eblip-19158 151 23 few few ADJ eblip-19158 151 24 outlying outlying ADJ eblip-19158 151 25 points point NOUN eblip-19158 151 26 ( ( PUNCT eblip-19158 151 27 hand hand NOUN eblip-19158 151 28 , , PUNCT eblip-19158 151 29 2008 2008 NUM eblip-19158 151 30 ) ) PUNCT eblip-19158 151 31 . . PUNCT eblip-19158 152 1 thus thus ADV eblip-19158 152 2 , , PUNCT eblip-19158 152 3 even even ADV eblip-19158 152 4 though though SCONJ eblip-19158 152 5 the the DET eblip-19158 152 6 range range NOUN eblip-19158 152 7 of of ADP eblip-19158 152 8 scores score NOUN eblip-19158 152 9 given give VERB eblip-19158 152 10 for for ADP eblip-19158 152 11 these these DET eblip-19158 152 12 criteria criterion NOUN eblip-19158 152 13 was be AUX eblip-19158 152 14 high high ADJ eblip-19158 152 15 , , PUNCT eblip-19158 152 16 the the DET eblip-19158 152 17 high high ADJ eblip-19158 152 18 scores score NOUN eblip-19158 152 19 were be AUX eblip-19158 152 20 outliers outlier NOUN eblip-19158 152 21 — — PUNCT eblip-19158 152 22 the the DET eblip-19158 152 23 majority majority NOUN eblip-19158 152 24 of of ADP eblip-19158 152 25 scores score NOUN eblip-19158 152 26 given give VERB eblip-19158 152 27 for for ADP eblip-19158 152 28 each each DET eblip-19158 152 29 annotation annotation NOUN eblip-19158 152 30 was be AUX eblip-19158 152 31 actually actually ADV eblip-19158 152 32 very very ADV eblip-19158 152 33 near near ADP eblip-19158 152 34 the the DET eblip-19158 152 35 low low ADJ eblip-19158 152 36 mean mean NOUN eblip-19158 152 37 scores score NOUN eblip-19158 152 38 . . PUNCT eblip-19158 153 1 this this PRON eblip-19158 153 2 further far ADV eblip-19158 153 3 reinforces reinforce VERB eblip-19158 153 4 the the DET eblip-19158 153 5 notion notion NOUN eblip-19158 153 6 that that SCONJ eblip-19158 153 7 students student NOUN eblip-19158 153 8 are be AUX eblip-19158 153 9 consistently consistently ADV eblip-19158 153 10 failing fail VERB eblip-19158 153 11 to to PART eblip-19158 153 12 critically critically ADV eblip-19158 153 13 apply apply VERB eblip-19158 153 14 the the DET eblip-19158 153 15 evaluative evaluative ADJ eblip-19158 153 16 criteria criterion NOUN eblip-19158 153 17 currency currency NOUN eblip-19158 153 18 , , PUNCT eblip-19158 153 19 accuracy accuracy NOUN eblip-19158 153 20 , , PUNCT eblip-19158 153 21 and and CCONJ eblip-19158 153 22 purpose purpose NOUN eblip-19158 153 23 in in ADP eblip-19158 153 24 their their PRON eblip-19158 153 25 annotations annotation NOUN eblip-19158 153 26 . . PUNCT eblip-19158 153 27         SPACE eblip-19158 154 1 discussion discussion NOUN eblip-19158 154 2     SPACE eblip-19158 154 3 overall overall ADV eblip-19158 154 4 , , PUNCT eblip-19158 154 5 it it PRON eblip-19158 154 6 appears appear VERB eblip-19158 154 7 that that SCONJ eblip-19158 154 8 the the DET eblip-19158 154 9 results result NOUN eblip-19158 154 10 of of ADP eblip-19158 154 11 this this DET eblip-19158 154 12 study study NOUN eblip-19158 154 13 support support VERB eblip-19158 154 14 the the DET eblip-19158 154 15 information information NOUN eblip-19158 154 16 literacy literacy NOUN eblip-19158 154 17 assessment assessment NOUN eblip-19158 154 18 literature literature NOUN eblip-19158 154 19 in in ADP eblip-19158 154 20 terms term NOUN eblip-19158 154 21 of of ADP eblip-19158 154 22 students student NOUN eblip-19158 154 23 struggling struggle VERB eblip-19158 154 24 with with ADP eblip-19158 154 25 the the DET eblip-19158 154 26 application application NOUN eblip-19158 154 27 of of ADP eblip-19158 154 28 critical critical ADJ eblip-19158 154 29 thinking thinking NOUN eblip-19158 154 30 skills skill NOUN eblip-19158 154 31 . . PUNCT eblip-19158 155 1 in in ADP eblip-19158 155 2 their their PRON eblip-19158 155 3 rubric rubric ADJ eblip-19158 155 4 assessment assessment NOUN eblip-19158 155 5 of of ADP eblip-19158 155 6 student student NOUN eblip-19158 155 7 writing writing NOUN eblip-19158 155 8 , , PUNCT eblip-19158 155 9 emmons emmons PROPN eblip-19158 155 10 and and CCONJ eblip-19158 155 11 martin martin PROPN eblip-19158 155 12 ( ( PUNCT eblip-19158 155 13 2002 2002 NUM eblip-19158 155 14 ) ) PUNCT eblip-19158 155 15 indicated indicate VERB eblip-19158 155 16 that that SCONJ eblip-19158 155 17 students student NOUN eblip-19158 155 18 had have VERB eblip-19158 155 19 particular particular ADJ eblip-19158 155 20 difficulties difficulty NOUN eblip-19158 155 21 with with ADP eblip-19158 155 22 identifying identify VERB eblip-19158 155 23 the the DET eblip-19158 155 24 purpose purpose NOUN eblip-19158 155 25 of of ADP eblip-19158 155 26 different different ADJ eblip-19158 155 27 kinds kind NOUN eblip-19158 155 28 of of ADP eblip-19158 155 29 sources source NOUN eblip-19158 155 30 ; ; PUNCT eblip-19158 155 31 knight knight NOUN eblip-19158 155 32 ( ( PUNCT eblip-19158 155 33 2006 2006 NUM eblip-19158 155 34 ) ) PUNCT eblip-19158 155 35 also also ADV eblip-19158 155 36 stated state VERB eblip-19158 155 37 that that SCONJ eblip-19158 155 38 source source NOUN eblip-19158 155 39 evaluation evaluation NOUN eblip-19158 155 40 was be AUX eblip-19158 155 41 a a DET eblip-19158 155 42 particular particular ADJ eblip-19158 155 43 issue issue NOUN eblip-19158 155 44 in in ADP eblip-19158 155 45 student student NOUN eblip-19158 155 46 writing writing NOUN eblip-19158 155 47 . . PUNCT eblip-19158 156 1 the the DET eblip-19158 156 2 eng eng PROPN eblip-19158 156 3 102 102 NUM eblip-19158 156 4 annotated annotate VERB eblip-19158 156 5 bibliographies bibliography NOUN eblip-19158 156 6 reinforce reinforce VERB eblip-19158 156 7 these these DET eblip-19158 156 8 ideas idea NOUN eblip-19158 156 9 — — PUNCT eblip-19158 156 10 the the DET eblip-19158 156 11 only only ADJ eblip-19158 156 12 evaluative evaluative ADJ eblip-19158 156 13 criteria criterion NOUN eblip-19158 156 14 students student NOUN eblip-19158 156 15 considered consider VERB eblip-19158 156 16 on on ADP eblip-19158 156 17 a a DET eblip-19158 156 18 consistent consistent ADJ eblip-19158 156 19 basis basis NOUN eblip-19158 156 20 was be AUX eblip-19158 156 21 the the DET eblip-19158 156 22 author author NOUN eblip-19158 156 23 of of ADP eblip-19158 156 24 their their PRON eblip-19158 156 25 source source NOUN eblip-19158 156 26 and and CCONJ eblip-19158 156 27 how how SCONJ eblip-19158 156 28 that that DET eblip-19158 156 29 source source NOUN eblip-19158 156 30 related relate VERB eblip-19158 156 31 to to ADP eblip-19158 156 32 their their PRON eblip-19158 156 33 research research NOUN eblip-19158 156 34 topics topic NOUN eblip-19158 156 35 . . PUNCT eblip-19158 157 1 this this DET eblip-19158 157 2 study study NOUN eblip-19158 157 3 therefore therefore ADV eblip-19158 157 4 contributes contribute VERB eblip-19158 157 5 to to ADP eblip-19158 157 6 the the DET eblip-19158 157 7 literature literature NOUN eblip-19158 157 8 on on ADP eblip-19158 157 9 student student NOUN eblip-19158 157 10 application application NOUN eblip-19158 157 11 of of ADP eblip-19158 157 12 critical critical ADJ eblip-19158 157 13 thinking thinking NOUN eblip-19158 157 14 skills skill NOUN eblip-19158 157 15 by by ADP eblip-19158 157 16 providing provide VERB eblip-19158 157 17 specific specific ADJ eblip-19158 157 18 data datum NOUN eblip-19158 157 19 about about ADP eblip-19158 157 20 the the DET eblip-19158 157 21 degree degree NOUN eblip-19158 157 22 to to PART eblip-19158 157 23 which which PRON eblip-19158 157 24 students student NOUN eblip-19158 157 25 evaluate evaluate VERB eblip-19158 157 26 their their PRON eblip-19158 157 27 information information NOUN eblip-19158 157 28 resources resource NOUN eblip-19158 157 29 . . PUNCT eblip-19158 157 30     SPACE eblip-19158 157 31 table table NOUN eblip-19158 157 32 4 4 NUM eblip-19158 157 33 breakdown breakdown NOUN eblip-19158 157 34 of of ADP eblip-19158 157 35 scores score NOUN eblip-19158 157 36 for for ADP eblip-19158 157 37 evaluative evaluative ADJ eblip-19158 157 38 criteria criterion NOUN eblip-19158 157 39 in in ADP eblip-19158 157 40 student student NOUN eblip-19158 157 41 papers paper NOUN eblip-19158 157 42     SPACE eblip-19158 157 43 a a DET eblip-19158 157 44 practical practical ADJ eblip-19158 157 45 implication implication NOUN eblip-19158 157 46 of of ADP eblip-19158 157 47 this this DET eblip-19158 157 48 research research NOUN eblip-19158 157 49 is be AUX eblip-19158 157 50 an an DET eblip-19158 157 51 indication indication NOUN eblip-19158 157 52 of of ADP eblip-19158 157 53 where where SCONJ eblip-19158 157 54 reinforced reinforce VERB eblip-19158 157 55 information information NOUN eblip-19158 157 56 literacy literacy NOUN eblip-19158 157 57 instruction instruction NOUN eblip-19158 157 58 might might AUX eblip-19158 157 59 be be AUX eblip-19158 157 60 beneficial beneficial ADJ eblip-19158 157 61 to to ADP eblip-19158 157 62 students student NOUN eblip-19158 157 63 . . PUNCT eblip-19158 158 1 it it PRON eblip-19158 158 2 is be AUX eblip-19158 158 3 clear clear ADJ eblip-19158 158 4 that that SCONJ eblip-19158 158 5 students student NOUN eblip-19158 158 6 need need VERB eblip-19158 158 7 more more ADJ eblip-19158 158 8 support support NOUN eblip-19158 158 9 in in ADP eblip-19158 158 10 learning learn VERB eblip-19158 158 11 how how SCONJ eblip-19158 158 12 to to PART eblip-19158 158 13 identify identify VERB eblip-19158 158 14 whether whether SCONJ eblip-19158 158 15 a a DET eblip-19158 158 16 source source NOUN eblip-19158 158 17 is be AUX eblip-19158 158 18 appropriate appropriate ADJ eblip-19158 158 19 and and CCONJ eblip-19158 158 20 to to PART eblip-19158 158 21 articulate articulate VERB eblip-19158 158 22 how how SCONJ eblip-19158 158 23 it it PRON eblip-19158 158 24 contributes contribute VERB eblip-19158 158 25 to to ADP eblip-19158 158 26 their their PRON eblip-19158 158 27 argument argument NOUN eblip-19158 158 28 . . PUNCT eblip-19158 159 1 this this PRON eblip-19158 159 2 is be AUX eblip-19158 159 3 a a DET eblip-19158 159 4 complex complex ADJ eblip-19158 159 5 skill skill NOUN eblip-19158 159 6 set set VERB eblip-19158 159 7 and and CCONJ eblip-19158 159 8 needs need VERB eblip-19158 159 9 to to PART eblip-19158 159 10 be be AUX eblip-19158 159 11 taught teach VERB eblip-19158 159 12 in in ADP eblip-19158 159 13 both both CCONJ eblip-19158 159 14 the the DET eblip-19158 159 15 eng eng PROPN eblip-19158 159 16 102 102 NUM eblip-19158 159 17 classroom classroom NOUN eblip-19158 159 18 as as ADV eblip-19158 159 19 well well ADV eblip-19158 159 20 as as ADP eblip-19158 159 21 in in ADP eblip-19158 159 22 the the DET eblip-19158 159 23 library library NOUN eblip-19158 159 24 instruction instruction NOUN eblip-19158 159 25 session session NOUN eblip-19158 159 26 . . PUNCT eblip-19158 160 1 the the DET eblip-19158 160 2 first first ADJ eblip-19158 160 3 step step NOUN eblip-19158 160 4 the the DET eblip-19158 160 5 author author NOUN eblip-19158 160 6 took take VERB eblip-19158 160 7 was be AUX eblip-19158 160 8 to to PART eblip-19158 160 9 share share VERB eblip-19158 160 10 the the DET eblip-19158 160 11 results result NOUN eblip-19158 160 12 of of ADP eblip-19158 160 13 the the DET eblip-19158 160 14 rubric rubric ADJ eblip-19158 160 15 assessment assessment NOUN eblip-19158 160 16 with with ADP eblip-19158 160 17 the the DET eblip-19158 160 18 eng eng PROPN eblip-19158 160 19 102 102 NUM eblip-19158 160 20 coordinator coordinator NOUN eblip-19158 160 21 and and CCONJ eblip-19158 160 22 to to PART eblip-19158 160 23 initiate initiate VERB eblip-19158 160 24 a a DET eblip-19158 160 25 conversation conversation NOUN eblip-19158 160 26 about about ADP eblip-19158 160 27 what what PRON eblip-19158 160 28 elements element NOUN eblip-19158 160 29 of of ADP eblip-19158 160 30 source source NOUN eblip-19158 160 31 evaluation evaluation NOUN eblip-19158 160 32 the the DET eblip-19158 160 33 composition composition NOUN eblip-19158 160 34 program program NOUN eblip-19158 160 35 would would AUX eblip-19158 160 36 like like VERB eblip-19158 160 37 to to PART eblip-19158 160 38 emphasize emphasize VERB eblip-19158 160 39 in in ADP eblip-19158 160 40 future future ADJ eblip-19158 160 41 semesters semester NOUN eblip-19158 160 42 . . PUNCT eblip-19158 161 1 the the DET eblip-19158 161 2 author author NOUN eblip-19158 161 3 also also ADV eblip-19158 161 4 plans plan VERB eblip-19158 161 5 to to PART eblip-19158 161 6 share share VERB eblip-19158 161 7 these these DET eblip-19158 161 8 results result NOUN eblip-19158 161 9 with with ADP eblip-19158 161 10 the the DET eblip-19158 161 11 eng eng PROPN eblip-19158 161 12 102 102 NUM eblip-19158 161 13 instructors instructor NOUN eblip-19158 161 14 at at ADP eblip-19158 161 15 their their PRON eblip-19158 161 16 fall fall NOUN eblip-19158 161 17 orientation orientation NOUN eblip-19158 161 18 meeting meeting NOUN eblip-19158 161 19 and and CCONJ eblip-19158 161 20 to to ADP eblip-19158 161 21 stress stress NOUN eblip-19158 161 22 areas area NOUN eblip-19158 161 23 where where SCONJ eblip-19158 161 24 the the DET eblip-19158 161 25 library library NOUN eblip-19158 161 26 can can AUX eblip-19158 161 27 help help VERB eblip-19158 161 28 improve improve VERB eblip-19158 161 29 student student NOUN eblip-19158 161 30 performance performance NOUN eblip-19158 161 31 , , PUNCT eblip-19158 161 32 as as ADV eblip-19158 161 33 well well ADV eblip-19158 161 34 as as ADP eblip-19158 161 35 steps step NOUN eblip-19158 161 36 the the DET eblip-19158 161 37 eng eng PROPN eblip-19158 161 38 102 102 NUM eblip-19158 161 39 instructors instructor NOUN eblip-19158 161 40 can can AUX eblip-19158 161 41 take take VERB eblip-19158 161 42 in in ADP eblip-19158 161 43 their their PRON eblip-19158 161 44 own own ADJ eblip-19158 161 45 classrooms classroom NOUN eblip-19158 161 46 to to PART eblip-19158 161 47 help help VERB eblip-19158 161 48 students student NOUN eblip-19158 161 49 become become VERB eblip-19158 161 50 more more ADV eblip-19158 161 51 critical critical ADJ eblip-19158 161 52 of of ADP eblip-19158 161 53 their their PRON eblip-19158 161 54 information information NOUN eblip-19158 161 55 sources source NOUN eblip-19158 161 56 . . PUNCT eblip-19158 162 1 the the DET eblip-19158 162 2 author author NOUN eblip-19158 162 3 had have AUX eblip-19158 162 4 previously previously ADV eblip-19158 162 5 created create VERB eblip-19158 162 6 an an DET eblip-19158 162 7 eng eng PROPN eblip-19158 162 8 102 102 NUM eblip-19158 162 9 instructor instructor NOUN eblip-19158 162 10 portal portal NOUN eblip-19158 162 11 — — PUNCT eblip-19158 162 12 a a DET eblip-19158 162 13 website website NOUN eblip-19158 162 14 available available ADJ eblip-19158 162 15 to to ADP eblip-19158 162 16 eng eng PROPN eblip-19158 162 17 102 102 NUM eblip-19158 162 18 instructors instructor NOUN eblip-19158 162 19 that that PRON eblip-19158 162 20 provides provide VERB eblip-19158 162 21 information information NOUN eblip-19158 162 22 literacy literacy NOUN eblip-19158 162 23 activities activity NOUN eblip-19158 162 24 and and CCONJ eblip-19158 162 25 assignments assignment NOUN eblip-19158 162 26 to to PART eblip-19158 162 27 support support VERB eblip-19158 162 28 the the DET eblip-19158 162 29 learning learn VERB eblip-19158 162 30 outcomes outcome NOUN eblip-19158 162 31 of of ADP eblip-19158 162 32 eng eng PROPN eblip-19158 162 33 102 102 NUM eblip-19158 162 34 . . PUNCT eblip-19158 163 1 the the DET eblip-19158 163 2 portal portal PROPN eblip-19158 163 3 was be AUX eblip-19158 163 4 highly highly ADV eblip-19158 163 5 utilized utilize VERB eblip-19158 163 6 throughout throughout ADP eblip-19158 163 7 the the DET eblip-19158 163 8 2012 2012 NUM eblip-19158 163 9 - - SYM eblip-19158 163 10 2013 2013 NUM eblip-19158 163 11 school school NOUN eblip-19158 163 12 year year NOUN eblip-19158 163 13 , , PUNCT eblip-19158 163 14 but but CCONJ eblip-19158 163 15 the the DET eblip-19158 163 16 learning learn VERB eblip-19158 163 17 outcome outcome NOUN eblip-19158 163 18 of of ADP eblip-19158 163 19 source source NOUN eblip-19158 163 20 evaluation evaluation NOUN eblip-19158 163 21 was be AUX eblip-19158 163 22 perhaps perhaps ADV eblip-19158 163 23 buried bury VERB eblip-19158 163 24 beneath beneath ADP eblip-19158 163 25 a a DET eblip-19158 163 26 myriad myriad NOUN eblip-19158 163 27 of of ADP eblip-19158 163 28 other other ADJ eblip-19158 163 29 learning learn VERB eblip-19158 163 30 outcomes outcome NOUN eblip-19158 163 31 , , PUNCT eblip-19158 163 32 and and CCONJ eblip-19158 163 33 was be AUX eblip-19158 163 34 therefore therefore ADV eblip-19158 163 35 not not PART eblip-19158 163 36 sufficiently sufficiently ADV eblip-19158 163 37 highlighted highlight VERB eblip-19158 163 38 . . PUNCT eblip-19158 164 1 the the DET eblip-19158 164 2 author author NOUN eblip-19158 164 3 plans plan VERB eblip-19158 164 4 to to PART eblip-19158 164 5 streamline streamline VERB eblip-19158 164 6 the the DET eblip-19158 164 7 instructor instructor NOUN eblip-19158 164 8 portal portal PROPN eblip-19158 164 9 content content NOUN eblip-19158 164 10 to to PART eblip-19158 164 11 emphasize emphasize VERB eblip-19158 164 12 this this DET eblip-19158 164 13 skill skill NOUN eblip-19158 164 14 set set VERB eblip-19158 164 15 , , PUNCT eblip-19158 164 16 and and CCONJ eblip-19158 164 17 to to PART eblip-19158 164 18 discuss discuss VERB eblip-19158 164 19 with with ADP eblip-19158 164 20 eng eng PROPN eblip-19158 164 21 102 102 NUM eblip-19158 164 22 instructors instructor NOUN eblip-19158 164 23 at at ADP eblip-19158 164 24 orientation orientation NOUN eblip-19158 164 25 the the DET eblip-19158 164 26 possible possible ADJ eblip-19158 164 27 ways way NOUN eblip-19158 164 28 they they PRON eblip-19158 164 29 can can AUX eblip-19158 164 30 integrate integrate VERB eblip-19158 164 31 these these DET eblip-19158 164 32 activities activity NOUN eblip-19158 164 33 into into ADP eblip-19158 164 34 their their PRON eblip-19158 164 35 teaching teaching NOUN eblip-19158 164 36 . . PUNCT eblip-19158 164 37                                     SPACE eblip-19158 164 38 in in ADP eblip-19158 164 39 addition addition NOUN eblip-19158 164 40 , , PUNCT eblip-19158 164 41 the the DET eblip-19158 164 42 library library NOUN eblip-19158 164 43 needs need VERB eblip-19158 164 44 to to PART eblip-19158 164 45 target target VERB eblip-19158 164 46 source source NOUN eblip-19158 164 47 evaluation evaluation NOUN eblip-19158 164 48 more more ADV eblip-19158 164 49 consistently consistently ADV eblip-19158 164 50 during during ADP eblip-19158 164 51 the the DET eblip-19158 164 52 instruction instruction NOUN eblip-19158 164 53 sessions session NOUN eblip-19158 164 54 for for ADP eblip-19158 164 55 eng eng PROPN eblip-19158 164 56 102 102 NUM eblip-19158 164 57 . . PUNCT eblip-19158 165 1 according accord VERB eblip-19158 165 2 to to ADP eblip-19158 165 3 a a DET eblip-19158 165 4 survey survey NOUN eblip-19158 165 5 of of ADP eblip-19158 165 6 the the DET eblip-19158 165 7 librarians librarian NOUN eblip-19158 165 8 who who PRON eblip-19158 165 9 worked work VERB eblip-19158 165 10 with with ADP eblip-19158 165 11 eng eng PROPN eblip-19158 165 12 102 102 NUM eblip-19158 165 13 , , PUNCT eblip-19158 165 14 source source NOUN eblip-19158 165 15 evaluation evaluation NOUN eblip-19158 165 16 was be AUX eblip-19158 165 17 not not PART eblip-19158 165 18 covered cover VERB eblip-19158 165 19 across across ADP eblip-19158 165 20 all all DET eblip-19158 165 21 sections section NOUN eblip-19158 165 22 that that PRON eblip-19158 165 23 came come VERB eblip-19158 165 24 to to ADP eblip-19158 165 25 the the DET eblip-19158 165 26 library library NOUN eblip-19158 165 27 , , PUNCT eblip-19158 165 28 and and CCONJ eblip-19158 165 29 , , PUNCT eblip-19158 165 30 when when SCONJ eblip-19158 165 31 it it PRON eblip-19158 165 32 was be AUX eblip-19158 165 33 taught teach VERB eblip-19158 165 34 , , PUNCT eblip-19158 165 35 it it PRON eblip-19158 165 36 usually usually ADV eblip-19158 165 37 consisted consist VERB eblip-19158 165 38 of of ADP eblip-19158 165 39 briefly briefly ADV eblip-19158 165 40 showing show VERB eblip-19158 165 41 the the DET eblip-19158 165 42 students student NOUN eblip-19158 165 43 the the DET eblip-19158 165 44 source source NOUN eblip-19158 165 45 evaluation evaluation NOUN eblip-19158 165 46 handout handout NOUN eblip-19158 165 47 at at ADP eblip-19158 165 48 the the DET eblip-19158 165 49 end end NOUN eblip-19158 165 50 of of ADP eblip-19158 165 51 the the DET eblip-19158 165 52 session session NOUN eblip-19158 165 53 . . PUNCT eblip-19158 166 1 the the DET eblip-19158 166 2 results result NOUN eblip-19158 166 3 of of ADP eblip-19158 166 4 this this DET eblip-19158 166 5 study study NOUN eblip-19158 166 6 clearly clearly ADV eblip-19158 166 7 demonstrate demonstrate VERB eblip-19158 166 8 that that SCONJ eblip-19158 166 9 this this PRON eblip-19158 166 10 is be AUX eblip-19158 166 11 not not PART eblip-19158 166 12 enough enough ADJ eblip-19158 166 13 instruction instruction NOUN eblip-19158 166 14 on on ADP eblip-19158 166 15 source source NOUN eblip-19158 166 16 evaluation evaluation NOUN eblip-19158 166 17 , , PUNCT eblip-19158 166 18 and and CCONJ eblip-19158 166 19 the the DET eblip-19158 166 20 author author NOUN eblip-19158 166 21 has have AUX eblip-19158 166 22 met meet VERB eblip-19158 166 23 with with ADP eblip-19158 166 24 the the DET eblip-19158 166 25 instruction instruction NOUN eblip-19158 166 26 librarians librarian NOUN eblip-19158 166 27 to to PART eblip-19158 166 28 discuss discuss VERB eblip-19158 166 29 ways way NOUN eblip-19158 166 30 in in ADP eblip-19158 166 31 which which PRON eblip-19158 166 32 we we PRON eblip-19158 166 33 can can AUX eblip-19158 166 34 better well ADV eblip-19158 166 35 teach teach VERB eblip-19158 166 36 source source NOUN eblip-19158 166 37 evaluation evaluation NOUN eblip-19158 166 38 in in ADP eblip-19158 166 39 our our PRON eblip-19158 166 40 sessions session NOUN eblip-19158 166 41 . . PUNCT eblip-19158 167 1 the the DET eblip-19158 167 2 group group NOUN eblip-19158 167 3 agreed agree VERB eblip-19158 167 4 that that SCONJ eblip-19158 167 5 in in ADP eblip-19158 167 6 order order NOUN eblip-19158 167 7 to to PART eblip-19158 167 8 more more ADV eblip-19158 167 9 fully fully ADV eblip-19158 167 10 instruct instruct VERB eblip-19158 167 11 on on ADP eblip-19158 167 12 source source NOUN eblip-19158 167 13 evaluation evaluation NOUN eblip-19158 167 14 , , PUNCT eblip-19158 167 15 other other ADJ eblip-19158 167 16 learning learn VERB eblip-19158 167 17 outcomes outcome NOUN eblip-19158 167 18 that that PRON eblip-19158 167 19 are be AUX eblip-19158 167 20 currently currently ADV eblip-19158 167 21 taught teach VERB eblip-19158 167 22 in in ADP eblip-19158 167 23 the the DET eblip-19158 167 24 library library NOUN eblip-19158 167 25 session session NOUN eblip-19158 167 26 should should AUX eblip-19158 167 27 be be AUX eblip-19158 167 28 moved move VERB eblip-19158 167 29 out out ADP eblip-19158 167 30 of of ADP eblip-19158 167 31 the the DET eblip-19158 167 32 face face NOUN eblip-19158 167 33 - - PUNCT eblip-19158 167 34 to to ADP eblip-19158 167 35 - - PUNCT eblip-19158 167 36 face face NOUN eblip-19158 167 37 classroom classroom NOUN eblip-19158 167 38 and and CCONJ eblip-19158 167 39 into into ADP eblip-19158 167 40 the the DET eblip-19158 167 41 virtual virtual ADJ eblip-19158 167 42 one one NUM eblip-19158 167 43 . . PUNCT eblip-19158 168 1 the the DET eblip-19158 168 2 author author NOUN eblip-19158 168 3 is be AUX eblip-19158 168 4 in in ADP eblip-19158 168 5 the the DET eblip-19158 168 6 process process NOUN eblip-19158 168 7 of of ADP eblip-19158 168 8 creating create VERB eblip-19158 168 9 a a DET eblip-19158 168 10 web web NOUN eblip-19158 168 11 tutorial tutorial NOUN eblip-19158 168 12 that that PRON eblip-19158 168 13 will will AUX eblip-19158 168 14 have have VERB eblip-19158 168 15 students student NOUN eblip-19158 168 16 work work VERB eblip-19158 168 17 through through ADP eblip-19158 168 18 selecting select VERB eblip-19158 168 19 and and CCONJ eblip-19158 168 20 narrowing narrow VERB eblip-19158 168 21 a a DET eblip-19158 168 22 research research NOUN eblip-19158 168 23 topic topic NOUN eblip-19158 168 24 prior prior ADV eblip-19158 168 25 to to ADP eblip-19158 168 26 coming come VERB eblip-19158 168 27 to to ADP eblip-19158 168 28 the the DET eblip-19158 168 29 library library NOUN eblip-19158 168 30 for for ADP eblip-19158 168 31 instruction instruction NOUN eblip-19158 168 32 . . PUNCT eblip-19158 169 1 the the DET eblip-19158 169 2 class class NOUN eblip-19158 169 3 time time NOUN eblip-19158 169 4 formerly formerly ADV eblip-19158 169 5 spent spend VERB eblip-19158 169 6 on on ADP eblip-19158 169 7 topic topic NOUN eblip-19158 169 8 exploration exploration NOUN eblip-19158 169 9 can can AUX eblip-19158 169 10 then then ADV eblip-19158 169 11 be be AUX eblip-19158 169 12 better well ADV eblip-19158 169 13 spent spend VERB eblip-19158 169 14 on on ADP eblip-19158 169 15 helping help VERB eblip-19158 169 16 students student NOUN eblip-19158 169 17 navigate navigate VERB eblip-19158 169 18 through through ADP eblip-19158 169 19 search search NOUN eblip-19158 169 20 results result NOUN eblip-19158 169 21 and and CCONJ eblip-19158 169 22 apply apply VERB eblip-19158 169 23 evaluative evaluative ADJ eblip-19158 169 24 criteria criterion NOUN eblip-19158 169 25 to to ADP eblip-19158 169 26 their their PRON eblip-19158 169 27 potential potential ADJ eblip-19158 169 28 sources source NOUN eblip-19158 169 29 . . PUNCT eblip-19158 169 30         SPACE eblip-19158 170 1 one one NUM eblip-19158 170 2 limitation limitation NOUN eblip-19158 170 3 to to ADP eblip-19158 170 4 this this DET eblip-19158 170 5 study study NOUN eblip-19158 170 6 is be AUX eblip-19158 170 7 the the DET eblip-19158 170 8 restricted restricted ADJ eblip-19158 170 9 face face NOUN eblip-19158 170 10 - - PUNCT eblip-19158 170 11 to to ADP eblip-19158 170 12 - - PUNCT eblip-19158 170 13 face face NOUN eblip-19158 170 14 time time NOUN eblip-19158 170 15 that that PRON eblip-19158 170 16 librarians librarian NOUN eblip-19158 170 17 have have VERB eblip-19158 170 18 with with ADP eblip-19158 170 19 students student NOUN eblip-19158 170 20 ; ; PUNCT eblip-19158 170 21 the the DET eblip-19158 170 22 library library NOUN eblip-19158 170 23 can can AUX eblip-19158 170 24 only only ADV eblip-19158 170 25 support support VERB eblip-19158 170 26 teaching teach VERB eblip-19158 170 27 one one NUM eblip-19158 170 28 75 75 NUM eblip-19158 170 29 minute minute NOUN eblip-19158 170 30 instruction instruction NOUN eblip-19158 170 31 session session NOUN eblip-19158 170 32 for for ADP eblip-19158 170 33 each each DET eblip-19158 170 34 section section NOUN eblip-19158 170 35 of of ADP eblip-19158 170 36 eng eng PROPN eblip-19158 170 37 102 102 NUM eblip-19158 170 38 . . PUNCT eblip-19158 171 1 the the DET eblip-19158 171 2 use use NOUN eblip-19158 171 3 of of ADP eblip-19158 171 4 a a DET eblip-19158 171 5 web web NOUN eblip-19158 171 6 tutorial tutorial NOUN eblip-19158 171 7 to to PART eblip-19158 171 8 free free VERB eblip-19158 171 9 up up ADP eblip-19158 171 10 class class NOUN eblip-19158 171 11 time time NOUN eblip-19158 171 12 , , PUNCT eblip-19158 171 13 as as ADV eblip-19158 171 14 well well ADV eblip-19158 171 15 as as ADP eblip-19158 171 16 providing provide VERB eblip-19158 171 17 eng eng PROPN eblip-19158 171 18 102 102 NUM eblip-19158 171 19 instructors instructor NOUN eblip-19158 171 20 with with ADP eblip-19158 171 21 more more ADJ eblip-19158 171 22 resources resource NOUN eblip-19158 171 23 on on ADP eblip-19158 171 24 source source NOUN eblip-19158 171 25 evaluation evaluation NOUN eblip-19158 171 26 is be AUX eblip-19158 171 27 intended intend VERB eblip-19158 171 28 to to PART eblip-19158 171 29 alleviate alleviate VERB eblip-19158 171 30 some some PRON eblip-19158 171 31 of of ADP eblip-19158 171 32 this this DET eblip-19158 171 33 pressure pressure NOUN eblip-19158 171 34 . . PUNCT eblip-19158 172 1 the the DET eblip-19158 172 2 library library NOUN eblip-19158 172 3 is be AUX eblip-19158 172 4 also also ADV eblip-19158 172 5 limited limit VERB eblip-19158 172 6 in in ADP eblip-19158 172 7 terms term NOUN eblip-19158 172 8 of of ADP eblip-19158 172 9 what what PRON eblip-19158 172 10 individual individual ADJ eblip-19158 172 11 eng eng PROPN eblip-19158 172 12 102 102 NUM eblip-19158 172 13 instructors instructor NOUN eblip-19158 172 14 emphasize emphasize VERB eblip-19158 172 15 when when SCONJ eblip-19158 172 16 they they PRON eblip-19158 172 17 assign assign VERB eblip-19158 172 18 the the DET eblip-19158 172 19 annotated annotate VERB eblip-19158 172 20 bibliography bibliography NOUN eblip-19158 172 21 project project NOUN eblip-19158 172 22 . . PUNCT eblip-19158 173 1 since since SCONJ eblip-19158 173 2 there there PRON eblip-19158 173 3 is be VERB eblip-19158 173 4 no no DET eblip-19158 173 5 shared share VERB eblip-19158 173 6 rubric rubric NOUN eblip-19158 173 7 that that SCONJ eblip-19158 173 8 eng eng PROPN eblip-19158 173 9 102 102 NUM eblip-19158 173 10 instructors instructor NOUN eblip-19158 173 11 have have VERB eblip-19158 173 12 to to PART eblip-19158 173 13 use use VERB eblip-19158 173 14 for for ADP eblip-19158 173 15 the the DET eblip-19158 173 16 annotated annotate VERB eblip-19158 173 17 bibliographies bibliography NOUN eblip-19158 173 18 , , PUNCT eblip-19158 173 19 it it PRON eblip-19158 173 20 is be AUX eblip-19158 173 21 plausible plausible ADJ eblip-19158 173 22 that that SCONJ eblip-19158 173 23 there there PRON eblip-19158 173 24 is be VERB eblip-19158 173 25 a a DET eblip-19158 173 26 large large ADJ eblip-19158 173 27 amount amount NOUN eblip-19158 173 28 of of ADP eblip-19158 173 29 variability variability NOUN eblip-19158 173 30 in in ADP eblip-19158 173 31 what what PRON eblip-19158 173 32 the the DET eblip-19158 173 33 classroom classroom NOUN eblip-19158 173 34 teachers teacher NOUN eblip-19158 173 35 instruct instruct VERB eblip-19158 173 36 their their PRON eblip-19158 173 37 students student NOUN eblip-19158 173 38 to to PART eblip-19158 173 39 do do VERB eblip-19158 173 40 in in ADP eblip-19158 173 41 their their PRON eblip-19158 173 42 annotations annotation NOUN eblip-19158 173 43 . . PUNCT eblip-19158 174 1 including include VERB eblip-19158 174 2 eng eng PROPN eblip-19158 174 3 102 102 NUM eblip-19158 174 4 instructors instructor NOUN eblip-19158 174 5 in in ADP eblip-19158 174 6 the the DET eblip-19158 174 7 conversation conversation NOUN eblip-19158 174 8 about about ADP eblip-19158 174 9 how how SCONJ eblip-19158 174 10 students student NOUN eblip-19158 174 11 struggle struggle VERB eblip-19158 174 12 with with ADP eblip-19158 174 13 source source NOUN eblip-19158 174 14 evaluation evaluation NOUN eblip-19158 174 15 — — PUNCT eblip-19158 174 16 via via ADP eblip-19158 174 17 fall fall NOUN eblip-19158 174 18 orientation orientation NOUN eblip-19158 174 19 and and CCONJ eblip-19158 174 20 the the DET eblip-19158 174 21 instructor instructor NOUN eblip-19158 174 22 portal portal NOUN eblip-19158 174 23 — — PUNCT eblip-19158 174 24 is be AUX eblip-19158 174 25 one one NUM eblip-19158 174 26 step step NOUN eblip-19158 174 27 in in ADP eblip-19158 174 28 improving improve VERB eblip-19158 174 29 this this DET eblip-19158 174 30 issue issue NOUN eblip-19158 174 31 . . PUNCT eblip-19158 174 32         SPACE eblip-19158 175 1 finally finally ADV eblip-19158 175 2 , , PUNCT eblip-19158 175 3 the the DET eblip-19158 175 4 process process NOUN eblip-19158 175 5 of of ADP eblip-19158 175 6 applying apply VERB eblip-19158 175 7 the the DET eblip-19158 175 8 information information NOUN eblip-19158 175 9 literacy literacy NOUN eblip-19158 175 10 rubric rubric NOUN eblip-19158 175 11 can can AUX eblip-19158 175 12 also also ADV eblip-19158 175 13 be be AUX eblip-19158 175 14 improved improve VERB eblip-19158 175 15 in in ADP eblip-19158 175 16 the the DET eblip-19158 175 17 future future NOUN eblip-19158 175 18 . . PUNCT eblip-19158 176 1 the the DET eblip-19158 176 2 training training NOUN eblip-19158 176 3 sessions session NOUN eblip-19158 176 4 failed fail VERB eblip-19158 176 5 to to PART eblip-19158 176 6 establish establish VERB eblip-19158 176 7 sufficient sufficient ADJ eblip-19158 176 8 inter inter ADJ eblip-19158 176 9 - - ADJ eblip-19158 176 10 rater rater ADJ eblip-19158 176 11 reliability reliability NOUN eblip-19158 176 12 , , PUNCT eblip-19158 176 13 and and CCONJ eblip-19158 176 14 this this PRON eblip-19158 176 15 was be AUX eblip-19158 176 16 in in ADP eblip-19158 176 17 large large ADJ eblip-19158 176 18 part part NOUN eblip-19158 176 19 due due ADP eblip-19158 176 20 to to ADP eblip-19158 176 21 not not PART eblip-19158 176 22 enough enough ADJ eblip-19158 176 23 time time NOUN eblip-19158 176 24 allotted allot VERB eblip-19158 176 25 for for ADP eblip-19158 176 26 training training NOUN eblip-19158 176 27 , , PUNCT eblip-19158 176 28 particularly particularly ADV eblip-19158 176 29 in in ADP eblip-19158 176 30 discussing discuss VERB eblip-19158 176 31 differences difference NOUN eblip-19158 176 32 in in ADP eblip-19158 176 33 scores score NOUN eblip-19158 176 34 . . PUNCT eblip-19158 177 1 two two NUM eblip-19158 177 2 2 2 NUM eblip-19158 177 3 - - PUNCT eblip-19158 177 4 hour hour NOUN eblip-19158 177 5 training training NOUN eblip-19158 177 6 sessions session NOUN eblip-19158 177 7 ( ( PUNCT eblip-19158 177 8 over over ADP eblip-19158 177 9 two two NUM eblip-19158 177 10 days day NOUN eblip-19158 177 11 ) ) PUNCT eblip-19158 177 12 was be AUX eblip-19158 177 13 all all DET eblip-19158 177 14 the the DET eblip-19158 177 15 group group NOUN eblip-19158 177 16 could could AUX eblip-19158 177 17 commit commit VERB eblip-19158 177 18 to to ADP eblip-19158 177 19 the the DET eblip-19158 177 20 project project NOUN eblip-19158 177 21 , , PUNCT eblip-19158 177 22 but but CCONJ eblip-19158 177 23 in in ADP eblip-19158 177 24 the the DET eblip-19158 177 25 future future NOUN eblip-19158 177 26 , , PUNCT eblip-19158 177 27 the the DET eblip-19158 177 28 author author NOUN eblip-19158 177 29 will will AUX eblip-19158 177 30 emphasize emphasize VERB eblip-19158 177 31 the the DET eblip-19158 177 32 importance importance NOUN eblip-19158 177 33 of of ADP eblip-19158 177 34 a a DET eblip-19158 177 35 full full ADJ eblip-19158 177 36 day day NOUN eblip-19158 177 37 of of ADP eblip-19158 177 38 training training NOUN eblip-19158 177 39 so so SCONJ eblip-19158 177 40 that that SCONJ eblip-19158 177 41 more more ADJ eblip-19158 177 42 time time NOUN eblip-19158 177 43 can can AUX eblip-19158 177 44 be be AUX eblip-19158 177 45 spent spend VERB eblip-19158 177 46 working work VERB eblip-19158 177 47 through through ADP eblip-19158 177 48 differences difference NOUN eblip-19158 177 49 . . PUNCT eblip-19158 177 50     SPACE eblip-19158 177 51 additional additional ADJ eblip-19158 177 52 factors factor NOUN eblip-19158 177 53 that that PRON eblip-19158 177 54 might might AUX eblip-19158 177 55 influence influence VERB eblip-19158 177 56 student student NOUN eblip-19158 177 57 performance performance NOUN eblip-19158 177 58 should should AUX eblip-19158 177 59 also also ADV eblip-19158 177 60 be be AUX eblip-19158 177 61 examined examine VERB eblip-19158 177 62 . . PUNCT eblip-19158 178 1 the the DET eblip-19158 178 2 author author NOUN eblip-19158 178 3 did do AUX eblip-19158 178 4 gather gather VERB eblip-19158 178 5 information information NOUN eblip-19158 178 6 about about ADP eblip-19158 178 7 student student NOUN eblip-19158 178 8 performances performance NOUN eblip-19158 178 9 based base VERB eblip-19158 178 10 on on ADP eblip-19158 178 11 their their PRON eblip-19158 178 12 section section NOUN eblip-19158 178 13 type type NOUN eblip-19158 178 14 — — PUNCT eblip-19158 178 15 regular regular ADJ eblip-19158 178 16 , , PUNCT eblip-19158 178 17 themed theme VERB eblip-19158 178 18 , , PUNCT eblip-19158 178 19 or or CCONJ eblip-19158 178 20 distance distance NOUN eblip-19158 178 21 education education NOUN eblip-19158 178 22 — — PUNCT eblip-19158 178 23 as as ADV eblip-19158 178 24 well well ADV eblip-19158 178 25 as as ADP eblip-19158 178 26 differences difference NOUN eblip-19158 178 27 in in ADP eblip-19158 178 28 scores score NOUN eblip-19158 178 29 between between ADP eblip-19158 178 30 eng eng PROPN eblip-19158 178 31 102 102 NUM eblip-19158 178 32 classes class NOUN eblip-19158 178 33 that that PRON eblip-19158 178 34 came come VERB eblip-19158 178 35 to to ADP eblip-19158 178 36 the the DET eblip-19158 178 37 library library NOUN eblip-19158 178 38 for for ADP eblip-19158 178 39 instruction instruction NOUN eblip-19158 178 40 versus versus ADP eblip-19158 178 41 those those PRON eblip-19158 178 42 that that PRON eblip-19158 178 43 did do VERB eblip-19158 178 44 not not PART eblip-19158 178 45 ; ; PUNCT eblip-19158 178 46 though though SCONJ eblip-19158 178 47 it it PRON eblip-19158 178 48 was be AUX eblip-19158 178 49 required require VERB eblip-19158 178 50 to to PART eblip-19158 178 51 schedule schedule VERB eblip-19158 178 52 a a DET eblip-19158 178 53 library library NOUN eblip-19158 178 54 instruction instruction NOUN eblip-19158 178 55 session session NOUN eblip-19158 178 56 , , PUNCT eblip-19158 178 57 ten ten NUM eblip-19158 178 58 sections section NOUN eblip-19158 178 59 did do AUX eblip-19158 178 60 not not PART eblip-19158 178 61 come come VERB eblip-19158 178 62 to to ADP eblip-19158 178 63 the the DET eblip-19158 178 64 library library NOUN eblip-19158 178 65 . . PUNCT eblip-19158 179 1 more more ADJ eblip-19158 179 2 in in ADP eblip-19158 179 3 - - PUNCT eblip-19158 179 4 depth depth NOUN eblip-19158 179 5 statistical statistical ADJ eblip-19158 179 6 analysis analysis NOUN eblip-19158 179 7 needs need VERB eblip-19158 179 8 to to PART eblip-19158 179 9 be be AUX eblip-19158 179 10 done do VERB eblip-19158 179 11 in in ADP eblip-19158 179 12 order order NOUN eblip-19158 179 13 to to PART eblip-19158 179 14 interpret interpret VERB eblip-19158 179 15 that that DET eblip-19158 179 16 information information NOUN eblip-19158 179 17 and and CCONJ eblip-19158 179 18 the the DET eblip-19158 179 19 impact impact NOUN eblip-19158 179 20 , , PUNCT eblip-19158 179 21 if if SCONJ eblip-19158 179 22 any any PRON eblip-19158 179 23 , , PUNCT eblip-19158 179 24 that that SCONJ eblip-19158 179 25 these these DET eblip-19158 179 26 factors factor NOUN eblip-19158 179 27 have have VERB eblip-19158 179 28 on on ADP eblip-19158 179 29 student student NOUN eblip-19158 179 30 performance performance NOUN eblip-19158 179 31 . . PUNCT eblip-19158 179 32         SPACE eblip-19158 180 1 conclusions conclusion NOUN eblip-19158 180 2     SPACE eblip-19158 180 3 this this DET eblip-19158 180 4 study study NOUN eblip-19158 180 5 intended intend VERB eblip-19158 180 6 to to PART eblip-19158 180 7 accomplish accomplish VERB eblip-19158 180 8 two two NUM eblip-19158 180 9 goals goal NOUN eblip-19158 180 10 . . PUNCT eblip-19158 181 1 the the DET eblip-19158 181 2 first first ADJ eblip-19158 181 3 aim aim NOUN eblip-19158 181 4 of of ADP eblip-19158 181 5 this this DET eblip-19158 181 6 study study NOUN eblip-19158 181 7 was be AUX eblip-19158 181 8 to to PART eblip-19158 181 9 gather gather VERB eblip-19158 181 10 evidence evidence NOUN eblip-19158 181 11 about about ADP eblip-19158 181 12 how how SCONJ eblip-19158 181 13 students student NOUN eblip-19158 181 14 apply apply VERB eblip-19158 181 15 information information NOUN eblip-19158 181 16 literacy literacy NOUN eblip-19158 181 17 skills skill NOUN eblip-19158 181 18 in in ADP eblip-19158 181 19 the the DET eblip-19158 181 20 context context NOUN eblip-19158 181 21 of of ADP eblip-19158 181 22 an an DET eblip-19158 181 23 authentic authentic ADJ eblip-19158 181 24 assessment assessment NOUN eblip-19158 181 25 activity activity NOUN eblip-19158 181 26 , , PUNCT eblip-19158 181 27 and and CCONJ eblip-19158 181 28 to to PART eblip-19158 181 29 be be AUX eblip-19158 181 30 able able ADJ eblip-19158 181 31 to to PART eblip-19158 181 32 use use VERB eblip-19158 181 33 that that DET eblip-19158 181 34 information information NOUN eblip-19158 181 35 to to PART eblip-19158 181 36 establish establish VERB eblip-19158 181 37 a a DET eblip-19158 181 38 baseline baseline NOUN eblip-19158 181 39 from from ADP eblip-19158 181 40 which which PRON eblip-19158 181 41 to to PART eblip-19158 181 42 design design VERB eblip-19158 181 43 future future ADJ eblip-19158 181 44 library library NOUN eblip-19158 181 45 instruction instruction NOUN eblip-19158 181 46 . . PUNCT eblip-19158 182 1 the the DET eblip-19158 182 2 overall overall ADJ eblip-19158 182 3 scores score NOUN eblip-19158 182 4 on on ADP eblip-19158 182 5 the the DET eblip-19158 182 6 rubric rubric NOUN eblip-19158 182 7 indicate indicate VERB eblip-19158 182 8 that that SCONJ eblip-19158 182 9 there there PRON eblip-19158 182 10 is be VERB eblip-19158 182 11 considerable considerable ADJ eblip-19158 182 12 need need NOUN eblip-19158 182 13 for for ADP eblip-19158 182 14 further further ADJ eblip-19158 182 15 instruction instruction NOUN eblip-19158 182 16 on on ADP eblip-19158 182 17 the the DET eblip-19158 182 18 concept concept NOUN eblip-19158 182 19 of of ADP eblip-19158 182 20 how how SCONJ eblip-19158 182 21 and and CCONJ eblip-19158 182 22 why why SCONJ eblip-19158 182 23 to to PART eblip-19158 182 24 apply apply VERB eblip-19158 182 25 evaluative evaluative ADJ eblip-19158 182 26 criteria criterion NOUN eblip-19158 182 27 when when SCONJ eblip-19158 182 28 selecting select VERB eblip-19158 182 29 information information NOUN eblip-19158 182 30 resources resource NOUN eblip-19158 182 31 . . PUNCT eblip-19158 183 1 clearly clearly ADV eblip-19158 183 2 , , PUNCT eblip-19158 183 3 one one NUM eblip-19158 183 4 instruction instruction NOUN eblip-19158 183 5 session session NOUN eblip-19158 183 6 is be AUX eblip-19158 183 7 not not PART eblip-19158 183 8 enough enough ADJ eblip-19158 183 9 time time NOUN eblip-19158 183 10 to to PART eblip-19158 183 11 sufficiently sufficiently ADV eblip-19158 183 12 introduce introduce VERB eblip-19158 183 13 this this DET eblip-19158 183 14 concept concept NOUN eblip-19158 183 15 — — PUNCT eblip-19158 183 16 teaching teach VERB eblip-19158 183 17 source source NOUN eblip-19158 183 18 evaluation evaluation NOUN eblip-19158 183 19 is be AUX eblip-19158 183 20 a a DET eblip-19158 183 21 responsibility responsibility NOUN eblip-19158 183 22 that that PRON eblip-19158 183 23 should should AUX eblip-19158 183 24 be be AUX eblip-19158 183 25 shared share VERB eblip-19158 183 26 between between ADP eblip-19158 183 27 the the DET eblip-19158 183 28 eng eng PROPN eblip-19158 183 29 102 102 NUM eblip-19158 183 30 instructors instructor NOUN eblip-19158 183 31 and and CCONJ eblip-19158 183 32 the the DET eblip-19158 183 33 librarian librarian NOUN eblip-19158 183 34 in in ADP eblip-19158 183 35 the the DET eblip-19158 183 36 instruction instruction NOUN eblip-19158 183 37 session session NOUN eblip-19158 183 38 . . PUNCT eblip-19158 184 1 working work VERB eblip-19158 184 2 with with ADP eblip-19158 184 3 the the DET eblip-19158 184 4 eng eng PROPN eblip-19158 184 5 102 102 NUM eblip-19158 184 6 program program NOUN eblip-19158 184 7 to to PART eblip-19158 184 8 help help VERB eblip-19158 184 9 instructors instructor NOUN eblip-19158 184 10 embed embed VERB eblip-19158 184 11 this this DET eblip-19158 184 12 information information NOUN eblip-19158 184 13 literacy literacy NOUN eblip-19158 184 14 skill skill NOUN eblip-19158 184 15 set set VERB eblip-19158 184 16 within within ADP eblip-19158 184 17 their their PRON eblip-19158 184 18 own own ADJ eblip-19158 184 19 classrooms classroom NOUN eblip-19158 184 20 is be AUX eblip-19158 184 21 an an DET eblip-19158 184 22 important important ADJ eblip-19158 184 23 first first ADJ eblip-19158 184 24 step step NOUN eblip-19158 184 25 in in ADP eblip-19158 184 26 helping help VERB eblip-19158 184 27 students student NOUN eblip-19158 184 28 succeed succeed VERB eblip-19158 184 29 . . PUNCT eblip-19158 184 30         SPACE eblip-19158 184 31 a a DET eblip-19158 184 32 second second ADJ eblip-19158 184 33 aim aim NOUN eblip-19158 184 34 of of ADP eblip-19158 184 35 this this DET eblip-19158 184 36 study study NOUN eblip-19158 184 37 was be AUX eblip-19158 184 38 to to PART eblip-19158 184 39 provide provide VERB eblip-19158 184 40 a a DET eblip-19158 184 41 case case NOUN eblip-19158 184 42 study study NOUN eblip-19158 184 43 of of ADP eblip-19158 184 44 how how SCONJ eblip-19158 184 45 to to PART eblip-19158 184 46 create create VERB eblip-19158 184 47 and and CCONJ eblip-19158 184 48 apply apply VERB eblip-19158 184 49 a a DET eblip-19158 184 50 rubric rubric NOUN eblip-19158 184 51 to to PART eblip-19158 184 52 evaluate evaluate VERB eblip-19158 184 53 student student NOUN eblip-19158 184 54 information information NOUN eblip-19158 184 55 literacy literacy NOUN eblip-19158 184 56 skills skill NOUN eblip-19158 184 57 . . PUNCT eblip-19158 185 1 to to ADP eblip-19158 185 2 that that DET eblip-19158 185 3 end end NOUN eblip-19158 185 4 , , PUNCT eblip-19158 185 5 the the DET eblip-19158 185 6 author author NOUN eblip-19158 185 7 provided provide VERB eblip-19158 185 8 a a DET eblip-19158 185 9 transparent transparent ADJ eblip-19158 185 10 view view NOUN eblip-19158 185 11 of of ADP eblip-19158 185 12 the the DET eblip-19158 185 13 process process NOUN eblip-19158 185 14 of of ADP eblip-19158 185 15 rubric rubric ADJ eblip-19158 185 16 development development NOUN eblip-19158 185 17 and and CCONJ eblip-19158 185 18 training training NOUN eblip-19158 185 19 scorers scorer NOUN eblip-19158 185 20 to to PART eblip-19158 185 21 use use VERB eblip-19158 185 22 the the DET eblip-19158 185 23 rubric rubric NOUN eblip-19158 185 24 . . PUNCT eblip-19158 186 1 several several ADJ eblip-19158 186 2 lessons lesson NOUN eblip-19158 186 3 were be AUX eblip-19158 186 4 learned learn VERB eblip-19158 186 5 during during ADP eblip-19158 186 6 this this DET eblip-19158 186 7 process process NOUN eblip-19158 186 8 , , PUNCT eblip-19158 186 9 such such ADJ eblip-19158 186 10 as as ADP eblip-19158 186 11 how how SCONJ eblip-19158 186 12 to to PART eblip-19158 186 13 move move VERB eblip-19158 186 14 forward forward ADV eblip-19158 186 15 when when SCONJ eblip-19158 186 16 inter inter ADJ eblip-19158 186 17 - - ADJ eblip-19158 186 18 rater rater ADJ eblip-19158 186 19 reliability reliability NOUN eblip-19158 186 20 was be AUX eblip-19158 186 21 less less ADJ eblip-19158 186 22 than than ADP eblip-19158 186 23 desirable desirable ADJ eblip-19158 186 24 . . PUNCT eblip-19158 187 1 it it PRON eblip-19158 187 2 is be AUX eblip-19158 187 3 the the DET eblip-19158 187 4 author author NOUN eblip-19158 187 5 ’s ’s PART eblip-19158 187 6 hope hope NOUN eblip-19158 187 7 that that SCONJ eblip-19158 187 8 others other NOUN eblip-19158 187 9 will will AUX eblip-19158 187 10 build build VERB eblip-19158 187 11 upon upon SCONJ eblip-19158 187 12 the the DET eblip-19158 187 13 successes success NOUN eblip-19158 187 14 and and CCONJ eblip-19158 187 15 learn learn VERB eblip-19158 187 16 from from ADP eblip-19158 187 17 the the DET eblip-19158 187 18 limitations limitation NOUN eblip-19158 187 19 of of ADP eblip-19158 187 20 this this DET eblip-19158 187 21 study study NOUN eblip-19158 187 22 in in ADP eblip-19158 187 23 order order NOUN eblip-19158 187 24 to to PART eblip-19158 187 25 continue continue VERB eblip-19158 187 26 the the DET eblip-19158 187 27 important important ADJ eblip-19158 187 28 discussion discussion NOUN eblip-19158 187 29 of of ADP eblip-19158 187 30 authentic authentic ADJ eblip-19158 187 31 assessment assessment NOUN eblip-19158 187 32 of of ADP eblip-19158 187 33 information information NOUN eblip-19158 187 34 literacy literacy NOUN eblip-19158 187 35 skills skill NOUN eblip-19158 187 36 and and CCONJ eblip-19158 187 37 the the DET eblip-19158 187 38 ways way NOUN eblip-19158 187 39 in in ADP eblip-19158 187 40 which which PRON eblip-19158 187 41 academic academic ADJ eblip-19158 187 42 libraries library NOUN eblip-19158 187 43 can can AUX eblip-19158 187 44 better well ADV eblip-19158 187 45 support support VERB eblip-19158 187 46 student student NOUN eblip-19158 187 47 learning learning NOUN eblip-19158 187 48 . . PUNCT eblip-19158 187 49     SPACE eblip-19158 188 1 acknowledgements acknowledgement NOUN eblip-19158 188 2     SPACE eblip-19158 188 3 the the DET eblip-19158 188 4 author author NOUN eblip-19158 188 5 would would AUX eblip-19158 188 6 like like VERB eblip-19158 188 7 to to PART eblip-19158 188 8 thank thank VERB eblip-19158 188 9 anne anne PROPN eblip-19158 188 10 zald zald PROPN eblip-19158 188 11 , , PUNCT eblip-19158 188 12 cheryl cheryl PROPN eblip-19158 188 13 taranto taranto PROPN eblip-19158 188 14 , , PUNCT eblip-19158 188 15 and and CCONJ eblip-19158 188 16 samantha samantha PROPN eblip-19158 188 17 godbey godbey PROPN eblip-19158 188 18 for for ADP eblip-19158 188 19 their their PRON eblip-19158 188 20 assistance assistance NOUN eblip-19158 188 21 in in ADP eblip-19158 188 22 scoring scoring NOUN eblip-19158 188 23 student student NOUN eblip-19158 188 24 work work NOUN eblip-19158 188 25 using use VERB eblip-19158 188 26 the the DET eblip-19158 188 27 information information NOUN eblip-19158 188 28 literacy literacy NOUN eblip-19158 188 29 rubric rubric NOUN eblip-19158 188 30 ; ; PUNCT eblip-19158 188 31 ruby ruby PROPN eblip-19158 188 32 fowler fowler NOUN eblip-19158 188 33 for for ADP eblip-19158 188 34 providing provide VERB eblip-19158 188 35 the the DET eblip-19158 188 36 de de ADJ eblip-19158 188 37 - - ADJ eblip-19158 188 38 identified identify VERB eblip-19158 188 39 copies copy NOUN eblip-19158 188 40 of of ADP eblip-19158 188 41 annotated annotate VERB eblip-19158 188 42 bibliographies bibliography NOUN eblip-19158 188 43 ; ; PUNCT eblip-19158 188 44 jen jen PROPN eblip-19158 188 45 fabbi fabbi VERB eblip-19158 188 46 for for ADP eblip-19158 188 47 her her PRON eblip-19158 188 48 guidance guidance NOUN eblip-19158 188 49 throughout throughout ADP eblip-19158 188 50 the the DET eblip-19158 188 51 project project NOUN eblip-19158 188 52 ; ; PUNCT eblip-19158 188 53 and and CCONJ eblip-19158 188 54 megan megan NOUN eblip-19158 188 55 oakleaf oakleaf NOUN eblip-19158 188 56 for for ADP eblip-19158 188 57 leading lead VERB eblip-19158 188 58 the the DET eblip-19158 188 59 assessment assessment NOUN eblip-19158 188 60 workshop workshop NOUN eblip-19158 188 61 that that PRON eblip-19158 188 62 led lead VERB eblip-19158 188 63 to to ADP eblip-19158 188 64 the the DET eblip-19158 188 65 revision revision NOUN eblip-19158 188 66 of of ADP eblip-19158 188 67 the the DET eblip-19158 188 68 first first ADJ eblip-19158 188 69 draft draft NOUN eblip-19158 188 70 of of ADP eblip-19158 188 71 the the DET eblip-19158 188 72 rubric rubric NOUN eblip-19158 188 73 . . PUNCT eblip-19158 188 74             SPACE eblip-19158 189 1 references reference NOUN eblip-19158 189 2     SPACE eblip-19158 189 3 avery avery NOUN eblip-19158 189 4 , , PUNCT eblip-19158 189 5 e.f e.f PROPN eblip-19158 189 6 . . PROPN eblip-19158 189 7 ( ( PUNCT eblip-19158 189 8 2003 2003 NUM eblip-19158 189 9 ) ) PUNCT eblip-19158 189 10 . . PUNCT eblip-19158 190 1 assessing assess VERB eblip-19158 190 2 information information NOUN eblip-19158 190 3 literacy literacy NOUN eblip-19158 190 4 instruction instruction NOUN eblip-19158 190 5 . . PUNCT eblip-19158 190 6       SPACE eblip-19158 190 7 in in ADP eblip-19158 190 8 e.f e.f PROPN eblip-19158 190 9 . . PROPN eblip-19158 190 10 avery avery PROPN eblip-19158 190 11 ( ( PUNCT eblip-19158 190 12 ed ed NOUN eblip-19158 190 13 . . PUNCT eblip-19158 190 14 ) ) PUNCT eblip-19158 190 15 , , PUNCT eblip-19158 190 16 assessing assess VERB eblip-19158 190 17 student student NOUN eblip-19158 190 18 learning learn VERB eblip-19158 190 19 outcomes outcome NOUN eblip-19158 190 20 for for ADP eblip-19158 190 21 information information NOUN eblip-19158 190 22 literacy literacy NOUN eblip-19158 190 23 instruction instruction NOUN eblip-19158 190 24 in in ADP eblip-19158 190 25 academic academic ADJ eblip-19158 190 26 institutions institution NOUN eblip-19158 190 27 ( ( PUNCT eblip-19158 190 28 pp pp ADJ eblip-19158 190 29 . . PUNCT eblip-19158 191 1 1 1 NUM eblip-19158 191 2 - - SYM eblip-19158 191 3 5 5 NUM eblip-19158 191 4 ) ) PUNCT eblip-19158 191 5 . . PUNCT eblip-19158 191 6     SPACE eblip-19158 192 1 chicago chicago PROPN eblip-19158 192 2 , , PUNCT eblip-19158 192 3 il il PROPN eblip-19158 192 4 : : PUNCT eblip-19158 192 5 acrl acrl PROPN eblip-19158 192 6 . . PUNCT eblip-19158 192 7     SPACE eblip-19158 193 1 arter arter NOUN eblip-19158 193 2 , , PUNCT eblip-19158 193 3 j. j. PROPN eblip-19158 193 4 a. a. PROPN eblip-19158 193 5 & & CCONJ eblip-19158 193 6 mctighe mctighe PROPN eblip-19158 193 7 , , PUNCT eblip-19158 193 8 j. j. PROPN eblip-19158 193 9 ( ( PUNCT eblip-19158 193 10 2001 2001 NUM eblip-19158 193 11 ) ) PUNCT eblip-19158 193 12 . . PUNCT eblip-19158 194 1 scoring score VERB eblip-19158 194 2 rubrics rubric NOUN eblip-19158 194 3 in in ADP eblip-19158 194 4 the the DET eblip-19158 194 5 classroom classroom NOUN eblip-19158 194 6 : : PUNCT eblip-19158 194 7 using use VERB eblip-19158 194 8 performance performance NOUN eblip-19158 194 9 criteria criterion NOUN eblip-19158 194 10 for for ADP eblip-19158 194 11 assessing assess VERB eblip-19158 194 12 and and CCONJ eblip-19158 194 13 improving improve VERB eblip-19158 194 14 student student NOUN eblip-19158 194 15 performance performance NOUN eblip-19158 194 16 . . PUNCT eblip-19158 195 1 thousand thousand NUM eblip-19158 195 2 oaks oak NOUN eblip-19158 195 3 , , PUNCT eblip-19158 195 4 ca ca PROPN eblip-19158 195 5 : : PUNCT eblip-19158 195 6 corwin corwin PROPN eblip-19158 195 7 press press PROPN eblip-19158 195 8 . . PUNCT eblip-19158 195 9     SPACE eblip-19158 196 1 callison callison PROPN eblip-19158 196 2 , , PUNCT eblip-19158 196 3 d. d. PROPN eblip-19158 196 4 ( ( PUNCT eblip-19158 196 5 2000 2000 NUM eblip-19158 196 6 ) ) PUNCT eblip-19158 196 7 . . PUNCT eblip-19158 197 1 rubrics rubric NOUN eblip-19158 197 2 . . PUNCT eblip-19158 198 1 school school NOUN eblip-19158 198 2 library library NOUN eblip-19158 198 3 media medium NOUN eblip-19158 198 4 activities activity NOUN eblip-19158 198 5 monthly monthly ADV eblip-19158 198 6 , , PUNCT eblip-19158 198 7 17(2 17(2 NUM eblip-19158 198 8 ) ) PUNCT eblip-19158 198 9 , , PUNCT eblip-19158 198 10 34 34 NUM eblip-19158 198 11 - - SYM eblip-19158 198 12 36 36 NUM eblip-19158 198 13 , , PUNCT eblip-19158 198 14 42 42 NUM eblip-19158 198 15 . . PUNCT eblip-19158 198 16     SPACE eblip-19158 199 1 choinski choinski ADJ eblip-19158 199 2 , , PUNCT eblip-19158 199 3 e. e. PROPN eblip-19158 199 4 , , PUNCT eblip-19158 199 5 mark mark PROPN eblip-19158 199 6 , , PUNCT eblip-19158 199 7 a. a. PROPN eblip-19158 199 8 e. e. PROPN eblip-19158 199 9 , , PUNCT eblip-19158 199 10 & & CCONJ eblip-19158 199 11 murphey murphey PROPN eblip-19158 199 12 , , PUNCT eblip-19158 199 13 m. m. NOUN eblip-19158 199 14 ( ( PUNCT eblip-19158 199 15 2003 2003 NUM eblip-19158 199 16 ) ) PUNCT eblip-19158 199 17 . . PUNCT eblip-19158 200 1 assessment assessment NOUN eblip-19158 200 2 with with ADP eblip-19158 200 3 rubrics rubric NOUN eblip-19158 200 4 : : PUNCT eblip-19158 200 5 efficient efficient ADJ eblip-19158 200 6 and and CCONJ eblip-19158 200 7 objective objective ADJ eblip-19158 200 8 means mean NOUN eblip-19158 200 9 of of ADP eblip-19158 200 10 assessing assess VERB eblip-19158 200 11 student student NOUN eblip-19158 200 12 outcomes outcome NOUN eblip-19158 200 13 in in ADP eblip-19158 200 14 an an DET eblip-19158 200 15 information information NOUN eblip-19158 200 16 resources resource NOUN eblip-19158 200 17 class class NOUN eblip-19158 200 18 . . PUNCT eblip-19158 201 1 portal portal PROPN eblip-19158 201 2 : : PUNCT eblip-19158 201 3 libraries library NOUN eblip-19158 201 4 & & CCONJ eblip-19158 201 5 the the DET eblip-19158 201 6 academy academy PROPN eblip-19158 201 7 , , PUNCT eblip-19158 201 8 3(4 3(4 NUM eblip-19158 201 9 ) ) PUNCT eblip-19158 201 10 , , PUNCT eblip-19158 201 11 563 563 NUM eblip-19158 201 12 . . PUNCT eblip-19158 201 13     SPACE eblip-19158 202 1 cohen cohen PROPN eblip-19158 202 2 , , PUNCT eblip-19158 202 3 j. j. PROPN eblip-19158 202 4 ( ( PUNCT eblip-19158 202 5 1960 1960 NUM eblip-19158 202 6 ) ) PUNCT eblip-19158 202 7 . . PUNCT eblip-19158 203 1 a a DET eblip-19158 203 2 coefficient coefficient NOUN eblip-19158 203 3 of of ADP eblip-19158 203 4 agreement agreement NOUN eblip-19158 203 5 for for ADP eblip-19158 203 6 nominal nominal ADJ eblip-19158 203 7 scales scale NOUN eblip-19158 203 8 . . PUNCT eblip-19158 203 9     SPACE eblip-19158 203 10 educational educational ADJ eblip-19158 203 11 and and CCONJ eblip-19158 203 12 psychological psychological ADJ eblip-19158 203 13 measurement measurement NOUN eblip-19158 203 14 , , PUNCT eblip-19158 203 15 20(1 20(1 NUM eblip-19158 203 16 ) ) PUNCT eblip-19158 203 17 , , PUNCT eblip-19158 203 18 37 37 NUM eblip-19158 203 19 - - SYM eblip-19158 203 20 46 46 NUM eblip-19158 203 21 . . PUNCT eblip-19158 203 22     SPACE eblip-19158 203 23 conger conger NOUN eblip-19158 203 24 , , PUNCT eblip-19158 203 25 a.j a.j PROPN eblip-19158 203 26 . . PROPN eblip-19158 203 27 ( ( PUNCT eblip-19158 203 28 1980 1980 NUM eblip-19158 203 29 ) ) PUNCT eblip-19158 203 30 . . PUNCT eblip-19158 204 1 integration integration NOUN eblip-19158 204 2 and and CCONJ eblip-19158 204 3 generalization generalization NOUN eblip-19158 204 4 of of ADP eblip-19158 204 5 kappas kappa NOUN eblip-19158 204 6 for for ADP eblip-19158 204 7 multiple multiple ADJ eblip-19158 204 8 raters rater NOUN eblip-19158 204 9 . . PUNCT eblip-19158 204 10     SPACE eblip-19158 205 1 psychological psychological ADJ eblip-19158 205 2 bulletin bulletin NOUN eblip-19158 205 3 , , PUNCT eblip-19158 205 4 88(2 88(2 NUM eblip-19158 205 5 ) ) PUNCT eblip-19158 205 6 , , PUNCT eblip-19158 205 7 322 322 NUM eblip-19158 205 8 - - SYM eblip-19158 205 9 328 328 NUM eblip-19158 205 10 . . PUNCT eblip-19158 205 11     SPACE eblip-19158 206 1 creswell creswell PROPN eblip-19158 206 2 , , PUNCT eblip-19158 206 3 j. j. PROPN eblip-19158 206 4 w. w. PROPN eblip-19158 206 5 ( ( PUNCT eblip-19158 206 6 2005 2005 NUM eblip-19158 206 7 ) ) PUNCT eblip-19158 206 8 . . PUNCT eblip-19158 207 1 educational educational ADJ eblip-19158 207 2 research research NOUN eblip-19158 207 3 : : PUNCT eblip-19158 207 4 planning planning NOUN eblip-19158 207 5 , , PUNCT eblip-19158 207 6 conducting conducting NOUN eblip-19158 207 7 , , PUNCT eblip-19158 207 8 and and CCONJ eblip-19158 207 9 evaluating evaluate VERB eblip-19158 207 10 quantitative quantitative ADJ eblip-19158 207 11 and and CCONJ eblip-19158 207 12 qualitative qualitative ADJ eblip-19158 207 13 research research NOUN eblip-19158 207 14 ( ( PUNCT eblip-19158 207 15 2nd 2nd ADJ eblip-19158 207 16 ed ed NOUN eblip-19158 207 17 . . PUNCT eblip-19158 207 18 ) ) PUNCT eblip-19158 207 19 . . PUNCT eblip-19158 208 1 upper upper ADJ eblip-19158 208 2 saddle saddle PROPN eblip-19158 208 3 river river PROPN eblip-19158 208 4 , , PUNCT eblip-19158 208 5 nj nj PROPN eblip-19158 208 6 : : PUNCT eblip-19158 208 7 merrill merrill PROPN eblip-19158 208 8 . . PUNCT eblip-19158 208 9     SPACE eblip-19158 208 10 emmons emmon NOUN eblip-19158 208 11 , , PUNCT eblip-19158 208 12 m. m. NOUN eblip-19158 208 13 , , PUNCT eblip-19158 208 14 & & CCONJ eblip-19158 208 15 martin martin PROPN eblip-19158 208 16 , , PUNCT eblip-19158 208 17 w. w. PROPN eblip-19158 208 18 ( ( PUNCT eblip-19158 208 19 2002 2002 NUM eblip-19158 208 20 ) ) PUNCT eblip-19158 208 21 . . PUNCT eblip-19158 209 1 engaging engage VERB eblip-19158 209 2 conversation conversation NOUN eblip-19158 209 3 : : PUNCT eblip-19158 209 4 evaluating evaluate VERB eblip-19158 209 5 the the DET eblip-19158 209 6 contribution contribution NOUN eblip-19158 209 7 of of ADP eblip-19158 209 8 library library ADJ eblip-19158 209 9 instruction instruction NOUN eblip-19158 209 10 to to ADP eblip-19158 209 11 the the DET eblip-19158 209 12 quality quality NOUN eblip-19158 209 13 of of ADP eblip-19158 209 14 student student NOUN eblip-19158 209 15 research research NOUN eblip-19158 209 16 . . PUNCT eblip-19158 210 1 college college NOUN eblip-19158 210 2 & & CCONJ eblip-19158 210 3 research research PROPN eblip-19158 210 4 libraries library NOUN eblip-19158 210 5 , , PUNCT eblip-19158 210 6 63(6 63(6 PROPN eblip-19158 210 7 ) ) PUNCT eblip-19158 210 8 , , PUNCT eblip-19158 210 9 545 545 NUM eblip-19158 210 10 - - SYM eblip-19158 210 11 60 60 NUM eblip-19158 210 12 . . PUNCT eblip-19158 210 13     SPACE eblip-19158 211 1 fagerheim fagerheim PROPN eblip-19158 211 2 , , PUNCT eblip-19158 211 3 b. b. PROPN eblip-19158 211 4 a. a. PROPN eblip-19158 211 5 & & CCONJ eblip-19158 211 6 shrode shrode PROPN eblip-19158 211 7 , , PUNCT eblip-19158 211 8 f.g f.g NOUN eblip-19158 211 9 . . PROPN eblip-19158 211 10 ( ( PUNCT eblip-19158 211 11 2009 2009 NUM eblip-19158 211 12 ) ) PUNCT eblip-19158 211 13 . . PUNCT eblip-19158 212 1 information information NOUN eblip-19158 212 2 literacy literacy NOUN eblip-19158 212 3 rubrics rubric NOUN eblip-19158 212 4 within within ADP eblip-19158 212 5 the the DET eblip-19158 212 6 disciplines discipline NOUN eblip-19158 212 7 . . PUNCT eblip-19158 212 8     SPACE eblip-19158 213 1 communications communication NOUN eblip-19158 213 2 in in ADP eblip-19158 213 3 information information NOUN eblip-19158 213 4 literacy literacy NOUN eblip-19158 213 5 , , PUNCT eblip-19158 213 6 3(2 3(2 NUM eblip-19158 213 7 ) ) PUNCT eblip-19158 213 8 , , PUNCT eblip-19158 213 9 158 158 NUM eblip-19158 213 10 - - SYM eblip-19158 213 11 170 170 NUM eblip-19158 213 12 . . PUNCT eblip-19158 213 13     SPACE eblip-19158 214 1 fleiss fleiss PROPN eblip-19158 214 2 , , PUNCT eblip-19158 214 3 j.l j.l PROPN eblip-19158 214 4 . . PROPN eblip-19158 214 5 ( ( PUNCT eblip-19158 214 6 1971 1971 NUM eblip-19158 214 7 ) ) PUNCT eblip-19158 214 8 . . PUNCT eblip-19158 215 1 measuring measure VERB eblip-19158 215 2 nominal nominal ADJ eblip-19158 215 3 scale scale NOUN eblip-19158 215 4 agreement agreement NOUN eblip-19158 215 5 among among ADP eblip-19158 215 6 many many ADJ eblip-19158 215 7 raters rater NOUN eblip-19158 215 8 . . PUNCT eblip-19158 216 1 psychological psychological ADJ eblip-19158 216 2 bulletin bulletin NOUN eblip-19158 216 3 , , PUNCT eblip-19158 216 4 76(5 76(5 NUM eblip-19158 216 5 ) ) PUNCT eblip-19158 216 6 , , PUNCT eblip-19158 216 7 378 378 NUM eblip-19158 216 8 - - SYM eblip-19158 216 9 382 382 NUM eblip-19158 216 10 . . PUNCT eblip-19158 216 11     SPACE eblip-19158 216 12 gwet gwet PROPN eblip-19158 216 13 , , PUNCT eblip-19158 216 14 k. k. PROPN eblip-19158 216 15 ( ( PUNCT eblip-19158 216 16 2010 2010 NUM eblip-19158 216 17 ) ) PUNCT eblip-19158 216 18 . . PUNCT eblip-19158 217 1 handbook handbook NOUN eblip-19158 217 2 of of ADP eblip-19158 217 3 inter inter ADJ eblip-19158 217 4 - - ADJ eblip-19158 217 5 rater rater ADJ eblip-19158 217 6 reliability reliability NOUN eblip-19158 217 7 , , PUNCT eblip-19158 217 8 2nd 2nd ADJ eblip-19158 217 9 ed ed NOUN eblip-19158 217 10 . . PUNCT eblip-19158 218 1 gaithersburg gaithersburg PROPN eblip-19158 218 2 , , PUNCT eblip-19158 218 3 md md PROPN eblip-19158 218 4 : : PUNCT eblip-19158 218 5 advanced advanced ADJ eblip-19158 218 6 analytics analytic NOUN eblip-19158 218 7 . . PUNCT eblip-19158 218 8     SPACE eblip-19158 218 9 hand hand NOUN eblip-19158 218 10 , , PUNCT eblip-19158 218 11 d.j d.j PROPN eblip-19158 218 12 . . PROPN eblip-19158 218 13 ( ( PUNCT eblip-19158 218 14 2008 2008 NUM eblip-19158 218 15 ) ) PUNCT eblip-19158 218 16 . . PUNCT eblip-19158 219 1 statistics statistic NOUN eblip-19158 219 2 : : PUNCT eblip-19158 219 3 a a DET eblip-19158 219 4 very very ADV eblip-19158 219 5 short short ADJ eblip-19158 219 6 introduction introduction NOUN eblip-19158 219 7 . . PUNCT eblip-19158 219 8     SPACE eblip-19158 220 1 oxford oxford NOUN eblip-19158 220 2 : : PUNCT eblip-19158 220 3 oxford oxford PROPN eblip-19158 220 4 university university PROPN eblip-19158 220 5 press press NOUN eblip-19158 220 6 . . PUNCT eblip-19158 220 7     SPACE eblip-19158 221 1 van van NOUN eblip-19158 221 2 helvoort helvoort NOUN eblip-19158 221 3 , , PUNCT eblip-19158 221 4 j. j. PROPN eblip-19158 221 5 ( ( PUNCT eblip-19158 221 6 2010 2010 NUM eblip-19158 221 7 ) ) PUNCT eblip-19158 221 8 . . PUNCT eblip-19158 222 1 a a DET eblip-19158 222 2 scoring scoring NOUN eblip-19158 222 3 rubric rubric NOUN eblip-19158 222 4 for for ADP eblip-19158 222 5 performance performance NOUN eblip-19158 222 6 assessment assessment NOUN eblip-19158 222 7 of of ADP eblip-19158 222 8 information information NOUN eblip-19158 222 9 literacy literacy NOUN eblip-19158 222 10 in in ADP eblip-19158 222 11 dutch dutch ADJ eblip-19158 222 12 higher high ADJ eblip-19158 222 13 education education NOUN eblip-19158 222 14 . . PUNCT eblip-19158 222 15     SPACE eblip-19158 223 1 journal journal NOUN eblip-19158 223 2 of of ADP eblip-19158 223 3 information information NOUN eblip-19158 223 4 literacy literacy NOUN eblip-19158 223 5 , , PUNCT eblip-19158 223 6 4(1 4(1 NOUN eblip-19158 223 7 ) ) PUNCT eblip-19158 223 8 , , PUNCT eblip-19158 223 9 22 22 NUM eblip-19158 223 10 - - SYM eblip-19158 223 11 39 39 NUM eblip-19158 223 12 . . PUNCT eblip-19158 223 13     SPACE eblip-19158 224 1 hoffman hoffman PROPN eblip-19158 224 2 , , PUNCT eblip-19158 224 3 d. d. PROPN eblip-19158 224 4 & & CCONJ eblip-19158 224 5 labonte labonte PROPN eblip-19158 224 6 , , PUNCT eblip-19158 224 7 k. k. PROPN eblip-19158 224 8 ( ( PUNCT eblip-19158 224 9 2012 2012 NUM eblip-19158 224 10 ) ) PUNCT eblip-19158 224 11 . . PUNCT eblip-19158 225 1 meeting meet VERB eblip-19158 225 2 information information NOUN eblip-19158 225 3 literacy literacy NOUN eblip-19158 225 4 outcomes outcome NOUN eblip-19158 225 5 : : PUNCT eblip-19158 225 6 partnering partner VERB eblip-19158 225 7 with with ADP eblip-19158 225 8 faculty faculty NOUN eblip-19158 225 9 to to PART eblip-19158 225 10 create create VERB eblip-19158 225 11 effective effective ADJ eblip-19158 225 12 information information NOUN eblip-19158 225 13 literacy literacy NOUN eblip-19158 225 14 assessment assessment NOUN eblip-19158 225 15 . . PUNCT eblip-19158 225 16     SPACE eblip-19158 226 1 journal journal NOUN eblip-19158 226 2 of of ADP eblip-19158 226 3 information information NOUN eblip-19158 226 4 literacy literacy NOUN eblip-19158 226 5 , , PUNCT eblip-19158 226 6 6(2 6(2 NUM eblip-19158 226 7 ) ) PUNCT eblip-19158 226 8 , , PUNCT eblip-19158 226 9 70 70 NUM eblip-19158 226 10 - - SYM eblip-19158 226 11 85 85 NUM eblip-19158 226 12 . . PUNCT eblip-19158 226 13     SPACE eblip-19158 227 1 huba huba PROPN eblip-19158 227 2 , , PUNCT eblip-19158 227 3 m. m. PROPN eblip-19158 227 4 e. e. PROPN eblip-19158 227 5 & & CCONJ eblip-19158 227 6 freed freed PROPN eblip-19158 227 7 , , PUNCT eblip-19158 227 8 j. j. PROPN eblip-19158 227 9 e. e. PROPN eblip-19158 227 10 ( ( PUNCT eblip-19158 227 11 2000 2000 NUM eblip-19158 227 12 ) ) PUNCT eblip-19158 227 13 . . PUNCT eblip-19158 228 1 learner learner NOUN eblip-19158 228 2 - - PUNCT eblip-19158 228 3 centered center VERB eblip-19158 228 4 assessment assessment NOUN eblip-19158 228 5 on on ADP eblip-19158 228 6 college college NOUN eblip-19158 228 7 campuses campus NOUN eblip-19158 228 8 : : PUNCT eblip-19158 228 9 shifting shift VERB eblip-19158 228 10 the the DET eblip-19158 228 11 focus focus NOUN eblip-19158 228 12 from from ADP eblip-19158 228 13 teaching teach VERB eblip-19158 228 14 to to ADP eblip-19158 228 15 learning learning NOUN eblip-19158 228 16 . . PUNCT eblip-19158 229 1 boston boston PROPN eblip-19158 229 2 : : PUNCT eblip-19158 229 3 allyn allyn PROPN eblip-19158 229 4 and and CCONJ eblip-19158 229 5 bacon bacon NOUN eblip-19158 229 6 . . PUNCT eblip-19158 229 7     SPACE eblip-19158 230 1 information information NOUN eblip-19158 230 2 literacy literacy NOUN eblip-19158 230 3 competency competency NOUN eblip-19158 230 4 standards standard NOUN eblip-19158 230 5 for for ADP eblip-19158 230 6 higher high ADJ eblip-19158 230 7 education education NOUN eblip-19158 230 8 . . PUNCT eblip-19158 231 1 ( ( PUNCT eblip-19158 231 2 2000 2000 NUM eblip-19158 231 3 ) ) PUNCT eblip-19158 231 4 . . PUNCT eblip-19158 232 1 retrieved retrieve VERB eblip-19158 232 2 august august PROPN eblip-19158 232 3 23 23 NUM eblip-19158 232 4 , , PUNCT eblip-19158 232 5 2013 2013 NUM eblip-19158 232 6 from from ADP eblip-19158 232 7 http://www.acrl.org/ala/mgrps/divs/acrl/standards/standards.pdf http://www.acrl.org/ala/mgrps/divs/acrl/standards/standards.pdf PROPN eblip-19158 232 8     SPACE eblip-19158 232 9 johnson johnson PROPN eblip-19158 232 10 , , PUNCT eblip-19158 232 11 r. r. PROPN eblip-19158 232 12 l. l. PROPN eblip-19158 232 13 , , PUNCT eblip-19158 232 14 penny penny PROPN eblip-19158 232 15 , , PUNCT eblip-19158 232 16 j. j. PROPN eblip-19158 232 17 a. a. PROPN eblip-19158 232 18 , , PUNCT eblip-19158 232 19 & & CCONJ eblip-19158 232 20 gordon gordon PROPN eblip-19158 232 21 , , PUNCT eblip-19158 232 22 b. b. PROPN eblip-19158 232 23 ( ( PUNCT eblip-19158 232 24 2009 2009 NUM eblip-19158 232 25 ) ) PUNCT eblip-19158 232 26 assessing assess VERB eblip-19158 232 27 performance performance NOUN eblip-19158 232 28 : : PUNCT eblip-19158 232 29 designing designing NOUN eblip-19158 232 30 , , PUNCT eblip-19158 232 31 scoring scoring NOUN eblip-19158 232 32 , , PUNCT eblip-19158 232 33 and and CCONJ eblip-19158 232 34 validating validate VERB eblip-19158 232 35 performance performance NOUN eblip-19158 232 36 tasks task NOUN eblip-19158 232 37 . . PUNCT eblip-19158 233 1 new new PROPN eblip-19158 233 2 york york PROPN eblip-19158 233 3 : : PUNCT eblip-19158 233 4 the the DET eblip-19158 233 5 guilford guilford PROPN eblip-19158 233 6 press press PROPN eblip-19158 233 7 . . PUNCT eblip-19158 233 8     SPACE eblip-19158 234 1 knight knight NOUN eblip-19158 234 2 , , PUNCT eblip-19158 234 3 l. l. PROPN eblip-19158 234 4 a. a. PROPN eblip-19158 234 5 ( ( PUNCT eblip-19158 234 6 2006 2006 NUM eblip-19158 234 7 ) ) PUNCT eblip-19158 234 8 . . PUNCT eblip-19158 235 1 using use VERB eblip-19158 235 2 rubrics rubric NOUN eblip-19158 235 3 to to PART eblip-19158 235 4 assess assess VERB eblip-19158 235 5 information information NOUN eblip-19158 235 6 literacy literacy NOUN eblip-19158 235 7 . . PUNCT eblip-19158 236 1 reference reference NOUN eblip-19158 236 2 services service NOUN eblip-19158 236 3 review review PROPN eblip-19158 236 4 , , PUNCT eblip-19158 236 5 34(1 34(1 NUM eblip-19158 236 6 ) ) PUNCT eblip-19158 236 7 , , PUNCT eblip-19158 236 8 43 43 NUM eblip-19158 236 9 - - SYM eblip-19158 236 10 55 55 NUM eblip-19158 236 11 . . PUNCT eblip-19158 237 1 doi doi NOUN eblip-19158 237 2 : : PUNCT eblip-19158 237 3 101108/00907320510631571 101108/00907320510631571 NUM eblip-19158 237 4     SPACE eblip-19158 237 5 landis landis PROPN eblip-19158 237 6 , , PUNCT eblip-19158 237 7 j. j. PROPN eblip-19158 237 8 r. r. PROPN eblip-19158 237 9 , , PUNCT eblip-19158 237 10 & & CCONJ eblip-19158 237 11 koch koch PROPN eblip-19158 237 12 , , PUNCT eblip-19158 237 13 g. g. PROPN eblip-19158 237 14 g. g. PROPN eblip-19158 238 1 ( ( PUNCT eblip-19158 238 2 1977 1977 NUM eblip-19158 238 3 ) ) PUNCT eblip-19158 238 4 . . PUNCT eblip-19158 239 1 the the DET eblip-19158 239 2 measurement measurement NOUN eblip-19158 239 3 of of ADP eblip-19158 239 4 observer observer NOUN eblip-19158 239 5 agreement agreement NOUN eblip-19158 239 6 for for ADP eblip-19158 239 7 categorical categorical ADJ eblip-19158 239 8 data datum NOUN eblip-19158 239 9 . . PUNCT eblip-19158 240 1 biometrics biometric NOUN eblip-19158 240 2 , , PUNCT eblip-19158 240 3 33(1 33(1 NUM eblip-19158 240 4 ) ) PUNCT eblip-19158 240 5 , , PUNCT eblip-19158 240 6 159 159 NUM eblip-19158 240 7 - - SYM eblip-19158 240 8 174 174 NUM eblip-19158 240 9 . . PUNCT eblip-19158 240 10 retrieved retrieve VERB eblip-19158 240 11 august august PROPN eblip-19158 240 12 23 23 NUM eblip-19158 240 13 , , PUNCT eblip-19158 240 14 2013 2013 NUM eblip-19158 240 15 from from ADP eblip-19158 240 16 http://www.jstor.org/stable/2529310 http://www.jstor.org/stable/2529310 PROPN eblip-19158 240 17     SPACE eblip-19158 240 18 maki maki NOUN eblip-19158 240 19 , , PUNCT eblip-19158 240 20 p. p. NOUN eblip-19158 240 21 ( ( PUNCT eblip-19158 240 22 2010 2010 NUM eblip-19158 240 23 ) ) PUNCT eblip-19158 240 24 . . PUNCT eblip-19158 241 1 assessing assess VERB eblip-19158 241 2 for for ADP eblip-19158 241 3 learning learn VERB eblip-19158 241 4 : : PUNCT eblip-19158 241 5 building build VERB eblip-19158 241 6 a a DET eblip-19158 241 7 sustainable sustainable ADJ eblip-19158 241 8 commitment commitment NOUN eblip-19158 241 9 across across ADP eblip-19158 241 10 the the DET eblip-19158 241 11 institution institution NOUN eblip-19158 241 12 ( ( PUNCT eblip-19158 241 13 2nd 2nd ADJ eblip-19158 241 14 ed ed NOUN eblip-19158 241 15 . . PUNCT eblip-19158 241 16 ) ) PUNCT eblip-19158 241 17 . . PUNCT eblip-19158 242 1 sterling sterling NOUN eblip-19158 242 2 , , PUNCT eblip-19158 242 3 va va NOUN eblip-19158 242 4 : : PUNCT eblip-19158 242 5 stylus stylus ADJ eblip-19158 242 6 . . PUNCT eblip-19158 242 7     SPACE eblip-19158 242 8 mertler mertler NOUN eblip-19158 242 9 , , PUNCT eblip-19158 242 10 c. c. PROPN eblip-19158 242 11 a. a. PROPN eblip-19158 242 12 ( ( PUNCT eblip-19158 242 13 2001 2001 NUM eblip-19158 242 14 ) ) PUNCT eblip-19158 242 15 . . PUNCT eblip-19158 243 1 designing design VERB eblip-19158 243 2 scoring scoring NOUN eblip-19158 243 3 rubrics rubric NOUN eblip-19158 243 4 for for ADP eblip-19158 243 5 your your PRON eblip-19158 243 6 classroom classroom NOUN eblip-19158 243 7 . . PUNCT eblip-19158 244 1 practical practical ADJ eblip-19158 244 2 assessment assessment NOUN eblip-19158 244 3 , , PUNCT eblip-19158 244 4 research research NOUN eblip-19158 244 5 & & CCONJ eblip-19158 244 6 evaluation evaluation NOUN eblip-19158 244 7 , , PUNCT eblip-19158 244 8 7(25 7(25 NUM eblip-19158 244 9 ) ) PUNCT eblip-19158 244 10 . . PUNCT eblip-19158 244 11     SPACE eblip-19158 245 1 montgomery montgomery PROPN eblip-19158 245 2 , , PUNCT eblip-19158 245 3 k. k. PROPN eblip-19158 245 4 ( ( PUNCT eblip-19158 245 5 2002 2002 NUM eblip-19158 245 6 ) ) PUNCT eblip-19158 245 7 . . PUNCT eblip-19158 246 1 authentic authentic ADJ eblip-19158 246 2 tasks task NOUN eblip-19158 246 3 and and CCONJ eblip-19158 246 4 rubrics rubric NOUN eblip-19158 246 5 : : PUNCT eblip-19158 246 6 going go VERB eblip-19158 246 7 beyond beyond ADP eblip-19158 246 8 traditional traditional ADJ eblip-19158 246 9 assessments assessment NOUN eblip-19158 246 10 in in ADP eblip-19158 246 11 college college NOUN eblip-19158 246 12 teaching teaching NOUN eblip-19158 246 13 . . PUNCT eblip-19158 247 1 college college NOUN eblip-19158 247 2 teaching teaching NOUN eblip-19158 247 3 , , PUNCT eblip-19158 247 4 50(1 50(1 PROPN eblip-19158 247 5 ) ) PUNCT eblip-19158 247 6 , , PUNCT eblip-19158 247 7 34 34 NUM eblip-19158 247 8 - - SYM eblip-19158 247 9 39 39 NUM eblip-19158 247 10 . . PUNCT eblip-19158 248 1 doi doi NOUN eblip-19158 248 2 : : PUNCT eblip-19158 248 3 10.1080/87567550209595870 10.1080/87567550209595870 NUM eblip-19158 248 4     SPACE eblip-19158 248 5 moskal moskal PROPN eblip-19158 248 6 , , PUNCT eblip-19158 248 7 b. b. PROPN eblip-19158 248 8 ( ( PUNCT eblip-19158 248 9 2003 2003 NUM eblip-19158 248 10 ) ) PUNCT eblip-19158 248 11 . . PUNCT eblip-19158 249 1 recommendations recommendation NOUN eblip-19158 249 2 for for ADP eblip-19158 249 3 developing develop VERB eblip-19158 249 4 classroom classroom NOUN eblip-19158 249 5 performance performance NOUN eblip-19158 249 6 assessments assessment NOUN eblip-19158 249 7 and and CCONJ eblip-19158 249 8 scoring scoring NOUN eblip-19158 249 9 rubrics rubric NOUN eblip-19158 249 10 . . PUNCT eblip-19158 250 1 practical practical ADJ eblip-19158 250 2 assessment assessment NOUN eblip-19158 250 3 , , PUNCT eblip-19158 250 4 research research NOUN eblip-19158 250 5 and and CCONJ eblip-19158 250 6 evaluation evaluation NOUN eblip-19158 250 7 , , PUNCT eblip-19158 250 8 8(14 8(14 NUM eblip-19158 250 9 ) ) PUNCT eblip-19158 250 10 . . PUNCT eblip-19158 250 11     SPACE eblip-19158 251 1 oakleaf oakleaf NOUN eblip-19158 251 2 , , PUNCT eblip-19158 251 3 m. m. NOUN eblip-19158 251 4 ( ( PUNCT eblip-19158 251 5 2007 2007 NUM eblip-19158 251 6 ) ) PUNCT eblip-19158 251 7 . . PUNCT eblip-19158 252 1 using use VERB eblip-19158 252 2 rubrics rubric NOUN eblip-19158 252 3 to to PART eblip-19158 252 4 collect collect VERB eblip-19158 252 5 evidence evidence NOUN eblip-19158 252 6 for for ADP eblip-19158 252 7 decision decision NOUN eblip-19158 252 8 - - PUNCT eblip-19158 252 9 making making NOUN eblip-19158 252 10 : : PUNCT eblip-19158 252 11 what what PRON eblip-19158 252 12 do do AUX eblip-19158 252 13 librarians librarian NOUN eblip-19158 252 14 need need VERB eblip-19158 252 15 to to PART eblip-19158 252 16 learn learn VERB eblip-19158 252 17 ? ? PUNCT eblip-19158 253 1 evidence evidence NOUN eblip-19158 253 2 based base VERB eblip-19158 253 3 library library NOUN eblip-19158 253 4 and and CCONJ eblip-19158 253 5 information information NOUN eblip-19158 253 6 practice practice NOUN eblip-19158 253 7 , , PUNCT eblip-19158 253 8 2(3 2(3 NUM eblip-19158 253 9 ) ) PUNCT eblip-19158 253 10 , , PUNCT eblip-19158 253 11 27 27 NUM eblip-19158 253 12 - - SYM eblip-19158 253 13 42 42 NUM eblip-19158 253 14 . . PUNCT eblip-19158 253 15     SPACE eblip-19158 254 1 oakleaf oakleaf NOUN eblip-19158 254 2 , , PUNCT eblip-19158 254 3 m. m. NOUN eblip-19158 254 4 ( ( PUNCT eblip-19158 254 5 2009 2009 NUM eblip-19158 254 6 ) ) PUNCT eblip-19158 254 7 . . PUNCT eblip-19158 255 1 using use VERB eblip-19158 255 2 rubrics rubric NOUN eblip-19158 255 3 to to PART eblip-19158 255 4 assess assess VERB eblip-19158 255 5 information information NOUN eblip-19158 255 6 literacy literacy NOUN eblip-19158 255 7 : : PUNCT eblip-19158 255 8 an an DET eblip-19158 255 9 examination examination NOUN eblip-19158 255 10 of of ADP eblip-19158 255 11 methodology methodology NOUN eblip-19158 255 12 and and CCONJ eblip-19158 255 13 interrater interrater ADJ eblip-19158 255 14 reliability reliability NOUN eblip-19158 255 15 . . PUNCT eblip-19158 255 16     SPACE eblip-19158 255 17 journal journal NOUN eblip-19158 255 18 of of ADP eblip-19158 255 19 the the DET eblip-19158 255 20 american american ADJ eblip-19158 255 21 society society NOUN eblip-19158 255 22 for for ADP eblip-19158 255 23 information information NOUN eblip-19158 255 24 science science NOUN eblip-19158 255 25 and and CCONJ eblip-19158 255 26 technology technology NOUN eblip-19158 255 27 , , PUNCT eblip-19158 255 28 60(5 60(5 NOUN eblip-19158 255 29 ) ) PUNCT eblip-19158 255 30 , , PUNCT eblip-19158 255 31 969 969 NUM eblip-19158 255 32 - - SYM eblip-19158 255 33 983 983 NUM eblip-19158 255 34 . . PUNCT eblip-19158 256 1 doi doi NOUN eblip-19158 256 2 : : PUNCT eblip-19158 256 3 10.1002 10.1002 NUM eblip-19158 256 4 / / SYM eblip-19158 256 5 asi.21030 asi.21030 NOUN eblip-19158 256 6     SPACE eblip-19158 256 7 oakleaf oakleaf NOUN eblip-19158 256 8 , , PUNCT eblip-19158 256 9 m. m. NOUN eblip-19158 256 10 & & CCONJ eblip-19158 256 11 kaske kaske PROPN eblip-19158 256 12 , , PUNCT eblip-19158 256 13 n. n. PROPN eblip-19158 256 14 ( ( PUNCT eblip-19158 256 15 2009 2009 NUM eblip-19158 256 16 ) ) PUNCT eblip-19158 256 17 . . PUNCT eblip-19158 257 1 guiding guide VERB eblip-19158 257 2 questions question NOUN eblip-19158 257 3 for for ADP eblip-19158 257 4 assessing assess VERB eblip-19158 257 5 information information NOUN eblip-19158 257 6 literacy literacy NOUN eblip-19158 257 7 in in ADP eblip-19158 257 8 higher high ADJ eblip-19158 257 9 education education NOUN eblip-19158 257 10 . . PUNCT eblip-19158 258 1 portal portal PROPN eblip-19158 258 2 : : PUNCT eblip-19158 258 3 libraries library NOUN eblip-19158 258 4 & & CCONJ eblip-19158 258 5 the the DET eblip-19158 258 6 academy academy PROPN eblip-19158 258 7 , , PUNCT eblip-19158 258 8 9(2 9(2 NUM eblip-19158 258 9 ) ) PUNCT eblip-19158 258 10 , , PUNCT eblip-19158 258 11 273 273 NUM eblip-19158 258 12 - - SYM eblip-19158 258 13 286 286 NUM eblip-19158 258 14 . . PUNCT eblip-19158 259 1 doi doi NOUN eblip-19158 259 2 : : PUNCT eblip-19158 259 3 10.1353 10.1353 NUM eblip-19158 259 4 / / SYM eblip-19158 259 5 pla.0.0046 pla.0.0046 NOUN eblip-19158 259 6     SPACE eblip-19158 259 7 rockman rockman NOUN eblip-19158 259 8 , , PUNCT eblip-19158 259 9 i. i. PROPN eblip-19158 259 10 f. f. PROPN eblip-19158 259 11 ( ( PUNCT eblip-19158 259 12 2002 2002 NUM eblip-19158 259 13 ) ) PUNCT eblip-19158 259 14 . . PUNCT eblip-19158 260 1 strengthening strengthen VERB eblip-19158 260 2 connections connection NOUN eblip-19158 260 3 between between ADP eblip-19158 260 4 information information NOUN eblip-19158 260 5 literacy literacy NOUN eblip-19158 260 6 , , PUNCT eblip-19158 260 7 general general ADJ eblip-19158 260 8 education education NOUN eblip-19158 260 9 , , PUNCT eblip-19158 260 10 and and CCONJ eblip-19158 260 11 assessment assessment NOUN eblip-19158 260 12 efforts effort NOUN eblip-19158 260 13 . . PUNCT eblip-19158 261 1 library library ADJ eblip-19158 261 2 trends trend NOUN eblip-19158 261 3 , , PUNCT eblip-19158 261 4 51(2 51(2 NUM eblip-19158 261 5 ) ) PUNCT eblip-19158 261 6 , , PUNCT eblip-19158 261 7 185 185 NUM eblip-19158 261 8 - - SYM eblip-19158 261 9 198 198 NUM eblip-19158 261 10 . . PUNCT eblip-19158 261 11     SPACE eblip-19158 261 12 stemler stemler NOUN eblip-19158 261 13 , , PUNCT eblip-19158 261 14 s. s. PROPN eblip-19158 261 15 e. e. PROPN eblip-19158 261 16 ( ( PUNCT eblip-19158 261 17 2004 2004 NUM eblip-19158 261 18 ) ) PUNCT eblip-19158 261 19 . . PUNCT eblip-19158 262 1 a a DET eblip-19158 262 2 comparison comparison NOUN eblip-19158 262 3 of of ADP eblip-19158 262 4 consensus consensus NOUN eblip-19158 262 5 , , PUNCT eblip-19158 262 6 consistency consistency NOUN eblip-19158 262 7 , , PUNCT eblip-19158 262 8 and and CCONJ eblip-19158 262 9 measurement measurement NOUN eblip-19158 262 10 approaches approach NOUN eblip-19158 262 11 to to ADP eblip-19158 262 12 estimating estimate VERB eblip-19158 262 13 interrater interrater ADJ eblip-19158 262 14 reliability reliability NOUN eblip-19158 262 15 . . PUNCT eblip-19158 263 1 practical practical ADJ eblip-19158 263 2 assessment assessment NOUN eblip-19158 263 3 , , PUNCT eblip-19158 263 4 research research NOUN eblip-19158 263 5 & & CCONJ eblip-19158 263 6 evaluation evaluation NOUN eblip-19158 263 7 , , PUNCT eblip-19158 263 8 9(4 9(4 NUM eblip-19158 263 9 ) ) PUNCT eblip-19158 263 10 . . PUNCT eblip-19158 263 11     SPACE eblip-19158 264 1 stevens stevens PROPN eblip-19158 264 2 , , PUNCT eblip-19158 264 3 d. d. PROPN eblip-19158 264 4 d. d. PROPN eblip-19158 264 5 & & CCONJ eblip-19158 264 6 levi levi PROPN eblip-19158 264 7 , , PUNCT eblip-19158 264 8 a. a. NOUN eblip-19158 264 9 ( ( PUNCT eblip-19158 264 10 2005 2005 NUM eblip-19158 264 11 ) ) PUNCT eblip-19158 264 12 . . PUNCT eblip-19158 265 1 introduction introduction NOUN eblip-19158 265 2 to to ADP eblip-19158 265 3 rubrics rubric NOUN eblip-19158 265 4 : : PUNCT eblip-19158 265 5 an an DET eblip-19158 265 6 assessment assessment NOUN eblip-19158 265 7 tool tool NOUN eblip-19158 265 8 to to PART eblip-19158 265 9 save save VERB eblip-19158 265 10 grading grading NOUN eblip-19158 265 11 time time NOUN eblip-19158 265 12 , , PUNCT eblip-19158 265 13 convey convey VERB eblip-19158 265 14 effective effective ADJ eblip-19158 265 15 feedback feedback NOUN eblip-19158 265 16 , , PUNCT eblip-19158 265 17 and and CCONJ eblip-19158 265 18 promote promote VERB eblip-19158 265 19 student student NOUN eblip-19158 265 20 learning learning NOUN eblip-19158 265 21 ( ( PUNCT eblip-19158 265 22 1st 1st ADJ eblip-19158 265 23 ed ed NOUN eblip-19158 265 24 . . PUNCT eblip-19158 265 25 ) ) PUNCT eblip-19158 265 26 . . PUNCT eblip-19158 266 1 sterling sterling NOUN eblip-19158 266 2 , , PUNCT eblip-19158 266 3 va va NOUN eblip-19158 266 4 : : PUNCT eblip-19158 266 5 stylus stylus ADJ eblip-19158 266 6 . . PUNCT