


































Frontiers of Contemporary Education 
ISSN 2690-3520 (Print) ISSN 2690-3539 (Online) 

Vol. 2, No. 2, 2021 

www.scholink.org/ojs/index.php/fce 

67 
 

Original Paper 

Designing Indicators for a Placement Test: Drawbacks and 

Affordances 

Gabriel Brito Amorim
1
 

1
 Languages Center, Federal University of Espírito Santo, Brazil 

 

Received: September 9, 2021    Accepted: October 3, 2021     Online Published: December 29, 2021 

doi:10.22158/fce.v2n2p67                  URL: http://dx.doi.org/10.22158/fce.v2n2p67 

 

Abstract 

One of the greatest challenges when it comes to assessment is the lack of clarity in the objectives or, to 

make it worse, the absence of objectives whatsoever for what results the curriculum, course, or the test 

instrument wishes to achieve. According to Raupp and Reichle (2003), this kind of information usually 

exists as expectations pushing the examiner to transform expectations, general goals or poorly stated 

objectives into measurable objectives. Raupp and Reichle (2003) emphasize that very precise indicators 

need to be written or selected to measure whether or not the course/curriculum objectives proposed were 

achieved. However, this job becomes rather challenging when objectives are not clearly stated or are too 

vague. Indicators, then, ideally, resonate curricular goals, teaching methodologies, and testing. If one of 

these components is not well calibrated or is dissonant with the others, the whole assessment process is 

jeopardized (CEFRL, 2007). Indicators for placement testing, then, need to be carefully written or 

selected so that they can reflect an intended proficiency level as well as the school’s curricular goals, 

teaching methodologies, and testing procedures (Brown, 2004; Richards, 2009). At the Federal 

University of Espírito Santo’s Language Center (CL), the discontentment of instructors and coordination 

staff with the lack of reliability, validity and practicality with the CL’s placement test motivated debates 

and work on the refinement of the indicators used for the referred exam. Through a literature review and 

with a practical example from a language center in Brazil, this paper addresses how 

teachers/practitioners can make informed decisions to write or select their own indicators for placement 

testing and, therefore, monitor the progress of their students and/or the success or pitfalls of their 

classes/programs/curricula and plan for possible welcome changes. 

Keywords 

Placement Testing, Assessment, Indicators 

 

 



www.scholink.org/ojs/index.php/fce           Frontiers of Contemporary Education             Vol. 2, No. 2, 2021 

68 
Published by SCHOLINK INC. 

Summary  

Through a literature review and with a practical example from a language center in Brazil, this paper 

addresses how teachers/practitioners can make informed decisions to write or select their own indicators 

for placement testing and, therefore, monitor the progress of their students and/or the success or pitfalls 

of their classes/programs/curricula and plan for possible welcome changes. 

 

1. Introduction 

According to Raupp and Reichle (2003), in order for one to obtain efficient results in assessment, it is 

necessary to work with some guiding principles. The aforementioned authors recommend four 

principles: 1) assessment as a collaborative process of exploration, examining, questioning and search 

for other perspectives, 2) the availability and the use of valid and reliable information to make the 

process more efficient, 3) the use of the information obtained as the key point of the process; and, lastly, 

4) assessment as a powerful and valuable catapult to increase the efficiency of projects and programs (p. 

18).  

Raupp and Kolb (1990) point out that, before designing an assessment system or instrument, one 

should consider the reason or reasons to assess. The authors list 10 possible reasons and claim that an 

assessment proposal may fall into one or more categories. The one we shall be focusing on this paper is 

number 2, “to make informed decisions”. Just as one ponders why to assess, they must also consider 

what to assess. Raupp and Kolb (1990) affirm that the information collected in the assessment 

corroborate a previous idea about the object being assessed. In other words, when placing an incoming 

student into a language program, the student might present certificates or diplomas from other schools, 

which indicate his/her overall proficiency level. A well-planned and well-executed placement test will 

collect valuable data that will confirm or refute that information and help the examiner place that 

student into the current language program.    

When a patient goes to the doctor seeking a diagnosis and treatment to a health problem, he/she 

presents precious health information that help the doctor diagnose the problem and prescribe an 

effective treatment. The description of the symptoms, clinical exams, the history of the patient and the 

experience of the doctor will collaborate in the recovery of the patient. Currently, there is software that 

helps the doctor diagnose patients. This software provides a more precise diagnosis as it analyzes the 

data collected and contrasts with medical notes, medicine books, etc. After this thorough analysis, the 

software offers the doctor a list of possible health problems and prescriptions for the doctor to close the 

diagnosis. The doctor, then, decides on how to treat the patient. A placement test works similarly to the 

work of this software, but in a more analogical way.  

The problem we faced at the Languages Center (CL) of the Federal University of Espírito Santo (UFES, 

Brazil) was very much like that. The discontentment of instructors and coordination staff with the lack 

of reliability, validity and practicality with the CL’s placement test motivated debates and work on the 

refinement of the indicators used for the referred exam. Also, the staff saw the need to refine the test 



www.scholink.org/ojs/index.php/fce           Frontiers of Contemporary Education             Vol. 2, No. 2, 2021 

69 
Published by SCHOLINK INC. 

instrument in order to align with the refined indicators.  

The CL was established in 1997 as a university extension project to offer language courses to the 

academic community at UFES and the community at large at a reasonable cost. Additionally, the CL 

provides an intensive language teacher education program for the Letters students at UFES. The CL 

holds an average of 5,000 students and 60 teachers (permanent and trainees). Since its conception the CL 

offers regular language courses that range from 8 semesters to 10 semesters of duration. The largest 

program is the English program, with the highest number of students, teachers and coordinators. For the 

purposes of this paper, we will be addressing the challenges faced by the English department only.   

In the early 2000’s, the CL hired an assessment specialist to assist the teachers and coordination staff 

transition from a paper-and-pencil assessment to a performance-based assessment system in the four 

abilities, namely, reading, writing, listening and speaking. The urge to change grew out of a 

dissatisfaction of both teachers and students with the mismatch between teaching and testing in the 

school. At that time, a brand-new performance-based assessment system was developed with the 

participation of permanent teachers and trainees. The system was and is used across language 

departments at the center. A principal document was elaborated with learning objectives for, mainly, 

three levels: beginner, intermediate and advanced. The document reflected the school’s views on 

language and language learning and had the communitive approach as its central theoretical basis. An 

assessment matrix, ranging from grade A to grade D, was designed with indicators that reflected the 

expected performance of the test-takers in the four abilities. The instruments were also redesigned and 

included tasks for reading comprehension, listening comprehension, writing expression and oral 

expression. As for the placement tests for incoming students of the English department (focus of this 

paper), a multiple-choice exam with 100 questions was created and encompassed a listening 

comprehension section, a grammar and vocabulary section and a reading comprehension section. There 

was also a second type of placement test which consisted of an essay exam and an oral interview.  

In 2013, due to the university’s internationalization process, the CL felt the need to adjust its assessment 

documents to international documents like the Common European Framework of Reference for 

Languages (CEFRL). So, the original documents that divided the benchmark levels were adapted to 

A1—corresponding to the beginner levels which encompassed levels 1, 2 and 3—, A2—corresponding 

to the pre-intermediate levels which encompassed levels 4, 5 and 6—, B1—or the intermediate levels, 

englobing levels 7 and 8—, and, lastly, B2—which stands for the upper-intermediate levels 9 and 10. The 

language courses that were only 8 semesters long adopted a different configuration. Since this time, the 

CL teachers and coordination staff have debated on the refinement of the assessment matrix and, 

consequently of the instruments they have used, including the placement tests.  

The motivation to rethink and, possibly, redesign the assessment documents and placement testing 

instruments of the English department came up as teachers and students reported that students were being 

misplaced over the course of years. In many occasions, the teachers, along with the coordination staff, 

had to adjust students’ levels after the placement testing procedures. When this situation became too 



www.scholink.org/ojs/index.php/fce           Frontiers of Contemporary Education             Vol. 2, No. 2, 2021 

70 
Published by SCHOLINK INC. 

frequent, teachers and coordinators decided it was time to change (Note 1).  

The laborious task of refining the assessment documents started back then when an assessment 

committee was formed and when it was decided that work should begin with an immersion in the 

literature of the area in order to make informed decisions. A series of teacher education sessions on 

assessment was carried out with different specialists and members of the staff. According to Raupp and 

Reichle (2003), for the daunting task of (re)designing assessment documents to be successful, it is 

essential that the designers know what they are aiming for. In order for one to know what a (language) 

program goal is, the objectives or indicators need to be carefully and clearly written. That was the 

beginning of our work.  

The objective of this paper is to present the refinement process of the indicators of the placement tests 

used in the CL at UFES. After an in-depth literature review and the analysis of assessment documents 

both from the institution and also of public domain (like the CEFRL), the designers began working on 

the assessment documents, which culminated in a revisited and calibrated assessment matrix and new 

placement test instruments. The research and data collection process was a collaborative task amongst 

the members of the CL staff, including myself, reporting the results of this endeavor.  

 

2. Methodology 

Given the history of the CL and the problems or difficulties the staff was facing with placement testing, 

teachers and coordination staff decided to delve into the literature of the area, which included theories, 

public documents like the CEFRL and internal assessment documents. As the team advanced in their 

readings and discussion prior to the beginning of the heavy work, the methodology to carry out this 

task became clearer to all. The team decided for a qualitative approach (Creswell, 2007) for its 

participatory and self-reflective nature (Rossman & Rallis, 1998).  

The team did not have a research question in mind, but a hypothesis that the assessment matrix being 

used to evaluate the incoming students was not really aligned with the learning objectives of the course 

nor with the test instruments being used. If this hypothesis could be translated into research questions, 

something along these lines would come up: To what extent are the assessment documents (matrix and 

instruments) influencing the misplacement of incoming students? How can the calibration of these 

documents improve results of placement testing?  

The characteristics of the qualitative research fit well into the scenario that the CL teachers and staff 

were inserted. They were inside the problem and could have access to information first-hand. As 

Creswell (2007) points out, qualitative research utilizes multiple methods of data collection and are 

more interactive and humanistic (p. 188). According to Creswell (2007), the analysis of documents, 

interviews and observations are some of the data collection methods applied in this kind of research. 

Another trait of the qualitative research is the flexibility of the process of data collection, which may 

change as the researcher encounters better opportunities to understand better the phenomenon in 

question.  



www.scholink.org/ojs/index.php/fce           Frontiers of Contemporary Education             Vol. 2, No. 2, 2021 

71 
Published by SCHOLINK INC. 

The CL research was of interpretative nature. That means that the investigator makes an interpretation 

of the phenomenon (or the data) through his/her own lenses (Creswell, 2007). In the case of the CL 

endeavor, the researchers (considering that all members of the staff contributed to the construction of 

this research) were very experienced teachers and coordinators, who have participated in the 

construction of the assessment documents of the CL and also had a lot of background knowledge on 

assessment. So, they were very aware of the context in which they were inserted to make the 

conclusions they needed to make and the alterations or calibrations they need to make.  

As per the phases of the research, first, the team observed the inconsistency of placement test results 

for some time (recurring topic in staff meetings and in the assessment committee meetings) and 

decided to invest time and effort in digging the problem to its roots. We call it the Observation Phase. 

Next, in phase 2, the team decided to delve into the literature of the area and into public and internal 

assessment documents in order to build on their expertise and find a direction to the problem posed. 

Staff meetings to discuss an appropriate manner to handle the problem were scheduled regularly. We 

call it the Theoretical Embedding phase. Then, in phase 3, using the data they had collected, the team 

started calibrating the internal assessment documents and designed one specifically for placement 

testing. We call it the Calibration Phase. Finally, in phase 4, the team began designing new test 

instruments for placement tests and piloting them in subsequent semesters. This was the Design and 

Piloting Phase. The four phases of the project were completed between 2018-2019.  

In the next section, a literature review is presented aiming to show the reader the theories underlying 

the changes made by the CL team. This body of work shed some light into the doubts the team had 

about how the assessment documents were interfering in the placement test results and served as 

guiding principles for the calibration of the internal documents.  

 

3. Literature Review 

As mentioned elsewhere, the objective of this literature review is to provide the theoretical foundations 

for the assumptions made by the CL team in regards to the institution’s assessment documents and their 

direct or indirect influence in the placement test results. As part of the research endeavor carried out by 

the team, they went back to the theory, from placement testing (Brown, 2004) to reference documents 

like the CEFRL (2001) to find the answers they needed.  

Coombe, Folse and Hubley (2007) start off by listing eight theoretical concepts, namely: 1) utility: 

supported by Bachman and Palmer’s (1996) test instrument utility model, Coombe, Folse and Hubley 

(2007) attest that this is the most important principle to give legitimacy to the evaluation process, that is, 

an evaluation instrument must be created for a specific objective and with a target audience in mind; 2) 

validity: refers to its purpose, that is, whether it evaluates what it intends to evaluate, one of its 

ramifications being construct validity, that is, whether the methodology chosen to evaluate is consistent 

with the instrument used; 3) reliability: refers to the consistency of the instrument, that is, if it presents 

the same results (or similar results) if applied on different occasions and by different examiners; 4) 



www.scholink.org/ojs/index.php/fce           Frontiers of Contemporary Education             Vol. 2, No. 2, 2021 

72 
Published by SCHOLINK INC. 

practicality: when thinking about the evaluation instrument, one must also consider the costs for 

development and maintenance, the administration time, the necessary resources, among other aspects, 

and for a large-scale evaluation it is necessary to determine practical factors for the collection of 

information about the object to be evaluated; 5) washback: it is related to the effect that the evaluation 

results produce, which can be positive or negative, and one should not conduct the evaluation in a way 

to mask the results for both positives and negatives; 6) authenticity: refers to the fact that the 

instrument is linked to a real situation, a real objective such as pointing out solutions or directions for a 

current problem; 7) transparency: it relates to the clarity of the information to be collected, the way it 

will be collected and why it will be collected, which should be made available to the parties involved in 

the evaluation process and 8) security: it relates to the preparation and administration of the evaluation 

instrument and in the case of large-scale assessments, for example, it is common for an instrument to 

be designed and administered by a group of people, which can compromise both the confidentiality of 

the instrument and the provision of information to the target audience, which may mask the results 

obtained (p. XXII). 

Brown (2004) suggests the foundations of designing and revising tests by providing 5 critical questions: 

1) What is the purpose of the test? 2) What are the objectives of the test? 3) How will the test 

specifications reflect both the purpose and the objectives of the test? 4) How will the test tasks be 

selected and the separate items arranged? What kind of scoring, grading, and/or feedback is expected? 

(p. 42). Knowing the purpose of a specific type of test helps to identify the foundational principles of 

assessment, namely, utility, validity, reliability, practicality, authenticity, transparency, security and 

wash back and, therefore, helps to determine whether a test is effective or not.  

The purpose of a placement test, for example,  

[…] is to place a student into a particular level or section of a language curriculum or 

school. A placement test usually, but not always, includes a sampling of the material to 

be covered in the various courses in a curriculum; a student’s performance on the test 

should indicate the point at which the student will find material neither too easy nor too 

difficult but appropriately challenging (Brown, 2004, p. 45). 

So, the placement test must fit the program or curriculum for which the incoming student is applying. 

There are some standardized tests available (some even online) which may be very practical for the 

institution administering the exam, but they might not really reflect the goals of the curriculum the 

student is going to encounter in that specific school or program.  

Richards (2009) explains that most schools and language programs describe their curricular goals in 

terms of learning objectives and aims. The author claims that “[T]he nature of aims and objectives, 

however, is not necessarily straightforward because they refer to knowledge, skills, and values that 

educational planners believe learners need to develop” (p. 112, emphasis in the original). This means 

that the role of teaching and learning of a language is reflected in the curricular goals of a language 

program. Richards (2009) helps us understand the complex task of describing curricular goals by 



www.scholink.org/ojs/index.php/fce           Frontiers of Contemporary Education             Vol. 2, No. 2, 2021 

73 
Published by SCHOLINK INC. 

giving us three common assumptions: 1) People are generally motivated to pursue specific goals; 2) 

The use of goals in teaching improves the effectiveness of teaching and learning; and 3) A program 

will be effective to the extent that its goals are sound and clearly described.  

As mentioned elsewhere in this manuscript, stemming from a dissatisfaction of both teacher and 

students about their assessment system and teaching practices, the CL at UFES started rethinking its 

curricular goals back in the early 2000’s, which resulted in the design of a brand-new 

performance-based assessment system. Part of the tasks back then was to define curricular goals for the 

language program and design institutional assessment documents to guide the staff. After the 

adjustment and refinement of the documents around 2013, the staff from all departments felt the need 

to revisit the assessment documents. This refers to Phase 1 and 2 described in the methodology section. 

Alongside this movement of change, the teachers and coordination staff of the English department 

began to work on the placement testing documents and procedures (Phase 3) (Note 2).   

During Phase 3, it became very clear to the English department staff that considerable time and effort 

should be spent on the indicators or descriptors (We will use indicators henceforth.) that were the direct 

link to the program curricular goals. According to Almeida (2018), important and valuable data can be 

collected through indicators, which can be assessed both quantitatively and qualitatively. Indicators 

present evidence of whether a goal was achieved or not.  

A document that served as reference and model for the English department staff when redesigning their 

own indicators was the CEFRL. The CEFRL was intentionally created with an action-oriented 

approach to aid language practitioners to elaborate their own frameworks, according to their specific 

needs and contexts across Europe. The manuscript became largely diffused and has served as 

theoretical basis for language programs around the world. Here is the opening paragraph of the 

document:   

The Common European Framework provides a common basis for the elaboration of 

language syllabuses, curriculum guidelines, examinations, textbooks, etc. across Europe. 

It describes in a comprehensive way what language learners have to learn to do in order 

to use a language for communication and what knowledge and skills they have to 

develop so as to be able to act effectively. The description also covers the cultural 

context in which language is set. The Framework also defines levels of proficiency which 

allow learners’ progress to be measured at each stage of learning and on a life-long basis 

(CEFRL, 2001, p. 1).  

Barrett et al. (2018) put forth some criteria for the formulation of indicators, namely: 1) brevity, that is, 

having a maximum of 25 words; 2) positivity, expressing a skill or goal in affirmative sentences; 3) 

clarity, that is, no jargon and simple structure; 4) independence, in other words, being independent of 

another indicator; and 5) accuracy, that is, describing whether a certain behavior or a certain goal was 

achieved or not.  

 



www.scholink.org/ojs/index.php/fce           Frontiers of Contemporary Education             Vol. 2, No. 2, 2021 

74 
Published by SCHOLINK INC. 

Taking the literature from the area and the analysis of the internal and public assessment documents 

into account, the CL staff initiated Phase 4 which encompassed the design of new assessment 

instruments aligned with the redesigned assessment documents. In the following section, we present 

one of the results of this research endeavor, specifically taken from the English department staff. This 

department engaged in producing an assessment matrix for placement testing aligned with the other 

institutional assessment documents which, in turn, were aligned with the revised program curricular 

goals. As we present the assessment matrix for placement testing, we will make the connections with 

the literature.  

 

4. Results and Discussion 

As mentioned before in this manuscript, the main outcomes of this project were the redesign of the 

institutional assessment documents and, subsequently, the assessment instruments. The English 

Department at the CL, in particular, underwent a surgical process of (re)construction of the assessment 

matrix for placement testing and its instruments, which will be used here as illustration of how the staff 

used the theory and knowledge of the program curricular goals to attenuate the issue they were having 

with misplaced students.  

The process of rethinking the assessment documents for placement testing started with the redesign of 

indicators for that purpose, which had to be carefully and meticulously aligned with the course 

curriculum. Before we analyze the process of creating the indicators, we shall describe the redesign of 

the placement testing instruments. There are two types of placement tests at the CL’s English 

Department: 1) one multiple-choice exam with a reading section, a grammar and vocabulary section 

and a listening section; and 2) one with a written test (essay) and an oral interview.  

As part of a screening process prior to the exam, the incoming students fill out an application form in 

which they describe their language learning experience. Depending on the information they fill in the 

form, the coordination staff will assign test 1 (multiple-choice) or test 2 (essay and interview) for the 

candidate. Test 1 is geared towards candidates who have had less numbers of hours of instruction and 

test 2 is assigned to candidates who have had an extensive number of hours of instruction (around 

300h).  

Test 1 was completely redesigned question by question going from a 100-item test to an 80-item test 

(with a listening section, a reading section and a grammar and vocabulary section), which contained 

questions from all across the curriculum (Brown, 2004) and were aligned with curricular goals of 

teaching and learning (Richards, 2009). Decreasing the number of questions made the test more 

practical to administer (the CL administers up to 250-300 tests each turn—usually happening twice in 

the academic semester), which increased security, validity, reliability and provoked positive washback 

(Coombe, Folse & Humbley, 2007).  

 

 



www.scholink.org/ojs/index.php/fce           Frontiers of Contemporary Education             Vol. 2, No. 2, 2021 

75 
Published by SCHOLINK INC. 

Test 2 was completely redesigned as well. The test is still divided between an essay part and an oral 

interview. This time, candidates are presented with 2 images and guiding questions and have 40 

minutes to write a 150-word-minimum essay about 1 one them. Upon finishing their writing exam, 

candidates are directed to the interview in which they are supposed to talk about the same image they 

have chosen in the writing part. The role of the interviewer is more of a facilitator to welcome the 

candidate, to unwind by talking about personal information and then approaching the topic with some 

discussion questions (the same as in the writing part). The interviewer is only supposed to interfere if 

he/she perceives that the candidate needs, and will benefit from, a supportive or provocative question. 

The two parts of the test add validity and reliability to the test as it provides more data for a 

better-informed decision about the level of the candidate. The examiner has the chance to verify and 

explore both in written and also in oral form different structures of the English language, from simple 

to more sophisticated, vocabulary range, pronunciation and, ultimately, the candidate’s command of the 

language (CEFRL, 2001; Coombe, Folse & Humbley, 2007).   

The meticulous process of designing the indicators for placement test 2 (written test and oral interview) 

was informed by the literature presented here and resulted in an assessment matrix for written 

expression and another one for oral expression. These two documents have helped the staff attenuate 

the problem of misplaced students in the sense that: 1) the indicators have been aligned with the course 

or program curricular goals (Richards, 2009); 2) with this alignment of the indicators and curricular 

goals, validity and reliability (Coombe, Folse & Humbley, 2007) were increased; 3) the specifications 

or the indicators of the test reflect its purpose (Brown, 2004); 4) the indicators echo the specifications 

of the CEFRL (2001), making it easily relatable to standardized proficiency levels; 5) the indicators are 

carefully and accurately written expressing the skill or goal the learner is expected to achieve (Barrett 

et al., 2018); and finally, 6) because this project was undertaken as a collaborative task, the staff of the 

English Department was able to have a more comprehensive understanding of the complexity of why, 

what and how to assess in order to get better results (Raupp & Kolb, 1990; Raupp & Reichle, 2003).  

Table 1 shows the Oral Assessment Matrix for Placement Testing, one of the products of the project, 

which is now in its Phase 4, the piloting stage. Some adjustments have been made on the punctuation 

for the levels at the CL and in the instruments of Test 2. Candidates are now required to take the 

multiple-choice test, then the essay exam and, lastly, do the oral interview. There was consensus that 

the candidates screened for test 2 required that more information was collected in order for the 

examiner to make a more informed decision about his/her level (Raupp & Kolb, 1990; Raupp & 

Reichle, 2003). The document was designed in Portuguese, candidates’ and teachers’ native language, 

for the sake of transparency and also for it to be a model for the other departments at the CL.  

 

 

 

 



www.scholink.org/ojs/index.php/fce           Frontiers of Contemporary Education             Vol. 2, No. 2, 2021 

76 
Published by SCHOLINK INC. 

Table 1. Placement Testing Matrix for Oral Assessment 

Níveis/Faixas de 

Pontuação/Pontuação 

VALORES FLUÊNCIA VOCABULÁRIO GRAMÁTICA PRONÚNCIA 

B2 13-16 pontos 

ING 9 e ING 10 
4 

Discurso com certa 

espontaneidade, 

coerente, com raras 

pausas para aprimorar 

e/ou reformular ideias, 

sem prejuízo ao fluxo 

da fala. 

Expõe opinião sobre 

temas globais e ideias 

abstratas com 

variedade lexical. 

De modo geral, faz 

uso de estruturas 

complexas com raras 

inadequações que não 

comprometem a 

inteligibilidade da fala. 

Pronúncia clara com 

entonação próxima a 

da língua alvo. Raros 

lapsos que não 

comprometem a 

inteligibilidade da fala. 

B1 9-12 pontos 

ING 7 e ING 8 
3 

Discurso coerente, 

porém, simples, com 

pausas curtas para 

resolver algum 

problema de 

construção linguística, 

exigindo algum 

esforço para 

expressar-se. 

Expõe posicionamento 

sobre temas cotidianos 

e ideias em geral com 

léxico simples. 

Faz uso de estruturas 

gramaticais complexas 

com algumas 

inadequações 

decorrentes de 

interferência de L1. 

Pronúncia, em geral, 

clara, com entonação 

regular e lapsos 

pontuais. 

Menor ou igual a 

ING 4 e ING 3 

 

2 

Discurso 

compreensível com 

pausas evidentes para 

resolver algum 

problema de 

construção linguística, 

que causam 

interrupções 

frequentes ao fluxo da 

fala. 

Discorre sobre temas 

do cotidiano e 

expressa ideias com 

léxico simples e 

interferências 

ocasionais da L1. 

Faz uso de estruturas 

simples com 

inadequações 

decorrentes de 

interferência da L1 e 

que, por vezes, 

comprometem a 

inteligibilidade da fala. 

Pronúncia com 

interferência da L1 

e/ou de outras línguas  

que, por vezes, 

compromete a 

inteligibilidade da fala. 

A decisão final se dará 

a partir do 

alinhamento entre 

Entrevistador e 

Corretor. 

1 

Discurso com 

constantes pausas e 

hesitações, que 

causam interrupções 

ao fluxo da fala. 

Expressa ideias 

simples com frases 

curtas, com 

interferências 

constantes da L1. 

Faz uso de estruturas 

sem flexão de tempos 

verbais e/ou 

concordância, com 

ordem invertida de 

palavras. 

Pronúncia com forte 

interferência da L1 

e/ou de outras línguas 

que frequentemente 

afeta a inteligibilidade 

da fala. 

Source: Núcleo de Línguas, Programa Cursos de Línguas (UFES). 



www.scholink.org/ojs/index.php/fce           Frontiers of Contemporary Education             Vol. 2, No. 2, 2021 

77 
Published by SCHOLINK INC. 

5. Conclusion: Drawbacks & Affordances 

In this paper, we described the process of redesign of indicators of the assessment documents of the 

Languages Center at the Federal University of Espírito Santo, Brazil. The staff of the English 

Department, in special, took on an extra project, which was to redesign the placement testing 

documents. The desire to implement such project grew out of the observation from the teachers and 

coordination staff of misplaced students after an adjustment to the assessment system around 2013. 

After observing the phenomenon, the English Department staff took action and set off on a qualitative 

research approach which encompassed a theoretical embedding phase, a calibration phase and a 

piloting phase. The outcome was a new assessment matrix for placement test type 2 (see above) for oral 

and written expression and also new placement testing instruments. As per the hypothesis and research 

questions we posed, as we observe the consistency of results of placements for the past year, this 

suggests that the assessment documents, indeed, to some extent influenced the misplacements 

happening before the changes. However, a more detailed study should be carried out to confirm.  

As for the drawbacks encountered on the way, speaking specifically of the English Department and 

their attempt to redesign the placement testing documents, because this is the largest department at the 

CL, with the greatest concentration of novice teachers, it was (still is!) difficult for novice teachers to 

conduct the placement test (type 2) using these documents. The novice teachers may not have a very 

comprehensive understanding of the program curricular goals and how they branch out into the 

benchmark levels. Not only do they have little grasp of the curricular goals for different proficiency 

levels, but also, they had little experience teaching the levels of the course to see it in practice. This fact 

concentrated the work of placement tests on more experienced teachers, who represent a smaller 

contingent at the CL. Ideally, all teachers should be able to conduct the placement test. This issue has 

been recognized and training sessions are in order to tackle the problem. A strategy used when 

designing the indicators was to attribute numerical values for the areas and indicators so that the sum of 

the indicators represents the level placed. Another difficulty faced by the staff when designing the 

indicators was to make them independent from one another (Barrett et al., 2018). Because we 

understand the skill or linguistic knowledge being assessed in holistic way, it was hard not to make one 

indicator dependent on the other. There were countless and long debates on one single word or line, 

which made the process quite exhausting. 

On the brighter side of things, we can count some affordances from this undertaking. The staff has 

noticed less complaints about misplacements both from teachers and from students. There have been, 

of course, some small level adjustments, which are part of the placement process. As mentioned 

elsewhere in this paper, the fact that this was a teacher-led project, as opposed to a top-down decision 

made by the coordination staff, the redesign of the indicators was beneficial to all as we had to go back 

to the foundations of assessment and reassure our language teaching and learning philosophies. This 

exercise helped us look at both the internal (e.g., the CL’s assessment documents) and the public 

documents (e.g., the CEFRL) and make the necessary links between them to have a clearer idea of what 



www.scholink.org/ojs/index.php/fce           Frontiers of Contemporary Education             Vol. 2, No. 2, 2021 

78 
Published by SCHOLINK INC. 

our program curricular goals are and also to have a more global view on language proficiency. In times 

of the internationalization of education worldwide, this understanding is crucial to make the institution 

move forward. Finally, the experience of designing indicators for a placement test in dialogue with 

internal and external documents gave us the experience to work with assessment grids, which are used 

worldwide to assess students and help us minimize the subjectivity of any assessment endeavor.  

 

Acknowledgements  

The author would like to acknowledge the work of the UFES’ Languages Center English Department 

staff during this project. Special thanks to: Sandra Prates, Leni Puppin, Wendell Máximo, Gustavo 

Reges, Danilo Marinho, Manoel Sampaio Júnior, Danuza Fonseca e Sandlei Moraes.  

 

References 

Almeida, J. (2018). Approaches and Tools for Internationalisation at Home. Bolzano, Italy. 08 set. 

Conference. 

Bachman, L., & Palmer, A. (1996). Language testing in practice: Designing and developing useful 

language tests. Oxford University Press. 

Barrett, M. et al. (2018). Reference Framework of Competences for Democratic Culture, Descriptors 

for democratic culture. Council of Europe, 2. 

Brown, H. D. (2004). Language Assessment: Principles and Classroom Practices: United Stated of 

America. A Pearson Education. 

Coombe, C. A., Folse, K. S., & Hubley, N. J. (2007). A practical guide to assessing English language 

learners. University of Michigan Press. 

Council of Europe. Council for Cultural Co-operation. Education Committee. Modern Languages 

Division. (2001). Common European Framework of Reference for Languages: Learning, teaching, 

assessment. Cambridge University Press. 

Creswell, J. W. (2007). Projeto de pesquisa métodos qualitativo, quantitativo e misto. In Projeto de 

pesquisa métodos qualitativo, quantitativo e misto. Artmed. 

Raupp, M., & Kolb, F. A. (1990). Evaluation Management Handbook. Andover, MA: Network, Inc. 

Raupp, M., & Reichle, A. (2003). Avaliação: Ferramenta para melhores projetos (p. 251). Santa Cruz 

do Sul: EDUNISC. 

Richards, J. C. (2001). Curriculum development in language teaching. Ernst Klett Sprachen. 

https://doi.org/10.1017/CBO9780511667220 

Rossman, G. B., & Rallis, S. (1998). Analyzing and interpreting data. In G. B. Rossman, & S. Rallis 

(Eds.), Learning in the field: An introduction to qualitative research. Sage Publications.  

 

 

 

https://doi.org/10.1017/CBO9780511667220


www.scholink.org/ojs/index.php/fce           Frontiers of Contemporary Education             Vol. 2, No. 2, 2021 

79 
Published by SCHOLINK INC. 

Notes 

Note 1. The CL uses a unified performance-based assessment system and the staff from all departments 

felt the need to change/redesign/refine the assessment documents. The staff of the English department 

went a step further and decided to invest time and energy on assessment documents for its placement 

testing procedures. In this paper, we are referring to the whole process of refinement of the documents, 

but with a special focus on the placement testing procedures and documents of the English department.  

Note 2. There were two tasks going on at the same time: 1) the redesign of the institutional assessment 

documents being carried out by the CL’s whole staff, and 2) the redesign of a specific assessment 

document/matrix for placement testing purposes being carried out by the English department staff only. 

We are focusing on the latter. 

 


