https://doi.org/10.21009/JISAE

JISAE (Journal of Indonesian Student Assessment and Evaluation)

ISSN : P-ISSN: 2442-4919│E-ISSN: 2597-8934
Vol 7 No 2 (2021)

Website : http://journal.unj.ac.id/unj/index.php/jisae

CONTENT VALIDITY AND CONSTRUCT VALIDITY

ON THE DEVELOPMENT OF ACADEMIC ANXIETY

QUESTIONNAIRE

Muawanah1

STABN Sriwijaya, Indonesia

Ahsanul Khair Asdar2

STABN Sriwijaya, Indonesia

ABSTRACT

This study was a research and development which aimed to
develop an academic anxiety questionnaire for students through

content validity and construct validity testing using ADDIE model.

This research involved 205 students in the first trial phase and 220

students in the second trial phase. They were selected using the
cluster random sampling method. The data were collected using a

survey method by distributing the questionnaires using the google

form. The results showed that the academic anxiety questionnaire

was developed by containing the dimensions of cognitive

symptoms, emotional symptoms, and physical symptoms. The
coefficient of content validity through the Aiken's V by four experts

for each item was more than 0.200, while the reliability was 0.893.

The first trial phase showed that eight items dropped through the

Confirmatory Factor Analysis (CFA) with Construct Reliability
(CR) and Variance Extracted (VE) values for each measurement

model were 0.881 and 0.202, while the CR and VE values for the

structural model were 0.888 and 0.730. The second trial phase

showed that four items dropped with the CR and VE values for the

measurement model were 0.873 and 0.213, while the CR and VE
values for the structural model were 0.956 and 0.880. The final

academic anxiety questionnaire contains 27 items with a modified

Likert scale (four choices).

Keywords: Development, Questionnaire, Academic Anxiety,
ADDIE, CFA, Content Validity, Construct Validity,

Reliability

Address for

Correspondence:

1muawanah@stabn-
sriwijaya.ac.id
2ahsanul.khair@stabn-

sriwijaya.ac.id

INTRODUCTION

To improve the quality of education in Indonesia, the government administrates the

examinations, including the National Examination at all educational levels. These various
forms of it will cause anxiety for the students, the anxiety of failure. Nasution and Fasti

stated that being too anxious about facing the national examination will affect the students'
academic life, such as difficulty concentrating. It also has the worst effect on their learning

methods, academic competition, and self-confidence (Variansyaah & Listiara, 2017).
Anxiety occurs because someone is worried that it will disrupt their emotional

security or personal security. Anxiousness is closely related to the strength or weakness of

self-confidence, and more broadly, it is closely related to the personal characteristics.

Anxiety can occur when someone faces certain circumstances. For example, when facing an

exam, fear of not being able to meet the demands of academic assignments, and so on
(Setyobroto, 2004; Siddiqui & Rehman, 2014).

http://journal.unj.ac.id/unj/index.php/jisae

65 |JISAE (Journal of Indonesian Student Assessment and Evaluation)|Volume 7 Number 2

Anxiety is an unpleasant affective expressed in many terms, such as anxiety, worry,
concern, etc. Affective feelings refer to the subjective emotions of a person, namely how a

person feels anxious, angry, afraid, depressed, guilty, or embarrassed. Anxiety as an
intervening variable is a hypothetical or theoretical reaction caused by almost any stimulus

and ultimately has further consequences. In other words, anxiety is a response that mediates
or be between a stimulus and another response. Intervening variables can be known

indirectly from the antecedents and observable consequences. So no one cannot take the

anxiety but can observe the condition of the stimulus that causes anxiety and its
psychological or behavioral consequences (Lazarus, 1969; Borich, 1996; Elliot, 2000; Jamaris,

2010; Prawitasari, 2012; Santrock, 2014; Shahrouri, 2016).
The main challenge faced by every educational institution is how and what to do so

that graduates can realize the goals and ideals of the institution. In simple words, we can say

what experiences should be given to students during their education to make their profiles
or competencies developed from their profiles or competencies as new students. The

introduction of new students profiles is the key to presenting the experiences to the
students. The impact of the experience can not be in line with expectations when they are

inappropriate. It can even be contrary to what we want (Hanifah & Abadi, 2018).
The students not only get anxiety when they go through the educational process.

They also can be anxious at the beginning of their entry into college. It is in line with

previous research conducted by Sanitiara and Nazriati that medical students in the first year
are at risk of experiencing more stress. In the first year of university, many students were

anxious at a high level. It could be caused by academic pressure, not being familiar with the

new educational environment, and high expectations from family, the surrounding

community, themself, and educators (Sanitiara & Nazriati, 2014).

New students are very vulnerable to psychological pressure. These pressures arise
because they experience changes in their roles and academic environment. When they enter

the university, they face a different learning and teaching process from their former school
environment. In this environment, they will know the various abilities possessed by other

students that they do not know (Ati, Kurniawati, & Nurwanti, 2015). In this phase, new
students are vulnerable to their abilities because they compare themselves to the other

students.

Not only as a new student or during the educational process, but a student can also
experience anxiety when they are about to complete a series of education in college. It can

be categorized as anxiety in determining the future. Anxiety about the future is an
unpleasant emotion associated with various problems that must be faced in its development

period that affect the affective, cognitive, and behavioral aspects (Siburian, Karyono, dan
Kaloeti, 2010: 41). Feldman, Olds, and Papalia state that choosing to continue their

education or enter the world of work is a problem experienced by students after

completing higher education (Apriliana, 2016). The students who decide to enter the
workforce after graduation will face a new status as job seekers or unemployed. This

narrative ultimately explains that students will always experience academic anxiety but to a
different degree.

Academic anxiety is a common phenomenon that commonly occurs among all

students at various levels of education. Anxiety has a good impact on students who respond
positively, but on the contrary, it will hurt students who respond negatively (Siddiqui &

Rehman, 2014; Sanitiara & Nazriati, 2014; Irman, 2015). Bandura explained that academic
anxiety is an unpleasant feeling of worry because of confidence in one's ability to cope the

academic tasks (Prawitasari, 2012).
There are four characteristics of academic anxiety. They are the pattern of anxiety

engendering mental activity, misdirected attention, physiological distress, and inappropriate

(Ottens, 1991). Meanwhile, Cassady and Johnson stated that academic anxiety could be
monitored by cognitive symptoms, emotional symptoms, and bodily symptoms (Cassady &

Johnson, 2002). Casbarro also explained that anxiety is a collaboration and a combination of
three uncontrolled things. They are cognitive manifestations, affective manifestations, and

motoric behavior (Suratmi & Taufik, 2017).
Stuart and Laraia suggested that the anxiety experienced by a person can be

categorized into four levels, namely mild, moderate, severe, and very severe. Regardless of

the form of the anxiousness experienced by students, in the end, it will affect their academic
achievement in the lecture process, including how they get these achievements. One of the

existing problems in education is integrity, where students commit academic fraud, such as
cheating or even plagiarism (Annisa, Surani, & Mirwanti, 2018). The next problem is what

kind of instruments can measure and detect academic anxiety experienced by students,

especially in the university? It ultimately attracted the interest of researchers to develop an
instrument to measure the students' academic anxiousness through content validity and

construct validity testing.

METHOD
This study was a research and development. It's aimed to develop an academic anxiety

instrument for students using the ADDIE model. The sample used in this study was 205

students for the first trial phase and 220 students for the second trial phase. They were
selected using the cluster random sampling technique. Content validity analysis was carried

out through Aiken's V and Hoyt formula involving four experts, while construct validity was

measured through factor analysis using Confirmatory Factor Analysis (CFA).

RESULTS AND DISCUSSIONS
1. Analyze

The safe and comfortable learning process contributes to determining the student's
success in learning. However, the psychological factors of a student are hard to be identified.

It is because of their ability to cover up and hide these psychological factors at their age.
One of these psychological factors is academic anxiety. So to facilitate the measurement of

academic anxiety, it is necessary to develop an academic anxiety instrument.

According to the fact, there is no scientific research that measures student academic
anxiety at the college level. Cassady and Johnson stated that academic anxiety can be

monitored through cognitive symptoms, emotional symptoms, and bodily symptoms (Irman,
2015). Casbarro also explained that anxiety is a collaboration and a combination of three

uncontrolled things. They are cognitive manifestations, affective manifestations, and motoric
behavior (Suratmi & Taufik, 2017). So, academic anxiety can be interpreted as a description

of the anxiety experienced by students in the academic nuances and expressed in the form

of scores obtained through an academic anxiety scale with dimensions of cognitive
symptoms, emotional symptoms, and physical symptoms.

2. Design
The initial grille of the academic anxiety instrument is shown in Table 1.

Tabel 1 Initial Grille of The Academic Anxiety Instrument

No. Dimension Indicators
Items

Favorable Unfavorable

1.
Cognitive

Symptoms

Anxious or afraid of failure 1 and 2 3

Irrelevant in thinking 4 and 6 5

Limitations in thinking 7, 8, and 9 -

2.
Emotional

Symptoms

Easy to get dizzy 11 and 12 10

Easy to feel panicked or scared 13 14 and 15

Easy to feel tense 16 and 17 18

Having trouble sleeping 19, 20, and 21 -

67 |JISAE (Journal of Indonesian Student Assessment and Evaluation)|Volume 7 Number 2

No. Dimension Indicators
Items

Favorable Unfavorable

3.
Physical

Symptoms

Roll the eyes when angry 23 and 24 22

Shortness of breath 25, 26, and 27 -

The facial muscles look tense 28, 29, and 30 -

The intonation of the voice became louder 32 and 33 31

Experiencing cardiovascular symptoms 34 and 36 35

Experiencing urogenital symptoms 37, 38, and 39 -

Total 30 9

Each item compiled at this stage is equipped with a measurement scale in the form of a

Likert scale with five response options, namely Strongly Disagree, Disagree, Neutral, Agree,
and Strongly Agree.

3. Development
At this stage, the content validity coefficient is calculated by using Aiken's V formula.

Overall, the validity coefficient of the draft of each item is more than 0.200. It fulfills the

criteria of content validity. Meanwhile, based on the results of the calculation of the

reliability coefficient of Hoyt, the reliability coefficient was 0.893. So it means that the draft

of the academic anxiety instrument was reliable according to expert judgment.
4. Implementation
a. The First Trial Phase

The result of first stage showed that seven items are invalid because their t-value is
lower than 1.963 (df = 699; = 0.05). They are items number 2, 6, 13, 20, 25, 27, and 31.

Since there are seven invalid items, the second stage of the first-order CFA testing is carried
out by removing them. Figure 1 and Figure 2 show that 32 items are declared valid because

their t-value is more than 1.965 (df = 461; = 0.05). It means that the relationship between

the manifest variables and the dimensions is significant. Thus, the 32 items can be declared
feasible to use.

Figure 1 The Standardized Solution Figure 2 The T-Value

of The First-Order CFA of The First-Order CFA

The first stage of second-order analysis showed that item number 34 is invalid (t-value
= 1.86). The t-value is lower than 1.965 (df = 491 and = 0.05). A re-analysis was carried out

by removing item number 34 from the academic anxiety instrument. Figure 3 and Figure 4
show that the remaining 31 statement items on the academic anxiety instrument are valid

because their t-value is more than 1.966 (df = 431 and = 0.05). It means that the academic
anxiety instrument consists of 31 statement items in the first trial phase. All items developed

are declared capable of measuring indicators, dimensions, and variables.

Figure 3 The Standardized Solution Figure 4 The T-Value

of The Second-Order CFA of The Second -Order CFA

Figure 5 The Factor Analysis Figure 6 The Factor Analysis
Structural Model Standardized Solution Structural Model T-Value

of The First Trial Phase of The First Trial Phase

Figure 5 and Figure 6 show that the factor loading on the dimensions that make up the
construct is valid, with the factor loading value more than 0.5 and the t-value more than

1.966 (df = 431 and = 0.05). The factor loading on the dimension of cognitive symptoms is

69 |JISAE (Journal of Indonesian Student Assessment and Evaluation)|Volume 7 Number 2

0.74, the dimension of emotional symptoms is 0.80, and the dimension of physical symptoms
is 1.00. The higher the loading factor value, the more reliable the dimensions to form the

construct are. The t-value for each dimension sequentially is 4.26, 6.63, and 4.54.
The t-value of 31 items on the academic anxiety instrument is more than 1.965 (df =

461 and = 0.05). It means that items in the academic anxiety instrument are valid and
feasible to use. The calculation results showed that CR = 0.881 and VE = 0.202. So, the 31

items are good. Meanwhile, for the structural model, the CR = 0.888 and the VE = 0.730.

This values have fulfilled the requirements and far exceeds the critical limit. So, the reliability
of the measurement model (construct) is good.

b. The Second Trial Phase
The result of first stage showed that four items are invalid because their t-value is

lower than 1.966 (df = 431; = 0.05). They are items number 2, 11, 16, and 28. Since there

are four invalid items, the second stage of the first-order CFA testing is carried out by
removing them. Figure 7 and Figure 8 show that 27 items are declared valid because their t-

value is more than 1.965 (df = 461; = 0.05). Thus the 27 items can be declared eligible to be
used. Figure 8 shows that the t-value of all manifest variables is more than 1.967 (df = 321; =

0.05). It means that the relationship between manifest variables and dimensions is significant.

Figure 7 The Standardized Solution Figure 8 The T-Value

of The First-Order CFA of The First-Order CFA

Figure 10 shows that the t-value of 27 items on the academic anxiety instrument is

more than 1.967 (df = 321 and = 0.05). It means that items are declared construct valid. So,

the academic anxiety instrument consists of 27 items. All items can measure the indicators,

dimensions, and variables that were the focus of this study. Figure 11 and Figure 12 show
that the factor loading on the dimensions that make up the construct is valid, with the factor

loading value more than 0.5 and the t-value more than 1.967 (df = 321 and = 0.05). The

factor loading on the dimensions of cognitive symptoms is 0.93, the dimension of emotional
symptoms is 0.92, and the dimension of physical symptoms is 0.96. The higher the loading

factor value, the more reliable the dimensions to form the construct are. The t-value for
each dimension sequentially is 5.50, 3.26, and 7.87.

The t-value of 27 items on the academic anxiety instrument is more than 1.967 (df =
321 and = 0.05). It means that items in the developed academic anxiety instrument are valid.

The calculation results showed that CR = 0.873 and VE = 0.213. So, the 27 items are good.

Meanwhile, for the structural model, the CR = 0.956 and the VE = 0.880. These values have
fulfilled the requirements and far exceed the critical limit. So, the reliability of the

measurement model (construct) is good.

Figure 9 The Basic Model Standardized Figure 10 The Basic Model T-Value
Solution The Second-Order CFA The Second-Order CFA

of The Second Trial Phase of The Second Trial Phase

Figure 11 The Factor Analysis Figure 12 The Factor Analysis

Structural Model Standardized Solution Structural Model T-Value

of The Second Trial Phase of The Second Trial Phase

71 |JISAE (Journal of Indonesian Student Assessment and Evaluation)|Volume 7 Number 2

5. Evaluation
Due to the tendency of respondents in the first trial phase and second trial phase, they

tend to choose a neutral response. So, the measurement scale is adjusted for each
statement item. The measuring scale used is a modified Likert scale. The resulting academic

anxiety instrument finally contains 27 items. Each item is equipped with four choices, namely
strongly disagree, disagree, agree, and strongly agree. Based on the results, the indicators

that make up the dimensions of cognitive symptoms are anxiety or afraid of failure,

irrelevant in thinking, and limitations in thinking. The indicators that make up the dimensions
of emotional symptoms are easy to get dizzy, easy to feel panicked or scared, easy to feel

tense, and having trouble slipping. Meanwhile, the indicators that make up the dimensions of
physical symptoms are rolling the eyes when angry, shortness of breath, the facial muscles

looking tense, the intonation of the voice becoming louder, experiencing cardiovascular

symptoms, and experiencing urogenital symptoms.

CONCLUSION
Based on the results, the conclusions of this study are: (1) the academic anxiety

instrument was developed by ADDIE model, (2) the coefficient of content validity through
the use of Aiken's V formulation for each item is more than 0.200 with a reliability

coefficient = 0.893, (3) the first trial phase showed that eight items had to be dropped

through CFA so the CR and VE values of 31 items for the measurement model were 0.881
and 0.202, while the CR and VE values for the structural model were 0.888 and 0.730, and

(4) the second trial showed that four items had to be dropped through CFA so the CR and

VE values for the measurement model were 0.873 and 0.213, while the CR and VE values

for the structural model were 0.956 and 0.880.

REFERENCES

Annisa, L., Surani, & Mirwanti, R. (2018). Tingkat Kecemasan Mahasiswa Keperawatan dalam
Menghadapi Ujian Berbasis Computer Based Test. MEDIASAINS: Jurnal Ilmiah Ilmu-Ilmu

Kesehatan, 16(2), 67-75.
Apriliana, R. (2016). Hubungan antara Kecemasan terhadap Karir Masa Depan dan Konsep

Diri Akademik dengan Prokrastinasi Skripsi pada Mahasiswa. Jurnal Proyeksi, 11(1), 24-

35.
Asdar, A. K. (2018). Ketidakwajaran Skor Seleksi Penerimaan Mahasiswa Baru. Vijjacariya, 1-

16.
Ati, E., Kurniawati, Y., & Nurwanti, R. (2015). Peran Impostor Syndrome dalam Menjelaskan

Kecemasan Akademik pada Mahasiswa Baru. Jurnal Mediapsi, 1(1), 1-9.
Borich, G. (1996). Effective Teaching Methods. New Jersey: Prentice Hall.

Cassady, J., & Johnson, R. (2002). Cognitive Test Anxiety and Acaemic Performance.

Contemporary Educational Psychology, 27, 270-295.
Elliot, S. N. (2000). Educational Psychology: Effective Teaching, Effective Learning. London:

McGraw-Hill.
Hanifah, & Abadi, A. (2018). Hubungan antara Pemahaman Konsep dan Kecemasan

Menghadapi Mata Kuliah Teori Grup dengan Prestasi Akademik Mahasiswa. Kreano:

Jurnal Matematika Kreatif-Inovatif, 9(2), 156-153.
Irman. (2015). Pengelolaan Kecemasan Akademik Siswa melalui Pelatihan Regulasi Emosi.

Jurnal Al-Qalb, 7(2), 62-71.
Jamaris, M. (2010). Orientasi Baru dalam Psikologi Pendidikan. Jakarta: Yayasan Penamas Murni.

Lazarus, R. S. (1969). Patterns of Adjustment and Human Effectiveness. New York: McGraw-Hill
Book Company.

Manoppo, Y., & Mardapi, D. (2014). Analisis Metode Cheating pada Tes Berskala Besar.
Jurnal Penelitian dan Evaluasi Pendidikan Tahun 18, 115-128.

Ottens, A. J. (1991). Coping with Academic Anxiety. New York: The Rosen Publishing Group.
Prawitasari, J. E. (2012). Psikologi Terapan Melintas Batas Disiplin Ilmu. Jakarta: Erlangga.

Sanitiara, & Nazriati, E. (2014). Hubungan Kecemasan Akademis dengan Regulasi Diri dalam
Belajar pada Mahasiswa Tahun Pertama Fakultas Kedokteran Universitas Riau Tahun

2013/2014. Jurnal Online Mahasiswa Fakultas Kedokteran, 1(2), 1-9.

Santrock, J. W. (2014). Psikologi Pendidikan . Jakarta: Salemba Humanika.
Setyobroto, S. (2004). Psikologi suatu Pengantar. Jakarta: Percetakan Solo.

Shahrouri, E. (2016). Sources of Academic Anxiety Among Undergraduate Students
Contemporary Study Between Private and Fovernment Universities. Journal of

Emerging Trends in Educational Research and Policy Studies (JETERAPS), 7(2), 118-124.

Siddiqui, M. A., & Rehman, A. U. (2014). An Interactional Study of Academic Anxiety in
Relation to Socio-Economic Status, Gender, and School Type Among Secondary

School Students. International Journal of Educatonal Research and Technology, 5(2), 74-79.
Suratmi, A. R., & Taufik, M. (2017). Hubungan antara Tingkat Kecemasan dengan Hasil

Belajar Mahasiswa di Program Studi Pendidikan Biologi Untirta. Jurnal Pembelajaran
Biologi, 4(1), 71-76.

Variansyaah, V., & Listiara, A. (2017). Hubungan Orientasi Tujuan Performa dengan

Kecemasan Akademik pada Siswa Kelas X di SMA Negeri "A" Semarang. Jurnal Empati,
6(1), 419-424.