





































English Language Teaching Educational Journal   ISSN 2621-6485 

Vol. 7, No. 2, August 2024, pp. 90-108  

       https://doi.org/10.12928/eltej.v7i2.11572        http://journal2.uad.ac.id/index.php/eltej/index          eltej@pbi.uad.ac.id  

OpenAI ChatGPT vs Google Gemini: A study of AI chatbots’ 

writing quality evaluation and plagiarism checking 

Siraprapa Kotmungkun a,1, Wichuta Chompurach b,2,*, Piriya Thaksanan c,3 

a, b, c Kasetsart University Chalermphrakiat Sakon Nakhon Province Campus, 59/6 Moo 1, Vor Por Ror Or 366 Road, Chiang Khruea, 

Sakon Nakhon, Thailand 
1 siraprapa.ko@ku.th ; 2 wichuta.c@ku.th*; 3 peereeya.t@ku.th   

 

A R T I C L E  I N F O 

 

A B ST R ACT   

 

 

Article history 

Received 14 September 2024 

Revised 21 September 2024 

Accepted 24 October 2024 

 This study explores the writing quality of two AI chatbots, OpenAI 
ChatGPT and Google Gemini. The research assesses the quality of the 
generated texts based on five essay models using the T.E.R.A. software, 
focusing on ease of understanding, readability, and reading levels using 
the Flesch-Kincaid formula. Thirty essays were generated, 15 from each 
chatbot, and evaluated for plagiarism using two free detection tools—
SmallSEOTools and Check-Plagiarism—as well as one paid tool, 
Turnitin. The findings revealed that both ChatGPT and Gemini 
performed well in terms of word concreteness but demonstrated 
weaknesses in narrativity. ChatGPT showed stronger performance in 
referential and deep cohesion, while Gemini excelled in narrativity, 
syntactic simplicity and word concreteness. However, a significant 
concern was the degree of plagiarism detected in texts from both AI tools, 
with ChatGPT's essays exhibiting a higher likelihood of plagiarism 
compared to Gemini’s. These findings highlight the potential limitations 
and risks associated with using AI-generated writing. 

 

This is an open access article under the CC–BY-SA license. 

    

 

 
Keywords 

AI Chatbot  

L2 writing  

Plagiarism  

Writing Evaluation  

Google Gemini  

OpenAI ChatGPT 

 

 

How to Cite: Kotmungkun, S., Chompurach, W., & Thaksanan, P. (2024). OpenAI ChatGPT vs Google Gemini: 
A study of AI chatbots’ writing quality evaluation and plagiarism checking. English Language Teaching 
Educational Journal, 7 (2), 90-108. https://doi.org/10.12928/eltej.v7i2.11572    

1. Introduction 

Over the past decade, it has become the new era of artificial intelligence (AI). The term "artificial 
intelligence" (AI) was coined by McCarthy et al. in 1955 to describe machines and processes that can 
mimic human intelligence and make decisions like humans (McCarthy et al., 1955; Zhai, 2023). AI 
has increasingly integrated into our everyday lives and routine activities (Lee & Park, 2023). The 
development of AI models, such as OpenAI ChatGPT (Generative Pre-trained Transformer) and 
Google Gemini, is one of significant technological breakthroughs that has attracted a lot of attention 
in various fields including healthcare, education, research, journalism, and industry (Evans et al., 
2023; Zhai, 2023). This was because of the high quality of their outputs across a wide range of topics 
and their language and content accuracy (Stahl & Eke, 2023). Moreover, since the particular AI 
models use natural language (NL) text generation, the models can generate “human-sounding” (p. 
107) texts in different languages, from a word level to a paragraph level, on almost any topic and in 
any writing style (Jovanovic, 2022).  

  

https://doi.org/10.12928/eltej.v7i2.11662
http://journal2.uad.ac.id/index.php/eltej/index
mailto:eltej@pbi.uad.ac.id
mailto:wichuta.c@ku.th*
mailto:peereeya.t@ku.th
http://creativecommons.org/licenses/by-sa/4.0/
https://doi.org/10.12928/eltej.v7i2.11572
http://creativecommons.org/licenses/by-sa/4.0/
http://crossmark.crossref.org/dialog/?doi=10.12928/eltej.v7i2.11572&domain=pdf
https://orcid.org/0009-0009-8572-6612
https://orcid.org/0000-0002-0883-9711
https://orcid.org/0009-0008-3686-9935


ISSN 2621-6485 English Language Teaching Educational Journal 91 
 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

In terms of education, especially in EFL (English as a Foreign Language) writing classrooms, 
technologies have been used for various purposes. For example, machine translation i.e. Google 
Translate has been used in English writing classrooms because it was viewed as a helpful, reliable tool 
helping learners complete their writing tasks (Chompurach, 2021). Also, AI technologies have been 
used in completing writing tasks because of their benefits that were time-saving and easy to use as 
well as access (Suaverdez & Suaverdez, 2023). Therefore, the particular technologies have been used 
in L2 writing from brainstorming to draft editing (Lingard, 2023); moreover, learners could receive 
their writing skills’ strengths, weaknesses, and concerning comments from AI (Barrot, 2023). 
However, there have been ethical considerations in the use of AI technologies in writing that were 
authorships, the potential for biases and discrimination, and the risk of plagiarism (Suaverdez & 
Suaverdez, 2023). 

1.1. Large Language Models: OpenAI ChatGPT and Google Gemini 

In November 2022, OpenAI, a research laboratory focused on artificial intelligence, released 
ChatGPT (Tlili et al., 2023). ChatGPT, an OpenAI-developed web-based AI system that utilizes GPT 
large language models (LLM), is projected to revolutionize numerous facets of society (Stahl & Eke, 
2023). According to OpenAI, ChatGPT dialogue format enables it to respond to follow-up questions, 
admit errors, challenge faulty assumptions, and refuse inappropriate requests (OpenAI, 2022; Zhai, 
2023). In terms of ChatGPT3.5, it is the first version of ChatGPT which is a large language model 
from OpenAI. It is built on top of InstructGPT, which is a GPT3.5 model that has been fine-tuned on 
a large dataset of conversational text. ChatGPT3.5 has also been fine-tuned using reinforcement 
learning from human feedback, which helps it to generate text that is more aligned with human 
preferences (Espejel et al., 2023). 

In December 2023, the evolution of Google’s Large Language Models (LLMs) called Google Bard 
took a significant step forward with the advent of Large Multimodal Models (LMMs) called Google 
Gemini. For Bard, it has already made a remarkable impact. This means users can access Bard through 
an easy-to-use interface, which allows them to collaborate directly with a generative AI system 
(Espejel et al., 2023). For Gemini, LMMs seek to build upon the strengths of LLMs by incorporating 
multi-sensory processing capabilities. This expansion aims to achieve more robust and general 
intelligence, enabling computers to interact with humans on a level closer to how humans naturally 
communicate with each other (Yang et al., 2023 as cited in Lee et al., 2023). 

As mentioned earlier, generative AI (GenAI) is used in various fields including education. This is 
because GenAI is a powerful tool that can be used to enhance students' learning experience in higher 
education and can respond to user prompts to generate highly original outputs, which can be used for 
a variety of educational purposes (Chan & Hu, 2023) including second language (L2) writing (Barrot, 
2023). 

1.2. AI Large Language Models Use in Education Settings 

During the first decade of the 21st century, significant progress in machine learning and natural 
language processing enabled the development of more sophisticated AI-powered educational tools 
(Zhai, 2023). For example, these tools could tailor their approach to the individual needs of each 
learner, provide customized instruction, and even assess assignments (Zhai, 2023). With these various 
benefits, many scholars, instructors, and educators have paid attention to both benefits and drawbacks 
of these models to education settings. 

In 2023, the study of Baidoo-Anu and Ansah was conducted to explore how interactive ChatGPT 
was and what its potential benefits and drawbacks on education were. In doing so, the researchers 
analyzed the research articles on ChatGPT from November 2022 - March 2023; the data revealed both 
these LLMs’ pros and cons. On the one hand, ChatGPT was viewed as a personal tutor, an automated 
essay evaluation, a translator, an interactive virtual tutor, and adaptive learning provider. On the other 
hand, these LLMs were perceived as a virtual tutor lacking human interaction, a limited-understanding 
system, a biased model, a less creative chatbot, a model depending on trained data, a limited contextual 
understanding system, and a limited ability instructor.  

These findings in terms of the LLM benefits are in line with the study of Evans et al (2023). Evans 
et al (2023) have conducted a study to explore the impacts of ChatGPT on access-efficiency, 
employment, and education. The findings revealed ChatGPT can be used to improve access and 
efficiency in a variety of industries and for individuals with disabilities or language barriers. ChatGPT 



92 English Language Teaching Educational Journal   ISSN 2621-6485 

 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

can be used to create translation tools that can help non-native speakers understand and communicate 
in a foreign language. For employment, ChatGPT created new job opportunities in AI and NL 
processing but also raising concerns about job displacement. Moreover, for education, ChatGPT can 
be used to improve language learning and personalized education. It can provide instant feedback, 
explanations, and examples to help learners improve their grammar, vocabulary, and comprehension. 
Additionally, ChatGPT can be used to enhance the accessibility and inclusivity of education by 
providing support for students with disabilities or language barriers. This research showed ChatGPT 
can help to break down language barriers and give everyone equal access to information, services, and 
education. 

Similarly, Tlili et al (2023) also found that ChatGPT helped users by providing clear and concise 
explanations of complex topics, using language that was appropriate for the target audience. The 
researchers have conducted a qualitative research study to investigate chatbots in education. To do 
this, the study focused on ChatGPT as an example of an advanced chatbot that was being used by 
early adopters. The findings of the study indicated that ChatGPT has the potential to transform 
education in a variety of ways. The majority of participants in the study indicated that ChatGPT was 
effective in increasing the likelihood of educational success by providing users (teachers and students) 
with basic knowledge of various topics. Furthermore, ChatGPT was acknowledged by the participants 
as an efficient tool for providing information of a wide range of complex topics in a clear and concise 
language. Conversely, a few participants held the opposing view that the misuse of ChatGPT by 
learners could also diminish their innovative capacities and critical thinking skills. 

In terms of university students’ voices on Gen AI technologies including ChatGPT and Google 
Bard, Chan and Hu (2023) have conducted a survey with closed-ended and open-ended questions to 
explore 399 university students’ use and perceptions toward GenAI technologies mentioned earlier. 
The findings showed the students understood the limitations of GenAI abilities, but the overall 
students still had positive attitude toward these technologies. Moreover, the students found these 
GenAI technologies were great tools because of 24/7 availability and the technologies can help them 
save time. However, they had got concerns about the way these technologies can undermine the 
university education value.   

From previous studies on the impacts of the LLMs on education, the findings showed ChatGPT’s 
both benefits and drawbacks. On the one hand, ChatGPT still lacked human interactions (Baidoo-Anu 
& Ansah, 2023) and could reduce users’ innovative and critical thinking ability (Tlili et al., 2023). On 
the other hand, ChatGPT demonstrated exceptional competencies in various fields including machine 
translation (MT) (Baidoo-Anu & Ansah, 2023; Evans et al., 2023; Tlili et al., 2023). Students’ voices, 
however, showed that these GenAI technologies were great tools (Chan & Hu, 2023). Nevertheless, 
the potential impacts of ChatGPT and the other chatbot powered by Google called Gemini on 
education remain less explored. The following section delves into the use of LLMs in L2 writing, 
examining how these AI tools have been utilized and their potential implications for language learning 
and teaching.        

1.3. Use of Large Language Model in L2 Writing 

Writing is one of the most essential skills that L2 learners have to develop (Ariyanti, 2016; Hyland, 
2003; Meiranti, 2012), and it is a complex linguistic skill that requires cognitive processing to 
effectively convey thoughts and emotions in a written form (Hasnawati et al., 2023). Through this 
writing activity, L2 students were expected to master grammars, vocabulary words, punctuations 
(Meiranti, 2012), and they should take these aspects into account that are a clear intention, a well-
organized framework, appropriate word selection, effective language use, and coherent thought 
expression (Hyland & Jiang, 2017). In terms of essay writing models, Ariyanti’s study (2016) 
provided definitions of each essay type. 

1. Descriptive Essay: Provide a clear and detailed portrayal of the topic to effectively convey its 
essence; 

2. Classification Essay: Provides concise definitions of their characteristics, allowing readers to 
understand the distinctions between them clearly; 

3. Cause and Effect Essay: Employs the appropriate vocabulary and expressions that effectively 
convey the relationship between causes and their corresponding effects; 



ISSN 2621-6485 English Language Teaching Educational Journal 93 
 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

4. Comparison and Contrast Essay: Goes beyond identifying similarities and differences by 
structuring comparisons effectively and employing appropriate clauses of comparison, contrast, 
and concession; 

5. Argumentative Essay: Focuses on presenting well-reasoned arguments, supported by credible 
evidence, with careful evaluation of argument strength. 

With the complexity of L2 writing and the benefits of AI, several educators have adopted these 
technologies into their writing classrooms. This following research has reported the use of AI 
technologies in L2 writing classrooms. The study of Fitria (2023) reported the benefits, especially on 
English writing. In her research, Fitria (2023) studied how ChatGPT wrote English essays. Adopting 
a qualitative research paradigm, the researcher collected data from documents and observation and 
adopted Miles et al’s (2018) three steps including data reduction, data display, and conclusion to 
analyze the obtained data. The data revealed ChatGPT can address a wide range of prompts, including 
English essays encompassing descriptive narratives on any topics. ChatGPT adhered to the 
conventions of event sequencing and essay structure, employing both active and passive voice 
constructions while it also maintained appropriate tense usage aligned with the essay topic.  

For academic writing, Mohammed et al (2023) conducted a research study to explore the utilization 
of ChatGPT among Arab postgraduate students in India, with a particular focus on its impact on their 
academic writing. Through a questionnaire administered to 40 postgraduates enrolled in five 
universities across Kerala, the study revealed the perceived benefits and tangible effects of ChatGPT 
on their academic writing, language proficiency, and overall academic performance. The findings 
reported more than a half of the participants (n=29) were familiar with ChatGPT. In terms of the 
potential benefits of ChatGPT use, the highest mean score (mean= 3.07) was for the statement saying 
"ChatGPT texts are highly reliable and need not be second checked for reliability" (p. 11). Also, the 
data reported that ChatGPT was viewed as an effective tool for variety tasks because it helped them 
generate ideas, gather information, and enhance their writing skills. Therefore, most of the students 
used ChatGPT in these tasks including researching topics, translating texts, and writing assignment.  

In terms of second language (L2) writing, Barrot (2023) has explored the several advantages and 
challenges of ChatGPT. In the study, the researcher reviewed ten studies and suggested the potential 
benefits of the integration of this technology in L2 writing classrooms. The data revealed since 
ChatGPT enabled users to revise their requests and rejected any unsuitable ones, L2 learners could 
draw upon these features to conduct sustained and coherent conversations without the anxiety of being 
criticized. Also, ChatGPT provided extensive range of writing assistance, from simple to complex 
tasks. It could produce human-like texts which generally exhibited coherence and grammatical 
accuracy. This could make it a beneficial resource for users to enhance their writing skills and refine 
their language usage. Moreover, ChatGPT could generate outlines in various formats, including 
sentence, topic, alphanumeric, and decimal system structures. This feature provided L2 students with 
access to both a guiding framework and a foundation for crafting a well-structured outline tailored to 
their chosen topics. In terms of feedback, ChatGPT provided users their writing strengths, weaknesses, 
and specific comments by automatically grading following predefined criteria. Therefore, instead of 
completely prohibiting ChatGPT, educators can investigate strategies for integrating these AI-
powered tools into their teaching practices and harness their benefits (Barrot, 2023).  

In 2023, Suaverdez and Suaverdez (2023) have conducted a study to examine writing texts or 
papers by AI generators and test them with AI detectors. The researchers collected writing texts from 
different sources that were an original man-made paper, papers from AI chatbots that were ChatGPT 
(both free and paid), Phin (free), and Moonbeam (free). The results showed from a free tool called 
“Contentatscale”, the paper from ChatGPT (free) received the highest percentage (96%) and the paid 
one received 91% interpreted as “likely to be human” (p. 6). Moreover, OpenAI classifier (free) 
reported that papers from both free and paid ChatGPT were “very unlikely AI-generated” (p. 6) while 
ones from Phin (free) and Moonbeam (free) were reported “unclear if it is AI- generated” (p. 6). 
However, Origibnality.AI detected that papers from both free and paid ChatGPT and Phin (free) were 
100% AI while one from Moonbeam (free) were 99% AI. In terms of plagiarism checking from the 
paid checker, the paper from Phin received the highest percentage (12%) of plagiarized content while 
ones from ChatGPT free and paid and Moonbeam received 8%, 1% and 0%, respectively. For the free 
checker, the paper from Moonbeam received the highest percentage (3%) of plagiarized content while 
ones from ChatGPT paid and free and Phin received 1%, 0% and 0%, respectively. The researchers 



94 English Language Teaching Educational Journal   ISSN 2621-6485 

 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

also recommended careful evaluation, strategic planning, and establishment of guidelines and 
regulations for the utilization of AI tools in academic writing and school tasks to minimize the 
potential negative consequences. 

Reviewing the literature, the researchers found some mixed messages. On the one hand, the 
potential benefits of ChatGPT to English or L2 writing have been reported (Barrot, 2023; Fitria, 2023; 
Mohammed et al., 2023). On the other hand, there have been worries and concerns about the ethical 
issues of the AI-model use in L2 writing (Suaverdez & Suaverdez, 2023). Moreover, there has been 
limited studies exploring Gemini’s benefits and drawbacks with using in L2 writing. The present 
research aims to explore and compare the writing quality of ChatGPT and Google Gemini with the 
Text Ease and Readability Assessor (T.E.R.A.). The researchers chose to focus on these two tools, 
ChatGPT and Google Gemini, as they are at the forefront of language model innovation (Rane et al., 
2024). Both platforms demonstrate exceptional capabilities in engaging in complex conversations, 
producing textual content, and generating a wide variety of creative outputs, making them ideal for a 
comprehensive analysis of AI-generated writing innovation (Rane et al., 2024). Also, three plagiarism 
checkers, TurnItIn and the other free plagiarism checkers: check-plagiarism.com and 
smallseotools.com were applied to detect the percentages of plagiarized content of each chatbot.  

Research Questions 

1. What is the writing quality level of OpenAI ChatGPT and Google Gemini based on five essay 
models in terms of the easability and readability? 

2. What is the percentage of plagiarized content in the generated texts from OpenAI ChatGPT and 
Google Gemini as detected by plagiarism checkers? 

2. Methods 

The present study has been conducted following the multi-method qualitative approach (Davis et 
al., 2011). To do so, the researchers can acquire in-depth understanding regarding the quality of AI 
chatbots’, ChatGPT and Google Gemini, L2 generated writing tasks.  

2.1. Data Collection Tools 

1) Basic Prompts to Gather Data from AI Chatbots 

To enable both AI chatbots to provide expected responses, the researchers adopted the prompt 
engineering technique from Rayhan (2023). Prompt engineering is an essential tool for refining the 
accuracy and authenticity of AI language models because it can be crafted to align with the objectives 
of specific tasks or domains (Rayhan, 2023). To design effective prompts, the researchers have 
adhered to the design principles for basic prompts which consist of a few words or a short sentence 
and can be either generalized or task-specific. In doing so, the researchers can ensure that the AI 
language model generates accurate and relevant outputs for a wide range of tasks and users. The Table 
1 shows the basic prompts being used in the present study based on the types of essay models. 

Table 1.  Basic Prompts Used in the Present Study 

Essay Models Basic Prompts 

Descriptive Write a 300-word essay describing the most beautiful place in Thailand. 

Classification Write a 300-word essay classifying genres of music.  

Cause and Effect Write a 300-word essay about causes and effects of global warming. 

Comparison and Contrast Write a 300-word essay about living in a city vs. living in the country. 

Argumentative  Write a 300-word essay about the benefits of social media. 

 

  



ISSN 2621-6485 English Language Teaching Educational Journal 95 
 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

As a result, Table 2 shows the data the researchers obtained.  

Table 2.  The Overall Data Obtained from the Two AI Chatbots 

AI Chatbots Essays Written by AI Chatbots 
Word 

Count 

Average Word Count 

by Writing Type 

ChatGPT 

ChatGPT Descriptive 1 357 

359.33 ChatGPT Descriptive 2 368 

ChatGPT Descriptive 3 353 

ChatGPT Classification 1 370 
333.33 

 
ChatGPT Classification 2 321 

ChatGPT Classification 3 309 

ChatGPT Cause and Effect 1 314 
320.67 

 
ChatGPT Cause and Effect 2 318 

ChatGPT Cause and Effect 3 330 

ChatGPT Comparison and Contrast 1 339 

331 ChatGPT Comparison and Contrast 2 412 

ChatGPT Comparison and Contrast 3 440 

ChatGPT Argumentative 1 330 

330.67 ChatGPT Argumentative 2 326 

ChatGPT Argumentative 3 337 

Average Word Count by ChatGPT 335  

Gemini 

Gemini Descriptive 1 416 
398.67 

 
Gemini Descriptive 2 367 

Gemini Descriptive 3 413 

Gemini Classification 1 426 

399.33 Gemini Classification 2 382 

Gemini Classification 3 390 

Gemini Cause and Effect 1 339 

397 Gemini Cause and Effect 2 412 

Gemini Cause and Effect 3 440 

Gemini Comparison and Contrast 1 383 

385 Gemini Comparison and Contrast 2 393 

Gemini Comparison and Contrast 3 379 

Gemini Argumentative 1 428 

435.33 Gemini Argumentative 2 481 

Gemini Argumentative 3 397 

Average Word Count by Gemini 403.07  

 

2) T.E.R.A. (The Text Ease and Readability Assessor) Program 

After gathering data from the AI chatbots, the researchers are applying the T.E.R.A. program 
(McNamara et al., 2014) to assess the generated texts in terms of their easability and readability. To 
do so, the program analyzes text on five components and Flesch-Kincaid Grade Level. 

1. Narrativity: Generally, more story-like texts have higher narrativity scores, making them 
easier to read, though exceptions exist. 

2. Syntactic Simplicity: Higher simplicity scores occur in texts with fewer clauses and words 
per sentence, and greater structural similarity within paragraphs. 

3. Word Concreteness: Texts with more concrete words are easier to read, while abstract words 
lower readability. 

4. Referential Cohesion: Overlapping words or concepts between sentences improve cohesion, 
though low cohesion may sometimes prompt deeper reader engagement. 

5. Deep Cohesion: Refers to the seamless connection of ideas through varied use of connecting 
words. 

6. Flesch-Kincaid Grade Level: This formula estimates readability, providing valuable insights 
for educational purposes. 

Each of these components is evaluated for a specific text by comparing it to a massive collection 
of other texts in a corpus (Graesser, McNamara, & Kulikowich, 2011 as cited in McNamara et al., 
2014). Also, in addition to providing a concise overview, T.E.R.A. also determines the grade level of 
the text employing the Flesch-Kincaid Grade Level readability formula (Kincaid, Fishburne, Rogers, 



96 English Language Teaching Educational Journal   ISSN 2621-6485 

 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

& Chissom, 1975 cited in McNamara et al., 2014). Therefore, by adopting the particular program, the 
researchers could obtain the data concerning the easability and readability of AI chatbots’ generated 
texts following each essay model type. 

3) Plagiarism Checkers 

To examine the plagiarism percentage of the obtained data, the researchers applied the three 
plagiarism checkers: one is paid (TurnItIn) and the other two are free (Check-Plagiarism and 
SmallSEOTools’ plagiarism detector). For TurnItIn, it is a paid plagiarism checker that compares 
submissions against over 47 billion web pages and premium academic content in multiple languages, 
providing detailed categorized matches for high-stakes writing (Turnitin, n.d.). The two free tools, 
Check-Plagiarism (check-plagiarism.com) and SmallSEOTools’ plagiarism detector 
(smallseotools.com), offer reliable detection services for students and writers. Check-Plagiarism 
conducts a deep search to identify potential plagiarism, while SmallSEOTools scans text against 
billions of webpages for thorough analysis (Small SEO Tools, n.d.; Check Plagiarism, 2024). 

2.2. Research Procedures 

The researchers conducted a pilot study to determine whether the basic prompts yielded relevant 
data. The prompts were then refined and used with both OpenAI ChatGPT and Google Gemini, 
ensuring consistency by using identical prompts for each chatbot. The 30 essays generated were 
evaluated using the T.E.R.A. writing evaluation program, and plagiarism detection was performed 
using Turnitin, Check-Plagiarism, and Small SEO Tools. Finally, qualitative document analysis was 
applied to address the two research questions.  

2.3. Data Analysis  

To analyze the obtained data, the researchers adopted document analysis. Document analysis 
entails a systematic examination and interpretation of data to extract meaning, enhance 
comprehension, and acquire empirical knowledge (Corbin & Strauss, 2008 as cited in Bowen, 2009; 
Rapley, 2007 as cited in Bowen, 2009). This study adopted document analysis to examine data, 
specifically the assessed results from the T.E.R.A. program and plagiarism checker outputs. Document 
analysis, as described by Corbin and Strauss (2008 as cited in Bowen, 2009), involves a process of 
skimming, thorough reading, and interpretation. This method incorporates elements of content 
analysis, organizing data into categories that align with the research questions. (Corbin & Strauss, 
2008 as cited in Bowen, 2009; Strauss & Corbin, 1998 as cited in Bowen, 2009). The documents 
include background papers, books, diaries, event programs, program proposals, scripts, reports, survey 
data, and public records (Bowen, 2009), for example. Therefore, the researchers were able to interpret 
and assess the quality of the texts generated by the AI chatbots, OpenAI ChatGPT and Google Gemini. 
Additionally, the researchers were able to determine the percentage of plagiarized content produced 
by the AI chatbots. 

3. Findings and Discussion 

The results of the present research are reported following the two research questions. 

3.1. RQ 1: What is the writing quality level of OpenAI ChatGPT and Google Gemini based on 

five essay models in terms of the easability and readability? 

To report the results concerning the writing quality of the two AI Chatbots based on five essay 
models in terms of the easability and readability, the researchers divided them into three parts which 
are as follows: 

1) Overall Analysis of Essays Generated by Two AI Chatbots 

From Table 2, there were 30 writing essays – fifteen generated by ChatGPT and the others 
generated by Gemini. All essays were generated in March 2024. The overall word count in average is 
369 words. Comparing ones from ChatGPT to Gemini’s, the researchers found when given the prompt 
to write a 300-word essay about each writing type, the average word count of writing generated by 
ChatGPT was closer to the specified number. This means that the average word count of the 15 essays 
by ChatGPT is 335 words, with a minimum of 309 words and a maximum of 370 words. For the 
average word count of essays generated by Gemini, it was 403 words, with a minimum of 339 words 
and a maximum of 481 words. These findings indicate that while both AI models were capable of 



ISSN 2621-6485 English Language Teaching Educational Journal 97 
 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

generating essays in response to the same prompts, ChatGPT was more precise in meeting the 
specified word count, whereas Gemini generated more verbose content. 

2) Easability and Readability of Essays Generated by Two AI Chatbots 

The data obtained from the T.E.R.A. program (McNamara et al., 2014), used to assess the 
easability and readability of 30 writing essays generated by the two AI chatbots, showed the following 
results. Among the five components (narrativity, syntactic simplicity, word concreteness, referential 
cohesion, and deep cohesion), the essays generated by both ChatGPT and Gemini received the highest 
percentage for word concreteness and the lowest percentage for narrativity. On average, the 15 essays 
generated by ChatGPT had a maximum of 75% for word concreteness and a minimum of 3% for 
narrativity. In contrast, the 15 essays generated by Gemini had a maximum of 80% for word 
concreteness and a minimum of 4% for narrativity. Higher scores indicate that the essays were easier 
to read. The findings provide valuable insights into the linguistic and structural characteristics of AI-
generated content. Both chatbots scored highest in word concreteness and lowest in narrativity, 
indicating that while the essays were generally easy to understand, they lacked storytelling or narrative 
elements. This suggests that both AI models prioritize clarity and straightforward language over 
engaging, narrative-driven writing. 

Comparing these two AI chatbots, Gemini reported higher scores in narrativity (4%), syntactic 
simplicity (69%), and word concreteness (80%). This indicates that Gemini’s writing was slightly 
more narrative-driven, easier to understand syntactically, and used more concrete language. These 
qualities could make Gemini’s content more suitable for users seeking direct and easily digestible 
information. In contrast, ChatGPT achieved higher scores in referential cohesion (14%) and deep 
cohesion (51%), suggesting that its essays were better at linking ideas and maintaining logical 
consistency throughout the text. This means that ChatGPT’s content might be more effective for 
complex topics that require a clear flow of ideas and a well-structured argument.  For narrativity and 
deep cohesion, the two AI chatbots had very slight differences in scores—only a 1% difference. The 
minimal difference in scores suggests that while there were distinctions in their linguistic capabilities, 
the gap was not significant. Therefore, both tools could be considered relatively close in terms of 
narrative and cohesive quality. 

Table 3.  Means of Text Ease and Readability Scores Coh-Metrixx Component Scores 

AI 

Chatbot 

Word 

Count 
Narrativity 

Syntactic 

Simplicity 

Word 

Concretenes

s 

Referentia

l Cohesion 

Deep  

Cohesio

n 

Flesch-

Kincaid 

Grade 

Level 

ChatGPT 335 3% 55% 75% 14% 51% 14 

Gemini 403 4% 69% 80% 5% 50% 13 

 

Besides analyzing the easability and readability of the texts, the T.E.R.A. program also reported 
the Flesch-Kincaid Grade Level to indicate the reading levels (basic, average, and advanced) of the 
analyzed texts. These levels were aligned with school levels (kindergarten, elementary, middle school, 
high school, college, and post-grad) and the corresponding age ranges of the readers (5-8, 8-11, 11-
14, 14-17, 17-20, and 20). The essays generated by ChatGPT had Flesch-Kincaid Grade Level scores 
ranging from 12 (minimum) to 17 (maximum), with an average score of 14. This means that these 
essays were at an advanced reading level, corresponding to college to post-grad school levels, and 
targeted readers aged 17-20. On the other hand, the essays generated by Gemini showed a slightly 
wider range of scores, with grade level scores ranging from 10 (minimum) to 16 (maximum) and an 
average of 13. This means that these essays were at average to advanced reading levels, corresponding 
to high school to post-grad school levels, and targeted readers aged 14-20. The analysis shows that 
both ChatGPT and Gemini generated essays aimed at readers with advanced reading abilities, but with 
some differences. ChatGPT’s content was more complex, suited for readers with high literacy skills, 
making it ideal for advanced learners. In contrast, Gemini produced content accessible to a broader 
audience, ranging from high school to college students, making it suitable for a wider range of 
educational settings. These differences highlight ChatGPT’s focus on higher-level language and 
Gemini’s flexibility in accommodating various reading levels. 

 



98 English Language Teaching Educational Journal   ISSN 2621-6485 

 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

3) Easability and Readability of Essays Generated by Two AI Chatbots Based on Five Essay Models  

The analysis of the easability, readability, and reading level of texts generated by ChatGPT and 
Gemini for each essay model is shown in Tables 4 and 5.  

The results of the texts generated by ChatGPT are shown in Table 4 as follows. Firstly, in 
narrativity, the ChatGPT argumentative essays achieved the highest percentage (5%) while the lowest 
percentages were found in two essay models: classification (2%) and cause and effect (2%). Secondly, 
in syntactic simplicity, the highest percentage was achieved by ChatGPT’s Comparison and Contrast 
essays (74%) whereas the descriptive essays had the lowest percentage at 26%. Thirdly, in word 
concreteness, the descriptive essays obtained the highest percentage at 99% while the cause and effect 
ones had the lowest at 59%. Fourthly, in referential cohesion, the highest percentage was achieved by 
the classification model (31%) while the comparison and contrast essays had the lowest at only 4%. 
For the last component, deep cohesion, the highest percentage was found in the argumentative writing 
(86%) while the lowest percentage was found in the comparison and contrast essays (19%). The 
analysis of Flesch-Kincaid Grade Level shows that “ChatGPT Classification” and “ChatGPT 
Argumentative” essays achieved the highest score of 16 while “ChatGPT Descriptive” had the lowest 
score of 12. 

Table 4.  Mean Values of Each Essay Model Generated by ChatGPT 

Comparison of 

Five Essay Models 

Narrativit

y 

Syntactic 

Simplicity 

Word 

Concretenes

s 

Referentia

l Cohesion 

Deep  

Cohesio

n 

Flesch-

Kincaid 

Grade Level 

Descriptive 4% 26% 99% 15% 41% 12 

Classification  2% 50% 91% 31% 58% 16 

Cause and Effect  2% 59% 59% 11% 54% 15 

Comparison and 

Contrast  
3% 74% 64% 4% 19% 13 

Argumentative  5% 67% 61% 8% 86% 16 

 

The results of the texts generated by Gemini are shown in Table 5 as follows. For the first 
component, narrativity, “Gemini Argumentative” essays had the highest percentage at 10% while the 
lowest percentage (3%) was found in three essay models: “Gemini Descriptive”, “Gemini Cause and 
Effect”, and “Gemini Comparison and Contrast”. Secondly, in syntactic simplicity, both “Gemini 
Classification” and “Gemini Cause and Effect” had the highest percentage at 76% while “Gemini 
Descriptive” and “Gemini Argumentative” had the lowest percentage at 61%. Thirdly, in word 
concreteness, the highest percentage was found in “Gemini Descriptive” (97%) whereas “Gemini 
Argumentative” had the lowest at 64%. Fourthly, in referential cohesion, the essay model with the 
highest percentage was “Gemini Argumentative” (16%) while “Gemini Comparison and Contrast” 
had the lowest at only 1%. For the last component, deep cohesion, the highest percentage was shown 
by “Gemini Argumentative” (87%) while “Gemini Descriptive” had the lowest at 25%. The analysis 
of Flesch-Kincaid Grade Level shows that “Gemini Argumentative” achieved the highest score of 15 
while the lowest score of 11 was found in “Gemini Descriptive”. 

Table 5.  Mean Values of Each Essay Model Generated by Gemini 

Comparison of 

Five Essay Models 

Narrativit

y 

Syntactic 

Simplicity 

Word 

Concretenes

s 

Referentia

l Cohesion 

Deep  

Cohesio

n 

Flesch-

Kincaid 

Grade 

Level 

Descriptive 3% 61% 97% 2% 25% 11 

Classification  4% 76% 93% 3% 36% 12 

Cause and Effect  3% 76% 66% 2% 54% 14 

Comparison and 

Contrast  
3% 72% 80% 1% 51% 12 

Argumentative  10% 61% 64% 16% 87% 15 

 



ISSN 2621-6485 English Language Teaching Educational Journal 99 
 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

These findings suggest that AI-generated content from ChatGPT and Gemini can significantly 
contribute to enhancing educational practices. However, understanding the specific capabilities and 
limitations of each AI tool is crucial for maximizing their effectiveness in learning and teaching 
contexts. ChatGPT is more precise in meeting specified word counts, making it ideal for tasks 
requiring strict adherence to length, such as academic assignments. In contrast, Gemini produces more 
detailed and verbose content, which is useful for creative writing or content creation where elaboration 
is valued. The findings also revealed that both ChatGPT and Gemini similarly and well performed 
regarding word concreteness. This means both AI chatbots are effective for generating easy-to-
understand educational content, useful for creating study materials. However, their low scores in 
narrativity suggest they are less ideal for storytelling tasks. Furthermore, the findings of this current 
study suggest that ChatGPT and Gemini have their strengths in a particular way. ChatGPT has a 
positive tendency to outperform Gemini in terms of referential cohesion and deep cohesion domains. 
Meanwhile, Gemini tends to perform better than ChatGPT regarding narrativity, syntactic simplicity, 
and word concreteness. This means ChatGPT is better for assignments involving complex ideas and 
structured arguments, while Gemini is more suitable for tasks requiring clarity and simplicity. In 
addition, the two AI tools differ in the reading levels of the texts they produce. ChatGPT tends to 
create content suitable for college to post-graduate readers, making it ideal for higher education. It’s 
a useful tool for generating challenging materials for advanced learners or as study aids for university 
students. In comparison, Gemini can generate texts ranging from high school to post-graduate levels, 
making it a more adaptable tool that fits a wider variety of educational settings. 

Although the two chatbots differently outperform from one another, both of these OpenAI tools 
are recognized for their usefulness in assisting language learning. The outcomes of this study accord 
with the previous studies relevant to the effective use of chatbot in English writing learning (Fitria, 
2023; Barrot, 2023; Zhang et al., 2023; Seyyedrezaei et al., 2022; Yan, 2023). Lee and Lee (2022) 
conducted a meta-analysis exploring the effect size of using chatbot assisted language learning. The 
outcomes of the study indicated a positive effect on language learning across all language skills. The 
differences in linguistic features and performance between ChatGPT and Gemini suggest that 
educators and students can benefit from selecting AI tools that match their specific needs. By 
understanding the strengths and limitations of each tool, they can make better informed decisions for 
their writing objectives. 

3.2. RQ 2: What is the percentage of plagiarized content in the generated texts from OpenAI 

ChatGPT and Google Gemini as detected by plagiarism checkers? 

To report the results concerning the plagiarized-content percentage of the generated writing texts 
from the two AI Chatbots by plagiarism checkers, the researchers divided them into three parts which 
are as follows: 

1) Free Plagiarism Checkers: SmallSEOTools and Check-Plagiarism 

Using the two free plagiarism checkers, SmallSEOTools and Check-Plagiarism, identified 12 out 
of the 30 essays as containing plagiarized content as shown in Table 6. The former checker found that 
10 essays (five generated by ChatGPT and the others by Gemini) containing plagiarism while the 
latter found 11 essays (five generated by ChatGPT and the other six by Gemini) containing it. In terms 
of the writing model, the average percentage showed that cause and effect essays from the two AI 
Chatbots contained the highest percentage of plagiarized content. 

The analysis revealed that both free plagiarism checkers similarly identified two essays — 
“ChatGPT Cause and Effect 2” and “Gemini Cause and Effect 3”— as containing the highest 
percentages of plagiarized content. According to the SmallSEOTools checker, “ChatGPT Cause and 
Effect 2” had 17% plagiarism while “Gemini Cause and Effect 3” had 12%. For the Check-Plagiarism 
checker, the data showed “ChatGPT Cause and Effect 2” contained 18% plagiarism, followed by 
“Gemini Cause and Effect 3” with 13%. 

The analyzed data revealed that among the five writing models, cause-and-effect essays contained 
the highest percentage of plagiarism while the classification essays generated by ChatGPT and the 
argumentative essays by Gemini had the lowest percentage (0%). The cause-and-effect essays 
produced by both AI chatbots were most frequently found to contain plagiarized content. Specifically, 
two out of the three essays generated by ChatGPT were reported to have plagiarism, which was similar 
to Gemini. Two out of three essays generated by Gemini also contained plagiarized content. In 
contrast, the analysis identified two writing models with no detected plagiarism (0%). The first was 



100 English Language Teaching Educational Journal   ISSN 2621-6485 

 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

the classification essays generated by ChatGPT. When checked using SmallSEOTools and Check-
Plagiarism, all three classification essays were found to have 0% plagiarism. The second was the 
argumentative essays generated by Gemini. All argumentative essays, when tested with both free 
plagiarism checkers, also showed no plagiarized content (0%). 

Table 6.  The Percentage of Plagiarized Content Found in AI Chatbots’ Writing by Free Plagiarism Checkers 

Essays Written by AI Chatbots 

Free Plagiarism Checkers 
Percentage of  

Plagiarized Content of 

each Writing Model 
smallseotools 

.com 

check-

plagiarism.com 

ChatGPT Descriptive 1 5% 6% 

1.83% ChatGPT Descriptive 2 0% 0% 

ChatGPT Descriptive 3 0% 0% 

ChatGPT Classification 1 0% 0% 

0.00% ChatGPT Classification 2 0% 0% 

ChatGPT Classification 3 0% 0% 

ChatGPT Cause and Effect 1 0% 0% 

7.67% ChatGPT Cause and Effect 2 17% 18% 

ChatGPT Cause and Effect 3 5% 6% 

ChatGPT Comparison and Contrast 1 5% 6% 

1.83% ChatGPT Comparison and Contrast 2 0% 0% 

ChatGPT Comparison and Contrast 3 0% 0% 

ChatGPT Argumentative 1 0% 0% 

2.83% ChatGPT Argumentative 2 0% 0% 

ChatGPT Argumentative 3 11% 6% 

Average Percentage 0.03% 0.03% 3% 

Gemini Descriptive 1 4% 0% 

1.50% Gemini Descriptive 2 0% 5% 

Gemini Descriptive 3 0% 0% 

Gemini Classification 1 0% 0% 

1.33% Gemini Classification 2 4% 4% 

Gemini Classification 3 0% 0% 

Gemini Cause and Effect 1 0% 0% 

6.33% Gemini Cause and Effect 2 4% 9% 

Gemini Cause and Effect 3 12% 13% 

Gemini Comparison and Contrast 1 0% 0% 
 

2.50% 
Gemini Comparison and Contrast 2 0% 4% 

Gemini Comparison and Contrast 3 5% 6% 

Gemini Argumentative 1 0% 0% 

0% Gemini Argumentative 2 0% 0% 

Gemini Argumentative 3 0% 0% 

Average Percentage 0.02% 0.03% 2% 

 

2) Paid Plagiarism Checker: TurnItIn 

The analysis of data from the paid plagiarism checker on essays generated by AI chatbots revealed 
that 21 out of 30 essays contained plagiarized content. In terms of the percentage of plagiarism, three 
essays generated by ChatGPT were found to have over 50% plagiarism: "ChatGPT Cause and Effect 
2" (55%), "ChatGPT Cause and Effect 3" (54%), and "ChatGPT Cause and Effect 1" (50%). For 



ISSN 2621-6485 English Language Teaching Educational Journal 101 
 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

Gemini, the three essays with the highest plagiarism percentages were "Gemini Cause and Effect 2" 
(37%), "Gemini Cause and Effect 3" (26%), and "Gemini Argumentative 2" (10%) as shown in Table 
7. 

Table 7.  The Percentage of Plagiarized Content Found in AI Chatbots’ Writing by a Paid Plagiarism Checker 

Essays Written by AI Chatbots TurnItIn 

Percentage of  

Plagiarized Content of 

each Writing Model 

ChatGPT Descriptive 1 8%   

ChatGPT Descriptive 2 7% 5.67% 

ChatGPT Descriptive 3 2%  

ChatGPT Classification 1 0% 

1.33% ChatGPT Classification 2 0% 

ChatGPT Classification 3 4% 

ChatGPT Cause and Effect 1 50% 

53.00% ChatGPT Cause and Effect 2 55% 

ChatGPT Cause and Effect 3 54% 

ChatGPT Comparison and Contrast 1 13%  

ChatGPT Comparison and Contrast 2 9% 9.33% 

ChatGPT Comparison and Contrast 3 6%  

ChatGPT Argumentative 1 17% 

30.00% ChatGPT Argumentative 2 33% 

ChatGPT Argumentative 3 40% 

Average Percentage 19.87%  

Gemini Descriptive 1 0% 

0.67% Gemini Descriptive 2 2% 

Gemini Descriptive 3 0% 

Gemini Classification 1 0% 

0.00% Gemini Classification 2 0% 

Gemini Classification 3 0% 

Gemini Cause and Effect 1 8% 

23.67% Gemini Cause and Effect 2 37% 

Gemini Cause and Effect 3 26% 

Gemini Comparison and Contrast 1 0% 

2.33% Gemini Comparison and Contrast 2 3% 

Gemini Comparison and Contrast 3 4% 

Gemini Argumentative 1 0% 

6.00% Gemini Argumentative 2 10% 

Gemini Argumentative 3 8% 

Average Percentage 6.53%  

 

  



102 English Language Teaching Educational Journal   ISSN 2621-6485 

 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

For ChatGPT, 13 out of 15 essays were reported to contain plagiarism, with the exceptions of 
"ChatGPT Classification 1" and "ChatGPT Classification 2." In contrast, eight out of 15 essays 
generated by Gemini were found to have plagiarism. Additionally, across the writing models, the data 
showed that essays generated by ChatGPT contained a higher average percentage of plagiarism than 
those generated by Gemini. 

Based on the five writing models, the highest average percentage of plagiarized content was found 
in the cause-and-effect essays from both ChatGPT (x̄ = 53%) and Gemini (x̄ = 23.67%). The highest 
percentage was 55% ("ChatGPT Cause and Effect 2") while the lowest was 8% ("Gemini Cause and 
Effect 1"). In contrast, the classification writing model had the lowest average percentage of 
plagiarized content (x̄ = 1.33% for ChatGPT and x̄ = 0% for Gemini) compared to the other models. 
Five out of the six classification essays had no plagiarism, except for "ChatGPT Classifying 3," which 
contained 4%.  

For the argumentative writing model, five essays contained plagiarism. The highest percentage 
was 40% ("ChatGPT Argumentative 3"), followed by 33% ("ChatGPT Argumentative 2"), with the 
lowest being 0% ("Gemini Argumentative 1"). In the comparison-and-contrast writing model, five out 
of six essays were also found to contain plagiarism. The highest percentage was in "ChatGPT 
Comparison and Contrast 1" (13%), while "Gemini Comparison and Contrast 1" showed no 
plagiarism. 

3) Both Free and Paid Plagiarism Checkers: SmallSEOTools, Check-Plagiarism, and TurnItIn 

When both free and paid plagiarism checkers were used to detect the content plagiarism in all 30 
essays generated by the two AI Chatbots, the analysis identified a significant portion (23 out of 30) 
containing plagiarized content. In average, the essay with the highest percentage of plagiarism was 
“ChatGPT Cause and Effect 2” (30%), followed by “ChatGPT Cause and Effect 3” (21.67%) and 
“ChatGPT Argumentative 3” (19%). This means the essays contained the highest percentage were 
generated by ChatGPT. In addition, the analysis showed that the average percentage of the fifteen 
essays generated by ChatGPT was higher than ones by Gemini as shown in Table 8.  

Among the 23 essays containing plagiarism, the analysis found that some plagiarized content was 
detected only by the free plagiarism checkers while some was detected only by the paid software. For 
plagiarized content detected only by the free software, "Gemini Descriptive 1" showed 4% plagiarism 
using SmallSEOTools, and "Gemini Classifying 2" showed 4% plagiarism using both free tools. 
Regarding plagiarism detected only by the paid software, there were 11 essays, as shown in Table 8. 
Among these, the highest percentages of plagiarism were found in "ChatGPT Cause and Effect 1" 
(50%), followed by "ChatGPT Argumentative 2" (33%) and "ChatGPT Argumentative 1" (17%). 

Furthermore, Table 8 showed that the average percentage of plagiarism across the 15 essays 
generated by ChatGPT was higher than those generated by Gemini. Specifically, the average 
plagiarism percentage for ChatGPT essays was 6.64% while for Gemini, it was 2.19%.  

In terms of writing models, ChatGPT's essays had a higher mean percentage of plagiarism in four 
out of five models as shown in Table 9. For descriptive writing, ChatGPT’s essays had an average of 
3.11% plagiarism, compared to 1.22% for Gemini. In cause-and-effect writing, ChatGPT's essays 
averaged 22.78%, while Gemini's averaged 12.11%. ChatGPT’s comparison and contrast essays had 
an average of 4.33%, compared to 2.44% for Gemini. In argumentative writing, ChatGPT's essays 
had an average of 11.89%, whereas Gemini's averaged 2%. The only writing model where Gemini 
had a higher average than ChatGPT was classification, with 0.89% for Gemini and 0.44% for 
ChatGPT. 

  



ISSN 2621-6485 English Language Teaching Educational Journal 103 
 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

Table 8.  The Percentage of Plagiarized Content Found in AI Chatbots’ Writingby Both Free and Paid 

Plagiarism Checkers 

Essays Written by AI Chatbots 

Plagiarism Checkers 

x̄ smallseo 

tools.com 

check-plagiarism 

.com 
TurnItIn 

ChatGPT Descriptive 1 5% 6% 8% 6.33% 

ChatGPT Descriptive 2 0% 0% 7% 2.33% 

ChatGPT Descriptive 3 0% 0% 2% 0.67% 

ChatGPT Classification 1 0% 0% 0% 0.00% 

ChatGPT Classification 2 0% 0% 0% 0.00% 

ChatGPT Classification 3 0% 0% 4% 1.33% 

ChatGPT Cause and Effect 1 0% 0% 50% 16.67% 

ChatGPT Cause and Effect 2 17% 18% 55% 30.00% 

ChatGPT Cause and Effect 3 5% 6% 54% 21.67% 

ChatGPT Comparison and Contrast 1 5% 6% 13% 8.00% 

ChatGPT Comparison and Contrast 2 0% 0% 9% 3.00% 

ChatGPT Comparison and Contrast 3 0% 0% 6% 2.00% 

ChatGPT Argumentative 1 0% 0% 17% 5.67% 

ChatGPT Argumentative 2 0% 0% 33% 11.00% 

ChatGPT Argumentative 3 11% 6% 40% 19.00% 

Average Percentage 0.03% 0.03% 19.87% 6.64% 

Gemini Descriptive 1 4% 0% 0% 1.33% 

Gemini Descriptive 2 0% 5% 2% 2.33% 

Gemini Descriptive 3 0% 0% 0% 0.00% 

Gemini Classification 1 0% 0% 0% 0.00% 

Gemini Classification 2 4% 4% 0% 2.67% 

Gemini Classification 3 0% 0% 0% 0.00% 

Gemini Cause and Effect 1 0% 0% 8% 2.67% 

Gemini Cause and Effect 2 4% 9% 37% 16.67% 

Gemini Cause and Effect 3 12% 13% 26% 17.00% 

Gemini Comparison and Contrast 1 0% 0% 0% 0.00% 

Gemini Comparison and Contrast 2 0% 4% 3% 2.33% 

Gemini Comparison and Contrast 3 5% 6% 4% 5.00% 

Gemini Argumentative 1 0% 0% 0% 0.00% 

Gemini Argumentative 2 0% 0% 10% 3.33% 

Gemini Argumentative 3 0% 0% 8% 2.67% 

Average Percentage 0.02% 0.03% 6.53% 2.19% 

 

  



104 English Language Teaching Educational Journal   ISSN 2621-6485 

 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

Table 9.  The Average Percentage of Plagiarized Content Found by both Free and Paid Checkers in Each 

Writing Model 

Five Essay Models 
AI Chatbots 

x̄ 
ChatGPT Gemini 

Descriptive 3.11% 1.22% 2.17% 

Classification 0.44% 0.89% 0.67% 

Cause and Effect 22.78% 12.11% 17.44% 

Comparison and Contrast 4.33% 2.44% 3.39% 

Argumentative 11.89% 2.00% 6.94% 

 

With a significant portion (23 out of 30) containing plagiarized, the analysis sheds light on the 
potential of AI-generated content while also highlights critical concerns regarding plagiarism. The 
study’s findings indicated that some written texts generated by ChatGPT were more prone to 
plagiarism detection than those produced by Gemini. Of the five writing models, the causes and effect 
essays were mostly plagiarized for both AI tools specially composed by the ChatGPT. The outcomes 
of this current study differ from those of Khalil and Er (2023) in that essay writing, particularly 
scientific essays produced by ChatGPT, tended to be recognized for their high originality. In contrast, 
this current study found that Turnitin detected the cause-and-effect essays as plagiarism at the highest 
level, similar to the free software plagiarism detector, which also showed that cause and effect writing 
produced by ChatGPT and Gemini exhibited the highest degree of plagiarism. At this point, cause and 
effect essays commonly address mostly debated topics, for instance, climate change or the effects of 
technology. These issues have been widely discussed, resulting in the unintended employment of 
identical ideas and language features from previous work. This is in line with Aydın and Karaarslan 
(2022), arguing that ChatGPT seemed to underperform in generating original work after paraphrasing 
compared to other chatbots.  

In comparing free versus paid checkers, this study highlights the differences between the free 
plagiarism tools and the paid program. As mentioned, Turnitin performed better in detecting 
plagiarism. These findings align with those reported by Suaverdez and Suaverdez (2023), though there 
are some differences. In their study, the paid tool identified a higher amount of plagiarized content 
compared to the free checkers. Similarly, the present study found that the paid tool outperformed the 
two free programs, particularly in detecting plagiarism in the cause-and-effect essays. However, the 
findings also indicated that the free checkers performed better in certain writing models generated by 
Google Gemini, including descriptive, classification, and comparison and contrast essays. While 
SmallSEOTools and Check-Plagiarism identified a lower percentage of plagiarism compared to 
Turnitin, they still proved reliable to some extent in detecting plagiarized content.  

The findings shed light on the importance of caution when relying on AI tools and encourage users 
to be cautious when using AI tools for academic or professional writing. Furthermore, this analysis 
provides valuable insights into the reliability of AI tools in generating original content and the 
effectiveness of different plagiarism detection systems. It highlights the need for caution and vigilance 
when using AI-generated texts, particularly in educational contexts where originality and academic 
integrity are critical. 

4. Conclusion and Implications 

The present study has explored and compared the writing quality of OpenAI ChatGPT and Google 
Gemini based on five essay models in terms of the easability and readability using the T.E.R.A 
program. Moreover, it has revealed the plagiarized content percentage of the texts generated by the 
two AI chatbots using three plagiarism checkers. The results revealed that both ChatGPT and Gemini 
similarly and well performed regarding word concreteness; however, they showed limitations in 
narrativity. ChatGPT was likely to excel over Gemini in areas like referential cohesion and deep 
cohesion. On the other hand, Gemini outperformed ChatGPT in aspects such as narrativity, syntactic 
simplicity, and word concreteness. Some essays written by ChatGPT and Gemini were detected for 



ISSN 2621-6485 English Language Teaching Educational Journal 105 
 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

their plagiarism. Some written texts generated by ChatGPT were more likely to be detected for 
plagiarism compared to those created by Gemini.  

AI has been increasingly used across disciplines. In an educational setting, chatbot has a great 
impact on how educators and teachers design courses, create teaching and learning materials, and 
assess students’ learning. It is worth noting that chatbot should be incorporated into teaching and 
learning with good care. As seen in this study, one AI tool performs differently from one another. As 
a matter of fact that AI tool has its unique ability, it is good for a teacher to select an appropriate tool 
for a particular teaching and learning objective. Take a material design, for example, where a teacher 
should take a special care for the levels of the students, choosing the most appropriate AI tool is a key 
to success. Some AI tools, like Gemini might involve a variety of age ranges more than others. In this 
case, in a high school context, Gemini could produce some model texts that are appropriate for high 
school students. In addition, one major concern that should be mentioned regarding the degree of 
plagiarism found in texts generated by AI tools. Based on the outcomes of this study, a percentage of 
plagiarism could be found in both ChatGPT and Gemini. This means that teachers and students should 
not heavily rely on AI tools. To raise students’ awareness of plagiarism, they should be encouraged 
to use AI as language learning assistants to avoid producing writing tasks containing plagiarism which 
is a violation of ethics. Also, they should practice necessary English skills and be trained how to use 
AI properly and efficiently in language learning. Additionally, plagiarism-detection software should 
be incorporated into classroom. 

 

Acknowledgment  

The researchers would like to express sincere gratitude to the Division of Research Administration 
and Academic Services, Kasetsart University Chalermphrakiat Sakon Nakhon Province Campus, 
Thailand, for providing the necessary resources and financial support. The researchers also extend 
appreciation to the Head of the Languages Department, Asst. Prof. Dr. Dentisak Dokchandra, as well 
as departmental colleagues and all administrative officers in the Faculty of Liberal Arts and 
Management Science, for their invaluable guidance, moral support, and encouragement. Their 
assistance and suggestions greatly contributed to the successful completion of this research. 

 

Declarations  

Author contribution : All authors, Siraprapa Kotmungkun, Wichuta Chompurach, and 
Piriya Thaksanan contributed equally to this research, participating in 
all stages, including conceptualization, data collection, analysis, and 
writing. Wichuta Chompurach, the corresponding author, led the 
manuscript writing and coordinated the collaboration with Siraprapa 
Kotmungkun and Piriya Thaksanan. All authors reviewed and revised 
the manuscript and approved the final version. 

Funding statement : The research is funded under the Division of Research Administration 
and Academic Services, Kasetsart University Chalermphrakiat Sakon 
Nakhon Province Campus, Thailand. 

Conflict of interest : The authors declare no conflict of interest. 

Ethics declaration : This research was conducted using AI-generated written data from 
OpenAI ChatGPT and Google Gemini. No human participants were 
involved in the study, and as such, the research does not raise ethical 
concerns regarding human subjects. The AI-generated texts were used 
solely for the purpose of assessing writing quality and plagiarism 
detection. All procedures followed were in compliance with ethical 
guidelines for non-human data research. 

Additional information : No additional information is available for this paper. 

 

  



106 English Language Teaching Educational Journal   ISSN 2621-6485 

 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

 

REFERENCES  

Ariyanti, A. (2016). Shaping students’ writing skills: The study of fundamental aspects in mastering 

academic writing. Indonesian Journal of EFL and Linguistics, 1(1), 63-77. Retrieved from 

http://www.indonesian-efl-journal.org  

Aydın, Ö., & Karaarslan, E. (2022). OpenAI ChatGPT generated literature review: Digital twin in 

healthcare. Emerging Computer Technologies, 2, 22-31. İzmir Akademi Dernegi. 

http://dx.doi.org/10.2139/ssrn.4308687 

Baidoo-Anu, D., & Owusu Ansah, L. (2023). Education in the era of generative Artificial Intelligence 

(AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. 

Journal of AI, 7(1), 52-62. https://doi.org/10.61969/jai.1337500  

Barrot, J. S. (2023). Using automated written corrective feedback in the writing classrooms: Effects 

on L2 writing accuracy. Computer Assisted Language Learning, 36(6), 584–607. 

https://doi.org/10.1080/09588221.2021.1936071  

Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research 

Journal, 9(2), 27-40. https://doi.org/10.3316/QRJ0902027  

Chan, C. K. Y., & Hu, W. (2023). Students’ voices on generative AI: perceptions, benefits, and 

challenges in higher education. International Journal of Educational Technology in Higher 

Education, 20, 43. https://doi.org/10.1186/s41239-023-00411-8  

Check Plagiarism. (2024). Check plagiarism: Free online plagiarism checker. https://www.check-

plagiarism.com/ 

Chompurach, W. (2021). “Please let me use Google Translate”: Thai EFL students’ behavior and 

attitudes toward Google Translate use in English writing. English Language Teaching, 

14(12), 23-35. https://doi.org/10.5539/elt.v14n12p23  

Davis, D. F., Golicic, S. L., & Boerstler, C. N. (2011). Benefits and challenges of conducting multiple 

methods research in marketing. Journal of the Academy of Marketing Science, 39, 467–479. 

https://doi.org/10.1007/s11747-010-0204-7  

Espejel et al. (2023). GPT-3.5, GPT-4, or BARD? Evaluating LLMs reasoning ability in zero-shot 

setting and performance boosting through prompts. Natural Language Processing Journal, 

5, 100032. https://doi.org/10.1016/j.nlp.2023.100032 

Evans et al. (2023). ChatGPT impacts on access-efficiency, employment, education and ethics: The 

socio-economics of an AI language model. BizEcons Quarterly, 16, 1-17.   

Fitria, T. N. (2023). Artificial intelligence (AI) technology in OpenAI ChatGPT application: A 

review of ChatGPT in writing English essay. ELT Forum: Journal of English Language 

Teaching, 12(1), 44-58. https://doi.org/10.15294/elt.v12i1.64069  

Hasnawati, H., Mujahidin, E., & Tanjung, H. (2023). Analyzing students’ difficulties in writing 

English essay. International Journal of Social Science and Human Research, 6(10), 5954-

5959. https://doi.org/10.47191/ijsshr/v6-i10-22  

Hyland, K. (2003). Second language writing. Cambridge University Press. 

https://doi.org/10.1017/CBO9780511667251  

Hyland, K., & Jiang, F. K. (2017). Is academic writing becoming more informal? English for Specific 

Purposes, 45, 40-51. https://doi.org/10.1016/j.esp.2016.09.001  

http://www.indonesian-efl-journal.org/
https://dx.doi.org/10.2139/ssrn.4308687
https://doi.org/10.61969/jai.1337500
https://doi.org/10.1080/09588221.2021.1936071
https://doi.org/10.3316/QRJ0902027
https://doi.org/10.1186/s41239-023-00411-8
https://www.check-plagiarism.com/
https://www.check-plagiarism.com/
https://doi.org/10.5539/elt.v14n12p23
https://doi.org/10.1007/s11747-010-0204-7
https://doi.org/10.1016/j.nlp.2023.100032
https://doi.org/10.15294/elt.v12i1.64069
https://doi.org/10.47191/ijsshr/v6-i10-22
https://doi.org/10.1017/CBO9780511667251
https://doi.org/10.1016/j.esp.2016.09.001


ISSN 2621-6485 English Language Teaching Educational Journal 107 
 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

Jovanovic, M. (2022). Generative artificial intelligence: Trends and prospects. Computer, 55(10), 

107-112. https://doi.org/10.1109/MC.2022.3192720  

Khalil, M., & Er, E. (2023). Will ChatGPT get you caught? Rethinking of plagiarism detection. In 

Proceedings of the 25th International Conference on Human-Computer Interaction (HCII 

2023) (Vol. 14040, pp. 475–487). https://arxiv.org/pdf/2302.04335  

Lee, G. G., Shi, L., Latif, E., Gao, Y., Bewersdorf, A., Nyaaba, M., et al. (2023). Multimodality of 

AI for education: Towards artificial general intelligence. arXiv. 

https://arxiv.org/abs/2312.06037 

Lee, H., & Lee, J. H. (2022). The effects of robot-assisted language learning: A meta-analysis. 

Educational Research Review, 35, 100425. https://doi.org/10.1016/j.edurev.2021.100425  

Lee, J. & Park, J. (2023). AI as “Another I”: Journey map of working with artificial intelligence from 

AI-phobia to AI-preparedness. Organizational Dynamics, 52, 100994. 

https://doi.org/10.1016/j.orgdyn.2023.100994  

Lingard, L. (2023). Writing with ChatGPT: It illustrates its capacity, limitations & implications for 

academic writers. Perspectives on Medical Education, 12(1), 261–270. 

https://doi.org/10.5334/pme.1072  

McCarthy, J., Minsky, M., Rochester, N., & Shannon, C. (1955). A proposal for Dartmouth summer 

research project on artificial intelligence. AI Magazine, 27, 12. 

McNamara, D. S., Graesser, A. C., McCarthy, P., & Cai, Z. (2014). Automated evaluation of text and 

discourse with Coh-Metrix. Cambridge University Press. 

https://doi.org/10.1017/CBO9780511894664  

Meiranti, R. (2012). Improving students’ writing skills through field trip method. English Review: 

Journal of English Education, 1(1), 89-96.   

Miles, M. B., Huberman, A. M., & Saldaña, J. (2018). Qualitative data analysis: A methods 

sourcebook (4th ed.). SAGE Publications. 

Mohammed, A., Al-ghazali, A. & Alqohfa, K. (2023). Exploring ChatGPT uses in higher studies: A 

case study of Arab postgraduates in India. Journal of English Studies in Arabia Felix, 2(2), 

8–15. https://doi.org/10.56540/jesaf.v2i2.55  

OpenAI. (2022). ChatGPT: Optimizing language models for dialogue. OpenAI. 

https://openai.com/blog/chatgpt/ (Accessed November 29, 2023) 

Rane, N., Choudhary, S., & Rane, J. (2024). Gemini or ChatGPT? Capability, performance, and 

selection of cutting-edge generative artificial intelligence (AI) in business management. 

Studies in Economics and Business Relations, 5(1), 40-50. 

https://doi.org/10.48185/sebr.v5i1.1051  

Rayhan, A. (2023). Mastering prompt engineering: Techniques for creating powerful and effective 

AI language models. Independently Published. 

Seyyedrezaei, M. S., Amiryousefi, M., Gimeno-Sanz, A., & Tavakoli, M. (2022). A meta-analysis 

of the relative effectiveness of technology-enhanced language learning on ESL/EFL writing 

performance: Retrospect and prospect. Computer Assisted Language Learning, 1–34. 

https://doi.org/10.1080/09588221.2022.218782 

Small SEO Tools. (n.d.). Small SEO tools: Text analysis tools. Small SEO tools. 

https://smallseotools.com/ 

https://doi.org/10.1109/MC.2022.3192720
https://arxiv.org/pdf/2302.04335
https://arxiv.org/abs/2312.06037
https://doi.org/10.1016/j.edurev.2021.100425
https://doi.org/10.1016/j.orgdyn.2023.100994
https://doi.org/10.5334/pme.1072
https://doi.org/10.1017/CBO9780511894664
https://doi.org/10.56540/jesaf.v2i2.55
https://openai.com/blog/chatgpt/
https://doi.org/10.48185/sebr.v5i1.1051
https://doi.org/10.1080/09588221.2022.218782
https://smallseotools.com/


108 English Language Teaching Educational Journal   ISSN 2621-6485 

 Vol. 7, No. 2, August 2024, pp. 90-108 

 Kotmungkun, S., et al (OpenAI ChatGPT vs Google Gemini: A study of AI.....) 

Stahl, B. C. & Eke, D. (2023). The ethics of ChatGPT – Exploring the ethical issues of an emerging 

technology. International Journal of Information Management, 74, 102700. 

https://doi.org/10.1016/j.ijinfomgt.2023.102700  

Suaverdez, J., & Suaverdez, U. (2023). Chatbots impact on academic writing. Global Journal of 

Business and Integral Security, (2). Retrieved from 

https://www.gbis.ch/index.php/gbis/article/view/150  

Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. 

(2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots 

in education. Smart Learning Environments, 10(1), 15. https://doi.org/10.1186/s40561-023-

00237-x 

Turnitin. (n.d.). Turnitin: Empower students to do their best, original work. Turnitin. 

https://www.turnitin.com/ 

Yan, D. (2023). Impact of ChatGPT on learners in a L2 writing practicum: An exploratory 

investigation. Education and Information Technologies, 28, 13943–13967. 

https://doi.org/10.1007/s10639-023-11742-4 

Zhai, X. (2023). ChatGPT user experience: Implications for education. SSRN Electronic Journal. 

https://dx.doi.org/10.2139/ssrn.4312418  

Zhang, S., Shan, C., Lee, J. S. Y., & et al. (2023). Effect of chatbot-assisted language learning: A 

meta-analysis. Education and Information Technologies, 28, 15223–15243. 

https://doi.org/10.1007/s10639-023-11805-6 

 

https://doi.org/10.1016/j.ijinfomgt.2023.102700
https://www.gbis.ch/index.php/gbis/article/view/150
https://doi.org/10.1186/s40561-023-00237-x
https://doi.org/10.1186/s40561-023-00237-x
https://www.turnitin.com/
https://doi.org/10.1007/s10639-023-11742-4
https://dx.doi.org/10.2139/ssrn.4312418
https://doi.org/10.1007/s10639-023-11805-6

	1. Introduction
	1.1. Large Language Models: OpenAI ChatGPT and Google Gemini
	1.2. AI Large Language Models Use in Education Settings
	1.3. Use of Large Language Model in L2 Writing

	2. Methods
	2.1. Data Collection Tools
	1) Basic Prompts to Gather Data from AI Chatbots
	2) T.E.R.A. (The Text Ease and Readability Assessor) Program
	3) Plagiarism Checkers

	2.2. Research Procedures
	2.3. Data Analysis

	3. Findings and Discussion
	3.1. RQ 1: What is the writing quality level of OpenAI ChatGPT and Google Gemini based on five essay models in terms of the easability and readability?
	1) Overall Analysis of Essays Generated by Two AI Chatbots
	2) Easability and Readability of Essays Generated by Two AI Chatbots
	3) Easability and Readability of Essays Generated by Two AI Chatbots Based on Five Essay Models

	3.2. RQ 2: What is the percentage of plagiarized content in the generated texts from OpenAI ChatGPT and Google Gemini as detected by plagiarism checkers?
	1) Free Plagiarism Checkers: SmallSEOTools and Check-Plagiarism
	2) Paid Plagiarism Checker: TurnItIn
	3) Both Free and Paid Plagiarism Checkers: SmallSEOTools, Check-Plagiarism, and TurnItIn


	4. Conclusion and Implications
	Acknowledgment
	Declarations


