































   Advancements in Agricultural Development 
  Volume 6, Issue 1, 2025 
  agdevresearch.org 

 

1. Amy Harder, Associate Dean and Professor, 1376 Storrs Rd., United 4134, Storrs, CT  06269-4134, amy.harder@uconn.edu, 

 https://orcid.org/0000-0002-7042-2028  
2. Lendel Kade Narine, Extension Associate Professor, Utah State University, 4900 Old Main Hill, Logan, UT 84322, 

lendel.narine@usu.edu,  https://orcid.org/0000-0001-6962-2770 
3. Stacey Stearns, Communications Specialist, University of Connecticut Extension, 1376 Storrs Road, Unit 4134, Storrs, CT 

06269-4134, stacey.stearns@uconn.edu,  https://orcid.org/0000-0002-4711-7244 
 

97 

 

Decision-Making and Data Quality: Applying Fraud Response 
Strategies to Clean Survey Panel Data 

 
A. Harder1, L. K. Narine2, S. Stearns3 

 
 

Article History 
Received: December 18, 2024 
Accepted: February 3, 2025 
Published: March 8, 2025 
 
 
Keywords 
SDG 4: Quality Education; online 
surveys; survey straightlining;  
low-quality indicators; 
extension professionals   

Abstract 
Survey panels provide extension professionals with a valuable tool for 
collecting data on a wide range of topics without overburdening their 
program participants. Paid data panels are particularly useful for 
gathering unbiased feedback about Extension programs. However, some 
survey participants in these panels engage in satisficing or straightlining 
behaviors to earn rewards with minimal effort, which compromises data 
quality. This study explored whether survey panelists’ perceptions of 
online survey items varied based on response quality. It compared normal 
and low-quality responses across broad issue areas and investigated 
whether age, education, income, or gender identity influenced these 
differences. Analysis of 94 respondents in each group revealed no 
significant data quality differences based on age, education, income, or 
gender identity. There was a statistically significant difference in data 
quality when using an open-ended question requiring greater cognitive 
effort. We recommend adopting more conservative data cleaning 
strategies. While this approach has limitations, its benefits are 
particularly valuable when the data informs an organization’s strategic 
priorities. 

 

mailto:amy.harder@uconn.edu
https://orcid.org/0000-0002-7042-2028
mailto:lendel.narine@usu.edu
https://orcid.org/0000-0001-6962-2770
mailto:stacey.stearns@uconn.edu
https://orcid.org/0000-0002-4711-7244


Harder et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i1.573   98 
 

Introduction and Problem Statement 
 
Online survey technologies have made it increasingly easy for researchers to contract with for-
profit survey providers to collect data from convenience samples. In the U.S., extension 
professionals have used survey panel vendors to investigate questions about a diverse array of 
topics, including programming priorities, consumer perceptions, willingness to pay, and water 
conservation (Harder et al., 2023; Holt et al., 2015; Kelly et al., 2019; Warner & Diaz, 2021). 
Using a data panel can prevent extension clientele from becoming fatigued by multiple survey 
requests, particularly when the clientele population may be limited. Furthermore, contracted 
data panels tend to provide respondents who are not familiar with extension services and are 
less likely to have biased answers (Warner & Diaz, 2021) which can be useful when the 
questions being asked are relevant to the general population.  
 
However, the integrity of the data obtained from survey panel systems is challenged by the 
potential for respondents’ dishonest behavior associated with the incentive system used by 
many panel providers. Some participants will engage in unethical behaviors to gain the 
incentive or multiple incentives (Chandler & Paolacci, 2017; Lawlor et al., 2021) or put forth the 
minimal effort needed to receive the reward in a strategy often described as satisficing (Hamby 
& Taylor, 2016; Roberts et al., 2019). Evidence of such behavior has been found in the 
extension literature (Harder et al., 2023; Narine et al., 2020; Warner & Diaz, 2021). Without 
data cleaning, fraudulent and/or low-quality responses can increase bias, erode data quality, 
and negatively impact organizational and programmatic decisions.  
 

Conceptual Framework 
 
Conceptually, the ability to identify fraudulent responses relies upon the identification of 
variables likely to be associated with such behaviors. Intermixed in the discussion of how to 
identify valid responses is the potential influence of demographics on response quality. Pratt-
Chapman et al. (2021) found age, race/ethnicity, marital status, income, and college completion 
characteristics were significantly different when comparing retained versus excluded responses 
to a web-based healthcare survey. Cognitive ability was found to be consistently related to 
lower quality responses caused by satisficing behaviors in a study by Kaminska et al. (2010); 
similarly, Zhang and Conrad (2014) found less educated respondents more likely to exhibit 
speeding and straightlining behaviors. Further, they found younger respondents tended to 
speed more often than older respondents. Fortunato et al. (2022) reported similar results and 
found men were more likely to exhibit “errors produced when respondents fail to provide 
quality responses through various satisficing behaviors” (p. 456), which they described as 
shirking. Based on the literature, researchers should consider the degree to which the 
demographic characteristics of the sample population may account for undesirable survey 
response behaviors. 
 
Pozzar et al. (2020) identified fraudulent responses as “those that strongly suggested 
automation or respondent misrepresentation” (Results, para. 2). Sometimes, less certainty 

https://doi.org/10.37433/aad.v6i1.573


Harder et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i1.573   99 
 

exists regarding the validity of responses. Low-quality or suspicious indicators are those “that 
could reasonably be attributed to respondent error or coincidence” (Pozzar et al., 2020, Results, 
para. 2). Multiple strategies for identifying and removing fraudulent or low-quality responses 
exist (e.g., Arndt et al., 2022; Belliveau & Yakovenko, 2022; Yarrish et al., 2019). Previous 
research by Spreen et al. (2020) found significant differences between the responses of 
“qualified, nonproblematic respondents” (p. 848) and their counterparts who were flagged for 
failing screening metrics. Qualtrics (2023) offers the use of Relevant ID Fraud scores which are 
designed to provide a numerical estimate of the likelihood that a response has been provided 
by a bot. Belliveau et al. (2022) and Lawlor et al. (2021) also recommended protocols for 
improving the validity of survey panel data.  
 
Commonly, straightlining and speeding are associated with fraudulent or low-quality responses. 
Straightlining can be defined as repetitively selecting the same response option across items, 
based on Zhang and Conrad’s (2014) description of the behavior. Speeding – the practice of 
going through a survey faster than reading comprehension allows – can be estimated based on 
an average reading speed of 300 words per minute (Carver, 1992).  
 
Given dueling concerns about protecting data integrity while not erroneously removing valid 
responses and decreasing the sample size (Johnson et al., 2023), the threshold of low-quality 
indicators that should be used to guide data cleaning decisions is unclear. Previous researchers 
(e.g., Revilla & Ochoa, 2015; Yarrish et al., 2019) cautioned against the quality of responses with 
multiple negative indicators. Is a single low-quality indicator sufficient to warrant a response’s 
removal? We sought to answer this question to increase the likelihood that data obtained from 
panels in applied extension research can be trusted to guide strategies to advance practice. 
 

Purpose 
 
The purpose of our study was to determine if survey panelists’ perceptions of items in an online 
survey are significantly different based on response quality. Objectives were to (a) assess 
differences in perceptions of broad issue areas between low-quality responses and normal 
responses, and (b) determine if any differences in perceptions were attributed to or moderated 
by age, education, income, and gender identity.  
 

Methods 
 
Data were collected in 2023 for a needs assessment conducted in Connecticut to assess adult 
residents’ perceptions of various issues with the intent of using the results to inform Extension 
program planning. We contracted with a for-profit survey panel vendor to obtain 1,000 usable 
responses from a convenience sample chosen to increase the odds that subgroups would be 
sufficiently large for meaningful analysis. The survey instrument contained three sections and is 
available as an appendix. Previous versions of the instrument were used by Narine et al. (2020) 
and Harder et al. (2023). The version used for our inquiry was reviewed for face validity by two 

https://doi.org/10.37433/aad.v6i1.573


Harder et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i1.573   100 
 

lifelong Connecticut residents with multiple years of extension experience; minor revisions to 
reflect local context were made based on their feedback. 
 
Data collection began with a soft launch via Qualtrics on August 25, 2023. The contracted panel 
provider sent 22 responses for our review. Data collection was completed on September 9, 
2023, with 1,030 responses. An unknown number of responses were removed by the 
contracted panel provider due to duplication of IP addresses, speeding, and bad open text (A. 
Olea, personal communication, October 4, 2023). We also removed responses due to Relevant 
ID Fraud scores, underage respondents, speeding, straightlining, and illogical open text.  
 
Low-quality (LQ) open-ended responses were then identified, such as when a respondent 
provided text like “I don’t know” unless an additional logical explanation was included. Unclear 
responses (e.g., “Is very informative and help to have you healthy safely”) and refusal to 
respond were also considered LQ, consistent with Revilla and Ochoa (2015) and Schmidt et al. 
(2020). We removed 20 responses with two or more low-quality indicators, leaving 94 
responses with one low-quality indicator.  
 
Data analysis sought to determine if participants’ perceptions toward issue areas differed based 
on their response quality, age, education, income, and gender identity. After the screening 
process, the LQ response group consisted of 94 observations. However, there were 876 
respondents in the normal quality (NQ) group. A random sample of 94 observations was taken 
from the NQ group using the Select Cases function in SPSS. As a result, the LQ response group 
(n = 94) and NQ group (n = 94) were equal in sample size. A Chi-square test was used to 
compare the LQ and NQ groups based on age, education, income, and gender identity. A priori 
p was set to 0.05 for statistical significance; results of the Chi-square test indicated there were 
no statistically significant differences in age (X2 = 11.09, p = .085), education (X2 = 4.02,  
p = .546), income (X2 = 1.01, p = .798), and gender identity (X2 = 0.43, p = .513) between the LQ 
and NQ groups. This implies the groups exhibited similar background characteristics.  
 
The survey instrument gathered data on respondents’ perceptions toward 45 items. A principal 
component analysis (PCA) was used to reduce the data into broad issue areas, each comprising 
highly intercorrelated items. However, four youth-focused issue items had low factor loading 
scores and did not emerge as a unique factor. Therefore, the final PCA was conducted on 41 
issue items. This reduction was appropriate to explain the variation in perceptions towards 
issues based on response quality without having to repeat the analysis for each issue item 
(Warner, 2012). Each factor was treated as an interval variable because a construct score was 
calculated from the raw scores of items within the factor (Carifio & Perla, 2008; Likert, 1932).  
 
An independent samples t-test with Cohen’s d estimate of effect size was used to determine if 
there were statistically significant differences in perceptions towards each issue area based on 
response quality. A series of two-way ANOVAs were conducted to assess the potential role of 
demographic factors on the differences in perceptions between the LQ response group and the 
NQ group. Perceptions toward each issue area were examined based on the interaction effect 
of group membership and age, education, income, and gender identity. The interaction effect 

https://doi.org/10.37433/aad.v6i1.573


Harder et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i1.573   101 
 

was necessary to conclude whether differences in perceptions varied because of demographics 
and group membership. Therefore, the statistical significance of the interaction term 
determined if the influence of group membership on perceptions depended on age, education, 
income, and gender identity. 
 

Findings 
 
Data Reduction  
A PCA with a varimax rotation was used to extract latent components from the 41 issue items, 
hereafter referred to as issue areas. The scree plot indicated four components were ideal and 
together explained 62% of the common variance across items. The Kaiser–Meyer–Olkin 
measure was 0.91 which indicates good sampling adequacy. Bartlett’s test was statistically 
significant (x2 = 6313.85, p < 0.001), which suggests the data was not an identity matrix and 
therefore, a PCA was appropriate (Jolliffe, 2002). The four factors or issue areas, all with 
eigenvalues above 2.00, were labeled as follows: (a) Agriculture (9 items, Cronbach’s α = .92), 
(b) Natural Resources (8 items, Cronbach’s α = .93), (c) Wellbeing and Family (10 items, 
Cronbach’s α = .92), and (d) Resilience and Health (14 items, Cronbach’s α = .93).  
 
Perceptions of Issue Areas by Response Quality  
As shown in Table 1, findings from the independent samples t-tests showed there were 
statistically significant differences in respondents’ perceptions of all issue areas based on 
response quality. Looking at Cohen’s d, the effect sizes were large for Agriculture and Natural 
Resources and medium for Well-being and Family, and Resilience and Health. Based on Cohen’s 
d, there was a 71% percent chance that an individual picked at random in the low-quality group 
had a lower mean score compared to those in the normal group for Agriculture and Natural 
Resources. For Resilience and Health, there was a 66% chance that an individual picked at 
random in the low-quality group had a lower score compared to individuals in the normal 
group. Lastly, for Wellbeing and Family, there was a 62% chance that an individual picked at 
random in the low-quality group had a lower score compared to individuals in the normal 
group. The effect was more pronounced for the issue areas of Agriculture and Natural 
Resources, but the low-quality response group rated all issue areas significantly lower 
compared to the respondents in the normal group.  
 
  

https://doi.org/10.37433/aad.v6i1.573


Harder et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i1.573   102 
 

Table 1 
 
Differences in Perceptions based on Response Quality 
Issue Area Group M SD t p (two-sided) d 
Agriculture LQ 3.52 0.76 5.43 <.001 0.79 

NQ 4.06 0.58 
Wellbeing and Family LQ 3.41 0.82 2.93 <.01 0.43 

NQ 3.77 0.85 
Resilience and Health LQ 3.26 0.73 4.08 <.001 0.60 

NQ 3.68 0.69 
Natural Resources LQ 3.24 0.83 5.39 <.001 0.79 

NQ 3.87 0.78 
Note. LQ = Low-Quality Group, NQ = Normal Group. 
 
Table 2 shows the direct and moderating effects of the response group and age based on the 
two-way ANOVA. As established in Table 1, response quality had a statistically significant direct 
effect on perceptions toward all issue areas. However, age did not have a direct effect on 
perceptions across any issue area. Lastly, results show the interaction between the response 
quality group and age did not have a statistically significant effect on perceptions. This suggests 
age did not influence the differences in perceptions between the low-quality response group 
and the normal group.  
 
Table 2 
 
Effect of Age and Response Quality on Perceptions 
Issue Area Effect f p ηp

2 
Agriculture Quality 18.44 <.001 .10 

Age 1.19 .31 .04 
Quality x Age 0.31 .93 .01 

     
Wellbeing and 

Family 
Quality 7.98 <.01 .04 
Age 0.70 .65 .02 
Quality x Age 0.56 .76 .02 

     
Resilience and 

Health 
Quality 13.79 <.001 .07 
Age 1.37 .23 .05 
Quality x Age .61 .73 .02 

     
Natural 

Resources 
Quality 16.52 <.001 .09 
Age 1.05 .40 .04 
Quality x Age .46 .84 .02 

 

https://doi.org/10.37433/aad.v6i1.573


Harder et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i1.573   103 
 

Like the results in Table 2, Table 3 shows response quality had a direct and statistically 
significant effect on perceptions across all issue areas. Yet, income did not have a direct effect 
on perceptions, and the interaction between the response quality group and income did not 
have a statistically significant effect on perceptions. Therefore, respondents’ income did not 
influence the differences in perceptions between the low-quality response group and the 
normal group. 
 
Table 3 
 
Effect of Income and Response Quality on Perceptions 
Issue Area Effect f p ηp

2 
Agriculture Quality 21.49 <.001 .11 

Income 0.22 .88 .00 
Quality x Income 1.43 .24 .02 

     
Wellbeing and 

Family 
Quality 6.16 <.01 .03 
Income 1.67 .17 .03 
Quality x Income 1.23 .30 .02 

     
Resilience and 

Health 
Quality 9.92 <.001 .05 
Income 1.85 .14 .03 
Quality x Income 1.12 .34 .02 

     
Natural 

Resources 
Quality 19.50 <.001 .10 
Income 2.26 .08 .04 
Quality x Income 1.98 .12 .03 

 
Consistent with previous results, Table 4 shows the established direct effect of the response 
quality group on perceptions across all issue areas. It also indicates gender identity did not 
directly influence perceptions, while the interaction between group membership and gender 
identity did not influence perceptions. Gender identity was not a moderating variable in the 
model which indicates it did not account for the differences in respondents’ perceptions on all 
four issue areas between the low-quality response group and normal group.  
 
  

https://doi.org/10.37433/aad.v6i1.573


Harder et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i1.573   104 
 

Table 4 
 
Effect of Gender Identity and Response Quality on Perceptions 
Issue Area Effect f p ηp

2 
Agriculture Quality 29.33 <.001 .14 

Gender Identity 0.04 .84 .00 
Quality x Gender Identity 0.05 .82 .00 

     
Wellbeing and 

Family 
Quality 8.75 <.001 .05 
Gender Identity 3.20 .08 .02 
Quality x Gender Identity 0.39 .53 .00 

     
Resilience and 

Health 
Quality 16.76 <.001 .08 
Gender Identity 3.92 .05 .02 
Quality x Gender Identity 0.96 .33 .01 

     
Natural 

Resources 
Quality 28.50 <.001 .13 
Gender Identity 0.38 .54 .00 
Quality x Gender Identity 0.01 .93 .00 

 
Respondents’ educational level did not have a statistically significant effect on their perceptions 
of issue areas (see Table 5). Likewise, the interaction between group membership and 
education was not an influential factor across all issue areas. Therefore, education did not 
moderate the effect of response quality on perceptions toward issue areas. 
 
Table 5 
 
Effect of Education and Response Quality on Perceptions 
Issue Area Effect f p ηp

2 
Agriculture Quality 22.46 <.001 .11 

Education 1.11 .36 .03 
Quality x Education 0.38 .86 .01 

     
Wellbeing and 

Family 
Quality 4.11 .04 .02 
Education 1.68 .14 .05 
Quality x Education 0.82 .53 .02 

     
Resilience and 

Health 
Quality 12.18 <.001 .07 
Education 1.16 .33 .03 
Quality x Education 0.53 .76 .02 

     
Natural 

Resources 
Quality 22.25 <.001 .11 
Education 1.34 .25 .04 
Quality x Education 0.17 .97 .01 

https://doi.org/10.37433/aad.v6i1.573


Harder et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i1.573   105 
 

Conclusions, Discussion, and Recommendations 
 
Researchers conducting web-based surveys with paid panels to inform extension strategic 
planning and programming decisions should consider adopting more conservative strategies for 
cleaning data than have previously been indicated in the literature. While some researchers 
(Revilla & Ochoa, 2015; Yarrish et al., 2019) have implemented or recommended methods 
based on removing responses with multiple low-quality indicators, we found that responses 
with a single low-quality indicator were significantly different from those without low-quality 
indicators. Further, those differences could not be attributed to age, income, or gender 
identity, which might have been expected based on prior research, which found that younger, 
less educated, and/or male respondents are more likely to demonstrate satisficing or other 
undesirable survey-taking behaviors (Fortunato et al., 2022; Pratt-Chapman et al., 2021; Zhang 
& Conrad, 2014). 
 
An important component of the methods we applied to clean the data was the inclusion of an 
open-ended question, which required respondents to demonstrate more cognitive effort than 
needed for the other instrument items. Over 80% (n = 78) of the 94 respondents who were 
flagged for a single low-quality indicator received that designation because they had indicated 
they did not know or were not sure of the answer to the question, refused to answer the 
question, or provided an unclear response to the question. These respondents were not guilty 
of other satisficing behaviors and passed the other screening checks we conducted, and their 
responses would have been included in the final data set had it not been for their low-quality 
response to the cognitively demanding question. In this way, the open-ended responses alerted 
us to a problem we would have otherwise missed. We advise extension researchers to consider 
incorporating at least one cognitively demanding question in their survey instruments so it can 
be used as a screening tool in the data cleaning process. 
 
Extension researchers face drawbacks when adopting a more conservative data-cleaning 
strategy. Extension researchers may have a reduced sample size or need more participants due 
to the higher number of responses discarded. Additional budget requirements and data 
cleaning can be inconvenient; however, low-quality data that negatively impacts the 
organizations’ strategic priority areas is a larger problem (Johnson et al., 2023). Extension 
researchers also need to be cognizant of the ethical issues of removing data and use a data 
cleaning system that ensures data integrity (Lawlor et al., 2021). We believe the drawbacks of a 
conservative data cleaning strategy outweigh the benefit of using high-quality data that helps 
guide the organization and influence policy decisions to advance agricultural development. 
Implications and recommendations arising from extension education research are frequently 
based on survey data, and researchers should strive for data accuracy and integrity via 
consistent and transparent data cleaning processes. 
 
Our study was conducted with paid survey panelists in one state in the U.S., which is a 
limitation of the study. The results may not be generalizable to other geographic locations or 
other online survey respondents recruited through different methods. However, we believe the 

https://doi.org/10.37433/aad.v6i1.573


Harder et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i1.573   106 
 

extensive literature covering concerns about respondent behavior for online surveys is one 
argument for sharing our results; the problem is clearly not limited to a single state in a single 
country nor has a definitive solution for fixing the problem been identified yet. There is a need 
for those of us who use online surveys to guide organizational, programmatic, and 
development policy decisions to share what we learn as we strive to protect the integrity of 
data used for those decisions. Secondly, the insignificant cost to experiment with the inclusion 
of a single cognitively demanding question in an online survey is so low that researchers may 
elect to experiment with this strategy in their own investigations with very little risk. We urge 
those who do so to publish their results to advance the discussion of data integrity in online 
survey research. 
 

Acknowledgments 
 
Author Contributions: A. Harder – conceptualization, methodology, formal analysis, 
investigation, writing – original draft, project administration; L. Narine – methodology, formal 
analysis, writing – original draft; S. Stearns – writing – original draft, writing – reviewing and 
editing 
 

References 
 
Arndt, A. D., Ford, J. B., Babin, B. J., & Luong, V. (2022). Collecting samples from online services: 

how to use screeners to improve data quality. International Journal of Research in 
Marketing, 39(1), 117–133. https://doi.org/10.1016/j.ijresmar.2021.05.001   

 
Belliveau, J., Soucy, K. I., & Yakovenko, I. (2022). The validity of Qualtrics panel data for research 

on video gaming and gaming disorder. Experimental and Clinical Psychopharmacology, 
30(4), 424–431. https://doi.org/10.1037/pha0000575 

 
Belliveau, J., & Yakovenko, I. (2022). Evaluating and improving the quality of survey data from 

panel and crowd-sourced samples: A practical guide for psychological research. 
Experimental and Clinical Psychopharmacology, 30(4), 400–408. 
https://doi.org/10.1037/pha0000564 

 
Carifio, J., & Perla, R. (2008). Resolving the 50-year debate around using and misusing Likert 

scales. Medical Education, 42(12), 1150–1152. https://doi.org/10.1111/j.1365-
2923.2008.03172.x  

 
Carver, R. P. (1992). Reading rate: Theory, research, and practical implications. Journal of 

Reading, 36(2), 84–95. https://api.semanticscholar.org/CorpusID:143405844 
 
Chandler, J. J., & Paoloacci, G. (2017). Lie for a dime: When most prescreening responses are 

honest but most study participants are imposters. Social Psychology and Personality 
Science, 8(5), 500–508. https://doi.org/10.1177/1948550617698203  

 

https://doi.org/10.37433/aad.v6i1.573
https://doi.org/10.1016/j.ijresmar.2021.05.001
https://psycnet.apa.org/doi/10.1037/pha0000575
https://psycnet.apa.org/doi/10.1037/pha0000564
https://doi.org/10.1111/j.1365-2923.2008.03172.x
https://doi.org/10.1111/j.1365-2923.2008.03172.x
https://doi.org/10.1177/1948550617698203


Harder et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i1.573   107 
 

Fortunato, D., Hibbing, M. V., & Hibbing, T. P. (2022). Hurdles to inference: The demographic 
correlates of survey breakoff and shirking. Social Science Quarterly, 103(2), 455–465. 
https://doi.org/10.1111/ssqu.13128  

 
Hamby, T., & Taylor, W. (2016). Survey satisficing inflates reliability and validity measures: An 

experimental comparison of college and Amazon Mechanical Turk samples. Educational 
and Psychological Measurement, 76(6), 912–932. 
https://doi.org/10.1177/0013164415627349  

 
Harder, A., Craig, D., Israel, G., Benge, M., & Caillouet, O. (2023). Exploring the possibilities of a 

standardized questionnaire for assessing residents’ needs. Journal of Extension, 61(2), 
Article 1. https://doi.org/10.34068/joe.61.02.01   

 
Holt, J., Rumble, J. N., Telg, R., & Lamm, A. (2015) The message or the channel: An experimental 

design of consumers' perceptions of a local food message and the media channels used 
to deliver the information. Journal of Applied Communications, 99(4). 
https://doi.org/10.4148/1051-0834.1053  

 
Johnson, M. S., Adams, V. M., & Byrne, J. (2023). Addressing fraudulent responses in online 

surveys: Insights from a web-based participatory mapping study. People and Nature, 
6(1), 147–164. https://doi.org/10.1002/pan3.10557  

 
Jolliffe, I. T. (2002). Principal component analysis (2nd ed.). Springer. 
 
Kaminska, O., McCutcheon, A. L., Billiet, J. (2010). Satisficing among reluctant respondents in a 

cross-national context. Public Opinion Quarterly, 74(5), 956–984. 
https://doi.org/10.1093/poq/nfq062  

 
Kelly, M. R., Getchis, T., Concepcion, A., & Bovay, J. (2019). Using online panels to inform 

extension programming. Journal of Extension, 57(5), Article 24. 
https://doi.org/10.34068/joe.57.05.24 

 
Lawlor, J., Thomas, C., Guhin, A. T., Kenyon, K., Lerner, M. D., & Drahota, A. (2021). Suspicious 

and fraudulent online survey participation: Introducing the REAL framework. 
Methodological Innovations, 14(3). https://doi.org/10.1177/20597991211050467  

 
Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 

22(140), 1–55. https://psycnet.apa.org/record/1933-01885-001 
 
Narine, L. K., Ali, A. D., & Hill, P. A. (2020). Application of a three-phase needs assessment 

framework to identify priority issue areas for extension programming. Journal of 
Extension, 58(4), Article 24. https://open.clemson.edu/joe/vol58/iss4/24/  

 
Pozzar, R., Hammer, M. J., Underhill-Blazey, M., Wright, A. A., Tulsky, J. A., Hong, F., Gunderson, 

D. A., & Berry, D. L. (2020). Threats of bots and other bad actors to data quality 
following research participant recruitment through social media: Cross-sectional 

https://doi.org/10.37433/aad.v6i1.573
https://doi.org/10.1111/ssqu.13128
https://doi.org/10.1177/0013164415627349
https://doi.org/10.34068/joe.61.02.01
https://doi.org/10.4148/1051-0834.1053
https://doi.org/10.1002/pan3.10557
https://doi.org/10.1093/poq/nfq062
https://doi.org/10.34068/joe.57.05.24
https://doi.org/10.1177/20597991211050467
https://open.clemson.edu/joe/vol58/iss4/24/


Harder et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i1.573   108 
 

questionnaire. Journal of Medical Internet Research, 22(10), e23021. 
https://doi.org/10.2196/23021  

 
Pratt-Chapman, M., Moses, J., & Arem, H. (2021). Strategies for the identification and 

prevention of survey fraud: Data analysis of a web-based survey. JMIR Cancer, 7(3), 
e30730. https://doi.org/10.2196/30730  

 
Qualtrics. (2023, October 1). Fraud detection. https://www.qualtrics.com/support/survey-

platform/survey-module/survey-checker/fraud-detection/ 
 
Revilla, M., & Ochoa, C. (2015). What are the links in a web survey among response time, 

quality, and auto-evaluation of the efforts done? Social Science Computer Review, 33(1), 
97–114. https://doi.org/10.1177/0894439314531214  

 
Roberts, C., Gilbert, E., Allum, N., Eisner, L. (2019). Research synthesis: Satisficing in surveys: A 

systematic review of the literature. Public Opinion Quarterly, 83(3), 598–626. 
https://doi.org/10.1093/poq/nfz035  

 
Schmidt, K., Gummer, T., & Roßmann, J. (2020). Effects of respondent and survey characteristics 

on the response quality of an open-ended attitude question in web surveys. methods, 
data, analyses, 14(1), 3–34. https://doi.org/10.12758/mda.2019.05  

 
Spreen, T. L., House, L. A., & Gao, Z. (2020). The impact of varying financial incentives on data 

quality in web panel surveys. Journal of Survey Statistics and Methodology, 8(5), 832–
850. https://doi.org/10.1093/jssam/smz030  

 
Warner, L. A., & Diaz, J. M. (2021). Amplifying the theory of planned behavior with 

connectedness to water to inform impactful water conservation program planning and 
evaluation. The Journal of Agricultural Education and Extension, 27(2), 229–253. 
https://doi.org/10.1080/1389224X.2020.1844771 

 
Warner, R. M. (2012). Applied statistics: From bivariate through multivariate techniques. Sage 

Publications. 
 
Yarrish, C., Groshon, L., Mitchell, J. D., Appelbaum, A., Klock, S., Winternitz, T., & Friedman-

Wheeler, D. G. (2019). Finding the signal in the noise: Minimizing responses from bots 
and inattentive humans in online research. The Behavior Therapist, 42(7), 235–242. 

 
Zhang, C., & Conrad, F. G. (2014). Speeding in Web surveys: The tendency to answer very fast 

and its association with straightlining. Survey Research Methods, 8(2), 127–135. 
https://doi.org/10.18148/srm/2014.v8i2.5453   

 
© 2025 by authors. This article is an open access article distributed under the terms and conditions of 
the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/). 
 

https://doi.org/10.37433/aad.v6i1.573
https://doi.org/10.2196/23021
https://doi.org/10.2196/30730
https://www.qualtrics.com/support/survey-platform/survey-module/survey-checker/fraud-detection/
https://www.qualtrics.com/support/survey-platform/survey-module/survey-checker/fraud-detection/
https://doi.org/10.1177/0894439314531214
https://doi.org/10.1093/poq/nfz035
https://doi.org/10.12758/mda.2019.05
https://doi.org/10.1093/jssam/smz030
https://doi.org/10.1080/1389224X.2020.1844771
https://doi.org/10.18148/srm/2014.v8i2.5453

