Advancements in Agricultural Development Volume 6, Issue 1, 2025 agdevresearch.org 1. Amy Harder, Associate Dean and Professor, 1376 Storrs Rd., United 4134, Storrs, CT 06269-4134, amy.harder@uconn.edu, https://orcid.org/0000-0002-7042-2028 2. Lendel Kade Narine, Extension Associate Professor, Utah State University, 4900 Old Main Hill, Logan, UT 84322, lendel.narine@usu.edu, https://orcid.org/0000-0001-6962-2770 3. Stacey Stearns, Communications Specialist, University of Connecticut Extension, 1376 Storrs Road, Unit 4134, Storrs, CT 06269-4134, stacey.stearns@uconn.edu, https://orcid.org/0000-0002-4711-7244 97 Decision-Making and Data Quality: Applying Fraud Response Strategies to Clean Survey Panel Data A. Harder1, L. K. Narine2, S. Stearns3 Article History Received: December 18, 2024 Accepted: February 3, 2025 Published: March 8, 2025 Keywords SDG 4: Quality Education; online surveys; survey straightlining; low-quality indicators; extension professionals Abstract Survey panels provide extension professionals with a valuable tool for collecting data on a wide range of topics without overburdening their program participants. Paid data panels are particularly useful for gathering unbiased feedback about Extension programs. However, some survey participants in these panels engage in satisficing or straightlining behaviors to earn rewards with minimal effort, which compromises data quality. This study explored whether survey panelists’ perceptions of online survey items varied based on response quality. It compared normal and low-quality responses across broad issue areas and investigated whether age, education, income, or gender identity influenced these differences. Analysis of 94 respondents in each group revealed no significant data quality differences based on age, education, income, or gender identity. There was a statistically significant difference in data quality when using an open-ended question requiring greater cognitive effort. We recommend adopting more conservative data cleaning strategies. While this approach has limitations, its benefits are particularly valuable when the data informs an organization’s strategic priorities. mailto:amy.harder@uconn.edu https://orcid.org/0000-0002-7042-2028 mailto:lendel.narine@usu.edu https://orcid.org/0000-0001-6962-2770 mailto:stacey.stearns@uconn.edu https://orcid.org/0000-0002-4711-7244 Harder et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v6i1.573 98 Introduction and Problem Statement Online survey technologies have made it increasingly easy for researchers to contract with for- profit survey providers to collect data from convenience samples. In the U.S., extension professionals have used survey panel vendors to investigate questions about a diverse array of topics, including programming priorities, consumer perceptions, willingness to pay, and water conservation (Harder et al., 2023; Holt et al., 2015; Kelly et al., 2019; Warner & Diaz, 2021). Using a data panel can prevent extension clientele from becoming fatigued by multiple survey requests, particularly when the clientele population may be limited. Furthermore, contracted data panels tend to provide respondents who are not familiar with extension services and are less likely to have biased answers (Warner & Diaz, 2021) which can be useful when the questions being asked are relevant to the general population. However, the integrity of the data obtained from survey panel systems is challenged by the potential for respondents’ dishonest behavior associated with the incentive system used by many panel providers. Some participants will engage in unethical behaviors to gain the incentive or multiple incentives (Chandler & Paolacci, 2017; Lawlor et al., 2021) or put forth the minimal effort needed to receive the reward in a strategy often described as satisficing (Hamby & Taylor, 2016; Roberts et al., 2019). Evidence of such behavior has been found in the extension literature (Harder et al., 2023; Narine et al., 2020; Warner & Diaz, 2021). Without data cleaning, fraudulent and/or low-quality responses can increase bias, erode data quality, and negatively impact organizational and programmatic decisions. Conceptual Framework Conceptually, the ability to identify fraudulent responses relies upon the identification of variables likely to be associated with such behaviors. Intermixed in the discussion of how to identify valid responses is the potential influence of demographics on response quality. Pratt- Chapman et al. (2021) found age, race/ethnicity, marital status, income, and college completion characteristics were significantly different when comparing retained versus excluded responses to a web-based healthcare survey. Cognitive ability was found to be consistently related to lower quality responses caused by satisficing behaviors in a study by Kaminska et al. (2010); similarly, Zhang and Conrad (2014) found less educated respondents more likely to exhibit speeding and straightlining behaviors. Further, they found younger respondents tended to speed more often than older respondents. Fortunato et al. (2022) reported similar results and found men were more likely to exhibit “errors produced when respondents fail to provide quality responses through various satisficing behaviors” (p. 456), which they described as shirking. Based on the literature, researchers should consider the degree to which the demographic characteristics of the sample population may account for undesirable survey response behaviors. Pozzar et al. (2020) identified fraudulent responses as “those that strongly suggested automation or respondent misrepresentation” (Results, para. 2). Sometimes, less certainty https://doi.org/10.37433/aad.v6i1.573 Harder et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v6i1.573 99 exists regarding the validity of responses. Low-quality or suspicious indicators are those “that could reasonably be attributed to respondent error or coincidence” (Pozzar et al., 2020, Results, para. 2). Multiple strategies for identifying and removing fraudulent or low-quality responses exist (e.g., Arndt et al., 2022; Belliveau & Yakovenko, 2022; Yarrish et al., 2019). Previous research by Spreen et al. (2020) found significant differences between the responses of “qualified, nonproblematic respondents” (p. 848) and their counterparts who were flagged for failing screening metrics. Qualtrics (2023) offers the use of Relevant ID Fraud scores which are designed to provide a numerical estimate of the likelihood that a response has been provided by a bot. Belliveau et al. (2022) and Lawlor et al. (2021) also recommended protocols for improving the validity of survey panel data. Commonly, straightlining and speeding are associated with fraudulent or low-quality responses. Straightlining can be defined as repetitively selecting the same response option across items, based on Zhang and Conrad’s (2014) description of the behavior. Speeding – the practice of going through a survey faster than reading comprehension allows – can be estimated based on an average reading speed of 300 words per minute (Carver, 1992). Given dueling concerns about protecting data integrity while not erroneously removing valid responses and decreasing the sample size (Johnson et al., 2023), the threshold of low-quality indicators that should be used to guide data cleaning decisions is unclear. Previous researchers (e.g., Revilla & Ochoa, 2015; Yarrish et al., 2019) cautioned against the quality of responses with multiple negative indicators. Is a single low-quality indicator sufficient to warrant a response’s removal? We sought to answer this question to increase the likelihood that data obtained from panels in applied extension research can be trusted to guide strategies to advance practice. Purpose The purpose of our study was to determine if survey panelists’ perceptions of items in an online survey are significantly different based on response quality. Objectives were to (a) assess differences in perceptions of broad issue areas between low-quality responses and normal responses, and (b) determine if any differences in perceptions were attributed to or moderated by age, education, income, and gender identity. Methods Data were collected in 2023 for a needs assessment conducted in Connecticut to assess adult residents’ perceptions of various issues with the intent of using the results to inform Extension program planning. We contracted with a for-profit survey panel vendor to obtain 1,000 usable responses from a convenience sample chosen to increase the odds that subgroups would be sufficiently large for meaningful analysis. The survey instrument contained three sections and is available as an appendix. Previous versions of the instrument were used by Narine et al. (2020) and Harder et al. (2023). The version used for our inquiry was reviewed for face validity by two https://doi.org/10.37433/aad.v6i1.573 Harder et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v6i1.573 100 lifelong Connecticut residents with multiple years of extension experience; minor revisions to reflect local context were made based on their feedback. Data collection began with a soft launch via Qualtrics on August 25, 2023. The contracted panel provider sent 22 responses for our review. Data collection was completed on September 9, 2023, with 1,030 responses. An unknown number of responses were removed by the contracted panel provider due to duplication of IP addresses, speeding, and bad open text (A. Olea, personal communication, October 4, 2023). We also removed responses due to Relevant ID Fraud scores, underage respondents, speeding, straightlining, and illogical open text. Low-quality (LQ) open-ended responses were then identified, such as when a respondent provided text like “I don’t know” unless an additional logical explanation was included. Unclear responses (e.g., “Is very informative and help to have you healthy safely”) and refusal to respond were also considered LQ, consistent with Revilla and Ochoa (2015) and Schmidt et al. (2020). We removed 20 responses with two or more low-quality indicators, leaving 94 responses with one low-quality indicator. Data analysis sought to determine if participants’ perceptions toward issue areas differed based on their response quality, age, education, income, and gender identity. After the screening process, the LQ response group consisted of 94 observations. However, there were 876 respondents in the normal quality (NQ) group. A random sample of 94 observations was taken from the NQ group using the Select Cases function in SPSS. As a result, the LQ response group (n = 94) and NQ group (n = 94) were equal in sample size. A Chi-square test was used to compare the LQ and NQ groups based on age, education, income, and gender identity. A priori p was set to 0.05 for statistical significance; results of the Chi-square test indicated there were no statistically significant differences in age (X2 = 11.09, p = .085), education (X2 = 4.02, p = .546), income (X2 = 1.01, p = .798), and gender identity (X2 = 0.43, p = .513) between the LQ and NQ groups. This implies the groups exhibited similar background characteristics. The survey instrument gathered data on respondents’ perceptions toward 45 items. A principal component analysis (PCA) was used to reduce the data into broad issue areas, each comprising highly intercorrelated items. However, four youth-focused issue items had low factor loading scores and did not emerge as a unique factor. Therefore, the final PCA was conducted on 41 issue items. This reduction was appropriate to explain the variation in perceptions towards issues based on response quality without having to repeat the analysis for each issue item (Warner, 2012). Each factor was treated as an interval variable because a construct score was calculated from the raw scores of items within the factor (Carifio & Perla, 2008; Likert, 1932). An independent samples t-test with Cohen’s d estimate of effect size was used to determine if there were statistically significant differences in perceptions towards each issue area based on response quality. A series of two-way ANOVAs were conducted to assess the potential role of demographic factors on the differences in perceptions between the LQ response group and the NQ group. Perceptions toward each issue area were examined based on the interaction effect of group membership and age, education, income, and gender identity. The interaction effect https://doi.org/10.37433/aad.v6i1.573 Harder et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v6i1.573 101 was necessary to conclude whether differences in perceptions varied because of demographics and group membership. Therefore, the statistical significance of the interaction term determined if the influence of group membership on perceptions depended on age, education, income, and gender identity. Findings Data Reduction A PCA with a varimax rotation was used to extract latent components from the 41 issue items, hereafter referred to as issue areas. The scree plot indicated four components were ideal and together explained 62% of the common variance across items. The Kaiser–Meyer–Olkin measure was 0.91 which indicates good sampling adequacy. Bartlett’s test was statistically significant (x2 = 6313.85, p < 0.001), which suggests the data was not an identity matrix and therefore, a PCA was appropriate (Jolliffe, 2002). The four factors or issue areas, all with eigenvalues above 2.00, were labeled as follows: (a) Agriculture (9 items, Cronbach’s α = .92), (b) Natural Resources (8 items, Cronbach’s α = .93), (c) Wellbeing and Family (10 items, Cronbach’s α = .92), and (d) Resilience and Health (14 items, Cronbach’s α = .93). Perceptions of Issue Areas by Response Quality As shown in Table 1, findings from the independent samples t-tests showed there were statistically significant differences in respondents’ perceptions of all issue areas based on response quality. Looking at Cohen’s d, the effect sizes were large for Agriculture and Natural Resources and medium for Well-being and Family, and Resilience and Health. Based on Cohen’s d, there was a 71% percent chance that an individual picked at random in the low-quality group had a lower mean score compared to those in the normal group for Agriculture and Natural Resources. For Resilience and Health, there was a 66% chance that an individual picked at random in the low-quality group had a lower score compared to individuals in the normal group. Lastly, for Wellbeing and Family, there was a 62% chance that an individual picked at random in the low-quality group had a lower score compared to individuals in the normal group. The effect was more pronounced for the issue areas of Agriculture and Natural Resources, but the low-quality response group rated all issue areas significantly lower compared to the respondents in the normal group. https://doi.org/10.37433/aad.v6i1.573 Harder et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v6i1.573 102 Table 1 Differences in Perceptions based on Response Quality Issue Area Group M SD t p (two-sided) d Agriculture LQ 3.52 0.76 5.43 <.001 0.79 NQ 4.06 0.58 Wellbeing and Family LQ 3.41 0.82 2.93 <.01 0.43 NQ 3.77 0.85 Resilience and Health LQ 3.26 0.73 4.08 <.001 0.60 NQ 3.68 0.69 Natural Resources LQ 3.24 0.83 5.39 <.001 0.79 NQ 3.87 0.78 Note. LQ = Low-Quality Group, NQ = Normal Group. Table 2 shows the direct and moderating effects of the response group and age based on the two-way ANOVA. As established in Table 1, response quality had a statistically significant direct effect on perceptions toward all issue areas. However, age did not have a direct effect on perceptions across any issue area. Lastly, results show the interaction between the response quality group and age did not have a statistically significant effect on perceptions. This suggests age did not influence the differences in perceptions between the low-quality response group and the normal group. Table 2 Effect of Age and Response Quality on Perceptions Issue Area Effect f p ηp 2 Agriculture Quality 18.44 <.001 .10 Age 1.19 .31 .04 Quality x Age 0.31 .93 .01 Wellbeing and Family Quality 7.98 <.01 .04 Age 0.70 .65 .02 Quality x Age 0.56 .76 .02 Resilience and Health Quality 13.79 <.001 .07 Age 1.37 .23 .05 Quality x Age .61 .73 .02 Natural Resources Quality 16.52 <.001 .09 Age 1.05 .40 .04 Quality x Age .46 .84 .02 https://doi.org/10.37433/aad.v6i1.573 Harder et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v6i1.573 103 Like the results in Table 2, Table 3 shows response quality had a direct and statistically significant effect on perceptions across all issue areas. Yet, income did not have a direct effect on perceptions, and the interaction between the response quality group and income did not have a statistically significant effect on perceptions. Therefore, respondents’ income did not influence the differences in perceptions between the low-quality response group and the normal group. Table 3 Effect of Income and Response Quality on Perceptions Issue Area Effect f p ηp 2 Agriculture Quality 21.49 <.001 .11 Income 0.22 .88 .00 Quality x Income 1.43 .24 .02 Wellbeing and Family Quality 6.16 <.01 .03 Income 1.67 .17 .03 Quality x Income 1.23 .30 .02 Resilience and Health Quality 9.92 <.001 .05 Income 1.85 .14 .03 Quality x Income 1.12 .34 .02 Natural Resources Quality 19.50 <.001 .10 Income 2.26 .08 .04 Quality x Income 1.98 .12 .03 Consistent with previous results, Table 4 shows the established direct effect of the response quality group on perceptions across all issue areas. It also indicates gender identity did not directly influence perceptions, while the interaction between group membership and gender identity did not influence perceptions. Gender identity was not a moderating variable in the model which indicates it did not account for the differences in respondents’ perceptions on all four issue areas between the low-quality response group and normal group. https://doi.org/10.37433/aad.v6i1.573 Harder et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v6i1.573 104 Table 4 Effect of Gender Identity and Response Quality on Perceptions Issue Area Effect f p ηp 2 Agriculture Quality 29.33 <.001 .14 Gender Identity 0.04 .84 .00 Quality x Gender Identity 0.05 .82 .00 Wellbeing and Family Quality 8.75 <.001 .05 Gender Identity 3.20 .08 .02 Quality x Gender Identity 0.39 .53 .00 Resilience and Health Quality 16.76 <.001 .08 Gender Identity 3.92 .05 .02 Quality x Gender Identity 0.96 .33 .01 Natural Resources Quality 28.50 <.001 .13 Gender Identity 0.38 .54 .00 Quality x Gender Identity 0.01 .93 .00 Respondents’ educational level did not have a statistically significant effect on their perceptions of issue areas (see Table 5). Likewise, the interaction between group membership and education was not an influential factor across all issue areas. Therefore, education did not moderate the effect of response quality on perceptions toward issue areas. Table 5 Effect of Education and Response Quality on Perceptions Issue Area Effect f p ηp 2 Agriculture Quality 22.46 <.001 .11 Education 1.11 .36 .03 Quality x Education 0.38 .86 .01 Wellbeing and Family Quality 4.11 .04 .02 Education 1.68 .14 .05 Quality x Education 0.82 .53 .02 Resilience and Health Quality 12.18 <.001 .07 Education 1.16 .33 .03 Quality x Education 0.53 .76 .02 Natural Resources Quality 22.25 <.001 .11 Education 1.34 .25 .04 Quality x Education 0.17 .97 .01 https://doi.org/10.37433/aad.v6i1.573 Harder et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v6i1.573 105 Conclusions, Discussion, and Recommendations Researchers conducting web-based surveys with paid panels to inform extension strategic planning and programming decisions should consider adopting more conservative strategies for cleaning data than have previously been indicated in the literature. While some researchers (Revilla & Ochoa, 2015; Yarrish et al., 2019) have implemented or recommended methods based on removing responses with multiple low-quality indicators, we found that responses with a single low-quality indicator were significantly different from those without low-quality indicators. Further, those differences could not be attributed to age, income, or gender identity, which might have been expected based on prior research, which found that younger, less educated, and/or male respondents are more likely to demonstrate satisficing or other undesirable survey-taking behaviors (Fortunato et al., 2022; Pratt-Chapman et al., 2021; Zhang & Conrad, 2014). An important component of the methods we applied to clean the data was the inclusion of an open-ended question, which required respondents to demonstrate more cognitive effort than needed for the other instrument items. Over 80% (n = 78) of the 94 respondents who were flagged for a single low-quality indicator received that designation because they had indicated they did not know or were not sure of the answer to the question, refused to answer the question, or provided an unclear response to the question. These respondents were not guilty of other satisficing behaviors and passed the other screening checks we conducted, and their responses would have been included in the final data set had it not been for their low-quality response to the cognitively demanding question. In this way, the open-ended responses alerted us to a problem we would have otherwise missed. We advise extension researchers to consider incorporating at least one cognitively demanding question in their survey instruments so it can be used as a screening tool in the data cleaning process. Extension researchers face drawbacks when adopting a more conservative data-cleaning strategy. Extension researchers may have a reduced sample size or need more participants due to the higher number of responses discarded. Additional budget requirements and data cleaning can be inconvenient; however, low-quality data that negatively impacts the organizations’ strategic priority areas is a larger problem (Johnson et al., 2023). Extension researchers also need to be cognizant of the ethical issues of removing data and use a data cleaning system that ensures data integrity (Lawlor et al., 2021). We believe the drawbacks of a conservative data cleaning strategy outweigh the benefit of using high-quality data that helps guide the organization and influence policy decisions to advance agricultural development. Implications and recommendations arising from extension education research are frequently based on survey data, and researchers should strive for data accuracy and integrity via consistent and transparent data cleaning processes. Our study was conducted with paid survey panelists in one state in the U.S., which is a limitation of the study. The results may not be generalizable to other geographic locations or other online survey respondents recruited through different methods. However, we believe the https://doi.org/10.37433/aad.v6i1.573 Harder et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v6i1.573 106 extensive literature covering concerns about respondent behavior for online surveys is one argument for sharing our results; the problem is clearly not limited to a single state in a single country nor has a definitive solution for fixing the problem been identified yet. There is a need for those of us who use online surveys to guide organizational, programmatic, and development policy decisions to share what we learn as we strive to protect the integrity of data used for those decisions. Secondly, the insignificant cost to experiment with the inclusion of a single cognitively demanding question in an online survey is so low that researchers may elect to experiment with this strategy in their own investigations with very little risk. We urge those who do so to publish their results to advance the discussion of data integrity in online survey research. Acknowledgments Author Contributions: A. Harder – conceptualization, methodology, formal analysis, investigation, writing – original draft, project administration; L. Narine – methodology, formal analysis, writing – original draft; S. Stearns – writing – original draft, writing – reviewing and editing References Arndt, A. D., Ford, J. B., Babin, B. J., & Luong, V. (2022). Collecting samples from online services: how to use screeners to improve data quality. International Journal of Research in Marketing, 39(1), 117–133. https://doi.org/10.1016/j.ijresmar.2021.05.001 Belliveau, J., Soucy, K. I., & Yakovenko, I. (2022). The validity of Qualtrics panel data for research on video gaming and gaming disorder. Experimental and Clinical Psychopharmacology, 30(4), 424–431. https://doi.org/10.1037/pha0000575 Belliveau, J., & Yakovenko, I. (2022). Evaluating and improving the quality of survey data from panel and crowd-sourced samples: A practical guide for psychological research. Experimental and Clinical Psychopharmacology, 30(4), 400–408. https://doi.org/10.1037/pha0000564 Carifio, J., & Perla, R. (2008). Resolving the 50-year debate around using and misusing Likert scales. Medical Education, 42(12), 1150–1152. https://doi.org/10.1111/j.1365- 2923.2008.03172.x Carver, R. P. (1992). Reading rate: Theory, research, and practical implications. Journal of Reading, 36(2), 84–95. https://api.semanticscholar.org/CorpusID:143405844 Chandler, J. J., & Paoloacci, G. (2017). Lie for a dime: When most prescreening responses are honest but most study participants are imposters. Social Psychology and Personality Science, 8(5), 500–508. https://doi.org/10.1177/1948550617698203 https://doi.org/10.37433/aad.v6i1.573 https://doi.org/10.1016/j.ijresmar.2021.05.001 https://psycnet.apa.org/doi/10.1037/pha0000575 https://psycnet.apa.org/doi/10.1037/pha0000564 https://doi.org/10.1111/j.1365-2923.2008.03172.x https://doi.org/10.1111/j.1365-2923.2008.03172.x https://doi.org/10.1177/1948550617698203 Harder et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v6i1.573 107 Fortunato, D., Hibbing, M. V., & Hibbing, T. P. (2022). Hurdles to inference: The demographic correlates of survey breakoff and shirking. Social Science Quarterly, 103(2), 455–465. https://doi.org/10.1111/ssqu.13128 Hamby, T., & Taylor, W. (2016). Survey satisficing inflates reliability and validity measures: An experimental comparison of college and Amazon Mechanical Turk samples. Educational and Psychological Measurement, 76(6), 912–932. https://doi.org/10.1177/0013164415627349 Harder, A., Craig, D., Israel, G., Benge, M., & Caillouet, O. (2023). Exploring the possibilities of a standardized questionnaire for assessing residents’ needs. Journal of Extension, 61(2), Article 1. https://doi.org/10.34068/joe.61.02.01 Holt, J., Rumble, J. N., Telg, R., & Lamm, A. (2015) The message or the channel: An experimental design of consumers' perceptions of a local food message and the media channels used to deliver the information. Journal of Applied Communications, 99(4). https://doi.org/10.4148/1051-0834.1053 Johnson, M. S., Adams, V. M., & Byrne, J. (2023). Addressing fraudulent responses in online surveys: Insights from a web-based participatory mapping study. People and Nature, 6(1), 147–164. https://doi.org/10.1002/pan3.10557 Jolliffe, I. T. (2002). Principal component analysis (2nd ed.). Springer. Kaminska, O., McCutcheon, A. L., Billiet, J. (2010). Satisficing among reluctant respondents in a cross-national context. Public Opinion Quarterly, 74(5), 956–984. https://doi.org/10.1093/poq/nfq062 Kelly, M. R., Getchis, T., Concepcion, A., & Bovay, J. (2019). Using online panels to inform extension programming. Journal of Extension, 57(5), Article 24. https://doi.org/10.34068/joe.57.05.24 Lawlor, J., Thomas, C., Guhin, A. T., Kenyon, K., Lerner, M. D., & Drahota, A. (2021). Suspicious and fraudulent online survey participation: Introducing the REAL framework. Methodological Innovations, 14(3). https://doi.org/10.1177/20597991211050467 Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22(140), 1–55. https://psycnet.apa.org/record/1933-01885-001 Narine, L. K., Ali, A. D., & Hill, P. A. (2020). Application of a three-phase needs assessment framework to identify priority issue areas for extension programming. Journal of Extension, 58(4), Article 24. https://open.clemson.edu/joe/vol58/iss4/24/ Pozzar, R., Hammer, M. J., Underhill-Blazey, M., Wright, A. A., Tulsky, J. A., Hong, F., Gunderson, D. A., & Berry, D. L. (2020). Threats of bots and other bad actors to data quality following research participant recruitment through social media: Cross-sectional https://doi.org/10.37433/aad.v6i1.573 https://doi.org/10.1111/ssqu.13128 https://doi.org/10.1177/0013164415627349 https://doi.org/10.34068/joe.61.02.01 https://doi.org/10.4148/1051-0834.1053 https://doi.org/10.1002/pan3.10557 https://doi.org/10.1093/poq/nfq062 https://doi.org/10.34068/joe.57.05.24 https://doi.org/10.1177/20597991211050467 https://open.clemson.edu/joe/vol58/iss4/24/ Harder et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v6i1.573 108 questionnaire. Journal of Medical Internet Research, 22(10), e23021. https://doi.org/10.2196/23021 Pratt-Chapman, M., Moses, J., & Arem, H. (2021). Strategies for the identification and prevention of survey fraud: Data analysis of a web-based survey. JMIR Cancer, 7(3), e30730. https://doi.org/10.2196/30730 Qualtrics. (2023, October 1). Fraud detection. https://www.qualtrics.com/support/survey- platform/survey-module/survey-checker/fraud-detection/ Revilla, M., & Ochoa, C. (2015). What are the links in a web survey among response time, quality, and auto-evaluation of the efforts done? Social Science Computer Review, 33(1), 97–114. https://doi.org/10.1177/0894439314531214 Roberts, C., Gilbert, E., Allum, N., Eisner, L. (2019). Research synthesis: Satisficing in surveys: A systematic review of the literature. Public Opinion Quarterly, 83(3), 598–626. https://doi.org/10.1093/poq/nfz035 Schmidt, K., Gummer, T., & Roßmann, J. (2020). Effects of respondent and survey characteristics on the response quality of an open-ended attitude question in web surveys. methods, data, analyses, 14(1), 3–34. https://doi.org/10.12758/mda.2019.05 Spreen, T. L., House, L. A., & Gao, Z. (2020). The impact of varying financial incentives on data quality in web panel surveys. Journal of Survey Statistics and Methodology, 8(5), 832– 850. https://doi.org/10.1093/jssam/smz030 Warner, L. A., & Diaz, J. M. (2021). Amplifying the theory of planned behavior with connectedness to water to inform impactful water conservation program planning and evaluation. The Journal of Agricultural Education and Extension, 27(2), 229–253. https://doi.org/10.1080/1389224X.2020.1844771 Warner, R. M. (2012). Applied statistics: From bivariate through multivariate techniques. Sage Publications. Yarrish, C., Groshon, L., Mitchell, J. D., Appelbaum, A., Klock, S., Winternitz, T., & Friedman- Wheeler, D. G. (2019). Finding the signal in the noise: Minimizing responses from bots and inattentive humans in online research. The Behavior Therapist, 42(7), 235–242. Zhang, C., & Conrad, F. G. (2014). Speeding in Web surveys: The tendency to answer very fast and its association with straightlining. Survey Research Methods, 8(2), 127–135. https://doi.org/10.18148/srm/2014.v8i2.5453 © 2025 by authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/). https://doi.org/10.37433/aad.v6i1.573 https://doi.org/10.2196/23021 https://doi.org/10.2196/30730 https://www.qualtrics.com/support/survey-platform/survey-module/survey-checker/fraud-detection/ https://www.qualtrics.com/support/survey-platform/survey-module/survey-checker/fraud-detection/ https://doi.org/10.1177/0894439314531214 https://doi.org/10.1093/poq/nfz035 https://doi.org/10.12758/mda.2019.05 https://doi.org/10.1093/jssam/smz030 https://doi.org/10.1080/1389224X.2020.1844771 https://doi.org/10.18148/srm/2014.v8i2.5453