Advancements in Agricultural Development Volume 5, Issue 2, 2024 agdevresearch.org 1. Lauri M. Baker, Associate Professor, Agricultural Education and Communication Department, University of Florida, 1408 Sabal Palm Drive, Level 2, Gainesville, FL 32611, lauri.m.baker@ufl.edu, https://orcid.org/0000-0002-4241-60773 2. Ashley McLeod-Morin, Associate Director of Strategic Communication, Southeastern Coastal Center for Agricultural Health and Safety, University of Florida, 1408 Sabal Palm Drive, Level 2, P.O. Box 110126, Gainesville, FL 32611, ashleynmcleod@ufl.edu, https://orcid.org/0000-0002-8649-9783 3. Anissa M. Mattox, Research Coordinator, UF/IFAS Center for Public Issues Education in Agriculture and Natural Resources, University of Florida, 1408 Sabal Palm Drive, Level 2, Gainesville, FL 32611, azagonel@ufl.edu, https://orcid.org/0000-0001-6359-58204 135 All in A Moment: Continuous Response Measurement Method for Analyzing Dynamic Science Communication Content Lauri M. Baker1, Ashley McLeod-Morin2, and Anissa Mattox3 Article History Received: June 1, 2023 Accepted: July 31, 2023 Published: January 31, 2024 Keywords perception analyzer dials; real time response measurement; dual processing theory Abstract Science communication strategy plays a crucial role in effectively conveying scientific information to target audiences. While various models exist for developing communication strategies, many focus on measuring the effectiveness of communication efforts and adjusting these based on audience feedback. However, traditional methods of evaluating communication effectiveness often measure one component at a time and do not consider how people make decisions in real-life situations. This paper proposes the use of continuous response measurement (CRM) as a method to evaluate science communication in agricultural social science research. CRM allows for real-time measurement of how individuals make decisions in response to dynamic communication content. This paper compares different types of CRM, including in-person and virtual dial CRM, and provides resources for researchers interested in implementing this methodology. The paper also discusses various research designs that can be used with CRM, such as experimental designs, survey designs, time-series designs, focus groups, and coding content and behavior. The benefits and limitations of CRM are outlined, highlighting the need for immediate feedback and real-time response in science communication campaigns. In-person CRM is discussed, including the selection of stimuli, response prompts, data collection procedures, and data analysis. Virtual CRM is also examined, highlighting its advantages in terms of flexibility and cost-effectiveness. The paper concludes by discussing data output and analysis methods for CRM data. Overall, this paper serves as a methodological proposal for the use of CRM in agricultural social science research, emphasizing the importance of real-time measurement and response in science communication. mailto:lauri.m.baker@ufl.edu https://orcid.org/0000-0002-4241-60773 mailto:ashleynmcleod@ufl.edu https://orcid.org/0000-0002-8649-9783 mailto:azagonel@ufl.edu https://orcid.org/0000-0001-6359-58204 Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 136 Introduction and Problem Statement Science communication strategy involves developing content that resonates with target audiences while emphasizing major tenants of science communication: timeliness, accessibility, trustworthiness, credibility, and usefulness. While models for developing a communication strategy vary, many models are cyclical in nature with an emphasis on measuring the effectiveness of communication efforts followed by adjusting communication based on feedback from target audiences (Percy, 2023; Villarreal, 2010). Primary and secondary consumer research play a vital role in developing a communication strategy and evaluating the strategy. Traditional methods for evaluating communication efforts include both quantitative and qualitative methodologies including surveys, sales data, behavioral observations, focus groups, interviews, and social media listening (Juska, 2021). Each of these methods has benefits in understanding how effective science communication is, but these tend to measure effectiveness in an artificial way, by focusing on measuring one component of a strategy at a time. However, modern behavioral economics indicates that people make decisions based on more than one factor at a time (Grayot, 2020). When you ask a question on a survey about a respondent’s perceptions of a campaign, you are forcing them to think about one element of a message at a time, but in real life they are making decisions based on multiple components at a time. Dual processing theory proposes that people make decisions two different ways (a) intuitively and (b) deliberately (De Neys & Pennycook, 2019). When you ask a question in a survey or interview you force people to deliberately decide based on one element. In a natural setting, they would decide intuitively based on a blend of message, graphics, audio, logo, color scheme, speaker characteristics, etc. For these reasons, it is imperative that researchers establish and embrace new methods for gathering real-time data from participants. Currently, many methods do not allow for real-time response or immediate feedback to change course in science communication campaigns. Continuous response measurement (CRM) offers a way to measure how people make decisions on dynamic, communication content in real time from moment-to-moment. CRM is commonly used in the communication industry for real-time measurement, but it has not yet been widely adopted in social science research (Maier et al., 2009). The purpose of this paper is to propose the use of CRM in agricultural social science research by describing the process and how to analyze data reaped from the method. Through these objectives, the authors will also compare types of CRM, including in-person dial testing and virtual dial testing, and provide resources for other researchers who want to apply this methodology in their own research. Continuous Response Measurement To effectively respond to public concerns and solve real problems, immediate feedback is needed. One of the public criticisms of scientists during the COVID-19 pandemic was a lack of transparent and immediate response (Hamel et al., 2020; Kenen & Roubein, 2020; Pollard & Davis, 2021). Particularly in the agriculture and natural resource sectors, immediate response is often necessary due to the essence and urgency of the industries, such as dealing with natural disasters (Mike et al., 2020), food safety issues (Opat et al., 2018), biosecurity risks (Sellnow et al., 2017), and animal welfare issues (Steede et al., 2018). CRM offers a way for researchers to https://doi.org/10.37433/aad.v5i2.357 Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 137 operate in a more immediate way than traditional research methods while being more authentic in the way messages are measured. Messages are a “continuous stream of sensory stimuli arranged in patterns,” (Biocca et al., 1994, p. 15) which convey meaning through and are interpreted based on individuals’ cognitive state. Purpose Science communication strategy is vital for delivering content to target audiences effectively. However, a gap exists in the ability of current methods to measure real-time response and feedback. This paper aims to address this gap by reviewing relevant literature that can serve as a guide to researchers, specifically those in agricultural social science research, considering Continuous Response Management (CRM) as a methodology. The following research objectives guided this study: RO1: Describe the planning process for conducting a CRM study. RO2: Describe how to analyze data gathered from a CRM study. Methods To address the research objectives of this study, research synthesis methodology was used to comprehensively review and synthesize existing literature on CRM methods to explain how to plan and conduct CRM studies, data analysis approaches for CRM, and best practices for CRM in academic settings. The authors’ search strategy included a systematic search of relevant scholarly databases (e.g., PubMed, Scopus, Web of Science) to retrieve articles published up until January 1, 2023. Additionally, grey literature, offline sources, and popular press references were explored to minimize publication bias and ensure all relevant resources were found. Search Terms and Inclusion Criteria The search strategy employed a combination of keywords and Boolean operators to identify relevant studies. These keywords included the exact terms and variations of the following terms: continuous response measurement, perception analyzer dials, online/virtual dial testing, dial testing. Articles were included if these met the predefined inclusion criteria based on if the articles used CRM methods, explained CRM planning, or analysis of CRM data. Study Selection Two independent reviewers screened the retrieved literature based on title and abstract, followed by a full-text review of potentially relevant studies. Any discrepancies between reviewers were resolved through discussion or consultation with a third reviewer. Articles were included if they met the predefined inclusion criteria and provided pertinent information related to the research purpose. Studies were excluded if they did not meet the inclusion criteria, were duplicates, or lacked relevant data. https://doi.org/10.37433/aad.v5i2.357 Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 138 Data Extraction A standardized data extraction form was utilized to extract key information from included studies, such as author information, publication year, study design, analysis methods, interventions/exposures, research stimuli, research setting. The methodological quality and risk of bias of included studies were assessed using the Critical Appraisal Skills Programme Systematic Review Checklist (CASP, 2018) by two independent reviewers. Discrepancies were resolved through discussion or involvement of a third reviewer. Synthesis of Results and Reporting An a priori qualitative narrative synthesis was conducted to deductively summarize the findings from the included studies within the framework of the two study objectives. This deductive coding approach matched the purpose of this research as recommended in qualitative coding procedures (Silverman, 2019). The synthesis methods in this paper adhere to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to ensure transparency and comprehensive reporting of the review process (Page et al., 2021). Findings RO1: Describe the planning process for conducting a CRM study. CRM technologies allow for measurement in self-reported shifts in message processes using electronic technologies (Biocca et al., 1994). CRM has been used in public speaking, advertising, film and television, political and health communication (Biocca et al., 1994), and agricultural communication (LaGrande et al., 2021; Opat et al., 2022; Tarpley et al., 2020). Research designs within this method can use experimental, survey and quasi-experimental, time-series, focus groups, and coding content and behavior designs (Biocca et al., 1994). Further descriptions of type of research design can be seen in Table 1. The majority of previously published work, regardless of the type of research design, has focused on in-person use of CRM, but CRM technology providers have recently started introducing online CRM options (Paull, 2023). https://doi.org/10.37433/aad.v5i2.357 Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 139 Table 1 Research Designs using CRM with Design Considerations and Previous Studies Design Considerations Previous Studies Experimental Experimental design is the most used design in CRM research and has been used with standard between- subject and within subject factorial design. Can use CRM to substitute for self-report measures. Biocca & David, 1990a; Biocca et al., 1987; Caspi et al., 2019; Jennings et al., 2021; Jeon & Lang, 2021; Keppel, 1982; Reeves et al., 1983; Reeves, 1984; Rossiter & Thornton, 2004; Thorson & Reeves, 1985; West & Biocca, 1992; Saks et al, 2016; Wöllner & Auhagen, 2008. Quasi experimental and survey Television research typically selects this design. Within this design, CRM is used as an extension of a survey with survey data collected within the CRM software or in a post/pre survey. In in-person settings it is difficult to obtain representative samples in this design, but the online CRM option is ideal for this design. Baggley, 1987; Baggaley, 1986a, 1986b; Biocca & David, 1990a; Beville, 1985; D'Ambrosio, 2019; Millard, 1989; Papakonstantinou et al., 2002; Philport, 1980; West & Bocca, 1992; West et al., 1991. Time series This design works well for public opinion on scientific issues that changes over time in response to current events or legislation. It can also be used when purposeful intervention is used by the researcher to measure the effectiveness of an intervention over time. It can be used in a single group with multiple interventions, multiple groups with a single intervention, or multiple groups with multiple interventions. Glass et al., 1975 (for more on time-series design generally); Palmgren, 1996; Wöllner & Auhagen, 2008; Focus groups Advertising and marketing studies have historically preferred this design, but in recent years science communication researchers have also used this design. It allows for mixed method data collection with quantitative data from CRM technology and qualitative data from focus group discussion about CRM results from the moment-to-moment data collection. This design allows for collection of individual thoughts on dynamic content to be collected before group discussion may alter individual responses. This design also allows for deeper understanding of the meaning of CRM data collection and offers participants a chance to interpret meaning rather than researchers relying solely on quantitative data analysis to decipher. Baggaley, 1987; Boussalis & Coan, 2021; Chandler et al., 2022; Cummins et al., 2018; Hughes, 2015; Schwarzkopf, 2021. Coding content and behavior CRM allows for quick coding of messages, context, and interaction from live or recorded content. Researchers can use the tool to record codes from multiple coders simultaneously to better understand participants behaviors and responses to content (when participants are the subject being viewed) or codes for variables like camera distance, presence of controversial images, changes in speech patterns, visual ques. Biocca & David, 1990b; Palmer & Cunningham, 1987; Waddell, & Williamson, 2017. https://doi.org/10.37433/aad.v5i2.357 Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 140 Choosing a research stimulus is one of the most important steps in collecting CRM data (Lawson et al., 2020). CRM can be used to evaluate any dynamic content from a persuasive campaign video, a political debate, a teaching presentation, or a podcast. Table 2 provides a list of recommended stimuli and unique considerations for each type that arose from the themes in the literature. Once a stimulus is selected, researchers must choose a response prompt. Response prompts may include the trustworthiness of the content, emotion, or the persuasiveness of the message. Participants can only respond to one option at a time, so it is suggested that researchers limit the options they ask participants to respond to in a study. Participants respond to the prompt on a continuous scale with responses ranging from 0 to 100. Table 2 Recommended Stimulus Types for CRM and Considerations for Each Stimulus Unique Considerations Prior Studies Using Radio or podcasts CRM use began with radio content, but modern-day researchers have used it with podcasts. While radio content and podcasts can be listened to in a group setting, it is typically content people listen to alone, so an in-person CRM setting may be unnatural for this content; virtual CRM may provide a more natural context for these stimuli. Bolce et al., 1996; Opat et al., 2022. Live events, public speeches, or lectures Any live event can be used with in-person CRM, although outside events can be challenging for hardware with difficulty viewing screens on dials and monitors and potential for adverse weather. With a live event as stimuli, it is important to record the event so you have the content for reference later and to make sure you have a precise starting point for when dial use began with participants. Alternatively live events can be recorded to use as video stimuli in in-person or virtual CRM. Biocca & David, 1990a; Cummins et al., 2018; Hughes, 2015; West & Biocca, 1992. Videos Videos can range from lengthy heavily produced content like full length documentaries to social media videos. CRM software has a minimum time of 30 seconds, so shorter social media videos need to be combined for use. The type of video content will depend on if it is most appropriate for use in a group in- person CRM setting or an individual CRM setting, which is more typical and effective in a virtual CRM setting. Biocca et al., 1987; Philport, 1980; LaGrande et al., 2021; Reeves et al., 1983; Reeves, 1984; Tarpley et al., 2020; Thorson & Reeves, 1985. In-person CRM CRM is a measurement tool that allows researchers to pinpoint specific perceptions of communication messages and elements of communication in real-time (Izenson, 2016). Traditionally, CRM is conducted in-person using handheld dials (Figure 1) that participants use to respond to the research stimulus. https://doi.org/10.37433/aad.v5i2.357 Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 141 Figure 1 Participant Using a Handheld Dial in An In-Person CRM Setting It is recommended to have participants practice using the dials before responding to the research prompt. For most participants, this is the first time they have used a CRM dial. Researchers should ensure the participants are comfortable using the dial and understand the prompt. For example, if the prompt is trustworthiness, the researchers should provide an example of how extremely untrustworthy and extremely trustworthy is rationalized in the study. As participants respond to the prompt, researchers should observe the participants and how they are using the dials. Researchers may need to remind participants to turn their dial continuously. If a participant is inactive, researchers may need to remove their data from the study. For a thorough step-by-step guide on how to use in-person CRM in an agricultural social science context, refer to Lawson et al. (2020). Quantitative data is automatically collected through CRM software, but to gain additional insight it is recommended that researchers pause at key moments in data collection and/or at the end of data collection to show results and discuss the reasons why participants felt the way they did about the critical moments in the data. Using a qualitative methodology for this questioning route is recommended. While in-person dial testing offers many unique benefits, limitations do exist. Two of the most burdensome limitations are the cost and participant recruitment. Software and equipment, including the console and 25 handheld dials, necessary to collect in-person CRM data will cost ~ $25,000. Each participant must use a handheld dial when participating in a CRM study so if researchers have 25 dials, only 25 people can be included in each study session. To collect data from 120 participants, the session would need to be held at least four times. Even if researchers have access to more handheld dials, it is recommended to keep the study groups a manageable size to avoid nonresponse and technical difficulties with the dials. Additional costs will be incurred to recruit participants and record and transcribe qualitative data. https://doi.org/10.37433/aad.v5i2.357 Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 142 Virtual CRM Online CRM has recently been introduced by CRM technology providers, such as DialSmith so, as such, fewer academic publications exist. However, online CRM provides researchers with more flexibility compared to in-person dial testing. Recorded video or audio is uploaded to the dial testing platform. Participants use a unique web link to view or listen to the content being tested, and participants use their own computer or tablet to indicate their moment-to-moment response. Respondents receive instructions and have an opportunity to practice rating content not related to the study. A slider appears on the screen and participants are asked to respond to the research prompt by moving the slider (Izenson, 2023). The feedback is automatically available to researchers through the reporting portal. Visuals of the data, including moment-to- moment video overlays, are available in the reporting portal but researchers can also download raw data to conduct additional analyses. The online CRM platform supports recorded audio and video files, including .mp4 and .mov files, but cannot be used to test live streaming content (Paull, 2023). Live stimuli must use in- person CRM. Since online CRM participants do not have to be physically present, a more geographically diverse sample can be obtained (Alzuhn, 2023). Researchers are not limited by the number of handheld devices when conducting online CRM so more responses can be collected over a shorter period (Alzuhn, 2023). Online CRM can also be more cost effective, particularly when researchers have a limited number of studies they wish to conduct. Costs vary depending on the type of content being tested, length of content, and number of desired participants. Costs range from $1,500 to $2,000 USD to upload the content and manage the project with an additional cost to recruit participants. One distinct disadvantage to online CRM is respondent engagement, particularly when testing lengthier content. When dial testing in- person, moderators can observe when participants are no longer engaging with the handheld device and can remind them to turn the dial or indicate when a response should be removed from data analysis. With online CRM, researchers cannot observe participants and help with focus, but a reminder is displayed on the screen to keep rating the content if a respondent stops moving the onscreen slider. DialSmith reports that online and in-person CRM results are similar in response sentiments (Paull, 2023). Reliability and Validity As with any type of research study, establishing reliability is an important step to gathering valid and useful data. Reliability is concerned with determining if the same results can be expected if measured the same way (Maier et al., 2009). This can be tested in CRM through test-retest, split-half, and parallel-test designs. Reliability can be a challenge in test-retest because of the spontaneous nature of psychological measurement within CRM studies, but some studies have obtained reliability this way. Split-half is a more common method for determining reliability in CRM studies, but parallel-test designs can also be used (Maier et al., 2009). External validity, or generalization, in CRM studies is a matter of properly designing the study regarding sampling procedures that are appropriate and representativeness of the target population. Additionally, it is important to match the measurement setting as close to a real life setting as possible and select the appropriate stimuli and data collection site. Like other experimental and quasi-experimental designs, pre-testing of CRM instruments can help https://doi.org/10.37433/aad.v5i2.357 Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 143 establish validity (Maier et al., 2009). For focus group designs, researchers should focus on appropriate sampling for their research questions and consider transferability to other settings. Verbatim transcripts, audit trails, data triangulation, member checks, and multiple coders can be used to ensure data accuracy (Lincoln & Guba, 1985). Limitations The cost of in-person and online CRM can be a significant limitation. In-person CRM requires more software and equipment, which can cost an estimated $25,000 USD. Online CRM can be a more cost efficient alternative but still comes with a cost of $2,000 USD or more depending on recruitment costs. Limited training is available for CRM data collection and analysis, which can be a limitation for researchers wanting to implement CRM research. Some CRM software companies, such as DialSmith, offer training on their specific software at an additional cost. These companies also offer consulting services to provide input on overall study design, data collection, and data analysis. CRM data is continuous, resulting in a data point per second per participant. As such, this methodology results in a large amount of data, even when working with a short stimulus and a small sample size, which can be a limitation. Researchers must be extremely organized and attentive to details to effectively manage the large amount of data and tie participant responses to stimuli. RO2: Describe how to analyze data gathered from a CRM study. Output of data from CRM technologies allows researchers to interpret in multiple ways. When collecting data in-person, you can export data as: PowerPoint, Excel, or SPSS files. When conducting virtual CRM, the same data output as in-person CRM is provided. Data visualizations are more readily available in the virtual CRM interface and, when using DialSmith, a project manager can support the data output and analysis phase. Analysis Data can be analyzed in two different ways. We have divided this into real time and reflective analysis. Real time analysis can provide immediate feedback, course correction, and/or discussion within a focus group setting. CRM allows researchers to view data in real time during collection in aggregate, by a single participant, or by a specific demographic. Summary statistics for each study question are projected on the researcher monitor throughout the data collection and immediately following. Data are saved on the designated CRM computer within the CRM software for future analysis. A researcher can choose to show the results to participants to collect their perceptions of the results and to discuss peaks and valleys in the data (Figure 2). This will offer additional insight beyond the quantitative data analysis and fits into the focus group design described in Table 1. Peaks and valleys in the data allow for visual understanding of the data through identification of high and low points. Data can be grouped by other study variables to visually determine differences (Figure 2). Depending on the research goal, some science communication projects may be able to use the real-time data analysis to adjust communication campaigns without deeper analysis. However, to publish data in a journal, reflective analysis is needed. https://doi.org/10.37433/aad.v5i2.357 Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 144 Figure 2 Example of Real Time Data Analysis Demonstrating Peaks and Valleys Note. Data outputs are visualized as peaks and valleys, with 50 being the neutral point on a 100-point scale (DialSmith, 2023). Reflective analysis offers additional understanding for research purposes. Data can be exported from the CRM software directly to SPSS statistical software or Excel in ASCII (American Standard Code for Information Interchange) file format for statistical analyses. The next steps in data analysis depend on the research questions and study design. If researchers collected survey data separately from the CRM data, they need to merge the data files and clean the data. Researchers may choose to run crosstabs or mean comparisons initially to understand the data. In the virtual CRM option, crosstabs will be run by the project manager at DialSmith. Because of the large quantity of data, researchers often chose to compress the data from 1 second increments to 5 or 10 second increments depending on the study size. Normalizing the data through Z scores is another step researchers may take before moving to deeper analysis, particularly when working to identify statistical significance of peaks and valleys in the data. Researchers may select to identify Z scores greater than 1.96 or less than -1.96 (Biocca et al., 1994), or they may choose to run independent samples t-test with an a priori established level. Once peaks and valleys are established as significant, then it is important to review the stimuli to determine what was happening at these critical moments. This will help researchers determine what is happening at these moments that may be affecting participants’ responses and to understand patterns in responses. Researchers may decide to categorize what was happening at critical moments to interpret the data. For example, if a particular word is used in all peaks, or if a particular speaker is on screen during every valley. https://doi.org/10.37433/aad.v5i2.357 Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 145 From this point, the data are handled similarly to how you would handle any other quantitative data set based on the study design and research questions. Studies have used basic descriptive statistics and mean comparisons. However, with such large data sets, deeper analysis methods are available for use and can offer deeper understanding. Beyond means and peaks and valleys, researchers can understand group differences with ANOVA. They can also create alternative series to look at standard deviation series and absolute, mean, and percentage activity series (Biocca et al., 1994). Time series analysis may include autoregressive integrated moving average (ATIMA) techniques (Box et al., 2015). Confirmatory factor analysis may also be an appropriate method as an extension of regression models; spectral or Fourier analysis may be appropriate to understand complex problems that were periodic in nature within the CRM study design (Biocca et al., 1994). For those interested in a deeper understanding of data analysis related to CRM, Biocca et al. (1994) provides formulas, examples, and strategies. Focus group designs typically combine the peak and valley analysis with qualitative data analysis methods (Williams & Moser, 2019) to compare quantitative with qualitative results in a mixed methods design (O’Cathain et al., 2010). Focus group results can offer insight into participants thought processes while viewing stimuli more than CRM results alone. Conclusions, Discussion, and Recommendations The benefits of CRM in agricultural social science research are abundant, including the depth of data collected, real-time response and visualization of the data, reduction of recall bias, and limiting group think. CRM allows for researchers to quickly collect a large amount of data that can be analyzed in a variety of ways, including audience segmentations, experimental designs, qualitative research, and self-report items (Lawson et al., 2020). Even though data can be analyzed in multiple ways, data can be viewed in real-time as participants are responding and immediately exported into visualizations that can be shared with participants or relevant stakeholder groups. When asking participants to respond to a survey question or a question posed during an interview or focus group, participants must often reflect on a past event or experience and may not remember accurately. Since CRM is collected in real-time of the participants interacting with the stimuli, the need for recall is removed. Additionally, CRM offers a way to measure messaging in a dual process way while all elements of a message are considered at the same time rather than isolating one specific component. Groupthink is often a limitation in qualitative research and occurs when one participant or a small group of participants influences others’ opinions (Janis, 1997). In CRM, you can collect individual opinions without the influence of others, but also have the option to collect qualitative data in a focus group using CRM to gain individual and group insight into specific points in the data. Participants respond to CRM individually without the influence of other participants, limiting groupthink. CRM has not been used to its full potential in agricultural social science research, but it could take science communication evaluation to the next level with quantification of perceptions and response to communication stimuli in a moment-to-moment format. Multiple opportunities exist to enhance strategic science communication in agriculture and natural resources. In- https://doi.org/10.37433/aad.v5i2.357 Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 146 person CRM use can offer real time testing of science and outreach products and allow for discussion with stakeholders on the reasons behind their views on science communication products. With the instantaneous nature of social media, CRM can render a similar environment and gather feedback. Moreover, the immediate response and course correction offers the ability to respond rapidly during a crisis or as a crisis develops. Because in-person CRM requires more equipment and a larger research team, it may not be an option for all researchers. However, online CRM has a lower entry point with costs allocated by online response and support for developing integration into surveys. This option also allows for help in the recruitment of participants. In addition to being a unique and useful research tool in science communication, CRM also provides opportunities for Extension, teaching, and leadership. Notably, CRM can be used to determine preferred teaching styles, perceptions of presenters, and response to messages used. Utilizing CRM can provide real-time insight to effective messaging, while also providing insight on how to become better teachers, presenters, and leaders. Acknowledgements This publication was possible because of support from the UF/IFAS Center for Public Issues Education in Agriculture & Natural Resources (PIE Center), https://piecenter.com References Alzuhn, E. (2023). Moment-to-moment cheat sheet: Online research vs in-person research. DIALSMITH. http://www.dialsmith.com/blog/moment-research-best-online-research/ Baggaley, J. (1987). Continual response measurement: Design and validation. Canadian Journal of Educational Communication, 16(3), 217–238. https://doi.org/10.21432/T2VG8M Baggaley, J. (1986a). Developing a televised health campaign: I. Smoking prevention. Media in Education & Development, 19, 29–43. Baggaley, J. (1986b). Developing a televised health campaign: II. Skin cancer prevention. Media in Education & Development, 19, 173–176. Beville, H. M. (1985). Audience ratings: Radio, television, and cable. Lawrence Erlbaum Associates. Biocca, F., Neuwirth, K., Oshagan, H., Zhongdang, P., & Richards, J. (1987, May). Prime-and- probe methodology: An experimental technique for studying film and television [Paper presentation]. International Communication Association, Montreal, QC, Canada. Biocca, F., David, P., & West, M. (1994). Continuous response measurement (CRM): A computerized tool for research on the cognitive processing of communication messages. https://doi.org/10.37433/aad.v5i2.357 https://piecenter.com/ http://www.dialsmith.com/blog/moment-research-best-online-research/ https://doi.org/10.21432/T2VG8M Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 147 In A. Lang (Ed.), Measuring psychological responses to media messages (pp. 15–65). Lawrence Erlbaum Associates. Biocca, F., & David, P. (1990a). How camera distance affects the perception of candidates during a presidential debate [Unpublished manuscript]. Center for Research in Journalism and Mass Communication, University of North Carolina at Chapel Hill. Biocca, F., & David, P. (1990b, May). Micro-shifts in audience opinions: A second-by-second analysis of the first 1988 presidential debate [Paper presentation]. International Communication Association, Dublin, Ireland. Bolce, L., De Maio, G., & Muzzio, D. (1996). Dial-in democracy: Talk radio and the 1994 election. Political Science Quarterly, 111(3), 457–481. https://doi.org/10.2307/2151971 Boussalis, C., & Coan, T. G. (2021). Facing the electorate: Computational approaches to the study of nonverbal communication and voter impression formation. Political Communication, 38(1-2), 75–97. https://doi.org/10.1080/10584609.2020.1784327 Box, G. E., Jenkins, G. M., Reinsel, G. C., & Ljung, G. M. (2015). Time series analysis: forecasting and control (5th ed.). John Wiley & Sons. Caspi, A., Bogler, R., & Tzuman, O. (2019). “Judging a book by its cover”: The dominance of delivery over content when perceiving charisma. Group & Organization Management, 44(6), 1067–1098. https://doi.org/10.1177/1059601119835982 Chandler, R., Ross, H., & Guillaume, D. (2022). Innovative perception analysis of HIV prevention messaging for black women in college: A proof of concept study. BMC Public Health, 22(1), 1255. https://doi.org/10.1186/s12889-022-13564-4 Critical Appraisal Skills Programme. (2018). CASP Checklists: Critical Appraisal Checklists. CASP. https://casp-uk.net/casp-tools-checklists Cummins, R. G., Smith, D. W., Callison, C., & Mukhtar, S. (2018). Using continuous response assessment to evaluate the effectiveness of extension education products. The Journal of Extension, 56(3), 26. https://tigerprints.clemson.edu/joe/vol56/iss3/26 D'Ambrosio, L. (2019, December 15). Understanding the adoption of and education about new auto technologies among older adults. ROSAP. https://rosap.ntl.bts.gov/view/dot/43817 De Neys, W., & Pennycook, G. (2019). Logic, fast and slow: Advances in dual process theorizing. Current Directions in Psychological Science, 28(5), 503-509. https://doi.org/10.1177/0963721419855658 DialSmith. (2023). Our technology. https://www.dialsmith.com/technology/ https://doi.org/10.37433/aad.v5i2.357 https://doi.org/10.2307/2151971 https://doi.org/10.1080/10584609.2020.1784327 https://doi.org/10.1177/1059601119835982 https://doi.org/10.1186/s12889-022-13564-4 https://casp-uk.net/casp-tools-checklists https://tigerprints.clemson.edu/joe/vol56/iss3/26 https://rosap.ntl.bts.gov/view/dot/43817 https://doi.org/10.1177/0963721419855658 https://www.dialsmith.com/technology/ Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 148 Glass, G. V., Willson, V. L., and Gottman, J. M. (1975). Design and Analysis of Time Series Experiments. Colorado Associated University Press. Grayot, J. D. (2020). Dual process theories in behavioral economics and neuroeconomics: A critical review. Review of Philosophy and Psychology, 11(1), 105–136. https://doi.org/10.1007/s13164-019-00446-9 Hamel, L., Kearney, A., Kirzinger, A., Lopes, L., Muñana, C., & Brodie, M. (2020, September 10). KFF health tracking poll - September 2020: Top issues in 2020 election, the role of misinformation, and views on a potential coronavirus vaccine. KFF. https://www.kff.org/coronavirus-covid-19/report/kff-health-tracking-poll-september- 2020/ Hughes, S. R. (2015). Moving the needle: A comparative analysis of message reception during televised presidential debates [Doctoral dissertation, Texas Tech University]. Texas Tech University Libraries. http://hdl.handle.net/2346/66113 Izenson, B. (2016). Your memory stinks: How flawed recall & memory bias pollute market research and what can be done about it. DIALSMITH. https://www.dialsmith.com/blog/flawed-recall-memory-bias-pollute-market-research- can-done/ Izenson, B. (2023). What is Online Dial Testing? DIALSMITH. https://www.dialsmith.com/blog/what-is-online-dial-testing/ Janis, I. L. (1997). Groupthink. In R. P. Vecchio (Ed.), Leadership: Understanding the dynamics of power and influence in organizations (pp. 163–176). University of Notre Dame Press. https://psycnet.apa.org/record/1997-36918-010 Jennings, F. J., Allen, M. W., & Le Vu Phuong, T. (2021). More plastic than fish: Partisan responses to an advocacy video opposing single-use plastics. Environmental Communication, 15(2), 218–234. https://doi.org/10.1080/17524032.2020.1819363 Jeon, Y. A., & Lang, A. (2021). The vicious cycle of stressors, food advertising and overeating: How stigmatizing anti-obesity PSAs, which precede food commercials, influence subsequent implicit and explicit attitudes toward high-and low-calorie food. Media Psychology, 24(5), 606–636. https://www.doi.org/10.1080/15213269.2020.1773854 Juska, J. M. (2021). Integrated marketing communication: Advertising and promotion in a digital world (2nd ed.). Routledge. https://doi.org/10.4324/9780367443382 Kenen, J., & Roubein, R. (2020, March 31). Why America is scared and confused: Even the experts are getting it wrong. POLITICO. https://www.politico.com/news/2020/03/31/experts-coronavirus-cdc-158313 Keppel, G. (1982). Design and analysis: A researcher's handbook. Prentice Hall. https://doi.org/10.37433/aad.v5i2.357 https://doi.org/10.1007/s13164-019-00446-9 https://www.kff.org/coronavirus-covid-19/report/kff-health-tracking-poll-september-2020/ https://www.kff.org/coronavirus-covid-19/report/kff-health-tracking-poll-september-2020/ http://hdl.handle.net/2346/66113 https://www.dialsmith.com/blog/flawed-recall-memory-bias-pollute-market-research-can-done/ https://www.dialsmith.com/blog/flawed-recall-memory-bias-pollute-market-research-can-done/ https://www.dialsmith.com/blog/what-is-online-dial-testing/ https://psycnet.apa.org/record/1997-36918-010 https://doi.org/10.1080/17524032.2020.1819363 https://www.doi.org/10.1080/15213269.2020.1773854 https://doi.org/10.4324/9780367443382 https://www.politico.com/news/2020/03/31/experts-coronavirus-cdc-158313 Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 149 LaGrande, L. E., Meyers, C., Cummins, G. R., & Baker, M. (2021). A moment-to-moment analysis of trust in agricultural messages. Journal of Applied Communications,105(2), 2. https://doi.org/10.4148/1051-0834.2375 Lawson, C., Fischer, L., LaGrande, L., & Opat, K. (2020). Do touch that dial: A guide to continuous response measurement in agricultural communications. Journal of Applied Communications, 104(3), 1–18. https://doi.org/10.4148/1051-0834.2333 Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic Inquiry. Sage Publications, Inc. Mike, M. R., Rampold, S. D., Telg, R. W., & Lindsey, A. B. (2020). Utilizing extension as a resource in disaster response: Florida extension’s communication efforts during the 2017 hurricane season. Journal of Applied Communications, (104)1, 1–13. https://doi.org/10.4148/1051-0834.2308 Maier, J., Maier, M, Maurer, M., Reinemann, C., & Meyer, V. (2009). Real time response measurement in the social sciences: Methodological perspectives and applications. Peter Lang. Millard, W. (1989). Research using the Millard System (Televac) [Research report]. W. J. Millard. O’Cathain, A., Murphy, E., & Nicholl, J. (2010). Three techniques for integrating data in mixed methods studies. bmj, 341. https://doi.org/10.1136/bmj.c4587 Opat, K., Magness, H., & Irlbeck, E. (2018). Blue Bell's Facebook posts and responses during the 2015 Listeria crisis: A Case Study. Journal of Applied Communications, (102)4, 1–16. https://doi.org/10.4148/1051-0834.2232 Opat, K., Irlbeck, E., Cummins, R. G., Li, N., & Boren-Alpizar, A. E. (2022). Get the story straight: Comparing narrative and logical-scientific communication to capture Gen Z’s interest in science podcasts. Journal of Radio & Audio Media, 1–19. https://doi.org/10.1080/19376529.2022.2145481 Page, M. J., McKenzie, J. E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., McGuiness, L. A., Stewart, L. A., Thomas, J., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. bmj 372(71). https://www.doi.org/10.1136/bmj.n71 Palmgren, J. (1996). Analysis of longitudinal data. Statistics in Medicine, 15(11), 123– 1232. https://doi.org/10.1002/(SICI)1097-0258(19960615)15:11<1231::AID- SIM282>3.0.CO;2-Z https://doi.org/10.37433/aad.v5i2.357 https://doi.org/10.4148/1051-0834.2375 https://doi.org/10.4148/1051-0834.2333 https://doi.org/10.4148/1051-0834.2308 https://doi.org/10.1136/bmj.c4587 https://doi.org/10.4148/1051-0834.2232 https://doi.org/10.1080/19376529.2022.2145481 https://www.doi.org/10.1136/bmj.n71 https://doi.org/10.1002/(SICI)1097-0258(19960615)15:11%3C1231::AID-SIM282%3E3.0.CO;2-Z https://doi.org/10.1002/(SICI)1097-0258(19960615)15:11%3C1231::AID-SIM282%3E3.0.CO;2-Z Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 150 Palmer, M., & Cunningham, R. (1987). Computerizing the collection and coding of communication interaction data: The AIDE system [Unpublished manuscript]. Communication Arts Dept., University of Wisconsin at Madison. Papakonstantinou, E., Hargrove, J. L., Huang, C., Crawley, C. C., Canolty, N. L. (2002). Assessment of perceptions of nutrition knowledge and disease using a group interactive system: The Perception Analyzer®. Journal of the American Dietetic Association, 102(11), 1663–1668, https://doi.org/10.1016/S0002-8223(02)90354-8 Paull, A. (2023). Online dial testing FAQ. DIALSMITH. https://www.dialsmith.com/blog/online- dial-testing-faq-frequently-asked-questions/ Percy, L. (2023). Strategic integrated marketing communications (4th ed.). Routledge. https://doi.org/10.4324/9781003169635 Philport, J. (1980). The psychology of viewer program evaluation. In J. Anderson (Ed.), Proceedings of the 1980 technical conference on qualitative television ratings (pp. B1– B17). Corporation for Public Broadcasting. Pollard, M. S., & Davis, L. M. (2021). Decline in trust in the Centers for Disease Control and Prevention during the COVID-19 pandemic. RAND Corporation. https://www.rand.org/pubs/research_reports/RRA308-12.html Reeves, B. (1984, June). Attention to television: Psychological theories and chronometric measures (pp. 251–279). University of Wisconsin. Reeves, B., Rothschild, M., & Thorson, E. (1983). Evaluation of the Tell-Back audience response system [Research Report for ABC]. Mass Communication Research Center, University of Wisconsin-Madison. Rossiter, J. R., & Thornton, J. (2004). Fear-pattern analysis supports the fear-drive model for antispeeding road-safety TV ads. Psychology & Marketing, 21(11), 945–960. https://psycnet.apa.org/doi/10.1002/mar.20042 Saks, J., Compton, J. L., Hopkins, A., & El Damanhoury, K. (2016). Dialed in: Continuous response measures in televised political debates and their effect on viewers. Journal of Broadcasting & Electronic Media, 60(2), 231–247. http://dx.doi.org/10.1080/08838151.2016.1164164 Schwarzkopf, S. (2021). Ten little jurors in the training camp: a genealogy of audience simulation. Journal of Cultural Economy, 14(6), 732–749. https://doi.org/10.1080/17530350.2021.1882535 https://doi.org/10.37433/aad.v5i2.357 https://doi.org/10.1016/S0002-8223(02)90354-8 https://www.dialsmith.com/blog/online-dial-testing-faq-frequently-asked-questions/ https://www.dialsmith.com/blog/online-dial-testing-faq-frequently-asked-questions/ https://doi.org/10.4324/9781003169635 https://www.rand.org/pubs/research_reports/RRA308-12.html https://psycnet.apa.org/doi/10.1002/mar.20042 http://dx.doi.org/10.1080/08838151.2016.1164164 https://doi.org/10.1080/17530350.2021.1882535 Baker et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i2.357 151 Sellnow, T. L., Parker, J. S., Sellnow, D. D., Littlefield, R. S., Helsel, E. M., Getchell, M. C., Smith, J. M., & Merrill, S. C. (2017). Improving biosecurity through instructional crisis communication: Lessons learned from the PEDv outbreak. Journal of Applied Communications, 101(4), 1–15. https://doi.org/10.4148/1051-0834.1298 Silverman, D. (2019). Interpreting qualitative data (6th ed.). SAGE Publications. Steede, G. M., Meyers, C., Li, N., Irlbeck, E., & Gearhart, S. (2018). A sentiment and content analysis of Twitter content regarding the use of antibiotics in livestock. Journal of Applied Communications, 102(4), 1–16. https://doi.org/10.4148/1051-0834.2225 Tarpley, T. G., Fischer, L. M., Steede, G. M., Cummins, R. G., & McCord, A. (2020). How much transparency is too much? A moment-to-moment analysis of viewer comfort in response to animal slaughter videos. Journal of Applied Communications, 104(2), 6. https://doi.org/10.4148/1051-0834.2302 Thorson, E., & Reeves, B. (1985). Effects of over-time measures of viewer liking and activity during programs and commercials on memory for commercials. In R. Lutz (Ed.), Advances in consumer research (pp. 549-553). Association of Consumer Research. Villarreal, R. (2010). Integrated marketing communication strategy. Wiley International Encyclopedia of Marketing. https://doi.org/10.1002/9781444316568.wiem01027 Waddell, G., & Williamon, A. (2017). Measuring the audience. In Lee, S. (Ed.), Scholarly research for musicians. (pp. 148-155). Routledge. West, M., & Biocca, F. (1992). "What if your wife were murdered": Audience responses to a verbal gaffe in the 1988 Los Angeles presidential debates [Paper presentation]. The Annual Conference of the American Association for Public Opinion Research, St. Petersburg, FL. West, M., Biocca, F., & David, P. (1991). "You're no Jack Kennedy": Audience responses to a verbal barb in the 1988 Omaha vice-presidential debates [Paper presentation]. The Annual Conference of the American Association for Public Opinion Research, Phoenix, AZ. Williams, M., & Moser, T. (2019). The art of coding and thematic exploration in qualitative research. International Management Review, 15(1), 45-55. http://www.imrjournal.org/uploads/1/4/2/8/14286482/imr-v15n1art4.pdf Wöllner, C., & Auhagen, W. (2008). Perceiving conductors' expressive gestures from different visual perspectives. An exploratory continuous response study. Music Perception, 26(2), 129-143. https://doi.org/10.1525/mp.2008.26.2.129 © 2024 by authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/). https://doi.org/10.37433/aad.v5i2.357 https://doi.org/10.4148/1051-0834.1298 https://doi.org/10.4148/1051-0834.2225 https://doi.org/10.4148/1051-0834.2302 https://doi.org/10.1002/9781444316568.wiem01027 http://www.imrjournal.org/uploads/1/4/2/8/14286482/imr-v15n1art4.pdf https://doi.org/10.1525/mp.2008.26.2.129