Research Article 93 Ved ABSTRACT Introduction: Increasing reinforcement received from cannabis-free activities, relative to reinforcement from cannabis-related activities, is one way to reduce harmful cannabis use. Thus, accurate measurement of cannabis reinforcement is important. Using convergent mixed methods, we developed the Adolescent Reinforcement Survey Schedule-Cannabis Use Version (ARSS-CUV). ARSS-CUV, adapted from the alcohol use version, measures cannabis reinforcement by asking individuals how frequently they engaged in, and how much they enjoyed, different activities when using and not using cannabis. Method: Young adults (N = 65; Mage = 20.4 years [SD = 1.8]) completed measures of cannabis use, the ARSS-CUV, and provided feedback on included activities, via focus groups. Following Standards for Educational and Psychological Testing framework, this study examined evidence of measurement validity based on item content. Results: Quantitative findings revealed that peer interactions were the most reinforcing activities, whereas activities related to family were least reinforcing. Qualitative findings indicated some confusion with question wording. Participants also indicated the importance of environmental context when using cannabis and noted who they use cannabis with may be more important than the activity they are doing. Changes were made to survey flow and response choices after participant feedback. Conclusions: ARSS- CUV includes revisions in activities solicited and response format. The revised ARSS-CUV provides opportunities to advance measurement of an important construct (i.e., reinforcement) in the study of cannabis use. Psychometric properties of the ARSS-CUV across different populations and contexts of use (e.g., polysubstance use) should be examined. Key words: = marijuana; behavioral economics; college students; substance-free reinforcement; qualitative Cannabis use is most prevalent among young adults between 18 and 25 years of age (34.5%) (Substance Abuse and Mental Health Services Administration, 2021). Past year cannabis use is reported by 44% of college students, and by 43% of young adults who are not in college (Schulenberg et al., 2021). Heavy cannabis use is a critical public health concern as it is associated with numerous deleterious outcomes, including increased risk for cannabis use disorder (CUD) (Cerdá et al., 2020), negative effects on individuals’ cardiovascular health (Cohen et al., 2019), and motor vehicle crashes (Rogeberg & Elvik, 2016). With the rapid increase in legalization of medical and recreational cannabis in concordance with shifting normative Nioud M. Gebru1, Tyler G. James2, Ricarda Foxx3, Michelle Castro3, Ali M. Yurasek3 1Center for Alcohol and Addiction Studies, Department of Behavioral and Social Sciences, Brown University School of Public Health, Providence, RI, USA 2 Department of Family Medicine & Center for Disability Health and Wellness, University of Michigan Medical School, Ann Arbor, MI, USA. 3 Department of Health Education & Behavior; Center for Behavioral Economic Health Research; University of Florida, Gainesville, FL, USA Cannabis 2024, Volume 7 (2) © Author(s) 2024 researchmj.org 10.26828/cannabis/2023/000185 Measuring Cannabis Reinforcement among Young Adults: A Mixed Methods Examination Corresponding Author: Nioud (Neo) Gebru, Ph.D., Brown University, 121 S. Main St., Box G-S121-3, Providence, RI 02912 USA, Phone: 401-863-1000, Email: neo_gebru@brown.edu Cannabis, A Publication of the Research Society on Marijuana 94 perceptions (Wallace et al., 2020), there is a need to identify factors associated with, and intervene upon, harmful cannabis use among young adults. Behavioral economic evidence suggests increasing reinforcement received from drug-free activities, relative to reinforcement from drug-related activities, is one way to reduce harmful cannabis use (Leventhal et al., 2015). Thus, accurately quantifying reinforcement gained from cannabis use is of critical importance, yet measurement of cannabis reinforcement remains understudied. Behavioral economic theory posits that cannabis use is most likely to occur when it is readily available, inexpensive, and when there are few cannabis-free alternative reinforcers to compete with its use (Higgins et al., 2004). Cannabis use generally provides an immediate reinforcement (e.g., anxiety reduction, euphoria), whereas many cannabis-free activities (e.g., school or work attendance, chores) provide delayed reinforcement (e.g., graduation, promotion) (Bickel et al., 2014). Individuals who use cannabis heavily may under engage in activities with the potential to compete with cannabis use (such as exercising, studying, or working) as the benefits of these activities are often delayed. Previously, the Adolescent Reinforcement Survey Schedule – Substance Use Version (ARSS- SUV) has been used to reliably capture reinforcement from substance-related activities (Murphy et al., 2005). The ARSS-SUV seeks to measure the two core components of reinforcement – strength of the reinforcer (i.e., magnitude) and rate of reinforcement (i.e., frequency) (Acuff et al., 2019). Specifically, the ARSS-SUV asks participants to indicate the frequency with which they engage in a variety of different activities (e.g., go to parties with friends) with and without using alcohol or drugs and how much they enjoyed each activity with and without using alcohol or drugs. Frequency and enjoyment ratings are multiplied to compute substance- related and substance-free reinforcement, from which a reinforcement ratio can also be calculated (Acuff et al., 2019). In addition to a total score of reinforcement, the ARSS-SUV can also be broken down into sub-scales of activities, including peer- related activities, dating, and chores. Reinforcement ratios from ARSS-SUV have been related to several substance use outcomes (Acuff et al., 2019) including past year alcohol use and alcohol-related negative consequences (Hallgren et al., 2016). The ARSS-SUV allows for a broad assessment of activity engagement and enjoyment for non- specified substance use (i.e., use of alcohol or drugs) to produce a substance-involved versus substance-free reinforcement ratio. Therefore, it does not allow for the identification of substance- specific substance-involved and substance-free activities, making it difficult to isolate what activities (and in turn, reinforcement) are related to a specific drug (e.g., cannabis) use. Hallgren and colleagues (2016) adapted the ARSS to assess alcohol-only reinforcement (ARSS-AUV), which revealed a unique factor structure compared to the ARSS-SUV. Their study also found that the nature of the activity is less predictive of frequency of alcohol use than with whom the activity is performed (i.e., context), highlighting the importance of examining context and substance-specific reinforcement (Hallgren et al., 2016). A cannabis adaptation of the ARSS-SUV is also needed to improve cannabis intervention efforts. For instance, Dennhart and colleagues (2015) tested a brief intervention to reduce substance use and found that reinforcement ratio at baseline was predictive of changes in cannabis use at 6-month follow-up. Specifically, alcohol using college students who also used cannabis and had high reinforcement from substances were more likely to reduce their substance use and were more responsive to the intervention (Dennhardt et al., 2015). However, because reinforcement was measured via the ARSS-SUV, it is challenging to disaggregate whether reported substance-related reinforcement at baseline was from alcohol, cannabis, or use of some other substance. Adapting the ARSS-SUV to cannabis by only changing substances (i.e., replacing “alcohol and drugs” with “cannabis” only) without further refinement may not be sufficient. Compared to measuring alcohol, many aspects of cannabis complicate measurement of cannabis-specific reinforcement, including lack of standards (e.g., hits, joints, grams). For instance, one study found that the marijuana purchase task, which had been adapted from the alcohol purchase task, had differential latent factor structure when compared, in part due to such measurement issues (Aston et al., 2017). Measurement and Measuring Cannabis Reinforcement 95 validity issues are further complicated by several possible routes of administration (e.g., vaping, edibles, spotting) and the illicit nature of cannabis at the federal level in the U.S. Thus, a cannabis-specific reinforcement measure designed with such considerations for the contextual factors associated with cannabis and its use is needed. Reinforcement gained from cannabis use may also vary from reinforcement from using other substances, due to differences in cannabis use contexts. For example, cannabis use may involve different types of social networks, be more or less available and accessible, and be used in different locations (Meldrum & Leimberg, 2018; Phillips et al., 2020). Hence, activities that young adults particularly engage in and enjoy (i.e., find reinforcing) with and without using cannabis might not be fully captured by current versions of the ARSS, limiting measurement of, and our understanding and ability to intervene on, cannabis reinforcement. Further, it is important to examine the factor structure of the ARSS subscales, to ensure that each subscale is capturing relevant activities. Enhancing measurement of cannabis reinforcement can help in developing and implementing more tailored interventions aimed at reducing harmful cannabis use. Thus, we sought to develop the ARSS-Cannabis Use Version (ARSS-CUV) using a convergent mixed methods approach. This mixed methods study was based on the Standards for Educational and Psychological Testing, which provides a framework for collecting evidence of validity (American Educational Research Association et al., 2014). This study was focused primarily on evidence of validity based on content, including the wording and response format of items. One primary concern when measuring reinforcing activities is ensuring the activities are relevant to the intended population. Failing to capture relevant activities leads to a phenomenon known as “construct underrepresentation,” which is one of the biggest threats to validity (American Educational Research Association et al., 2014, p. 12). Therefore, the purpose of this study was to begin the process of testing and revising a Preliminary ARSS-CUV to be cannabis-use specific by aligning ARSS-SUV content to activities of interest to young adults who use cannabis. METHODS Study Design This study used a convergent mixed methods design, in which qualitative and quantitative data were collected to gain a broader understanding of a specific phenomenon (Figure 1). Consistent with guidance in mixed methods research studies (Fetters, 2020; Fetters & Molina-Azorin, 2017), we integrated data on multiple dimensions. First, with the intention to merge the two databases to capture the reinforcing activities of young adults who use cannabis, we used the Preliminary ARSS- CUV to guide interviews and match constructs asked in both data collection segments (Moseholm & Fetters, 2017). Secondly, we used an identical sample where the participants were in both the quantitative and qualitative aims (Fetters, 2020). Recruitment and Screening Procedures Participants were 65 young adults between ages of 18 and 26 years old who reported using cannabis at least 3 times in the past month. Participants were recruited through printed flyers posted at public locations (e.g., restaurants, stores) near a large university in north central Florida. Interested participants contacted the research team via phone or email and were provided with an overview of study procedures and a link to the screening survey. The online screening survey included an informed consent form followed by questions to determine eligibility. Eligible participants were then scheduled for a focus group session and were sent a link to an online survey to be completed before the focus group. This survey included the informed consent and collected data on drug use and related activities, including the Preliminary ARSS-CUV. All participants completed the Preliminary ARSS- CUV before the focus group session. At the end of the focus group, participants were compensated with a $30 VisaTM card. All procedures were approved by the university’s institutional review board. Cannabis, A Publication of the Research Society on Marijuana 96 Figure 1. Convergent Mixed Methods Procedural Diagram of the Current Study. Quantitative Measures Screening Survey. Participants were presented with questions on age, gender, race, employment status, and income. Participants were also asked to report how many days they had used cannabis in the past month. Preliminary Adult Reinforcement Survey Schedule – Cannabis Use Version (ARSS-CUV). A preliminary, modified version of a 26-item ARSS- SUV (Murphy et al., 2005) was used to determine past-month engagement and enjoyment from various cannabis-free and cannabis-related activities. Modifications made to the ARSS-SUV were minimal, primarily asking about participant engagement in activities with or without cannabis (instead of alcohol or drugs). Activity frequency and enjoyment ratings were assessed using 5- point Likert Scales ranging from 0-4. Frequency ratings ranged from 0 (zero times per week) to 4 (more than once a day), and enjoyment ratings ranged from 0 (unpleasant or neutral) to 4 (extremely pleasant). A cross-product score, reflecting reinforcement derived from an activity with or without cannabis, is calculated by multiplying the frequency and enjoyment ratings for each activity (Correia et al., 2003). The relative reinforcing value of cannabis use, or the reinforcement ratio (R-ratio) was then computed [(cannabis-related total)/(cannabis-free total + cannabis-related total)]. DSM-5 Cannabis Use Disorder (CUD) Checklist. A categorical variable (i.e., none, mild, moderate, severe) was created indicating CUD severity based on participant responses indicating experiencing any DSM-5 CUD related symptoms in the past year (American Psychiatric Association, 2013). Focus Group Procedures Nine in person focus groups were conducted in a conference room located within a research space. There were 5-10 people per focus group (M = 7.2), and sessions lasted approximately 75 minutes. Focus groups were moderated by the principal investigator and trained research assistants, and a trained note-taker was present during all sessions. Data Collection • Survey of cannabis and other drug use, Preliminary ARSS- Cannabis Use Version, and demographic measures. Data Collection • Focus groups on Preliminary ARSS- Cannabis Use Version revisions, and activities and social factors related cannabis use. Data Analysis • Classical Test Theory statistics, including reinforcement ratio and cannabis-free cross- product means and correlations. Data Analysis • Conventional content analysis of reinforcing activities related to cannabis use and ARSS questionnaire format revisions. Integrative Data Analysis • Weaving. Quantitative Qualitative Revised ARSS- Cannabis Use Version Integration: • Matching quantitative and qualitative data collection topics. • Connecting samples across both quantitative and qualitative aims. Integration: • Merging quantitative and qualitative findings. • Explaining quantitative results using qualitative findings. Measuring Cannabis Reinforcement 97 At the start of the focus group, participants were instructed to refer to each other by the numbered name tags at their seats and that the session would be audio recorded. They were informed that the purpose of the focus group was to provide feedback about a questionnaire assessing cannabis use and activities young adults do while using cannabis. Participants received a blank copy of the Preliminary ARSS-CUV and the Pleasant Events Schedule (PES; Correia et al., 2002) which were used to stimulate discussion. In the first hour, participants were asked to look at the Preliminary ARSS-CUV and circle activities people their age commonly engage in and cross out activities they thought people their age are less likely to engage in. They then provided feedback on how to adapt the measure to be more relevant to cannabis use. After a 5–10-minute break, the remaining 30 minutes were spent discussing how other drugs impact cannabis use, enjoyment of activities pre- cannabis use versus post-cannabis use, and any other relevant activities that may not have come up in the discussion. Data Analysis Following the convergent mixed methods design, data analysis was conducted prior to merging results to develop mixed methods meta- inferences. Quantitative analysis. ARSS-CUV indices were scored using R statistical software (version 4.2.2; R Core Team 2022). Cannabis-related and cannabis-free reinforcement was calculated by multiplying frequency and enjoyment for each activity, from which reinforcement ratios were calculated. Data were cleaned in IBM SPSS Statistics (Version 25) and analyzed in SAS version 9.4 (SAS Institute Inc). Descriptive statistics of the cannabis reinforcement ratio were examined. As a test of internal structure, we originally intended to conduct an Exploratory Factor Analysis (EFA) to determine if the factor structure of the preliminary measure was similar to that reported by Murphy and colleagues (2005). However, due to the small sample size, the EFA solution did not converge. Eigenvalue analysis indicated a five-factor solution may be appropriate for the data. Given the five factors sub-scales identified for the ARSS-SUV (Murphy et al., 2005), the factor structure for the Preliminary ARSS-CUV was expected to be similar. Thus, internal consistency reliability estimates and correlations between factors are reported. ARSS-CUV factor scores were calculated based on the average response scores for non-missing data (i.e., if a factor consisted of 3 items but only 2 were completed, the demonimator would be a 2; Holmes et al., 1987). Qualitative analysis. Transcripts of focus group recordings were managed and analyzed using MAXQDA (VERBI Software, Berlin, Germany). Analysis focused on the addition or revision of items for content representation on the ARSS-CUV, and format considerations of the measure. The first two focus group transcripts were free coded by the second and last author. Coding decisions were discussed to develop new codes and resolve discrepancies through consensus (Saldaña, 2015). Then, the two coders separately coded the remaining seven focus group transcripts, before the coded transcripts were merged in MAXQDA. Lastly, the second author audited the transcripts to ensure the code scheme was applied consistently. Mixed methods. Mixed methods data analysis was facilitated by assessing quantitative and qualitative results that were matched on constructs. Instead of reporting results separately, we report findings from both data sources based on the similarity of constructs in an integration process known as weaving (Fetters et al., 2013). RESULTS Sample Characteristics Participants were on average 20.4 years old (SD = 1.77), with the vast majority (96.9%) being college students (Table 1). Approximately half were White (49.2%), and slightly over half were women (55.4%). Participants used cannabis an average of 19.9 days per month. Of the participants who met the symptom count criteria for DSM-5 CUD diagnosis (83.1%), almost half were classified as mild CUD. Evidence of Validity based on Internal Structure Estimates of mean item responses, internal consistency reliability, and R-ratio correlations are provided in Table 2. Cannabis, A Publication of the Research Society on Marijuana 98 Table 1. Demographic Characteristics. Characteristic % (n) Age in years, M (SD), range 20.40 (1.77) 18 – 26 Race White 49.23% (32) Hispanic or Latino 21.54% (14) Asian 16.92% (11) Black or African American 1.54% (1) Other 1.54% (1) Biracial or multiracial 9.23% (6) Gender Woman 55.38% (36) Man 43.08% (28) Non-binary 1.54% (1) Attend college Yes 96.92% (63) No 3.08% (2) Cannabis use days, past month, M (SD), range 19.94 (8.81) 3 – 31 CUD diagnostic criteria, DSM-5 None (0 – 1 symptom) 16.92% (11) Mild (2 – 3 symptoms) 38.46% (25) Moderate (4 – 5 symptoms) 24.62% (16) Severe (6+ symptoms) 20.00% (13) Table 2. Internal Structure Based on Assumed Factors of Scores on the preliminary ARSS-CUV. Factor Reinforcement Ratio Score (SD) Internal consistency (Unstandardized ⍺) Reinforcement Ratio Correlations Cannabis -related cross product Cannabis- free cross product Dating activity Peer Inter- action Sibling & family interaction Sexual activity Chores Dating activity 0.31 (0.20) [n=31] 0.86 0.92 - - - - Peer interaction 0.37 (0.19) [n=53] 0.88 0.92 0.70*** [n=29] - - - Sibling & family interaction 0.16 (0.19) [n=51] 0.77 0.89 0.76*** [n=29] 0.63*** [n=50] - - Sexual activity 0.28 (0.25) [n=59] 0.88 0.91 0.59** [n=30] 0.47** [n=53] 0.58*** [n=51] - Chores 0.29 (0.24) [n=56] 0.67 0.84 0.48** [n=29] 0.45** [n=53] 0.36** [n=50] 0.28* [n=56] Total 0.31 (0.17) [n=28] 0.90 0.94 0.89*** [n=28] 0.89*** [n=28] 0.88*** [n=28] 0.63** [n=28] 0.68*** [n=28] Note. Reinforcement ratio correlations present p-values that are not corrected for multiple comparisons. Interpret with caution. * p < .05, ** p < .01, *** p < .001. Measuring Cannabis Reinforcement 99 Unstandardized Cronbach’s alpha for both the cannabis use and cannabis-free measures were high (⍺s = 0.90 and 0.94, respectively). Mean R- ratio was highest for the Peer interaction factor (R-ratio mean = 0.359), and lowest for Sibling and family interaction (R-ratio mean = 0.141). Internal consistency reliability was high for both the cannabis-related and cannabis-free cross products for all factors except for the cannabis- related cross product for Chores. R-ratio correlations were significant for all factors, and ranged in strength from weak to strong effect (Pearson rs 0.297-0.897). Evidence of Validity Based on Content and Response Processes Evidence of validity based on content and response processes were identified based on item- level statistics and focus group responses to prompts about the activities that were and were not included in the Preliminary ARSS-CUV (which are the same activities from the ARSS- SUV). This section first provides an overview of the activities included on the survey, and activities that the participants indicated were relevant to them, followed by information about the response formats of the survey. Item content. Descriptive, item-level statistics are shown in Table 3. Items (i.e., activities) with R-ratio approaching 1.0 are more reinforcing when using cannabis; items with higher cannabis- free reinforcement values indicate the activities were more reinforcing without cannabis. Table 3. Descriptive Item-Level Statistics Original ARSS-SUV Factor Item Reinforcement Ratio Cannabis-free Reinforcement Value n Mean (SD) Range n Mean (SD) Range Dating_1 Go places with dates or potential romantic partners 60 0.28 (0.29) 0 to 1 63 3.54 (4.24) 0 to 16 Dating_2 Talk with dates or potential romantic partners 58 0.29 (0.25) 0 to 1 62 6.68 (6.37) 0 to 16 Dating_3 Go out to eat with dates or potential romantic partners 39 0.28 (0.28) 0 to 1 62 3.24 (4.06) 0 to 16 Dating_4 Flirt with dates or potential romantic partners 58 0.35 (0.31) 0 to 1 62 5.42 (5.56) 0 to 16 Dating_5 Get compliments from dates or potential romantic partners 49 0.33 (0.24) 0 to 0.86 61 6.41 (5.78) 0 to 16 Dating_6 Go on dates 37 0.30 (0.31) 0 to 1 62 3.05 (3.73) 0 to 16 Dating_7 Kiss dates or potential romantic partners 47 0.34 (0.29) 0 to 1 63 5.34 (5.79) 0 to 16 Leisure_1 Exercise or participate in sports 58 0.16 (0.23) 0 to 0.80 62 6.53 (5.52) 0 to 16 Peer_1 Go out to eat with friends 59 0.44 (0.28) 0 to 1 63 6.25 (4.39) 0 to 16 Peer_2 Talk with same sex friends 58 0.44 (0.27) 0 to 1 62 11.31 (5.03) 0 to 16 Peer_3 Go places with friends 58 0.40 (0.24) 0 to 1 62 8.92 (5.42) 0 to 16 Peer_4 Go for a walk with friends 60 0.25 (0.31) 0 to 1 63 3.51 (4.48) 0 to 16 Peer_5 Talk on the phone with friends 60 0.29 (0.29) 0 to 1 63 5.68 (5.62) 0 to 16 Peer_6 Go to parties with friends 56 0.52 (0.36) 0 to 1 62 3.68 (4.46) 0 to 16 Cannabis, A Publication of the Research Society on Marijuana 100 Peer_7 Talk with friends about day’s activities 58 0.36 (0.22) 0 to 1 62 10.21 (5.41) 0 to 16 Peer_8 Get compliments from friends 58 0.33 (0.24) 0 to 0.82 62 6.89 (5.38) 0 to 16 Peer_9 Meet new people my age 60 0.28 (0.27) 0 to 1 63 5.19 (4.87) 0 to 16 Peer_10 Go hang out where friends meet 55 0.31 (0.27) 0 to 1 62 5.89 (5.27) 0 to 16 Peer_11 Interact with people of own age and sex 58 0.36 (0.23) 0 to 1 61 9.80 (5.27) 0 to 16 Peer_12 Write email, text messages, or letters to friends 57 0.38 (0.23) 0 to 1 60 10.43 (5.03) 0 to 16 Family_1 Go places with siblings or family members 52 0.19 (0.29) 0 to 1 61 3.33 (4.23) 0 to 16 Family_2 Talk with siblings or family members 57 0.19 (0.24) 0 to 0.67 60 7.83 (5.18) 0 to 16 Family_3 Go out to eat with siblings or family members 59 0.12 (0.23) 0 to 1 62 3.57 (4.56) 0 to 16 Family_4 Tell secrets to siblings or family members 59 0.10 (0.25) 0 to 1 62 1.92 (3.77) 0 to 16 Family_5 Spend weekends or vacations with siblings/family 59 0.12 (0.24) 0 to 1 62 3.69 (4.65) 0 to 16 Sex_1 Caressing with a date/romantic partner 59 0.32 (0.29) 0 to 1 62 5.48 (5.23) 0 to 16 Sex_2 Oral sex with a date/romantic partner 59 0.28 (0.29) 0 to 1 62 4.07 (4.45) 0 to 16 Sex_3 Sexual intercourse with a date/romantic partner 59 0.29 (0.29) 0 to 1 62 4.69 (4.93) 0 to 16 Sex_4 Weekends/vacations with romantic partner 59 0.21 (0.32) 0 to 1 62 2.1 (3.57) 0 to 16 Chores_1 Going to school 56 0.17 (0.25) 0 to 1 60 7.28 (4.78) 0 to 16 Chores_2 Studying 56 0.22 (0.33) 0 to 1 60 5.28 (4.83) 0 to 16 Chores_3 Doing chores at home 58 0.47 (0.34) 0 to 1 61 4.48 (4.29) 0 to 16 Note. Items with reinforcement ratios closer to 1.000 indicate activities that were more reinforcing when using cannabis than when not using cannabis. Dating. Dating or activities with potential romantic partners were relatively less reinforcing when engaged in after cannabis use (R-ratio range: 0.208-0.351). The dating item with the highest cannabis-free reinforcement value was flirt with dates or potential romantic partners. Focus groups indicated multiple opportunities to revise the content of items. First, participants indicated confusion on what ‘romantic partners’ meant. This difference in understanding the item wording reportedly influenced their item response, and how researchers may interpret the response data. “The other thing was that I put four [on the Likert-type survey, which is the highest score for frequency and enjoyment of each activity] for everything where romantic partners were concerned, but I am in a relationship. I think that blurred the lines – it may if I’m in a relationship, but if I’m not, it seemed like I just go on a lot of dates and with different people…I don’t think that when under the influence of drugs, I would go on a date with a stranger.” (Focus Group [FG] 1; Participant [P]3) In addition, some highlighted concerns with understanding the items. As some items related to Measuring Cannabis Reinforcement 101 hanging out with friends indicate hanging out with same sex friends, it led to confusion with interpreting the flirting items. “Yeah, it seems inherently heteronormative. What does that mean if I’m bi? Then, it goes into all of the flirting questions. I’m like, ‘I don’t get it.’” (FG 6; P1) Leisure. There was only one item related to leisure: exercising or participating in sports. This was associated with low cannabis reinforcement (R-ratio = 0.16). In focus groups, participants indicated that there should be a distinction between solo-exercising (e.g., weightlifting) and playing recreational sports which have a social component. Participants encouraged a number of new activities to be added to the measure including: sleep, listening to music, watching TV, movies, or YouTube, exercising, arts and crafts, shopping, playing video games, scrolling through social media, and relaxing through meditation or yoga. These activities were indicated as being differentially reinforcing depending on cannabis use. “A lotta stuff you do normally would just be heightened by smoking, listening to music, just eating anything, going out and shopping, getting groceries. [Smoking cannabis] just makes it better.” (FG 6; P6) Peer. Items within the peer factor had the highest reinforcement ratios and cannabis-free reinforcement values on the scale. Going to parties with friends had the highest cannabis reinforcement of all items (R-ratio=0.516). However, participants indicated confusion with this item, and that the item may not have been accurately measuring the intended construct. From Focus Group 6: “Also, maybe going out is different than going to a party ‘cause you’re at a public bar versus a house or something. Going out versus a party…” (P8) “There’s a difference between a house party and a bar scene.” (P3) From Focus Group 2: “Honestly, partying, I stopped doing a lot. I know that’s maybe not directly the question you’re asking, but I definitely stopped partying a lot after I started smoking a lot.” (P2) “Yeah, me too. I never really went to parties that much, and if I do now, they’re weed parties.” (P4) Furthermore, participants indicated it was important to differentiate between scenes where drug use was more expected (e.g., concert) than going out to a nightclub, or if cannabis was being used with friends. Highly reinforcing cannabis-free activities were related to communication with friends (cannabis-free reinforcement values > 10). However, participants indicated these items may be unnecessarily gender specific and outdated (e.g., writing letters). Participants also wanted more clarity on the type of conversations that they were having. When high, ‘you can have really deep conversations’; participants distinguished this level of conversation from ‘chatting.’ Participants indicated that items asking getting compliments from friends were irrelevant and should be removed from the survey. From Focus Group 4: “I don’t really understand ‘getting compliments’ [as an activity].” (P5) “It’s on there twice, too. Get compliments from dates and get compliments from friends...” (P6) “That’s not an activity. I’m thinking of activities as something I go out of my way to do.” (P2) One item that was more controversial was walking with friends. Some participants indicated that it was not a relevant activity for them to engage in; some participants, however, indicated that they routinely ‘explore all of campus high.’ This item had the lowest reinforcement ratio on the peer factor (R-ratio = 0.249). Family. The items related to family had the lowest reinforcement ratios and cannabis-free reinforcement value ranges of the entire Cannabis, A Publication of the Research Society on Marijuana 102 instrument. Telling secrets to siblings or family members (R-ratio = 0.102) was perceived as an irrelevant activity by participants despite being sensitive to individual differences and contexts. “I also really liked the question about [telling] secrets because I feel like I can't tell my family stuff 'cause I know my sister will tell my mom right away. …. I really like that question 'cause it applies very differently to people.” (FG 2; P5) In general, however, participants felt that it was inappropriate to be using cannabis while around family (supporting the lower R-ratios). From Focus Group 8: “I feel like a lot of the family member [questions] may be a bit tricky ‘cause I’m not tryin’ to hang out with my mom while I’m blazed.” (P10) “I don’t wanna speak for everyone, but [smoking with family] is just not right. If you’re home with your family, then you walk out [to smoke] and come back in, unless it’s Thanksgiving. That’s a different story.” (P9) Sex. Caressing romantic partners had the highest reinforcement ratio of the items related to sexual intimacy (R-ratio = 0.319); yet, this item was perceived to be ‘weird’ and redundant. “I’d also say joining caressing, oral sex and sexual intercourse [as a single activity] maybe. That’s physical touch with your romantic partner, unless there’s a reason to—why those need to be [asked separately]…” (FG 7; P8) In contrast, during focus groups, participants described a variety of sexually intimate behaviors they felt were applicable to the cannabis- reinforcing activities including masturbation, sex with a partner who is also high, and sex with a partner who is not high. “Personally, with my boyfriend, if he's high and I'm not, I don't enjoy it as much 'cause I feel like we're not in the same place, and then vice versa. Also, if we're both high, it can be really, really enjoyable...” (FG 2; P2) Chores. The chores domain had the second highest mean R-ratio score, which was driven by doing chores at home having the second highest item-level R-ratio of the entire questionnaire (R- ratio = 0.472). Doing chores at home (e.g., laundry, cleaning dishes) was perceived as enjoyable when high. “Yeah. I know there’s one question about doing chores at home. It’s definitely more enjoyable to do stuff like that when you’re high like cleaning. I feel like time is less important. It’s like, ‘Oh, I have time to do things I need to do,’ so I’ll cook. I’ll do little stuff that I’ve been meaning to do.” (FG 3; P1) Some participants, however, felt that the activities they were doing at home (e.g., organizing) would not be considered a chore. The items related to schoolwork and studying were perceived as relevant to participants, and may differentiate between participants’ usage patterns (e.g., some participants indicated they ‘definitely wouldn’t’ go to school or study high, while others go ‘to class high almost every single day’). Similar to other activities, cannabis was a way to ‘enhance’ otherwise monotonous activities. Participants also recommended adding working as an activity. Other activities that were recommended in this domain were cooking and grocery shopping. Item response format. Focus groups provided several format considerations. There was substantial discussion regarding confusion given the format of the ARSS-SUV and use of numerical values for ‘enjoyment’ and ‘frequency’ for activities when cannabis was and was not being used. “Yeah, it was just a lot to think about, the difference and then how many times for each scenario if you were using [cannabis or not]. Especially because it’s so repetitive, I was just like—I put this for this answer. I’m just gonna put the same [answer] cuz it’s the same question for me.”(FG3;P6) This is a major concern for reliability and validity of responses, as participants indicated that they ‘completely ignored the key’ for responses and used the actual frequency instead of the intended frequency measure. Measuring Cannabis Reinforcement 103 This concern is also related to participant burden. In addition to the length of the survey, other considerations included the usability of the survey on mobile phones. Lastly, participants were not sure what to include for ‘enjoyment’ when the ‘frequency’ was never (value of 0). This was particularly relevant for activities, such as vacations, which are infrequent. Mixed Methods Findings We integrated findings from the quantitative and qualitative data sources to develop a revised version of the ARSS-CUV. The revised preliminary survey and scoring instructions is provided in Supplemental File 1. DISCUSSION Substance use is related to the availability and reinforcing aspect of substance-free and substance-involved activities (Bickel et al., 2014). Much of the evidence has utilized measures of reinforcement, such as the ARSS-SUV which broadly assesses reinforcement from activities with and without using alcohol or drugs. This study was the first to attempt to adapt a reinforcement measure specifically for cannabis use by utilizing mixed-methods procedures to test a modified, cannabis version of the ARSS and collect formative data to guide measurement refinement. The preliminary ARSS-CUV had high internal consistency, and reinforcement ratios between each of the five factors were significantly correlated. The ARSS-CUV measures the relative reinforcing value of engaging in activities with cannabis compared to without cannabis. Assessment of engagement in different activities via the ARSS-CUV allows for quantification of the relative enjoyment and frequency (i.e., magnitude and rate of reinforcement; R-ratio) of individuals’ behavioral allocation with and without cannabis. Future studies will examine if cannabis- reinforcement as measured by the revised ARSS- CUV is associated to cannabis use and CUD symptom severity. Further, recent evidence indicates that behavioral economic interventions that promote cannabis-free activity engagement among adults with CUD are promising in reducing cannabis use (Coughlin et al., 2023). Moreover, research shows that substance-free reinforcement moderates treatment response, such that those who reported low levels of substance-free reinforcement at baseline showed greater reductions in heavy drinking following brief intervention (Murphy et al., 2012). Future research may test whether cannabis-free reinforcement as measured by the ARSS-CUV is predictive of treatment response, and whether increasing cannabis-free reinforcement is a viable intervention target. The revised ARSS-CUV was tailored to a young adult population by ensuring that the survey items (i.e., activities) were relevant and salient. Availability and access to potentially reinforcing activities is an important consideration when measuring reinforcement. Future studies may need to test and adapt the ARSS-CUV if measuring cannabis reinforcement in samples that are considerably different (e.g., in- treatment populations). Further, given the length and structure of the ARSS-CUV, as highlighted by participants’ comments regarding its repetitive nature, future research should test and validate briefer versions to assess cannabis-related and cannabis-free reinforcement. The five-factor subscales of the ARSS-CUV was informed by the ARSS-SUV (Murphy et al., 2005). Of the five subscales, the Peer interaction factor had the highest reinforcement ratio, while the Siblings and family factor had the lowest. Overall, internal consistency was high for both cannabis-related and cannabis-free cross products across all factors (except for Chores) and reinforcement ratio correlations were also significant for all factors. The findings related to internal structure, however, are secondary to the evidence related to the content of the measure. We identified specific activities young adults who use cannabis engage in and find reinforcing, which are discussed below. Dating. Within the dating subscale, activities involving romantic partners were less reinforcing when using cannabis. Young adults may not be comfortable interacting with dates while under the influence of cannabis, which is consistent with their verbal responses, suggesting feelings of discomfort when using cannabis with individuals they do not know well. This may in part be due to confusion surrounding how “potential romantic partner” was defined. Participants felt that their decision to use cannabis with romantic partners is dependent on length of time knowing the person Cannabis, A Publication of the Research Society on Marijuana 104 (long-term relationship versus “hooking up”), which might influence reinforcement. Hence, it may be important to assess and control for relationship status when measuring cannabis reinforcement in this population. Leisure. Participants strongly encouraged addition of leisure activities including playing video games, communicating via social media, arts and crafts, and separating exercise from sports, which were incorporated in the revised version. Interestingly, many participants encouraged the addition of sleep as it was a commonly reported motive for using cannabis in this sample. Yet, how to best assess cannabis-free vs. cannabis-involved reinforcement from sleep remains unclear. Some participants recommended defining sleep as “downtime before bed” in which the frequency and enjoyment (perceived sleep quality) of using cannabis before bed would be assessed. The current ARSS-CUV does not include sleep, however, future research should examine how to properly define and assess sleep-related cannabis reinforcement. Peer. Reinforcement ratios and cannabis-free reinforcement values were highest among the peer factors. This is consistent with findings from the alcohol specific reinforcement literature which highlights that the social component of substance use may be more important from a reinforcement and risk aspect than the type or nature of the activity (Hallgren et al., 2016). Future studies could assess whether activities were conducted alone or with peers. Type of peer-involved activity was also important from a reinforcement standpoint with parties providing more cannabis- related reinforcement and communication with peers providing high levels of cannabis-free reinforcement. Future research will examine relations between social groups and social network affiliations to better understand reinforcement from activities such as interacting with people of own age. The revised measure differentiates between house parties and going out to bars/clubs as this distinction may influence cannabis use and reinforcement. Similarly, we eliminated the same-sex wording throughout. Family. Family related items had the lowest reinforcement ratios and cannabis-free reinforcement value ranges on the measure. Most young adults were not comfortable using cannabis when with parents, whereas some were comfortable using around siblings or cousins. Time spent with family, especially parents, may serve as a protective factor for risky cannabis use. This is consistent with evidence suggesting young adults adjust their substance use based on the environment, including parental influence (Miller et al., 2016). Future research could separately assess activity reinforcement with different types of family members (e.g., siblings; parents). Further, because the sample was primarily college students, the frequency of engagement in family-related activities may be lower due to participants being away at school which may impact interpretation of reinforcement levels. Hence clarifying or assessing opportunity to engage in family-related activities may be useful when assessing cannabis-free and involved reinforcement with this population. Sex. Although participants reported that sexual activity with cannabis was highly reinforcing during the qualitative focus group discussions, quantitavely, sexual activity had the second lowest raw reinforcement ratio score (only higher than sibling and family interactions). These contrasting findings may be explained in part by the phrasing of the question in survey which was perceived by particpants as ‘weird’ and redundant. Based on participant recommendations, the sex-related questions were simplified and reduced to one question encompassing multiple sexual related behaviors for the main measure. Sexual activities while under the influence of cannabis (i.e., high from cannabis use), and in situations where both the individual and the sexual partner are both “high” appear to be particularly enjoyable and may thus be differentially reinforcing. This increased reinforcement from “sex while high” may also increase odds of future cannabis use and higher consumption, which may increase odds of CUD, and should be examined in future studies, using the sex-module to the ARSS-CUV. Participants also acknowledged that sex is, in many ways, a unique activity compared to other activities included in the ARSS-CUV, suggesting it may warrant a separate, specific sex reinforcement module to be developed in the future. Chores. The chores domain had the second highest mean reinforcement ratio score, indicating high levels of enjoyment when doing chores while using cannabis, especially home- related chores. Changes made to this section include adding examples of common chores Measuring Cannabis Reinforcement 105 completed at home, adding a chore conducted outside of the home (e.g., grocery shopping), and adding cooking as its own question among other minor changes. Academic-related activities like studying seemed to be a protective factor for some participants while others noted experiencing reinforcement from using cannabis when engaging in these activities. This is consistent with behavioral economic literature demonstrating that academic and career related next-day responsibilities (e.g., test) are related to reductions in substance use demand (Ferguson et al., 2021; Skidmore et al., 2011). Those who verbally indicated enjoying using cannabis during academic activities may be less sensitive to environmental contingencies which places them at increased risk for cannabis related consequences. This study has limitations. The use of a small convenience sample in the quantitative aim led to the inability to explore the factor structure, even though the current sample (N = 65) is comparable to the ARSS-SUV study (Murphy et al., 2005; N = 54). However, in mixed methods studies, the alignment between samples across data sources (i.e., use of quantitative and qualitative data) is a relative strength, bolstering confidence in findings (Perez et al., 2023). In addition, there were very few non-college students (n = 2); thus, generalizability to the broader young adult population may be limited. Methods (e.g., use of self-report, focus groups) may introduce some bias (e.g., desirability) despite efforts to minimize their effects. Conclusions This study is the first to adapt a reinforcement measure specifically for cannabis use using a mixed methods design. While our results provide some initial support for the validity of the ARSS- CUV, there are several opportunities to improve the measurement of cannabis reinforcement. Future research should test psychometric properties of this measure across different populations. REFERENCES Acuff, S. F., Dennhardt, A. A., Correia, C. J., & Murphy, J. G. (2019). Measurement of substance-free reinforcement in addiction: A systematic review. Clinical Psychology Review, 70, 79–90. https://doi.org/10.1016/j.cpr.2019.04.003 American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing (U.S.). (2014). Standards for educational and psychological testing. AERA. American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders. American Psychiatric Publishing. https://dsm.psychiatryonline.org/doi/book/10.1 176/appi.books.9780890425596 Aston, E. R., Farris, S. G., MacKillop, J., & Metrik, J. (2017). Latent factor structure of a behavioral economic marijuana demand curve. Psychopharmacology, 234(16), 2421– 2429. https://doi.org/10.1007/s00213-017- 4633-6 Bickel, W. K., Johnson, M. W., Koffarnus, M. N., MacKillop, J., & Murphy, J. G. (2014). The behavioral economics of substance use disorders: Reinforcement pathologies and their repair. Annual Review of Clinical Psychology, 10, 641–677. https://doi.org/10.1146/annurev-clinpsy- 032813-153724 Cerdá, M., Mauro, C., Hamilton, A., Levy, N. S., Santaella-Tenorio, J., Hasin, D., Wall, M. M., Keyes, K. M., & Martins, S. S. (2020). Association between recreational marijuana legalization in the United States and changes in marijuana use and cannabis use disorder from 2008 to 2016. JAMA Psychiatry, 77(2), 165–171. https://doi.org/10.1001/jamapsychiatry.2019.3 254 Cohen, K., Weizman, A., & Weinstein, A. (2019). Positive and negative effects of cannabis and cannabinoids on health. Clinical Pharmacology & Therapeutics, 105(5), 1139–1147. https://doi.org/10.1002/cpt.1381 Correia, C. J., Carey, K. B., & Borsari, B. (2002). Measuring substance-free and substance- related reinforcement in the natural environment. Psychology of Addictive Behaviors: Journal of the Society of Psychologists in Addictive Behaviors, 16(1), 28–34. Cannabis, A Publication of the Research Society on Marijuana 106 Correia, C. J., Carey, K. B., Simons, J., & Borsari, B. E. (2003). Relationships between binge drinking and substance-free reinforcement in a sample of college students: A preliminary investigation. Addictive Behaviors, 28(2), 361–368. https://doi.org/10.1016/S0306- 4603(01)00229-5 Coughlin, L. N., Bonar, E. E., Wieringa, J., Zhang, L., Rostker, M. J., Augustiniak, A. N., Goodman, G. J., & Lin, L. (Allison). (2023). Pilot trial of a telehealth-delivered behavioral economic intervention promoting cannabis- free activities among adults with cannabis use disorder. Journal of Psychiatric Research, 163, 202–210. https://doi.org/10.1016/j.jpsychires.2023.05.01 2 Dennhardt, A. A., Yurasek, A. M., & Murphy, J. G. (2015). Change in delay discounting and substance reward value following a brief alcohol and drug use intervention. Journal of the Experimental Analysis of Behavior, 103(1), 125–140. https://doi.org/10.1002/jeab.121 Ferguson, E., Bush, N., Yurasek, A., & Boissoneault, J. (2021). The effect of next day responsibilities and an adaptive purchase task on cannabis demand. Drug and Alcohol Dependence, 227, 108919. https://doi.org/10.1016/j.drugalcdep.2021.108 919 Fetters, M. D. (2020, October). The mixed methods research workbook: Activities for designing, implementing, and publishing projects (1 edition). SAGE Publications, Inc. SAGE Publications Inc. Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving integration in mixed methods designs: Principles and practices. Health Services Research, 48(6pt2), 2134– 2156. https://doi.org/10.1111/1475-6773.12117 Fetters, M. D., & Molina-Azorin, J. F. (2017). The Journal of Mixed Methods Research starts a new decade: The mixed methods research integration trilogy and its dimensions. Journal of Mixed Methods Research, 11(3), 291–307. https://doi.org/10.1177/1558689817714066 Hallgren, K. A., Greenfield, B. L., & Ladd, B. O. (2016). Psychometric properties of the adolescent reinforcement survey schedule – Alcohol use version with college student drinkers. Substance Use & Misuse, 51(7), 812– 822. https://doi.org/10.3109/10826084.2016.115560 9 Higgins, S., Heil, S., & Plebani, J. (2004). Clinical implications of reinforcement as a determinant of substance use disorders. Annual Review of Psychology, 55, 431–461. https://doi.org/10.1146/annurev.psych.55.0909 02.142033 Holmes, G. R., Heckel, R. V., Chestnut, E., Harris, N., & Cautela, J. (1987). Factor analysis of the Adolescent Reinforcement Survey Schedule (ARSS) with college freshmen. Journal of Clinical Psychology, 43(4), 386–390. https://doi.org/10.1002/1097- 4679(198707)43:4<386::AID JCLP2270430408>3.0.CO;2-E Leventhal, A. M., Bello, M. S., Unger, J. B., Strong, D. R., Kirkpatrick, M. G., & Audrain- McGovern, J. (2015). Diminished alternative reinforcement as a mechanism underlying socioeconomic disparities in adolescent substance use. Preventive Medicine, 80, 75– 81. https://doi.org/10.1016/j.ypmed.2015.05.021 Meldrum, R. C., & Leimberg, A. (2018). Unstructured socializing with peers and risk of substance use: Where does the risk begin? Journal of Drug Issues, 48(3), 452–471. https://doi.org/10.1177/0022042618774263 Miller, M. B., Merrill, J. E., Yurasek, A. M., Mastroleo, N. R., & Borsari, B. (2016). Summer versus school-year alcohol use among mandated college students. Journal of Studies on Alcohol and Drugs, 77(1), 51–57. https://doi.org/10.15288/jsad.2016.77.51 Moseholm, E., & Fetters, M. D. (2017). Conceptual models to guide integration during analysis in convergent mixed methods studies. Methodological Innovations, 10(2), 2059799117703118. https://doi.org/10.1177/2059799117703118 Murphy, J. G., Correia, C. J., Colby, S. M., & Vuchinich, R. E. (2005). Using behavioral theories of choice to predict drinking outcomes following a brief intervention. Experimental and Clinical Psychopharmacology, 13(2), 93– 101. https://doi.org/10.1037/1064-1297.13.2.93 Murphy, J. G., Dennhardt, A. A., Skidmore, J. R., Borsari, B., Barnett, N. P., Colby, S. M., & Martens, M. P. (2012). A randomized controlled trial of a behavioral economic Measuring Cannabis Reinforcement 107 supplement to brief motivational interventions for college drinking. Journal of Consulting and Clinical Psychology, 80, 876– 886. https://doi.org/10.1037/a0028763 Perez, A., Howell Smith, M. C., Babchuk, W. A., & Lynch-O’Brien, L. I. (2023). Advancing quality standards in mixed methods research: Extending the legitimation typology. Journal of Mixed Methods Research, 17(1), 29–50. https://doi.org/10.1177/15586898221093872 Phillips, J. G., Evans, M., Hughes, B., & Ogeil, R. P. (2020). Patterns of Cannabis Consumption, Social Networks, and Foraging. Journal of Drug Issues, 50(1), 63–76. https://doi.org/10.1177/0022042619887501 Rogeberg, O., & Elvik, R. (2016). The effects of cannabis intoxication on motor vehicle collision revisited and revised. Addiction, 111(8), 1348– 1359. https://doi.org/10.1111/add.13347 Saldaña, J. (2015). The coding manual for qualitative researchers. Sage. Schulenberg, J. E., Patrick, M. E., Johnston, L. D., O’Malley, P. M., Bachman, J. G., & Miech, R. A. (2021). Monitoring the Future National Survey Results on Drug Use, 1975-2020. Volume II, College Students & Adults Ages 19- 60. In Institute for Social Research. Institute for Social Research. https://eric.ed.gov/?id=ED615085 Skidmore, J. R., & Murphy, J. G. (2011). The effect of drink price and next-day responsibilities on college student drinking: A behavioral economic analysis. Psychology of Addictive Behaviors : Journal of the Society of Psychologists in Addictive Behaviors, 25(1), 57–68. https://doi.org/10.1037/a0021118 Stoner, S. (2018). Marijuana and sexual risk behavior among youth and emerging adults: What do we know? Seattle: Alcohol & Drug Abuse Institute, University of Washington. http://adai.uw.edu/pubs/pdf/2018MarijuanaR SB.pdf Substance Abuse and Mental Health Services Administration. (2021). Key substance use and mental health indicators in the United States: Results from the 2020 national survey on drug use and health. Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. https://www.samhsa.gov/data/ Wallace, G. T., Parnes, J. E., Prince, M. A., Conner, B. T., Riggs, N. R., George, M. W., & Shillington, A. M. (2020). Associations between marijuana use patterns and recreational legislation changes in a large Colorado college student sample. Addiction Research & Theory, 28(3), 211–221. https://doi.org/10.1080/16066359.2019.162200 Funding and Acknowledgements: This research was supported by research funds from the Department of Health Education and Behavior at the University of Florida. Dr. Gebru’s contribution to the manuscript was supported by NIAAA T32 AA07459. Dr. Yurasek’s contribution to the manuscript was supported by National Institute on Drug Abuse (NIDA) grant K23 DA046565. All funding sources had no role in the study design, collection, analysis or interpretation of the data, writing the manuscript, or the decision to submit the paper for publication. All authors report no conflicts of interest. Copyright: © 2024 Authors et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction, provided the original author and source are credited, the original sources is not modified, and the source is not used for commercial purposes. https://creativecommons.org/licenses/by/4.0/