Feasibility and Acceptability of a Mobile App for Prolonged Grief Disorder Symptoms Research Articles Feasibility and Acceptability of a Mobile App for Prolonged Grief Disorder Symptoms Anaïs Aeschlimann 1 , Nicolas Gordillo 2 , Taro Ueno 3 , Andreas Maercker 1 , Clare Killikelly 1,4 [1] Department of Psychology, University of Zurich, Zurich, Switzerland. [2] Department of Informatics, University of Zurich, Zurich, Switzerland. [3] SUSMED Inc., Tokyo, Japan. [4] Department of Psychiatry, University of British Columbia, Vancouver, Canada. Clinical Psychology in Europe, 2024, Vol. 6(1), Article e10881, https://doi.org/10.32872/cpe.10881 Received: 2022-12-19 • Accepted: 2024-01-15 • Published (VoR): 2024-03-28 Handling Editor: Cornelia Weise, Philipps-University of Marburg, Marburg, Germany Corresponding Author: Clare Killikelly, Department of Psychology, University of Zurich, Binzmuehlestrasse 14/17, CH-8050 Zurich, Switzerland. E-mail: c.killikelly@psychologie.uzh.ch Supplementary Materials: Materials [see Index of Supplementary Materials] Abstract Background: Mobile apps provide a unique platform for mental health assessment and monitoring. They can provide real time, accessible data on symptoms of mental disorders that may yield rich data for detailed clinical assessment and help individuals gain insight into their current mental state. We developed one of the first apps for tracking symptoms of prolonged grief disorder. Method: In this pilot feasibility study, we assess the feasibility and acceptability of a new mobile app mGAGE for use once a day for 3 weeks. 27 participants completed mental health assessments at t1 and t2. Results: Adherence to the app protocol was very high with 100% for the first two weeks of use. A surprising finding was the improvement of grief symptoms at t2. Debriefing interviews revealed general qualitative categories including positive feedback, negative feedback and specific recommendations. Overall, the app was found to be feasible for use for the first two weeks and acceptable for bereaved individuals. Conclusions: This app could provide valuable data for in depth clinical assessment, may support individuals to gain greater insight into their symptoms and may have a therapeutic effect in terms of improved grief symptoms. Implications for future studies including use in larger intervention studies are discussed. This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International License, CC BY 4.0, which permits unrestricted use, distribution, and reproduction, provided the original work is properly cited. https://crossmark.crossref.org/dialog/?doi=10.32872/cpe.10881&domain=pdf&date_stamp=2024-03-28 https://orcid.org/0000-0001-7049-5131 https://orcid.org/0000-0002-1537-6719 https://orcid.org/0000-0003-3697-4301 https://orcid.org/0000-0001-6925-3266 https://orcid.org/0000-0003-2661-4521 https://www.psychopen.eu/ https://cpe.psychopen.eu/ https://creativecommons.org/licenses/by/4.0/ https://creativecommons.org/licenses/by/4.0/ https://creativecommons.org/licenses/by/4.0/ Keywords prolonged grief disorder, e-mental health, mobile app, self-monitoring intervention Highlights • Prolonged grief disorder is a new mental health disorder in the ICD-11: This is one of the first mobile apps developed to assess the new ICD-11 PGD. • The app is found to be easy to use and acceptable by bereaved individuals. This may improve clinical assessment as it provides real time data on the intensity and stability of grief over a 2-week period. • The act of self-monitoring may have an intervention effect; after two weeks of reporting grief symptoms there was a significant improvement in grief symptoms. The death of a loved one represents one of the most severe life stressors (Breslau et al., 1998). In a majority of those affected (80-90%), acute grief symptoms dissipate after a certain time, however, approximately 10% experience a prolonged and severe grief reac­ tion (Lundorff et al., 2017). With the introduction of the diagnostic category prolonged grief disorder (PGD) in 2018, the World Health Organization’s International Classifica­ tion of diseases (ICD-11) allows the diagnosis of disordered grief reactions for the first time. PGD is characterized by two core symptoms relating to longing or yearning for the deceased and persistent preoccupation with the deceased, accessory symptoms of emotional distress and a time and functional impairment criterion. The criteria also take into account varying cultural norms and practices, stating that symptoms must exceed the typical duration and intensity in an individual’s culture and context (Maercker et al., 2013). However, the applicability of the diagnostic criteria, including the time criterion, have not yet been established beyond the Western-European contexts (Stelzer et al., 2020). If left untreated, individuals affected by PGD are at risk of experiencing a range of further serious health and psychosocial problems including increased rates of cardio­ vascular problems, high blood pressure, harmful health behaviors, substance abuse, or suicidality (Fujisawa et al., 2010; Kersting et al., 2011; Maercker et al., 2008; Prigerson et al., 2009). Being a relatively recent diagnostic category however, PGD may be difficult for clinicians to differentiate from normal grief (Keeley et al., 2016). Furthermore, PGD does not respond well to interventions that are intended and effective for other bereave­ ment-related mental health problems, e.g., depression, but rather calls for interventions specifically tailored to the precursor to PGD, complicated grief (M. K. Shear et al., 2016). Hence, a correct diagnosis is highly relevant for identifying individuals in need of treatment and providing suitable interventions. Although recent evidence has shown the effectiveness of interventions specifically tailored to PGD, research still remains limited and further investigation is necessary (Boelen et al., 2006; Bryant et al., 2014; Killikelly & Maercker, 2017; Shear et al., 2005). mGAGE Mobile Assessment for Grief 2 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ Another domain of research, which is still lacking, concerns providing the individual with the right support at the right time (Wagner, 2013; Wakefield, 2012). Currently, the long-term trajectories of PGD are poorly understood and further research on the heter­ ogeneity in the fluctuations of symptoms over time is needed (Bonanno & Malgaroli, 2020; Sveen et al., 2018). Some individuals may need immediate psychotherapy support while others may benefit from self-help and monitoring (Johannsen et al., 2019). In addition, recall bias can be problematic for individuals who provide a one-off self-report questionnaire on symptoms. From depression to psychosis there can be variability in the presentation and reporting of symptoms day by day. Digital technology provides the opportunity to more reliably monitor symptoms daily over a longer period of time to ensure a robust assessment and valid measurement (Lenferink et al., 2022). Alongside more accurate diagnosis, symptom tracking and self-monitoring has two uses for inter­ ventions. Firstly, daily monitoring alongside psychological and behavioral interventions would allow participants to see the effect of interventions as they unfold day to day. Secondly, the mere act of tracking and monitoring symptoms could provide more insight into symptoms and their severity. In recent years, digitalization has gained importance in the domain of mental health, bearing the potential to overcome access barriers, as well as expanding the availability and quality of mental health treatment (Chandrashekar, 2018; Torous et al., 2019). Inno­ vative solutions to self-management of mental health problems are especially relevant, given the large treatment gap, meaning that only a small proportion of individuals in need of treatment receive professional help (Kohn et al., 2004). E-mental health interven­ tions delivered via smartphones bring many advantages including immediate support, constant availability, anonymity, low cost, greater access and equity of mental health resources (Olff, 2015). There has been an increase in the number of available smartphone apps for monitor­ ing and management of mental health symptoms (Wang et al., 2018). This method of ambulant monitoring possesses a number of advantages, which have been demonstrated for various mental health problems (Myin-Germeys et al., 2011; Wichers et al., 2011). This type of data may deliver insights on symptom triggers, relapse signatures, real-time effects of treatment and can provide early detection of change in symptoms (Birchwood et al., 2000; Gleeson et al., 2005; Palmier-Claus, 2011). This has the potential of facilitating earlier interventions, preventing relapse and avoiding hospital admissions, thus also reducing costs (Whelan et al., 2015). Recently ‘My Grief App’ was developed by a Swedish collaborator for bereaved parents after the death of a child (Eklund et al., 2021, 2022). It includes modules focused on psychoeducation and intervention with some brief symptom tracking (one item, grief severity). In a pilot study 13 parents used the app for 4 weeks and it was found to be an acceptable and useful intervention. A follow-up randomized control trial is currently underway to assess the effectiveness in reducing PGD symptoms. In terms of grief monitoring, a recent study confirmed the feasibility Aeschlimann, Gordillo, Ueno et al. 3 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ and acceptability of grief symptom monitoring using a mobile app and experience sampling methodology (Lenferink et al., 2022). Bereaved individuals responded to PGD symptom questions five times a day for two weeks. Adherence was variable with a high drop out rate of 35-40%. Our current study adds to this new wave of symptom tracking research by developing a mobile app to directly assess ICD-11 PGD symptoms using a validated questionnaire, the International Prolonged Grief Disorder Scale (IPGDS) and using a less intensive monitoring frequency. A Mobile Self-Report Tool to Assess PGD: mGAGE Taking into consideration the current knowledge on the benefits of symptom monitoring and harnessing the potential of digitalization, we designed and developed the mobile Grief Assessment Guide and E-resource (mGAGE). mGAGE was designed employing the user-centered design process and two focus groups of bereaved individuals. The first step in our development process was to design an app for use by bereaved individuals in general, not only those with clinically severe PGD. Focus group partici­ pants were recruited from a convenience sample and using the snowball technique and the inclusion criteria included adults who had experienced the death of a loved one at least 6 months ago. The first focus group (n = 4) openly explored the need for and qualities of an app for grief, including advantages and disadvantages of such an app. The second focus group (n = 4) elaborated on a preliminary design for the app and gathered feedback on the design and specific planned features of the app. Examples of the focus group questions include: what type of information about grief would you like to know? How many questions would you answer? What kind of feedback would be helpful for you? One important consideration in the further development of this app is the purpose of the app. We have sought to ensure that the user centered design (Schnall et al., 2016) is applied to all stages of the app development. User centered design is defined as an itera­ tive developmental process that incorporates user feedback across different stages of app development. This can be achieved through interviews and focus groups. Related to this, the findings from the current study confirm that it is important to provide participants with the choice of how and when to use the app. Several participants identified the need for personalization of the app. In the next iterative round of development, we will add options to ensure that participants can choose when they complete the app, for how long they would like to use it and how frequently. Moreover, self-monitoring tools, such as mGAGE, have the potential to promote a more empowered and active role of patients in treatment (Huber et al., 2011). Future research should explore these possibilities in the form of a randomized controlled trial with an active control group and importantly, with a clinical sample of those diagnosed with PGD. mGAGE Mobile Assessment for Grief 4 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ In the current version of the mGAGE app it can be delivered via an iOS and Android app, which can be used online on mobile devices. Users create a personalized login ID and are then led to two introduction pages containing information on PGD and the mGAGE app respectively. The app sends users a daily reminder to fill out the integrated questionnaire on PGD symptoms. The completion of the questionnaire takes around 5 minutes. At the end of the questionnaire, users have the option to utilize the diary function, which additionally allows them to record their mood, thoughts, behaviors or actions (see Figure 1). After completion, users receive a feedback concerning the severity of their symptoms, as well as help-seeking recommendations. If the user scores above the cut-off score, indicating that they may suffer from PGD, a direct link to different local services and resources including psychological support appears at the bottom of the page. Furthermore, mGAGE includes a graph function (see Figure 1), which allows users to visualize their assessment history, helping them track the course of their symptoms. Additionally, mGAGE provides users with a support resource page, which lists different professional support resources for bereaved individuals. Figure 1 Screenshot of the mGAGE App Welcome Page and Symptom Tracking Screens Symptom course Diary Track your mood, thoughts, behavior, and activities here regularly. Studies show that this promotes awareness for our emotions and that this can lead to a reduction of psychological problems and promote coping strategies. Figure 1: Screen shot of the mGAGE app welcome page and symptom tracking screens. The aim of the present study is to (1) evaluate the feasibility of the app in terms of adherence and use of the app as well as (2) the acceptability of the mGAGE app in terms of feedback and evaluation of the app by bereaved individuals. We also explore the variability of IPGDS scores across the 2 weeks and provide insights that may guide the app use to evaluate the effectiveness of PGD interventions. Aeschlimann, Gordillo, Ueno et al. 5 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ Method Study Design We conducted an observational feasibility study with the aim of developing and assess­ ing the feasibility and acceptability of a new mobile grief symptom tracking tool to assist bereaved individuals in self-tracking and monitoring their grief symptoms. The study protocol was approved by the ethics review board of the University of Zurich. Participants and Recruitment Participants were bereaved individuals, who had experienced the loss of a close person. Inclusion criteria specified: 1) fluency in German; 2) age of 18 or older; 3) ability to provide written informed consent; 4) loss of a loved one (family or friend) at least 6 months prior; 5) use of a smartphone. Exclusion criteria were a severe mental health disorder (e.g., Major depression, suicidality, current schizophrenia) or currently being an in-patient. Participants were remunerated for their time to complete the pre-assessment with 30 Swiss francs (CHF) and the post-assessment with 50 CHF. Furthermore, participants who were Psychology students had the option of being compensated with course credit (4 hours) as an alternative. Additionally, participants had the option of being reimbursed up to 20 CHF for travel expenses related to study participation. Recruitment took place between August 24th and November 4th, 2020. Methods of recruitment included the posting of advertisements and recruitment links on different Facebook groups and mailing lists of the University of Zurich, as well as through WhatsApp groups. Four people who had agreed to participate dropped out before t1. Reasons for drop out included personal reasons and lack of readiness to discuss the death (10 people were interested in the study but did not meet criteria). Measures Participants were assessed twice in-person at the Department of Psychology of the Uni­ versity of Zurich: for the pre-assessment (t1) and the post-assessment (t2). Assessments were conducted by the project manager and a bachelor’s student in Psychology under the supervision of a clinical psychologist. The bachelor’s student was previously trained by the project manager on the study procedures, including the administration of the questionnaires. Additionally, participants were encouraged to provide daily online data regarding their grief symptoms for 3 weeks between t1 and t2. One participant continued to use the app for longer than the required 21 days (up to 27 days). However, this was only 1 participant and beyond the scope of our protocol. The duration of assessment of up to 3 weeks was determined as a limit based on previous experience sampling methods mGAGE Mobile Assessment for Grief 6 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ with bereaved individuals who found that adherence to self-monitoring significantly declined after 2 weeks (Lenferink et al., 2022; Mintz et al., 2023). Socio-Demographic Data Data on socio-demographic information (age, gender, nationality, marital status, educa­ tion, psychotherapy experience) was collected at t1. Mental Health Outcomes At t1 and t2, we assessed several mental health outcomes using the measures and descriptions listed here. Grief was assessed using the International ICD-11 Prolonged Grief Disorder Scale, including both the Standard Subscale and the Cultural Supplement, IPGDS Killikelly et al., 2020). The standard subscale consists of 13 items based on the ICD-11 definition of PGD, while the cultural supplement consists of an additional 19 possible items that can be used for treatment planning and cultural acceptability. Symptoms are rated on a 5-point scale: 1 = almost never (less than once a month), 2 = rarely (monthly), 3 = sometimes (weekly), 4 = often (daily), and 5 = always (several times a day). An impairment item as well as screening items for the length of time since bereavement and the violation of socio-cultural norms was also included. PGD strict criteria requires the fulfillment of the following criteria: one of Items 1 or 2, 1 or more of Items 3-12 and the impairment criteria (Item 13) all rated 4 or above (Lenferink et al., 2021). PGD moderate criteria is the same as the strict criteria except all items are rated 3 or above. Finally, the Maciejewski et al. (2016) criteria includes one of Items 1 or 2, 3-5 of Items 3-12 and no impairment criteria, all rated 4 or higher (Breen & O’Connor, 2007; Maciejewski et al., 2016). Depression: The Patient Health Questionnaire (PHQ) is a self-administered version of the PRIME – MD diagnostic instrument for common mental disorders (Kroenke et al., 2001). The PHQ-9 is the depression module, which scores each of the 9 DSM – IV criteria as “0” (not at all) to “3” (nearly every day). A cutoff score of ≥ 10 has been recommended to indicate moderate to severe depression (Kroenke, Spitzer, & Williams, 2001). Anxiety: The 7-item Generalized Anxiety Disorder Scale (GAD-7) is a practical self- report anxiety questionnaire that proved valid in primary care according to DSM-IV (Spitzer et al., 2006). Scores for all 7 items range from 0 (Not at all) and 3 (Nearly every day). A cutoff score of 8 or above is recommended to identify possible anxiety disorder. Post traumatic stress disorder: International Trauma Questionnaire (ITQ) is the ICD-11 based post-traumatic stress disorder (PTSD) measure (Cloitre et al., 2018). The first 9 items of the scale relate to core symptoms of PTSD including re-experiencing, avoidance and sense of current threat and functional impairment. For the purposes of this current study we used only the first 9 items to assess PTSD. For the diagnostic algo­ rithm see Cloitre et al. (2018) Furthermore, we assessed subjective wellbeing (WHO-five wellbeing index, WHO-5, Bech et al., 2003) The total raw score, ranging from 0 to 25, is Aeschlimann, Gordillo, Ueno et al. 7 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ multiplied by 4: 0 indicates worst well-being and 100 indicates the best well-being. Daily mobile app data regarding grief symptoms was collected using the Standard Subscale from the IPGDS (13 items) which was recently psychometrically validated in terms of reliability and validity (Killikelly et al., 2020). Previous research using similar daily sampling or experience sampling methodology has purported the importance of allowing participants the option to personalize the assessment method and tailor questions based on their current needs or experiences. Here we pilot the use of personalized assessment by including three idiosyncratic, personally relevant items chosen from the cultural supplement alongside the standard scale of 13 items (van Os et al., 2017). Feasibility, Clinical Utility and Acceptability Outcomes To examine the feasibility of the intervention, we analyzed usage data, specifically (i) percent of participant completing all entries (ii) average entries completed per week. Ac­ ceptability was assessed during an unstructured exit interview, where participants were asked about their experience using mGAGE and their suggestions for improvement in a potential updated version. Additionally, we employed a questionnaire on acceptability to assess the quality of health-related mobile apps at t2 (mobile app rating scale, MARS, Stoyanov et al., 2015). The MARS evaluates the quality of mobile apps (engagement, functionality, visual aesthetics, information quality and subjective quality subscales) on a scale from 1 (Inadequate) to 5 (Excellent). Procedure Preceding t1, participants were sent an e-mail confirming their appointment for t1, including the study information sheet, the informed consent form, directions to the De­ partment of Psychology and an information sheet about the mGAGE app. Additionally, participants received a reminder e-mail one day before t1. Assessments were conducted in our offices by the project manager or a Psychology student, who received training on administering the assessment. Participants provided written informed consent. Participants then completed a questionnaire battery about their current mental health status and grief symptoms. Subsequently the participants had the option to select up to three items from the cultural supplement of the IPGDS to be included in their online questionnaire on the mGAGE app. A brief introduction to the app was given where a team member helped participants download the app and create a personal user ID, as well as elaborating all app functionalities and answering any questions participants had (see Supplementary Materials on mobile app information). Participants were instructed to complete the IPGDS within the mGAGE app once a day for three weeks. The IPGDS in this case meant the Standard Subscale of the IPGDS and the potential maximum of 3 additional individual items selected from the cultural supplement of the IPGDS by the participants. Additionally, an information sheet, summa­ mGAGE Mobile Assessment for Grief 8 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ rizing all app functionalities was provided. Furthermore, all participants received a list of resources and links for bereaved individuals. Three weeks after t1, participants were invited to return for the post-assessment t2 and received a reminder e-mail one day before t2. At t2, participants were asked to com­ plete the same questionnaires as for t1 and an additional questionnaire concerning their experience with the mGAGE app. Upon completion, a brief unstructured exit interview was conducted assessing the acceptability of the app. After this, participants received a short debriefing to discuss their experience of participating in the study. Analyses All statistical analyses were conducted in SPSS version 24. Inspection of histograms and the Kolmogorov–Smirnov test statistic (i.e., significance indicates that the distribution of the data significantly differs from a normal distribution) was used to determine whether parametric or non-parametric testing was appropriate. The main outcome measures for grief (IPGDS t1 and t2) were normally distributed while all other measures were normal­ ly distributed at one time point. To ensure consistency, the results of the parametric tests (paired samples t-tests) were confirmed with non-parametric tests (Mann-Whitney- Wilcoxon test). Feasibility and acceptability variables included percentage of participants completing the daily entry and averaged over the week. The debriefing interview was analyzed using qualitative thematic analysis. Firstly, the interview data was in vivo transcribed into short relevant sentences and translated into English. Secondly the text was coded and grouped into large categories following iterative categorization (Neale, 2016). Finally, the codes and themes were reviewed by CK and AA and final consensus codes were determined. Results Table 1 presents the demographic characteristics of the participants. Participants were mostly University educated (40.7%) young (average age 25.7) women (88.9%). Time since loss ranged from 6 months to more than 10 years. Almost 30% of participants had pre­ vious experience with psychotherapy. Mean scores on the mental health outcome meas­ ures are presented in Table 2. Paired sample t-tests revealed no significant differences between t1 and t2 on all mental health measures, except for the IPGDS standard scale (13 items) (t1 mean 21.3 vs. t2 mean 18.5, p = .002). Table 3 presents the diagnostic algorithm findings. None of the participants met criteria for a strict diagnosis of PGD, while one met criteria for moderate PGD at t1 and two participants at t2. Three participants met criteria for PTSD at t2. Aeschlimann, Gordillo, Ueno et al. 9 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ Table 1 Sociodemographic, Loss-Related and Symptom Characteristics Variable Total sample (n = 27) n % Gender Male 3 11.1 Female 24 88.9 Education Maturaa 16 59.3 College/university 11 40.7 Relationship status Singleb 26 96.3 Married 1 3.7 Time since loss 6 to 12 months ago 8 29.6 1 to 5 years ago 13 48.1 10 to 20 years ago 2 7.4 Other Previous psychotherapy for griefc 8 29.7 Current psychotherapyc 2 7.4 Note. N = 27. Participants were on average 25.70 years old (SD = standard deviation). aEquivalent to high school education or above in Switzerland. bIs defined here as non-married individuals. cReflects the number and percentage of participants answering “yes” to this question. In order to assess the feasibility of mGAGE use, variables related to adherence to the app were examined. Only one participant completed all entries over the total possible 21 days. Up to Day 15 all participants completed the required daily entry. From Day 16 to 21, there was a drop in adherence. As revealed in Figure 3, the average number of entries drops in Week 3 (67.7%) compared to Week 1 and 2 (100%). The average IPGDS score was calculated for Days 1 to 15 (with all data points complete for all participants) to examine the variability in the average score. (See Figures 2 and 3). Paired samples t-test compared the highest scored day (Day 2, 20.0) vs. lowest average scored day (Day 11, 18.52) and confirmed a statistically significant difference (p = .049). mGAGE Mobile Assessment for Grief 10 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ Table 2 Questionnaire Data at Pre (T1) Assessment and Post (T2) Assessment Variable T1 Pre T2 Post Difference test M SD M SD Paired samples t-test PGD sum score (13 standard items) 21.3 6.4 18.5 6.3 .002* PGD cultural supplement 26.5 5.9 26.2 8.7 .779 PHQ9 4.2 2.7 4.4 3.9 .790 GAD7 4.2 3.2 3.8 3.1 .374 ITQ Re 1.3 1.7 1.6 1.9 .387 ITQ Av 1.6 2.2 1.3 2.0 .235 ITQ Th .6 1.0 .7 1.3 .574 WHO wellbeing scale 15.1 5.2 15.4 4.8 .704 PG13 17.6 5.7 16.8 6.4 .204 Note. M = mean; SD = standard deviation; PGD = prolonged grief disorder; PHQ9 = patient health questionnaire 9; GAD7 = generalized anxiety disorder 7; ITQ Re = International Trauma Questionnaire – Reexperiencing subscale; ITQ Av = International Trauma Questionnaire – Avoidance subscale; ITQ Th = International Trauma Questionnaire – Threat subscale; WHO = world health organization wellbeing scale; PG13 = prolonged grief 13. *p < .05. **p < .01. ***p < .001. Table 3 Diagnostic Algorithm Comparison Diagnostic Test T1 Pre T2 Post % n % n PGD Strict 0.0 0.0 PGD Moderate 3.7 1 7.4 2 Maciejewski 2016 criteria 3.7 1 0.0 PTSD 0.0 11.1 3 Note. PGD = prolonged grief disorder; PTSD = posttraumatic stress disorder. Acceptability of mGAGE Overall mean scores on the MARS questionnaire ranged from 3 to 5 indicating average to very good acceptability for the app (across all questions mean score of 4). The lowest scored question was ‘would you pay for this app’ (average 2). The highest scored ques­ tions were ease of use, navigation and gestural designs (all scored average of 5). For the question ‘overall star rating’ (Question 11) participants’ mean score was 4/5. Aeschlimann, Gordillo, Ueno et al. 11 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ Figure 2 Daily Average IPGDS Score for All Participants 17,50 18,00 18,50 19,00 19,50 20,00 20,50 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A ve ra ge Day Average IPGDS score * * Figure 2 Daily average IPGDS score for all participants Note. IPGDS= international prolonged grief disorder scale, * statistically significant differenceNote. IPGDS = International Prolonged Grief Disorder Scale. *Statistically significant difference between average daily IPGDS on Day 2 and Day 11; p < .05. Figure 3 Adherence per WeekFigure 3 Adherence per week 0 20 40 60 80 100 120 Week 1 Week 2 Week 3Pe rc en t o f pa rt ic ip an ts c om pl et ed Week Average Entries per Week Note. Percent of participants completing the daily questionnaireNote. Percent of participants completing the daily questionnaire. mGAGE Mobile Assessment for Grief 12 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ Thematic analysis was conducted on the debriefing interviews. Overall, three broad cate­ gories of responses were identified, each with related themes (see Table 4 for exemplar quotes). Table 4 Summary of Qualitative Feedback and Example Quotes Themes Example quotes Positive Feedback Useful links especially the links to get help were helpful, Self-monitoring and reflection But she liked the app, it helped her to reflect and process her grieving thoughts. Ease of Use easy to use, also good for older people, fulfilled its purpose, Good design the design was very simple but good; Negative Feedback Effect on symptoms He said even though he thought, that he had already processed his grief symptoms, the app made him a bit sad in the beginning Notifications and technical issues the reminders didn't work on a daily basis. Some days the app reminded her, some days not Repetitive After two weeks he thought, that the questions were getting boring and annoying, cause they were always the same Design issues but the design wasn't that appealing Specific recommendations Personalization and timing the participant would have liked the app to react to the questions and ask more specific questions Variation she would have preferred the questions to be in a different order each time, cause it was a bit repetitive Other functions tips like breathing exercises for when score is red (instead of just contact numbers) IPGDS changes the IPGDS should have more gradations or possibly a slider bar for more sensitivity Chart improvement it would be nice to have a summary of the scores if the app should be used for a longer period of time (e.g. a smaller graph) Other Even though she liked the app, she wouldn't pay for it. Her grief symptoms aren't as strong anymore as they used to be, that's why she didn't find the questions disturbing. Note. IPGDS = International Prolonged Grief Disorder Scale. Aeschlimann, Gordillo, Ueno et al. 13 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ The first category was ‘positive feedback’ with related themes including useful links, self- monitoring and reflection, ease of use, and good design. The second category was ‘negative feedback’ with related themes including effect on symptoms, notifications and technical issues, repetitive, and design issues. The third category ‘specific recommendations’ inclu­ ded related themes such as personalization and timing, variation, other functions, IPGDS changes, chart improvement. Discussion Main Findings This pilot study provides preliminary support for the feasibility and acceptability of the mGAGE app for use by bereaved individuals. Feasibility, assessed through adherence to the app, was extremely high for the first two weeks of use (100%). Acceptability, assessed by high ratings on the mobile app rating scale (MARS) questionnaire (mean score of 4/5), was also confirmed. Qualitative feedback from debriefing interviews revealed sev­ eral themes supporting the acceptability of the app (useful links, self-monitoring and reflection, ease of use, and good design). However, participants also identified possible negative effects on mood as well as specific recommendations for improving the app. The final aim of this study was to explore variability in grief symptoms over two weeks. Here we confirmed high variability (a statistically significant difference between highest and lowest mean scores) over the two weeks. This has important implications for better understanding variation and intensity of grief scores in real time as outlined below. Additionally, the process of self-monitoring may have a therapeutic effect. At t2 participants had significantly lower scores on the International Prolonged Grief Dis­ order Scale (IPGDS). This significant reduction in symptoms may indicate that online self-monitoring is a beneficial intervention and could be used to supplement face to face therapy, which is in accordance with results from previous studies (e.g. Bakker & Rickard, 2018; Kauer et al., 2012). However, this result should be interpreted with caution as no control group was included in this study. The finding that one participant met moderate diagnostic criteria for PGD and two participants met criteria for PTSD at T2 although not at T1 attests to the need for a control group to unpick the effect of PGD symptom monitoring compared to other possible confounding or moderating effects on T2 outcomes. Previous research has confirmed that adherence to e-mental health apps wavers. Response rates of around 60-80% have been found in previous e-mental health studies of depression (70%; Putnam & McSweeney, 2008), substance misuse (88.8%; Phillips et al., 2014), and trauma (67.5%; Dewey et al., 2015). Evidently the current study found excep­ tionally high adherence; 100% of responses completed by all participants. However, after two weeks adherence declines substantially (Week 3 average response 67.7%). This is also mGAGE Mobile Assessment for Grief 14 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ found in other areas of e-mental health. After an initial burst of interest participants typically decrease their use of health apps after two weeks (Dorsey et al., 2017). Reasons for decline in use include poor user centered design, lack of incentive and decreased internal motivation (Torous et al., 2019). Importantly, it should be noted that many participants mentioned technical problems as a reason for decline in use. The current study developed the mGAGE app following a user centered iterative design and we plan to build on the current findings to redesign elements of the app to improve acceptability. Mohr et al. (2011) developed a model of ‘supportive accountability’ which seeks to improve adherence to e-mental health interventions by adding person-to-person contact throughout the intervention. They argue that participants are more likely to continue the intervention if they experience contact with a ‘coach’ who is supportive, trustworthy and kind. To improve adherence to the current app we would consider adding contact with a weekly coach to check-in and provide support and encouragement. However, based on the current findings we may consider limiting the required use of the app to two weeks. After two weeks participants have provided daily real-time data on their grief symptoms. This may be enough data to capture a comprehensive clinical picture for further assessment and treatment by a clinician. Additionally, this may provide the individual with more insight into their current symptoms. One interesting finding of the current study was the possible therapeutic effect of self-monitoring. Previous literature has revealed that the repeated act of completing an outcome measure may be an intervention in itself (Amble et al., 2015). This study found that participants had decreased grief scores at t2. Previous studies have also found that daily self-monitoring may improve mental health symptoms. For example, after 30 days of mood monitoring with a new mobile app, MoodPrism, participants experienced a significant decrease in symptoms of depression and anxiety, and this was directly related to app use (Bakker & Rickard, 2018). The act of daily self-reflection and improved insight into symptoms is the goal of several cognitive-behavioral therapy interventions for grief and other disorders (Boelen et al., 2011; Kavanagh, 1990; Spuij et al., 2015). These are found to be predictors of positive affect, cognitive reappraisal as well as emotional self-awareness (Kauer et al., 2012; O’Toole et al., 2014). Emotional self-awareness is thought to be a key factor that may improve self-regulation and wellbeing (Barrett & Gross, 2001). Overall participants rated the app as highly acceptable. However, they identified some important areas for improvement. Several participants identified that daily self- monitoring may not have a positive effect on mood, but instead remind bereaved individuals of their sadness and grief. One participant identified that she thought she experienced more negative symptoms after using the app daily. Another worried that it could be difficult for people to be reminded of their loss everyday (see Table 4). This is an important consideration and will be an important topic for the next phase of research, particularly when conducting research with a clinical sample (Wykes & Brown, 2016). Aeschlimann, Gordillo, Ueno et al. 15 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ As mentioned above, we would consider adding the model of supportive accountability, not only to improve intrinsic motivation to complete the app but to offer participants support, as well as for risk assessment. Clinical Implications and Future Research Overall, the findings of this pilot study confirm that the mGAGE app may have an important use in future randomized controlled trials and grief interventions. Firstly, it can deliver real time daily data on symptom variability and intensity, which provides researchers with a tool to accurately investigate the heterogeneity in the fluctuations of PGD symptoms over time in future research (Bonanno & Malgaroli, 2020; Sveen et al., 2018). This may also be used to inform clinical assessments and treatment options by providing more depth and richer information than a singular self-report assessment. Secondly, daily monitoring can be used to track the effectiveness of grief interventions. As change in symptom severity may vary and effects may waver, mGAGE could be used as an addition to standard, face-to-face therapy and help to track progress and effective­ ness, as suggested by Torous and Roux (2017). It could also be used in conjunction with existing online grief interventions (Wagner et al., 2006). Additionally, mGAGE has the potential to aid clinicians in identifying individuals with PGD. By facilitating early iden­ tification of cases and accelerating access to appropriate treatment, it could also prevent hospitalization or relapses (Whelan et al., 2015). Thirdly, the act of self-monitoring may be a useful intervention by increasing emotional self-awareness. Developing insight into the severity and variability of symptoms may be an effective therapeutic tool (Bakker & Rickard, 2018). Limitations This study was limited in the following ways. The sample size was small and homoge­ neous. It was a largely female sample of the same age group and education level. We did not include individuals from a clinical sample. This is the next required step to ensure acceptability for PGD diagnosis. There were also several technical issues with the app (such as server unavailable, only worked with internet connection) that may have prevented optimal data collection. In terms of the qualitative debriefing interviews, there may have been a social desirability effect as the participants were not blinded to the interviewer. The finding that two participants met criteria for PTSD or moderate PGD at T2 needs further investigation. Currently no participants at t1 or t2 met ‘strict’ criteria for PGD is reassuring as our intention was not to investigate a clinical sample in this pilot study. The diagnostic algorithm for PGD is currently under debate with no clear consensus on whether the ‘strict’ ‘moderate’ or another algorithm for ICD-11 PGD may yield the most reliable and valid diagnosis (Boelen & Lenferink, 2020). In the present study if a participant met criteria for moderate PGD, currently this does not necessarily mGAGE Mobile Assessment for Grief 16 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://www.psychopen.eu/ indicate disorder. However, it may mean elevated symptoms. This should be monitored and follow up with a case control analysis or an RCT including a control group. Another significant limitation in the sample that requires follow up is the heterogeneity in the time criteria. Time since death is a significant predictor of grief symptom severity and although all included participants experienced a death more than 6 months ago, there was still large variability in the duration since loss. The impact of time since loss on daily sampling and grief symptom variability should be assessed in future studies. Funding: CK and TU have established a research partnership to develop a mobile app. This is funded by the Innovation Partnership Grant, IPG 05-122017 from the University of Zurich. This study is part of a larger study in collaboration with SUSMED inc. Acknowledgments: We would like to acknowledge the participation of bereaved participants during the COVID-19 pandemic and the hard work of interns at the Universtiy of Zurich. Competing Interests: AA, CK, NG and AM report no conflict of interest. Taro Ueno is member of Japanese based startup company, SUSMED inc. Author Contributions: CK and AA wrote the manuscript and conducted the study. NG developed the app. AM and TU advised on the study and the project management. Ethics Statement: Ethical approval was obtained from the University of Zurich. Supplementary Materials The Supplementary Materials contain the following item (for access, see Aeschlimann et al., 2024): • Information pamphlet in German and English describing the functions and features of the mGAGE app for prolonged grief disorder. Index of Supplementary Materials Aeschlimann, A., Gordillo, N., Ueno, T., Maercker, A., & Killikelly, C. (2024). Supplementary materials to "Feasibility and acceptability of a mobile app for prolonged grief disorder symptoms" [Information pamphlet]. PsychOpen GOLD. https://doi.org/10.23668/psycharchives.14238 References Amble, I., Gude, T., Stubdal, S., Andersen, B. J., & Wampold, B. E. (2015). The effect of implementing the Outcome Questionnaire-45.2 feedback system in Norway: A multisite randomized clinical trial in a naturalistic setting. Psychotherapy Research, 25(6), 669–677. https://doi.org/10.1080/10503307.2014.928756 Aeschlimann, Gordillo, Ueno et al. 17 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://doi.org/10.23668/psycharchives.14238 https://doi.org/10.1080/10503307.2014.928756 https://www.psychopen.eu/ Bakker, D., & Rickard, N. (2018). Engagement in mobile phone app for self-monitoring of emotional wellbeing predicts changes in mental health: MoodPrism. Journal of Affective Disorders, 227, 432–442. https://doi.org/10.1016/j.jad.2017.11.016 Barrett, L. F., & Gross, J. J. (2001). Emotional intelligence: A process model of emotion representation and regulation. In T. J. Mayne & G. A. Bonanno (Eds.), Emotions: Current issues and future directions (pp. 286–310). Guilford Press. Bech, P., Olsen, L., Kjoller, M., & Rasmussen, N. (2003). Measuring well-being rather than the absence of distress symptoms: A comparison of the SF-36 Mental Health subscale and the WHO-Five well-being scale. International Journal of Methods in Psychiatric Research, 12(2), 85– 91. https://doi.org/10.1002/mpr.145 Birchwood, M., Spencer, E., & McGovern, D. (2000). Schizophrenia: Early warning signs. Advances in Psychiatric Treatment, 6(2), 93–101. https://doi.org/10.1192/apt.6.2.93 Boelen, P. A., van den Hout, M. A., & Bout, J. V. D. (2006). A cognitive-behavioral conceptualization of complicated grief. Clinical Psychology: Science and Practice, 13(2), 109–128. https://doi.org/10.1111/j.1468-2850.2006.00013.x Boelen, P. A., de Keijser, J., van den Hout, M. A., & van den Bout, J. (2011). Factors associated with outcome of cognitive-behavioural therapy for complicated grief: A preliminary study. Clinical Psychology & Psychotherapy, 18(4), 284–291. https://doi.org/10.1002/cpp.720 Boelen, P. A., & Lenferink, L. I. M. (2020). Comparison of six proposed diagnostic criteria sets for disturbed grief. Psychiatry Research, 285, Article 112786. https://doi.org/10.1016/j.psychres.2020.112786 Bonanno, G. A., & Malgaroli, M. (2020). Trajectories of grief: Comparing symptoms from the DSM-5 and ICD-11 diagnoses. Depression and Anxiety, 37(1), 17–25. https://doi.org/10.1002/da.22902 Breen, L. J., & O’Connor, M. (2007). The fundamental paradox in the grief literature: A critical reflection. Omega, 55(3), 199–218. https://doi.org/10.2190/OM.55.3.c Breslau, N., Kessler, R. C., Chilcoat, H. D., Schultz, L. R., Davis, G. C., & Andreski, P. (1998). Trauma and posttraumatic stress disorder in the community: The 1996 Detroit Area Survey of Trauma. Archives of General Psychiatry, 55(7), 626–632. https://doi.org/10.1001/archpsyc.55.7.626 Bryant, R. A., Kenny, L., Joscelyne, A., Rawson, N., Maccallum, F., Cahill, C., Hopwood, S., Aderka, I., & Nickerson, A. (2014). Treating prolonged grief disorder: A randomized clinical trial. JAMA Psychiatry, 71(12), 1332–1339. https://doi.org/10.1001/jamapsychiatry.2014.1600 Chandrashekar, P. (2018). Do mental health mobile apps work: Evidence and recommendations for designing high-efficacy mental health mobile apps. mHealth, 4(3), Article 6. https://doi.org/10.21037/mhealth.2018.03.02 Cloitre, M., Shevlin, M., Brewin, C. R., Bisson, J. I., Roberts, N. P., Maercker, A., Karatzias, T., & Hyland, P. (2018). The International Trauma Questionnaire: Development of a self-report measure of ICD-11 PTSD and complex PTSD. Acta Psychiatrica Scandinavica, 138(6), 536–546. https://doi.org/10.1111/acps.12956 mGAGE Mobile Assessment for Grief 18 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://doi.org/10.1016/j.jad.2017.11.016 https://doi.org/10.1002/mpr.145 https://doi.org/10.1192/apt.6.2.93 https://doi.org/10.1111/j.1468-2850.2006.00013.x https://doi.org/10.1002/cpp.720 https://doi.org/10.1016/j.psychres.2020.112786 https://doi.org/10.1002/da.22902 https://doi.org/10.2190/OM.55.3.c https://doi.org/10.1001/archpsyc.55.7.626 https://doi.org/10.1001/jamapsychiatry.2014.1600 https://doi.org/10.21037/mhealth.2018.03.02 https://doi.org/10.1111/acps.12956 https://www.psychopen.eu/ Dewey, D., McDonald, M. K., Brown, W. J., Boyd, S. J., Bunnell, B. E., & Schuldberg, D. (2015). The impact of ecological momentary assessment on posttraumatic stress symptom trajectory. Psychiatry Research, 230(2), 300–303. https://doi.org/10.1016/j.psychres.2015.09.009 Dorsey, E. R., “Yvonne” Chan, Y.-F., McConnell, M. V., Shaw, S. Y., Trister, A. D., & Friend, S. H. (2017). The use of smartphones for health research. Academic Medicine, 92(2), 157–160. https://doi.org/10.1097/ACM.0000000000001205 Eklund, R., Eisma, M. C., Boelen, P. A., Arnberg, F. K., & Sveen, J. (2021). Mobile app for prolonged grief among bereaved parents: Study protocol for a randomised controlled trial. BMJ Open, 11(12), Article e052763. https://doi.org/10.1136/bmjopen-2021-052763 Eklund, R., Eisma, M. C., Boelen, P. A., Arnberg, F. K., & Sveen, J. (2022). My Grief app for prolonged grief in bereaved parents: A pilot study. Frontiers in Psychiatry, 13, Article 872314. https://doi.org/10.3389/fpsyt.2022.872314 Fujisawa, D., Miyashita, M., Nakajima, S., Ito, M., Kato, M., & Kim, Y. (2010). Prevalence and determinants of complicated grief in general population. Journal of Affective Disorders, 127(1-3), 352–358. https://doi.org/10.1016/j.jad.2010.06.008 Gleeson, J. F., Rawlings, D., Jackson, H. J., & McGorry, P. D. (2005). Early warning signs of relapse following a first episode of psychosis. Schizophrenia Research, 80(1), 107–111. https://doi.org/10.1016/j.schres.2005.07.019 Huber, M., Knottnerus, J. A., Green, L., van der Horst, H., Jadad, A. R., Kromhout, D., Leonard, B., Lorig, K., Loureiro, M. I., van der Meer, J. W. M., Schnabel, P., Smith, R., van Weel, C., & Smid, H. (2011). How should we define health? BMJ, 343, Article d4163. https://doi.org/10.1136/bmj.d4163 Johannsen, M., Damholdt, M. F., Zachariae, R., Lundorff, M., Farver-Vestergaard, I., & O’Connor, M. (2019). Psychological interventions for grief in adults: A systematic review and meta-analysis of randomized controlled trials. Journal of Affective Disorders, 253, 69–86. https://doi.org/10.1016/j.jad.2019.04.065 Kauer, S. D., Reid, S. C., Crooke, A. H. D., Khor, A., Hearps, S. J. C., Jorm, A. F., Sanci, L., & Patton, G. (2012). Self-monitoring using mobile phones in the early stages of adolescent depression: Randomized controlled trial. Journal of Medical Internet Research, 14(3), Article e67. https://doi.org/10.2196/jmir.1858 Kavanagh, D. J. (1990). Towards a cognitive-behavioural intervention for adult grief reactions. The British Journal of Psychiatry, 157(3), 373–383. https://doi.org/10.1192/bjp.157.3.373 Keeley, J. W., Reed, G. M., Roberts, M. C., Evans, S. C., Robles, R., Matsumoto, C., Brewin, C. R., Cloitre, M., Perkonigg, A., Rousseau, C., Gureje, O., Lovell, A. M., Sharan, P., & Maercker, A. (2016). Disorders specifically associated with stress: A case-controlled field study for ICD-11 mental and behavioural disorders. International Journal of Clinical and Health Psychology, 16(2), 109–127. https://doi.org/10.1016/j.ijchp.2015.09.002 Kersting, A., Brähler, E., Glaesmer, H., & Wagner, B. (2011). Prevalence of complicated grief in a representative population-based sample. Journal of Affective Disorders, 131(1-3), 339–343. https://doi.org/10.1016/j.jad.2010.11.032 Aeschlimann, Gordillo, Ueno et al. 19 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://doi.org/10.1016/j.psychres.2015.09.009 https://doi.org/10.1097/ACM.0000000000001205 https://doi.org/10.1136/bmjopen-2021-052763 https://doi.org/10.3389/fpsyt.2022.872314 https://doi.org/10.1016/j.jad.2010.06.008 https://doi.org/10.1016/j.schres.2005.07.019 https://doi.org/10.1136/bmj.d4163 https://doi.org/10.1016/j.jad.2019.04.065 https://doi.org/10.2196/jmir.1858 https://doi.org/10.1192/bjp.157.3.373 https://doi.org/10.1016/j.ijchp.2015.09.002 https://doi.org/10.1016/j.jad.2010.11.032 https://www.psychopen.eu/ Killikelly, C., & Maercker, A. (2017). Prolonged grief disorder for ICD-11: The primacy of clinical utility and international applicability. European Journal of Psychotraumatology, 8(sup6), Article 1476441. https://doi.org/10.1080/20008198.2018.1476441 Killikelly, C., Zhou, N., Merzhvynska, M., Stelzer, E.-M., Dotschung, T., Rohner, S., Sun, L. H., & Maercker, A. (2020). Development of the International Prolonged Grief Disorder Scale for the ICD-11: Measurement of core symptoms and culture items adapted for Chinese and German- speaking samples. Journal of Affective Disorders, 277, 568–576. https://doi.org/10.1016/j.jad.2020.08.057 Kohn, R., Saxena, S., Levav, I., & Saraceno, B. (2004). The treatment gap in mental health care. Bulletin of the World Health Organization, 82, 858–866. Kroenke, K., Spitzer, R. L., & Williams, J. B. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613. https://doi.org/10.1046/j.1525-1497.2001.016009606.x Lenferink, L. I. M., Boelen, P. A., Smid, G. E., & Paap, M. C. S. (2021). The importance of harmonising diagnostic criteria sets for pathological grief. The British Journal of Psychiatry, 219(3), 473–476. https://doi.org/10.1192/bjp.2019.240 Lenferink, L. I. M., van Eersel, J. H. W., & Franzen, M. (2022). Is it acceptable and feasible to measure prolonged grief disorder symptoms in daily life using experience sampling methodology? Comprehensive Psychiatry, 119, Article 152351. https://doi.org/10.1016/j.comppsych.2022.152351 Lundorff, M., Holmgren, H., Zachariae, R., Farver-Vestergaard, I., & O’Connor, M. (2017). Prevalence of prolonged grief disorder in adult bereavement: A systematic review and meta- analysis. Journal of Affective Disorders, 212, 138–149. https://doi.org/10.1016/j.jad.2017.01.030 Maciejewski, P. K., Maercker, A., Boelen, P. A., & Prigerson, H. (2016). “Prolonged grief disorder” and “persistent complex bereavement disorder”, but not “complicated grief”, are one and the same diagnostic entity: An analysis of data from the Yale Bereavement Study. World Psychiatry, 15(3), 266–275. https://doi.org/10.1002/wps.20348 Maercker, A., Brewin, C., & Bryant, R. (2013). Proposals for mental disorders specifically associated with stress in the International Classification of Diseases-11. Lancet, 381(9878), 1683–1685. https://doi.org/10.1016/S0140-6736(12)62191-6 Maercker, A., Forstmeier, S., Enzler, A., Krüsi, G., Hörler, E., Maier, C., & Ehlert, U. (2008). Adjustment disorders, posttraumatic stress disorder, and depressive disorders in old age: Findings from a community survey. Comprehensive Psychiatry, 49(2), 113–120. https://doi.org/10.1016/j.comppsych.2007.07.002 Mintz, E., Toner, E. R., Skolnik, A., Pan, A. Y., Frumkin, M., Baker, A., Simon, N., & Robinaugh, D. (2023). Ecological momentary assessment with bereaved adults: Feasibility, acceptability, and measurement reactivity and their relationship to prolonged grief symptom severity. PsyArXiv. https://doi.org/10.31234/osf.io/hw7jm mGAGE Mobile Assessment for Grief 20 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://doi.org/10.1080/20008198.2018.1476441 https://doi.org/10.1016/j.jad.2020.08.057 https://doi.org/10.1046/j.1525-1497.2001.016009606.x https://doi.org/10.1192/bjp.2019.240 https://doi.org/10.1016/j.comppsych.2022.152351 https://doi.org/10.1016/j.jad.2017.01.030 https://doi.org/10.1002/wps.20348 https://doi.org/10.1016/S0140-6736(12)62191-6 https://doi.org/10.1016/j.comppsych.2007.07.002 https://doi.org/10.31234/osf.io/hw7jm https://www.psychopen.eu/ Mohr, D. C., Cuijpers, P., & Lehman, K. (2011). Supportive accountability: A model for providing human support to enhance adherence to eHealth interventions. Journal of Medical Internet Research, 13(1), Article e30. https://doi.org/10.2196/jmir.1602 Myin-Germeys, I., Birchwood, M., & Kwapil, T. (2011). From environment to therapy in psychosis: A real-world momentary assessment approach. Schizophrenia Bulletin, 37(2), 244–247. https://doi.org/10.1093/schbul/sbq164 Neale, J. (2016). Iterative categorization (IC): A systematic technique for analysing qualitative data. Addiction, 111(6), 1096–1106. https://doi.org/10.1111/add.13314 Olff, M. (2015). Mobile mental health: A challenging research agenda. European Journal of Psychotraumatology, 6(s4), Article 27882. https://doi.org/10.3402/ejpt.v6.27882 O’Toole, M. S., Jensen, M. B., Fentz, H. N., Zachariae, R., & Hougaard, E. (2014). Emotion differentiation and emotion regulation in high and low socially anxious individuals: An experience-sampling study. Cognitive Therapy and Research, 38(4), 428–438. https://doi.org/10.1007/s10608-014-9611-2 Palmier-Claus, J. (2011). The clinical uses of momentary assessment. Acta Psychiatrica Scandinavica, 124(4), 241–242. https://doi.org/10.1111/j.1600-0447.2011.01761.x Phillips, M. M., Phillips, K. T., Lalonde, T. L., & Dykema, K. R. (2014). Feasibility of text messaging for Ecological Momentary Assessment of marijuana use in college students. Psychological Assessment, 26(3), 947–957. https://doi.org/10.1037/a0036612 Prigerson, H. G., Horowitz, M. J., Jacobs, S. C., Parkes, C. M., Aslan, M., Goodkin, K., Raphael, B., Marwit, S. J., Wortman, C., Neimeyer, R. A., Bonanno, G., Block, S. D., Kissane, D., Boelen, P., Maercker, A., Litz, B. T., Johnson, J. G., First, M. B., & Maciejewski, P. K. (2009). Prolonged grief disorder: Psychometric validation of criteria proposed for DSM-V and ICD-11. PLoS Medicine, 6(8), Article e1000121. https://doi.org/10.1371/journal.pmed.1000121 Putnam, K. M., & McSweeney, L. B. (2008). Depressive symptoms and baseline prefrontal EEG alpha activity: A study utilizing Ecological Momentary Assessment. Biological Psychology, 77(2), 237–240. https://doi.org/10.1016/j.biopsycho.2007.10.010 Schnall, R., Rojas, M., Bakken, S., Brown, W., Carballo-Dieguez, A., Carry, M., Gelaude, D., Mosley, J. P., & Travers, J. (2016). A user-centered model for designing consumer mobile health (mHealth) applications (apps). Journal of Biomedical Informatics, 60, 243–251. https://doi.org/10.1016/j.jbi.2016.02.002 Shear, K., Frank, E., Houck, P. R., & Reynolds, C. F. (2005). Treatment of complicated grief: A randomized controlled trial. Journal of the American Medical Association, 293(21), 2601–2608. https://doi.org/10.1001/jama.293.21.2601 Shear, M. K., Reynolds, C. F., Simon, N. M., Zisook, S., Wang, Y., Mauro, C., Duan, N., Lebowitz, B., & Skritskaya, N. (2016). Optimizing treatment of complicated grief. JAMA Psychiatry, 73(7), 685–694. https://doi.org/10.1001/jamapsychiatry.2016.0892 Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097. https://doi.org/10.1001/archinte.166.10.1092 Aeschlimann, Gordillo, Ueno et al. 21 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://doi.org/10.2196/jmir.1602 https://doi.org/10.1093/schbul/sbq164 https://doi.org/10.1111/add.13314 https://doi.org/10.3402/ejpt.v6.27882 https://doi.org/10.1007/s10608-014-9611-2 https://doi.org/10.1111/j.1600-0447.2011.01761.x https://doi.org/10.1037/a0036612 https://doi.org/10.1371/journal.pmed.1000121 https://doi.org/10.1016/j.biopsycho.2007.10.010 https://doi.org/10.1016/j.jbi.2016.02.002 https://doi.org/10.1001/jama.293.21.2601 https://doi.org/10.1001/jamapsychiatry.2016.0892 https://doi.org/10.1001/archinte.166.10.1092 https://www.psychopen.eu/ Spuij, M., Dekovic, M., & Boelen, P. A. (2015). An open trial of ‘grief-help’: A cognitive-behavioural treatment for prolonged grief in children and adolescents. Clinical Psychology & Psychotherapy, 22(2), 185–192. https://doi.org/10.1002/cpp.1877 Stelzer, E.-M., Zhou, N., Maercker, A., O’Connor, M.-F., & Killikelly, C. (2020). Prolonged grief disorder and the cultural crisis. Frontiers in Psychology, 10, Article 2982. https://doi.org/10.3389/fpsyg.2019.02982 Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., & Mani, M. (2015). Mobile App Rating Scale: A new tool for assessing the quality of health mobile apps. JMIR mHealth and uHealth, 3(1), Article e27. https://doi.org/10.2196/mhealth.3422 Sveen, J., Johannesson, K. B., Cernvall, M., & Arnberg, F. K. (2018). Trajectories of prolonged grief one to six years after a natural disaster. PLoS One, 13(12), Article e0209757. https://doi.org/10.1371/journal.pone.0209757 Torous, J., Andersson, G., Bertagnoli, A., Christensen, H., Cuijpers, P., Firth, J., Haim, A., Hsin, H., Hollis, C., Lewis, S., Mohr, D. C., Pratap, A., Roux, S., Sherrill, J., & Arean, P. A. (2019). Towards a consensus around standards for smartphone apps and digital mental health. World Psychiatry, 18(1), 97–98. https://doi.org/10.1002/wps.20592 Torous, J., & Roux, S. (2017). Patient-driven innovation for mobile mental health technology: Case report of symptom tracking in schizophrenia. JMIR Mental Health, 4(3), Article e7911. https://doi.org/10.2196/mental.7911 van Os, J., Verhagen, S., Marsman, A., Peeters, F., Bak, M., Marcelis, M., Drukker, M., Reininghaus, U., Jacobs, N., Lataster, T., Simons, C., Lousberg, R., Gülöksüz, S., Leue, C., Groot, P. C., Viechtbauer, W., & Delespaul, P. (2017). The experience sampling method as an mHealth tool to support self-monitoring, self-insight, and personalized health care in clinical practice. Depression and Anxiety, 34(6), 481–493. https://doi.org/10.1002/da.22647 Wagner, B. (2013). Komplizierte Trauer. Springer. Wagner, B., Knaevelsrud, C., & Maercker, A. (2006). Internet-based cognitive-behavioral therapy for complicated grief: A randomized controlled trial. Death Studies, 30(5), 429–453. https://doi.org/10.1080/07481180600614385 Wakefield, J. C. (2012). Should prolonged grief be reclassified as a mental disorder in DSM-5? Reconsidering the empirical and conceptual arguments for complicated grief disorder. The Journal of Nervous and Mental Disease, 200(6), 499–511. https://doi.org/10.1097/NMD.0b013e3182482155 Wang, K., Varma, D. S., & Prosperi, M. (2018). A systematic review of the effectiveness of mobile apps for monitoring and management of mental health symptoms or disorders. Journal of Psychiatric Research, 107, 73–78. https://doi.org/10.1016/j.jpsychires.2018.10.006 Whelan, P., Machin, M., Lewis, S., Buchan, I., Sanders, C., Applegate, E., Stockton, C., Preston, S., Bowen, R. A., Ze, Z., Roberts, C., Davies, L., Wykes, T., Tarrier, N., Kapur, S., & Ainsworth, J. (2015). Mobile early detection and connected intervention to coproduce better care in severe mental illness. In I. N. Sarkar, A. Georgiou, & P. M. Azevedo Marques (Eds.), MEDINFO 2015: eHealth-enabled Health (pp. 123-126). https://doi.org/10.3233/978-1-61499-564-7-123 mGAGE Mobile Assessment for Grief 22 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://doi.org/10.1002/cpp.1877 https://doi.org/10.3389/fpsyg.2019.02982 https://doi.org/10.2196/mhealth.3422 https://doi.org/10.1371/journal.pone.0209757 https://doi.org/10.1002/wps.20592 https://doi.org/10.2196/mental.7911 https://doi.org/10.1002/da.22647 https://doi.org/10.1080/07481180600614385 https://doi.org/10.1097/NMD.0b013e3182482155 https://doi.org/10.1016/j.jpsychires.2018.10.006 https://doi.org/10.3233/978-1-61499-564-7-123 https://www.psychopen.eu/ Wichers, M., Simons, C. J. P., Kramer, I. M. A., Hartmann, J. A., Lothmann, C., Myin-Germeys, I., van Bemmel, A. L., Peeters, F., Delespaul, P., & van Os, J. (2011). Momentary assessment technology as a tool to help patients with depression help themselves. Acta Psychiatrica Scandinavica, 124(4), 262–272. https://doi.org/10.1111/j.1600-0447.2011.01749.x Wykes, T., & Brown, M. (2016). Over promised, over-sold and underperforming? – E-health in mental health. Journal of Mental Health, 25(1), 1–4. https://doi.org/10.3109/09638237.2015.1124406 Clinical Psychology in Europe (CPE) is the official journal of the European Association of Clinical Psychology and Psychological Treatment (EACLIPT). PsychOpen GOLD is a publishing service by Leibniz Institute for Psychology (ZPID), Germany. Aeschlimann, Gordillo, Ueno et al. 23 Clinical Psychology in Europe 2024, Vol. 6(1), Article e10881 https://doi.org/10.32872/cpe.10881 https://doi.org/10.1111/j.1600-0447.2011.01749.x https://doi.org/10.3109/09638237.2015.1124406 https://www.psychopen.eu/ mGAGE Mobile Assessment for Grief (Introduction) A Mobile Self-Report Tool to Assess PGD: mGAGE Method Study Design Participants and Recruitment Measures Procedure Analyses Results Acceptability of mGAGE Discussion Main Findings Clinical Implications and Future Research Limitations (Additional Information) Funding Acknowledgments Competing Interests Author Contributions Ethics Statement Supplementary Materials References