Mental health apps in a college setting: openness, usage, and attitudes
Original Article

Mental health apps in a college setting: openness, usage, and attitudes

Adam Kern1, Victor Hong2, Joyce Song2, Sarah Ketchen Lipson3, Daniel Eisenberg4

1Washington University in St. Louis, School of Social Work, St. Louis, MO, USA; 2Department of Psychiatry, University of Michigan, Ann Arbor, MI, USA; 3Department of Health Law Policy and Management, School of Public Health, Boston University, Boston, MA, USA; 4Department of Health Management and Policy, and Population Studies Center, University of Michigan, Ann Arbor, MI, USA

Contributions: (I) Conception and design: J Song, A Kern, V Hong; (II) Administrative support: D Eisenberg, SK Lipson, A Kern; (III) Provision of study material or patients: D Eisenberg, SK Lipson, A Kern; (IV) Collection and assembly of data: SK Lipson, A Kern; (V) Data analysis and interpretation: A Kern, SK Lipson, V Hong; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

Correspondence to: Victor Hong. Department of Psychiatry, University of Michigan, 1500 E. Medical Center Drive, Ann Arbor, MI 48109, USA. Email: vhong@med.umich.edu.

Background: The ubiquity of smartphones and the development of mental health apps (MHAs) calls for evaluation of consumers’ attitudes towards and usage of MHAs. Due to the increasing demand for mental health services on college campuses, research is especially needed to evaluate MHAs as a potentially viable treatment modality in that setting.

Methods: The study team developed survey questions related to MHAs, added these to the Healthy Minds Study, and used Qualtrics as the platform. The participants were 741 students, age 18 and older, from a large Midwest public university. Students could answer a varying number of multiple choice questions based on embedded display logic, and the survey required 20–25 minutes for most participants to complete. Based on embedded display logic in the survey and how questions were answered, students could receive anywhere from 1–20 questions. Questions were primarily categorical (e.g., “Yes”, “Maybe”, “No”), with the remaining questions in free response format. The survey was fielded in April, 2016.

Results: 26.1% of respondents were open to using an MHA yet only 7.3% had used an MHA. 9.0% of respondents preferred to use an MHA versus seeing a mental health professional. 13.2% of respondents felt that MHAs do have an evidence base. 23.8% of users felt that MHAs helped with their mental health. Those who reported receiving mental health services within the past 12 months were significantly more open to using MHAs than those who had not received services. Convenience, immediate availability, and confidentiality were common reasons for interest in MHAs.

Conclusions: There is interest in, but limited usage, of MHAs among university students, providing evidence of MHAs as a potentially untapped treatment modality for this population. Further research could help assess how best to integrate this technology into the university and college mental health system.

Keywords: College students; mobile apps; mental health; treatment; technology


Received: 06 February 2018; Accepted: 17 May 2018; Published: 30 June 2018.

doi: 10.21037/mhealth.2018.06.01


Introduction

Within the last decade, the emergence and growing ubiquity of mobile smartphones has dramatically impacted the personal, social, and occupational routines of the global population (1). Unsurprisingly, numerous mobile apps now exist to help diagnose, track, and manage health conditions, with varying degrees of success (2). The presence of mental health apps (MHAs) has also increased; there are MHAs to address depression, schizophrenia, and bipolar disorder, among other mental health conditions (3-5). Additionally, there are treatment-related MHAs incorporating interventions such as mindfulness and Cognitive Behavioral Therapy (6-8). Further research and collaboration are needed to evaluate these tools and optimize their integration into the broader field of behavioral health care (9-11). The American Psychiatric Association and others recognize the need for a framework for evaluating MHAs and have recently provided recommendations of how to do so (12,13).

The growing American “college mental health crisis” motivates providers and administrators to find innovative solutions to meet students’ mental health needs (14). College and university students are rarely without a smartphone on hand, using them not only to communicate, but also to take photos, search the Internet, engage in social media and access mobile applications (apps).

As providers grapple with dramatic increases in the number of students seeking treatment, the supply of college and university mental health resources often fails to meet the demand, and many students struggle to cope with challenges such as long wait times and limited provider options (15). Mental health issues on campus affect many aspects of well-being and academic functioning. According to the 2015 American College Health Association (ACHA) survey, in a 12-month period, 85% of students felt overwhelmed, 35% felt so depressed that it was difficult to function, while 24%, 15%, and 20% reported anxiety, depression, and sleep difficulties, respectively (16). The traditional models of counseling and appointment management may no longer be adequate for some college students, who may prefer readily available, constantly accessible tools (17). MHAs present a theoretical augmentation strategy for managing mental health issues in college student populations.

Previous research has examined attitudes regarding MHAs in general adult samples (18,19), but attitudes of college students concerning the use of MHAs have not been previously evaluated. Furthermore, there is a paucity of literature regarding any aspect of MHAs in the college population. This article aims to contribute to the growing literature regarding MHAs by presenting outcomes of a study investigating the potential usefulness of MHAs, and attitudes toward using them, among a college student population.


Methods

Participants

Approximately 4,000 students from a large Midwestern Public University were randomly selected from the entire student population and recruited to participate in the Healthy Minds Study (HMS), an annual web-based survey that examines mental health, service utilization, and related issues among undergraduate and graduate students. Those who met the inclusion criteria for the random sample were degree-seeking students ages 18 or older, and these students were compiled and randomized from the participating institution’s Registrar. Given the voluntary nature of this study, not every question was answered by every participant, resulting in some variables having incomplete data. The average time to participate in this study was 20–25 minutes, though this varied considerably due to embedded display logic in the survey (i.e., students could receive anywhere from 1–20 questions in the MHAs section depending on how they answered questions). This study was approved by the institutional review board at the participating institution and administered in April of 2016. The study number was HUM00100169.

Measures

MHA questions

For this study, the HMS team created and piloted a new set of survey questions at this particular university to assess knowledge of and attitudes concerning MHAs. The new questions were administered in the spring 2016 semester. A working definition was displayed at the beginning of the MHA questions to provide a uniform understanding. The study team defined a MHA as, “A mobile application that is designed to track or treat a mental health condition (e.g., depression, anxiety, psychosis, personality disorder, bipolar disorder, insomnia, post-traumatic stress disorder, substance use disorder, eating disorder, ADHD, etc.). This does not include an app that would relate to other aspects of wellness (diet, exercise, stress, general health). Examples of mental health apps include but are not limited to: Talkspace, SAM, Moodkit, PTSD Coach, CBT-I Coach, Operation Reach Out, Optimism.

Primary outcome measures

Select MHA questions were sorted into three categories of interest: “Openness” to MHAs, “Usage” of MHAs, and “Attitudes” toward MHAs. Given the exploratory nature and aim of this study, the researchers felt that highlighting these particular categories of questions were critical to help inform the structure of future studies. The researchers collapsed the variables’ original response options into dichotomous categories in order to better understand the significant relationships revealed by the chi-square tests. The exact wording, display logic, and operationalization of these variables are further outlined in Table S1.

Table S1
Table S1 Primary outcomes: definitions from survey questions
Full table

Student subgroups

Gender included “Male”, “Female”, or “Other”, with “Other” including all students who endorsed a gender option other than “Male” or “Female”. There were a very small number of students in the “Other” gender category; the researchers kept those students’ data to be as inclusive as possible, but generated a binary gender variable consisting of “Male” and “Female” for purposes of analyses. Age was separated into two categories: 18–22, and 23 or older. Race/ethnicity was condensed to “White” versus “Non-white”, with “Non-white” including all students who endorsed a race/ethnicity category other than “White”. Some race/ethnicity categories were not endorsed by an adequate number of students to be able to conduct analyses, explaining the creation and utilization of binary “White” and “Non-white” categories. Student type consisted of “Undergraduate” versus “Graduate”, with “Graduate” including all students who endorsed enrollment in any graduate program.

Mental health was calculated based on students’ scores on the embedded mental health screens in HMS. The three validated screens included depression, measured by the Patient Health Questionnaire-9 (PHQ-9); anxiety, measured by the Generalized Anxiety Disorder-7 (GAD-7); and eating disorders, measured by the five item SCOFF questionnaire (20-22). The mental health screen variable was operationalized as “Positive” versus “Negative”, with “Positive” screens including students scoring over 10 on the PHQ-9, over 10 on the GAD-7, and/or over 2 on the SCOFF. Previous mental health diagnosis was categorized as “Yes” versus “No”, with “Yes” representing students who indicated being previously diagnosed with either depression, anxiety, attention disorder or learning disability, eating disorder, psychosis, personality disorder, and/or substance abuse disorder. Lastly, previous mental health treatment was structured as “Yes” versus “No”, with “Yes” including students who indicated receiving counseling or therapy for mental or emotional health from a health professional within the past 12 months.

Procedure

Administering the study

Qualtrics was used as the survey platform, and students were recruited by email. The data collection period lasted for approximately two and a half weeks, with up to three reminder emails for those had yet to participate or decline to participate. Of the students who consented to participate, 876 students started the survey, and 741 reached the MHA questions. Of the 741 students who arrived at the MHA questions, approximately 721 (97.3%) indicated ownership of a smartphone and were thus asked the MHAs questions. Based on embedded skip logic, the remaining 20 students who did not indicate ownership of a smartphone were not asked the MHA questions. Students with a smartphone were asked if they would be open to using a MHA. Those who answered “No” were directed to two final questions. Students who responded “Yes” or “Maybe” went on to answer 16 additional MHA questions based on embedded display logic (with the exception of the question: “Do you think mental health apps have an evidence base?”, which was asked of all students eligible to answer the remaining MHA questions, regardless of whether they were open to using a MHA). The study team selected this path of embedded logic to specifically gather more data from those students who were actually open to using a MHA. Cash sweepstakes prizes were used as incentives for participation.

Data analysis

Analyses were conducted using Stata statistical software. Using the participating school’s administrative data (e.g., gender, age, race-ethnicity), the researchers generated probability weights to help adjust for nonresponse bias.

Applying the response weights, the study team ran univariate frequency tests to estimate the distribution of responses among all students answering MHA questions in the “Openness”, “Usage”, and “Attitudes” question categories. Bivariate analyses then examined differences across subgroups on these MHA questions through chi-square tests, displayed in Tables 1, 2, and 3, respectively. Given the format of the “free text” questions, which were separate from the “Openness”, “Usage”, and “Attitudes questions, data for these responses were compiled by themes, rather than by specific answers.

Table 1
Table 1 Openness toward mental health apps
Full table
Table 2
Table 2 Use of mental health apps
Full table
Table 3
Table 3 Attitudes toward mental health apps
Full table

Results

Sample characteristics

The analytic sample (see Table 4) included 264 males (36.6%), 445 females (61.7%), and 12 students (1.7%) identifying as something other than “Male” or “Female.” This imbalanced representation of gender (higher female response rate than male) is typical compared to other schools administering HMS. The majority of respondents (N=455, 63.1%) were ages 18 to 22, with 266 (36.9%) students reporting ages 23 or older. Most respondents (N=442, 63.0%) were undergraduate students, and a majority identified as White (N=494, 69.1%).

Table 4
Table 4 Sample characteristics
Full table

Overall openness, usage, and attitudes towards MHAs

Of the students who reported owning a smartphone, 26.1% replied “Yes” to being open to using a MHA, whereas 73.9% replied “Maybe/No”. Within the MHA “Usage” question category (among students owning smartphone and responding “Yes” or “Maybe” to being open to using MHAs), 7.3% of students indicated having used MHAs. 9.0% of students preferred a MHA over seeing a mental health professional if experiencing a mental health condition. Regarding attitudes toward MHAs, 13.2% of students felt that MHAs do have an evidence base, with 86.8% reporting “Maybe/Unsure/No”. Additionally, 17.3% replied that MHAs could possibly improve their academic performance if they were experiencing a mental health condition.

Openness to MHAs among subgroups

As shown in Table 1, a significantly higher percentage of female students (32.9%) were open to MHAs relative to males (19.3%), χ2(1, N=709) =15.5, P<0.001. Those who reported receiving mental health services within the past 12 months were significantly more open (33.6%) than those who had not received services (23.8%), χ2(1, N=720) =6.56, P=0.01. Students with a previous mental health diagnosis were more open to using MHAs (30.9%) compared to 24.9% of students with no previous diagnoses, but the difference was not statistically significant.

Usage of MHAs among subgroups

Students who replied “Yes” or “Maybe” to being open to MHAs were asked whether they had ever used a MHA. Females (9.5%) had used MHAs significantly more than males (4.4%), χ2(1, N=562) =4.9, P=0.03 (see Table 2). Students with a previous mental health diagnosis had a significantly higher usage (13.6%) compared to those with no previous mental health diagnosis (4.2%), χ2(1, N=545) =16.5, P<0.001. Furthermore, only 3.5% of students who did not receive mental health services within the past 12 months had used a MHA, whereas 18.0% of those who had received services endorsed use of a MHA, χ2(1, N=572) =36.5, P<0.001. Non-significant but noteworthy differences included younger students (ages 18–22) with higher use of MHAs (8.4%) versus those 23 or older (5.3%), 8.7% of white students versus 5.1% non-white students, and 9.4% of those with a positive mental health screen versus 5.7% of those with a negative mental health screen.

Students who reported “Yes” or “Maybe” to being open to MHAs were also asked if they would prefer to use a MHA over seeing a mental health professional if experiencing a mental health condition (see Table 2). Non-white students had a significantly higher preference (11.9%) towards using a MHA compared to white students (7.3%), χ2(1, N=565) =4.66, P=0.03. 4.3% of students who received mental health services in the past 12 months preferred to use MHAs, which was significantly less than those who had not received services (10.7%), χ2(1, N=569) =8.33, P=0.004. Another finding, though not significant, was that males (7.1%) were less partial to using MHAs over a mental health professional compared to females (10.6%), which was also the trend for older students, ages 23 or older (6.7%), compared to younger students, ages 18–22 (10.2%).

Attitudes toward MHAs among subgroups

For the first question in this category, students who answered “Yes” to having a smartphone were asked, “Do you think mental health apps have an evidence base?” There were no significant differences among subgroups for this question (see Table 3), though younger students (ages 18–22, 14.0%) believed MHAs have an evidence base somewhat more so than older students (ages 23 or older, 11.7%).

The second question in this category, “Do you think a mental health app could possibly improve your academic performance, if you were experiencing a mental health condition?”, was asked to students who answered “Yes” to having a smartphone, and replied “Yes” or “Maybe” to being open to MHAs (see Table 3). Both age and student type showed significant differences for this question. 20.1% of students ages 18–22 felt a MHA could improve academic performance if experiencing a mental health condition versus 13.2% of students ages 23 or older, χ2(1, N=568) =5.11, P=0.02. As for student type, significantly more undergraduate students (19.4%) than graduate students (11.8%) felt MHAs could help academic performance, χ2(1, N=556) =4.41, P=0.04. There were no differences based on race/ethnicity or presence of a positive mental health screen.

Other notable findings (non-category and free response questions)

Among those reporting “Yes” or “Maybe” to openness to a MHA, 54.3% reported using apps for other aspects of their well-being. Among the respondents who were at least “Maybe” open to MHAs, the most common responses for why they would use a MHA were: To track mood and/or anxiety, accessibility, confidentiality, and immediate availability (see Table 5). Those who reported they would prefer use of a MHA to meeting with a mental health professional cited reason of convenience, anonymity and reduced stigma, immediate availability, treatment of milder mental health issues, and affordability (see Table 5). 36.1% of respondents answered “Yes” when asked if they would be worried about privacy issues when using a MHA.

Table 5
Table 5 Free text responses categorized by the most popular themes
Full table

Those who stated they were not open to using a MHA were asked why. The most common reason was that these students do not have current mental health issues (see Table 5). Some felt a MHA would be too impersonal, while others stated a desire to reduce use of their smartphones.

Of the students at least “Maybe” open to MHAs who also reported use of MHAs (7.3%), the design and reliability of the app were important factors in whether they liked the MHA. 3.7% of these students used the apps multiple times daily, 22.2% used them daily, and 74.1% used them weekly or less. Regarding the length of time that they used the MHA, 26.7% of students used them for one week or less, 49.5% used them for between one and four weeks, and 23.8% used them for more than four weeks. Finally, 23.7% of MHA users felt MHAs helped with their mental health symptoms, 48.9% felt MHAs maybe helped, and 27.4% felt they did not help.


Discussion

The results of this study demonstrate appreciable interest among college students in the use of MHAs (with over one-quarter of respondents indicating that they would be open to using a MHA). Among student subgroups, the results demonstrated trends towards younger students both using and being more open to using MHAs, which correlates with previous surveys about MHA usage (23). When examining gender differences, females both used and were more open to using MHAs. Several studies have demonstrated that females have reported utilizing more mental health services than males, both in the general population and among college students (24,25). However, only a small proportion of students in the present study reported actually using MHAs. The gap between the interest in and actual use of MHAs points to a need for further research, improved app development, and empirical validation of MHAs.

When considering race/ethnicity in the lens of White versus Non-White students, there were non-significant trends towards White students having higher percentages of openness and usage. Despite less openness and usage among Non-White students, they significantly preferred using an MHA to seeing a mental health professional more than White students. While it would be speculative to propose reasons for this difference, other studies have demonstrated less utilization of mental health services among racial and ethnic minority groups, both in the college and general population (24,25). Whether other factors such as stigma among racial and ethnic minority groups or lack of mental health providers of racial and ethnic minority backgrounds play a role in this would require further examination.

Among students who reported using a MHA, most used them weekly or less and for a relatively short period of time (less than 4 weeks). Further investigation into why use was infrequent and short would be valuable when considering potential applications of this technology. For example, one cannot know whether this limited use could be explained by something such as poor MHA design or something quite different such as student workload and/or distraction. Implementing a team approach to using a MHA, involving both the mental health provider and the patient, may increase the efficacy and longevity of its use (26). Despite the short-term nature of MHA use in this study, nearly one-in-four respondents felt that their mental health was improved by MHA use.

Students commonly cited the convenience, immediate availability, and confidentiality of MHAs as motivators for use. These attitudes may reflect a broader cultural shift in the desire for convenient, immediate, and constant availability of all services, health related or otherwise. MHAs may also appeal to students who grapple with the stigma associated with mental illness and its treatment. Although stigma among college students has declined in recent years, it is still a barrier to mental health services (27). The data shows that a small cohort of students may prefer to use MHAs versus in-person visits with a mental health professional due to what they consider greater convenience and availability. In order to reach those students, innovative non-traditional modes of treatment warrant further consideration. A minority of students indicated disinterest in using a MHA. Most of the disinterested respondents stated they did not have a mental health problem (i.e., that MHAs were not relevant to them). Others would prefer a more personal interaction with a mental health provider, and this attitudinal disparity suggests that if MHAs are to be optimally incorporated into care, then it could be useful to first briefly screen students regarding treatment preferences prior to recommending MHA use. Within this emerging field of wellness-related applications, validating their evidence base, user-friendliness and confidentiality seem essential (28). One study suggested that limiting the use of text and maximizing multimedia in a MHA could help engage users (29). Another study found benefits to incorporating target users in the development of MHAs (30).

There are several limitations to consider in the context of this study. The survey response rate was 18.5%. Although nonresponse weights were used to ensure the results were representative in terms of demographic characteristics, it is possible that the participant sample was biased in terms of interest or usage of MHAs. There is also the potential for a selection bias, as participants self-selected to participate in the MHA module. The survey questions were also not part of a measure validated for reliability, and there was not a definition provided to guide respondents on what being “open” to using an MHA meant. Additionally, the sample only included students at one large, public university. Given the diversity of students across regions, types of colleges and universities attended, and other potential variables, it would be premature to assume these findings would be generalizable to the nation’s college and university students as a whole. Future research would ideally include samples from other sites including a variety of geographically diverse campuses, community and junior colleges, and smaller private schools. The use of yes/no responses versus a Likert scale reduced the specificity of the responses. Finally, the survey questions did not address MHAs for specific psychiatric diagnoses. It is possible that student attitudes towards MHAs might vary based on the specific mental health issues they face; further inquiry into this could be valuable.


Conclusions

Given the context of not only the ubiquity of smartphones, but also the growing demand for college mental health services, the use of MHAs could theoretically help to bolster mental health prevention and intervention efforts for college students. Some have proposed that MHA use could also reduce the costs of providing treatment services, an important consideration given that demand for mental health services exceeds availability at many colleges and universities (31). It is notable that over half of respondents in this study had previously used an app for other health reasons, suggesting that wellness-related apps are not foreign to the college and university student community. Certainly, MHAs need to be empirically validated prior to their integration into the delivery of behavioral health care on campuses. However, it is important to recognize that there are inherent challenges in researching mobile mental health treatments, including the demand for apps to be developed and released quickly (32). Furthermore, treatment providers would need to learn more about these tools to sufficiently educate and guide their patients in proper use (33). The college campus environment serves as an ideal setting for the dissemination and use of MHAs as a potential treatment modality, and this study supports that there is student interest. Further research could help assess whether MHAs have a sufficient foothold with students such that integrating this technology into their behavioral health care could be feasible and beneficial.


Acknowledgements

None.


Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: This study was approved by the institutional review board at the participating institution and administered in April of 2016. The study number was HUM00100169.


References

  1. “Mobile Fact Sheet.” Pew Research Center, Washington, D.C. (February 5, 2018). Retrieved April 12, 2018. Available online: http://www.pewinternet.org/fact-sheet/mobile/
  2. Free C, Phillips G, Galli L, et al. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: A systematic review. PLoS Med 2013;10. [Crossref] [PubMed]
  3. Shen N, Levitan MJ, Johnson A, et al. Finding a depression app: a review and content analysis of the depression app marketplace. JMIR Mhealth Uhealth 2015;3. [Crossref] [PubMed]
  4. Firth J, Torous J. Smartphone Apps for Schizophrenia: A Systematic Review. JMIR Mhealth Uhealth 2015;3. [Crossref] [PubMed]
  5. Nicholas J, Larsen ME, Proudfoot J, et al. Mobile Apps for Bipolar Disorder: A Systematic Review of Features and Content Quality. J Med Internet Res 2015;17. [Crossref] [PubMed]
  6. BinDhim NF, Alanazi EM, Aljadhey H, et al. Does a Mobile Phone Depression-Screening App Motivate Mobile Phone Users With High Depressive Symptoms to Seek a Health Care Professional's Help? J Med Internet Res. 2016;18. [Crossref] [PubMed]
  7. Mani M, Kavanagh DJ, Hides L, et al. Review and Evaluation of Mindfulness-Based iPhone Apps. JMIR Mhealth Uhealth 2015;3. [Crossref] [PubMed]
  8. Huguet A, Rao S, McGrath PJ, et al. A Systematic Review of Cognitive Behavioral Therapy and Behavioral Activation Apps for Depression. PLoS One 2016;11. [Crossref] [PubMed]
  9. Donker T, Petrie K, Proudfoot J. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013;15. [Crossref] [PubMed]
  10. Stoyanov SR, Hides L, Kavanagh DJ, et al. Development and Validation of the User Version of the Mobile Application Rating Scale (uMARS). JMIR Mhealth Uhealth 2016;4. [Crossref] [PubMed]
  11. Chan S, Godwin H, Gonzalez A, et al. Review of use and integration of mobile apps into psychiatric treatments. Curr Psychiatry Rep 2017;19:96. [Crossref] [PubMed]
  12. Zagorski, N. (2017, January 6). APA to Provide Framework to Evaluate Mobile Health Apps. Psychiatric News. Available online: http://psychnews.psychiatryonline.org/
  13. Chan S, Torous J, Hinton L, et al. Towards a framework for evaluating mobile mental health apps. Telemed J E-Health. 2015;21:1038-41. [Crossref] [PubMed]
  14. Kadison R, DiGeronimo TF. College of the overwhelmed: The campus mental health crisis and what to do about it. San Francisco, CA, US: Jossey-Bass, 2004.
  15. Center for Collegiate Mental Health. (2016, January). 2015 Annual Report (Publication No. STA 15-108).
  16. American College Health Association. American College Health Association-National College Health Assessment II: Reference Group Executive Summary Fall 2015. Hanover, MD: American College Health Association,2016.
  17. Clarke G, Yarborough BJ. Evaluating the promise of health IT to enhance/expand the reach of mental health services. Gen Hosp Psychiatry 2013;35:339-44. [Crossref] [PubMed]
  18. Proudfoot J, Parker G, Hadzi Pavlovic D. Community attitudes to the appropriation of mobile phones for monitoring and managing depression, anxiety, and stress. J Med Internet Res 2010;12. [Crossref] [PubMed]
  19. Torous J, Chan SR, Yee-Marie Tan S, et al. Patient Smartphone Ownership and Interest in Mobile Apps to Monitor Symptoms of Mental Health Conditions: A Survey in Four Geographically Distinct Psychiatric Clinics. JMIR Ment Health 2014;1. [Crossref] [PubMed]
  20. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001;16:606-13. [Crossref] [PubMed]
  21. Spitzer RL, Kroenke K, Williams JB, et al. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006;166:1092-7. [Crossref] [PubMed]
  22. Morgan JF, Reid F, Lacey JH. The SCOFF questionnaire: assessment of a new screening tool for eating disorders. BMJ 1999;319:1467-8. [Crossref] [PubMed]
  23. Krebs P, Duncan DT. Health App Use Among US Mobile Phone Owners: A National Survey. JMIR Mhealth Uhealth 2015;3. [Crossref] [PubMed]
  24. Hunt JB, Eisenberg D, Lu L, et al. Racial/Ethnic Disparities in Mental Health Care Utilization among U.S. College Students: Applying the Institution of Medicine Definition of Health Care Disparities. Acad Psychiatry 2015;39:520-6. [Crossref] [PubMed]
  25. Substance Abuse and Mental Health Services Administration. (2015). Racial/Ethnic Differences in Mental Health Service Use among Adults. Available online: https://www.samhsa.gov/data/
  26. Cavanagh K, Millings A. (Inter) personal computing: the role of the therapeutic relationship in e-mental health. J Contemp Psychother 2013;43:197-206. [Crossref]
  27. National Alliance on Mental Illness. College Students Speak: A Survey Report on Mental Health. Arlington, VA: 2012.
  28. Armontrout J, Torous J, Fisher M, et al. Mobile Mental Health: Navigating New Rules and Regulations for Digital Tools. Curr Psychiatry Rep 2016;18:91. [Crossref] [PubMed]
  29. Kenny R, Dooley B, Fitzgerald A. Developing mental health mobile apps: Exploring adolescents' perspectives. Health Informatics J 2016;22:265-75. [Crossref] [PubMed]
  30. Goodwin J, Cummins J, Behan L, et al. Development of a mental health smartphone app: perspectives of mental health service users. J Ment Health 2016;25:434-40. [Crossref] [PubMed]
  31. Powell AC, Chen M, Thammachart C. The Economic Benefits of Mobile Apps for Mental Health and Telepsychiatry Services When Used by Adolescents. Child Adolesc Psychiatr Clin N Am 2017;26:125-33. [Crossref] [PubMed]
  32. Olff M. Mobile mental health: a challenging research agenda. Eur J Psychotraumatol 2015;6:27882. [Crossref] [PubMed]
  33. East ML, Havard BC. Mental Health Mobile Apps: From Infusion to Diffusion in the Mental Health Social System. JMIR Ment Health 2015;2. [Crossref] [PubMed]
doi: 10.21037/mhealth.2018.06.01
Cite this article as: Kern A, Hong V, Song J, Lipson SK, Eisenberg D. Mental health apps in a college setting: openness, usage, and attitudes. mHealth 2018;4:20.