Autoren:	Jungmann, Stefanie Klan, Timo Kuhn, Sebastian Jungmann, Florian
Titel:	Accuracy of a chatbot (Ada) in the diagnosis of mental disorders : comparative case study with lay and expert users
Online-Publikationsdatum:	27-Nov-2019
Erscheinungsdatum:	2019
Sprache des Dokuments:	Englisch
Zusammenfassung/Abstract:	Background: Health apps for the screening and diagnosis of mental disorders have emerged in recent years on various levels (eg, patients, practitioners, and public health system). However, the diagnostic quality of these apps has not been (sufficiently) tested so far. Objective: The objective of this pilot study was to investigate the diagnostic quality of a health app for a broad spectrum of mental disorders and its dependency on expert knowledge. Methods: Two psychotherapists, two psychology students, and two laypersons each read 20 case vignettes with a broad spectrum of mental disorders. They used a health app (Ada—Your Health Guide) to get a diagnosis by entering the symptoms. Interrater reliabilities were computed between the diagnoses of the case vignettes and the results of the app for each user group. Results: Overall, there was a moderate diagnostic agreement (kappa=0.64) between the results of the app and the case vignettes for mental disorders in adulthood and a low diagnostic agreement (kappa=0.40) for mental disorders in childhood and adolescence. When psychotherapists applied the app, there was a good diagnostic agreement (kappa=0.78) regarding mental disorders in adulthood. The diagnostic agreement was moderate (kappa=0.55/0.60) for students and laypersons. For mental disorders in childhood and adolescence, a moderate diagnostic quality was found when psychotherapists (kappa=0.53) and students (kappa=0.41) used the app, whereas the quality was low for laypersons (kappa=0.29). On average, the app required 34 questions to be answered and 7 min to complete. Conclusions: The health app investigated here can represent an efficient diagnostic screening or help function for mental disorders in adulthood and has the potential to support especially diagnosticians in their work in various ways. The results of this pilot study provide a first indication that the diagnostic accuracy is user dependent and improvements in the app are needed especially for mental disorders in childhood and adolescence.
DDC-Sachgruppe:	150 Psychologie 150 Psychology
Veröffentlichende Institution:	Johannes Gutenberg-Universität Mainz
Organisationseinheit:	FB 02 Sozialwiss., Medien u. Sport FB 04 Medizin
Veröffentlichungsort:	Mainz
ROR:	https://ror.org/023b0x485
DOI:	http://doi.org/10.25358/openscience-235
URN:	urn:nbn:de:hebis:77-publ-594315
Version:	Published version
Publikationstyp:	Zeitschriftenaufsatz
Nutzungsrechte:	CC BY
Informationen zu den Nutzungsrechten:	https://creativecommons.org/licenses/by/4.0/
Zeitschrift:	JMIR formative research 3 4
Seitenzahl oder Artikelnummer:	e13863
Verlag:	JMIR Publications
Verlagsort:	Toronto
Erscheinungsdatum:	2019
ISSN:	2561-326X
URL der Originalveröffentlichung:	http://dx.doi.org/10.2196/13863
DOI der Originalveröffentlichung:	10.2196/13863
Enthalten in den Sammlungen:	JGU-Publikationen