Overview of the CLEF ehealth evaluation lab 2018

Hanna Suominen; Liadh Kelly; Lorraine Goeuriot; Aurélie Névéol; Lionel Ramadier; Aude Robert; Evangelos Kanoulas; Rene Spijker; Leif Azzopardi; Dan Li; Jimmy; João Palotti; Guido Zuccon

doi:10.1007/978-3-319-98932-7_26

Overview of the CLEF ehealth evaluation lab 2018

Hanna Suominen^*, Liadh Kelly, Lorraine Goeuriot, Aurélie Névéol, Lionel Ramadier, Aude Robert, Evangelos Kanoulas, Rene Spijker, Leif Azzopardi, Dan Li, Jimmy, João Palotti, Guido Zuccon

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

61 Citations (Scopus)

Abstract

In this paper, we provide an overview of the sixth annual edition of the CLEF eHealth evaluation lab. CLEF eHealth 2018 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins, clinical staff, and health scientists in understanding, accessing, and authoring eHealth information in a multilingual setting. This year’s lab offered three tasks: Task 1 on multilingual information extraction to extend from last year’s task on French and English corpora to French, Hungarian, and Italian; Task 2 on technologically assisted reviews in empirical medicine building on last year’s pilot task in English; and Task 3 on Consumer Health Search (CHS) in mono- and multilingual settings that builds on the 2013–17 Information Retrieval tasks. In total 28 teams took part in these tasks (14 in Task 1, 7 in Task 2 and 7 in Task 3). Herein, we describe the resources created for these tasks, outline our evaluation methodology adopted and provide a brief summary of participants of this year’s challenges and results obtained. As in previous years, the organizers have made data and tools associated with the lab tasks available for future research and development.

Original language	English
Title of host publication	Experimental IR Meets Multilinguality, Multimodality, and Interaction - 9th International Conference of the CLEF Association, CLEF 2018, Proceedings
Editors	Eric SanJuan, Fionn Murtagh, Jian Yun Nie, Laure Soulier, Linda Cappellato, Patrice Bellot, Josiane Mothe, Chiraz Trabelsi, Nicola Ferro
Publisher	Springer Verlag
Pages	286-301
Number of pages	16
ISBN (Print)	9783319989310
DOIs	https://doi.org/10.1007/978-3-319-98932-7_26
Publication status	Published - 2018
Event	9th International Conference of the CLEF Association, CLEF 2018 - Avignon, France Duration: 10 Sept 2018 → 14 Sept 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11018 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	9th International Conference of the CLEF Association, CLEF 2018
Country/Territory	France
City	Avignon
Period	10/09/18 → 14/09/18

Access to Document

10.1007/978-3-319-98932-7_26

Cite this

Suominen, H., Kelly, L., Goeuriot, L., Névéol, A., Ramadier, L., Robert, A., Kanoulas, E., Spijker, R., Azzopardi, L., Li, D., Jimmy, Palotti, J., & Zuccon, G. (2018). Overview of the CLEF ehealth evaluation lab 2018. In E. SanJuan, F. Murtagh, J. Y. Nie, L. Soulier, L. Cappellato, P. Bellot, J. Mothe, C. Trabelsi, & N. Ferro (Eds.), Experimental IR Meets Multilinguality, Multimodality, and Interaction - 9th International Conference of the CLEF Association, CLEF 2018, Proceedings (pp. 286-301). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11018 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-98932-7_26

Suominen, Hanna ; Kelly, Liadh ; Goeuriot, Lorraine et al. / Overview of the CLEF ehealth evaluation lab 2018. Experimental IR Meets Multilinguality, Multimodality, and Interaction - 9th International Conference of the CLEF Association, CLEF 2018, Proceedings. editor / Eric SanJuan ; Fionn Murtagh ; Jian Yun Nie ; Laure Soulier ; Linda Cappellato ; Patrice Bellot ; Josiane Mothe ; Chiraz Trabelsi ; Nicola Ferro. Springer Verlag, 2018. pp. 286-301 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{210e46a4f2cc4131b46389530485c9ce,

title = "Overview of the CLEF ehealth evaluation lab 2018",

abstract = "In this paper, we provide an overview of the sixth annual edition of the CLEF eHealth evaluation lab. CLEF eHealth 2018 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins, clinical staff, and health scientists in understanding, accessing, and authoring eHealth information in a multilingual setting. This year{\textquoteright}s lab offered three tasks: Task 1 on multilingual information extraction to extend from last year{\textquoteright}s task on French and English corpora to French, Hungarian, and Italian; Task 2 on technologically assisted reviews in empirical medicine building on last year{\textquoteright}s pilot task in English; and Task 3 on Consumer Health Search (CHS) in mono- and multilingual settings that builds on the 2013–17 Information Retrieval tasks. In total 28 teams took part in these tasks (14 in Task 1, 7 in Task 2 and 7 in Task 3). Herein, we describe the resources created for these tasks, outline our evaluation methodology adopted and provide a brief summary of participants of this year{\textquoteright}s challenges and results obtained. As in previous years, the organizers have made data and tools associated with the lab tasks available for future research and development.",

keywords = "Entity linking, Evaluation, Health records, Information extraction, Information retrieval, Medical informatics, Self-diagnosis, Systematic reviews, Test-set generation, Text classification, Text segmentation, Total recall",

author = "Hanna Suominen and Liadh Kelly and Lorraine Goeuriot and Aur{\'e}lie N{\'e}v{\'e}ol and Lionel Ramadier and Aude Robert and Evangelos Kanoulas and Rene Spijker and Leif Azzopardi and Dan Li and Jimmy and Jo{\~a}o Palotti and Guido Zuccon",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2018.; 9th International Conference of the CLEF Association, CLEF 2018 ; Conference date: 10-09-2018 Through 14-09-2018",

year = "2018",

doi = "10.1007/978-3-319-98932-7_26",

language = "English",

isbn = "9783319989310",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "286--301",

editor = "Eric SanJuan and Fionn Murtagh and Nie, {Jian Yun} and Laure Soulier and Linda Cappellato and Patrice Bellot and Josiane Mothe and Chiraz Trabelsi and Nicola Ferro",

booktitle = "Experimental IR Meets Multilinguality, Multimodality, and Interaction - 9th International Conference of the CLEF Association, CLEF 2018, Proceedings",

address = "Germany",

}

Suominen, H, Kelly, L, Goeuriot, L, Névéol, A, Ramadier, L, Robert, A, Kanoulas, E, Spijker, R, Azzopardi, L, Li, D, Jimmy, Palotti, J & Zuccon, G 2018, Overview of the CLEF ehealth evaluation lab 2018. in E SanJuan, F Murtagh, JY Nie, L Soulier, L Cappellato, P Bellot, J Mothe, C Trabelsi & N Ferro (eds), Experimental IR Meets Multilinguality, Multimodality, and Interaction - 9th International Conference of the CLEF Association, CLEF 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11018 LNCS, Springer Verlag, pp. 286-301, 9th International Conference of the CLEF Association, CLEF 2018, Avignon, France, 10/09/18. https://doi.org/10.1007/978-3-319-98932-7_26

Overview of the CLEF ehealth evaluation lab 2018. / Suominen, Hanna; Kelly, Liadh; Goeuriot, Lorraine et al.
Experimental IR Meets Multilinguality, Multimodality, and Interaction - 9th International Conference of the CLEF Association, CLEF 2018, Proceedings. ed. / Eric SanJuan; Fionn Murtagh; Jian Yun Nie; Laure Soulier; Linda Cappellato; Patrice Bellot; Josiane Mothe; Chiraz Trabelsi; Nicola Ferro. Springer Verlag, 2018. p. 286-301 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11018 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Overview of the CLEF ehealth evaluation lab 2018

AU - Suominen, Hanna

AU - Kelly, Liadh

AU - Goeuriot, Lorraine

AU - Névéol, Aurélie

AU - Ramadier, Lionel

AU - Robert, Aude

AU - Kanoulas, Evangelos

AU - Spijker, Rene

AU - Azzopardi, Leif

AU - Li, Dan

AU - Jimmy,

AU - Palotti, João

AU - Zuccon, Guido

N1 - Publisher Copyright: © Springer Nature Switzerland AG 2018.

PY - 2018

Y1 - 2018

N2 - In this paper, we provide an overview of the sixth annual edition of the CLEF eHealth evaluation lab. CLEF eHealth 2018 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins, clinical staff, and health scientists in understanding, accessing, and authoring eHealth information in a multilingual setting. This year’s lab offered three tasks: Task 1 on multilingual information extraction to extend from last year’s task on French and English corpora to French, Hungarian, and Italian; Task 2 on technologically assisted reviews in empirical medicine building on last year’s pilot task in English; and Task 3 on Consumer Health Search (CHS) in mono- and multilingual settings that builds on the 2013–17 Information Retrieval tasks. In total 28 teams took part in these tasks (14 in Task 1, 7 in Task 2 and 7 in Task 3). Herein, we describe the resources created for these tasks, outline our evaluation methodology adopted and provide a brief summary of participants of this year’s challenges and results obtained. As in previous years, the organizers have made data and tools associated with the lab tasks available for future research and development.

AB - In this paper, we provide an overview of the sixth annual edition of the CLEF eHealth evaluation lab. CLEF eHealth 2018 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins, clinical staff, and health scientists in understanding, accessing, and authoring eHealth information in a multilingual setting. This year’s lab offered three tasks: Task 1 on multilingual information extraction to extend from last year’s task on French and English corpora to French, Hungarian, and Italian; Task 2 on technologically assisted reviews in empirical medicine building on last year’s pilot task in English; and Task 3 on Consumer Health Search (CHS) in mono- and multilingual settings that builds on the 2013–17 Information Retrieval tasks. In total 28 teams took part in these tasks (14 in Task 1, 7 in Task 2 and 7 in Task 3). Herein, we describe the resources created for these tasks, outline our evaluation methodology adopted and provide a brief summary of participants of this year’s challenges and results obtained. As in previous years, the organizers have made data and tools associated with the lab tasks available for future research and development.

KW - Entity linking

KW - Evaluation

KW - Health records

KW - Information extraction

KW - Information retrieval

KW - Medical informatics

KW - Self-diagnosis

KW - Systematic reviews

KW - Test-set generation

KW - Text classification

KW - Text segmentation

KW - Total recall

UR - http://www.scopus.com/inward/record.url?scp=85051062027&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-98932-7_26

DO - 10.1007/978-3-319-98932-7_26

M3 - Conference contribution

SN - 9783319989310

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 286

EP - 301

BT - Experimental IR Meets Multilinguality, Multimodality, and Interaction - 9th International Conference of the CLEF Association, CLEF 2018, Proceedings

A2 - SanJuan, Eric

A2 - Murtagh, Fionn

A2 - Nie, Jian Yun

A2 - Soulier, Laure

A2 - Cappellato, Linda

A2 - Bellot, Patrice

A2 - Mothe, Josiane

A2 - Trabelsi, Chiraz

A2 - Ferro, Nicola

PB - Springer Verlag

T2 - 9th International Conference of the CLEF Association, CLEF 2018

Y2 - 10 September 2018 through 14 September 2018

ER -

Suominen H, Kelly L, Goeuriot L, Névéol A, Ramadier L, Robert A et al. Overview of the CLEF ehealth evaluation lab 2018. In SanJuan E, Murtagh F, Nie JY, Soulier L, Cappellato L, Bellot P, Mothe J, Trabelsi C, Ferro N, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction - 9th International Conference of the CLEF Association, CLEF 2018, Proceedings. Springer Verlag. 2018. p. 286-301. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-98932-7_26

Overview of the CLEF ehealth evaluation lab 2018

Abstract

Publication series

Conference

Access to Document

Other files and links

Cite this