Task 1: ShARe/CLEF eHealth evaluation lab 2013

Sameer Pradhan; Noemie Elhadad; Brett R. South; David Martinez; Lee Christensen; Amy Vogel; Hanna Suominen; Wendy W. Chapman; Guergana Savova

Task 1: ShARe/CLEF eHealth evaluation lab 2013

Sameer Pradhan, Noemie Elhadad, Brett R. South, David Martinez, Lee Christensen, Amy Vogel, Hanna Suominen, Wendy W. Chapman, Guergana Savova

Research output: Contribution to journal › Conference article › peer-review

26 Citations (Scopus)

Abstract

This report outlines the Task 1 of the ShARe/CLEF eHealth evaluation lab pilot. This task focused on identification (1a) and normalization (1b) of diseases and disorders in clinical reports. It used annotations from the ShARe corpus. A total of 22 teams competed in Task 1a and 17 of them also participated Task 1b. The best systems had an F1 score of 0.75 (0.80 Precision, 0.71 Recall) in Task 1a and an accuracy of 0.59 in Task 1b. The organizers have made the text corpora, annotations, and evaluation tools available for future research and development.

Original language	English
Journal	CEUR Workshop Proceedings
Volume	1179
Publication status	Published - 2013
Event	2013 Cross Language Evaluation Forum Conference, CLEF 2013 - Valencia, Spain Duration: 23 Sept 2013 → 26 Sept 2013

Cite this

@article{9ee99729733b4984a61eff889c0276cc,

title = "Task 1: ShARe/CLEF eHealth evaluation lab 2013",

abstract = "This report outlines the Task 1 of the ShARe/CLEF eHealth evaluation lab pilot. This task focused on identification (1a) and normalization (1b) of diseases and disorders in clinical reports. It used annotations from the ShARe corpus. A total of 22 teams competed in Task 1a and 17 of them also participated Task 1b. The best systems had an F1 score of 0.75 (0.80 Precision, 0.71 Recall) in Task 1a and an accuracy of 0.59 in Task 1b. The organizers have made the text corpora, annotations, and evaluation tools available for future research and development.",

keywords = "Medical informatics, Natural language processing, Reference standard generation, Text normalization",

author = "Sameer Pradhan and Noemie Elhadad and South, {Brett R.} and David Martinez and Lee Christensen and Amy Vogel and Hanna Suominen and Chapman, {Wendy W.} and Guergana Savova",

year = "2013",

language = "English",

volume = "1179",

journal = "CEUR Workshop Proceedings",

issn = "1613-0073",

publisher = "CEUR-WS",

note = "2013 Cross Language Evaluation Forum Conference, CLEF 2013 ; Conference date: 23-09-2013 Through 26-09-2013",

}

TY - JOUR

T1 - Task 1

T2 - 2013 Cross Language Evaluation Forum Conference, CLEF 2013

AU - Pradhan, Sameer

AU - Elhadad, Noemie

AU - South, Brett R.

AU - Martinez, David

AU - Christensen, Lee

AU - Vogel, Amy

AU - Suominen, Hanna

AU - Chapman, Wendy W.

AU - Savova, Guergana

PY - 2013

Y1 - 2013

N2 - This report outlines the Task 1 of the ShARe/CLEF eHealth evaluation lab pilot. This task focused on identification (1a) and normalization (1b) of diseases and disorders in clinical reports. It used annotations from the ShARe corpus. A total of 22 teams competed in Task 1a and 17 of them also participated Task 1b. The best systems had an F1 score of 0.75 (0.80 Precision, 0.71 Recall) in Task 1a and an accuracy of 0.59 in Task 1b. The organizers have made the text corpora, annotations, and evaluation tools available for future research and development.

AB - This report outlines the Task 1 of the ShARe/CLEF eHealth evaluation lab pilot. This task focused on identification (1a) and normalization (1b) of diseases and disorders in clinical reports. It used annotations from the ShARe corpus. A total of 22 teams competed in Task 1a and 17 of them also participated Task 1b. The best systems had an F1 score of 0.75 (0.80 Precision, 0.71 Recall) in Task 1a and an accuracy of 0.59 in Task 1b. The organizers have made the text corpora, annotations, and evaluation tools available for future research and development.

KW - Medical informatics

KW - Natural language processing

KW - Reference standard generation

KW - Text normalization

UR - http://www.scopus.com/inward/record.url?scp=84922022518&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:84922022518

SN - 1613-0073

VL - 1179

JO - CEUR Workshop Proceedings

JF - CEUR Workshop Proceedings

Y2 - 23 September 2013 through 26 September 2013

ER -

Task 1: ShARe/CLEF eHealth evaluation lab 2013

Abstract

Other files and links

Fingerprint

Cite this