Boosting Long-Tail Data Classification with Sparse Prototypical Networks

Research output: Chapter in book/report/conference proceedingConference contributionResearchpeer review

Authors

  • Alexei Figueroa
  • Jens Michalis Papaioannou
  • Conor Fallon
  • Alexandra Bekiaridou
  • Keno Bressem
  • Stavros Zanos
  • Felix Gers
  • Wolfgang Nejdl
  • Alexander Löser

Research Organisations

External Research Organisations

  • DATEXIS
  • Elmezzi Graduate School of Molecular Medicine
  • Northwell Health System
  • Technical University of Munich (TUM)
View graph of relations

Details

Original languageEnglish
Title of host publicationMachine Learning and Knowledge Discovery in Databases
Subtitle of host publicationResearch Track - European Conference, ECML PKDD 2024, Proceedings
EditorsAlbert Bifet, Jesse Davis, Tomas Krilavičius, Meelis Kull, Eirini Ntoutsi, Indrė Žliobaitė
PublisherSpringer Science and Business Media Deutschland GmbH
Pages434-449
Number of pages16
ISBN (electronic)978-3-031-70368-3
ISBN (print)9783031703676
Publication statusPublished - 2024
EventEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2024 - Vilnius, Lithuania
Duration: 9 Sept 202413 Sept 2024

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14947 LNAI
ISSN (Print)0302-9743
ISSN (electronic)1611-3349

Abstract

Clinical Decision Support Systems (CDSS) have become ubiquitous in healthcare facilities, leveraging the increasing presence of Electronic Health Records (EHR). Predicting clinical outcomes from clinical text, such as identifying diagnoses based on the admission state of patients, is among the core tasks that a CDSS must address. The state-of-the-art for this task has been set by transformer encoder models, recently superseded by encoders enhanced with a prototypical network. This task remains a significant challenge due to the substantial imbalance of the outcome labels, which is characterized by a long-tailed distribution where the majority of diagnoses are under-represented. Motivated by recent biologically inspired findings in deep learning, we propose S-Proto, a novel, efficient, and sparse prototypical layer. Our method achieves state-of-the-art performance in outcome diagnosis prediction, without compromising on the explainability characteristics of prototypical encoders. Quantitative results demonstrate that our approach is robust to the challenges presented by clinical notes, and transfers successfully to a second, unseen dataset. Qualitative evaluation with medical doctors shows that S-Proto is capable of disaggregating the representations of a disease that manifests differently in patient cohorts.

Keywords

    Long-Tail, NLP, Prototypical Networks, Sparsity

ASJC Scopus subject areas

Cite this

Boosting Long-Tail Data Classification with Sparse Prototypical Networks. / Figueroa, Alexei; Papaioannou, Jens Michalis; Fallon, Conor et al.
Machine Learning and Knowledge Discovery in Databases: Research Track - European Conference, ECML PKDD 2024, Proceedings. ed. / Albert Bifet; Jesse Davis; Tomas Krilavičius; Meelis Kull; Eirini Ntoutsi; Indrė Žliobaitė. Springer Science and Business Media Deutschland GmbH, 2024. p. 434-449 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14947 LNAI).

Research output: Chapter in book/report/conference proceedingConference contributionResearchpeer review

Figueroa, A, Papaioannou, JM, Fallon, C, Bekiaridou, A, Bressem, K, Zanos, S, Gers, F, Nejdl, W & Löser, A 2024, Boosting Long-Tail Data Classification with Sparse Prototypical Networks. in A Bifet, J Davis, T Krilavičius, M Kull, E Ntoutsi & I Žliobaitė (eds), Machine Learning and Knowledge Discovery in Databases: Research Track - European Conference, ECML PKDD 2024, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14947 LNAI, Springer Science and Business Media Deutschland GmbH, pp. 434-449, European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2024, Vilnius, Lithuania, 9 Sept 2024. https://doi.org/10.1007/978-3-031-70368-3_26
Figueroa, A., Papaioannou, J. M., Fallon, C., Bekiaridou, A., Bressem, K., Zanos, S., Gers, F., Nejdl, W., & Löser, A. (2024). Boosting Long-Tail Data Classification with Sparse Prototypical Networks. In A. Bifet, J. Davis, T. Krilavičius, M. Kull, E. Ntoutsi, & I. Žliobaitė (Eds.), Machine Learning and Knowledge Discovery in Databases: Research Track - European Conference, ECML PKDD 2024, Proceedings (pp. 434-449). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14947 LNAI). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-70368-3_26
Figueroa A, Papaioannou JM, Fallon C, Bekiaridou A, Bressem K, Zanos S et al. Boosting Long-Tail Data Classification with Sparse Prototypical Networks. In Bifet A, Davis J, Krilavičius T, Kull M, Ntoutsi E, Žliobaitė I, editors, Machine Learning and Knowledge Discovery in Databases: Research Track - European Conference, ECML PKDD 2024, Proceedings. Springer Science and Business Media Deutschland GmbH. 2024. p. 434-449. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2024 Aug 22. doi: 10.1007/978-3-031-70368-3_26
Figueroa, Alexei ; Papaioannou, Jens Michalis ; Fallon, Conor et al. / Boosting Long-Tail Data Classification with Sparse Prototypical Networks. Machine Learning and Knowledge Discovery in Databases: Research Track - European Conference, ECML PKDD 2024, Proceedings. editor / Albert Bifet ; Jesse Davis ; Tomas Krilavičius ; Meelis Kull ; Eirini Ntoutsi ; Indrė Žliobaitė. Springer Science and Business Media Deutschland GmbH, 2024. pp. 434-449 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
Download
@inproceedings{17ec0da1b5ff421f8fc0579d82178c5d,
title = "Boosting Long-Tail Data Classification with Sparse Prototypical Networks",
abstract = "Clinical Decision Support Systems (CDSS) have become ubiquitous in healthcare facilities, leveraging the increasing presence of Electronic Health Records (EHR). Predicting clinical outcomes from clinical text, such as identifying diagnoses based on the admission state of patients, is among the core tasks that a CDSS must address. The state-of-the-art for this task has been set by transformer encoder models, recently superseded by encoders enhanced with a prototypical network. This task remains a significant challenge due to the substantial imbalance of the outcome labels, which is characterized by a long-tailed distribution where the majority of diagnoses are under-represented. Motivated by recent biologically inspired findings in deep learning, we propose S-Proto, a novel, efficient, and sparse prototypical layer. Our method achieves state-of-the-art performance in outcome diagnosis prediction, without compromising on the explainability characteristics of prototypical encoders. Quantitative results demonstrate that our approach is robust to the challenges presented by clinical notes, and transfers successfully to a second, unseen dataset. Qualitative evaluation with medical doctors shows that S-Proto is capable of disaggregating the representations of a disease that manifests differently in patient cohorts.",
keywords = "Long-Tail, NLP, Prototypical Networks, Sparsity",
author = "Alexei Figueroa and Papaioannou, {Jens Michalis} and Conor Fallon and Alexandra Bekiaridou and Keno Bressem and Stavros Zanos and Felix Gers and Wolfgang Nejdl and Alexander L{\"o}ser",
note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.; European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2024 ; Conference date: 09-09-2024 Through 13-09-2024",
year = "2024",
doi = "10.1007/978-3-031-70368-3_26",
language = "English",
isbn = "9783031703676",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Science and Business Media Deutschland GmbH",
pages = "434--449",
editor = "Albert Bifet and Jesse Davis and Tomas Krilavi{\v c}ius and Meelis Kull and Eirini Ntoutsi and Indrė {\v Z}liobaitė",
booktitle = "Machine Learning and Knowledge Discovery in Databases",
address = "Germany",

}

Download

TY - GEN

T1 - Boosting Long-Tail Data Classification with Sparse Prototypical Networks

AU - Figueroa, Alexei

AU - Papaioannou, Jens Michalis

AU - Fallon, Conor

AU - Bekiaridou, Alexandra

AU - Bressem, Keno

AU - Zanos, Stavros

AU - Gers, Felix

AU - Nejdl, Wolfgang

AU - Löser, Alexander

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

PY - 2024

Y1 - 2024

N2 - Clinical Decision Support Systems (CDSS) have become ubiquitous in healthcare facilities, leveraging the increasing presence of Electronic Health Records (EHR). Predicting clinical outcomes from clinical text, such as identifying diagnoses based on the admission state of patients, is among the core tasks that a CDSS must address. The state-of-the-art for this task has been set by transformer encoder models, recently superseded by encoders enhanced with a prototypical network. This task remains a significant challenge due to the substantial imbalance of the outcome labels, which is characterized by a long-tailed distribution where the majority of diagnoses are under-represented. Motivated by recent biologically inspired findings in deep learning, we propose S-Proto, a novel, efficient, and sparse prototypical layer. Our method achieves state-of-the-art performance in outcome diagnosis prediction, without compromising on the explainability characteristics of prototypical encoders. Quantitative results demonstrate that our approach is robust to the challenges presented by clinical notes, and transfers successfully to a second, unseen dataset. Qualitative evaluation with medical doctors shows that S-Proto is capable of disaggregating the representations of a disease that manifests differently in patient cohorts.

AB - Clinical Decision Support Systems (CDSS) have become ubiquitous in healthcare facilities, leveraging the increasing presence of Electronic Health Records (EHR). Predicting clinical outcomes from clinical text, such as identifying diagnoses based on the admission state of patients, is among the core tasks that a CDSS must address. The state-of-the-art for this task has been set by transformer encoder models, recently superseded by encoders enhanced with a prototypical network. This task remains a significant challenge due to the substantial imbalance of the outcome labels, which is characterized by a long-tailed distribution where the majority of diagnoses are under-represented. Motivated by recent biologically inspired findings in deep learning, we propose S-Proto, a novel, efficient, and sparse prototypical layer. Our method achieves state-of-the-art performance in outcome diagnosis prediction, without compromising on the explainability characteristics of prototypical encoders. Quantitative results demonstrate that our approach is robust to the challenges presented by clinical notes, and transfers successfully to a second, unseen dataset. Qualitative evaluation with medical doctors shows that S-Proto is capable of disaggregating the representations of a disease that manifests differently in patient cohorts.

KW - Long-Tail

KW - NLP

KW - Prototypical Networks

KW - Sparsity

UR - http://www.scopus.com/inward/record.url?scp=85204376875&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-70368-3_26

DO - 10.1007/978-3-031-70368-3_26

M3 - Conference contribution

AN - SCOPUS:85204376875

SN - 9783031703676

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 434

EP - 449

BT - Machine Learning and Knowledge Discovery in Databases

A2 - Bifet, Albert

A2 - Davis, Jesse

A2 - Krilavičius, Tomas

A2 - Kull, Meelis

A2 - Ntoutsi, Eirini

A2 - Žliobaitė, Indrė

PB - Springer Science and Business Media Deutschland GmbH

T2 - European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2024

Y2 - 9 September 2024 through 13 September 2024

ER -

By the same author(s)