TY - GEN

T1 - The Touché23-ValueEval Dataset for Identifying Human Values behind Arguments

AU - Mirzakhmedova, Nailia

AU - Kiesel, Johannes

AU - Alshomary, Milad

AU - Heinrich, Maximilian

AU - Handke, Nicolas

AU - Cai, Xiaoni

AU - Barriere, Valentin

AU - Dastgheib, Doratossadat

AU - Ghahroodi, Omid

AU - SadraeiJavaheri, MohammadAli

AU - Asgari, Ehsaneddin

AU - Kawaletz, Lea

AU - Wachsmuth, Henning

AU - Stein, Benno

N1 - Publisher Copyright: © 2024 ELRA Language Resource Association: CC BY-NC 4.0.

PY - 2024/5/1

Y1 - 2024/5/1

N2 - While human values play a crucial role in making arguments persuasive, we currently lack the necessary extensive datasets to develop methods for analyzing the values underlying these arguments on a large scale. To address this gap, we present the Touché23-ValueEval dataset, an expansion of the Webis-ArgValues-22 dataset. We collected and annotated an additional 4780 new arguments, doubling the dataset`s size to 9324 arguments. These arguments were sourced from six diverse sources, covering religious texts, community discussions, free-text arguments, newspaper editorials, and political debates. Each argument is annotated by three crowdworkers for 54 human values, following the methodology established in the original dataset. The Touché23-ValueEval dataset was utilized in the SemEval 2023 Task 4. ValueEval: Identification of Human Values behind Arguments, where an ensemble of transformer models demonstrated state-of-the-art performance. Furthermore, our experiments show that a fine-tuned large language model, Llama-2-7B, achieves comparable results.

AB - While human values play a crucial role in making arguments persuasive, we currently lack the necessary extensive datasets to develop methods for analyzing the values underlying these arguments on a large scale. To address this gap, we present the Touché23-ValueEval dataset, an expansion of the Webis-ArgValues-22 dataset. We collected and annotated an additional 4780 new arguments, doubling the dataset`s size to 9324 arguments. These arguments were sourced from six diverse sources, covering religious texts, community discussions, free-text arguments, newspaper editorials, and political debates. Each argument is annotated by three crowdworkers for 54 human values, following the methodology established in the original dataset. The Touché23-ValueEval dataset was utilized in the SemEval 2023 Task 4. ValueEval: Identification of Human Values behind Arguments, where an ensemble of transformer models demonstrated state-of-the-art performance. Furthermore, our experiments show that a fine-tuned large language model, Llama-2-7B, achieves comparable results.

KW - Corpus (Creation, Annotation, etc.)

KW - Document Classification

KW - Text categorisation

UR - http://www.scopus.com/inward/record.url?scp=85195967977&partnerID=8YFLogxK

M3 - Conference contribution

SP - 16121

EP - 16134

BT - Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)

A2 - Calzolari, Nicoletta

A2 - Kan, Min-Yen

A2 - Hoste, Veronique

A2 - Lenci, Alessandro

A2 - Sakti, Sakriani

A2 - Xue, Nianwen

CY - Torino, Italia

ER -