Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection

Publikation: Beitrag in Buch/Bericht/Sammelwerk/KonferenzbandAufsatz in KonferenzbandForschungPeer-Review

Autorschaft

Externe Organisationen

  • Universität Bielefeld
  • Ludwig-Maximilians-Universität München (LMU)
Forschungs-netzwerk anzeigen

Details

OriginalspracheEnglisch
Titel des SammelwerksProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics
UntertitelHuman Language Technologies
ErscheinungsortAlbuquerque, New Mexico
Seitenumfang28
Band1
ISBN (elektronisch)979-8-89176-189-6
PublikationsstatusVeröffentlicht - 29 Apr. 2025

Abstract

Recent advances on instruction fine-tuning have led to the development of various prompting techniques for large language models, such as explicit reasoning steps. However, the success of techniques depends on various parameters, such as the task, language model, and context provided. Finding an effective prompt is, therefore, often a trial-and-error process. Most existing approaches to automatic prompting aim to optimize individual techniques instead of compositions of techniques and their dependence on the input. To fill this gap, we propose an adaptive prompting approach that predicts the optimal prompt composition ad-hoc for a given input. We apply our approach to social bias detection, a highly context-dependent task that requires semantic understanding. We evaluate it with three large language models on three datasets, comparing compositions to individual techniques and other baselines. The results underline the importance of finding an effective prompt composition. Our approach robustly ensures high detection performance, and is best in several settings. Moreover, first experiments on other tasks support its generalizability.

Zitieren

Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection. / Spliethöver, Maximilian; Knebler, Tim; Fumagalli, Fabian et al.
Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies. Band 1 Albuquerque, New Mexico, 2025.

Publikation: Beitrag in Buch/Bericht/Sammelwerk/KonferenzbandAufsatz in KonferenzbandForschungPeer-Review

Spliethöver, M, Knebler, T, Fumagalli, F, Muschalik, M, Hammer, B, Hüllermeier, E & Wachsmuth, H 2025, Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection. in Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies. Bd. 1, Albuquerque, New Mexico. <https://aclanthology.org/2025.naacl-long.122.pdf>
Spliethöver, M., Knebler, T., Fumagalli, F., Muschalik, M., Hammer, B., Hüllermeier, E., & Wachsmuth, H. (2025). Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Band 1). https://aclanthology.org/2025.naacl-long.122.pdf
Spliethöver M, Knebler T, Fumagalli F, Muschalik M, Hammer B, Hüllermeier E et al. Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection. in Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies. Band 1. Albuquerque, New Mexico. 2025
Spliethöver, Maximilian ; Knebler, Tim ; Fumagalli, Fabian et al. / Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies. Band 1 Albuquerque, New Mexico, 2025.
Download
@inproceedings{0e2c1918b8a24c1dbedead0659b9ba0e,
title = "Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection",
abstract = "Recent advances on instruction fine-tuning have led to the development of various prompting techniques for large language models, such as explicit reasoning steps. However, the success of techniques depends on various parameters, such as the task, language model, and context provided. Finding an effective prompt is, therefore, often a trial-and-error process. Most existing approaches to automatic prompting aim to optimize individual techniques instead of compositions of techniques and their dependence on the input. To fill this gap, we propose an adaptive prompting approach that predicts the optimal prompt composition ad-hoc for a given input. We apply our approach to social bias detection, a highly context-dependent task that requires semantic understanding. We evaluate it with three large language models on three datasets, comparing compositions to individual techniques and other baselines. The results underline the importance of finding an effective prompt composition. Our approach robustly ensures high detection performance, and is best in several settings. Moreover, first experiments on other tasks support its generalizability.",
author = "Maximilian Splieth{\"o}ver and Tim Knebler and Fabian Fumagalli and Maximilian Muschalik and Barbara Hammer and Eyke H{\"u}llermeier and Henning Wachsmuth",
year = "2025",
month = apr,
day = "29",
language = "English",
volume = "1",
booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics",

}

Download

TY - GEN

T1 - Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection

AU - Spliethöver, Maximilian

AU - Knebler, Tim

AU - Fumagalli, Fabian

AU - Muschalik, Maximilian

AU - Hammer, Barbara

AU - Hüllermeier, Eyke

AU - Wachsmuth, Henning

PY - 2025/4/29

Y1 - 2025/4/29

N2 - Recent advances on instruction fine-tuning have led to the development of various prompting techniques for large language models, such as explicit reasoning steps. However, the success of techniques depends on various parameters, such as the task, language model, and context provided. Finding an effective prompt is, therefore, often a trial-and-error process. Most existing approaches to automatic prompting aim to optimize individual techniques instead of compositions of techniques and their dependence on the input. To fill this gap, we propose an adaptive prompting approach that predicts the optimal prompt composition ad-hoc for a given input. We apply our approach to social bias detection, a highly context-dependent task that requires semantic understanding. We evaluate it with three large language models on three datasets, comparing compositions to individual techniques and other baselines. The results underline the importance of finding an effective prompt composition. Our approach robustly ensures high detection performance, and is best in several settings. Moreover, first experiments on other tasks support its generalizability.

AB - Recent advances on instruction fine-tuning have led to the development of various prompting techniques for large language models, such as explicit reasoning steps. However, the success of techniques depends on various parameters, such as the task, language model, and context provided. Finding an effective prompt is, therefore, often a trial-and-error process. Most existing approaches to automatic prompting aim to optimize individual techniques instead of compositions of techniques and their dependence on the input. To fill this gap, we propose an adaptive prompting approach that predicts the optimal prompt composition ad-hoc for a given input. We apply our approach to social bias detection, a highly context-dependent task that requires semantic understanding. We evaluate it with three large language models on three datasets, comparing compositions to individual techniques and other baselines. The results underline the importance of finding an effective prompt composition. Our approach robustly ensures high detection performance, and is best in several settings. Moreover, first experiments on other tasks support its generalizability.

M3 - Conference contribution

VL - 1

BT - Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics

CY - Albuquerque, New Mexico

ER -

Von denselben Autoren