CAIDAS at SemEval-2025 Task 7: Enriching Sparse Datasets with LLM-Generated Content for Improved Information Retrieval
D. Benchert, S. Meßlinger, S. Goller, J. Kaiser, J. Pfister, and A. Hotho. Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), page 1623--1638. Vienna, Austria, Association for Computational Linguistics, (July 2025)
Abstract
The focus of SemEval-2024 Task 7 is the retrieval of relevant fact-checks for social media posts across multiple languages. We approach this task with an enhanced bi-encoder retrieval setup, which is designed to match social media posts with relevant fact-checks using synthetic data from LLMs. We explored and analyzed two main approaches for generating synthetic posts. Either based on existing fact-checks or on existing posts. Our approach achieved an S@10 score of 89.53\% for the monolingual task and 74.48\% for the crosslingual task, ranking 16th out of 28 and 13th out of 29, respectively. Without data augmentation, scores would have been 88.69 (17th) and 72.93 (15th).
%0 Conference Paper
%1 benchert-etal-2025-caidas
%A Benchert, Dominik
%A Meßlinger, Severin
%A Goller, Sven
%A Kaiser, Jonas
%A Pfister, Jan
%A Hotho, Andreas
%B Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
%C Vienna, Austria
%D 2025
%E Rosenthal, Sara
%E Rosá, Aiala
%E Ghosh, Debanjan
%E Zampieri, Marcos
%I Association for Computational Linguistics
%K nlp myown author:kaiser author:hotho from:janpf author:pfister
%P 1623--1638
%T CAIDAS at SemEval-2025 Task 7: Enriching Sparse Datasets with LLM-Generated Content for Improved Information Retrieval
%U https://aclanthology.org/2025.semeval-1.214/
%X The focus of SemEval-2024 Task 7 is the retrieval of relevant fact-checks for social media posts across multiple languages. We approach this task with an enhanced bi-encoder retrieval setup, which is designed to match social media posts with relevant fact-checks using synthetic data from LLMs. We explored and analyzed two main approaches for generating synthetic posts. Either based on existing fact-checks or on existing posts. Our approach achieved an S@10 score of 89.53\% for the monolingual task and 74.48\% for the crosslingual task, ranking 16th out of 28 and 13th out of 29, respectively. Without data augmentation, scores would have been 88.69 (17th) and 72.93 (15th).
%@ 979-8-89176-273-2
@inproceedings{benchert-etal-2025-caidas,
abstract = {The focus of SemEval-2024 Task 7 is the retrieval of relevant fact-checks for social media posts across multiple languages. We approach this task with an enhanced bi-encoder retrieval setup, which is designed to match social media posts with relevant fact-checks using synthetic data from LLMs. We explored and analyzed two main approaches for generating synthetic posts. Either based on existing fact-checks or on existing posts. Our approach achieved an S@10 score of 89.53{\%} for the monolingual task and 74.48{\%} for the crosslingual task, ranking 16th out of 28 and 13th out of 29, respectively. Without data augmentation, scores would have been 88.69 (17th) and 72.93 (15th).},
added-at = {2025-08-08T13:18:45.000+0200},
address = {Vienna, Austria},
author = {Benchert, Dominik and Me{\ss}linger, Severin and Goller, Sven and Kaiser, Jonas and Pfister, Jan and Hotho, Andreas},
biburl = {https://www.bibsonomy.org/bibtex/2718b2f4d802274a1adf075edee15bc6b/dmir},
booktitle = {Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)},
editor = {Rosenthal, Sara and Ros{\'a}, Aiala and Ghosh, Debanjan and Zampieri, Marcos},
interhash = {ad9583887d374bd0715928d223b95fac},
intrahash = {718b2f4d802274a1adf075edee15bc6b},
isbn = {979-8-89176-273-2},
keywords = {nlp myown author:kaiser author:hotho from:janpf author:pfister},
month = jul,
pages = {1623--1638},
publisher = {Association for Computational Linguistics},
timestamp = {2025-08-08T13:18:45.000+0200},
title = {{CAIDAS} at {S}em{E}val-2025 Task 7: Enriching Sparse Datasets with {LLM}-Generated Content for Improved Information Retrieval},
url = {https://aclanthology.org/2025.semeval-1.214/},
year = 2025
}