Zero- and Few-Shots Knowledge Graph Triplet Extraction with Large Language Models

Andrea Papaluca*, Daniel Krefl, Sergio J. Rodríguez Méndez, Artem Lensky, Hanna Suominen

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

In this work, we tested the Triplet Extraction (TE) capabilities of a variety of Large Language Models (LLMs) of different sizes in the Zero- and Few-Shots settings. In detail, we proposed a pipeline that dynamically gathers contextual information from a Knowledge Base (KB), both in the form of context triplets and of (sentence, triplets) pairs as examples, and provides it to the LLM through a prompt. The additional context allowed the LLMs to be competitive with all the older fully trained baselines based on the Bidirectional Long Short-Term Memory (BiLSTM) Network architecture. We further conducted a detailed analysis of the quality of the gathered KB context, finding it to be strongly correlated with the final TE performance of the model. In contrast, the size of the model appeared to only logarithmically improve the TE capabilities of the LLMs. We release the code on GitHub 1 for reproducibility.

Original languageEnglish
Title of host publicationKaLLM 2024 - 1st Workshop on Knowledge Graphs and Large Language Models, Proceedings of the Workshop
EditorsRussa Biswas, Lucie-Aimee Kaffee, Oshin Agarwal, Pasquale Minervini, Sameer Singh, Gerard de Melo
PublisherAssociation for Computational Linguistics (ACL)
Pages12-23
Number of pages12
ISBN (Electronic)9798891761476
Publication statusPublished - 2024
Event1st Workshop on Knowledge Graphs and Large Language Models, KaLLM 2024 - Bangkok, Thailand
Duration: 15 Aug 2024 → …

Publication series

NameKaLLM 2024 - 1st Workshop on Knowledge Graphs and Large Language Models, Proceedings of the Workshop

Conference

Conference1st Workshop on Knowledge Graphs and Large Language Models, KaLLM 2024
Country/TerritoryThailand
CityBangkok
Period15/08/24 → …

Fingerprint

Dive into the research topics of 'Zero- and Few-Shots Knowledge Graph Triplet Extraction with Large Language Models'. Together they form a unique fingerprint.

Cite this