TY - GEN
T1 - Contrastive Language-Entity Pre-training for Richer Knowledge Graph Embedding
AU - Papaluca, Andrea
AU - Krefl, Daniel
AU - Lensky, Artem
AU - Suominen, Hanna
N1 - Publisher Copyright:
© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.
PY - 2025
Y1 - 2025
N2 - In this work we propose a pretraining procedure that aligns a graph encoder and a text encoder to learn a common multi-modal graph-text embedding space. The alignment is obtained by training a model to predict the correct associations between Knowledge Graph nodes and their corresponding descriptions. We test the procedure with two popular Knowledge Bases: Wikidata (formerly Freebase) and YAGO. Our results indicate that such a pretraining method allows for link prediction without the need for additional fine-tuning. Furthermore, we demonstrate that a graph encoder pretrained on the description matching task allows for improved link prediction performance after fine-tuning, without the need for providing node descriptions as additional inputs. We make available the code used in the experiments on GitHub(https://github.com/BrunoLiegiBastonLiegi/CLEP) under the MIT license to encourage further work.
AB - In this work we propose a pretraining procedure that aligns a graph encoder and a text encoder to learn a common multi-modal graph-text embedding space. The alignment is obtained by training a model to predict the correct associations between Knowledge Graph nodes and their corresponding descriptions. We test the procedure with two popular Knowledge Bases: Wikidata (formerly Freebase) and YAGO. Our results indicate that such a pretraining method allows for link prediction without the need for additional fine-tuning. Furthermore, we demonstrate that a graph encoder pretrained on the description matching task allows for improved link prediction performance after fine-tuning, without the need for providing node descriptions as additional inputs. We make available the code used in the experiments on GitHub(https://github.com/BrunoLiegiBastonLiegi/CLEP) under the MIT license to encourage further work.
KW - Knowledge Graphs
KW - Multi-modal Learning
UR - http://www.scopus.com/inward/record.url?scp=85219206043&partnerID=8YFLogxK
U2 - 10.1007/978-981-97-8702-9_16
DO - 10.1007/978-981-97-8702-9_16
M3 - Conference contribution
AN - SCOPUS:85219206043
SN - 9789819787012
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 233
EP - 246
BT - Pattern Recognition and Artificial Intelligence - 4th International Conference, ICPRAI 2024, Proceedings
A2 - Wallraven, Christian
A2 - Liu, Cheng-Lin
A2 - Ross, Arun
PB - Springer Science and Business Media Deutschland GmbH
T2 - 4th International Conference on Pattern Recognition and Artificial Intelligence, ICPRAI 2024
Y2 - 3 July 2024 through 6 July 2024
ER -