Knowledge Graphs and Natural Language: two sides of the same coin

Abstract

In recent years, the fields of Natural Language Processing (NLP) and Knowledge Graphs (KG) have witnessed remarkable advancements, each independently contributing to the enhancement of various AI applications. Knowledge Graphs, structured representations of knowledge, offer a powerful framework for organizing and connecting information, facilitating efficient knowledge retrieval and reasoning. On the other hand, NLP techniques enable machines to understand, generate, and communicate in human language, bridging the gap between human and machine communication. The symbiotic relationship between Knowledge Graphs and Natural Language Processing and Understanding (NLP/NLU) is increasingly recognised as pivotal for advancing AI capabilities. In my thesis in computer science, I delve into the interplay between these two domains, exploring how they complement each other to achieve deeper semantic understanding and more sophisticated reasoning, by proposing and evaluating machine learning methods that integrate them seamlessly. More specifically, the aim is to guide the reader through the boundary connecting NLP to KGs, presenting which routes could be followed to achieve different levels of integration between the two modalities and which degree of improvement has to be expected under each different scenario. The core of the thesis is constituted by three peer-reviewed papers (of which, two were best-paper awarded) that explore different aspects of the integration between Knowledge Graphs and Natural Language. Ranging from the more simple combination of graph and text embeddings through concatenation, to the deeper construction of a multi-modal aligned text-graph space and to the more high level usage of external KGs as reservoirs of commonsense knowledge, the thesis demonstrated how the integration of the two data modalities, and their corresponding encoder models, often enabled better modeling capabilities. This hybrid integration was beneficial in both language related tasks, such as Relation/Triplet Extraction (RE/TE) and Question Answering (QA), and graph associated tasks, such as Link Prediction (LP). Various standard datasets commonly used in literature were adapted and enriched to allow for the joint processing of graph and text information, serving as benchmarks for quantitatively evaluating the improvement over the baseline. The set of tested models included Transformer-based architectures, ranging from their first iterations (e.g., Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformer (GPT)), to the more recent Large Language Models (LLM) (e.g., LLM Meta AI (LLaMA) and Falcon). On the graph side, instead, TransE embeddings as well as declinations of the Graph Convolutional Networks (GCN), like Relational GCN (RGCN) and Compositional GCN (CompGCN), were considered. Each one of these studies presented a different approach to achieve the integration and tested for different facets of reasoning and understanding natural language. However, they all demonstrated how pivotal the interaction of standard NLP models with KGs is for processing natural language. In particular, they evidenced as sometimes (specifically in the LLM case) the integration of an external KG both, leads to larger improvements and constitutes a cheaper approach, compared to, for instance, training and making use of larger, more complex language models. Therefore, the thesis offers a guideline for future research into how the integration of data modalities (and their corresponding encoder models) can enable better modeling capabilities in analysis tasks from answering questions to creating knowledge graphs. Show more