Dissertation Defense
Irene Li
Title: Neural Graph Transfer Learning in Natural Language Processing Tasks
Advisor: Dragomir Radev
Other committee members:
Marynel Vázquez
Yang Cai
Elisa Celis
Abstract:
Natural language is essential in our daily lives as we rely on languages to communicate and exchange information. A fundamental goal for natural language processing (NLP) is to let the machine understand natural language to help or replace human experts to mine knowledge and complete tasks.
Many NLP tasks deal with sequential data. For example, a sentence is considered as a sequence of works. However, not all tasks can be formulated using sequence models. Specifically, graph-structured data is also fundamental in NLP, including entity linking, entity classification, relation extraction, abstractive meaning representation, and knowledge graphs. In this scenario, BERT-based pretrained models may not be suitable. Graph Convolutional Neural Network (GCN) is a deep neural network model designed for graphs. It has shown great potential in text classification, link prediction, question answering and so on.
This dissertation presents novel graph models for NLP tasks, including text classification, prerequisite chain learning, and coreference resolution. We focus on different perspectives of graph convolutional network modeling: for text classification, a novel graph construction method is proposed which allows interpretability for the prediction; for prerequisite chain learning, we propose multiple aggregation functions that utilize neighbors for better information exchange; for coreference resolution, we study how graph pretraining can help when labeled data is limited. Moreover, an important branch is to apply pretrained language models for the mentioned tasks. So, this dissertation also focuses on the transfer learning method that generalizes pretrained models to other domains, including medical, cross-lingual, and web data. Finally, we propose a new task called unsupervised cross-domain prerequisite chain learning, and study novel graph-based methods to transfer knowledge over graphs.