Banca de QUALIFICAÇÃO: SILVAN FERREIRA DA SILVA JÚNIOR

Uma banca de QUALIFICAÇÃO de DOUTORADO foi cadastrada pelo programa.
STUDENT : SILVAN FERREIRA DA SILVA JÚNIOR
DATE: 29/07/2022
TIME: 14:00
LOCAL: Google Meet
TITLE:

A Neuro-Symbolic Approach to Logical Reasoning with Cross-Modal Alignment and Memory Augmented Transformers

KEY WORDS:

Neuro-Symbolic, Artificial Intelligence, Logic Reasoning,
Transformers.

PAGES: 70
BIG AREA: Engenharias
AREA: Engenharia Elétrica
SUMMARY:

Modern machine learning is heavily based on connectionist models that are capable of achieving groundbreaking results in several tasks involving perception and feature extraction. Nevertheless, this approach tends to work as "gray boxes" since it lacks explainability, which can also bring questions about the trustworthiness of the system. In this thesis, we propose a Neuro-Symbolic system capable of extracting structured information about the objects in the environment, e.g. persons or objects in an image or sentences in a text, as well as converting a question in natural language into an intermediary query and execute it. The entities in the query, i.e., parts of the text relating to characteristics of the objects, are extracted and passed to a pre-trained text encoder to produce embedding vectors. The same process is made to the objects, using a convenient pre-trained model, depending on the domain of the objects. The embeddings from entities and objects are passed to an alignment model to measure the correspondence between them, to construct an alignment matrix that has information about the object-entity pair, to be used in the logical reasoning module and evaluate the input queries. For this thesis proposal, the logical reasoner is implemented and tested in two ways: as an interpreter, that executes an abstract syntax tree constructed from the intermediary query language, and as a memory augmented Transformer, pre-trained in a self-supervised way with synthetic data for, in an auto-regressive approach, compute the output by reading and writing in the memory sequentially. Experiments show that good results and a high level of explainability and robustness can be reached. Additionally, the framework proposed in this thesis allows for extending this approach to different domains, such as text and video.

COMMITTEE MEMBERS:
Presidente - 2579664 - ALLAN DE MEDEIROS MARTINS
Interno - 2885532 - IVANOVITCH MEDEIROS DANTAS DA SILVA
Externo à Instituição - FRANCISCO DE ASSIS BRITO FILHO - UFERSA