Intro

Marcell Fekete

My main motivation is to create better performing NLP tools for diverse languages with a special focus on data-poor settings. In researching data-efficient methods such as language adapters, I integrate expertise from my background in Linguistics (BA Linguistics, University of Cambridge), NLP and deep learning (MA Human Language Technology, Vrije Universiteit Amsterdam).

I started my PhD in September 2022 with Johannes Bjerva at Aalborg University in Copenhagen, Denmark. Already in my first year, my extended abstract Gradual Language Model Adaptation Using Fine-Grained Typology was accepted to SIGTYP at EACL 2023 in Dubrovnik.

In the following years, I aim to build on my existing knowledge and develop my skills to fulfill my goals, especially with respect to Uralic languages that receive little attention in current NLP research.

Work

Gradual Language Model Adaptation Using Fine-Grained Typology

Extended abstract accepted at the SIGTYP workshop, presented at EACL 2023 in Dubrovnik. View poster →

Typological Challenges for the Application of Multilingual Language Models in the Digital Humanities

Abstract accepted for the submission of a full chapter in the volume Multilingual DH. Collaboration with Lisa Beinborn and Johannes Bjerva.

Cross-lingual Transfer of Multilingual Language Models Using Stacked Language Adapters

Thesis internship project with the Vrije Universiteit Amsterdam and TAUS. Image credits to AdapterHub.

Dimensionality Reduction of Multilingual Sentence Embeddings using Autoencoders

Internship project with TAUS. Image credits to Arden Dertat on Towards Data Science.

Part-of-speech Tagging Using the Hidden Layers of Multilingual BERT

Comparing layer-wise performance between English and Hungarian. Cross-lingual linguistic probing using multilingual BERT, a project for LOT (Netherlands Research School of Linguistics).

Joint Processing of Attribution Components with Contextual Embeddings

Group project at Vrije Universiteit Amsterdam, using DistilBERT to carry out attribution linking.

Chronicling Novelty: Tracking the Development of Numeracy between the 15th and 19th Centuries in Dutch Speakers

Group project at Vrije Universiteit Amsterdam as part of the larger project Chronicling Novelty.

About

I am a PhD Fellow at Aalborg University in Copenhagen, Denmark, working within the Multilingual Modelling for Resource-Poor Languages project under the supervision of Johannes Bjerva.

My academic background spans Linguistics (BA, University of Cambridge) and Natural Language Processing (MA Human Language Technology, Vrije Universiteit Amsterdam). I am particularly interested in Uralic languages and data-efficient cross-lingual transfer methods.