Bengali.AI Speech Recognition: Multi-Module ASR Pipeline
From July to October 2023, I led a team of 2 in the Kaggle competition Bengali.AI Speech Recognition, and we won a silver medal, ranking 22nd of 744 teams.
- Our solution writeup on Kaggle, by my teammate Mutian Hong: 22nd Place Solution for the Bengali.AI Speech Recognition Competition
Pipeline
We developed an automatic speech recognition (ASR) pipeline for a low-resource language (Bangla/Bengali). With supervised finetuning from augmented low-resource speech datasets, the inference pipeline mainly consists of:
- denoising;
- inference with the main ASR decoder model, a finetuned Wav2Vec2, whose decoder probabilities are fused with those of an n-gram draft language model;
- post-processing that restores the punctuation of the plain words with a BERT-based punctuation prediction model.
My part
I worked mainly on the n-gram and punctuation enhancement and on the finetuning of the Wav2Vec2 model.