Case study · 2024
Tani Speech Corpus Pipeline
An end-to-end audio collection, alignment, and transcription pipeline for under-documented Tani languages, built so contributors with patchy internet can record from a phone.
- Year
- 2024
- Role
- Founder + Engineer
- Status
- still building
- Stack
- Python, Whisper, FFmpeg, FastAPI, PostgreSQL, S3, Svelte
Challenge
Most existing speech tooling assumes a high-resource language and a fast, stable connection. Tani-family languages have neither. Contributors are often on intermittent rural data, and there is no off-the-shelf model that handles the phonotactics well.
Process
I designed an offline-first mobile capture flow that buffers locally and resumes on reconnect. On the server, a queue runs forced alignment with a custom phoneme inventory and produces a draft transcription that a human reviewer can correct. Every correction feeds back into the model.
Outcome
Over 60 hours of clean, aligned Galo and Nyishi speech captured across 14 contributors. The transcription model's word error rate has dropped from 41% to 17% across the last three retraining cycles.
The thing nobody warns you about with low-resource language work is how much of the engineering ends up being about the connection, not the model.
I started this assuming I’d be fine-tuning ASR models all day. In practice, the first three months were almost entirely about making sure a contributor sitting on a slow 3G signal in a village in West Siang district could open the app, record a clean 90-second sample, and have it actually reach the server an hour later when they walked closer to a tower. Without that, the model doesn’t matter.
Now that the pipeline is reliable, the modeling work is the fun part. The phonotactic constraints in Tani languages are different enough from Mandarin or English that a vanilla Whisper finetune underperforms a smaller, phoneme-aware model trained from scratch. That was a surprise.