In this article, I will show how we converted Dunstan Matekenya’s Chichewa Whisper model to ONNX and running it within Python. Chichewa is one of the national languages of Malawi, fondly known as the warm heart of Africa. Our home. Chichewa is considered a low-resource language and there are not that many resources around transcription and audio based models for it.
Fortunately, there is an ongoing effort to change this status quo, which we are eager to support by putting into practical implementation. More on that in the future.
For this article, we will focus on converting to onnx and running the model locally.
Firstly, we need to access the model from here: https://huggingface.co/dmatekenya/whisper-large-v3-chichewa-variant-b-normalized-transcript. This is the most recent model at the time of writing, though this process works with all the models published on the huggingface profile.
What is ONNX?
ONNX, or Open Neural Network Exchange, is an open standard file format used to represent and share machine learning and deep learning models. ONNX was started as an initiative by Microsoft engineers to standardize on a common format for ML models. Microsoft do have their moments, to be honest ;).
The cool thing about ONNX is that you can take the same model (.onnx) file and use it in other languages like Java (See this article)
Converting the Chichewa Whisper model to ONNX
Next, we will need to convert the model to ONNX. To achieve this, we will be using Huggingface’s Optimum-CLI project.
We prefer uv and recommend it for use over virtualenv or other similar tools - due to it’s speed and ergonomics.
$ mkdir chichewa-whisper-onnx
$ cd chichewa-whisper-onnx
$ uv init
$ uv add "optimum[exporters]" "optimum[onnx]"
Next, we run the conversion using optimum-cli. You will need about 6GB to 10GB of RAM and disk space to complete this step.
NOTE: You will need about 6GB to 10GB of RAM and disk space to complete this step.
It will also be faster to download the model if you have a Huggingface token configured to avoid HF’s rate limiting for unauthenticated requests.
% export HF_TOKEN="your HF token"
$ uv run optimum-cli export onnx --model "dmatekenya/whisper-large-v3-chichewa" whisper-chichewa-onnx
The conversion may take a while, depending on your hardware/environment. If it completes successfully, you should have the new directory whisper-chichewa-onnx with generated .onnx files along with some other files.
It could look something like this
$ ls whisper-chichewa-onnx
added_tokens.json decoder_model.onnx_data encoder_model.onnx preprocessor_config.json
config.json decoder_with_past_model.onnx encoder_model.onnx_data vocab.json
decoder_model.onnx decoder_with_past_model.onnx_data generation_config.json model.onnx
Now that we have gotten this far, we can try to play with the model using the onnxruntime library in Python.
Recording sample Chichewa audio and converting to .wav
First, record a short sample of audio with some Chichewa and place it in a file name sample.wav. Whisper expects audio in WAV format, you may have to use
ffmpeg to convert e.g. an .m4a to .wav.
$ ffmpeg -i input.m4a -ar 16000 -ac 1 sample.wav
Testing the ONNX model via onnx runtime
Next we will use the onnxruntime library in Python to confirm it works.
Before we proceed, add the following dependencies for the script to work : librosa for audio processing, onnxruntime and the usual transformers, thought we need to pin to version 5.9.0
$ uv add onnxruntime "librosa>=0.11.0" "transformers>=5.9.0"
Then replace the main.py with the following:
import librosa
from transformers import WhisperProcessor
from optimum.onnxruntime import ORTModelForSpeechSeq2Seq
import onnxruntime as ort
# --- CONFIGURATION ---
model_id = "dmatekenya/whisper-large-v3-chichewa"
whisper_base_model_id = "openai/whisper-large-v3"
whisper_base_model_language = "shona"
onnx_model_dir = "chichewa-kumva-w3-onnx"
audio_path = "sample.wav"
# --- SETUP CPU OPTIMIZATIONS ---
sess_options = ort.SessionOptions()
sess_options.intra_op_num_threads = 4
# --- LOAD PROCESSOR AND MODEL ---
print("Loading processor and ONNX model...")
processor = WhisperProcessor.from_pretrained(whisper_base_model_id, \
local_files_only=False, \
language=whisper_base_model_language, \
task="transcribe")
model = ORTModelForSpeechSeq2Seq.from_pretrained(
onnx_model_dir,
provider="CPUExecutionProvider", # you can omit this and let the library use available GPUs
session_options=sess_options,
use_merged=False
)
# --- LOAD AND PREPARE AUDIO ---
print(f"Loading audio from {audio_path}...")
audio_input, sample_rate = librosa.load(audio_path, sr=16000)
inputs = processor(audio_input, sampling_rate=16000, return_tensors="pt")
attention_mask = inputs.get("attention_mask")
# --- RUN INFERENCE ---
print("Running inference...")
generated_ids = model.generate(
inputs["input_features"], # <--- FIXED: Safer dictionary access
attention_mask=attention_mask,
pad_token_id=processor.tokenizer.eos_token_id,
max_length=255,
language="sn",
task="transcribe"
)
transcription = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(f"\nTranscript from model:\n{transcription}")
We can now run this via uv:
$ uv run main.py
If that works, you should hopefully get a transcription of the Chichewa audio.
Note on model quality
It’s important to note that the Chichewa model is trained on very few hours of data (actually the author’s haven’t published the dataset) and is fine-tuned on Shona/Kiswahili so the transcription won’t be perfect. From our conversations with the authors, we understand that there are some challenges getting more data samples and training infrastructure.
We are looking into how to support this effort and welcome the AI research community in Malawi to join us to help make the model the best it can be.
Conclusion
In this experimental/beaker article, we showed how to convert a whisper model to ONNX and run it via Python using the onnxruntime. This is a starting point for some very exciting work down the line to improve Speech transcription and ASR use cases for a low resource language, Chichewa.