Overview

ARK-ASR-3B is a 3-billion-parameter multilingual automatic speech recognition model built by Audio8 that combines a Whisper-style audio encoder with an MLP adapter and a Qwen decoder. The model operates at 16 kHz sampling rate and achieves state-of-the-art performance on the Hugging Face Open ASR Leaderboard English short-form benchmark with an average word error rate of 5.04% and a real-time factor (RTFx) of 490.98. The architecture uses custom arkasr remote code to inject audio representations into the decoder by replacing audio placeholder token embeddings before transcript generation. The model supports 19 languages including Chinese, English, German, Japanese, French, Korean, Spanish, Polish, Italian, Romanian, Hungarian, Czech, Dutch, Finnish, Croatian, Slovak, Slovene, Estonian, and Lithuanian. It is distributed in safetensors checkpoint format and requires loading with trust_remote_code=True in the Hugging Face Transformers library. The accompanying training, inference, and evaluation code is open-source and available on GitHub, along with vLLM serving infrastructure for production deployment.

Best use cases

Multilingual transcription for meeting recordings and professional audio. ARK-ASR-3B excels at processing longer-form audio with complex acoustic conditions like background noise and multiple speakers, as evidenced by strong performance on the AMI (8.79% WER) and Earnings22 (8.23% WER) benchmarks. This makes it suitable for enterprise applications where accuracy on real-world meeting and earnings call recordings is critical. The model maintains quality across diverse acoustic domains, making it reliable for professional transcription services.

Chinese language speech recognition with character error rates below 5%. The model achieves 1.80% character error rate on AISHELL-1 and 4.58% on WenetSpeech test-net, positioning it as a strong choice for Mandarin transcription tasks. Organizations processing Chinese audio content benefit from both the low error rate and the model's ability to handle conversational Mandarin with minimal errors, making it suitable for customer service recordings, podcast transcription, and Chinese language media applications.

High-throughput batch transcription with efficient inference. With an RTFx of 490.98, the model processes audio roughly 490 times faster than real-time on GPUs. This efficiency allows batch processing of large audio datasets with manageable computational overhead. The model supports batch sizes up to 40 (as shown in the reference implementation) and outputs both cleaned and raw predictions, making it practical for pipelines that need to handle thousands of audio files without excessive infrastructure scaling.

Specialized audio domains including parliamentary speech and spontaneous English. Performance on GigaSpeech (6.98% WER) and VoxPopuli (5.47% WER) shows the model generalizes well to parliamentary and diverse European language audio. This makes it suitable for government, regulatory, and international organization applications where transcribing formal or semi-formal speech is necessary.

Production deployment with OpenAI-compatible API endpoints. The bundled vLLM serving infrastructure exposes both custom /asr endpoints and OpenAI-style /v1/audio/transcriptions endpoints, allowing direct drop-in deployment without custom server implementation. Organizations can stand up production transcription services using the provided deploy_ark_asr_vllm_service.sh script without architectural changes to existing systems expecting OpenAI-compatible interfaces.

Limitations

Restricted to a fixed 19-language set. Unlike some competing models supporting 1000+ languages, ARK-ASR-3B only supports the 19 explicitly listed languages. Organizations needing transcription for languages outside this set cannot use this model without retraining. The language limitation is particularly acute for rare, endangered, or low-resource languages.

Significant VRAM requirements for inference. A 3-billion-parameter model with bfloat16 precision requires approximately 6GB of VRAM at minimum for single-sample inference, and considerably more for batch processing. The reference implementation uses batch size 40 with an A100 or similar high-memory GPU. Inference on consumer GPUs with less than 12GB VRAM becomes challenging, limiting deployment to organizations with access to enterprise-grade hardware.

Inference speed requires GPU acceleration. While the RTFx of 490.98 is fast on GPUs, the model runs extremely slowly on CPU. The code examples specify device = "cuda" if torch.cuda.is_available() else "cpu" but do not specify CPU inference times, implying CPU usage is impractical. Organizations without GPU access cannot reasonably deploy this model.

Audio length limitations and potential quality degradation on very long utterances. The reference inference code sets audio_max_length=30 * 16000 samples, which equals 30 seconds of audio at 16 kHz. Longer utterances require segmentation, introducing risk of incorrect segmentation boundaries and transcription errors at break points. The model provides no guidance on optimal segment lengths or handling of speech that spans multiple segments.

Limited error analysis or failure mode documentation. The model card reports aggregate metrics but does not provide detailed failure mode analysis, common error patterns, or guidance on which acoustic conditions degrade performance most significantly. Users cannot easily predict when the model will fail or by how much quality will degrade for specific audio characteristics.

No fine-tuning guidance or infrastructure. While the model loads with Transformers, the README provides no instructions for domain-specific fine-tuning or adaptation. The reference implementation focuses on inference only, leaving organizations that need to adapt the model to proprietary domains without clear guidance on training procedures or data requirements.

Apache 2.0 license requires attribution. While commercially usable, the license requires proper attribution and compliance notices in any distribution. Some organizations may have policies restricting use of copyleft-adjacent licenses even if technically permissive.

How it compares

ARK-ASR-3B serves as the state-of-the-art English ASR baseline in this comparison. Its 5.04% average WER represents current best-in-class accuracy on standard benchmarks, making it the reference point against which other models are evaluated.

meta-omnilingual-asr-7b by Meta supports 1,693 languages compared to ARK-ASR-3B's 19, making it mandatory for organizations needing multilingual coverage. However, the 7B parameter count and larger model size likely introduce higher VRAM requirements and slower inference, suitable only when language breadth outweighs speed and efficiency concerns. Choose ARK-ASR-3B if you operate primarily in the supported 19 languages and require maximum inference speed; choose Meta's model if you need true omnilingual capability.

Qwen3-ASR-1.7B provides a smaller, faster alternative with support for 52 languages and 22 Chinese dialects. The 1.7B parameter count approximately one-third the size of ARK-ASR-3B enables lower VRAM requirements and faster inference, though likely at some accuracy tradeoff (specific performance metrics not provided in the available documentation). Choose ARK-ASR-3B if accuracy on English benchmarks is paramount and you have sufficient hardware; choose Qwen3-ASR-1.7B if you need broader language coverage and can accept slightly lower accuracy for faster deployment.

Qwen3-ASR-0.6B represents the extreme efficiency end of the spectrum at 0.6B parameters, supporting the same 52 languages as the 1.7B variant. This model enables deployment on resource-constrained hardware and mobile devices where ARK-ASR-3B is infeasible, but expects substantially lower accuracy on challenging audio. Choose ARK-ASR-3B for production systems where accuracy is non-negotiable; choose Qwen3-ASR-0.6B when hardware constraints or edge deployment requirements force acceptance of lower accuracy.

higgs-tts-3-4b is a text-to-speech model, not automatic speech recognition, and solves the inverse problem. These models serve complementary roles: ARK-ASR-3B converts speech to text; Higgs TTS converts text to expressive speech. Use ARK-ASR-3B when you need transcription; use Higgs TTS when you need synthesis.

Technical specifications

ARK-ASR-3B consists of three primary components: a Whisper-style audio encoder with rotary position embeddings (RoPE), an MLP adapter for feature projection, and a Qwen-based text decoder. The model processes 16 kHz monaural audio input and generates text output with optional post-processing cleanup. Checkpoint format is safetensors, enabling fast and safe loading.

The model operates as an autoregressive transformer where audio features are extracted by the encoder, projected through the MLP adapter, and injected into the decoder by replacing special audio placeholder token embeddings. Generation uses beam search or greedy decoding with configurable output length (default maximum 256 tokens) and custom bad-words filtering to prevent non-transcription control tokens from appearing in output.

Architecture and parameters:

  • Model scale: 3 billion parameters in the decoder LLM

  • Encoder: Whisper-style audio encoder with RoPE

  • Adapter: MLP projection layer

  • Decoder: Qwen-based language model

  • Total model size: approximately 6GB in bfloat16 precision

Input and output specifications:

  • Input: 16 kHz mono or multi-channel audio (supports audio_max_length=30*16000= 480,000 samples = 30 seconds)

  • Input format: WAV files or raw audio tensors

  • Output: UTF-8 text transcription with optional metadata preservation

  • Output includes both pred_text(cleaned) andpred_text_raw(unprocessed)

Training and evaluation:

  • Evaluated on Hugging Face Open ASR Leaderboard using standard JER/WER metrics

  • Training based on on-policy distillation from stronger ASR teacher models (OPD framework)

  • Training code derived from THUNLP/OPD and Volcengine/veRL repositories

Framework and inference:

  • Primary framework: Hugging Face Transformers (trust_remote_code=Truerequired)

  • Recommended attention implementation: SDPA (scaled dot-product attention)

  • Supported dtype: bfloat16 on CUDA, float32 on CPU

  • Batch inference: supports configurable batch sizes (reference uses batch_size=40)

  • Serving: vLLM-compatible with custom model registration

Computational requirements:

  • VRAM: minimum 6GB for single inference, 20GB+ for batch_size=40 with bfloat16

  • Real-time factor: 490.98x (processes audio 490 times faster than real-time on GPU)

  • Inference time: approximately 30ms per second of audio on A100-class GPUs

  • CPU inference: impractical; no timing provided

Supported languages (19 total):

Chinese, English, German, Japanese, French, Korean, Spanish, Polish, Italian, Romanian, Hungarian, Czech, Dutch, Finnish, Croatian, Slovak, Slovene, Estonian, Lithuanian.

Model inputs and outputs

Inputs

  • Audio file path(string): filesystem path to WAV file or other supported audio format

  • Audio tensor(float32 or float16): raw audio samples at 16 kHz sampling rate, shape (batch_size, max_samples) with max_samples ≤ 480,000 (30 seconds)

  • Chat template(dict): conversation format with user message containing audio type and optional text prompt

  • Generation parameters(optional): max_new_tokens (default 256), temperature, top_p, do_sample

Outputs

  • Transcription text(string): UTF-8 encoded transcript of audio content

  • Cleaned prediction(- pred_text): post-processed transcript with ASR-specific token filtering

  • Raw prediction(- pred_text_raw): unprocessed model output before cleanup

  • Input metadata(dict): original input fields preserved in batch inference output JSONL

  • Token IDs(tensor): underlying model generation in token space if raw outputs requested

Getting started

import torch from transformers import AutoModelForCausalLM, AutoProcessor, AutoTokenizer model_path = "AutoArk-AI/ARK-ASR-3B" audio_path = "path/to/your/audio.wav" device = "cuda" if torch.cuda.is_available() else "cpu" torch_dtype = torch.bfloat16 if device == "cuda" else torch.float32 processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True) tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained( model_path, trust_remote_code=True, torch_dtype=torch_dtype, attn_implementation="sdpa", ).to(device) model.eval() def build_bad_words_ids(tokenizer): eos_ids = tokenizer.eos_token_id keep_ids = {eos_ids} if isinstance(eos_ids, int) else set(eos_ids or []) bad_ids = set(tokenizer.all_special_ids) - keep_ids bad_ids.update( token_id for token, token_id in tokenizer.get_added_vocab().items() if token.startswith("<") and token.endswith(">") and token_id not in keep_ids ) return [[token_id] for token_id in sorted(bad_ids)] conversation = [ { "role": "user", "content": [ {"type": "audio", "path": audio_path}, {"type": "text", "text": "Transcribe this audio."}, ], } ] inputs = processor.apply_chat_template( conversation, add_generation_prompt=True, return_tensors="pt", sampling_rate=16000, audio_padding="longest", text_kwargs={"padding": "longest"}, audio_max_length=30 * 16000, ) inputs = inputs.to(device) if "audios" in inputs: inputs["audios"] = inputs["audios"].to(dtype=torch_dtype) bad_words_ids = build_bad_words_ids(tokenizer) with torch.inference_mode(): outputs = model.generate( **inputs, do_sample=False, max_new_tokens=256, pad_token_id=tokenizer.pad_token_id, eos_token_id=tokenizer.eos_token_id, bad_words_ids=bad_words_ids, ) transcription = tokenizer.batch_decode( outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=True, )[0] print(f"Transcription: {transcription}")
For batch JSONL processing of many audio files:

```
git clone https://github.com/AutoArk/open-audio-opd
cd open-audio-opd
pip install -e .

Create input JSONL with one entry per line:

{"audio":"/path/to/audio.wav","text":"","task":"asr","begin_time":-1,"end_time":-1}

python scripts/infer/ark_asr_transformers.py \
--input /path/to/input.jsonl \
--output predictions.jsonl \
--model_path AutoArk-AI/ARK-ASR-3B \
--processor_path AutoArk-AI/ARK-ASR-3B \
--batch_size 40 \
--dtype bfloat16 \
--attn_impl sdpa
```
For production deployment with vLLM and OpenAI-compatible endpoints:

```
MODEL=AutoArk-AI/ARK-ASR-3B GPU=0 PORT=8025 \
scripts/vllm/deploy_ark_asr_vllm_service.sh start

Test with a curl request:

curl -X POST http://127.0.0.1:8025/v1/audio/transcriptions \
-F file=@audio.wav \
-F model=ark-asr
```

Frequently asked questions

Q: Can I use ARK-ASR-3B for commercial applications?

A: Yes. The model is licensed under Apache 2.0, which permits commercial use, modification, and distribution. You must include a copy of the license and provide attribution to the original creators. You should verify compliance with all Apache 2.0 requirements for your specific use case.

Q: What GPU VRAM do I need to run this model?

A: Minimum 6GB for single inference with bfloat16 precision; 20GB or more for batch size 40 (the reference batch size). CPU inference is impractical and not recommended. An A100, H100, L40S, or equivalent enterprise GPU is recommended for production batch transcription.

Q: How does ARK-ASR-3B compare to Qwen3-ASR-1.7B for English transcription?

A: ARK-ASR-3B achieves 5.04% average WER on English benchmarks, representing current state-of-the-art. Qwen3-ASR-1.7B is not evaluated on the same English benchmarks, but its smaller parameter count (1.7B vs 3B) typically trades some accuracy for faster inference and lower VRAM requirements. Choose ARK-ASR-3B if English accuracy is critical; choose Qwen3-ASR-1.7B if you need broader language support and can accept potential accuracy reduction.

Q: What happens if I input audio longer than 30 seconds?

A: The model's audio_max_length parameter is set to 480,000 samples (30 seconds at 16 kHz) by default. Audio longer than this must be segmented before processing. The reference implementation does not automatically segment long audio, so you must split files manually or modify the inference script. Segmentation introduces risk of incorrect boundaries and transcription errors where speech spans segments.

Q: Can I fine-tune this model on my own domain-specific data?

A: The model loads in Transformers with trust_remote_code=True, enabling standard fine-tuning with LoRA or full parameter updates. However, the README provides no fine-tuning guidance, training hyperparameters, or data requirements. You must refer to the open-source training code in the AutoArk/open-audio-opd repository and adapt the OPD on-policy distillation framework to your use case. Fine-tuning infrastructure exists but is not documented in the model card.

Q: Does ARK-ASR-3B perform language identification automatically?

A: No. The model expects input in one of its 19 supported languages but does not identify which language is being spoken. For multilingual audio where language varies by segment, you must implement separate language identification before running transcription, or run multiple language-specific ASR passes.

Q: What is the inference latency for a typical short audio clip?

A: The model achieves RTFx of 490.98, meaning a 10-second audio clip processes in approximately 20 milliseconds on an A100-class GPU. Actual latency depends on batch overhead, audio preprocessing, and hardware. Single-sample inference is faster than batch inference per-sample due to fixed overhead, but batch processing is more efficient per-audio-file when processing many files.

Q: Is the model still actively maintained and updated?

A: The repository linked in the documentation (AutoArk/open-audio-opd) contains the training and serving code. Maintenance status is not explicitly stated, but the arXiv paper (2605.28139) indicates the model represents current research as of 2026, suggesting active development. Check the GitHub repository for recent commits and issues to assess current maintenance status.