Overview
dots.llm1.inst is an instruction-tuned mixture-of-experts language model built by dots-studio that activates 14 billion parameters out of a total of 142 billion parameters during inference, delivering performance comparable to Qwen2.5-72B while requiring significantly less computational resources. The model uses a 62-layer architecture with 32 attention heads, multi-head attention with QK-Norm in each attention layer, and fine-grained MoE routing that selects top-6 experts out of 128 routed experts plus 2 shared experts. It supports a 32,768 token context window, handles both English and Chinese text, and uses the transformers library for inference.
The model was pretrained using a three-stage data processing framework on high-quality, non-synthetic corpora without synthetic data generation, and then instruction-tuned through supervised fine-tuning. All training happens under the MIT license, enabling both academic and commercial use.
Best Use Cases
Chat and instruction-following tasks. This model is specifically instruction-tuned, making it ideal for conversational AI applications, task completion, question-answering systems, and agents that must follow specific formatting or behavioral directives. The instruction-tuned variant performs better on tasks with explicit user instructions compared to the base model.
Multilingual content generation in English and Chinese. The pretraining included both languages with equal emphasis, making this model capable of generating coherent responses in either language. This is particularly valuable for companies serving Asian markets or maintaining multilingual products without maintaining separate models.
Production deployments with constrained hardware budgets. Because only 14B parameters activate at inference time despite the 142B total, this model offers the quality and capability of much larger models while reducing memory requirements, inference latency, and computational costs significantly. Organizations can run this on fewer GPUs than would be required for a full 72B parameter model.
Long-context document processing and analysis. The 32,768 token context window enables processing of lengthy documents, code repositories, or conversation histories in a single inference pass. This makes it suitable for summarization, code review analysis, or context-aware question-answering over large texts.
Research and benchmarking with training dynamics insights. The team released intermediate checkpoints spanning the entire training process, enabling researchers to study how language model capabilities emerge during training and providing concrete examples of learning dynamics in large-scale MoE systems.
Limitations
Inference requires specialized infrastructure. The documentation recommends tensor-parallel-size 8, meaning the model expects distributed inference across at least 8 GPUs. Running this on single or dual-GPU setups is not practical, which eliminates it from many edge deployment scenarios and typical developer laptops. The model requires GPU infrastructure with sufficient inter-node bandwidth for all-to-all MoE communication.
Limited evaluation transparency. While the technical report exists, detailed evaluation results are relegated to an external PDF without specific benchmark numbers or comparison tables in the primary documentation. Practitioners cannot easily determine exact performance margins against specific competitors from the model card alone.
MoE-specific considerations during fine-tuning. The mixture-of-experts architecture introduces routing complexity that may complicate custom fine-tuning compared to dense models. Load balancing across experts during SFT requires careful attention, and the specialized infrastructure recipe limits where fine-tuning can occur.
Relatively low download count despite strong performance. With only 7,375 downloads, this model has minimal production deployment evidence compared to widely-used alternatives, which means fewer community examples, fewer third-party optimizations, and potentially undiscovered edge cases in production use.
Hardware memory requirements are not specified. The documentation does not explicitly state VRAM requirements per GPU or total memory needed for inference at various batch sizes, forcing practitioners to estimate or test directly.
How It Compares
dots.llm1.base is the pre-instruction-tuned version with identical architecture and parameters. Choose dots.llm1.inst if you need the model to follow instructions, answer questions, and engage in chat-style interactions; use the base model if you plan extensive custom fine-tuning or need a foundation for specialized domain adaptation where instruction-tuning would be undesirable.
dots.vlm1.inst is a vision-language model from the same studio built on a 1.2 billion parameter backbone. This model is suitable only if you need to process and reason about images; dots.llm1.inst remains the choice for text-only applications and offers dramatically larger capability through its MoE architecture.
dots.tts-base is a 2-billion parameter text-to-speech system using continuous end-to-end autoregressive modeling. This model handles completely different modalities; if you need speech synthesis, use dots.tts-base; if you need language understanding and generation, use dots.llm1.inst.
dots.ocr is a multilingual document layout parsing vision-language model specialized for optical character recognition and document understanding. Choose dots.ocr for document extraction and layout parsing tasks; use dots.llm1.inst for processing the extracted text or for general language tasks without visual input.
Technical Specifications
Architecture and parameters: Mixture-of-experts model with 142 billion total parameters and 14 billion activated parameters at inference. Uses 62 transformer layers with 32 attention heads per layer. Multi-head attention includes QK-Norm in each attention layer for stability. Fine-grained MoE routing selects the top-6 experts from a pool of 128 routed experts plus 2 shared experts that always activate.
Training approach: Two-stage training consisting of pretraining on high-quality, non-synthetic corpora followed by supervised fine-tuning (SFT) for instruction-following capabilities. The three-stage data processing framework generates large-scale, high-quality, and diverse pretraining data at scale.
Infrastructure optimizations: Uses interleaved 1F1B pipeline scheduling combined with efficient grouped GEMM implementations to overlap MoE all-to-all communication with computation, boosting computational efficiency.
Input and output specifications:
- Context window: 32,768 tokens
- Supported languages: English, Chinese
- Output format: Text generation with configurable max_tokens and temperature parameters
- Quantization: Supports bfloat16 precision as documented in examples
Supported frameworks and libraries:
- Transformers (official support pending via PR #38143)
- vLLM (official support via PR #18254)
- SGLang (official support via PR #6471)
- Docker support via official Docker Hub images
License: MIT, permitting commercial and private use with minimal restrictions.
Model availability: Two variants available—dots.llm1.base (pretrained only) and dots.llm1.inst (instruction-tuned). Intermediate training checkpoints are released throughout the training process for research purposes.
Model Inputs and Outputs
Inputs
- Text prompts: Plain text strings for text completion or chat messages.
- Chat format: System role (optional), user role, and assistant role messages using the standard chat template.
- Generation parameters: max_new_tokens, temperature, and other standard generation configuration.
- Batch processing: Supports batched inference through vLLM or SGLang.
Outputs
- Generated text: Completion tokens decoded from logits without special tokens by default.
- API format: OpenAI-compatible API when served via vLLM or SGLang returns JSON with model name, messages, and usage statistics.
- Raw tokens: Model outputs logits over vocabulary; decoding to text is handled by tokenizer.
Getting Started
Text completion with Transformers:
```
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "rednote-hilab/dots.llm1.inst"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
device_map="auto",
torch_dtype=torch.bfloat16
)
messages = [
{"role": "user", "content": "Explain quantum computing in simple terms"}
]
input_tensor = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
)
outputs = model.generate(
input_tensor.to(model.device),
max_new_tokens=200
)
result = tokenizer.decode(outputs[0][input_tensor.shape[1]:], skip_special_tokens=True)
print(result)
```
Serving with vLLM:
vllm serve dots.llm1.inst --port 8000 --tensor-parallel-size 8
Then query via OpenAI-compatible API:
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "dots.llm1.inst",
"messages": [{"role": "user", "content": "Hello"}],
"max_tokens": 100
}'
Frequently Asked Questions
Q: Can I use dots.llm1.inst commercially?
A: Yes, the model is released under the MIT license, which permits commercial use, modification, and distribution with minimal restrictions. You must retain the license and copyright notice in any distributions.
Q: What GPU setup do I need to run this model?
A: The recommended configuration uses 8 GPUs with tensor parallelism (--tensor-parallel-size 8), as documented in the examples. The model uses all-to-all MoE communication, so GPUs should have high-bandwidth interconnect. Single or dual-GPU setups are not supported by the provided infrastructure.
Q: How does dots.llm1.inst compare to Qwen2.5-72B in performance?
A: The model achieves performance comparable to Qwen2.5-72B while activating only 14B parameters instead of 72B, resulting in significantly faster inference and lower memory requirements. Exact performance margins are detailed in the technical report.
Q: Can I fine-tune this model for my specific domain?
A: The model is instruction-tuned already, so standard supervised fine-tuning should work through Transformers. However, MoE fine-tuning introduces additional complexity around expert load balancing that requires careful attention and specialized infrastructure.
Q: Does this model support languages other than English and Chinese?
A: The pretraining explicitly supports English and Chinese. Other languages may work to some degree through transfer learning, but the model is not optimized or validated for languages outside these two.
Q: What is the difference between dots.llm1.base and dots.llm1.inst?
A: dots.llm1.base is the pretrained model without instruction-tuning, suitable for continued pretraining or domain-specific fine-tuning. dots.llm1.inst is instruction-tuned for chat and task completion, making it immediately usable for conversational applications.
Q: How does the 14B activated vs 142B total parameter split affect inference speed?
A: Only 14B parameters participate in forward passes during inference, making computation requirements approximately 10x lower than a dense 142B model. Exact speed depends on the MoE routing efficiency and hardware communication overhead.
Q: Is this model actively maintained, and will there be updates?
A: The model was released on June 6, 2025. The team maintains Docker images with specific vLLM versions and has open pull requests for integration into Transformers, suggesting active maintenance. Check the Hugging Face model page and GitHub repository for the latest updates.
Image source: AIModels.fyi
This is a simplified guide to an AI model called dots.llm1.inst maintained by dots-studio. If you like these kinds of analyses, join AIModels.fyi or follow us on Twitter.