IndexTTS 1
bilibili · Industrial-grade controllable zero-shot TTS · First released Feb 2025
IndexTTS is the first-generation system from bilibili's Index Speech Team. It builds on XTTS and Tortoise rather than starting from scratch, and its headline contribution is making Chinese pronunciation controllable — a problem that English-centric TTS research had largely ignored. For an international audience, the important detail is the release timeline: v1.0 (Mar 2025) was followed by v1.5 (May 2025), which specifically and significantly improved English performance and overall stability. If you evaluate IndexTTS 1, evaluate v1.5, not v1.0.
- Developer
- bilibili Index Speech Team
- Initial release
- Feb 2025 (weights Mar 2025)
- Architecture
- Autoregressive (GPT-style)
- Languages
- Chinese, English (+ dialects)
- VRAM
- ~11 GB
- Throughput
- ~1 min audio in ~10 s
Contribution 1 — Hybrid character + pinyin modeling
This is the core idea. Chinese polyphones (characters with multiple readings depending on context) are a persistent failure mode for TTS. IndexTTS let users inject pinyin directly into the input text, and trained the model to cope with it via stochastic augmentation:
- During training, roughly 20% of non-polyphonic characters are replaced with their pinyin form using a Bernoulli mask (α = 0.20). This forces the model to learn a robust mapping between graphemes and phonemes.
- Adversarial training reduced the mispronunciation rate of homophones from 8.7% to 0.9%.
- At inference you can write
zhong4 zhong4inline to disambiguate — the same mechanism also helps with rare and out-of-vocabulary characters.
Why this matters beyond Chinese: the same architecture trick is directly reusable for any language with irregular pronunciation. It is functionally similar to what CMU phoneme overrides do for English.
Contribution 2 — Precise pause / prosody control
Punctuation is mapped to explicit pause durations via timestamp embeddings (roughly 0.3 s for a comma, 0.8 s for a full stop). On classical Chinese long-form text, sentence-boundary accuracy reached 98.6%.
Contribution 3 — Encoder and vocoder upgrades
- A Conformer-based speech conditional encoder replaced the baseline conditioning stack, improving zero-shot cloning stability and speaker similarity.
- The speech-code decoder was replaced with BigVGAN2, a meaningful jump in output naturalness.
- The team published a comparison of Vector Quantization (VQ) vs Finite Scalar Quantization (FSQ) for codebook utilization: at a 6,000-hour data scale, FSQ reached close to 100% codebook utilization versus roughly 55% for VQ — a finding later inherited by the whole IndexTTS lineage.
Reported results (self-evaluated)
| Metric | Value | Notes |
|---|---|---|
| Word error rate (WER) | 1.3% | Chinese scenario |
| Speaker similarity (SS) | 0.776 | Zero-shot cloning |
| MOS | 4.01 | Subjective quality |
These figures were reported as beating contemporaneous open-source systems including CosyVoice2, Fish-Speech, FireRedTTS and F5-TTS, while using a simpler training pipeline and faster inference.
Use cases: when to pick 1
| Scenario | How to run it | Watch out for |
|---|---|---|
| Chinese-only batch narration (cost-sensitive) | Chunk the script and run it in one pass, locking timbre with a few seconds of reference audio | ~11 GB VRAM; no duration control, so frame-locked dubbing is out |
| Content dense with names, jargon or classical Chinese | Mixed "hanzi + pinyin" input fixes polyphones — homophone misreads drop from 8.7% to 0.9% | Pinyin annotations need a human pass; cost scales with script length |
| Reading where pause rhythm matters | Punctuation maps directly to pause durations; ~98.6% sentence-boundary accuracy on classical text | No continuous speed control — you tune rhythm by editing punctuation |
| A baseline for version comparison | When you're unsure whether you actually need 2/2.5, run 1 first | English requires v1.5 or later; v1.0 English is not a valid benchmark |
Verdict for international users
Use it if you need a light, stable Chinese-centric cloning model and you have ~11 GB of VRAM to spare. It remains a solid, well-tested baseline.
Skip it if you need emotional control, precise duration control, or more than two languages — those arrived only in IndexTTS2 and 2.5. Note the English-language quality gap between v1.0 and v1.5 was substantial.
Where it sits in the head-to-head data
IndexTTS 1 did not take part in the CV3-Eval head-to-head — that evaluation covers the newer generation (IndexTTS2.5, VoxCPM2, OmniVoice, CosyVoice 3, Qwen3-TTS, MOSS-TTS and others). Its role in this report is therefore generational baseline, not competitor. Its own absolute numbers are in the "Reported results" block above.
| Reference point | Figures | Basis |
|---|---|---|
| IndexTTS 1 (this model) | WER 1.3% / SS 0.776 / MOS 4.01 | Author-reported |
| IndexTTS2 (direct successor) | WER 1.88% / emotion similarity 0.887 | Author-reported, strong-emotion condition |
| IndexTTS2.5-RL (two generations on) | zh→en 3.55 WER / 67.47 SS | CV3-Eval — see 06.2 |
What it costs to run yourself
IndexTTS 1 never published a comparable RTF with stated hardware, so it cannot enter the GPU-hour table in 06.4. These are the only usable anchors:
| Item | Value | Note |
|---|---|---|
| VRAM | ~11 GB | Higher than 2.5's ~6 GB; a single RTX 4090 is enough |
| Author-claimed speed | ~10 s per minute of audio | Hardware unspecified — not comparable with any other RTF here |
| Conservative estimate (borrowing IndexTTS2's 0.3257) | 5.9 GPU-hours / million characters | ~$4.34 on on-demand 4090; ~$9.32 on A100 |
Licence and disclosure gate
| Item | Status |
|---|---|
| Code licence | Open source |
| Weights licence | Index Model License — needs line-by-line review; this report found no explicit commercial grant |
| Commercial use | Terms must be reviewed. Do not assume yes |
| Default watermark | Not stated → plan on the assumption there is none; add your own if you have a disclosure duty |
| Data residency | Runs fully offline |
If not this, then what
- Chinese cloning, and you can move to a newer release → IndexTTS2.5. Faster, five languages, finer pronunciation control (pinyin / CMU / kana), lower VRAM.
- The licence has to be clean → VoxCPM2 (Apache-2.0 for code and weights, 48 kHz).
- Extremely low footprint, CPU or edge → Kokoro 82M (outside this set, Apache-2.0, ~2–3 GB, but it cannot clone).
- You specifically need 1 as a baseline for a paper-style comparison → keep it, but always report both v1.0 and v1.5 version numbers or your conclusion will mislead people.
Ready to create with IndexTTS 1?
Open Text to Speech with this engine selected and start with your own voice model.