Yandex introduced Sona, a generative AI model that replaces traditional multi-stage recommendation cascades with a single unified system. Tested during a seven-day live production experiment on Yandex smart speakers, Sona successfully replaced more than 15 candidate generators, the pre-ranking stage, and the ranking stage with one served transformer.
The Sona architecture combines candidate generation and ranking around a shared user representation. The system utilizes a semantic tokenizer, an encoder with history compression that attends to 8,192 past events, a 2-layer decoder, and a ranking module trained via rollout distillation from a frozen teacher ranker. Running on NVIDIA Triton Inference Server with CUDA graphs, the model achieves 41 percent model FLOPs utilization, maintaining median end-to-end latency of 45 minutes and p99 latency of 60 minutes while updating serving weights every 10 minutes.
Why it matters: Sona streamlines production recommendation pipelines by eliminating hand-engineered features and multi-stage complexity in favor of a single generative architecture.