← back
Personalization in the Era of LLMs - Shivam Verma, Spotify
Takeaway
Combining behavioral representations with language interfaces can make personalization more expressive and controllable.
Summary
- Spotify connects user modeling, catalog understanding, and LLMs to make recommendations both personalized and steerable.
- Products such as AI DJ, prompted playlists, and taste profiles expose natural-language control over recommendations and preferences.
- The organization is moving from separate candidate-generation and ranking models toward a more unified foundation-model approach.
- Daily user embeddings summarize listening history at billion-user scale, with sequential modeling replacing older generalized autoencoder representations.
spotifypersonalizationrecommendation-systems
Original description
Spotify represents Ariana Grande and Bruno Mars as sequences of six tokens. The first two are shared because both are pop artists. The remaining tokens diverge to capture what makes each distinct. That is a Semantic ID, and it is how Spotify teaches open-weight LLMs to reason over a catalog of 100 million tracks the same way they reason over words. Shivam Verma from Spotify's AI foundation team walks through the three components they assembled to personalize LLMs at scale without full fine-tuning. User embeddings trained on streaming history across 750 million users form the base. Semantic IDs compress catalog vectors into tokens the model can autoregressively generate, predicting the next song or episode as the next token in a sequence. A soft tokenization layer projects a user's embedding directly into the LLM's token space, giving the frozen model a user-specific token to attend over. Podcast next-episode recommendations are already running on this stack in production. Speaker info: https://x.com/kaffeinated / shivam13verma