Netflix’s LLM-Inspired Recommendation Foundation Model at Scale

Netflix’s recent work on "LLM-powered recommendations" offers a rare public view into how foundation models are deployed at a global scale. Rather than introducing new interfaces, it shows how generative models function as core infrastructure under strict latency, cost, and freshness constraints. For enterprise AI teams, Netflix illustrates how value emerges from system design, not standalone models.

Context and Source Foundations

Netflix’s "LLM-powered recommendations" (in the sense used by Netflix engineers) is less about deploying a general-purpose chatbot for browsing, and more about bringing the transformer/foundation-model playbook to recommender systems. The goal is to train a large, semi-supervised, sequence model on massive user interaction histories, then reuse it across many recommendation surfaces (homepage rows, title-to-title, retrieval, ranking, etc.) via several integration patterns.

Scale is the core constraint. Netflix reported 260.28M global streaming paid memberships at the end of Q4 2023, later 301.63M by the end of Q4 2024, and then reported crossing 325M paid memberships during Q4 2025. In that environment, the recommendation stack must serve global, latency-sensitive interfaces and retrain or refresh frequently to keep up with catalog changes and drifting tastes.

Netflix has disclosed the most relevant LLM-adjacent recommendation details in four primary, engineer-authored sources (plus supporting papers and slides):

• Foundation Model for Personalized Recommendation (March 2025), which explains the model motivation, tokenization, long-context handling, and cold-start strategy.
• Integrating Netflix’s Foundation Model into Personalization applications (November 2025), which focuses on production integration patterns: embeddings, subgraph, and fine-tuning.
• Towards Generalizable and Efficient Large-Scale Generative Recommenders (January 2026), which details scaling to roughly one billion parameters, vocabulary and decoding efficiency, latency-misalignment mitigation, and cold-start adaptation via multimodal item towers.
• Post-Training Generative Recommenders with Advantage-Weighted Supervised Fine-tuning (A-SFT) (October 2025), which adapts LLM-style post-training ideas to recommendation under missing counterfactuals and noisy rewards.

Reference Architecture for Netflix’s LLM-Powered Recommendation Stack

Netflix’s public descriptions imply a layered discovery architecture where a large transformer-based recommendation foundation model sits alongside (not necessarily replacing) existing retrieval and ranking systems, feeding them with embeddings, features, or acting as an in-graph submodel.

A practical distilled architecture, consistent with Netflix’s described integration options, looks like this:

Offline / training plane (hours → days)

• Event and content data transformed into interaction tokens
• Foundation model pretraining and periodic retraining
• Frequent fine-tuning to incorporate newest behavior and newly launched titles
• Batch inference to publish profile and item embeddings to an embedding store, with model and version metadata

Online / serving plane (milliseconds → seconds)

• Candidate generation, often via embedding-based retrieval
• Ranking or re-ranking, where downstream models consume foundation-model embeddings or the foundation-model decoder runs as a shared subgraph per request
• Policy layers enforcing diversity, novelty, business constraints, exploration, and long-term reward proxies

This separation follows directly from Netflix’s explicit discussion of interaction tokenization, periodic retraining and daily fine-tuning, batch embedding refresh and publishing, and caching and latency constraints in online use.

What Model Did Netflix Actually Use?

Netflix’s public material describes a transformer-based generative recommender trained on tokenized interaction sequences with a next-event objective. This mirrors LLM training mechanics, but tokens represent user interactions and entities rather than natural-language subwords.

Netflix distinguishes several internally described model patterns:

Recommendation Foundation Model (FM) A large, centralized model trained on comprehensive histories, designed to produce reusable user and item representations and support downstream fine-tuning.

FM-Intent A hierarchical multi-task extension that explicitly predicts session intent and uses it to improve next-item prediction. Netflix reports a 7.4% offline improvement in next-item accuracy versus a strong baseline in experiments.

Large-scale generative recommender models (50M to 1B parameters) Scaling experiments that treat recommendations as generative modeling and analyze compute efficiency, training stability, and scaling laws.

Post-trained generative recommenders (A-SFT) A post-training algorithm that aligns a generative recommender with preference and value signals under real-world constraints.

Netflix also references experiments using the open-sourced HSTU architecture from Meta research in work related to reward modeling and post-training evaluation, illustrating cross-pollination between generative recommender research and Netflix’s production approach.

Data and Feature Pipelines

Interaction data to tokens for generative modeling

Netflix frames its data scale in LLM terms. With over 300 million users by the end of 2024, the company describes hundreds of billions of interactions and builds sequences by tokenizing user interactions, explicitly comparing the problem to tokenization trade-offs in language models.

A key disclosed tactic is merging adjacent actions on the same title to reduce redundancy while preserving meaningful aggregates such as total watch duration. Netflix notes that overly lossy tokenization removes signal, while excessively granular sequences exceed practical compute and latency limits.

Rich per-token context beyond item IDs

Netflix emphasizes that recommendation tokens are heterogeneous. They include action attributes such as locale, time, duration, and device, along with content information such as item IDs and metadata including genre and release country. Most such features, especially categorical ones, are embedded directly in the model, consistent with end-to-end representation learning.

Handling long histories under tight serving SLAs

Active users can accumulate thousands of interaction events, exceeding standard transformer context windows, while online recommendation services often operate under millisecond-level latency constraints.

Netflix’s mitigations span training and inference and include:

• Sparse attention mechanisms to extend usable context while controlling compute
• Sliding-window sampling during training so different segments of long histories are seen across epochs
• Key–value caching during inference to reuse past computations and reduce latency

Cold start for new titles and unseen entities

Cold start is unavoidable because new titles enter the catalog frequently. Netflix describes two complementary approaches:

• Incremental training and warm-starting, where previous parameters are reused, embedding layers are expanded, and new items are initialized using metadata-based heuristics
• Inference with unseen entities, blending ID-based embeddings with metadata-derived embeddings through age-aware mixing so new titles rely more on metadata while mature titles rely more on interaction history

In newer work on large-scale generative recommenders, Netflix extends this approach using multimodal semantic towers for vision, language, and knowledge-graph features, with masking strategies aligned to expected cold-start prevalence.

Scaling and Latency Optimization Strategies

Compute scaling and retraining cadence

Netflix contrasts recommendations with train-once LLM workflows. Recommendation models must be retrained frequently to reflect shifting tastes, seasonality, and catalog changes, making efficiency a first-order constraint.

Netflix reports training generative recommenders on trillions of tokens, periodically processing roughly two trillion tokens, and contextualizes this against the training scales of large language models. One disclosed training footprint involves 80 A100 GPUs for roughly 240 hours per cycle, supplemented by frequent fine-tuning.

Vocabulary size and decoding cost

Recommendation vocabulary can reach millions of items, far exceeding typical LLM vocabularies. Netflix highlights this as a core scaling challenge and describes two key mitigations:

• Sampled softmax, computing logits over sampled negatives instead of the full catalog
• Projected heads, reducing the dimensionality used in final logit computation

Netflix reports that this combination can reduce training costs by one to two orders of magnitude.

Serving latency and distribution shift

Caching and batch refresh introduce latency that can misalign training objectives with what users actually experience. Netflix evaluates performance under both sub-second online serving and long-latency serving with delays up to 48 hours.

This leads to an important architectural insight: embedding-based and cached serving are not just optimizations , they change the effective learning target.

Production integration patterns

Netflix documents three integration approaches for deploying foundation models:

• Embeddings, offering low adoption cost but risking staleness
• Subgraph integration, reducing staleness but increasing inference complexity
• Fine-tune-and-serve, offering strong task adaptation at the cost of managing many models and meeting strict SLAs

Each represents a different point in the latency, cost, and freshness trade-off space.

Alignment, Personalization Quality, and Diversity Controls

From next-token to multi-token prediction

Netflix identifies two mismatches between language-style next-token objectives and recommendation:

• Cached outputs may be served after the "next" item has already been consumed
• Recommendation targets are often order-insensitive, with multiple acceptable next items

Netflix addresses this by using multi-token prediction, supervising a set of future items over a time window aligned with serving latency, and weighting labels by utility signals such as watch time, novelty, or diversity.

Post-training alignment under real-world constraints

Netflix argues that classical RLHF-style post-training faces obstacles in recommendation: lack of counterfactuals, noisy rewards, and unknown logging policies. A-SFT is proposed as a middle ground between behavior cloning and offline reinforcement learning, weighting supervised updates by an advantage-like signal without overfitting to noisy rewards.

Long-term satisfaction as a system-level objective

Netflix consistently emphasizes that optimizing short-term engagement alone can misalign with long-term satisfaction and retention. Recommendation systems are therefore wrapped in a broader decision layer involving proxy rewards, delayed feedback modeling, and disciplined evaluation.

Measured Impact and Evaluation Evidence

Business value of personalization at Netflix scale

A 2025 working paper co-authored by Netflix researchers estimates that replacing Netflix’s recommender with a matrix-factorization system would reduce engagement by roughly 4%, while a popularity-based system would reduce engagement by roughly 12%, also reducing consumption diversity. While this does not isolate foundation models specifically, it anchors the economic importance of recommendation quality.

Reported offline gains

Netflix reports several offline evaluation results:

• FM-Intent achieves a 7.4% improvement in next-item prediction accuracy
• Scaling generative recommenders from 50M to 1B parameters yields substantial improvements, with careful attention to task-specific ceilings
• Multi-token prediction improves performance under high-latency serving, with minor trade-offs in short-term dependency tasks
• A-SFT improves alignment without overfitting to noisy reward models

Disclosed system constraints

Netflix highlights concrete constraints shaping engineering decisions:

• Certain online serving paths target p95 latency below one second
• Embedding-based approaches trade freshness for efficiency, while subgraph approaches increase inference cost
• Embeddings are refreshed through monthly pretraining, daily fine-tuning, and batch inference
• With over 300M paid memberships and multiple viewers per household, Netflix estimates a global audience exceeding 700M, making freshness and latency critical operational concerns

What remains undisclosed

Netflix has not publicly released live A/B lift numbers for foundation-model deployments, precise production parameter counts, or detailed online diversity trade-off curves. Instead, public materials focus on system design patterns, objective alignment, and training and serving efficiency,  suggesting these are the lessons Netflix considers transferable without exposing sensitive product details.

Implications for Enterprise AI System Design

The real value of Netflix’s disclosures lies in the repeatable pattern they illustrate across systems and models. Foundation models are becoming infrastructure rather than endpoints. For enterprise AI leaders, the key lesson is that value emerges from how large models are trained, refreshed, integrated, and constrained within real systems, balancing latency, cost, alignment, and long-term outcomes. As more organizations move beyond narrow task models, Netflix’s approach highlights where generative AI delivers real impact: scalable system design, disciplined objectives, and continuous adaptation under real-world constraints.