How Meta Gives Ads a Stable Language

Meta has built GenRecs, a new system for ad recommendation. Older systems often gave every ad or idea its own ID. This works for popular ads with much history. But it is weaker for a new ad. A new trail shoe has little user data, even when it is similar to known sports products. GenRecs instead gives ads a small set of semantic tokens. These tokens represent meaning, so they can be shared.

The system first studies several kinds of information. Multimodal models read ad text and images. They create an embedding, which is a list of numbers that represents content. The tokenizer also uses engagement signals, such as how people respond. It then changes the embedding into a short line of discrete tokens. Each token comes from a stable vocabulary.

This common vocabulary can describe ads, products, pages, and pixels. It supports both retrieval and ranking. Retrieval finds useful candidates from a large group. Ranking puts the best candidates in order. Because related items share tokens, a model can connect a new product with known interests. This improves cold start, the difficult period before an item has enough history.

Meta tested eight versions of its tokenizer. It used intrinsic metrics to check token quality before full recommendation tests. These measures were closely linked to accuracy in predicting clicks. Stronger tokens improved results for tail ads, which receive less activity. They also create a useful cycle. Better recommendations bring more engagement data, and that data can improve later models.

The team also improved its codebooks, which store the available tokens. It used diversity loss, dead code resets, and contrastive objectives. A two-tower system with a shared LLM encoder reached strong retrieval results with much less training data. Meta now wants better cold-start performance and efficient LLM-based serving. It also plans to join text, image, audio, and video in one shared representation.