xai-org/x-algorithm — X's For You Feed Algorithm
Transcript
QuickFacts
Welcome! Today we're exploring something really exciting — the x-algorithm repository from X AI. This is the actual code that powers the For You feed on X, and they've open sourced it for everyone to learn from. Let's dive in and see how it all works.
PlainEnglish
At its core, this system does one big job — it builds your personalized For You feed. It takes posts from people you follow, mixes in interesting content from the wider network, and uses machine learning to figure out what you'll probably want to see. It filters out spam and blocked accounts, then carefully weaves ads into the timeline. All of this happens in real-time for every request.
Architecture
Let's look at what this system is built on. The orchestration layer is written in Rust for speed and safety, while the machine learning models run in Python with JAX. Services talk to each other over gRPC. For data, there's an in-memory post store using DashMap, Redis for caching, and Strato as the main database. Everything is wired together with Kafka for streaming events, and specialized services like Phoenix for ML inference, Gizmoduck for user profiles, and SocialGraph for your following relationships.
Architecture
Here's the high-level architecture. When your client requests the For You feed, it hits Home Mixer — the orchestration layer. Home Mixer talks to Thunder to grab posts from accounts you follow. It calls Phoenix to retrieve and rank out-of-network content using machine learning. Grox provides safety checks and content understanding. Finally, Home Mixer assembles everything into a ranked feed and sends it back to you.
Architecture
Let's walk through how a single request flows through the system. First, we hydrate the query — loading your following list, engagement history, and bloom filters. Next, we source candidates from Thunder, Phoenix, and other sources in parallel. Then we hydrate each candidate with post data and author info, and filter out blocked accounts, spam, and duplicates. After that, we score everything using the Phoenix ML model. Finally, we select the top posts and blend in ads at safe positions.
Architecture
Phoenix is the brains of the operation, and it works in two stages. First, retrieval — your recent engagement history gets encoded into a user embedding, then a two-tower model does similarity search across millions of posts to find the top two hundred candidates. Second, ranking — a Grok-based transformer looks at each candidate in the context of your history, using a clever attention mask so candidates can't see each other. It predicts probabilities for fifteen different engagement types — likes, replies, shares, and so on. Finally, those predictions get combined into a single weighted score.
Architecture
Thunder is the in-memory post store that makes everything fast. It listens to Kafka for tweet create events. When a post arrives, Thunder stores the full post data and creates a tiny reference indexed by the author. Later, when Home Mixer needs in-network posts, it sends Thunder a list of following IDs. Thunder looks up each author's timeline, grabs their recent posts, and sends them back in milliseconds. No database queries, just pure in-memory lookups.
Architecture
Meanwhile, Grox handles content understanding. It's a dispatcher-engine architecture that processes posts through multiple classifiers. The spam detector uses a vision-language model to analyze posts with media. Safety classifiers look for policy violations. Embedders generate multimodal representations combining text, images, and video. Reply rankers score conversation quality. All of this runs asynchronously, feeding safety labels and embeddings back into the main pipeline.
Architecture
Let's zoom out and see where the code lives. Home-mixer is the orchestration layer — around eleven thousand lines of Rust with candidate pipelines, sources, and scorers. Phoenix holds the machine learning models — retrieval, ranking, and the Grok transformer. Thunder is the in-memory post store built on DashMap. Grox contains the content classifiers and embedders. And candidate-pipeline is a reusable framework that defines the trait-based pipeline stages everything else builds on.
Community
So who's behind this? The repo is actively maintained — the last commit was just five days ago, and there have been a dozen commits in the past month. Eight contributors have been active recently, out of twenty-three total. The core engineering team and ML team are the most active. It's a healthy, living project with steady updates and a focused group of maintainers keeping everything running smoothly.
PlainEnglish
And that's the X algorithm! What makes this special is that it's real production code — the same ML models, ranking logic, and pipeline architecture that serve millions of users every day. X AI has opened it up so engineers, researchers, and anyone curious can dig in and learn how modern feed ranking actually works. It's a remarkable resource, and we hope this walkthrough helps you explore it further. Happy coding!
How this was made
Lenzon read xai-org/x-algorithm and generated this walkthrough automatically. The narration above is the transcript of what it says.
Explain a pull request from your own repo
Point Lenzon at a repo or a pull request and get a narrated walkthrough like this one.
Try it