Last week I posted a topographic map of Qwen 2.5-7B's vocabulary, 10,000+ English words projected from 3,584-dimensional space into a terrain where similar words form mountains. That map was static, a snapshot of the model before it thinks.
This time I made it think.
I fed the model a question: "What material would be hardest for a craftsman to combine with gold using only fire: quartz, silver, or copper? Answer in one word." Then captured its internal state at every one of its 28 transformer layers. Then I projected each layer's hidden state back onto the vocabulary terrain and visualized which regions light up.
Models think across layers, which means that each layer transforms the model's internal representation a little further. Think of it like a 28-step thought process. The input enters as raw words, no understanding. Each layer adds context, combines meanings, and resolves ambiguities. By the final layer, the representation has been refined into a specific answer.
The challenge is that we can't normally see what's happening in the middle steps. The model's thinking at layer 14 exists as a 3,584-dimensional vector that isn't directly readable as words. It's like watching someone solve a math problem but only being able to see the final answer, the scratch work is in a language you can't read. The J-lens translates that scratch work back into vocabulary at every step.
What you're seeing in the GIF:
The grey mountains are the model's vocabulary landscape, fixed and unchanging. The amber glow shows which words the model's thinking is connected to at each layer depth. Bright means "this region is active." Dark means "the model isn't thinking about this right now."
White labels mark the strongest activations. Amber labels (⬡) mark words that are uniquely active at that specific layer compared to all others.
The story across layers:
Layers 0–5: The model is parsing basic language structure. Top words are "one", "how", "were", they are syntax, not content.
Layers 6–12: Task recognition. "Strategist", "answer", "puzzle" appear. The model has figured out it's being asked to solve something, but hasn't started thinking about materials yet.
Layers 13–22: "Answer" dominates for ten straight layers. This is the model's workspace, it's holding "I need to produce an answer" while working through the decision. The distinctive words evolve alongside it: "correct" and "decision" appear at layer 19, "single" at layer 20. It knows it needs one correct answer.
Layer 23: The pivot. "Copper" appears for the first time. The model transitions from processing the task format to processing the content.
Layer 28: Crystallization. The glow concentrates on a single cluster: copper, silver, quartz, silica, gold. The distinctive words at this layer are "hard", "solid", "specific", the actual reasoning about material properties. The model answers: copper (55% confidence), silver (14%), quartz (7%).
The model's final reply:
Copper. Copper would be the hardest material for a craftsman to combine with gold using only fire due to its lower melting point and tendency to form an oxide layer when heated, which can make it difficult to fuse with gold. In contrast, silver and quartz have higher melting points and are more compatible with gold when melted together. However, among the given options, copper is the most challenging to work with in this context.
Qwen got it wrong:
The model's inner deliberation showed "answer, correct, decision" in the mid-layers, it weighed all three options (copper 55%, silver 14%, quartz 7%), committed to copper with confidence, and got it wrong. The visualization of thinking doesn't guarantee the thinking is correct. The map shows you how the model reaches its answer, not whether the answer is right. I asked for one word, and Qwen couldn't help itself and answered a whole paragraph.
What we're still missing:
Most of the visible action happens in the last 4 layers, where the terrain suddenly erupts with activation before collapsing onto the final answer. There's clearly a lot happening in that transition. The model goes from "I know what kind of question this is" to "the answer is copper" in just a few layers, and right now we're seeing the explosion but not the detail of how it resolves. Zooming into that window, maybe tracking individual word trajectories frame by frame, is the next step.
Also worth noting: the J-lens covers layers 0–26, and the final layers fall back to a cruder method. So the most dramatic moment in the model's thinking is the one we're least equipped to read. Working on it.
Next up: the hunt for the em-dash.
Certain models have strong stylistic preferences — they love em-dashes. Somewhere in those 3,584 dimensions there's a direction that means "dramatic pause energy." I want to find it.
The method:
This uses the J-lens (from "Verbalizable Representations Form a Global Workspace in Language Models," 2026). At each transformer layer, the model's hidden state exists in a coordinate system that's been rotated by all previous layers. A naive projection back to vocabulary (called the logit lens) produces noise in the middle layers because it's reading a rotated map. The J-lens corrects for that rotation using a learned per-layer transformation (the averaged Jacobian of the network from that layer to the output), making the mid-layer thinking readable.
I ran both lenses on the same prompt. I don't show the logit lens here, but I have a similar visual for it. The logit lens showed the word "libertine" as the top activation for 22 out of 29 layers, pure geometric noise. The J-lens showed "answer", "puzzle", "hint", "true" , the actual deliberation.
Tools: Qwen2.5-7B-Instruct, PyTorch (hidden state capture), pre-fitted J-lens from HuggingFace, UMAP + KDE for the terrain, Plotly for the 3D visualization.
Visualization and research: me. Write-up polished with Claude's help.
Previous post: https://www.reddit.com/r/dataisbeautiful/comments/1vyq0ug/oc_i_turned_qwen_257bs_embedding_space_into_a/
