Researcher Skill Stack for Frontier Labs

Frontier labs hire for more than "knows transformers." They want people who can turn uncertainty into experiments, use compute wisely, and communicate results under ambiguity.

The Four Competence Layers

1. Mechanistic Understanding

You need to understand the actual computation:

• Embeddings become residual-stream vectors.
• Attention moves information between token positions.
• MLPs transform features position-wise.
• Normalization stabilizes optimization.
• Positional schemes give the model sequence geometry.
• Loss is next-token cross-entropy unless post-training changes the behavior target.

Core notes:

• Transformer Block Anatomy
• Attention Mechanics and KV Cache
• Data Tokenization and Pretraining Objective

2. Empirical Scaling Taste

You need intuitions about what changes with scale:

• Some methods look good only at small scale.
• Some methods are worse at small scale but win after enough compute.
• Data quality can dominate parameter count.
• Inference cost can decide whether an architecture matters.
• Communication can erase theoretical compute savings.

Core notes:

• Scaling Laws and Compute Optimal Training
• Mixture of Experts Architectures
• Frontier Model Systems and Inference

3. Experimental Discipline

Good researchers ask:

• What claim am I testing?
• What is the smallest experiment that could be informative?
• What baseline would make this claim look ordinary?
• Which metric is a proxy, and what does it fail to capture?
• Is the improvement from architecture, data, optimizer, training length, or evaluation leakage?

For every paper, write:

• Mechanism.
• Claimed bottleneck.
• Evidence.
• Missing baselines.
• Scaling guess.
• Failure modes.
• One reproduction plan.
• One falsification experiment.

4. Systems Awareness

Architecture is inseparable from systems:

• Attention FLOPs are not the only bottleneck.
• KV cache memory can dominate serving.
• MoE routing creates communication and load-balancing problems.
• Long context increases prefill cost and memory pressure.
• Quantization changes quality, latency, and memory.
• Batch size, parallelism, and hardware topology shape feasible model design.

Core notes:

• Attention Mechanics and KV Cache
• Frontier Model Systems and Inference
• Long Context and Efficient Sequence Models

Skills To Build Deliberately

Math

• Derive attention from matrix operations.
• Understand softmax saturation and temperature.
• Understand cross-entropy and KL divergence.
• Understand low-rank factorization and why LoRA can work.
• Read scaling-law equations without fear.

Implementation

• Write a decoder-only transformer from scratch.
• Add RoPE, RMSNorm, SwiGLU, GQA/MQA, KV cache.
• Profile training and inference.
• Implement sampling methods.
• Implement toy SFT and DPO.

Paper Reading

Use Paper Reading Ladder Frontier LLMs.

Do not only read abstracts. Reconstruct the experiment table. Ask why each baseline was chosen.

Writing

Write short research memos:

• "What problem is this paper actually solving?"
• "What would make this result fail?"
• "What experiment would I run with 1 GPU, 8 GPUs, and frontier-lab compute?"

Portfolio Ideas

• Tiny transformer implementation with clean notes.
• Small scaling-law reproduction.
• MoE toy model with routing visualization.
• Long-context attention benchmark.
• KV-cache memory estimator.
• DPO toy reproduction.
• Paper summaries with critical commentary.

Hiring Signal

The strongest signal is not claiming to know every new paper. It is showing that you can learn fast, reason from first principles, notice confounders, implement clean experiments, and explain tradeoffs.