WRITING
Can you pick the right architecture before training it?
Applying Amid et al.'s Neural Network Prior Kernel as an architecture screen. It does not rank networks by trainability — but it detects whether a design can express the task's rule, and re-ranks correctly when the rule changes. Seven experiments, twenty-four limitations, three of which corrected me.
One Layer Deeper: learning a hidden recurrence, exactly
A competition about depth. A tiny learned gate-population recurrence swept Easy and all ten Medium datasets (100%, certified to T=64 on seen and unseen moduli). Hard didn't fall — and the diagnosis, a float64 precision cliff then a hypothesis-class mismatch, was the real prize.
Making ctf-lite a real training environment
Turning a capture-the-flag eval into an RL training environment — the null result I kept, the diagnosis, and the difficulty-generator fix I built and measured.
Teaching a transformer to write backwards
A decoder-only model that answers forward-English questions in character-reversed English, generated tail-first — why the task is interesting, the design decisions, and the data-pipeline bugs that actually bit.
READING
Play Long-term Games With Long-term People
Naval Ravikant on choosing enduring industries and people you can build with repeatedly, where trust and compound interest have time to do their work.
Fault-tolerant quantum computing is within reach
Oratomic's case for utility-scale neutral atom quantum computers, high-rate error correction, and the cryptographic implications of practical fault tolerance.
The Precipice
Toby Ord's comprehensive analysis of existential risks facing humanity and how we can navigate the challenges of the 21st century. A philosophical and practical guide to safeguarding humanity's future.
Superintelligence
Nick Bostrom's seminal work on the paths, dangers, and strategies for artificial general intelligence. Explores the potential for AI to surpass human intelligence and the existential risks involved.
The Scaling Era
Dwarkesh Patel's in-depth interviews with leading AI researchers including Anthropic CEO Dario Amodei, DeepMind cofounder Demis Hassabis, and OpenAI cofounder Ilya Sutskever. A comprehensive portrait of large language models and a vision of what comes next.
Situational Awareness: The Decade Ahead
Leopold Aschenbrenner's comprehensive analysis of the path from GPT-4 to AGI to superintelligence. A deep dive into compute scaling, intelligence explosions, and the geopolitical implications of the AI race.
The upcoming GPT-3 moment for RL
RL will soon have its own GPT-3 moment. Rather than fine-tuning models on a small number of environments, we expect the field will shift toward massive-scale training across thousands of diverse environments.
The future of AI is already written
To a first approximation, the future course of civilization has already been fixed, predetermined by hard physical constraints combined with unavoidable economic incentives. Full automation is inevitable.
Life after work
Exploring what comes after full automation of human labor.
2 OLMo 2 Furious
AllenAI's fully-open OLMo 2 language models: architecture and training-stability fixes, a late-pretraining data curriculum, and Tülu-3-style post-training — competitive with open-weight peers at similar compute.
Speculative Decoding
A clever technique to speed up LLM inference by producing multiple tokens per forward pass. Explains the algorithm, its derivation, and newer variants.
The Mythical Man-Month
Fred Brooks' classic essay on software engineering management. Why adding manpower to a late software project makes it later.
How Complex Systems Fail
Richard Cook's insightful paper on the nature of complex system failures and why they're inevitable.
You Are Not Google
A reminder that most engineering teams don't need Google-scale solutions. Build what you actually need.
The Architecture of Open Source Applications
Deep dives into the architecture of successful open source projects. Learn from the best.