Attention Is All You Need
Introduced the Transformer — an encoder/decoder built entirely on self-attention, no recurrence — that underlies virtually every modern LLM.
Read the paper →Frontier
The Frontier
The papers that built this field — hand-picked, with one honest line on why each one matters.
Language Models are Few-Shot Learners
Showed that scaling a decoder-only transformer to 175B parameters unlocks strong few-shot, in-context learning without task-specific fine-tuning.
Read the paper →Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Paired a pretrained seq2seq model with a learned dense retriever over a vector index — the blueprint modern RAG pipelines still follow.
Read the paper →Training Language Models to Follow Instructions with Human Feedback
Introduced the RLHF fine-tuning recipe used to make base models follow instructions and prefer helpful, honest responses — the technique behind ChatGPT.
Read the paper →Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Showed that prompting a model to produce intermediate reasoning steps, rather than jumping to an answer, sharply improves multi-step problem solving.
Read the paper →ReAct: Synergizing Reasoning and Acting in Language Models
Interleaves reasoning traces with tool-calling actions and observations — the loop most modern agent frameworks are built on.
Read the paper →Constitutional AI: Harmlessness from AI Feedback
Replaces much of human feedback with AI-generated critiques judged against a written constitution, reducing reliance on human labelers for safety training.
Read the paper →Self-Consistency Improves Chain of Thought Reasoning in Language Models
Samples several diverse chain-of-thought paths for the same question and takes the majority final answer, trading extra compute for higher accuracy.
Read the paper →