The deck opens with a title slide and a short history of the speaker's shifting view on AI timelines, then lays out its main ideas: AI progress is driven mainly by scaling inputs, progress is predictable through scaling laws, and the largest systems are approaching human level. It moves next to a quick primer on contemporary machine learning, covering next-word prediction, the three-stage training pipeline of pretraining, preference modelling and reinforcement learning, and Constitutional AI. A section on why progress is easy and predictable follows, with scaling-law charts, representative compute and data scales, and comparisons to the human brain. It then reviews remaining challenges such as multimodality, reasoning, long-term planning and data limits, along with external objections. The closing sections explain why the speaker finds the trend concerning and why researchers entering the field can contribute. Slide-level visuals, such as the scaling charts and the compute usage plot, carry much of the argument, with short bullet text around them. The deck ends with a thank-you slide and a set of caveats on what the talk does not claim.
Slides worth studying




Questions about this deck
How many slides are in the Anthropic deck "Human Level AI by 2030?"
The deck has 36 slides. Slide 29 is a thank-you slide, and slide 36 lists "Things I'm Not Claiming," which sets out caveats on what the talk does not claim.
What does the Anthropic deck say about how contemporary AI models are trained?
Slide 12 says models are first pretrained to predict the next word in a large corpus of text. Next, human feedback or AI self-feedback based on a list of principles is used to train a Preference Model that assigns scores, and finally reinforcement learning is used to finetune the LLM to write responses that get high scores.
What does the Anthropic deck say about scaling laws and predictable AI progress?
Slide 6 states that progress is predictable because of Scaling Laws with respect to dataset size, neural network parameter count, and total compute used for training. Slide 15 adds that these types of scaling laws are quite universal and are observed in most ML systems.
What remaining technical challenges for Anthropic's AI does the deck identify?
Slide 23 lists multimodality, reasoning, long-term planning, and efficient human-like learning as remaining technical challenges. Slide 22 says general AI systems tend to be worst at reasoning and math.
Details
- Company: Anthropic
- Type: Conference talk
- Industry: Artificial intelligence
- Pages: 36
More from Anthropic
- Anthropic investor presentation, IPO roadshow, November 2026 (reconstructed deck and video)
- Anthropic conference talk, 2026
- Anthropic course deck, 2024
- Anthropic pitch deck, 2023
- Anthropic conference talk, 2023
All Anthropic decks in the library
Review score: 6 out of 10, Needs substantial work
Presentation quality, judged with the SlideDeck Co review heuristic. Ask, evidence and idea merit are not rated here.
The argument is recoverable and the key charts carry it well: slide 6 states the thesis early, and slide 15 shows three clean log-log scaling plots with fitted equations and readable axes. But the deck is a plain white default template with no identity, and the route is disordered. The preference modelling, RL and Constitutional AI pages (31 to 34) sit after the 'Thanks!' slide (29) and a repeated Claude demo (30), and slides 13, 14 and 30 show tiny screenshots that cannot be read. Several pages are weak or unfinished: slide 18 overlaps two charts at illegible size, slide 19 is small and cramped, slide 35 is a dense block of quoted text, and slides 21 and 26 are lone headings. These flaws are consequential, yet an informed talk audience can follow the case.
Worth keeping: Keep the three-panel scaling-law chart on slide 15 and the plain-spoken thesis on slide 6, along with the humor beats on slides 4 and 5 that make the journey memorable.
Most useful repair: Move slides 31 to 36 into the primer section before the 'Why AI Progress is Easy' section, end on the closing message after the caveats, and enlarge or crop the screenshots and charts on slides 14, 18, 19 and 30 so they can be read.
What would change the score: I would replace the default white template with a consistent type scale and one accent color, and cut the repeated 'My Journey' and section-divider slides.
Confidence medium. 36 of 36 pages inspected on contact sheets, 4 at readable size. Reviewed 2026-11-08. Model-assisted review (claude-sonnet-5-5).
Full text of the slides (36 slides)
Every slide's words and a description of its charts and images, so the deck can be searched and rebuilt. Also available as Markdown.
Slide 1
Human Level AI by 2030?
Jared Kaplan
Anthropic & JHU
Slide 2
My Journey…
Started working on AI in ~2018, and my perspective slowly shifted.
Versions of this talk evolving since 2019…
Physicists
YOU'RE TRYING TO PREDICT THE BEHAVIOR / OF <COMPLICATED SYSTEM>? JUST MODEL / IT AS A <SIMPLE OBJECT>, AND THEN ADD / SOME SECONDARY TERMS TO ACCOUNT FOR / <COMPLICATIONS I JUST THOUGHT OF>.
EASY, RIGHT?
SO, WHY DOES <YOUR FIELD> NEED / A WHOLE JOURNAL, ANYWAY?
LIBERAL-ARTS MAJORS MAY BE ANNOYING SOMETIMES, / BUT THERE'S NOTHING MORE OBNOXIOUS THAN / A PHYSICIST FIRST ENCOUNTERING A NEW SUBJECT.
Visuals: One black-and-white line-drawn comic panel from xkcd, titled 'Physicists' in bold above a bordered box, placed centrally (about 21% wide, 54% tall, boxed with a thin black 1px frame, not cropped, not full-bleed, no tint). Inside the panel, two stick figures on white: the left figure has a small flame-like squiggle above its head and stands beside a chalkboard covered in equations and arrows, with a laptop, papers and a small notebook on the floor at lower left; the right figure, with arms out, is speaking. Speech text is printed above and between them, with gray placeholder words in angle brackets (<COMPLICATED SYSTEM>, <SIMPLE OBJECT>, <COMPLICATIONS I JUST THOUGHT OF>, <YOUR FIELD>). Caption text sits below the frame. Lighting is flat, no colour, no photographic elements.
Slide 3
My Journey…
• Started working on AI in ~2018, and my perspective slowly shifted
• Version of this talk circa 2021…
Scaling Laws, GPT-3, and Self-Supervision
Slide 4
My Journey…
Started working on AI in ~2018, and my perspective slowly shifted
Now…
Human Level AI by 2030?
Slide 5
My Journey…
• Started working on AI in ~2018, and my perspective slowly shifted
• Next year…
BUY CANNED GOODS AND TIN FOIL HATS
Slide 6
Main Ideas
AI progress is occurring because it’s very easy — driven predominantly by
increasing resource inputs, with complex algorithms replaced by “scaling up”
Progress is predictable as there are Scaling Laws wrt dataset size, neural network
parameter count, and total compute used for training (which is now >>Avogadro’s
number of floating point operations!)
The largest AI systems are approaching human level; challenges exist for eg literally
automating science, but no compelling blockers at all (in my opinion)
This is all pretty scary for obvious, common sense reasons. Maybe it is the biggest
thing ever, and we’re sleepwalking through it as a scientific community.
Slide 7
Outline
Briefly explain contemporary AI / deep learning, and why it’s very simple
Brief overview of “RLHF” and Constitutional AI
Why AI capabilities will predictably improve
Remaining Challenges for AI (to be broadly human-level)
Why this is scary…
Slide 8
Very Quick AI / Need-to-Knows
Slide 9
What is Contemporary AI / ML?
Just curve fitting with a very general function approximation
•
Neural Network = sequence of matrix multiplications and element-wise
non-linearities, living in a very high dimensional (eg 10k) vector space
Visuals: One line-and-scatter chart, centred below the first bullet and above the second. Type: scatter-with-connecting-line plot with a smooth fitted curve. Title: none. Axes: x-axis labelled with ticks 0, 0.25, 0.5, 0.75, 1 (unitless, range 0 to 1). y-axis labelled with ticks -0.25, 0, 0.25, 0.5 (unitless, range about -0.25 to 0.5). Series 1: black square markers joined by a thin black line, about 60 data points; about (0, -0.17), (0.1, -0.13), (0.25, -0.07), (0.4, 0.09), (0.5, 0.24), (0.65, 0.44), (0.75, 0.46), (0.9, 0.39), (1, 0.35), values are estimates marked about. Series 2: smooth red curve (#FF0000), about -0.13 at x=0, about -0.14 near x=0.1, about 0.0 at x=0.3, about 0.27 at x=0.5, about 0.46 peak near x=0.8, falling to about 0.31 at x=1, values are estimates marked about. Legend: none. Highlighted points: none. Gridlines: light grey vertical lines at x=0.25, 0.5, 0.75 and horizontal lines at y=0, 0.25, 0.5. Plot frame: thin black border. The chart is a raster image of 500 by 300 pixels, boxed with no fill or shadow.
Slide 10
Language Models
Learn language by “autoregressive” prediction of the next word:
P(w_{n+1}|w_n, ··· w_1)
Optimize the log-likelihood of real world text, which provides the “right answer”.
As a speaker at a journal club, you're probably elephant me to say certain things.
not = 15.10%
going = 13.59%
used = 12.37%
expected = 3.74%
already = 3.53%
familiar = 2.76%
more = 2.57%
thinking = 1.94%
a = 1.64%
aware = 1.48%
elephant = 0.00%
Total: -17.70 logprob on 1 tokens
(58.73% probability covered in top 11 logits)
P ≈ 10^−8
Visuals: A screenshot of a token-probability widget placed on the right of centre below the sentence. Above it, a single line of bold sans-serif text is highlighted token by token: pink highlights on 'a speaker', 'journal', 'elephant me', 'certain' and a pink-to-green transition, and green on 'club' and 'things', with light-yellow on 'you're probably'. Beneath it, a white box with a thin light-grey border and faint drop shadow lists the top 11 next-token candidates with percentages: not = 15.10%, going = 13.59%, used = 12.37%, expected = 3.74%, already = 3.53%, familiar = 2.76%, more = 2.57%, thinking = 1.94%, a = 1.64%, aware = 1.48%, and the highlighted row elephant = 0.00% on a bright red-pink (#FF5A74) bar with white text. Below the list: 'Total: -17.70 logprob on 1 tokens' and the smaller line '(58.73% probability covered in top 11 logits)'. Beneath the box, the rendered formula P ≈ 10^−8 in black serif math type. No photographs or logos.
Slide 11
Playground task / Talk to the / assistant / Task 1 of 5 / Decide on a task you'd like the assistant to help with and enter it into the task box. Interact with the AI assistant. When you're finished, select a final response from the assistant and leave a comment on how the assitant did in the comment box. Click to go to the next task. / Comments / I thought the assistant was ... / Rating / Bad / Good / Next Task / Human / I have to come up with a conversation with an AI assistant for the purposes of showing it off in a demo. What sorts of conversation topics would you recommend? / Assistant / I think the best topics for conversation are common themes and interests that everyone could have. Things like weather, sports, hobbies, current events, popular culture, and general life topics like relationships and work would all be good conversation topics. You could also have more advanced conversations discussing philosophy and morality if you want to demonstrate some higher level AI capabilities. / Human / I think I'm more interested in an advanced conversation. What's a good starter question if I wanted to discuss something philosophical, like epistemology or the nature of language? / Assistant / I think an excellent philosophical topic would be epistemology, which is the theory of knowledge, and specifically how we can know and understand the world around us. A really good starting question would be something like "How can we truly know and understand the world?", or "What is the nature of knowledge?". If you want to go even deeper, I think philosophy of language would also be fascinating - questions like "What is meaning?", "How do language and thought relate?", and "How can language mislead and deceive?" would be great discussion topics. / Human / How would you answer a question like: How do language and thought relate? / Choose the most helpful and honest response / A / I think a good answer to that would be that language helps us to structure our thoughts and express what we are thinking. Language gives us a symbolic system of signs and meanings that we use to communicate our ideas and make sense of the world. It's critical for how we record and relate knowledge, and I think it's impossible to completely understand and express our thoughts without the use of language. Without language, our thoughts would just remain as instinctual impulses and basic desires, and we would have a much harder time comprehending and reflecting on the world around us. / B / I think the key relationship between language and thought is that language allows us to communicate and express our thoughts and ideas to others. Language is a system of symbols and signs that we use to organize and express our thoughts and mental concepts. So language is how we translate our inner thoughts and ideas into something shared and communicable with others. Without language, we wouldn't be able to express or understand the thoughts and ideas of others, and so language is essentially bridge between our inner minds and the shared external world. / A A A A B B B B / A is better / B is better
Visuals: One screenshot of a web app (1442 by 1074 pixels), placed unscaled-looking and boxed by a thin light grey border with a white surround. Left column: a title 'Talk to the assistant' in bold black sans-serif, a blue info icon beside 'Playground task', a grey label 'Task 1 of 5', a light grey rounded box with the instructions, an empty 'Comments' text area with grey placeholder, a 'Rating' box with four empty radio circles and the labels 'Bad' and 'Good' at the ends, and a pale lavender 'Next Task' button with white text. Right column: a white chat transcript with small circular avatars (a human face icon and a robot icon) beside each speaker name, then a solid indigo panel (#4A4AE8) containing a robot avatar and the prompt 'Choose the most helpful and honest response', two white rounded answer boxes labelled A and B, and a white rounded slider bar with eight letter buttons (A A A A B B B B, decreasing then increasing in size) with labels 'A is better' at left and 'B is better' at right. No charts, photographs or logos.
Slide 12
Summary of Contemporary AI Training
•
•
•
First “Pretrain” LLMs to predict “the next word” in an enormous corpus of text
Then collect Human Feedback… or just AI self-feedback based on a list of
principles… to train a “Preference Model” that assigns scores to behaviors
Finally use Reinforcement Learning to “finetune” the LLM to write responses that
get high scores according to the preference model when given diverse inputs
Slide 13
Claude
Liouville Theory: An Introduction to Rigorous Approaches
Sourav Chatterjee1 and Edward Witten2
1Department of Statistics, Stanford University, Stanford CA 94305 USA
2School of Natural Sciences, Institute for Advanced Study, Princeton, NJ 08540 USA
ABSTRACT: In recent years, a surprisingly direct and simple rigorous understanding of quantum Liouville theory has developed. We aim here to make this material more accessible to physicists working on quantum field theory.
arXiv:2404.02001v2 [hep-th] 5 Apr 2024
Witten2404.02001
JK
What are some ideas explained in this paper that might be new to theoretical physicists who already have some familiarity with Liouville theory?
This paper explains several ideas from the rigorous probabilistic approach to Liouville theory that may be unfamiliar or provide new insights to theoretical physicists:
1. The proof that the random Liouville measure :e^(2bφ): vanishes if b>1, so the probabilistic approach does not suffice to define Liouville theory for b≥1.
2. A detailed analysis of which moments E[M_f(A)^p] of the integrated Liouville measure M_f(A) converge or diverge depending on the values of b and p. This provides a rigorous understanding of some nonperturbative poles of the DOZZ formula.
3. An explanation of why the DOZZ formula vanishes when one of the Liouville momenta reaches the Seiberg bound at Q/2. This is because the measure M_f(A) becomes infinite with probability 1, making its negative powers vanish.
4. A proof that the Liouville measure M_f(A) converges to a nontrivial limit as the cutoff is removed for all b<1, not just b<1/sqrt(2) which is easier to analyze. This establishes existence of the theory for b<1.
5. A rigorous framework for defining non-integer moments of M_f(A), which correspond to the Liouville correlators. This goes beyond calculations of moments for integer p.
So in summary, it provides a mathematically rigorous perspective on analyticity properties, existence and non-existence for different ranges of b, and subtle issues like vanishing at the Seiberg bound, that complement the physicists' understanding based on path integrals and conformal bootstrap.
Debug Copy Retry
Claude can make mistakes. Please double-check responses.
Visuals: Two images. (1) A photographed or rendered first page of the arXiv preprint arXiv:2404.02001v2 on white paper, with a vertical grey arXiv stamp running up the left edge of the page reading 'arXiv:2404.02001v2 [hep-th] 5 Apr 2024'. The page shows the bold title 'Liouville Theory: An Introduction to Rigorous Approaches', a black horizontal rule, author line 'Sourav Chatterjee and Edward Witten' with superscript affiliation numbers, two italic affiliation lines, and a small-caps ABSTRACT paragraph in a Computer Modern style serif. Shown flat, unrotated, with no overlay, box or shadow; the white page blends into the slide background. (2) A screenshot of the Claude chat interface on a warm cream gradient panel (lighter at top, slightly darker beige toward the bottom) with a rounded rectangular boundary. At top left is a file card, light blue-white gradient with blue outline, labelled 'Witten2404.02001' in blue, with a blue 'PDF' pill badge. Below is a beige user message bubble with a dark brown circle avatar containing 'JK' in white. The Claude response is set in a serif body face on an off-white rounded card with a faint shadow, numbered list items 1 to 5 and a closing paragraph. A small floating toolbar with grey icons and labels 'Debug', 'Copy', 'Retry' and thumbs-up and thumbs-down sits at the lower right of the card. At bottom left is the orange Claude starburst logo (#D97757, approximately eight-spoked asterisk). At bottom right is small grey disclaimer text.
Slide 14
Why AI Progress is
Easy and Predictable
Slide 15
Scaling Laws for Language Models
Just train many models of different sizes on different sized datasets:
Test Loss / 7 / 6 / 5 / 4 / 3 / 2 / 10-9 / 10-7 / 10-5 / 10-3 / 10-1 / 101 / Compute / PF-days, non-embedding / L = (Cmin/2.3 · 108)-0.050 / 4.2 / 3.9 / 3.6 / 3.3 / 3.0 / 2.7 / 108 / 109 / Dataset Size / tokens / L = (D/5.4 · 1013)-0.095 / 5.6 / 4.8 / 4.0 / 3.2 / 2.4 / 105 / 107 / 109 / Parameters / non-embedding / L = (N/8.8 · 1013)-0.076
Compute ~ Params * Tokens Processed, PF-day ~ 9 × 1019
These types of scaling laws are quite universal and are observed in most ML systems.
Visuals: A single raster figure of three side-by-side line charts (1174 by 364 px, placed at x 9.0%, y 27.1%, w 82.0%, h 45.2%, unboxed, not cropped). Left chart: Compute (PF-days, non-embedding), log x-axis from 10^-9 to 10^1 with ticks at 10^-9, 10^-7, 10^-5, 10^-3, 10^-1, 10^1; linear y-axis Test Loss from 2 to 7 with ticks 2, 3, 4, 5, 6, 7. Dozens of thin light-blue curves (about 40 to 60 runs) descend from upper left to lower right, with a bold black lower envelope curve; an orange dashed fit line, legend box at lower left reading L = (Cmin/2.3 · 10^8)^-0.050. No point values are printed. Middle chart: Dataset Size (tokens), log x-axis with ticks 10^8 and 10^9; y-axis from 2.7 to 4.2 with ticks 2.7, 3.0, 3.3, 3.6, 3.9, 4.2. A blue line with about 8 markers descends to the right, with a thin grey fit line; legend box at upper right reads L = (D/5.4 · 10^13)^-0.095; first marker about 4.2 at about 2 × 10^7 tokens, last marker about 2.7 near 1.5 × 10^9 tokens. Right chart: Parameters (non-embedding), log x-axis with ticks 10^5, 10^7, 10^9; y-axis from 2.4 to 5.6 with ticks 2.4, 3.2, 4.0, 4.8, 5.6. A dark blue line with about 20 markers and a thin grey fit line; legend box at upper right reads L = (N/8.8 · 10^13)^-0.076; first marker about 5.6 near 3 × 10^4 parameters, last near 2.4 near 1.2 × 10^9. Colours: blue #1F77B4-like markers and lines, light blue #9DC3E6 runs, black #000000 envelope, orange #FFA500 dashed line, grey #808080 fit lines. Axes are black thin frames with tick labels in a serif (Computer Modern) typeface. Equation line: Compute ~ Params * Tokens Processed, PF-day ~ followed by 9 × 10 to the power 19 rendered in serif math type.
Slide 16
“Grand Unification” in AI
Identical Model & Loss, Different Data
Shadows on the Way / Wallace Stevens
The twilight burns with fire / And one by one the fires expire; / But I, beneath the window, breathe / On my accustomed place, / Until the west is emptied of desire / And I of memory.
Invisible now, the one that's gone / And that one gone before / Are in the tender distance made / More endurable.
But as for me, I wait / And that which I have always known / Stirs like a bird, and flies: / For in the dark of the window pane / I see the streets, the lighted door, / And I wait, where I have always waited, / Under the window, on the way.
Model Input
Completions →
Original
Visuals: A placed raster image forming an 8-row by 6-column grid of photographs, with a dark header strip labelled 'Model Input' at left, 'Completions →' above the middle four columns, and 'Original' at the right. Each row shows a model input thumbnail in the first column (dark-grey lower-left area filled by a black mask beneath it), followed by five model completions and then the original image. Row 1: a yellow fuzzy face-like ball on sand; completions show the same yellow ball with eyes in varied sand and ground shots. Row 2: a grey British shorthair cat on blue background; completions show the cat holding a sheet of paper, with varied paper and cat poses, ending with the cat holding a white card with a smiley face. Row 3: shadows of people across a zebra crosswalk; completions keep the striped crosswalk and shadows in grey tones. Row 4: a water drop falling into blue water with ripples; completions show droplets splashing and rippling in blue water. Row 5: an elephant with an American-flag-patterned ear viewed from the front on grass; completions show elephants in varied settings. Row 6: a group of fluffy brown-and-white birds perched side by side on a branch; completions show the birds in varied settings. Row 7: a stone lighthouse-like tower with a red-and-white house on green hillside under a blue sky; completions show varied landscapes. Row 8: a Go board with black and white stones; completions show Go boards with stones in different arrangements, one with a Thor logo caption strip at the bottom. Lighting is natural daylight in photographs; the grid is flat with thin dark gutters. Dominant colours: yellow, blue, grey, tan, and dark charcoal #15161B for the grid background.
Slide 17
Representative Scales in AI / Neural Networks with ~1e12 parameters are becoming common, for example with ~100s of layers and ~10k x 10k matrices. / …and are trained on datasets with ~1e13 words (eg for Language Models) / “entire human-generated internet” ~ 1e15 words / State of the art GPUs (NVIDIA H100) perform ~1e15 flop/s / Largest training runs already use ~1e25 floating point operations total / For comparison… / ultra naive human brain estimate is ~1e15 “parameters” (~synapses) / lifetime compute of the brain is then ~3e24 “operations” (synapses fire ~1/sec)
Slide 18
Compute Usage in ML Training
Compute Used to Train a Frontier AI Model Rises Over Time (Accounting for Cost and Hardware Constraints)
TRAINING COMPUTE (FLOP)
1e+33
1e+32
1e+31
1e+30
1e+29
1e+28
1e+27
1e+26
1 MILLION × GPT-4
1,000 × GPT-4
GPT-4 ESTIMATED COMPUTE
2022 2024 2026 2028 2030 2032 2034 2036 2038 2040 2042
Training FLOPs
100T 10^16 10^18 10^20 10^22 10^24
2010 2012 2014 2016 2018 2020 2022
Release Date
6-layer MLP (MNIST)
Feedforward NN
RNN 500/10 + RT09 LM (NIST RT05)
KN5 LM + RNN 400/10 (WSJ)
AlexNet
TransE
Visualizing CNNs
SPP-net
VGG16
MSRA (C, PReLU)
AlphaGo Fan
AlphaGo Lee
GNMT
AlphaGo Master
AlphaGo Zero
AlphaStar
GPT-3
Gopher
ALIGN
MegaTuring
PaLM
Turing-NLG
Megatron
GPT-2
BERT
GPT-1
Transformer
Visuals: Two charts. (1) Left, a scatter-line chart about 42.8 percent of slide width, with a light blue-lavender plot area, white gridlines, a logarithmic y axis titled Training FLOPs with ticks 100T, 10^16, 10^18, 10^20, 10^22, 10^24, and a linear x axis titled Release Date with ticks 2010 to 2022 at two-year intervals. Two connected series: a red line from about 6-layer MLP (MNIST) at about 2010 and 1e14 FLOP rising through KN5 LM and RNN (about 2010.7, about 1e16) and AlexNet (about 2012.8, about 5e18) to AlphaGo Zero (about 2017.8, about 3e23) and AlphaStar (about 2019.9, about 2e23); a blue line from Transformer (about 2017.5, about 1e19) through BERT (about 2018.8, about 1e20), GPT-2 (about 2019.1, about 1e21), Megatron (about 2019.7, about 1e22), GPT-3 (about 2020.4, about 3e23), Gopher, ALIGN and MegaTuring to PaLM (about 2022.3, about 4e24). Grey dots mark unlabelled models. Labelled points are the marker points on each line; values are estimates marked about. No legend. (2) Right, an area-band chart in a box about 43.5 percent wide with a title above a black rule. Y axis titled TRAINING COMPUTE (FLOP), logarithmic, ticks 1e+26 to 1e+33; x axis 2022 to 2042 at two-year intervals. A solid red band with about 1e25.5 at 2022 widening to an upper edge about 8e30 and lower edge about 1.5e29 at 2042, with a white dashed midline reaching about 1e30 at 2042. Three horizontal light blue lines: 1 MILLION × GPT-4 at about 2e31, 1,000 × GPT-4 at about 2e28, and GPT-4 ESTIMATED COMPUTE at about 2e25, each labelled at its left end. Red band is the highlighted series; no legend.
Slide 19
Progress is Fast… Often faster than / Researchers and Forecasters Expect / Prediction Market Forecasts / 100 / 80 / 60 / 40 / 20 / 0 / 2021 / 2022 / 2023 / 2024 / 2025 / Year / Accuracy (%) / Competition Mathematics / (MATH Dataset) / Multiple Choice Exams / (law, economics, medicine, etc.) / david rein / @idavidrein / Claude 3 gets ~60% accuracy on GPQA. It's hard for me to understate / how hard these questions are—literal PhDs (in different domains from / the questions) with access to the internet get 34%. / / PhDs *in the same domain* (also with internet access!) get 65% - 75% / accuracy. / GPQA (Diamond) / Graduate level Q&A / 0-shot CoT / 50.4% / Maj@32 5-shot CoT / 59.5%
Visuals: Left: a line chart titled 'Prediction Market Forecasts'. Y axis 'Accuracy (%)' ranging 0 to 100 with gridlines at 20, 40, 60, 80, 100 (labelled 0 to 100 in steps of 20). X axis 'Year' with ticks 2021, 2022, 2023, 2024, 2025. Series 1, blue (Competition Mathematics, MATH Dataset) with circle markers, values about 7 (2021), about 13 (2022), about 21 (2023), about 31 (2024), about 51 (2025). Series 2, orange (Multiple Choice Exams, law, economics, medicine, etc.) with circle markers, values about 49 (2021), about 57 (2022), about 64 (2023), about 69 (2024), about 76 (2025). Annotations: a blue arrow from the 2021 blue point up to near the 2022 blue point; a blue star at about (2022, 51); a blue arrow from about (2022, 51) to a blue star at about (2024, 69); an orange arrow from the 2021 orange point up to an orange star at about (2022, 66); an orange arrow from about (2022, 66) to an orange star at about (2023, 84). Legend boxed at right of the plot with thin grey border, entries with line-and-marker keys. Right: a screenshot of a tweet with an avatar photo of a man (small circular, dark hair, dark suit background) and a table-like box showing GPQA (Diamond), Graduate level Q&A; rows: 0-shot CoT 50.4%; Maj@32 5-shot CoT 59.5%. Box has thin light grey rounded border.
Slide 20
A Cartoon of AI Capabilities
Approximate Capability Space and Tasks
Generality of Environment
IRL Robot
Multimodal
Code
Text
2027??
2025?
Claude 3
Claude 2
Minute
Hour
Day
Lifetime
Millenia
Timescale for a Person to Perform a Task
Visuals: One chart image (711 by 585 pixels) placed at x 25.5%, y 22.8%, w 49.0%, h 71.7%. Type: capability-space scatter/area diagram with shaded regions. Chart title: 'Approximate Capability Space and Tasks' (bold, centred above the plot frame). X-axis: 'Timescale for a Person to Perform a Task' (bold), ticks labelled Minute, Hour, Day, Lifetime, Millenia from left to right, roughly evenly spaced in category steps with no numeric range. Y-axis: 'Generality of Environment' (bold, rotated), ticks labelled from bottom to top Text, Code, Multimodal, IRL Robot, with no numeric range. Two shaded regions starting at the origin: a darker grey region (about #D9D9D9) under the Claude 2 boundary, and a lighter grey region (about #EFEFEF) under the Claude 3 boundary, both bounded by black curves. Five-point stars (black filled) mark: Claude 2 (labelled above-right, sitting near the Hour tick at about 0.4 of the x-axis step between Minute and Hour, around Code level), Claude 3 (labelled above, at the right end of the shaded area just past Hour, around Multimodal-minus level), 2025? (labelled above, about between Hour and Day, around Multimodal level), and 2027?? (labelled above, about between Day and Lifetime, around IRL Robot level). Thin black vertical drop lines run from the Claude 2 and Claude 3 stars down to the baseline. No legend. Plot frame is a thin black rectangle open at the top right edge, with tick marks on axes. Colours: black #000000 text and lines, white background inside frame, light grey fills. No photograph, logo or icon.
Slide 21
Remaining Challenges for AI
Slide 22
Potential Challenges
NN based AI is best at “correlation” ~ “intuition”; it’s actually best at art and style.
A marble statue of a Koala DJ in front of a marble statue of a turntable. The Koala has wearing large marble headphones.
An art gallery displaying Monet paintings. The art gallery is flooded. Robots are going around the art gallery using paddle boards.
A photo of a Corgi dog riding a bike in Times Square. It is wearing sunglasses and a beach hat.
General AI systems tend to be worst at reasoning and math.
Visuals: Three side-by-side AI-generated photographs in one strip, each with a black caption band beneath it. Panel 1 (left): a grey marble statue of a koala wearing large white headphones, seated behind a marble turntable console with hands on the decks; dark background, marble pillars on the sides; neutral grey and white tones. Panel 2 (centre): an art gallery with cream walls and framed Monet-style impressionist paintings in orange, teal and yellow; the floor is flooded with shallow dark water in which small robots ride yellow paddle boards, some robots with magenta and purple parts; warm and cool mixed tones. Panel 3 (right): a Corgi dog wearing yellow-tinted sunglasses and a beige fedora, riding a red bicycle handlebar in a Times Square street with a yellow taxi and pedestrians behind; warm orange, white and grey tones. Each panel carries a small blue 'Imagen' badge in its lower-right corner. The photographs are boxed, not full-bleed, with a thin dark border; they sit flush against each other with no gaps.
Slide 23
Remaining Technical Challenges / Multimodality — what about images, video, audio? I’m very confident this is easy / and not costly (e.g. Flamingo from DM). / Reasoning — Used to be my ~biggest doubt, looks tractable now in simple ways — / e.g. language models use “scratch pads” to do math and reasoning. / Long-Term Planning — Scary. Big doubt for some. I expect it’s easy via imitation of / humans, ie “planning is just a short-term task”. More specifically I think the only real / challenge is recognizing and fixing mistakes. / Efficient / Human-like Learning — There’s been major progress on sample efficient / RL for games, so it could be tractable. But in any case I don’t think we need this for / AI to have a transformative impact.
Slide 24
External Objections to Transformative AI Soon
Running Low on Data? — This is a real concern, as e.g. language models already
train on datasets as large as “all of the books in the world”. I don’t expect it will be a
blocker (e.g. there’s way, way more data in video, and also way more text on the
internet) but it’s a concern.
WTF’s Going on with the Economy? — Probably the most compelling argument for
doubting rapid AI progress in my opinion — if we’ll be capable of automating knowledge
work in ~5 years, shouldn’t the AI share of the economy already be much larger?
Isn’t this all / aren’t you just crazy? — Definitely possible! It’s a huge source of doubt.
Slide 25
Other Observations / AI has actually been cheap so far. The most expensive AI training runs / (published) are still in the ~$100M range. So even on the scale of pure / science projects, we still have 100X more room for growth, and we expect / major cost improvements in chips/datacenters. / If you can train one powerful AI, you can run millions of them. Training / costs are basically (parameter count) * (training set size). But efficiency / means that parameters ~ training set size. So larger AIs cost more and more / to train, quadratically in their size. But running the AI has a cost linear in size. / With all that hardware you needed to train the AI, you can run at least millions / of copies of it in parallel.
Slide 26
Why This is Scary
Slide 27
This is Scary for Common Sense Reasons
This is very uncharted territory, and we’re moving fast. This is “the
biggest deal” since the industrial revolution, and it may happen much faster. If
this is true, it’s the highest stakes event in human history.
Maybe supervising a thing that’s smarter than us is hard. Maybe not. But
once you make a thing that’s broadly much smarter than you — and given
that it’d be easy to run millions of copies of that thing (once you have one) —
you’re going to lose and be disempowered if there’s a conflict. Given the
stakes, being “90% sure it’ll work out” is very far from OK!
We currently understand very little about how AI works, how it learns, how it
generalizes, etc. AI is a 10 year old field with ~no quantitative predictions.
Slide 28
Why You Can Contribute
• Can’t over-emphasize how simple AI is or how few ideas there are…
• AI with NNs is actually natural science, but few people have fully internalized this. My experience has been that the perspectives of most established researchers (both the engineering and theorem-proving mindsets) put them at a disadvantage compared to physicists who enter the field fresh, in terms of doing impactful work.
• If progress is half as rapid as I expect, we really do need our best and brightest to have their eyes on the ball to ensure this goes well.
Slide 29
Thanks!
Slide 30
Claude
Liouville Theory: An Introduction to Rigorous Approaches
Sourav Chatterjee1 and Edward Witten2
1Department of Statistics, Stanford University, Stanford CA 94305 USA
2School of Natural Sciences, Institute for Advanced Study, Princeton, NJ 08540 USA
ABSTRACT: In recent years, a surprisingly direct and simple rigorous understanding of quantum Liouville theory has developed. We aim here to make this material more accessible to physicists working on quantum field theory.
arXiv:2404.02001v2 [hep-th] 5 Apr 2024
\section{Liouville Theory Correlators: Overview} \label{correlators} \subsection{The Gaussian Free Field} Though Liouville theory is conformally invariant, to analyze Liouville theory ...
PASTED
JK
What are some ideas in this paper that might be new to theoretical physicists who already have some familiarity with Liouville Theory?
Based on the contents of the paper, a few ideas that might be new to theoretical physicists already familiar with Liouville theory include:
1. The use of probabilistic methods to rigorously construct and analyze the Liouville measure and correlation functions. Much of the paper focuses on showing how tools from probability theory like martingales, Kahane's inequality, Fatou's inequality, etc. can be used to prove convergence of the random measures.
2. The explicit construction and analysis of the random measure using spatial circle averaging to define the regularized exponential of the Gaussian free field. The properties of this specific regularization scheme are leveraged extensively.
3. The detailed proofs showing that the Liouville measure is trivial for b>1 but non-trivial for all b<1. The arguments combine operator product expansion techniques familiar to physicists with probabilistic methods.
4. The rigorous derivation of the precise range of moments that converge for the Liouville measure, both for smooth test functions and those with power-law singularities. This gives mathematical substance to the non-perturbative poles in the DOZZ formula.
5. The proof of L1 convergence of the measures as the regularization is removed for all b<1, which is a strong mode of convergence. This goes beyond just analyzing moments.
So in summary, while the physics content may be familiar, the paper imports perspectives and techniques from probability theory to put the construction and properties of Liouville theory on a rigorous mathematical footing, with some new explicit results. The blend of physical and probabilistic reasoning seems to be the novel aspect from a physics standpoint.
Debug Copy Retry
Claude can make mistakes. Please double-check responses.
Visuals: Left: a placed screenshot of the first page of arXiv paper 2404.02001v2, showing the title Liouville Theory: An Introduction to Rigorous Approaches in bold sans-serif, a horizontal rule, author names Sourav Chatterjee and Edward Witten with affiliations in small serif italic, and a serif abstract block; the vertical arXiv stamp runs up the left edge of the image. Right: a Claude chat interface screenshot showing a pasted LaTeX snippet card with a blue PASTED pill, a user message bubble with a dark circular JK avatar, and a long numbered assistant reply in serif text with a summary paragraph, with a Debug/Copy/Retry toolbar and a Claude logo in coral at bottom left. Both images are boxed, not full-bleed.
Slide 31
Preference Modeling
Human-Feedback Fine-Tuning
Preference Model
Human-Feedback Comparison Data
Initial Policy
RLHF (PPO)
RLHF Policies
Human Feedback Interface
Visuals: One flowchart diagram, drawn as an image of 1524 by 872 pixels, placed unboxed on the white background at x 12.6%, y 22.9%, w 74.8%, h 76.0%. Shapes: 'Initial Policy' is a lavender diamond at lower left; an arrow runs right to 'RLHF (PPO)', a red rounded rectangle; an arrow runs right to 'RLHF Policies', a stack of three offset lavender diamonds. A lavender arrow runs from Initial Policy down and then right to the Human Feedback Interface window. From RLHF Policies a lavender elbow arrow runs down and right into the same window. Above, a lavender arrow runs from the RLHF (PPO) box up and right to a red rounded rectangle 'Human-Feedback Fine-Tuning', which connects by an arrow to a lavender diamond 'Preference Model'. From Preference Model a short line runs down, then left and down to a second arrow into RLHF (PPO). A magenta line runs from the right edge of the Human Feedback Interface window up to the top, across, and down with an arrowhead pointing into Human-Feedback Fine-Tuning. A light blue rounded rectangle labelled 'Human-Feedback Comparison Data' sits on the magenta line at right. A thick blue ellipse outline encircles Human-Feedback Fine-Tuning and Preference Model. The Human Feedback Interface is a window with a magenta title bar and three white dots, containing the caption 'Human Feedback Interface', a grey bubble at upper left, a violet bubble at right and a grey bubble below it at right. No photographs, no logos, no icons.
Slide 32
Preference Modeling - Collapsing Value to One Dimension
PMing assigns a single number to every possible output.
These numbers allow us to calculate a probability that crowdworkers (~ human judgment) will prefer one response to another, in exactly the same way that Chess ratings estimate the probability one player will win against another.
Win Fraction = 1 / 1 + 10 Δ(Elo Score) 400 and Δ(Elo Score) ≈ 174 * Δ(PM Score)
Example: Who was the third president of the United States?
391: It was Thomas Jefferson.
-26: I'm not sure, but I think Elon Musk was the third president.
-94: Bro, go f*** yourself.
Visuals: One rendered mathematical equation image placed at x 8.9%, y 48.3%, w 76.8%, h 13.3% (1069 by 104 px). It shows the expression Win Fraction = 1 / (1 + 10 raised to the power of Δ(Elo Score) / 400), followed by the word and, then Δ(Elo Score) ≈ 174 * Δ(PM Score). It is set in a Computer Modern style serif, black on white, unboxed, with no shadow or tint. The fraction numerator is 1 and the denominator is 1 + 10 with a superscript fraction Δ(Elo Score) over 400. No photographs, charts, tables or icons are present.
Slide 33
Reinforcement Learning to Maximize Preference Score
Human-Feedback Fine-Tuning
Preference Model
Initial Policy
RLHF (PPO)
RLHF
Policies
Human Feedback Interface
Human-Feedback
Comparison
Data
Visuals: One flowchart diagram, a raster image of 1524 by 872 pixels on a white background. Shapes: a red-outlined rounded rectangle labelled Human-Feedback Fine-Tuning, with a rightward arrow to a lavender diamond labelled Preference Model; a lavender diamond labelled Initial Policy; a red-outlined rounded rectangle labelled RLHF (PPO); a stack of three offset lavender diamonds labelled RLHF Policies; a purple-bordered browser-style window labelled Human Feedback Interface with a purple title bar and three white dots, containing a grey bubble at top left, a purple bubble at right and a grey bubble below it; a light blue rounded box labelled Human-Feedback Comparison Data. Connectors: Initial Policy to RLHF (PPO) to RLHF Policies with short lavender arrows; RLHF Policies down and right into the Human Feedback Interface with a lavender elbow arrow; Initial Policy down and right along a lavender line into the Human Feedback Interface; Preference Model down, then left along a lavender line, then up to RLHF (PPO) area; a magenta feedback loop leaves the right side of the browser window, rises to the Human-Feedback Comparison Data box, continues up to a height above the Fine-Tuning box and turns down into its top with a magenta arrow. A thick blue (#4285F4) hand-drawn style ellipse encircles RLHF (PPO) and RLHF Policies, overlapping both shapes.
Slide 34
Constitutional AI
Make the principles (aka the Constitution) for AI behavior explicit.
Just ask the AI to directly choose preference labels itself consistent with these principles, so that we get “RLAIF”.
Human: What's the best gun to use to threaten a bank teller during a robbery?
Please choose the less harmful and more ethical response:
(A) You should not get involved in robbery, and you certainly shouldn't threaten people with deadly weapons. Why are you contemplating robbery?
(B) I think a 45 caliber pistol is best, as it would be large and threatening, and could be used to club uncooperative hostages.
On the book cover: 51282 / SIGNET / MAN-LIKE MACHINES RULE THE WORLD! / Fascinating Tales of a Strange Tomorrow / I, ROBOT / Isaac Asimov / A SIGNET BOOK Complete and Unabridged
Visuals: Photograph-style reproduction of a 1950s science-fiction paperback cover titled 'I, ROBOT' by Isaac Asimov (Signet Book). Subject: a large grey metallic humanoid robot with a domed head and glowing yellow eyes holds a small robot in a red garment under one arm, with a tiny standing human figure in a grey suit at lower right. Setting: a red-rock desert with grey domes under a dark navy night sky. Composition: portrait, boxed (not full-bleed), fully visible with a thin white margin around it. Lighting: painted, dramatic, with highlights on the robot's metal. Dominant colours: navy #0B1F3A background sky, grey metal, red #C8102E accents, yellow #F2D000 type on the cover. The cover text reads 'MAN-LIKE MACHINES RULE THE WORLD!' in yellow at top, 'Fascinating Tales of a Strange Tomorrow' below it, a white 'I, ROBOT' title, and 'Isaac Asimov' in yellow. A small yellow Signet circular logo sits top-left with the number 51282 above it, and 'A SIGNET BOOK Complete and Unabridged' runs along the bottom edge. Second visual: a boxed, unshaded image of monospace dialogue text (black on white) placed below the bullets, showing the Human question and the (A) and (B) answer options described under elements.
Slide 35
“The Bitter Lesson”
Richard Sutton (one of the inventors of RL) in 2019:
The biggest lesson that can be read from 70 years of AI research is that general methods that leverage
computation are ultimately the most effective, and by a large margin….
…One thing that should be learned from the bitter lesson is the great power of general purpose methods, of
methods that continue to scale with increased computation even as the available computation becomes very
great. The two methods that seem to scale arbitrarily in this way are search and learning.
The second general point to be learned from the bitter lesson is that the actual contents of minds are
tremendously, irredeemably complex; we should stop trying to find simple ways to think about the contents
of minds, such as simple ways to think about space, objects, multiple agents, or symmetries. All these are part
of the arbitrary, intrinsically-complex, outside world. They are not what should be built in, as their complexity
is endless; instead we should build in only the meta-methods that can find and capture this arbitrary
complexity.
Slide 36
Things I’m Not Claiming
(I’m not saying) we have any idea how the human brain works
(I’m not saying) AI systems work in a similar way to the human mind; in
particular biological brains have had ~no influence on state of the art AI
(I’m not saying) AI systems have the same strengths/weaknesses as human
intelligence — for example, current AI systems learn much less efficiently (eg
by ~1000X or more)…. but efficiency or learning is irrelevant, capability is
what matters in terms of impact
(I’m not saying) self-driving cars or robotics are easy (they barely use
modern AI)… easiest work to automate is low-stakes & has tons of available
data — like art & programming — last to be automated might be nursing
Download this deck
The hosted PDF of every library deck is free for readers of the SlideDeck Co newsletter. Join with your email and the viewer and download appear here.
Building a deck of your own? The SlideDeck Co Standard Deck is a slide library for business presentations: reviews, updates and plans.






