Teaching a Brute-Forcer to Dream: Fixed vs. Recursive Exploration

Google DeepMind's Dream-RSI lets an agent improve how it searches by "dreaming" against its own history instead of running costly real trials. Here's the fixed-vs-recursive difference, built as a directory brute-forcer, and what it means for Black Hat Python tooling.

$ cat table_of_contents.txt

Dream-RSI and the difference between fixed and recursive exploration

Sep 21, 2026 · @Dennis Gonzalez

Every tool that searches a large space answers one question over and over: what do I try next? A directory brute-forcer asks it for every line of a wordlist. A fuzzer asks it for every mutation. A chess engine asks it for every candidate move. The rule that answers is the exploration strategy — and a new paper from Google and Google DeepMind argues that improving that rule, not the underlying model, is where the real leverage sits.

The paper is Dream-RSI: Recursive Self-Improvement through Evolving Worlds, posted to arXiv on September 14, 2026 by seventeen authors from Google, Google DeepMind, the University of Maryland, and the University of Virginia. RSI stands for recursive self-improvement — not the trading indicator, and not a repetitive-strain injury, both of which a search cheerfully offers up first.

The bottleneck: exploration doesn’t scale

The paper frames a dilemma every security tinkerer will recognize. Fixed strategies fail to adapt as the search space grows, while online policy optimization — learning a better strategy as you go — means navigating a huge meta-search space under slow, expensive feedback. In plain terms: a hardcoded wordlist is dumb but cheap, and learning on the fly is smart but costly. Neither is satisfying once the target space gets large.

Dream-RSI’s move: dream in a replay simulator

The clever bit is this. Every time the agent explores, it leaves a discovery tree — a record of what it tried, what failed, and what worked. That accumulated history can itself serve as a replay simulator of the search space. So instead of paying for thousands of real trials to test a new strategy, the system dreams: it replays against its own logged history for free.

The loop has three stages, and it leaves the underlying coding agent completely unchanged — only the exploration-policy code changes; the models, evaluator, and execution interfaces stay fixed.

  1. Explore for real. Run the current strategy against the real environment (calling the model, compiling and running code), logging every path into a discovery tree.
  2. Build the simulator. Turn that history into a reusable replay simulator.
  3. Dream. A dedicated agent writes thousands of alternative strategies and scores them against the replay — cheaply, no new real runs — then deploys the winner, whose fresh discoveries enrich the simulator for the next round.

The chess analogy that sticks: you don’t re-set the board and replay a whole game to study it. You read the scorebook.

The strategic difference, by example

One nice detail from the sources: “Recursive Fixed Exploration” is the actual name of the baseline the paper compares against, and it maps almost exactly onto a toy directory brute-forcer. Picture a simulated site where the hidden paths cluster under a few prefixes. Two strategies:

Recursive Fixed Exploration (the baseline). A human writes the wordlist order once. It never changes. Run it against six fresh-but-similar targets and it costs the same every time:

FIXED strategy (never learns):
  real requests per scan: [78, 78, 78, 78, 78, 78]

Cycle six is exactly as dumb as cycle one. It has no memory.

Dream-RSI-style recursion. Same agent, same starting point — the only difference is that it rewrites its own ordering between rounds using the replay. Give three hidden regions different strengths (/api/v1 densest, /admin medium, /vendor sparse) and watch it sort them out on its own:

RECURSIVE strategy (dreams between scans):
  cycle 1: 78 real requests   ranking: /api/v1 > /admin > /vendor
  cycle 2: 42 real requests   ranking: /api/v1 > /admin > /vendor
  ...
  cycle 7: 42 real requests   ranking: /api/v1 > /admin > /vendor

Cost fell 78 -> 42 requests (46% cheaper), and it sorted the three
regions by strength on its own.

Cycle one is identical to fixed, because it has no log yet. After that it dreams: it scores hundreds of candidate orderings against its history and front-loads whatever pays off. Nothing about the model changed. The only thing that evolved is the order it chose to try things in — which it wrote for itself.

The paper’s real numbers are bigger versions of the same story: up to 162x fewer agent calls on a Lasso solver against one baseline, over 50x budget savings on three math problems, and roughly 1.8–2.4x fewer generations on GPU kernels — across algorithm engineering, mathematical optimization, and GPU kernel engineering. Because the improvement lives at the exploration layer, small models can benefit too.

The honest caveats

Two things are worth keeping straight before anyone reaches for the word “AGI.”

First, this is self-improvement in a narrow, contained sense. The weights don’t move. It’s the search scaffolding that rewrites itself — closer to a program tuning its own config than to a mind bootstrapping intelligence.

Second, reviewers have flagged what’s missing. One analysis notes that Dream-RSI and Recursive Fixed Exploration share the same agent, evaluator, initialization, and constraints, differing only in whether the policy updates between rounds — a clean comparison — but that the paper lacks a control that randomly picks among revised policies instead of using the best replay-scorer. In other words, some of the gain might come from simply iterating any revision, not from the replay ranking specifically. Good skepticism to carry.

Why this belongs on a security blog

If you’re working through Black Hat Python, you’ve already written fixed exploration without calling it that. The web-hacking chapter’s directory and vhost brute-forcers read a wordlist and march through it in order — pure Recursive Fixed Exploration. The mental upgrade Dream-RSI offers is: what if the scanner kept a log of every request it ever made, rehearsed better orderings against that log offline, and only spent real requests on the ordering that won?

That’s not hypothetical for offensive tooling. The same “get cheaper and better at searching a huge space” pattern is exactly what drives modern AI-assisted fuzzing and vulnerability discovery.

And there’s a real safety reason the toy uses a simulated target: the replay-first idea isn’t just efficient, it’s the disciplined way to iterate without hammering a live server you may not own. The Black Hat Python scanners send real traffic, so they only ever belong against targets you own or have written permission to test.