Reference bots: CLASSIC and ABYSS compared

This chapter examines the reference bots in the release. The repository includes their builds, seven-run scores, timing checks, replays, and pilot tests. We use those records to explain how each bot works and what its measured score shows.

Assemble the bot from the components we have studied

The strongest reference bots do two jobs within one VM time budget. Their power work earns or maintains boosts. Their pilot observes the arena, requests aim and movement, and routes toward gems. The game adds EXP only when the knight collects a gem, which is where the machine behavior finally meets the score.

The two reference bots have different surviving files. CLASSIC has a saved compiled file but not its complete original source. ABYSS has both its generator and the source it generated. We can trace the ABYSS pilot line by line. We cannot claim to know the CLASSIC pilot's exact movement code.

CLASSIC apex: what the saved bot does

The saved CLASSIC apex is 53,264 bytes, while its earlier version is 21,066 bytes. The apex keeps eight chains active across all four families, and that spread matters.

In the cap formula, K counts active chains, Fam counts their families, and Keff is the count the cap actually uses. Eight chains across four families give Keff=8. Put all eight in one family and Keff drops to 2, even though K stays 8.

What changed from the seven-chain predecessor?

The two saved files allow a comparison. The repository says the earlier bot has seven recognized chains. The apex keeps those seven and adds LOCKSMITH. Thus, the cap count changes from Keff=7 to Keff=8. LOCKSMITH does not write EXP directly.

The comparison is shown as cards below →

Why the bigger file earns LOCKSMITH

The VM uses the uploaded file's length as the end of its code region. It counts padding in that length. The apex extends the code region over its host-call arguments. The host call then reads bytes that the VM also counts as code.

The repository keeps both uploads, so we can check the byte sizes ourselves. The earlier file ends at 0x524A, while the apex ends at 0xD010. Its last nonzero byte is at 0x6F3C, and the remaining 24,787 bytes are zero. Those zeros still count toward the uploaded length, so the VM does not set code_end to the last useful instruction:

    pub fn new_with_mode(code: &[u8], mode: EconomyMode) -> Self {
        let mut mem = Mem::new();
        mem.load_image(code);
        let image = mem.bytes.clone();
        let code_end = code.len().min(VM_MEM_SIZE) as u32;

    fn new_with_mode(code, mode) {
        make fresh 64 KiB VM memory
        load the uploaded code bytes
        save a copy of those original bytes
        remember the end of the uploaded image

How does the stack reach that address? PUSH_ZERO is byte 03; it pushes one four-byte word. The apex contains 5,120 consecutive 03 bytes from 0x26E2 through 0x3AE1. These are actual instructions.

Starting at sp=0x8000, 5,120 net pushes move the stack by 0x5000 bytes to 0xD000. The README calls this the stack parking step. The zero padding extends code_end to 0xD010, just past the parked stack.

pub const VM_MEM_SIZE: usize = 0x10000;
pub const ADDR_MASK: u32 = 0xFFFF;
pub const IP_INIT: u32 = 0x0000;
pub const SP_INIT: u32 = 0x8000;
pub const PER_TICK_INSTR_BUDGET: u32 = 10_000;

outer VM limits {
    memory = 64 KiB, with addresses wrapping at its end
    first instruction = address 0
    stack begins at address 0x8000 and grows upward
    one game tick permits 10,000 VM instructions
}

pub mod op {
    pub const PUSH: u8 = 0x01;
    pub const PUSH_SHORT: u8 = 0x02;
    pub const PUSH_ZERO: u8 = 0x03;

push instruction choices {
    PUSH stores a full number
    PUSH_SHORT stores a small signed number
    PUSH_ZERO stores zero

        self.mem.store_u32(self.sp, v);
        if self.sp > self.sp_hi {
            self.sp_hi = self.sp;
        }
        self.sp = self.sp.wrapping_add(4);

        store one four-byte word at the current stack address
        if (this is the highest stack address reached) {
            remember it
        }
        advance the stack pointer by four bytes

Consider a call made with sp=0xD000. Before the VM removes stack values, it saves this address as base. First, it reads the argument count at base−4 = 0xCFFC. Lower addresses hold arguments and the call number.

All these addresses are less than 0xD010. Thus, a < code_end reports that the call frame overlaps the code region. The detector sets marshal_alias. It does not check for harmful argument values.

This code tests each call-frame word address. It checks for an overlap with the reserved challenge page or the image. The apex overlaps the image:

    fn note_syscall_frame(&mut self, base: u32, argc: u32) {
        for k in 1..=(argc + 2) {
            let a = base.wrapping_sub(4 * k) & ADDR_MASK;
            if (RESERVED_ALIAS_LO..RESERVED_ALIAS_HI).contains(&a) || a < self.code_end {
                self.marshal_alias = true;
            }

    fn note_syscall_frame(base, argc) {
        for (each service or argument word below the frame base) {
            if (it overlaps the original code or a reserved area) {
                mark the unusual frame placement
            }

That VM observation is only one part of LOCKSMITH. In the same tick, the host also checks for the LOCK word at 0xFF2C and a nonzero reply at 0xFF28. The host wrote the challenge at 0xFF24, and the required reply is econ_hash(challenge XOR LOCK_ECHO).

The apex README says its bytecode includes econ_hash. The host code gives the required formula. The saved binary does not identify which instructions calculate each reply.

pub(crate) fn lock_challenge(tick: u32) -> u32 {
    splitmix64((KEY_SEED as u64) ^ K_LOCKSMITH ^ ((tick as u64) << 32)) as u32
}
pub(crate) fn lock_response(c: u32) -> u32 {
    econ_hash(c ^ LOCK_ECHO)
}

fn lock_challenge(tick) {
    mix the secret seed, LOCKSMITH tag, and current tick
    return this tick's challenge
}
fn lock_response(challenge) {
    mix the challenge with the LOCKSMITH echo constant
    return the expected reply
}

    fn locksmith_armed(&self, mem: &Mem, l: PwnLatches) -> bool {
        if !l.marshal_alias || mem.load_u32(LOCK_MAGIC_ADDR) != LOCK_MAGIC {
            return false;
        }
        let r = mem.load_u32(LOCK_RESP_ADDR);
        r != 0 && r == lock_response(lock_challenge(self.tick))
    }

    fn locksmith_armed(mem, latches) {
        if (no unusual host-call frame or wrong magic word) { return false }
        read the stored answer
        return answer is nonzero and matches this tick's challenge
    }

To reproduce the chain, extend the image with padding and park the stack inside its extent. Make a real SYSCALL each tick to set marshal_alias. Read that tick's challenge, write the reply and magic word, and keep the pilot running.

Padding alone changes only the address comparison. It does not answer the challenge, fight enemies, or collect EXP. The earlier file has code_end=0x524A, which never reaches the normal operand stack. That is why the repository uses these two uploads as a before-and-after check for this eighth chain.

What the surviving files cannot tell us

The saved files show the chain difference, but they do not contain the original source. So we cannot identify the apex pilot's exact movement code, or assign its full score to LOCKSMITH alone.

The CLASSIC payout example starts after a chain passes the host check. Its meter changes limited knight fields. The game must still create a gem, and the knight must collect it.

Earlier chapters show example pilots. They do not reconstruct the missing CLASSIC apex source.

During a match, the bot must repeat the chain behaviors or their meters fall. Active meters give limited attack, survival, and collection bonuses, but more active power also raises HEAT, which makes survival harder.

A useful pilot stays where its attacks can hit several targets, survives contact, and collects the gems from those kills. The diver chapter gives one complete example. It does not claim that the saved apex uses the same pilot.

The same saved file scored a median 224,384 with older rewards and 22,704 after the A1 balance change. It still had eight recognized chains. Smaller bonuses slowed kills, which meant fewer gems, fewer level-ups, and fewer chances to stay alive. One multiplier cannot explain the full score change.

ABYSS apex: generator, compiled program, inner work, and pilot

For ABYSS, the repository has both the generator and the source it produced. A generator writes another program. It runs before the match. The normal compiler then turns the generated source into the uploaded program.

Follow how the ABYSS apex was built

The apex build reuses earlier programs. Its driver loads the Chasm generator, which loads Rift, which in turn pulls in the earlier Entry pilot and a saved test fixture.

The apex then asks Chasm's builder for source, a raw image, a compiled file, and build notes. This is the build order, not the order of actions during a match.

The two driver modules show these imports:

ROOT = HERE.parents[2]
CHASM_PATH = ROOT / "bots/exploit/abyss_chasm/generate.py"

_spec = importlib.util.spec_from_file_location("abyss_chasm_generate", CHASM_PATH)
if _spec is None or _spec.loader is None:
    raise RuntimeError("cannot load Chasm generator")
CHASM = importlib.util.module_from_spec(_spec)
sys.modules[_spec.name] = CHASM
_spec.loader.exec_module(CHASM)

Apex builder setup {
    locate Chasm's builder
    load its code as a Python module
    reuse its inner ABYSS construction steps
}

HERE = Path(__file__).resolve().parent
ROOT = HERE.parents[2]
RIFT_PATH = ROOT / "bots/exploit/abyss_rift/generate.py"

_spec = importlib.util.spec_from_file_location("abyss_rift_generate", RIFT_PATH)
if _spec is None or _spec.loader is None:
    raise RuntimeError("cannot load Rift generator")
RIFT = importlib.util.module_from_spec(_spec)
sys.modules[_spec.name] = RIFT
_spec.loader.exec_module(RIFT)

Chasm builder setup {
    locate Rift's builder
    load its code as a Python module
    reuse the earlier proof construction steps
}

The apex writes three files. The .bot file contains readable outer source. The runner receives the .bin file and extracts VM instructions from it. The .trace.json file records build and grading details. It does not run in the arena:

    check_or_write(BOT, source.encode(), args.check)
    check_or_write(BIN, artifact, args.check)
    check_or_write(TRACE, canonical_json(trace), args.check)

    when finishing the Apex build {
        save the readable bot source
        save the uploadable binary
        save a build record for checking later
    }

In one game tick, the pilot reads nearby enemies and aims. Gate work can run or pause. If the host accepts the result, it saves more proof. A Drive boost can then make a hit stronger.

The knight must still move to collect a dropped gem. Pilot work and Gate work share the outer VM's time limit. Too much work on either job can prevent the other from finishing.

The reference notes say the builder uses the public compiler without patching the compiled binary. Thus, the recorded build can come from source accepted by that compiler. A raw-binary method would be a different claim.

Accepted proof permits strong attack, survival, movement, and collection settings. The pilot uses these settings in the arena. Later, we compare the score with real gem pickups. We also examine tests that block pilot actions.

The saved Entry, Rift, and Chasm bots stop at earlier proof steps. They show how the system changes as the host accepts more proof. Their Gate work, time use, and match paths also differ. They are not the apex with only a smaller damage value.

The gameplay pilot inside the ABYSS apex

The generated apex source calls its pilot the Rift pilot, the gameplay code in the highest-scoring ABYSS reference. It does not orbit a group like diver. Instead, it aims at valuable nearby enemies and checks walls, using fewer calculations and waiting longer before it flees.

Use the interactive example before reading the functions. Start with 80% HP, three scanned enemies, and a gem closer than the nearest enemy. Then add enemies and reduce HP. When the escape condition passes, the pilot flees before it checks the gem. The example uses the pilot's strict < comparisons at the health and gem limits.

Construction choice 1: keep aim selection cheap

The target helper considers at most the first two scanned enemies. Its weights are thief 100, gnome 60, shaman 40, panda 35, bear 30, and default 10. For each candidate, it divides the weight by squared distance:

fn best_target(n, mx, my) {
    if (n > 2) { n = 2; }
    let best = 0; let score = 0 - 1; let i = 0;
    while (i < n) {
        let dx = enemy_x(i) - mx; let dy = enemy_y(i) - my;
        let candidate = value_of(enemy_type(i)) * 100000 / (dx*dx + dy*dy + 1);
        if (candidate > score) { score = candidate; best = i; }
        i = i + 1;
    }
    return best;
}

fn best_target(visible_enemies) {
    inspect only the first two scan records
    for (each inspected enemy) {
        score its type value against distance
        remember the best score
    }
    return that enemy's index
}

The bot can miss a better third enemy. It scores only two targets instead of up to sixteen. This leaves more VM time for other work.

Gate work and pilot work share VM time. A faster decision can let both jobs run during the tick.

Construction choice 2: shorten wall look-ahead

The ordinary gapkiter checks one, two, and three tile-scale steps ahead. The apex helper checks only the first step:

fn path_clear(hx, hy, dx, dy) {
    let ax = dx; if (ax < 0) { ax = 0 - ax; }
    let ay = dy; if (ay < 0) { ay = 0 - ay; }
    let mx = ax; let mn = ay; if (ay > ax) { mx = ay; mn = ax; }
    let m = mx + (mn * 7 / 16); if (m < 1) { m = 1; }
    let ux = (dx * 64) / m; let uy = (dy * 64) / m;
    if (is_solid(hx + ux, hy + uy)) { return 0; }
    return 1;
}

fn path_clear(direction) {
    sample one spot roughly one tile ahead
    if (that spot is a wall) { return false }
    return true
}

The apex still tests several directions and adjusts their lengths, but it does not look as far ahead. A clear first sample can lead to a wall farther away, so the pilot checks again on later ticks and can change direction. Each choice uses fewer scans and calculations.

Construction choice 3: escape on health pressure

The complete main pilot routine shows its priority order:

fn pilot() {
    sense();
    let n = enemy_count();
    if (n <= 0) {
        let mx = my_x(); let my = my_y();
        if (gem_count() > 0) {
            step_heading(mx, my, gem_x(0) - mx, gem_y(0) - my);
        }
        return;
    }

    let mx = my_x(); let my = my_y();
    let reach = my_reach();
    let target = best_target(n, mx, my);
    aim_at(enemy_x(target), enemy_y(target));

    let ndx = enemy_x(0) - mx; let ndy = enemy_y(0) - my;
    let nd2 = ndx*ndx + ndy*ndy;
    let hp_pct = my_hp() * 100 / (100 + 18 * (my_level() - 1));

    if (hp_pct < 45 || (hp_pct < 70 && n >= 8)) {
        step_escape(n, mx, my);
        return;
    }

    let dx = 0; let dy = 0;
    let gems = gem_count();
    let gx = 0; let gy = 0; let gd2 = 1 << 30;
    if (gems > 0) {
        gx = gem_x(0) - mx; gy = gem_y(0) - my; gd2 = gx*gx + gy*gy;
    }
    if (gems > 0 && gd2 < nd2 && gd2 < 340*340) {
        dx = gx; dy = gy;
    } else if (nd2 > reach*reach) {
        dx = ndx; dy = ndy;
    } else {
        dx = 0 - ndx; dy = 0 - ndy;
    }
    if (dx != 0 || dy != 0) { step_heading(mx, my, dx, dy); }
}

fn pilot() {
    read the knight and its short enemy scan
    if (no enemy is visible) {
        head toward the first visible gem, if there is one
        return
    }
    score at most two enemies and aim at the better target
    estimate danger from health and crowd size
    if (health is under pressure) {
        escape; return
    }
    compare the closest enemy with a useful gem
    request a move using a cheap wall check
}

First, the routine refreshes status and enemies. If it sees no enemy, it moves toward the first gem and returns. Otherwise, it reads knight position and weapon reach. It then selects an aim target and reads the nearest enemy's position relative to the knight.

The health estimate is HP * 100 / (100 + 18*(level-1)). The denominator is the knight's normal maximum HP, so the result is an integer percentage. The pilot flees below 45% HP, or below 70% if it scans at least eight enemies.

Unlike ordinary gapkiter, this pilot does not flee only because it sees many enemies. It waits until HP is low. Its stronger survival lets it stay near enemies for longer.

At 80% HP with ten enemies, the bot does not flee. At 60% HP with ten enemies, the second rule makes it flee. At 40% HP with one enemy, the first rule makes it flee.

The code uses <, not <=. At exactly 45% HP with few enemies, the first rule does not apply. At exactly 70% HP, the second rule does not apply. Check these limits when you read a replay.

Construction choice 4: simplify directional threat

Escape compares eight compass directions and gives a large penalty to a blocked one. It calculates enemy pressure from only the nearest enemy: for direction (dx,dy) and enemy displacement (ex,ey), it starts with max(0, ex*dx + ey*dy). It then selects the lowest pressure, and the strict comparison keeps the first candidate when pressures are equal.

Unlike gapkiter, it does not add pressure from every enemy. If the nearest enemy is east, eastward choices get a penalty. Westward choices start at zero before wall penalties.

Other enemies can still make west dangerous. The pilot ignores them to save time. This is a fast choice, not a complete safety check.

Construction choice 5: keep the collection loop alive

If the pilot does not flee, it calls gem_count(). It reads only gem_x(0) and gem_y(0). The host sorts scanned gems by distance. Thus, index 0 is the nearest listed gem inside the scan box.

If that gem is closer than the nearest enemy and within 340 pixels, the pilot moves toward it. Otherwise, it approaches an enemy beyond weapon reach, or backs away from one already in reach. It sends a move request only when the direction is not zero. Reading one gem saves work, but the pilot does not compare all gem values or routes.

The host sorts gems before the pilot reads them. It uses squared distance, then slot number for equal distances. It returns only the requested number of entries:

            let dx = (g.x - hx) as i64;
            let dy = (g.y - hy) as i64;
            cand.push((dx * dx + dy * dy, slot));
        }
        cand.sort_by(|a, c| a.0.cmp(&c.0).then(a.1.cmp(&c.1)));
        let cap = n_req.min(SCAN_GEMS_MAX) as usize;
        let n = cap.min(cand.len());
        for (i, &(_, slot)) in cand.iter().take(n).enumerate() {
            let g = &self.gems[slot];
            let base = result_addr.wrapping_add((i as u32) * 12);
            mem.store_i16(base, to_int(g.x) as i16);
            mem.store_i16(base.wrapping_add(2), to_int(g.y) as i16);

            when scanning gems {
                sort visible gems from nearest to farthest
                keep only as many as the small scan allows
                copy those gem records into the bot's VM memory
            }

A large pickup radius alone is not enough. The pilot moves toward drops while it stays near enemies it can hit. The game checks the actual distances and pickups. Later, a test blocks gem-directed moves, and its scores show the effect of this rule in the complete bot.

The pilot still aims and collects gems, but each choice uses less VM time. Stronger survival lets the knight stay near enemies longer. The score measures the whole bot, so it does not show that one shortcut helps every bot.

The current release ladder

The Stage 7 report records these scores from the packaged runner. The run numbers label the columns but do not reveal the seed values. These measurements come from the repository, and this guide did not rerun the exploit artifacts.

Swipe the score table sideways to see all seven runs →

What changed between those ABYSS rows?

Each request has three levels: SERRATE / BASTION / TRAWL. They control attack; survival and movement; and gem supply and collection. A proof step sets the highest allowed level. The bot must request an allowed level for each track and keep the short boost active. The saved reference notes give these requests:

The four stages are shown as cards below →

At every proof step, the game still decides whether hits land and gems are collected. The change from [8,8,8] to [12,12,12] gives much more attack and collection power.

Why one proof step cannot explain the score gap

The scores measure complete bots. The bots differ in build, Gate work, pilot timing, and match behavior. We cannot assign the gap between two rows to one proof step. Other code also changed.

Why the apex kept full TRAWL

The apex builder also tried a [16,16,2] request. Its lower TRAWL level reduced HEAT, but it also gave up gem supply and pickup help. The saved build notes report worse scores on hidden seeds. The builder kept [16,16,16] because the extra collected gems outweighed the extra danger in those runs.

The median scores rise from row to row. Individual run scores do not always rise. For example, Chasm scores below Rift in runs one and four. The milestone compares medians from seven runs, not every run.

Here, the ABYSS median is about 14.81 times the CLASSIC median. This ratio describes these bots and this test set. It does not apply to all ABYSS and CLASSIC bots.

A median selects a run

The grader sorts seven complete results and selects the fourth. It does not average values. It does not find separate medians for EXP, kills, time, and level. The selected result stays with its replay.

When you watch a replay, its EXP, kills, time, and level all come from the same run. The grader does not combine values from different seeds.

Within one grade, the runner sorts by EXP from high to low. It then sorts equal scores by shorter time and smaller seed. The leaderboard also sorts by EXP, but uses earlier creation time for equal scores. Kills do not affect leaderboard rank.

What the frozen artifacts establish

The repository keeps the submitted CLASSIC apex binary, its predecessor, and their hashes. These records are frozen, meaning saved at release and never changed, so their value as evidence is a stable identity and measured behavior. The current frozen median is 22,704. The older score of 224,384 used different rewards, so do not mix it into the current release comparison.

The ABYSS apex has source and a generator, and its README says the unmodified public compiler produces the artifact without patching the compiled binary. This shows that the submitted image can come from the public source language. Even so, the bot still has specialized code and packaging, so it is not a simple beginner example.

The apex has accepted proof on all three Drive tracks and a working pilot. The proof permits strong but limited bonuses, and the pilot turns them into kills and collected gems. Both parts are needed to explain the measured score.

Supply and score are distinct measurements

The ABYSS reference reports median real-gem supply of 362,435 and median EXP of 336,338. These are separate statistics for the seven-run set. Their ratio describes the set. It is not necessarily the pickup rate in the selected median-score run.

The more direct check compares values within each run. The picked-up gem value matches the score, and no pickup counts twice. The value of gems on the ground shows whether the bot needs more kills or better collection.

After the build was saved, practice scores were 349,198; 179,835; 92; 346,924; 316,000; 154,367; and 351,006. Their median was 316,000. The third run scored only 92, although the median remained high.

The report says Stage 6 used the official hidden seeds during tuning. These later practice runs show what happened after that work. They are not an independent test on unseen inputs.

What happens when we block one pilot input?

These tests use the same apex program and proof work. They do not remove code. The host blocks one request type at a time: movement, aim, or movement toward a live gem.

When the host blocks movement, the knight stays in place. When it blocks aim, the old aim stays. In the gem-route test, other moves still work. Gem routing is a move, not a separate host command. The bot can calculate a route even when the host rejects the move.

The gem-route check sits in the normal sys_move code. It runs before the host refreshes the short-lived records of recent movement and gem routing, so a blocked gem-route move renews neither record:

        if self.abyss_on() {
            let pickup_route = self.gems.iter().any(|gem| {
                if !gem.alive {
                    return false;
                }
                let gx = to_int(gem.x) - to_int(hx) - i32::from(dx);
                let gy = to_int(gem.y) - to_int(hy) - i32::from(dy);
                gx.saturating_mul(gx) + gy.saturating_mul(gy) <= 12 * 12
            });
            if self.pilot_pickup_routing_ablated() && pickup_route {
                return packed(hx, hy);
            }
            self.device_move_lease = crate::world::DEVICE_PILOT_LEASE_TICKS;
            if pickup_route {
                self.device_pickup_route_lease = crate::world::DEVICE_PILOT_LEASE_TICKS;

        when the bot requests a move {
            check whether its destination points near a live gem
            if (the move is rejected) { do not refresh its pickup route }
            otherwise {
                refresh movement; refresh pickup routing if near a gem

The host keeps short-lived records of recent movement, aim, and gem routing. The code calls them leases. A blocked request does not renew its lease.

The attack boost needs recent movement and aim. The larger pickup radius and magnet need a recent gem route. Thus, each test changes both the knight's actions and which boosts stay active.

Combat checks for both recent aim and movement. Pickup code checks the recent gem route separately:

        let device_piloted =
            !self.abyss_on() || (self.device_aim_lease > 0 && self.device_move_lease > 0);
        let attack_effects = bugs && device_piloted;
        let disc_mode = attack_effects && self.hero.disc_committed;

        before applying ABYSS attack power {
            require a recent aim request and a recent move request
            if (both are fresh) { allow the extra attack effects }
        }

        let pickup_route_live = !self.abyss_on() || self.device_pickup_route_lease > 0;
        let mut pr = self.hero.pickup_radius;
        if self.lever_effects_on() && pickup_route_live && self.hero.vacuum_radius > 0 {
            pr = pr.max(from_int(self.hero.vacuum_radius.min(self.eff_vacuum_cap())));

        before checking gem pickup {
            require a recent accepted route toward a live gem
            if (TRAWL is active and that route is fresh) {
                widen the pickup radius
            }

What the recorded runs say

Blocking movement or aim separately reduces the frozen median to 91. Blocking gem routing leaves 2,322, and blocking both movement and aim leaves 14,677. All are far below the complete pilot's 336,338.

Blocking both movement and aim scored better than blocking either one alone in these runs. The changed actions also change positions, enemy contacts, spawns, and pickups. The effects of two blocked requests do not equal the sum of their separate score losses.

These tests show that this reference bot needs its pilot. The numbers measure this bot on these runs. They do not give universal values for movement or aim.

Incomplete understanding can look successful

The release also tests six bots with deliberately incomplete proof work. They often beat a basic bot but score below the complete bot at their proof step. A failed proof step does not always crash the bot.

A failed upgrade can leave earlier accepted proof in place. The tests check the public result, saved old proof, and whether the bot can try again. They also check that failure does not grant higher power.

What the timing studies establish

The timing test tried to distinguish two private cases by run time. Random guesses would be correct about 50% of the time. The authors set 55% as the level of concern. Every recorded result stayed below it.

The Stage 7 report cites test version 14. Its highest accuracy on unseen pairs was 52.63671875%. The repository also has a completed version 15 test. It lists thirteen runs, each with 256 warm-up pairs and 2,048 measured pairs. Its highest listed accuracy is about 53.0273%.

These results apply to the tested builds, inputs, machines, and methods. They do not rule out every possible timing test. Saved hashes identify the measured program and test producer.

The Stage 7 report is useful, but the repository has newer timing data than its version-14 result.

The difficulty ratio was not measured

The original requirement sought an ABYSS-to-CLASSIC modeling-time ratio of 10–20×. It called for separate groups that had not seen the challenge. The report says the ratio was not measured and no longer blocks release.

The two-agent playtest lasted 205 and more than 240 minutes, and together the agents generated 2,877 binaries. Their best scores were 21,672 and 3,458, both on CLASSIC, and neither agent entered the ABYSS path. So the playtest shows a discovery barrier, but it cannot measure the requested ratio, which needs one new group per engine.

What the release checks actually show

Release tests check that repeated runs and replays match byte for byte. Other tests check score from pickups, limits on invalid input and full tables, local effects, the pilot, and spectator viewing. Each test supports a specific claim.

The results apply to the tested build and inputs. Rust's forbid(unsafe_code) helps prevent some memory errors. It does not prove that every protocol rule is correct. Matching replays show that those runs were recorded consistently. They do not cover every possible bot.

This distinction makes the evidence more useful: each result states what was tested, what stayed fixed, and which conclusion follows. It lets a later maintainer identify exactly which evidence needs renewal after a code change.