I Made Fortnite Festival's Missing Practice Mode

Disclaimer: this is done for educational and research purposes only. I will not be making the source code public. This post is a rewrite of the one I first published in early January, as of July 2026.

Fortnite Festival has no practice mode. Some charts are long or brutal and there's no way to loop a section without playing the whole thing top to bottom, which every rhythm game I've touched in the past decade lets you do. I've been playing them since I was about 10; drilling a pattern until it clicks is most of the fun. Rust is a language I've spent years in, and this was the right kind of problem for it, something I needed and something that would teach me, so I built the practice mode from scratch.

It didn't stay a looper. Festival's one API hands you song data and nothing else: no score, no live state, not even where you are in the song. So the client reads the game from the outside. It reverse-engineers the scoring off pixels recorded from the screen, and drives an overlay that syncs to the live game by pattern-matching my own playing against the chart. None of it touches the game's code or memory.

Some of it diverges from Festival on purpose, since this was built around how I wanted to practice rather than as a 1:1 clone, and most of those differences are settings you can tune. The client is the heart of it; a launcher wraps it, owning everything outside of playing itself and spawning the client straight into a count-in when you hit practice. That shape is about reuse, not tidiness. I wanted to watch replays of my real in-game sessions, and the client already had everything a replay needs: hit detection, scoring, a highway, a results screen. Capturing what happens in the real game and feeding it through the client I already trust beats building a second scorer that would drift out of agreement with the first. One engine judges everything, whether the notes came from a practice run or from the game itself.

Architecture#

The whole thing is a Rust workspace of three crates. core is the shared library: chart parsing, AES MIDI decryption, the SQLite database and settings. The launcher is a Tauri 2 app with a React frontend that owns everything outside the highway. The practice client is a third crate the launcher spawns as a separate process.

crates/
├── core/       # shared library: chart parse, AES decrypt, SQLite DB, settings
├── launcher/   # Tauri 2 + React UI: catalogue, analysis, sections, replays, overlay
└── practice/   # macroquad + egui gameplay client, spawned per session

The two processes deliberately share almost nothing: a SQLite database in WAL mode, an argv contract, and an exit code. The launcher owns the catalogue, section management and analysis, and spawns the client straight into a count-in when you hit practice. The client keeps its verified timing and scoring stack; the launcher never links against it, so neither can quietly break the other. It's all just a practice client under the hood.

Inside, the client is a game state machine where macroquad handles 3D rendering (the note highway) and egui handles menus and HUD as an immediate-mode GUI overlay; each frame polls input, advances game logic and renders. It does the hit detection, the multi-stem audio, the scoring and the highway, everything a rhythm game needs. What it dropped when it moved under the launcher is song browsing and loading, so the spawned client goes straight into gameplay instead of starting at a browser. Its state machine is short:

enum AppState {
    Practicing(PracticingState),    // live gameplay; a whole song is just a section
    Results(ResultsState),          // final score, accuracy, timing histogram
    WatchingReplay(WatchingReplayState),
}

The old Loading, SongSelect, Playing and Paused states are gone. Live gameplay always enters Practicing, where a full song is simply a section that spans the whole chart; from there it moves to Results, or, when the launcher spawns it with --replay, into WatchingReplay where recorded input events play back against the original chart.

Parsing MIDI Charts#

Each decrypted MIDI file contains instrument-specific tracks at four difficulty tiers where the MIDI note numbers map to fret positions:

Special markers encode gameplay mechanics beyond the basic fret notes: MIDI note 116 defines overdrive phrase boundaries where note-on starts a phrase and note-off ends it, base+6 through base+10 are lift notes and notes occurring on the same MIDI tick are detected as chords. Each parsed note is stored as a struct carrying its tick position, absolute time in milliseconds, fret number (0-4), sustain duration in ticks, milliseconds and beats, along with a NoteFlags bitfield using the bitflags crate for efficient per-note metadata:

pub struct Note {
    tick: u32,
    time_ms: f64,
    fret: u8,              // 0-4 (green through orange)
    sustain_ticks: u32,
    sustain_ms: f64,
    sustain_beats: f32,    // for display scaling
    flags: NoteFlags,      // TAP | OVERDRIVE | LIFT | CHORD
}

The same MIDI carries a SECTION track of named landmarks (intro, verse, chorus and so on); the launcher reads those as the anchors you build practice sections around.

The Clock#

The client's audio position originally came straight from the audio engine, and that was the wrong source of truth. The engine reports position in ~45ms steps (one audio buffer at a time), and the first release smoothed over the steps with interpolate-then-snap logic that traded stepping for wobble. In a game where the perfect window is ±25ms, a clock that quantises at 45ms is disqualified before it starts.

The fix inverts the relationship. A wall-anchored timer is the position: anchor the moment playback starts, advance on the monotonic clock, re-anchor on every seek, pause and speed change. The engine's reported position is only consulted as a drift check, through one small pure function that takes the predicted position and the raw report and returns a correction only when they disagree by more than 150ms. When I finally probed the engine headless for a full song, it tracked the wall clock within ±7ms over 412 seconds with all five stems in lockstep, which means the safety net has likely never fired in anger; the stepping, not the engine's accuracy, was the problem. At practice speeds the anchor advances at the playback rate, so slowing a section to 0.5x slows time itself: the highway, the count-in and the hit windows all stay glued to the audio for free, and because hit windows are defined in chart time, half speed also means double the real-time margin for your hands. That is the entire point of a practice speed.

Replay System#

The replay system in game/replay.rs records every input event during gameplay - fret presses, fret releases, overdrive activations - with their timestamps relative to the chart start. After a session it saves to the shared SQLite database as a single row: the run's summary (score, max combo, accuracy, per-rating hit counts) alongside the whole event stream, kept as a JSON array in one column and validated on write. Practice runs, a single section or a whole song, and live Fortnite sessions are all captured this way, and a run you quit halfway is kept too, stored up to the point you stopped and replaying exactly that far.

Playback feeds the recorded events back into the gameplay engine against the original chart, meaning the same hit detection, scoring and visual feedback run as they did during the original session. Seeking within a replay works by replaying all recorded events from the start up to the target timestamp to reconstruct the full game state (combo, score, overdrive meter, sustain states) at that point, which is straightforward since the event list is in chronological order.

Practice Mode#

Sections are created against a picture of the song rather than typed as timestamps. The launcher renders the whole chart as a notes-per-second heat strip, resampling the cached 500ms NPS buckets into a fixed number of columns with area weighting so long songs don't alias, and draws the chart's own section names underneath as landmarks. You drag across the strip and the create form prefills with the range; clicking a section label snaps to that section exactly and takes its name. Saved sections live in the shared database, and each row has its own play button. Launching one forwards the range on the command line as --range-start-ms and --range-end-ms, which the client turns into a synthetic section through the same code path a chart section uses, so the count-in, looping and auto-restart behave identically whether the section came from the chart or from a drag.

On the client side, the practice engine in game/practice.rs runs the loop. A configurable count-in plays the audio with no notes for a number of beats (default 4) before the section starts, giving you time to orient yourself before notes appear; auto-restart watches the miss count during a section and loops back to the start, count-in and all, once it crosses the threshold. Speed presets scale the same wall-anchored clock, so slowing a section slows the audio, the highway, the count-in and the hit windows together and nothing drifts out of line at any rate.

Audio comes in by dropping the source .mp4 on the window. Festival's files carry ten mono tracks in fixed pairs (drums, bass, lead, vocals, backing), so the importer probes the stream layout with ffprobe and merges each pair to a stereo .wav with an ffmpeg amerge filter, one stem at a time with per-stem progress in the UI. A file without that layout degrades to a single backing track rather than failing, and a half-failed extraction cleans up after itself so the gate never reports a confusing partial state.

Festival's Scoring, Reverse-Engineered#

The engine agrees with the game to within 25 points on a 335,000-point run. Every constant in it traces to a frame of recorded gameplay. The rules as they actually are: 30 points base per note, a perfect (±25ms) multiplies by 1.2 to 36, a good (±100ms) keeps the full base. The combo multiplier is delayed rather than immediate, stepping up every ten notes with the eleventh note as the first 2x, and it caps at 4x for lead and drums but 6x for bass and vocals. Sustains shorter than one beat pay nothing. The ones that qualify pay 12 points per beat, scaled by the combo multiplier. Overdrive doubles whatever is true at the moment, and an active overdrive window extends when another phrase completes inside it.

Reading the Game Off the Screen#

Festival won't hand you a score, and nothing in this project reads game memory or digs through the game's code; the approach everywhere is to observe from the outside. That turns out to be enough here, because the game publishes every number that matters, sixty times a second, in the corner of the screen. One run recorded at 60fps, the score counter cropped from every frame, binarised, and OCR'd: 18,671 frames. The counter snaps instantly on every note, and a cumulative, monotonic counter sampled at 60Hz is not a picture of the score. It is a ledger of every note's exact value.

On a full combo with no overdrive the decode is total. Combo equals note index, so the multiplier schedule is fully determined, and every increment factors uniquely as a perfect or a good at a known multiplier: per-note verdicts, extracted from pixels. Everything surprising came out of that one recording. The results screen number is a count-up animation, which matters if a screenshot catches it mid-roll and the game appears to disagree with itself by six and a half thousand points. Sustains under a beat pay nothing: an entire section of quarter-beat trails paid a flat +144 per note, no trickle. Qualifying sustains are multiplied: one isolated 1.5-beat hold, sitting before an eleven-second break with nothing else to contaminate the reading, paid exactly 72 points, which is 18 for the sustain times the 4x multiplier that supposedly never applies. A second recording with overdrive settled the window-extension rule the same way, from the lengths of the doubled regions alone.

The Unreachable Last Points#

The last few hundred points are not reachable, and the reason is more interesting than a bug. The design window for a perfect is ±25ms, but the game judges a keypress on its own engine tick, Unreal sampling input on its schedule, while the capture timestamps the same keypress on a wall clock. Measured from the outside, the effective boundary between the two clocks landed at 27.0ms in one session and 25.5ms in another, with mixed early/late signs that rule out a constant offset. A dense chart stacks enough notes right on that boundary that the wobble is worth a few hundred points on any fixed window. The engine uses the midpoint, 26.25ms, and stops there, because the residual is not an error in either scorer. It is two clocks watching one finger and disagreeing about the edge.

The verification loop is unchanged: play a session in the real game, capture every input with timestamps, score it through the one engine, and compare against what the game displayed, note by note when a recording exists. The game tells you everything, if you watch closely enough.

Linking Into Fortnite#

The companion side of the launcher needs to know what's happening inside a running song. Festival's API only lists songs; it says nothing about a live one. It turns out Fortnite tells you anyway, if you listen in the right places.

Tailing the Log#

The first place is the log file. Festival's internal codename is Pilgrim, and the game writes LogPilgrim* lines for every meaningful transition. Tail the log, run each line through a handful of regexes, and you get a clean event stream:

pub enum LogEvent {
    /// "Song data set. N gems found for …Difficulty… …TrackType…" — the song
    /// started on this instrument and difficulty.
    SongStarted { instrument: Instrument, difficulty: String },
    /// "local client finished loading song <id>" — the song id is now known.
    SongLoaded { song_id: String },
    /// Quickplay reached the intro state; this is what arms the matcher.
    SongIntro,
    EnteredBackstage,
    LeftBackstage,
    StoppingSong,
    // ...
}
 
static DIFFICULTY_REGEX: Lazy<Regex> = Lazy::new(|| {
    Regex::new(r"LogPilgrimGameEvaluator: \[....\] : Song data set\. [0-9]* gems found for ").unwrap()
});

That stream tells you which song started and on which instrument, but not where the player is inside it. The song-started line is honest about the fact and useless as a clock: it fires around the start, not at it, and in multiplayer the gap varies with loading and connection wait times, so anchoring position to the log timestamp would be wrong by a different amount every session. Fortnite exposes no playback position at all, and that's the interesting problem: the overlay draws a difficulty graph with a sweep line that has to track the song in real time, with nothing trustworthy to anchor it to. So the log event arms the matcher and nothing more.

The Player Is the Clock#

The answer is that the player is the clock. Fret presses are captured with GetAsyncKeyState polling on a dedicated thread, timestamped on the monotonic clock into a small ring buffer. The choice of polling over a keyboard hook is deliberate and it's about latency: a low-level hook (WH_KEYBOARD_LL) inserts itself into the input delivery chain, so every keystroke in the system synchronously waits on your callback before the game sees it, and a slow moment in your process becomes input lag in theirs. Polling never touches that chain. It reads the key state table Windows already maintains, the game's input path stays exactly as long as it was, and the worst the companion can do to your timing is nothing at all. The same poll reads the fret bindings from the shared settings, so it always watches the keys you actually play on. The last thing you want from software running beside a rhythm game is added latency, and the honest way to guarantee that is to never stand in the input path in the first place. When the log shows the song loading in, before the first note, the matcher arms and listens.

The chart's opening is reduced to onset groups: clusters of note times with the lanes they occupy. Incoming presses get clustered the same way, and the matcher tries to align the two sequences. It takes a window of the first few onset groups, derives the offset that would map a candidate press cluster onto the first onset, and checks whether the following clusters land within tolerance of the following onsets on compatible lanes. If the player flubbed a note or tapped a fret while waiting, a skip parameter slides the window forward so a pre-tap or a missed opening note can't poison the lock. Three distinct groups aligning within 120ms each (half an NPS bucket) commits the match, and the offset becomes the anchor: song position is just the monotonic clock minus that offset from then on.

/// Per-onset residual tolerance accepted for the opening alignment.
const MATCH_TOL_MS: f64 = 120.0;
/// Distinct onset groups that must align before we commit a lock.
const MATCH_GROUPS: usize = 3;
 
pub enum Lifecycle {
    Disarmed,
    /// Capturing opening presses, no anchor yet ("syncing...").
    Listening,
    /// Pattern-locked; position runs forward on the wall clock.
    Locked,
    /// No clean lock; wall-clock estimate from song start.
    Degraded,
}

Once locked, song position zero is pinned to a monotonic clock value and everything runs forward from there and never needs another press. If ten seconds pass without a clean lock (the player flubbed the intro), it degrades to a rough wall-clock estimate instead of showing nothing. You sync the overlay to the game by playing the game; the opening notes are the handshake.

Drawing the Overlay#

The overlay itself is a native Win32 layered window (per-pixel alpha, clickthrough, always-on-top) drawn with GDI rather than a webview, because it has to coexist with a fullscreen game without stealing input or a compositor's worth of resources. It sits in the corner and updates in real time off nothing but the lock.

What it draws is the chart as a difficulty graph: the whole song's notes-per-second curve with a sweep line riding across it at the locked position, so the dense stretch is visible before it arrives, and a live score checked against my previous scores on the chart so a run has something to beat.

Live Overdrive-Path Feedback#

The overlay's other half sits on top of the highway itself, pinned wherever I want it: a path box listing the overdrive activations CHOpt worked out for the chart, with the one you are on marked. That is the notation stacked over the game in the clip above. Making that mark track your run on its own is the hard bit, because Fortnite never tells the overlay where you are in the path any more than it tells it the time. It infers that too. A scoring pass runs over the run so far and counts the double-score windows you have actually opened, not the ones you whiffed, and that count is the index into the path list. The next-activation marker moves because the game moved, not because a timer said it should.

Overdrive activation is detected the same way position is: your own key. Hitting the bound overdrive key records an activation into the capture and steps the highlight forward. Whether that press did anything is decided afterward, by scoring it against the star power you had banked at that moment. Fire with an empty bank and you open no window, and the box flashes for a couple of seconds to tell you the activation was wasted. Underneath, a status line reads the windows you have opened against how many the path expects by now, so you can see at a glance whether you are on plan, ahead of it, or behind. There is no per-activation callout for a mistimed phrase; a skipped or early activation just shows up as running behind, and as the live score slipping under the leaderboard mark.

The count can drift. It rides on the lock holding, the clock staying aligned to the game, the scorer matching every activation to a window, and a 5ms input poll that can miss a fast double-tap, and any of those can leave it a window ahead or behind where you really are in the printed path. So two keys nudge the highlight by one in either direction, and a third freezes the auto-tracking once it is right, so a good manual correction is not overwritten on the next pass.

The scoring pass runs slow on purpose: it re-reads the run about once a second, so the leaderboard score trails Fortnite's instant counter by up to a beat. Recomputing the whole run continuously would put real work right next to the input capture, and the companion's one rule, the same reason it polls the keyboard instead of hooking it, is to never cost the game a millisecond. Running it light and a little late keeps the overlay cheap and off the input path.

Replay Analysis#

A replay is a real run fed back through the same engine, so the client can show you the run the way the game never does. The replay viewer has an analysis mode. Turn it on and every note wears its timing from the moment it spawns rather than the moment you would have hit it: gold when the press landed inside the perfect window, blue when it was early, pink when it was late, a dark X where the note was missed. A whole run's timing reads at a glance, so a section that is quietly rushing shows up as a wall of blue long before it ever costs you a note. It is colour on the highway, not numbers; the millisecond spread shows in a corner histogram, and the full per-note breakdown lives in the headless timing tools.

The clip below is the same Beyond the Flame run, played back in the client with the original Fortnite capture pinned in the corner. Both are driven by one chart time, so the client's highway and the game's own footage stay locked frame for frame. That is the end-to-end check: if the replay, the timing and the scoring agree with what actually happened, they agree with the video too.

Patching CHOpt for Lift Notes#

Path images come from CHOpt, a Star Power path optimiser made for Rock Band and the like with Fortnite Festival support. The launcher's relationship with it is stranger than it should be. It ships two builds side by side: stock CHOpt, and a fork with Festival's scoring model patched in. The counterintuitive part is which binary does what. The fork runs with --engine rb and produces the path text; stock runs with --engine fnf and produces the score and the image. It reads exactly backwards, and it's the empirically proven recipe: the fork's scoring model makes the right activation decisions but doesn't total correctly, stock totals correctly but paths worse, and no single binary gets both right. Path from the fork, score from standard. Every song runs through both, and the launcher stitches the results into one answer.

The image half of that arrangement is where lifts come in. Stock CHOpt supports Festival but didn't know lift notes existed: its parser's note range stopped short of the lift markers, so they were dropped before the renderer ever saw them. In the MIDI, lifts live at fixed offsets above each difficulty's base note (keys 102-106 on Expert, one per lane), with the wrinkle that the green-lane lift shares a key with a force marker, and in Festival charts the lift reading wins.

The patch is three small layers on a branch, and the first layer isn't in CHOpt at all. Chart parsing lives in SightRead, a separate library CHOpt pins as a submodule: SightRead reads the MIDI into the note stream CHOpt paths and draws over, so a note class it never emits is one CHOpt can never see. The lift flag has to be born there: a new bit on the note-flags enum, a collection pass that gathers lift note-ons per difficulty, and an application pass that marks the note sharing each lift's tick and lane. All of it runs only for the Fortnite track type, and it's purely additive; it marks existing notes and never creates or moves one, so the optimiser's input is untouched and paths and scores stay byte-identical. Rather than fork the parser library for a flag and two passes, the branch vendors it: the submodule becomes plain committed files with the patch applied, which keeps the whole change self-contained in one branch that CI can build without any upstream involvement.

In SightRead, the new flag sits alongside the existing note flags:

enum NoteFlags : std::uint32_t {
    // ...
    FLAGS_FORCE_STRUM = 1U << 7,
    FLAGS_LIFT        = 1U << 8,
    // ...
};

The collection pass walks the MIDI track once and records where lifts occur. Each difficulty's five lift keys are base + 6 through base + 10 in green-to-orange order (Expert base 96, so keys 102 to 106), so the lane is the key minus that base, and a velocity-zero note-on (a note-off in disguise) is skipped:

std::map<Difficulty, std::vector<std::tuple<int, int>>>
fortnite_lifts_from_midi(const MidiTrack& midi_track)
{
    constexpr int LIFT_GREEN_OFFSET = 6;
    constexpr int LIFT_LANE_COUNT = 5;
    constexpr std::array<std::tuple<int, Difficulty>, 4> LIFT_DIFFS {
        {{96, Difficulty::Expert}, {84, Difficulty::Hard},
         {72, Difficulty::Medium}, {60, Difficulty::Easy}}};
 
    std::map<Difficulty, std::vector<std::tuple<int, int>>> lifts;
    for (const auto& event : midi_track.events) {
        const auto* midi_event = std::get_if<MidiEvent>(&event.event);
        if (midi_event == nullptr) continue;
        if ((midi_event->status & 0xF0) != 0x90) continue;  // note-on only
        if (midi_event->data[1] == 0) continue;             // velocity 0 = note-off
        const int key = midi_event->data[0];
        for (const auto& [base, diff] : LIFT_DIFFS) {
            const int lane = key - (base + LIFT_GREEN_OFFSET);
            if (lane >= 0 && lane < LIFT_LANE_COUNT) {
                lifts[diff].emplace_back(event.time, lane);
                break;
            }
        }
    }
    return lifts;
}

The application pass sets the flag on any real note that shares a lift's tick and lane, and touches nothing else. That last part is the whole safety argument: positions, lengths, counts and every other flag stay exactly as they were, so the optimiser is handed an identical chart:

void apply_fortnite_lifts(std::map<Difficulty, std::vector<Note>>& notes,
                          const std::map<Difficulty,
                              std::vector<std::tuple<int, int>>>& lifts)
{
    for (auto& [diff, note_set] : notes) {
        const auto lift_iter = lifts.find(diff);
        if (lift_iter == lifts.cend()) continue;
        for (auto& note : note_set) {
            for (const auto& [lift_tick, lift_lane] : lift_iter->second) {
                if (note.position == Tick {lift_tick}
                    && note.lengths.at(lift_lane) != Tick {-1}) {
                    note.flags = static_cast<NoteFlags>(note.flags | FLAGS_LIFT);
                    break;
                }
            }
        }
    }
}

Both hang off a single line in the Fortnite note builder, run right after the normal note pass:

auto notes = notes_from_event_track(event_track, {}, TrackType::FortniteFestival);
apply_fortnite_lifts(notes, fortnite_lifts_from_midi(midi_track));

That is the entire SightRead layer. From there CHOpt has to carry the flag from a parsed note to a drawn one. When the image builder converts a note it fills a per-lane lift mask on DrawnNote, set only for the note's active lanes and only when the flag is present:

std::array<bool, COLOURS_SIZE> lift_lanes {};
if ((note.flags & FLAGS_LIFT) != 0U) {
    for (auto i = 0; i < COLOURS_SIZE; ++i) {
        lift_lanes.at(i) = note.lengths.at(i) != Tick {-1};
    }
}

The renderer reads that mask back out as a bitfield and decides what to draw. A note whose every active lane is a lift is drawn as triangles only, so its sprite is suppressed; a lift lane sitting over an otherwise normal note gets a triangle drawn on top of the usual sprite. The triangle is a right-pointing wedge in the lane's colour with a 1px black outline:

constexpr int HALF_WIDTH = 4;
constexpr int HALF_HEIGHT = 5;
 
const auto apex_x = x + HALF_WIDTH;
const auto base_x = x - HALF_WIDTH;
 
m_image.draw_triangle(apex_x, cy, base_x, cy - HALF_HEIGHT,
                      base_x, cy + HALF_HEIGHT, colour, 1.0F);

Regression on both engines confirmed non-Festival output byte-identical and Festival paths unchanged. The only difference is that release notes finally look like release notes.

Configuration#

Keybinds and the client's knobs live in the shared core settings, so the launcher's keybind UI and the practice client read the same file: rebind once and both obey. The launcher's settings dialog writes it, validating and canonicalising every key so a binding the client can't understand never reaches the client; the client loads it at launch. The knobs cover master volume (0.0-1.0), a global audio offset in milliseconds for system-level latency calibration, track speed, the Good hit window (the Perfect window is a fixed ±25ms and is never resized), lane colour preset, highway render distance, note size and sustain tail width, overdrive colour, and toggles for the timing histogram and the NPS graph.

Per-chart audio offsets are stored in {data_dir}/offsets/{shortname}.txt so songs that need individual calibration (due to audio encoding differences or stem sync issues) don't pollute the global offset value. The rest of the state lives in the shared database and a handful of working directories:

%APPDATA%/[client-name]/
├── settings.json          # shared settings and keybinds
├── [client-name].db       # shared SQLite (WAL): catalogue, sections, plays, replays, paths
├── offsets/               # per-chart audio offset calibration
├── cache/<song>/stems/    # extracted stems
├── midi/                  # decrypted MIDI, used as CHOpt input
├── chopt_images/          # rendered path images
└── assets/                # album art and instrument icons

Everything runs from local storage after initial setup, and both binaries stay light. I won't be making the client public.