How Audio to MIDI Transcription Actually Works

A MIDI file doesn't contain any sound at all — no waveform, no samples, nothing you could play back on its own. It's a list of instructions: which note, how loud, when it starts, and how long it lasts. Converting audio to MIDI means pulling that note-and-timing information back out of a finished recording that never had it attached in the first place, which is a fundamentally different, much harder problem than converting between two audio formats.

What audio-to-MIDI transcription actually does

An audio file is a waveform — a record of air pressure over time. A MIDI file is closer to sheet music: a sequence of note-on and note-off events, each with a pitch, a velocity, and a timestamp. Audio-to-MIDI transcription analyzes a waveform and detects what that underlying note sequence probably was — reconstructing the score from the performance, in effect, rather than reading the score directly.

How a note gets detected

The detector scans the audio for two related signals: an onset — a sudden rise in energy at a particular pitch, which usually marks the start of a note — and a sustained frameof energy at that same pitch, which marks the note continuing to ring out. Two threshold settings control how sensitive each check is. A lower onset threshold catches quieter or more subtle note starts, but also picks up more false positives from noise or bleed. A lower frame threshold holds notes open longer and catches quieter sustain, at the same trade-off. There's no single correct setting — it's a real trade-off between missing genuine notes and registering ones that were never played, and the right balance depends on the source recording.

Why a single melody transcribes cleaner than a full mix

Detecting "when did a note start and at what pitch" is a much easier job when there's one clear pitch to track at a time. A solo vocal line, a single instrument, or an isolated melody gives the detector an unambiguous signal. A dense mix with multiple instruments overlapping in the same frequency range, or a chord where several notes ring simultaneously, means disentangling overlapping energy at once — a genuinely harder detection problem, not just a matter of turning the sensitivity up. This is also why a stem-separated track (an isolated vocal or bass line, for example) transcribes far more reliably than the same part still buried in a full mix — thinning out the signal before transcription starts makes a real difference.

What the minimum and maximum frequency filters are for

These two settings simply tell the detector to ignore anything outside a pitch range before it starts listening. Setting a minimum frequency filters out low rumble or bleed from a bass or kick drum that isn't the part you're trying to capture. Setting a maximum frequency filters out hiss, cymbals, or high-frequency noise. Narrowing the range to just where your actual melody sits reduces false positives from everything outside it, without touching the onset or frame sensitivity at all.

What a minimum note length filters out

Short spurious blips — a transient click, a bit of noise that briefly crosses the onset threshold — can register as extremely short "notes" that were never actually played. Raising the minimum note length discards anything shorter than that duration, which cleans up a lot of stray notes at the cost of also discarding any genuinely fast, short notes in the source performance. Fast melodic runs need a shorter minimum than a slow, sustained vocal line.

What a transcribed MIDI file is actually useful for

A resulting MIDI file isn't a substitute for the original recording — it's a starting point. Producers use it to pull a melody or bassline out of a reference track and reassign it to a different instrument or synth patch in a DAW. It works as a rough first pass for building sheet music or a lead sheet in notation software, saving the slower work of transcribing by ear from scratch. And it lets you study or rework a melodic idea's timing and pitch independent of the original performance's tone or production.

Where it still struggles

Dense chords, fast polyphonic passages, and full-band mixes remain genuinely hard for any automatic transcription approach, not just this one — that's an open problem in audio processing generally, not a limitation specific to one implementation. Drums and unpitched percussion don't transcribe meaningfully at all, since the whole approach is built around tracking pitch over time. The most reliable results come from a single clear instrument or vocal melody with minimal competing sound in the same frequency range.

How AudioForges uses these settings

AudioForges exposes onset sensitivity, frame sensitivity, minimum note length, and frequency range directly, plus a set of presets — vocal, piano, bass, guitar, and fast passages — that start from a sensible combination of these for common sources. Pick the closest preset, or adjust the controls above manually if the default result isn't quite right. For a practical example, open the Audio to MIDI Converter and expand Advanced Settings.