Checking and editing the detection of notes within polyphonic material

In this tour, you will learn how to check the detection of notes in polyphonic material and how to reassign notes.

The fact that this is virtually always necessary where the material contains pronounced overtones is inherent in its nature: when searching for notes in polyphonic material, multiple interpretations of the frequency spectra encountered are always going to be possible and, with them, different ways of carving the spectra up into notes. So Melodyne cannot know with certainty whether the energy in a particular frequency range indicates the presence of a new note or some overtones of a lower one. It can and does happen, then, that prominent overtones are sometimes mistaken for fundamentals or that notes actually played are not shown because they have been assigned to other notes as overtones.

So the analysis yielded by the detection offers you the most plausible interpretation of the material, but it will generally be necessary to check through this and weed out superfluous notes (by deactivating them) as well as supply missing ones, by activating ‘potential’ notes that are currently inactive. As will become clear, it is very easy to perform the requisite reassignments and we urge you to do so before you begin editing your material.

There’s a good reason for this: Only if the notes displayed correspond exactly to the notes played will any sensible and artifact-free editing of the material be possible. If the notes displayed do not correspond to those actually played, you may find yourself editing what are merely spectral fragments in the mistaken belief that they are whole notes; or editing what you think is one note when in fact it is two, because the higher note has been taken for an overtone of the lower. So take the time to check through the note assignments when the detection is complete. Otherwise Melodyne will be unable to offer you the full benefit of its unique editing capabilities.

Activating and deactivating notes

After transferring or loading your audio, select the ± tool to switch to note assignment mode.

In the stand-alone implementation, note assignment mode only ever applies to one of the audio files in your project at a time; in the Melodyne plug-in, it applies to a single transfer. This means that in note assignment mode you cannot always see all the notes belonging to a given track simultaneously; only those derived from a single file or transfer. Click therefore on a note belonging to the first file or transfer you wish to work on, and then switch to note assignment mode to see the notes in question. When you have finished correcting the note assignments within this file or transfer, quit note assignment mode before selecting a note belonging to the next file or transfer you wish to work on and repeating the process. In this way, you can edit the note assignments of each file or transfer in turn.

Please bear in mind that for technical reasons, switching to this mode causes the undo history to be deleted, so any actions performed prior to the change of mode can no longer be reversed. If you select the ± tool whilst editing polyphonic material, additional control elements will appear beneath the toolbar.

The color of the editing background changes to remind you that in note assignment mode no ‘audible’ editing is possible. This mode is used to check Melodyne’s interpretation of the audio material and correct it where necessary. Notes that have been ‘swallowed’ (where a fundamental has been mistaken for an overtone) can be activated, which makes it possible later to edit them. Conversely, overtones that have been mistaken for fundamentals can be deactivated.

In this mode, the outline of active blobs is filled in (i.e. they are solid) whereas, with inactive blobs, only the hollow outline is seen.

When you click on a blob, you will hear the pitch of the corresponding note. Where a solid blob has been assigned to what is, in fact, merely one of the overtones of some other note, you can deactivate it by double-clicking on it. Now only the hollow outline of the blob will be seen and its energy in the frequency spectrum will be attributed to the note of which it can most plausibly be assumed to be an overtone.

In the example above, you can see that, in deactivating the higher A, we have caused the lower A to be redrawn more boldly: this is because the spectral energy previously ascribed to the higher note (when it was assumed to be a separate note) has been reassigned to the lower one (of which it is now considered to be an overtone).

Conversely, by double-clicking on a hollow blob, you can turn a potential note currently interpreted by Melodyne as an overtone into an active one. Only active notes can be edited later using the tools in the Melodyne toolbar, which is why all the notes played and only those notes should be represented by solid blobs. Otherwise you may find yourself editing not whole notes with their full overtone spectra but stray overtones divorced from the fundamentals to which they belong, which will yield poor results acoustically.

Now that you know how to activate and deactivate blobs, you have mastered the basics of note assignment in Melodyne. The more complex the overtone structure of the audio material, the more open it is to different interpretations when it comes to note assignment, and therefore the more work you will have to do to correct manually the inevitable errors of interpretation. Melodyne does offer you, however, a number of aids to ensure the procedure is as swift and effortless as possible.

The monitoring synthesizer

A considerable aid when checking and correcting note assignments is accessed by clicking the sine wave icon, which you will see beneath the toolbar. This is both a switch and a rotary control. When activated, it causes the sound of a synthesizer to replace the normal sound of each blob. To control the volume, click on the icon and drag the mouse pointer to the left or right.

This synthesizer replaces normal playback of the original recording when Melodyne is in note assignment mode. This allows you to hear the notes that are currently ‘active’ – i.e. represented by solid blobs.

Think of the solid blobs as representing a transcription of the music in the audio file. The synthesizer allows you now to check this transcription undistracted by the original sound. With it, you can very quickly determine whether all the notes actually played have been identified correctly as fundamentals as well as weed out notes that were never played – i.e. overtones that have been mistaken for fundamentals.

You can carry on activating and deactivating blobs even during playback with the synthesizer; this allows you to hear what you are doing and arrive more swiftly at the ideal situation, in which the solid blobs displayed represent all, and only, the notes actually played.

Note separation

When you place the mouse pointer just above a blob, the arrow turns into the note separation tool. With this, as with the normal note separation tool, you can split or join notes by introducing or removing note separations.

The note assignment slider

The double slider that appears beneath the toolbox in note assignment mode allows you to control the number of potential notes displayed and the number of active notes derived from them.

If you move the large right bracket (or ‘crescent’) in the slider to the left, fewer potential notes will be displayed. If you drag it to the right, more potential notes will appear. Choose a setting that ensures that only as many potential notes are displayed as you may conceivably wish to activate in the course of the subsequent editing. That will give you a clearer overview.

Now drag the orange knob on the slider (the ‘orange’) to the left and right. As you drag it to the left, you reduce the probability of the potential notes displayed becoming active notes, thereby reducing the number of active notes. As you drag it to the right, you increase that probability, thereby creating more active notes from the potential notes displayed.

There can never be more active than potential notes, so the orange can never pass through the crescent but merely pushes it to the right when it wants to go further, thereby causing additional potential notes to be displayed and activated simultaneously. Adjust the two sliders until the number of active notes displayed is as close as you can get to the number of notes that were actually played. Then proceed to the manual correction of individual notes.

Now and then, it can happen that a note that can be heard in the material is not detected as an active note, and, even with the right-hand bracket slider (the crescent) at its maximum setting, is not shown as a potential tone. If that happens, move the crescent fully to the right (to its maximum setting) and then move the mouse pointer over the position in the editing window where the missing tone ought to be. Around the mouse pointer, in the form of an “energy image”, notes will now appear that were detected neither as active nor as potential notes. When you have identified the missing note in this way, double-click on it to transform it into an active note. Thereafter, by subsequent double-clicking, you can toggle the status of these notes between “potential” and “active” just like that of any others.

The venetian blinds

With instruments in particular that generate powerful overtones, it can happen that over a wide range notes are detected that you perfectly well know are far higher (or lower) than any you actually played.

In such cases, the ‘venetian blinds’ come in handy; if you can’t see them at the top or bottom of the editing area, scroll upwards or downwards until you can. You can raise or lower the top blind by dragging its thick bottom edge and do the same with the top edge of the bottom blind, in this way delimiting the range within which Melodyne assigns notes. All notes partially concealed by the venetian blinds are automatically deactivated unless they have previously been activated by hand. You can still ‘reach through’ the venetian blinds, however, to turn notes on or off with a mouse click.

The venetian blinds also provide a useful first approximation that you can later correct by activating and deactivating notes singly by hand.

Pulling open note ends

It may happen that in the detection note separations are placed in such a way that the beginning or ending of a note is ‘swallowed’ – i.e. annexed by an adjacent note. In such cases, you can pull notes open, by dragging their front boundaries gently leftward or their rear boundaries to the right. (In order to gain access to the note separations, you must check the option Show Note Separations in the View menu. Position the tool over the front or back note separation line and drag it horizontally.

Be careful: if a deactivated (hollow) note borders on the edited note, you must first activate this before you can move the note borders.

Leaving note assignment mode

To quit note assignment mode simply select one of the other tools. As you leave the mode, Melodyne will perform a fresh analysis based on the changes you have made. This could take a few moments.