Editing polyphonic NOTES within a WAV FILE! (Melodyne 2)

[Cheez]

For those who has been following Musikmesse 08, you've probably already heard of this one - Melodyne version 2.

For those who heard of Melodyne for the first time, it's an innovative software that lets you edit recorded tracks like how you would edit midi. Except that the editing source is a wav file, not a midi file.

Version 2 is REALLY interesting - Direct Note Access lets you edit INDIVIDUAL notes within a polyphonic wav file. So if a guitarist or keyboardist plays a wrong note and is recorded in a track, you can actually edit that one note even if it is played in a chord. And you can change notes/chords to make it sound totally different (major key to minor key etc); you can time stretch individual notes etc. Remember this is editing a track, not midi. And it's editing individual notes within a recorded track! Amazing technology. See the video to believe it!

celemony_ :: Direct Note Access

Definitely a must for the recording/producer's arsenal of tools!

 

[mingguan]

Looks good.
Eh...
what is "polyphonic Notes"?

But, single track editor (uno) already US$199/=.

 

[Cheez]

That means if I play a chord C3, E3, G3, B3, D4, E4, G4 on a piano for example and record that, Melodyne can split that wav file into the individual notes. Then I can change any of the notes by just dragging it (eg C3 to D3 etc), change the duration of each note etc etc.

Imagine an entire audio track of an instrument played not with single notes but polyphonic notes (many notes sounding together at the same time) and the ability to alter any note within that audio track like you would with a midi file!!! Revolutionary!

The plugin version is probably the most versatile as it would sit in any DAW software.

 

[soft]

The technology is amazing. How are they able to identify the notes so accurately?

 

[since1982]

wow interesting.. seriously i thought .wav files are final and unamendable..

 

[MadWerewolfBoy]

Whoa, I always knew there're ways to change the pitch of a note in a single-line solo, but changing a note in chords is just out of this world!

 

[dirtchamber700]

"The first Melodyne product with Direct Note Access will be Version 2 of Melodyne plugin which will come onto the market in the Fall 2008."

I'm not too familiar with seasons, so when is this available?

Do think this is possible with distorted sounds ie. distorted guitars etc?

 

[godchuanz]

Seriously, this is over-hyped. It's still a good piece of software, but don't expect it to really do what it claims to do.

Source separation is considered, until today, an unsolved problem. If they have really solved this problem, there would be a huge uproar in the audio signal processing community.

Their onset detection algorithm for note segmentation is not that great too. It works well for really percussive notes, but fails terribly in the case of the singing voice.

 

[Cheez]

But there is an uproar currently re: Melodyne 2! I agree. For voice/singing, it may not work as well as for, say, guitar and piano. There may problem also for string and wind instruments. But at least they have broken the barrier. It can only get better from here on. And the test of the cake is in the pudding. Whether it is over-hyped or not, we cannot judge too early until the software is out.

 

[godchuanz]

What I mean is... before a software is made, they surely have algorithms developed for these functions. In the scientific research circles, source separation is an unsolved problem. Universities are still spending millions and billions of dollars in research. If Celemony scientists/engineers have somehow developed algorithms to solve this problem, there would surely have been some news of it.

The recent uproar (hype) is among the consumers, not among the scientists. If they have indeed solved this problem, I'm sure they'll have some sort of patents/publications on the technology. And major universities and scientist groups will gladly pay billions of dollars just to get a whiff of the algorithms in use. But of course, we know that this is not the case.

Probably, they just used one of the published state-of-the-art algorithms (which performs quite poorly, by the way). You can easily find such algorithms on IEEE and possibly from proceedings of many other signal processing conferences.

Last time, when I heard about how you can pitch-correct each note, I was also very excited. But the note segmentation really let them down, which is expected. Note onset detection in non-percussive signals is also an unsolved problem.

 

[gutturalpiss]

Absolutely, and that is why we cannot and should not say "this is it!"

 

[stunew78]

I've been engineering for 15 years and it is may day in day out job.
There is no 'problem', the nature of what we work with in mixed audio signals is complete and unchangeable. Complex polyphonic waveforms will never be able to be unbundled back to multitrack components.
It will never be able to happen. Never.

The only feasable approximation offers only that, and no solution; which is to intelligibly approximate what would be the closest assumption of the original multi-tracked component within the mix file.

I guarantee not seeing any computer in my lifetime ever being able to work this out without human input.

 

[godchuanz]

Hi stunew78,

Don't say that

Many people will be out of jobs if it was unanimously decided that this problem is unsolvable.
Yes, ultimately, the best source separation we can achieve will only still be an approximation. But maybe 10 years into the future, the approximation is good enough for studio use?

Let's remember that digital waveforms are also "approximations" of analog signals! The approximation is deemed so good that almost the whole world is using it without complains.

Maybe one day, some smart scientist will figure out an algorithm that works well enough. Until then, let's just stick to Melodyne etc. I believe it will still be somewhat functional.

 

[gutturalpiss]

While we're on this, take a look at Midingsolo (particularly the simple code).

It may not be that much difficult in research terms, but couple the progress of new programming techniques and languages we may see something we've never believed to be possible before. A good example regarding technology is 32-bit memory allocation in computing systems; nobody thought the mainstream need for registers would go beyond 2^32 addresses ever, but it happened and gave rise to 64-bit as we see from RISC machines during the 90s (the trigger to really mainstream x86_64 at least a decade later).

if [ "$FREQ" <= $((x)) ]; then
  $NOTE="AMinor"
  $PITCH="+0"
else
 $NOTE=$(($x + $y)) && $PITCH="+4" || $PITCH="unknown"
fi

Not meant to make sense :lol: Well, in anyway stuff like these are pitch-correction tools for monaural data so they're still of use no matter what, just that music is becoming less of a passion of more of a stinking business.

 

[stunew78]

Okay. I'm going to eat my words here. Big time.

Here's the link. celemony_ :: Direct Note Access

I've watched this video about 50 times in the last three days and I simply cannot believe what I am seeing and hearing.
I just don't believe it, I see it and hear it, but I can't believe it.

I need other opinions on this please

Hi stunew78,

Don't say that

Many people will be out of jobs if it was unanimously decided that this problem is unsolvable.
Yes, ultimately, the best source separation we can achieve will only still be an approximation. But maybe 10 years into the future, the approximation is good enough for studio use?

Let's remember that digital waveforms are also "approximations" of analog signals! The approximation is deemed so good that almost the whole world is using it without complains.

Maybe one day, some smart scientist will figure out an algorithm that works well enough. Until then, let's just stick to Melodyne etc. I believe it will still be somewhat functional.

 

[godchuanz]

Very nice demo

The last time when they first introduced their pitch correction software, it was this shocking for me too =D

Before we get too excited, we ought to understand that this is really good marketing. If it doesn't look good, it will not be shown. Notice that they are showing mainly clean guitar clips, which are much easier to process than say, an overdriven guitar. Also, notice that they have not shown us the sound of the individual notes within the polyphony. I suspect there might be phase issues if you play the notes individually, but when played as a whole chord, the effect is masked.

Melodyne produces very good software, I'm sure. I actually use their pitch correction function regularly. Just that... it does not really work as perfect as the way they show it to be. Most of the time, you'll find that it only works under some specific conditions. For example, in the pitch correction function, many note onsets simply don't show up.

I expect that the note separation issue to be similar. I'm sure it will work reasonably, but don't bet on it. Don't always think that "I can fix those wrong chords later" and end your recording sessions. I'd rather go for a few extra takes, just to be safe.

I'm quite excited to try this software though, can't wait =D