LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Musical Instrument Recognition Program help

I am currently working on a LabVIEW program to allow me to recognize a musical instrument by analyzing its waveform.

I am using LV 7.0 and 7.1. I have 7.0 on my home computer, and the computer labs at college have 7.1 (both running Windows XP Professional).

I have a few questions. In order for you to understand what I am talking about, I have included explanations of/(and a link to) my program.

You can go to my website if you want to see more about the program. It may be helpful to click the “Documents” button and view the "Senior Project Midterm Review Presentation", “Project Proposal Presentation”, and the "Final Project Proposal".

Website: http://home.pct.edu/~rhijas55/SeniorProject/index.htm

Program: http://home.pct.edu/~rhijas55/SeniorProject/documents/compare.zip

Open the “CompareHarmonics_RM.vi” file, and select WAV files from the “sound” folder. The recordings in the “sound/no” folder do not match up correctly. The others do. Make sure that the correct frequency is entered for the file you are playing.

Questions:

1. What are some ways that I could more precisely compare two arrays to see how well they match up?

2. Should the difference of the waveform’s harmonics be weighted greater when the harmonics are larger values (earlier harmonics for most instruments), than the differences of the later (around the 12th harmonic and on) harmonics which have much lower amplitudes?

3. Do any of you know of any characteristics that are always found in the waveform of certain instruments?

    4. Does anyone know if there is a way that LabVIEW can step through the contents of a folder, selecting the next file each time the loop iterates?

Since the objective of this project is to create a program that can recognize an instrument based on its waveform, I need to compare whatever instrument waveform I select, with the harmonics which are characteristic of different types of instruments.

 


Message Edited by rhijas on 03-15-2006 08:47 PM

0 Kudos
Message 1 of 29
(5,563 Views)

I am comparing the harmonics (Example of harmonics: http://hyperphysics.phy-astr.gsu.edu/hbase/audio/geowv.html#c1) of different types of musical instruments to find which match up the best. Basically, what my program is doing is that after I find the amplitude values of the harmonics of an instrument, they are converted relative to the greatest value, and stored in an array. The first array value is the amplitude of the first harmonic; the second array value is the amplitude of the second harmonic; and so on. I have analyzed different types of instruments, and stored these harmonic arrays as spreadsheet files. These spreadsheet files then will act as references to show what the harmonics of that type of instrument look like.

 

My program currently is finding the average difference between each array value (the arrays which are stored as spreadsheet files) and the coinciding array value of the current waveform.

 

What are some ways that I could more precisely compare two arrays to see how well they math up?

 

Should the difference of the waveform’s harmonics be weighted greater when the harmonics are larger values (earlier harmonics for most instruments), and then the differences of the later (around the 12th harmonic and on) harmonics which have much lower amplitudes?http://home.pct.edu/~rhijas55/SeniorProject/images/pictures/Harmonics.PNG) (

How could I go about doing this?

 

Do any of you know of any characteristics that are always found in the waveform of certain instruments? For example, I think I heard somewhere ( and both samples I have show) that clarinets have mainly odd harmonics, with very small even harmonics.

0 Kudos
Message 2 of 29
(5,538 Views)

Spreadsheet to Array:

The Bottom section of the Block diagram has many “Spread. to Array” and “Array Diff.” modules repeating the same thing over and over again. They take a spreadsheet file from the folder labeled “Harmonic Data Rel. Max”, convert the file to an array, and find the average difference between each array value and the coinciding array value of the current waveform. (These spreadsheet files contain the relative amplitude harmonic values of different types of instruments.) This average difference and the name of the instrument are put into arrays. Then the same thing is done with the next spreadsheet file until all of them have been compared with the current waveform’s harmonics.

 

Once all the stored files have been compared to the current waveform, the loop at the bottom of the Block Diagram finds for which instrument there was the least difference from the current waveform. The idea is that harmonics of instruments of the same type would have less difference than two different types of instruments.

 

What I would like to do is to find a way that I can put all the converting and comparing into a single loop so I don’t have to duplicate it for each stored instrument. I would want to look in the folder that holds all the spreadsheet files, and choose the first file. That file would be converted to an array, and then compared with the current waveform’s harmonics. The average difference and the instrument’s name would be put into arrays, and then the next loop would iterate. For each iteration of the loop, the next spreadsheet file would be selected from the folder, until all stored files have been analyzed, and their differences and names put in arrays. Then the arrays would be sent to the last section where it would determine which stored harmonic array hast the least difference from the current waveform’s harmonics.

 

Does anyone know if there is a way that LabVIEW can step through the contents of a folder, selecting the next file each time the loop iterates? That would make it a lot easier to add more stored harmonic files so I won’t have to repeat the code for each file added to the folder.

 

I should also be able to more successfully recognize instruments if I have more samples of instruments stored to account for variations among instruments.

0 Kudos
Message 3 of 29
(5,538 Views)
By the way, Do any of you know why the posting box won't let you make long posts? It said that there was a 5000 word limit. All three posts only added up to around 900 something characters.

Does it format everything in HTML, and count all the tag characters also? Of so, is there a way to get it to stop adding tags except for carrage returns? I am trying this in HTML mode, so maybe that is what you have to do?
0 Kudos
Message 4 of 29
(5,523 Views)

This sounds like a very interesting project.  I would be delighted to see your final program.  I play and own several musical instruments:  guitar, mandolin, dulcimer, fiddle, dobro, keyboards, wooden flute, clarinet, saxaphone (don't own one but play a little), drums (barely).

I suppose you would have to take a sample waveform from each instrument for analyzation and to see the variance between instrument waveforms.  However, you will have variation just within one instrument between different models.  A Martin box guitar sounds slightly different than an Ovation, which sounds different than a Yamaha, etc.  Also different strings can make different sounds.  Also different picks, especially finger picking, can make different sounds.  You have quite a challenge on your hands.

Here is a method to loop through files in a directory for processing each one:

Message Edited by tbob on 03-16-200610:46 AM

- tbob

Inventor of the WORM Global
Message 5 of 29
(5,510 Views)
Rhijas:  I tried opening your links but could not open any of them.  FTP to your folder prompts me for a login and password.  Can you correct the problem so that I can look at your project?
- tbob

Inventor of the WORM Global
0 Kudos
Message 6 of 29
(5,507 Views)
If you want to follow those links, don't click them. That doesn't work. Copy the address, and paste it into your address bar. It works for me, and I'm not using FTP.
Program: http://home.pct.edu/~rhijas55/SeniorProject/documents/Compare.zip

You can also get it Here: http://home.pct.edu/~rhijas55/SeniorProject/index.htm
Go to the Documents page, and chose "Instrument Comparison".

I'll take a look at that file program you posted. Does it allow you to select a different file from a certain folder each loop, without having to specify each file name? That is what I would like to do, so that I can add files to the folder without having to list every name.

Thats pretty neat that you can play all of those instruments. I guess since a bunch of them are strings, they may be similar. I have been using samples from the University of Iowa Electronic Music Studios website (http://theremin.music.uiowa.edu/MIS.html). I take the .AIFF files, and use Audacity (http://audacity.sourceforge.net/) to convert them to .WAV files which Labview can then read.

I have samples for: Alto and Soprano Saxophone, Double Bass, Bassoon, Bb Clarinet, Cello, Flute, Horn, Oboe, Tenor & Bass Trombone, Trumpet, Tuba and Violin. I would like to use some others, but I would have to get samples for those instruments. Do you know where I can find samples of an Acoustic Guitar or Piano? They would need to be MONO, and saved in a .WAV format (or a format that Audacity can convert to .WAV).

If you, or anyone else here have a microphone, you/they could make a recording of a Piano or Guitar playing sustained notes one at a time. I would just neet to get more than one sample so that I can have some to store as spreadsheet files, and then other unstored samples that I can use to try to match up with the stored samples of the same instrument.

Message Edited by rhijas on 03-16-2006 01:25 PM

0 Kudos
Message 7 of 29
(5,504 Views)

Here is how the program segment that I posted works:  The first subvi, List Directory returns a string array containing the file names of every file in the directory specified (C:\My Files) that match the pattern *.wav.  Both directory path and pattern can be edited at will.  The middle output on the right side of List Directory is the array of file names.  This connects to a For Loop.  The pink terminal that looks like this, [ [] ], means that Indexing is enabled.  (A solid colored box means indexing is disabled).  By enabling indexing, each element of the array will be processed by each iteration of the loop.  In other words, the first loop iteration uses the first array element, the second iteration uses the second array element, and so on until all array elements are used.  Then the loop exits.  So if you have a unique directory that contains nothing but the files you want to process, this code segment will work.  Any file you drop into that directory will be processed without changing the code.  If you want to store other files there, just have a common extension for the files you want to process and use the pattern input to List Directory (like *.wav).  This will cause only files ending with the wav extension to be processed, and all other files will be ignored.  You don't need the *.wav pattern input if you won't be putting in these other files that should not be processed.  So if all your files are .wav files, you can delete the *.wav pattern string from the code.  This will process all files in your directory.

To continue on, each file name is concatenated with the directory name and a complete pathname is created by the Build Path function.  Then the pathname is fed into the Open/Create/Replace File function which opens the file.  The file is then read, data is stored in the indicator, and the file is closed.  You must add your processing code to the loop.

Hope this explains everything.  I looked at your website.  Very interesting project to a musician/Labview programmer such as myself.  When I get time, I will look at your code. 

I can try to get some wav files for you of guitar and other string samples.  Are you in a terrible hurry?  It may not be until the weekend before I can find time to do this.  I would like to help out in this manner.  Am I correct in that you are looking for samples of the E note in the 4th octave?  Do you know what the fundamental frequency of this note is?  I could get samples from a box guitar, electric guitar (so many different tone settings for the pickup), dulcimer, mandolin, dobro.  Seems like you already have the piano and clarinet samples.  What about a harmonica?

 

- tbob

Inventor of the WORM Global
0 Kudos
Message 8 of 29
(5,473 Views)
tbob, thanks for showing me how to access files like that. I tested it out, and it works. Now I am able to add more sample instrument spreadsheet files to the folder without having to change the code in my Block Diagram.

No, I'm not in a hurry. I already have a lot of different instrument samples. I can always just use them. It just would be nice to have a large variety of instruments by the end of the project. Yes, I have been using mostly samples of the E not in the 4th octave, though I can use any note. I am just trying to use notes around E4 for now so that I don't have as great harmonic variation due to playing widely varying notes.

Is the Box Guitar just the standard acoustic guitar? If you have a normal acoustic guitar, you could make some samples of different notes (you could even make multiple samples of different notes). If you want to make some harmonica samples, you could. Is the sound of the harmonica affected by the mouth in adittion to the reeds inside, or is it just mainly the reeds?

What I need to work on next is to find a way to more accurately match up the harmonics of an instrument woth those of the same type of instrument out of a list of different kinds. When we look at the frequency spectrum of an instrument, we can see how the amplitudes of the harmonics relate to each other, and can notice when a different sample's harmonics are set in a different shape.To get a computer to do that, I need to figure out how to relate the two arrays of harmonic amplitudes somehow so that it will notice when the harmonic amplitudes match up, or when they are different.
0 Kudos
Message 9 of 29
(5,461 Views)
To match up the harmonics, I would try a normalization process. Calculate the ratio of each peak to the fundamental. That will eliminate amplitude variations between different data sets and also allow you to ignore small offsets in the tuning of the instruments. You analysis would then look at the array of ratios of harmonics.

Lynn
0 Kudos
Message 10 of 29
(5,450 Views)