Re: Feed forward Neural network (back propagation) code classification problem - C# Discussion Boards

Re: Feed forward Neural network (back propagation) code classification problem

2-Aug-10 19:01

Hi, Everbody.

i have writen c# Artifical intelliget Neural Network. I have 500 count dataset which is normalized -1 and 1 and.i have used learning rate and momentum coefficient .

i teaching my network with 400 count dataset. Network have 5 inputs and 5 outputs. my output is setuped as 00001/00010/00100/01000/10000
when my network teched with 10000 iterations and 400 datasets, my output have a problem.
my network teached in last class of 10000 so when i tested my datasets , output only have been predicted as true class of 10000.
as result my code can only learn last class and other classes can not predicted or poor predicected

Example Test and Learning dataset
5 neuron input

{{0.04418 ,0.18135 ,0.28143 ,0.34367 ,0.14938 ,},
{0.37418 ,0.35050 ,0.18184 ,0.07493 ,0.01855 ,},
{0.04540 ,0.17061 ,0.24506 ,0.32560 ,0.21333 ,},
{0.08494 ,0.25653 ,0.26922 ,0.26141 ,0.12790 ,},
{0.18135 ,0.36832 ,0.24799 ,0.15743 ,0.04491 ,},
{0.11887 ,0.29143 ,0.27581 ,0.22870 ,0.08518 ,},
{0.02758 ,0.26239 ,0.37247 ,0.29998 ,0.03759 ,},
{0.00781 ,0.15792 ,0.32145 ,0.41811 ,0.09470 ,},
{0.07811 ,0.31608 ,0.31901 ,0.24506 ,0.04174 ,},
{0.07664 ,0.32390 ,0.32829 ,0.21674 ,0.05443 ,},
....

5 neuron output
{
{1 ,0 ,0 ,0 ,0 },
{1 ,0 ,0 ,0 ,0 },
{0 ,1 ,0 ,0 ,0 },
{0 ,1 ,0 ,0 ,0 },
{0 ,1 ,0 ,0 ,0 },
{0 ,0 ,1 ,0 ,0 },
{0 ,0 ,1 ,0 ,0 },
{0 ,0 ,0 ,1 ,0 },
{0 ,0 ,0 ,1 ,0 },
{0 ,0 ,0 ,0 ,1 },
{0 ,0 ,0 ,0 ,1 }
.....

My trainin code blok is as folowing show

int iterasyon=100000,it=0;
while (it <iterasyon && mmse >= 0.00001)//iteration start section
            {
                mmse = 0;
                mae = 0;
                for (int nf = 0; nf < 400; nf++)//dataset count*********
                {
                    mmse += hesaplas(ndata, nf);
                    //mae += smean_abs_err;
                    trains(nf);

                }//nf

                mmse = 0.5 * (mmse / 400 * 5));
              

                if (it % 100 == 0)
                {
                    yazs.WriteLine(mmse.ToString());
                    yazs.Flush();
                }
                it++;

               

            }//it

modified on Tuesday, August 3, 2010 9:30 PM

OriginalGriff2-Aug-10 21:47

OriginalGriff

2-Aug-10 21:47

Firstly, when you post code, use the "code block" widget rather than the "inline code" - it preserves the formatting and makes it easier to read:

int iterasyon=100000;
while (it = 0.00001)//iteration start section
   {
   mmse = 0;
   mae = 0;
   for (int nf = 0; nf < 400; nf++)//dataset count*********
      {
      mmse += hesaplas(ndata, nf);
      //mae += smean_abs_err;
      trains(nf);
      }//nf
   mmse = 0.5 * (mmse / 400 * 5));

   if (it % 100 == 0)
      {
      yazs.WriteLine(mmse.ToString());
      yazs.Flush();
      }
   it++;
   }//it

Secondly, if you are going to post code, make sure it compiles. That won't: it will complain about

while (it = 0.00001)//iteration start section

"Cannot implicitly convert type xxx to bool"
Even if it did, it would execute the outer loop either once, or an infinite number of times.

Thirdly, don't use magic numbers: how do you know you have 400 entries? What is the

mmse = 0.5 * (mmse / 400 * 5));

supposed to do?

Real men don't use instructions. They are only the manufacturers opinion on how to put the thing together.

Re: Feed forward Neural network (back propagation) code classification problem

Keith Barrow3-Aug-10 10:53

Re: Feed forward Neural network (back propagation) code classification problem [modified]

3-Aug-10 10:53

Three things (over Original Griffs Reply)
1. Is there any correlation beween the input an the output (I cannot see any)? if not, the NN is not likely to find it.
2. How are you training the network, your code is hard to read. If you are repeating outputs 1,0,0,0,0 (400/10000?) times at the end of trainging, you are probably annealing the network to give this result no matter what. You should cycle through many outputs.
3. It is possible to overtrain a NN!

I'd start with something simpler to test you NN code:
1 Input 0 -->360 , and train output to sin
1 Input 0 -->360 , and train output to x^2
2 Inputs, train to Logical Opertors (remember the XOR problem)
3 Inputs, result = i1 * i2 * i3

Then test on a real dataset.

ragnaroknrol The Internet is For Porn[^]

Pete o'Hanlon: If it wasn't insulting tools, I'd say you were dumber than a bag of spanners.

karayel_kara3-Aug-10 15:40

Re: Feed forward Neural network (back propagation) code classification problem [modified]

3-Aug-10 15:40

Firstly Thanks interest.

my dataset is folowing is that:

data is 5x400 matrix
b_cikis is 5x400 matrix

[...
long data removed
...].

int iterasyon=100000,it=0;
while (it = 0.00001)//iteration start section
            {
                mmse = 0;
                mae = 0;
                for (int nf = 0; nf < 400; nf++)//dataset count*********
                {
                    mmse += hesaplas(data, nf);
                    //mae += smean_abs_err;
                    trains(nf);

                }//nf

                mmse = 0.5 * (mmse / 400 * 5));
              

                if (it % 100 == 0)
                {
                    yazs.WriteLine(mmse.ToString());
                    yazs.Flush();
                }
                it++;

               

            }//it

modified on Wednesday, August 4, 2010 10:25 AM

Keith Barrow4-Aug-10 1:26

Re: Feed forward Neural network (back propagation) code classification problem

4-Aug-10 1:26

One of the admins has trimmed your message, the actual data is irrelevant and it was taking a long time to open your message.

I only want clarifiaction on two things, not the actual test data:

1. What does the input vector represent? What is its relationship to the output vector? The input looks like a bunch of random numbers to me.
2. Are you iterating 100000times over each input/output pair OR
Are you iterating over each pair of items in the input/ output arrays then doing this 100000 times.

ragnaroknrol The Internet is For Porn[^]

Pete o'Hanlon: If it wasn't insulting tools, I'd say you were dumber than a bag of spanners.

modified on Thursday, August 5, 2010 3:10 AM

karayel_kara6-Aug-10 18:35

Re: Feed forward Neural network (back propagation) code classification problem [Long post]

6-Aug-10 18:35

Hi,
that is my actual daya set and i took my friend.
Datas is measured patient at hospital. As a result Datas is Biological datas, My Friend clasified nntool in matlab. he used difference learnin method which is Levenberg-Marquart, i used gradient desent with momentum algortihm(gdm). so i hope i will classify with my soft.
But test results is not good.
2.Yes
I train each pair in input/output arrays in each one iteration

for i:0:i<100000;i++// iteration
 for j:0:j<400;j++ // input/output array 
    {

calculate error;

      if error>0.000001
       { train network;
        update weigths;
       }
        else
        break;


     }

Keith Barrow7-Aug-10 1:04

Response to e-mail sent directly from OP

7-Aug-10 1:04

OK the first thing we can rule out tis training to one input repeatedly at the end, which is a pity as it's the easiest fix Smile | :)

.

I think I understand the problem domain now, You are trying to categorise a function given 5 input points. The first thing is, you do have more than one layer between the input layer and the output layer don't you? If not you might hit the linear separability problem, as some of your functions look paraboloid, I *think* these aren't linearly separable, and would require at least two tiers of processing neurons

In your OP You defined the trainging outputs to be:
2 X Category I input
3 X Category II input (is this correct? the others all take two)
2 x Category II input.

I stopped here. I then entered your test data into excel (red = Cat I, green = Cat II Blue = Cat III) and plotted their "function" graphs

{0.04418 ,0.18135 ,0.28143 ,0.34367 ,0.14938}

{0.37418 ,0.35050 ,0.18184 ,0.07493 ,0.01855}

{0.04540 ,0.17061 ,0.24506 ,0.32560 ,0.21333}

{0.08494 ,0.25653 ,0.26922 ,0.26141 ,0.12790}

{0.18135 ,0.36832 ,0.24799 ,0.15743 ,0.04491}

{0.11887 ,0.29143 ,0.27581 ,0.22870 ,0.08518}

{0.02758 ,0.26239 ,0.37247 ,0.29998 ,0.03759}

I suggest you do the same. AFAICT plots for 1 and 3 are near identical (despite being in different categories, as do plots 4 & 6), plots 1 & 2 are different (depsite being in the same category, plot 1 looks like the maximun of a quadratic, the other is sigmoid.), this is also the case for graphs 6 & 7. If your input data is correct, it is doubtful than at NN can train to these sets.

The situation looks a little better if the expected outputs are changed to
2 X Category I input
2 X Category II input (corrected?)
2 x Category II input.
In theory what you are doing is similar to OCR, but the graphs do not look consistent enough within a category, or different enough between categories for you approach to work. Hopefully I'm wrong though!!!!!!

ragnaroknrol The Internet is For Porn[^]

Pete o'Hanlon: If it wasn't insulting tools, I'd say you were dumber than a bag of spanners.

Keith Barrow7-Aug-10 9:36

7-Aug-10 9:36

OP (Karayel) e-mailed me directly with the following
> Hi ,
>
> i have 5 layer and have 3 hiden layer in network
> datas is not mine, i said that i took from my friends Lab. datas is measure from patients.
> they can be wrong.
>
> i have 5 category in my data set
> i am sending you my datas
[edit removed the data set!]

The fact that you have 3 hidden layers will solve the linear separability problem.

Actually, this is the crux of your problem:
"datas is measure[d] from patients. they can be wrong"
Firstly, I replotted the first three graphs of category one. The graphs one an three looked very similar (a line going up hitting a maximum, then going down again)(which is good), but graph 2 looked sigmoid (it could have been the same graph but shifted earlier).The problem is this breaks the pattern, and I suspect this happens in all the categories, if so the NN is unlikely to be any help IMO. People have tried to spot predict patterns in stockmarket prices with limited success, I think the situation you have is similar.

If the graphs are consistent (and you should plot samples from each category yourself to check) you may have a hope. If this is the case there are three things I'd Add
1) if this is a research project, is a NN really suitable in vivo? The system you are writing is inherently unprovable, and given this is a medical diagnostic tool (with potentially serious consequences), should it be trusted?
2) You are training lots of category 1 inputs, then lots of category 2,etc. If your NN anneals, it could well anneal to category 1 by doing it this way. You should cycle through one example of Cat I, Cat II Cat III etc while training, and repeat until the NN is as accurate as you need it. If you can develop a function to simulate expected data sets, that would be even better, as real data is unreliable, and trains unreliably. Ironically, it is the unreliable input that the NN sorts out, but I'd use generate training inputs.
3) Can you express what constitues category 1,2,3, etc formally in code? If so just coding it will be better.

ragnaroknrol The Internet is For Porn[^]

Pete o'Hanlon: If it wasn't insulting tools, I'd say you were dumber than a bag of spanners.

karayel_kara10-Aug-10 17:56

10-Aug-10 17:56

Hi i am writing again.
I want to test my code and
is there any fucntion which are 5 inputs and 5 out?
or square function example, 2 inputs x 2 outputs, 4 inputs x4 outputs.
May be , you can recommend a function for my testing.

in addition, i thank you so mouch for sharining your informations with me

Best regards.

Keith Barrow11-Aug-10 10:16

11-Aug-10 10:16

If your are just testing the NN works, I'd lower the inputs and outputs to 1 neuron, then train to some functions (sin, x^2 ln etc), this gives you a base line to ensure all is correct.
As for 5 inputs, 5 outputs, I'm not sure, the problem I applied my NN to was quite different. You could try a matrix transform of some sort, or categories 5 inputs. You might have better luck contacting a dedicated NN board (I'm sure I've seen them somewhere).

I have only used a NN once(for my MSc Dissertation) and that was ~10 years ago so I'm pretty rusty. Every couple of years I get the urge go back to it, but other things are shinier (and /or more remunerative Smile | :)

. I had to hand-check my NN there was a term in the momentum calculation (actually a "-" instead of a "+"!) that brought the NN close to the desired result, then would diverge again. That took two weeks (c++ on linux, nedit, but no real IDE).

Anyway, best of luck.

ragnaroknrol The Internet is For Porn[^]

Pete o'Hanlon: If it wasn't insulting tools, I'd say you were dumber than a bag of spanners.

karayel_kara12-Aug-10 18:15

12-Aug-10 18:15

tanks you too,
i used meumentum coefficient.
May be you will begin study NN again?
can you see your code ? and in there, can you write main block in your code?

best.

Keith Barrow12-Aug-10 23:12

This constraint cannot be enabled as not all values have corresponding parent values

12-Aug-10 23:12

karayel_kara wrote:
I used meumentum coefficient.

It's a good thing to use, I tested the property my NN without the coefficient, and learning was much slower, I just had a bug in mine!

karayel_kara wrote:
May be you will begin study NN again?

Not likely sadly, not much call for it in the business world and most of what I do is related to business.

karayel_kara wrote:
can you see your code ? and in there, can you write main block in your code?

If memory serves ~2000+ lines of [badly written, I had only just started developing!] c++, and I only have a hardcopy here, the CD is back in the UK. There are neural network frameworks available on CP. This one is really good:

http://www.codeproject.com/KB/recipes/aforge_neuro.aspx[^]

ragnaroknrol The Internet is For Porn[^]

Pete o'Hanlon: If it wasn't insulting tools, I'd say you were dumber than a bag of spanners.

Masking in C#

Pdaus2-Aug-10 18:28

Pdaus

2-Aug-10 18:28

hye all,

How do I mask my image such that i got the masked image. For example i want to mask round shape and discard the background. Anyone could guide me?

Re: Masking in C#

Pete O'Hanlon2-Aug-10 21:37

Pete O'Hanlon

2-Aug-10 21:37

Pdaus wrote:
Anyone could guide me?

Yup. Help yourself to any one of these[^] bad boys.

"WPF has many lovers. It's a veritable porn star!" - Josh Smith

As Braveheart once said, "You can take our freedom but you'll never take our Hobnobs!" - Martin Hughes.

My blog | My articles | MoXAML PowerToys | Onyx

Vimalsoft(Pty) Ltd2-Aug-10 11:42

Vimalsoft(Pty) Ltd

2-Aug-10 11:42

Good Day All

i have a function that builds a Dataset from 3 Xml Files. Let me take this time to explain the xml files.

There is Subjects.xml, this xml contains all the subjects and i have XML_Venue.xml that contains all the venues and i have TimeTableFull.xml it contains the times,Venues that the subjects will be attended at. Now that means TimeTableFull.xml has many occurances of subjects but on Different venues or same venues but different times. The Following code builds a dataset from the xml's

private DataSet CreateDataSet()
   {
        DataColumn[] keys = new DataColumn[1];
        DataSet dsFinalTimeTable = new DataSet();
        DataTable tbldt;

        tbldt = new DataTable("Subjects");
        //Add Staff Table 
        XmlDataDocument xmlDatadocStaff = new XmlDataDocument();
        xmlDatadocStaff.DataSet.ReadXml(@"J:\Pilot Project\App_Data\Subjects.xml");
        tbldt = xmlDatadocStaff.DataSet.Tables["Subjects"];
        keys = new DataColumn[1];
        keys[0] = tbldt.Columns["ID"];
        tbldt.PrimaryKey = keys;
        dsFinalTimeTable.Tables.Add(tbldt.Copy());


        //Add Venue Table 
        tbldt = new DataTable("Venue");
        XmlDataDocument xmlDatadocVenue = new XmlDataDocument();
        xmlDatadocVenue.DataSet.ReadXml(@"J:\Pilot Project\App_Data\XML_Venue.xml");
        tbldt =  xmlDatadocVenue.DataSet.Tables["VENUE"];
        keys = new DataColumn[1];
        keys[0] = tbldt.Columns["ID"];
        tbldt.PrimaryKey = keys;
        
        dsFinalTimeTable.Tables.Add(tbldt.Copy());

       //TimeTable
        tbldt = new DataTable("TimeTable");
        XmlDataDocument xmlDatadocTimeTable = new XmlDataDocument();
        xmlDatadocTimeTable.DataSet.ReadXml(@"J:\Pilot Project\App_Data\TimeTableFull.xml");
        tbldt = xmlDatadocTimeTable.DataSet.Tables[0];
        keys = new DataColumn[1];
        keys[0] = tbldt.Columns["ID"];
        tbldt.PrimaryKey = keys; 
        dsFinalTimeTable.Tables.Add(tbldt.Copy());
        
        
        //Setup RelationsTimeTable//
        DataRelation VenuesRelations = new DataRelation("VenueRel", dsFinalTimeTable.Tables["Venue"].Columns["ID"], dsFinalTimeTable.Tables["Appointment"].Columns["ID"]);
        DataRelation ModuleRelations = new DataRelation("ModuleRel",dsFinalTimeTable.Tables["Subjects"].Columns["ID"],dsFinalTimeTable.Tables["Appointment"].Columns["subjectid"]);
       
        dsFinalTimeTable.Relations.Add(VenuesRelations);
        dsFinalTimeTable.Relations.Add(ModuleRelations);

        return dsFinalTimeTable;
   }

and my problems comes when i have have to add relations on the module relations

dsFinalTimeTable.Relations.Add(ModuleRelations);

i receive the Following error

This constraint cannot be enabled as not all values have corresponding parent values

and i tried to use this line of code to remove this integrity enforcement

dsFinalTimeTable.EnforceConstraints = false;

and this left me with an empty grid and this does not look correct.

Can someone assist me. i have attached the 3 xml files.

http://www.vbforums.com/attachment.php?attachmentid=79604&d=1280784905

http://www.vbforums.com/attachment.php?attachmentid=79605&d=1280784953

http://www.vbforums.com/attachment.php?attachmentid=79606&d=1280785001

Thanks

Vuyiswa Maseko,

Spoted in Daniweb-- Sorry to rant. I hate websites. They are just wierd. They don't behave like normal code.

C#/VB.NET/ASP.NET/SQL7/2000/2005/2008
http://www.vuyiswamaseko.com
vuyiswa@its.co.za
http://www.itsabacus.co.za/itsabacus/

Re: This constraint cannot be enabled as not all values have corresponding parent values

Gopal.S2-Aug-10 17:14

Gopal.S

2-Aug-10 17:14

Hi Vuyiswa,

Please correct the data's of subjectid column in Appointment table (TimeTableFull.xml). It contains wrong values that should match the
'ID' column values of subjects table.

Thanks,

Gopal.S

Re: This constraint cannot be enabled as not all values have corresponding parent values

Vimalsoft(Pty) Ltd2-Aug-10 21:40

Vimalsoft(Pty) Ltd

2-Aug-10 21:40

Thank you very much. That was the case

Change (or disable) selection color of listview [SOLVED partially]

sodevrom2-Aug-10 8:11

sodevrom

2-Aug-10 8:11

Hello guys,
First, I don't want to use OwnerDraw, that is a condition I have Frown | :(

.

What I want is set a custom color to the selection of a listview. I want the color to be visible even if the listview is not focused.
I know about the HideSelection member, but that's not what I need.

I tried to overrwrite the onselectionindexchanged, and make the item.selected=false, and after that, set a custom back/fore color. This does not work because I have multiple items, and it deselects all items but one.

I need one of 2 things:
a)Keep selection visible (with blue color, not gray, this is the problem with HideSelection) always
b)Remember selected objects, but do not show the selection at all (because if the selection is visible, than the custom back/fore color are not visible), so I can "mimic" the selection by setting a custom back/fore color to the items.

Now I have been spending over 4 hours now on this stupid issue, and no luck. Anyone has a suggestion? It would help a lot.

*******************solution
Instead of setting the color in the on selected changed event, we can do it on mouse up. It's not perfect, but it's the best I got. The best way to do this is using a custom draw listview. Here is the code:

List<int> selections=new List<int>();

protected override void OnMouseUp(MouseEventArgs e)
        {
            selections = this.SelectedIndices.Cast<int>().ToList(); //store selection
            this.SelectedIndices.Clear(); //clear selection 

            for (int i = 0; i < this.Items.Count; i++)
            {
                if (this.selections.Contains(i))
                    this.Items[i].BackColor = Color.Blue; //if selected, make it blue
                else
                    this.Items[i].BackColor = Color.White; //if not selected, make it white
            }
            base.OnMouseUp(e);
        }

modified on Monday, August 2, 2010 9:01 PM

Re: Change (or disable) selection color of listview (not easy :( )

Eddy Vluggen2-Aug-10 8:23

Eddy Vluggen

2-Aug-10 8:23

sodevrom wrote:
a)Keep selection visible (with blue color, not gray, this is the problem with HideSelection) always

That's by design; whatever is focused is drawn in the "selected" color. That way I can (as a user) visually differentiate between the ListView that's focused (and will react to keyboard-input) and which won't. If both were using blue, it would imply that both have the input-focus. That's not possible in Windows, since there can only be one active control with input-focus.

There's no "neat" way to do this. Why no owner-drawing btw?

sodevrom wrote:
b)Remember selected objects, but do not show the selection at all (because if the selection is visible, than the custom back/fore color are not visible), so I can "mimic" the selection by setting a custom back/fore color to the items.

Remembering selected objects can be done by adding them to a generic list, whenever you select one. Sounds like you want to replace the default multiselect by your own version?

I are Troll Suspicious | :suss:

Re: Change (or disable) selection color of listview (not easy :( )

sodevrom2-Aug-10 8:28

sodevrom

2-Aug-10 8:28

Hello,
I know this is possible because I seen it in action somewhere.

I do not want to use owner draw, because I would have to implement tons of things, and I still want the list to look exactly like in windows (with the theme from windows).

Still waiting for some suggestions.

Thanks!

Re: Change (or disable) selection color of listview (not easy :( )

Eddy Vluggen2-Aug-10 8:47

Eddy Vluggen

2-Aug-10 8:47

sodevrom wrote:
I know this is possible because I seen it in action somewhere

Yup, it's possible Smile | :)

sodevrom wrote:
I do not want to use owner draw, because I would have to implement tons of things

You want non-standard behavior, and that has to be implemented somewhere. Owner-drawing might sound like overkill, especially if you've been looking at examples that draw a lot manually. The example on MSDN[^] looks rather short and might do the trick.

sodevrom wrote:
and I still want the list to look exactly like in windows

Make sure you're using the SystemColors[^] when drawing.

I are Troll Suspicious | :suss:

Re: Change (or disable) selection color of listview (not easy :( )

Keith Barrow2-Aug-10 8:35