How to calculate feed forward net weights?

Question

0.00/5 (No votes)

See more:

I can't get my feedforward net to work. it's a single layer one with a single neuron...
are any of these equations wrong?
output = sum of all (inputs x corresponding weight)
error = desired output - neuron output
weight change => new weight = old weight + learningrate*error*input for that synapse

if not maybe you can see whats wrong when i go through two patterns of 3 inputs each time:

Data | Weight | Output | Desired | Error | New Weight
--------------------------------------…
334.6 | 0.00125126 | 257.953 | 342.2 | 84.247 | 281.892
340.1 | 0.563585 |
340.7 | 0.193304 |
--------------------------------------…
340 | 281.892 | 293,693 ..... making weights rediculous and the program error
342 | 287.088 | hitting #INF fast because it cant handle it....
347 | 287.223 |

Many thanks!

Posted 22-Nov-12 9:13am

lostandconfused1234

Add a Solution

2 solutions

Solution 1

You didn't mention the neural network algorithm. This actually looks like backpropagation network (since you calculate error).

In any case (either feed forward or backpropagation):

output = f( sum of all (inputs x corresponding weight) )

where, f() is the activation function.

In binary neural nets (e.g. Hopfield net), f() is a step function
In continuous nets, f() is a sigmoid function such as tanh(x)

Posted 22-Nov-12 18:21pm

manoranjan

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Stefan_Lang · Accepted Answer · 2012-11-23T06:33:00

The problem is that your activation function (i. e. the formula for calculating the output) is a linear function. Because of that, no matter what weights you set, the entire neural network of yours is simply a linear function and can never approximate more than two target values at once.

Basically the best you can achieve is linear regression, but then there is no point for using NNs for linear regression since you can directly solve that a lot faster. (and anyway your correction formula is wrong). The main point of NNs is that they can solve nonlinear problems with the right type of activation function:

What you need is a nonlinear activation function, and a correction formula specifically suited to that function. Backpropagation uses a simple sigmoid function that allows for a simple error correction formula, but isn't so hard to use different functions with similar results.

There are various articles on CP about Backpropagation and other types of nueral networks, I suggest you hit search and see what helps you best.