|
Hi,
this is my first post on this forum.
First a Short intro to the Self Organizing Map
A Self Organizing Map (SOM) ,-sometimes called a Kohonen network-, maps multidimensional data onto a two dimensional map. A classic example is to classify colors by the Red, green and Blue (RGB) values. So if we have a large (10 000) set of randomly chosen colors, with different shades of, greens, reds, blues, pinks, etc. the SOM can map them on a 2-dimensional map where similar colors tend to end up on the same node on the map. If we chose a map with the dimensions 5 by 5 we get a map with 25 nodes. As an example the pink shades might end up in one corner of the map close to violet shades and the violets close to blue shades and so on. So similar colors tend to be close to each other. So if we wanted to classify those 10 000 colors in our dataset into 25 classes we could look at each colors node and use the information to se what class each of the colors belong to.
But how can we create clusters of several nodes based on their similarity to each other?
modified on Thursday, August 18, 2011 9:19 AM
|
|
|
|
|
SharpSim wrote: this is my first post on this forum.
And, as it seems, most probably the last ! 
|
|
|
|
|
Sometimes I get the impression that CodeProject is a forum of uneducated people.
|
|
|
|
|
WOw, are you stuck somewhere in the time-space of last week ?
|
|
|
|
|
This question may be more suited in this forum.
<a
href="http: www.codeproject.com="" forums="" 326859="" algorithms.aspx"="">http://www.codeproject.com/Forums/326859/Algorithms.aspx[^]
Regards
[Edit]
Fix link.
It was broke, so I fixed it.
modified on Thursday, August 18, 2011 7:58 AM
|
|
|
|
|
S Houghtelin wrote: This question may be more suited in this forum.
<ahref="http: www.codeproject.com="" forums="" 326859="" algorithms.aspx"="">http://www.codeproject.com/Forums/326859/Algorithms.aspx[^]
Can we move the question to that forum?
|
|
|
|
|
|
I knew it. I did not, but I knew someone would !
|
|
|
|
|
You forgot to chill it using Liquid Nitrogen.
Panic, Chaos, Destruction. My work here is done.
Drink. Get drunk. Fall over - P O'H
OK, I will win to day or my name isn't Ethel Crudacre! - DD Ethel Crudacre
I cannot live by bread alone. Bacon and ketchup are needed as well. - Trollslayer
Have a bit more patience with newbies. Of course some of them act dumb - they're often *students*, for heaven's sake - Terry Pratchett
|
|
|
|
|
Seconded. Motion is passed.
|
|
|
|
|
What an interesting question, well I think that a Self Organizing Map should ............ oh shiny! or in my case .... hmm chocolate! 
Ali
|
|
|
|
|
Welcome to the algorithms forum.
I'm not sure what triggered such negative responses to your question (originally posted to another forum which it wasn't appropriate for ? then moved here with all its baggage of snipes ?), but it seems to me your question is related to algorithms.
I'm not familiar with SOM, or Kohonen networks, but I come here, to this forum, to share my own algorithmic interests, and to learn, and be stimulated by, others interests, so I'm happy when someone introduces me to a concept I haven't encountered before.
One way I think you can make your question here more enjoyable for other participants is to share the context of your question: why you are interested in this type of algorithm, what this problem or question is in the context of your work, or your study.
If you share what you have done so far, in code, or in theory, or where you find your progress in implementing this type of algorithm/solution is blocked, as specifically as possible, then, I think you'll get responses that will be most helpful.
Your description of Kohonen networks, and the example given of organizing a color-space, reminds me of the wonderful program 'Visual Thesaurus'[^] which, when it came out several years ago, was a real 'revelation' ... to me.
VT 'maps' semantic affinities between words in a thesaurus using font-size and font-attributes to represent a virtual third dimension, and its 'dynamic:' the 'cloud' of associated words is being continually updated.
And, yes, the 'word-tag-clouds' seen on many website these days are a form of 'static' VT, imho.
I wonder if VT type algorithms might be considered an algorithmic problem of the type you are describing ?
Where I get interested in the issue I think is raised by your question is: the how of ... once you establish the evaluative criteria by which a multi-dimensional data set is 'distilled' into some useful summary representation ... you then express that representation in a form which you can use a compiler (or lexer-parser) to create : as visual interface, as meta-language (DSL), etc.
While some data-sets, I think, have an 'easier,' perhaps statistics-based, answer to how you represent summary information in a way that facilitates communicating essential 'meaning,' and enables sophisticated exploration through 'drill-down' in hierarchical, or 'flat,' visual interfaces ...
Other 'problem spaces,' ones I think are much more interesting, are going to have internal 'semantic' networks based on ? Bayesian probabilities ?
An example: a personal interest of mine is the evolution of Buddhist sacred iconography in S.E. Asia, where I live. There is a kind of an algorithmic component in the sense of a 'guiding meme' of the 32 lakshanas originating in Sri Lanka which express a sense of the normative visual attributes of representation of the teacher now known as 'the Buddha.'
And then, there's the whole complex chaotic history of how styles of representation of 'the Buddha' evolved through conquest and assimilation (capture and relocation of artisans, and or images, and supplies of raw materials, being a key goal in S.E. Asian geo-political warfare).
And then, there unique events, as when King Tilokkaraja of Chiang Mai, in the 15th. century, as part of establishing more centralized control of the nascent proto-polity of a more cohesive northern Thailand meta-state (Lanna), promoted the image of 'the Buddha' in 'royal raiment' as part of a larger campaign to have himself recognized as a divinely sanctioned ruler (dhammaraja, chakravartin).
So you have a leit-motiv (the lakshanas), you have historical mutation and change, you have top-down visual innovation and 'orthodoxy' imposed at times, and you have serendipitous changes, sometimes enabled by a changed supply of raw materials, or contact with other cultures, or innovation by particular craftspeople, or sudden availability of new technology
And then, it gets even more complex: at times 'retrograde' change occurs where the visual style of an older, or imagined archaic, style becomes fashionable, or required, or is re-vivified by some individual innovator.
And at the maximum 'edge' of change ... fractal ? ... you have meta-cultural differences which can be quite profound. To the 'western mind,' shaped so profoundly by ideas of absolute historical origins, a non-recurrent sequence of time, cumulative progress, Aristotelian syllogism, the division of art and science (trivium, quadrivium), mind and body, Psyche and Techne: a culture where time is experienced as an 'eternal return' of a boundless cycle of dynamic manifestations of opposites (what Jung would have called 'enantidromia') is 'alien' ground.
That's the kind of 'data-space' that I am interested in. How to make an 'interface' to that kind of data ... mmm ... there's the rub.
best, Bill
"In the River of Delights, Panic has not failed me." Jorge Luis Borges
modified on Thursday, August 18, 2011 9:04 PM
|
|
|
|
|
BillWoodruff wrote: originally posted to another forum which it wasn't appropriate for ? then moved here with all its baggage of snipes
Yup. Started life in the lounge. Didn't actually get stomped as hard as I expected.
Cheers,
Peter
Software rusts. Simon Stephenson, ca 1994.
|
|
|
|
|
It's rated low because it was posted in the Lounge and then moved here with the low votes intact. It seems like an interesting enough algorithmic post.
|
|
|
|
|
Thank you Bill
BillWoodruff wrote: If you share what you have done so far, in code, or in theory, or where you find your progress in implementing this type of algorithm/solution is blocked, as specifically as possible, then, I think you'll get responses that will be most helpful.
Well, the color sorting problem is one of the examples often used to describe the Self Organizing Map (SOM) algorithm.
There is several implementations of the SOM algorithm. There exist task specific SOM algorithms and general purpose SOM algorithms. Here is three general purpose implementations:
SOM Toolbox 2.0 (Matlab 5) By Tevo Kohonen and his research team , sort of open source
http://www.cis.hut.fi/somtoolbox/[^]
SOM_PAK (ansi c) also By Tevo Kohonen and his research team, sort of open source
http://www.cis.hut.fi/research/som_lvq_pak.shtml[^]
There is also an Octave/Matlab compatibel developed based on the above SOM Toolbox 2.0 called
Melikerion , open sourcehttp://www.finndiane.fi/software/melikerion/[^]
The Melikerion research is done by for the Finnish diabetic nephropathy study. But the software is general use.
Another good SOM/Kohonen software package is Ron Wehrens "kohonen: Supervised and unsupervised self-organising maps". It is written in R http://cran.r-project.org/[^]
Kohonen package for R
http://cran.r-project.org/web/packages/kohonen/index.html[^]
There exist also several SOM/Kohonen projects here on codeproject.com
I am studying individual persons career paths wich include many person level variables, such as, age, sex, education, graduation, family related variables, income, neighborhood, house, etc. I know how to classify my data on to a 2-d map with SOM_PAK and Melikerion. As the first step the data is scaled and the nodes of the SOM is set to initial values usually based on a random sample of the input data. Then one member of the input dataset is compared to all of the nodes by calculating the multidimensional euclidian distance. The node wich is closest to the input data "winns" and are updated with the values from the one input data member. Also the surrounding nodes are adjusted as a gradient where far nodes are adjusted less than the closest. This is repeated for the whole dataset several times until the map converges. The convergens is accomplished by decreasing the adjustment of the nodes. When the input data is clustered the SOM algorithm can reveal the clustering and that can be observed by visual inspection of the map. My basic problem is how two find the clusters made of several of the map nodes. The SOM creates a map where the nodes that are close to each other is more alike than nodes far apart.
I would like to find clusters of similar nodes and use those as the classification of my data. That informatin could be used in the analysis of the differences and similarities between different individuals careers and income levels related to their educational history, work career and other family characteristics.
|
|
|
|
|
It seems to me that this problem boils down to selecting the best projection of your problem space onto two axes. Do you already know how to create such a map? Are you just looking for cluster analysis algorithms? A search should turn up some 2D cluster analysis algorithms, it must be a solved problem.
|
|
|
|
|
You sent a shiver down my spine when I read your post. I've just being doing this as part of my Uni AI course, although we were doing it with analysis of birds contaminated by an oil spill, and had 17 inputs going to a 8x8 SOM grid.
We have been using JavaNNS for all the course work.
What you basically need to do is set up your network with each of the 3 input nodes (RGB) connected to every single node in your SOM.
You haven't said what language you are trying to create this with, but searching google there are heaps of results with libraries to use.
Here is one example article;
Implementing the Kohonen Neural Network[^]
|
|
|
|
|
I will try to formulate my question about Self Organizing Map (SOM) that I have. My research problem is that I want to classify the entries of a large dataset so that similar entries end up in clusters where the the variation within clusters are less than between the clusters. I know this can be accomplished with the SOM algorithm. Now to the point:
If the SOM is started with a large map lets say 10 by 10 nodes (100 nodes in total). So I map my data onto the 100 node map. Then I visually inspect the result and find that there is lets say 7 distinct clusters. Is it then approperiate to reduce the number of nodes to a map 3 by 3 nodes (9 nodes in total) and again run the SOM onto this much smaller map. And then use the nodes of this smaller map as my clusters?
|
|
|
|
|
Hi,
Interesting the way that one intellectual quest can lead to another: my connecting-nodes-in-hierachies quest led to thinking about property change notifications in WinForms, which took a detour into WPF where so much work has been done with INotifyPropertyChanged automation, which led to experimenting with auto-code generation with T4 and PostSharp.
Which led to: I wrote a (non-recursive) code-generator ... which I will write-up for CP as a Tip or something if there's any interest ... that will generate a complete C# class for evaluating all possible logical comparisons of #n boolean variables. Here's the 'guts' of what it produces if I call it with an integer argument of #3:
if (Is_A && Is_B && Is_C)
{
Console.WriteLine("Test0 passed");
return;
}
else if (!Is_A && Is_B && Is_C)
{
Console.WriteLine("Test1 passed");
return;
}
else if (Is_A && !Is_B && Is_C)
{
Console.WriteLine("Test2 passed");
return;
}
else if (!Is_A && !Is_B && Is_C)
{
Console.WriteLine("Test3 passed");
return;
}
else if (Is_A && Is_B && !Is_C)
{
Console.WriteLine("Test4 passed");
return;
}
else if (!Is_A && Is_B && !Is_C)
{
Console.WriteLine("Test5 passed");
return;
}
else if (Is_A && !Is_B && !Is_C)
{
Console.WriteLine("Test6 passed");
return;
}
else if (!Is_A && !Is_B && !Is_C)
{
Console.WriteLine("Test7 passed");
return;
}It has been many, many years since I once studied 'truth-tables' and 'decision-tables, etc. And, given a language ... C# ... that optimizes boolean expressions so that in "A && B && C" if 'A is not true, there is no evaluation of 'B and 'C ...
Assuming that any variable in the set of variables is as likely to be true or false in any given use case, it, of course, would not make any difference in what order you wrote out the tests. But, what interests me is how, when you have a sense of the probability distribution of evaluations, you might optimize.
So if you think that a test of 'A' is modal in your use-case, you move the evaluation of 'A to the front of the test; if you think evaluation of 'A is an edge-case infrequently evaluated, you write it last.
Where I am 'blocked' is in coming up with an algorithm to make a pass over the complete matrix of possibilities and optimize the tests based on some user supplied 'weighting.'
Perhaps the user of the code-generator should supply a string like "C,B,A" as its input parameter, and then interpret that as the order to write out the terms ?
Appreciate your opinions, and advice.
best, Bill
p.s. While the current code-generator is not Linquified, I'll probably end up going there. And I am also considering it might be better to generate a switch statement of the form:
switch (theSelector)
{
case "TTT:
break;
case "TFT:
break;
... more cases ...
case default;
break;
}
"In the River of Delights, Panic has not failed me." Jorge Luis Borges
modified on Wednesday, August 17, 2011 1:49 AM
|
|
|
|
|
Hey Bill, why not generate a pattern like:
if (Is_A)
{
if (Is_B)
{
if (Is_C)
{
Console.WriteLine("Test 0 passed");
}
else
{
Console.WriteLine("Test 1 passed");
}
}
else
{
if (Is_C)
{
Console.WriteLine("Test 2 passed");
}
else
{
Console.WriteLine("Test 3 passed");
}
}
}
else
{
if (Is_B)
{
if (Is_C)
{
Console.WriteLine("Test 4 passed");
}
else
{
Console.WriteLine("Test 5 passed");
}
}
else
{
if (Is_C)
{
Console.WriteLine("Test 6 passed");
}
else
{
Console.WriteLine("Test 7 passed");
}
}
}
I didn't check whether they have the same test numbers, but the idea should be clear - instead of evaluating O(2^k) things, evaluate only O(k)
The switch should be even faster, provided you use integers.
|
|
|
|
|
Hi David, excellent feedback, thanks !
I did consider writing out the nested form as you present here, but, given I was just getting into the arcane world of *.tt file editing in VS Studio, and T4 syntax (coming to it without a background in ASP.NET content/code escape syntax conventions), decided it was easier to write the more 'linear' output first. Nested 'style' is coming.
Yes, I like your idea of using integers for the conditions in a switch statement: I'm already using those values in the bit-shifting I'm doing in the code-generator, anyway. See any downside in nested switch statements ?
If the code-generator writes out a comment in each clause documenting implicit meaning of the integer in terms of the boolean variables, that's enough for code maintenance ?
I kind of like the idea of dual approach: perhaps during development and testing using the 'linear form' expression might be productive, while using the 'compiled-to-switch denser' form for possible greater execution speed ?
best, Bill
"In the River of Delights, Panic has not failed me." Jorge Luis Borges
|
|
|
|
|
Nothing like writing out by hand an example of what you want, and testing it against all possible inputs, to make you aware of issues you'll face in generating it auto-magically ... 
Wish I had the IL skills to look at how the compiler renders this and evaluate for efficiency. Is falling through to the 'default case' of a switch statement exactly the same as skipping over an 'if clause that has a 'return inside it ? I'd guess so.
Some obvious refactoring 'cries out' to be done in the 'brute force' example included here: mmm ... dare I utter the forbidden name: 'G*T*' ?
Would you really want to have beasts like this at large in your applications:
public static class NestedSwitch
{
public static string Test4(bool A, bool B, bool C, bool D)
{
int intSelectorA = A ? 8 : 0;
int intSelectorB = B ? 4 : 0;
int intSelectorC = C ? 2 : 0;
int intSelectorD = D ? 1 : 0;
switch (intSelectorA)
{
case 8:
switch (intSelectorB)
{
case 4:
switch (intSelectorC)
{
case 2:
switch (intSelectorD)
{
case 1:
return "ABCD";
default:
return "ABC!D";
}
default:
switch (intSelectorD)
{
case 1:
return "AB!CD";
default:
return "AB!C!D";
}
}
default:
switch (intSelectorC)
{
case 2:
switch (intSelectorD)
{
case 1:
return "A!BCD";
default:
return "A!BC!D";
}
default:
switch (intSelectorD)
{
case 1:
return "A!B!CD";
default:
return "A!B!C!D";
}
}
}
default:
switch (intSelectorB)
{
case 4:
switch (intSelectorC)
{
case 2:
switch (intSelectorD)
{
case 1:
return "!ABCD";
default:
return "!ABC!D";
}
default:
switch (intSelectorD)
{
case 1:
return "!AB!CD";
default:
return "!AB!C!D";
}
}
default:
switch (intSelectorC)
{
case 2:
switch (intSelectorD)
{
case 1:
return "!A!BCD";
default:
return "!A!BC!D";
}
default:
switch (intSelectorD)
{
case 1:
return "!A!B!CD";
default:
return "!A!B!C!D";
}
}
}
}
}
}
"In the River of Delights, Panic has not failed me." Jorge Luis Borges
|
|
|
|
|
I'm pretty sure the compiler is going to be silly about it and not optimize it (edit: that is to say, the nested switches) at all, but I would suggest this:
int all = intSelectorA | intSelectorB | intSelectorC | intSelectorD;
switch (all)
{
case 0:
return "!A!B!C!D";
....
case 15:
return "ABCD";
}
|
|
|
|
|
Thanks, David, I'm going to take a break, and enjoy what's left of the day here at GMT + 7, and then think about your comments, and Luc's comments, tonight, or early am tomorrow.
I feel pretty clear that I want to implement something with both a 'verbose' form for development and testing, and a 'mean-lean' form for final use. So any further thoughts about what would be the form(s) you'd like to see would be welcome.
I am a devotee of the cult of making generic dictionaries with keyvalue pairs whose keys are whatever and whose values are executable code: I'll have to see if I can bootleg that in here 
best, Bill
"In the River of Delights, Panic has not failed me." Jorge Luis Borges
|
|
|
|
|
For mean and lean I'd definitely go for the Single Big Switch, it will be compiled to a jump table so it's constant time and without the often-mispredicted branches that the Tree Of If's would have.
As for a dictionary with executable values, yes that'll work.. you could just make it an array though, because the key would just be an integer in the range 0 - 2^k without holes.
I don't expect that to be any faster than the Single Big Switch though because it's essentially doing the same thing, only the switch has less overhead because the jump table would contain the address directly rather than references to objects that contain the address (as in an array of delegates)
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
