|
yep. No big surprise, except locking the classifiers makes no sense to me, as it kills all parallelism and I don't see the need. IMO you either did something wrong, or forgot to tell me something...
did you compact the data with shorts?
Luc Pattyn
I only read code that is properly indented, and rendered in a non-proportional font; hint: use PRE tags in forum messages
Local announcement (Antwerp region): Lange Wapper? 59.24% waren verstandig genoeg om NEEN te stemmen; bye bye viaduct.
|
|
|
|
|
Yes, I used ushorts for all the data. I can't say exactly what the effects were for the classifiers, but making the image data ushorts definitely helped.
Do you maybe know how I could use parallism without locking the data? Locking it seems the only way to keep it safe, and like you say, nullifies the point of using parallism...
tvb
|
|
|
|
|
tvbarnard wrote: how I could use parallism
I already told you I would use N threads (N=#cores), and give them 1/Nth of the classifiers each. I have no experience with Parallel.For
Luc Pattyn
I only read code that is properly indented, and rendered in a non-proportional font; hint: use PRE tags in forum messages
Local announcement (Antwerp region): Lange Wapper? 59.24% waren verstandig genoeg om NEEN te stemmen; bye bye viaduct.
|
|
|
|
|
Yeah, was actually asking if you could elaborate on that, but now that you say it like that I understand 
tvb
|
|
|
|
|
tvbarnard wrote: the system uses at least 200,000 classifiers
what is a classifier? what thing in your source code equals 200,000?
can you please explain, maybe give some numeric examples, I want to see to what extent "locality of data" applies.
and how long does it take to do all you want done for just one image?
Luc Pattyn
I only read code that is properly indented, and rendered in a non-proportional font; hint: use PRE tags in forum messages
Local announcement (Antwerp region): Lange Wapper? 59.24% waren verstandig genoeg om NEEN te stemmen; bye bye viaduct.
|
|
|
|
|
Sorry for the confusion.
In context of my system, a classifier would describe the way an image is evaluated and "classified" to contain a certain object in the image.
In my case, a classifier is a set of points in the image that respectively either get added or subtracted and thus return a real value. Using a threshold one can then decide accroding to the real value, whether the image represents that object or not.
For example, say you have a classifier:
int[] classifier = new int[] { 5, 6, 22}
then the image would be evaluated by adding all pixel values at coordinates {5, 6, 22} (assuming the image has been reshaped as a vector, and not a matrix).
Hope this is clear enough!
The point of the whole excercise, is to evaluate a set of classifiers on all images (each) and determine which is the best classifier. So what makes the excercise so computationally intensive, is that there exist at least 200,000 classifiers.
To answer your question on how long it takes to do an image, I haven't time it since it doesn't take very long. But the problem becomes exponentially more difficult if you increase your amount of images to let say 3000. I would estimate 3000 images, with 200,000 classifiers takes about 4min. 4 min doesn't sound long, but this must be done 500 times 500x4min = 33hrs!
The reason why I need to optimize this, is because firstly, it's not suppose to take so long (many papers report much faster times), and I also don't personally have the time to wait 30+hrs.
tvb
|
|
|
|
|
Sorry, I still:
- don't understand how your textual explanation fits your code;
- what "f" represents in your code;
- which array/list/whatever holds a classifier;
- and now also where that "500" is coming from.
And I fail to understand it at the higher level, i.e. what is really happening.
But I gave you another reply about cache efficiency and parallelism errors. Maybe that helps you.
If you need more advice, make sure to show sufficient code and provide explanations that refer to actual variables; in my experience, if you fail to explain it properly, chances are huge that you will fail to make it run efficiently too.
Luc Pattyn
I only read code that is properly indented, and rendered in a non-proportional font; hint: use PRE tags in forum messages
Local announcement (Antwerp region): Lange Wapper? 59.24% waren verstandig genoeg om NEEN te stemmen; bye bye viaduct.
|
|
|
|
|
Very sorry, the problem is that the theory behind the code is very complex, so I try not to refer to the theory too much.
To answer:
1. A classifier is exactly the same as a feature, hence I use the terms interchangeably (sorry bout that)
2. The 500 refers to the amount of iterations the entire process must be carried out (where the entire process refers to the evaluation of all classifiers/features on all images)
Thanks for your willingness to help (I know it's a bit confusing)
If you give me a few minutes I'll come up with a more generic/descriptive code example for you.
tvb
|
|
|
|
|
Sorry, to answer your questions:
I load the images at the start of the program using pointers to traverse the image and store each pixel into the array (one by one). After that, I never reference the files themself again, so how I load the data shouldn't make any difference.
tvb
|
|
|
|
|
tvbarnard wrote: 2. Parrallel processing the images (but I ended up with errors).
What were the errors? How did you code it? Did you use the Parallel Framework[^]
only two letters away from being an asset
|
|
|
|
|
I am not sure which parallel libraries you are refering to, but to be specific, I used the following line of code:
Parallel.For(0, bound, delegate(int i)
{
);
I didn't get errors in compiling, just got errors in my results.
I know my results were wrong, because the results I got were immediately different.
Any other parallel approaches you would suggest?
I also tried using normal referencing instead of pointers but I still got incorrect results.
tvb
|
|
|
|
|
Only one Parallel Framework I'm aware of.
only two letters away from being an asset
|
|
|
|
|
A couple of suggestions...
It takes a lot longer to process doubles than ints. Can you use ints for pixels?
A good algorithm for discovering classifiers is the ID3 algorithm (http://en.wikipedia.org/wiki/ID3_algorithm[^]). It analyzes your training set and automatically produces a decision tree that will classify a new case. Each branch of the decision tree will compare a feature (in this case a pixel) of the new case with a threshold.
|
|
|
|
|
Oh yes, already did that 
Thanks, will have a look at the ID3. Unfortunately I'm running out of time, so can't change or add too much code.
tvb
|
|
|
|
|
what is their functions? I am completely confused.
When sending arp message in LAN,Winpcap is very nessecery?
What is about it?
any one help me?
thanks in advance!
|
|
|
|
|
I've had a look at the documentation on their site, and it appears to be all C++ not C#. You will need to write a wrapper and use PInvoke to call the functions and write equivalent data structures etc so that data can be marshalled correctly, unless there is already a .NET wrapper for it such as this[^].
|
|
|
|
|
There is a nice WinPcap wrapper you should try to use.
It uses C++/CLI instead of PInvoke, which has much better performance.
It is called Pcap.Net and available here:
http://pcapdotnet.codeplex.com
|
|
|
|
|
Hi guys.
I have a VPN connection to the Internet with 14kb/s. It's too low, but I can setup several VPN connections and it causes the speed increase ( 14kb/s per each VPN connection )
Now I'm going to establish several VPN connections to the Internet and configure them with C# !
Could you please guide me, how I can do it ?
Thanks.
|
|
|
|
|
can i go to next page in a multi pages crystalreport without using next page button just scrolling down and next pages will come one by one.....how?????
|
|
|
|
|
I run D:\Program Files\Microsoft Visual Studio 9.0\VC>Installutill.exe C:\Winservice\Release\LOG.exe to install windows service ,but The following problem occurred:
The Rollback phase completed successfully .
The transacted install has completed.
The installation failed, and the rollback has been performed.
------------
the installer class is as below:
public partial class Installer1 : Installer
{
ServiceInstaller ITServersLOG;
ServiceProcessInstaller ICCIMProcessInstaller1;
public Installer1()
{
InitializeComponent();
ITServersLOG = new ServiceInstaller();
ICCIMProcessInstaller1 = new ServiceProcessInstaller();
ICCIMProcessInstaller1.Account = ServiceAccount.LocalService ;
ITServersLOG.StartType = ServiceStartMode.Manual;
ITServersLOG.DisplayName = "ICCIMLOGService";
ITServersLOG.ServiceName = ITServersLOG.DisplayName;
Installers.Add(ITServersLOG);
Installers.Add(ICCIMProcessInstaller1);
}
}
how can i solve it?!! please help me!! 
|
|
|
|
|
how to searche in datagrid word to word
|
|
|
|
|
Does this[^] helps?
It's not necessary to be so stupid, either, but people manage it. - Christian Graus, 2009 AD
|
|
|
|
|
Hi
i've use TableAdapter (typed dataSet) in my app, i extent my business layer to validate some information. when data is not valid then throw an exception.
now i want to handle this error in presentation layer. for example display error value in red color in datagridview, but i don't know how to do that.
can anybody help me to handle it in presentation layer ?
thanks
|
|
|
|
|
What information are you passing with the exception back to the presentation layer? I suppose there is any unique identifier that can help you identify the row having faulty data. Then, using Cells.Style property, you can set the fore color of the cell.
It's not necessary to be so stupid, either, but people manage it. - Christian Graus, 2009 AD
|
|
|
|
|
hello
i want to ask, how i can get the current date format?
thx
Thaer
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
