|
I followed Your advice and went for SQLite database and entity framework EF6 and the total time needed for the same same code is now 1400 ms vs 16 sec before!
It took some time to learn and implement EF(hence the late reply) but it was worth the efford 
Thank You,
Groover
0200 A9 23
0202 8D 01 80
0205 00
|
|
|
|
|
Glad it worked out for you!
Didn't mean to say you should use EF; just that more "benchmark numbers" would be useful.
But yes; EF lets us think a little less about data access strategies and avoid some mistakes that can cause performance problems.
If you haven't already, these were good starting links for EF:
Entity Framework (EF) Documentation[^]
|
|
|
|
|
Only a detail
replace
for (int i = 0; i < length; i += 8)
{
int colNum;
if (i == 0) colNum = 1;
else colNum = i / 8 + 1;
with
int colNum=0;
for (int i = 0; i < length; i += 8)
{
colNum ++;
It will not really change the overall timing, but should improve a little.
Patrice
“Everything should be made as simple as possible, but no simpler.” Albert Einstein
|
|
|
|
|
You are absolutely right, I did mention that I write crappy code.
changed it to:
int colNum = 0;
for (int i = 0; i < length; i += 8)
{
colNum = i / 8 + 1;
and zero divided by 8 gives zero, so no need for checking that. 
Thanks,
Groover
0200 A9 23
0202 8D 01 80
0205 00
|
|
|
|
|
Your change correct, but mine is faster.
Patrice
“Everything should be made as simple as possible, but no simpler.” Albert Einstein
|
|
|
|
|
Hi guys,
So I have an object that I'm exposing via .NET remoting (for historical / political reasons I can't just drop this and go to something more modern), which I use to process a list of items, one call per item in my list. The remoting server is set up as Single call rather than Singleton, and so my understanding is that I should get a new instance of the server class on each invocation - i.e. for each item in my list.
This seems to work just fine for small datasets, but when I push it a little by processing a list containing approx. 200K items, it reaches around 19K processed, I see a timeout, and the call fails with a socket exception - "A connection attempt failed because the connected party did not properly respond after a period of time or established connection failed because connected host has failed to respond". The next call then succeeds and processing continues...but then fails again later.
There seems to be no significance in the number of records processed - this varies from around 19K to 19.1k and fails at different points and with different records.
Additionally, when I run the system in multi-threaded mode (Parallel.ForEach() & MaxDegreeOfParallelism == -1), then I'm seeing the same "profile" of failures, but instead of a single record fail followed by the next "working", I'm seeing about 10 failures before it comes back to life - which leads me to believe it's something like the server actually dying and being re-instantiated.
Anyone have any ideas? I can batch out the data and get a "new" remoting connection if I create a factory to do so (I'm injecting my remoting client), but this seems like avoiding the problem rather than understanding the cause and fixing it. Ultimately this will be used to process > 1.2M records
Cheers guys!
C# has already designed away most of the tedium of C++.
|
|
|
|
|
I would pass the whole lot of records at one time. Not only is it going fubar after 19k records, but if you process 1.2M records, imagine how much time you are adding with the network latency between each record.
The difficult we do right away...
...the impossible takes slightly longer.
|
|
|
|
|
You should open this as a ticket to your network providder
|
|
|
|
|
I am currently working on monitoring the folder on desktop. When the hex data files are dragged into the "Monitored" folder, it would automatically start to analyze the hex files and create the interpreted files into other folder.
I have created the Local Service process "Interpreter Service". The thing is that when I started the service then it abruptly stopped running. Upon checking Event Viewer, it reported the error:
Service cannot be started. System.ArgumentException: The directory name C:\Users\RBWorkstation\Desktop\Monitored Folder\ is invalid.
at System.IO.FileSystemWatcher.set_Path(String value)
at InterpreterService.UM_InterpreterService.OnStart(String[] args)
at System.ServiceProcess.ServiceBase.ServiceQueuedMainCallback(Object state)
The folder is right on my desktop! So anyone have any suggestion on how to fix this so this service can find the folder?
Thanks!
~ Ron Boucher
|
|
|
|
|
What account is your service running as? Does it have access to the folders in your profile?
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Help me to clarify the "account" you are speaking of? I'm on Administrator account.
|
|
|
|
|
Yes, but what account is the service running as?
When you created the service installer, you selected an account for the service to run as. That account needs to have sufficient permissions to access the folder it's trying to monitor.
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
As I said on the first post: Local Service.
|
|
|
|
|
There are multiple accounts on your machine. Regardless of you logging in as admin, normal services do NOT run with those rights. Open the services, find yours, right-click, and check which user.
Chances are it's "SYSTEM", wich does not have the same rights as your admin-account. The reason for that is that the service runs before someone (like you as admin) can loging and prove who he/she is - the service will already be started with the credentials provided.
Bastard Programmer from Hell 
If you can't read my code, try converting it here[^]
|
|
|
|
|
Ah, it wasn't clear that that was the account; I though you were just describing the type of service.
The Local Service account "has minimum privileges on the local computer and presents anonymous credentials on the network"[^]. It most likely doesn't have permission to access a folder within your profile.
Try changing it to run as Local System, or as your user account.
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Try running the service under Administrative account
|
|
|
|
|
please reply this question
|
|
|
|
|
Apparently you already know that C# is a programming language, or else you wouldn't have had the idea to ask about it here. So if you know that C# is a programming language, why don't you ask a more specific question that can be answered without writing an essay, which is already written, anyway (you do know Google, right?): C Sharp (programming language) - Wikipedia, the free encyclopedia[^]
If the brain were so simple we could understand it, we would be so simple we couldn't. — Lyall Watson
|
|
|
|
|
He is a new member, he joined about 4 days ago
he is just trying to see how this site works
|
|
|
|
|
Ratul Thakur wrote: He is a new member, he joined about 4 days ago
he is just trying to see how this site works
Apparently his definition of "seeing" is empirical research with extreme input values.
If the brain were so simple we could understand it, we would be so simple we couldn't. — Lyall Watson
|
|
|
|
|
It's a semitone above C natural. It's sometimes known as D flat.
This space for rent
|
|
|
|
|
Bloody musos!
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
PunctuationException: you seem to be missing a comma.
|
|
|
|
|
|
This question has many answers. Can you tell us in what sense you want answer for.
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
