C# Discussion Boards - CodeProject

Christian Graus5-Mar-09 12:43

Christian Graus

5-Mar-09 12:43

Doing what he suggested in your instance, makes you an idiot. If you couldn't understand my answer, then you're an idiot. You sure as hell should not be writing any code that anyone is paying for, or that will be used anywhere.

Christian Graus

Driven to the arms of OSX by Vista.

_Maxxx_5-Mar-09 14:09

_Maxxx_

5-Mar-09 14:09

I was about to 1-vote you for demeaning this person - then I saw his user details, and he is a software developer! (I had sort of assumed he was a kid with little or no experience)

Have a 5 instead!

___________________________________________
.\\axxx
(That's an 'M')

DaveyM696-Mar-09 1:03

DaveyM69

6-Mar-09 1:03

I agree totaly - I was working on the assumption that he *really* wanted to raise the event again (for reasons unknown) as he said in the question.

Of course, if all he wants to do is re-run the handling code for the Enter event, all he needs to do is call that method, or move the code out to a common method and call that from both handlers.

(Btw, TextBox.Enter and Button.Click do both have the same signature.)

Big Grin | :-D

Dave
BTW, in software, hope and pray is not a viable strategy. (Luc Pattyn)
Visual Basic is not used by normal people so we're not covering it here. (Uncyclopedia)

2 way Remoting from client to installed service and Access Denied issue

E_Gold7-Mar-09 19:16

E_Gold

7-Mar-09 19:16

Hi people, why you so angry ??
isn't this forum for asking question ?
English in not my mother tongue and maybe i don't explain my self well
if someone uninterested to help - don't help !!
Mad | :mad:

carbon_golem5-Mar-09 7:16

carbon_golem

5-Mar-09 7:16

I have a publish/subscribe application running on a single machine.

The system uses 2 ipc remoting channels, one set up on the installed service (which has no issue) and another set up on a client.

The client sends uri information to the service via the service's remoting channel so telemetry can be sent to the observing client. I can connect to the service and do everything I need to do with it's ipc channel, but when I try to connect to the client's channel I receive an Access Denied error.

The service is installed as "Local System" permission. I presume that this is a security issue but I can't seem to get it right... Any suggestions?

Thanks!

Scott P

"Simplicity carried to the extreme becomes elegance."
-Jon Franklin

Re: 2 way Remoting from client to installed service and Access Denied issue

led mike5-Mar-09 9:01

led mike

5-Mar-09 9:01

carbon_golem wrote:
I presume that this is a security issue

carbon_golem wrote:
Any suggestions?

My first suggestion is don't presume. Perhaps you could configure your system so that all it's processes are running under the same account and verify that the error is resolved under those conditions. If it is, some form of that could be a solution, correct?

Re: 2 way Remoting from client to installed service and Access Denied issue

carbon_golem5-Mar-09 9:11

carbon_golem

5-Mar-09 9:11

led mike wrote:
My first suggestion is don't presume.

100% correct. I should UNlearn what I have learned.

It turned out that it WAS and WASN'T a security issue. I set the default security with the way I created the IpcChannel. I had to set it up using Hashtable props that include the security settings. If I hadn't taken the shortcut and did it the right way I would have spotted this right off.

Thanks for the help, I appreciate it!

Scott P.

"Simplicity carried to the extreme becomes elegance."
-Jon Franklin

Re: 2 way Remoting from client to installed service and Access Denied issue - SOLVED

carbon_golem5-Mar-09 9:03

carbon_golem

5-Mar-09 9:03

I don't know why I didn't try this before. I had the client publishing it's IpcChannel with it's string uri only, and this won't work apparently if there's any kind of security needed. You need to specify the dictionary definitions when you create the channel as follows:

public static void Publish() {
            var serverProvider = new BinaryServerFormatterSinkProvider();
            var clientProvider = new BinaryClientFormatterSinkProvider();
            IDictionary props = new Hashtable();
            serverProvider.TypeFilterLevel = TypeFilterLevel.Full;
            props["name"] = "Client";
            props["portName"] = LogOnInformation.LogOnChannel;
            props["typeFilterLevel"] = "Full";
            props["exclusiveAddressUse"] = "false"; 
            props["authorizedGroup"] = "Everyone";
            props["rejectRemoteRequests"] = "true";
           
            var chan = new IpcChannel(props, clientProvider, serverProvider); // this has to be unique
            ChannelServices.RegisterChannel(chan, false);

            client = new LocationViewProxy(LogOnInformation);
            RemotingServices.Marshal(client, LogOnInformation.LogOnHost, typeof(LocationViewProxy));
        }

Thanks to those of you who looked.

Scott P.

"Simplicity carried to the extreme becomes elegance."
-Jon Franklin

Is the a max row count for SQLDataReader?

Ben Cocker5-Mar-09 6:22

5-Mar-09 6:22

Hi there,

I am reading a (rather large) MS-SQL stored procedure into C# using the SqlDataReader. I expect this to return slightly over 8,000,000 records, but it consistently maxes out at 726,301. I cannot for the life in me see why - when running the stored procedure (which has no input parameters) in SQL Server Management Studio it returns the correct number of records.

I am running VS 2008 Professional with SQL Server 2008 Developer Ed. The code is paraphrased as follows:

// Set up a list of data type Record (my own) to hold the output of the sp
List<record> records = new List<record>();

System.Data.SqlClient.SqlConnection Connection = new
System.Data.SqlClient.SqlConnection(ConnectionString);
System.Data.SqlClient.SqlCommand Command;

try
{
Connection.Open();
{
Command = new System.Data.SqlClient.SqlCommand("StoredProcedure", Connection);
Command.CommandType = CommandType.StoredProcedure;
System.Data.SqlClient.SqlDataReader Reader = Command.ExecuteReader();

while (Reader.Read())
{
Record NewRecord = new Record();

NewRecord.PAR1 = Reader.GetInt32(Reader.GetOrdinal("PAR1"));
NewRecord.PAR2 = Reader.GetInt32(Reader.GetOrdinal("PAR2"));... etc

records.Add(NewRecord);
}
}
}
catch
{
}
finally
{
Connection.Close();
}

Thanks a million for any help!

Ben

Luc Pattyn5-Mar-09 6:34

5-Mar-09 6:34

Hi,

why don't you catch an Exception and look at all its information, using Exception.ToString().
Chances are the problem has been reported but you showed no interest.
Could be a timeout, an out-of-memory condition, ...
Since the number of rows returns is constant, I do expect an OOM.

BTW: if 8 million is what you expect, how many records are there?
And if you were to want all existing records, why use a SP in the first place?

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

- before you ask a question here, search CodeProject, then Google
- the quality and detail of your question reflects on the effectiveness of the help you are likely to get
- use the code block button (PRE tags) to preserve formatting when showing multi-line code snippets

modified on Sunday, June 12, 2011 9:07 AM

Ben Cocker5-Mar-09 6:48

5-Mar-09 6:48

Hi Luc,

Thanks for your reply. In answer to your questions: I don't get an exception - it just terminates the while(Reader.Read()) loop at 7263001 records. I have actually slightly increased the field count of the query (which makes it also take much longer) - but this doesn't affect the number of records it tops out at, which made me think that timeout and oom might not be the reason.

I used a sp in order to be able to test and maintain the query on the server rather than just having a long text string select statement in my C# code.

Cheers, Ben

Luc Pattyn5-Mar-09 7:14

5-Mar-09 7:14

Hi Ben,

Ben Cocker wrote:
I don't get an exception

you may have misunderstood me. What I meant is your code (catch{}) just swallows exceptions if and when they occur, so it may seem you don't get any. Did you try a

...
   catch(Exception exc) {
       log(exc.ToString());
   }
...

where log is some logging method, a Console.WriteLine, a MessageBox.Show, whatever you fancy.

Also, I'm not familiar with the Record class, is it standard .NET stuff or is it yours? any idea what the size of its objects is? I hope it isn't duplicating information, such as table metadata all over.

BTW: "only 1.5GB" (from your other reply) is a phrase I never used so far

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

modified on Sunday, June 12, 2011 9:07 AM

Luc Pattyn5-Mar-09 8:04

5-Mar-09 8:04

Hi Ben,

adding a field to the query does not change the memory situation if that field eventually gets stored in a value type inside Record, such as an int or Int32, since such members get allocated memory whether you fill them or not.

If adding some NEW members to the Record class still keeps it failing at record 726,301 then I might be inclined to say something is abnormal with that record (say a NULL value, so your GetInt32 fails).

My odds are still very much in favor of O-O-M though.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

modified on Thursday, March 5, 2009 2:34 PM

Jon Rista5-Mar-09 10:27

Jon Rista

5-Mar-09 10:27

I have to agree with Luc, I think your running into a problem on that particular record, its throwing an exception, and the exception is getting swallowed. Stick a breakpoint in your catch clause and see what is going on.

Luc Pattyn5-Mar-09 10:45

5-Mar-09 10:45

... and never ever swallow exceptions like that. Sigh | :sigh:

Luc Pattyn [Forum Guidelines] [My Articles]

modified on Sunday, June 12, 2011 9:08 AM

Jon Rista5-Mar-09 12:18

Jon Rista

5-Mar-09 12:18

....never ever.....ever.....ever. Suspicious | :suss:

Ben Cocker5-Mar-09 22:54

5-Mar-09 22:54

Guys,

I've been astonished by the help I've received with this problem - thank you very much everyone.

I'm afraid to say that the problem was as stupid as me consuming the exception on a particular record. Best practice point to self, and I'm really sorry for wasting peoples time on such a simple thing. I normally go out of my way to Google for problems, but since 726301 is not a magic number/ceiling (as it was just an exception) I was really scratching my head over this one without seeing the obvious.

Anyway, thanks again for everyone's time, and for the suggestions.

Ben

Jon Rista5-Mar-09 6:35

Jon Rista

5-Mar-09 6:35

Well, my guess is that the sheer number of records your processing in-memory is just flat out insane. :P

The problem is most likely not a row count limitation of SqlDataReader...but rather a memory limitation. You are trying to create a massive List of Record objects...the overhead for that is going to be fairly high compared to just displaying rows of text in, say, Sql Management Studio. Depending on exactly how many columns you are setting on your Record object, the data types of each, and the lengths of strings that may exist in your result set...you could be looking at GIGS of memory usage here.

I am not exactly sure what your doing or what your needs are...but you should look into batching your work. Either chunk it up into significantly smaller data sets (say, 10000 records each) and process them one at a time...or distribute those chunks out to a server farm to process them in parallel. If you absolutely need to process all 8 million records at once...then you are probably doing something that calls for LAPACK or some other matrix or vector processor that can efficiently handle large data sets.

Ben Cocker5-Mar-09 6:54

5-Mar-09 6:54

Hi Jon,

Thanks for your reply, and the suggestions. Could I point you to my reply to Luc for a discussion of this out-of-memory limitation? And, I have been very careful to limit the number of fields I use and don't have any string columns, so although I take your point that 8 million rows is a beast, in SQL management studio the size of comma delimited output file is less than 1.5gig - I've got a 32 bit machine with maximum memory so I am skating on thin ice, but still I expected it to work out memory wise.

I'll look into LAPACK - thanks for the suggestion.

Ben

Dave Kreskowiak5-Mar-09 7:40

Dave Kreskowiak

5-Mar-09 7:40

Ben Cocker wrote:
SQL management studio the size of comma delimited output file is less than 1.5gig

So what? That's a file on disk that doesn't have the overhead of a storage class managing the data. THere is no limit on the Reader, since it only reads a single record at a time and returns the fields in that record. The real question is where is this data going?? Is it going into an ever-expanding collection?? If so, then you're running the machine out of memory.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007, 2008

Dragonfly_Lee5-Mar-09 18:12

Dragonfly_Lee

5-Mar-09 18:12

Well, maybe you are confused about the datareader and dataset in ADO.net. DataReader object just go forward for ONE record each time. Generally, it will not take you too much memory for only one record, right?

I Love KongFu~