Click here to Skip to main content
15,887,886 members
Home / Discussions / C#
   

C#

 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
Gerry Schmitz10-Jan-16 9:04
mveGerry Schmitz10-Jan-16 9:04 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
JD8610-Jan-16 16:33
JD8610-Jan-16 16:33 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
Gerry Schmitz10-Jan-16 17:40
mveGerry Schmitz10-Jan-16 17:40 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
JD8611-Jan-16 5:01
JD8611-Jan-16 5:01 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
JD8612-Jan-16 6:24
JD8612-Jan-16 6:24 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
Luc Pattyn12-Jan-16 15:50
sitebuilderLuc Pattyn12-Jan-16 15:50 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
JD8612-Jan-16 16:29
JD8612-Jan-16 16:29 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
Luc Pattyn12-Jan-16 17:16
sitebuilderLuc Pattyn12-Jan-16 17:16 
Hi,

thanks for all that info. Here are 3 remarks:

1) changing from explicit Dispose() calls to using constructs will not solve the problem by itself; it merely results in simpler code that is less likely to contain errors. So it should not be a priority.

2) I have an experiment I want to suggest to you with high priority: your code contains some 16 instances of
List<something> somename=new List<something>();


I want you to replace those by:
List<something> somename=new List<something>(someconstant);

i.e. specify an initial capacity for each and every list, where someconstant is a small factor larger than your current number of items; also preferably do not use more than 2 different values. So I'd suggest 0x00400000 for Message Tracking Logs and Mailbox Sizes, and 0x00040000 for everything else; yes those are hex numbers for a reason, and they exceed 4 million and quarter of a million respectively. The idea is to have the Lists of sufficient size right away (this will save a lot of data copying while populating an currently also growing the lists), and to improve the probability the same memory chunks will be used for newer generations of those lists.

This may seem mysterious, the rationale is: a List internally has an array holding the items; an array is an object; objects larger than around 80KB are kept in what is known as the "Large Object Heap". That heap used to have very poor memory management in the early days of .NET (say .NET 2.0); the issue was kept quiet by MS as long as possible, but it was (maybe or probably still is) a weakness of the .NET memory management. It has been said it got improved, however I don't know whether it has, how, and by how much. BTW: I expect behavior to be better on a 64-bit application.) If my suggested experiment behaves much better, it more or less proves the issue still exists and is relevant to your application.

Warning: the memory usage may initially be higher, however the "climbing steadily" should disappear completely.

3) I discovered one thing that is wasteful, and fixing that would then probably be the final solution I'd suggest: in your scenario you first build a list of all messages (sent, received), then repetitively use LINQ to reduce that list to a smaller list, and then basically are interested in the count of that smaller list. With a little effort, you could write a method that doesn't build the big list, it could create a list of lists (one for each user) hopefully resulting in many lists each of them much smaller than the current big list (a list with fewer than 10K items is bound to be managed well on the regular heap). If needed we can discuss this more later on, after we have the results from the experiment under (2).

That's it for now.

Smile | :)
Luc Pattyn [My Articles] Nil Volentibus Arduum

GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
JD8612-Jan-16 17:30
JD8612-Jan-16 17:30 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
Luc Pattyn12-Jan-16 17:46
sitebuilderLuc Pattyn12-Jan-16 17:46 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
JD8614-Jan-16 4:07
JD8614-Jan-16 4:07 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
JD8614-Jan-16 4:26
JD8614-Jan-16 4:26 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
JD8614-Jan-16 14:05
JD8614-Jan-16 14:05 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
Luc Pattyn14-Jan-16 18:26
sitebuilderLuc Pattyn14-Jan-16 18:26 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
JD8617-Jan-16 16:09
JD8617-Jan-16 16:09 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
Luc Pattyn17-Jan-16 17:10
sitebuilderLuc Pattyn17-Jan-16 17:10 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
JD8617-Jan-16 17:21
JD8617-Jan-16 17:21 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
Luc Pattyn17-Jan-16 18:32
sitebuilderLuc Pattyn17-Jan-16 18:32 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
JD8619-Jan-16 3:26
JD8619-Jan-16 3:26 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
JD8621-Jan-16 4:04
JD8621-Jan-16 4:04 
GeneralRe: Memory leak trouble with Linq to Sql and multiple threads Pin
Luc Pattyn21-Jan-16 15:06
sitebuilderLuc Pattyn21-Jan-16 15:06 
Questionbetter way to Linqify this ? Pin
BillWoodruff9-Jan-16 2:01
professionalBillWoodruff9-Jan-16 2:01 
AnswerRe: better way to Linqify this ? Pin
Sascha Lefèvre9-Jan-16 2:25
professionalSascha Lefèvre9-Jan-16 2:25 
GeneralRe: better way to Linqify this ? Pin
BillWoodruff11-Jan-16 8:06
professionalBillWoodruff11-Jan-16 8:06 
GeneralRe: better way to Linqify this ? Pin
Sascha Lefèvre11-Jan-16 8:37
professionalSascha Lefèvre11-Jan-16 8:37 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.