C# concurrentqueue shows out of memory exception after running for a few hours

Question

1.00/5 (1 vote)

See more:

Hi. I am working with data that is generated every second. I am trying to use ConcurrentQueue to enqueue those contents. I have added a Task which checks whether the queue count is greater than 500. If it is greater than 500, it dequeues the content to an array and I send the array. This is the how my application works. But after running for a few hours, I got the following message:

at (wrapper alloc) System.Object.AllocVector(intptr,intptr)
  at System.Collections.Concurrent.ConcurrentQueue`1+Segment[T]..ctor (System.Int32 boundedLength) [0x00006] in <f56c876907e742b0aa586f051fcce845>:0 
  at System.Collections.Concurrent.ConcurrentQueue`1[T].EnqueueSlow (T item) [0x00051] in <f56c876907e742b0aa586f051fcce845>:0 
  at System.Collections.Concurrent.ConcurrentQueue`1[T].Enqueue (T item) [0x00010] in <f56c876907e742b0aa586f051fcce845>:0

Please help me out. Give me somme suggestions to solve this problem. Should I use System.Generics.Queue instead of ConcurrentQueue?
I am using the following steps:
1. Read data from serial port and enqueues it to a queue

C#

ConcurrentQueue<int> queue1 = new ConcurrentQueue<int>();
queue1.enqueue(values)

2. To dequeue I use the following Task

C#

public async Task PublishMessage()
{
int waittime =1;
if (waittime != 0)
{
do
{
double[] testarray=new double[512];
if(queue1.Count>=512)
{
wavetest = new double[512];
for (int k = 0; k < 512; k++)
{
queue1.TryDequeue(out testarray[k]); 
}
SendMessage(testarray);
}
await Task.Delay(waittime);

}
while (true);
}

}

Thanks

What I have tried:

I tried seeing if the queue increases with time. But no. The queue count was around 250 every. It didn't increase rapidly with time.

Posted 1-May-18 0:46am

Member 13245297

Updated 1-May-18 6:22am

CHill60

v3

Add a Solution

Comments

Wendelius 1-May-18 6:50am

Can you post the code you use for adding to and removing from queue

Member 13245297 1-May-18 7:21am

I do the following:
1. Read data from serial port and enqueues it to a queue
ConcurrentQueue<int> queue1 = new ConcurrentQueue<int>();
queue1.enqueue(values)
2. To dequeue I use the following Task

public async Task PubMessage()
{
int waittime =1;
if (waittime != 0)
{
do
{
double[] testarray=new double[512];
if(queue1.Count>=512)
{
wavetest = new double[512];
for (int k = 0; k < 512; k++)
{
queue1.TryDequeue(out testarray[k]); 
}
SendMessage(testarray);
}
await Task.Delay(waittime);

}
while (true);
}

}

Patrice T 1-May-18 10:25am

Use Improve question to update your question.
So that everyone can pay attention to this information.

Member 13245297 1-May-18 7:21am

Thanks for replying

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Stryder_1 · Answer 1 · 2018-05-01T06:24:00

Solution 1

There are two things I see that would lead to out of memory:

1. The first is you are allocating a new double[512] every time the do-while loop iterates. If you move that statement to within the if block that checks for count>=512, this will now only allocate memory when you have reached the minimum count.

2. Second, this is an async Task. Is this called more than once? If so, the do-while loop is set up as an infinite loop. Each time this is called, it will start a new loop, but any previous calls will still have their loops running.

I also do not see a need for the line "wavetest = new double[512];"

Posted 1-May-18 6:24am

Stryder_1

Comments

Member 13245297 1-May-18 13:31pm

Hi. Thanks for pointing that mistake. I will rectify it. I couldn't understand the second point properly. Right now,I am actually starting the task PubMessage when my program begins execution and it will check whether the condition(Count>512) is met every milli second and if the condition is met it will send the message. So my main program will enqueue all the data points received every milli-second and the task PubMessage will periodically dequeue the contents and send it. You have any suggestion to optimize my solution? Thanks

Stryder_1 2-May-18 12:06pm

Since you only call this function once in your application, you probably will not be concerned with point #2. If you want to learn more about it though, search for Asynchronous programming.

From what I can tell, you do not really need a new array allocated with each minimum count. Instead, reuse the array you have already created. You would declare your array before the do-while loop. Then instead of instantiating an array with each loop, reset the values in the array to a number you would not receive from the serial port. For example, if you do not expect negative values, you could initialize each array value with -1. This will eliminate the allocation/deallocation processing for each loop iteration.