|
Hi,
Thank you for the reply.
I cannot buffer it until I get everything, as it is a logging function and I believe I could end up hogging too much memory.
What is the best practice for receiving chunks of data and then parsing it? Do I have to have synchronization for this on both my PC and microchip end? I was hoping that if I only streamed updated data every 250mS from the source, that I would not encounter issues like this.
|
|
|
|
|
if the data is text and has a line delimiter (one or two characters that won't appear in the middle of the message), I would not use events at all; I'd rather use a Thread or a BackGroundWorker with an eternal loop, performing a SerialPort.ReadLine() and some output operations. That way you get rid of all worries about partial messages, and you don't need any additional buffering. You should set your definition of the message delimiter explicitly through SerialPort.NewLine[^]
BTW: If the delimiter could vary, say \n or \r\n, then you should set the SerialPort to look for the shorter one, and remove the other character from each line you get.
Luc Pattyn [My Articles] Nil Volentibus Arduum
Fed up by FireFox memory leaks I switched to Opera and now CP doesn't perform its paste magic, so links will not be offered. Sorry.
|
|
|
|
|
I will try and implement that.
Thank you Luc.
|
|
|
|
|
Luc,
I think I tried this one before with SerialPort.NewLine and I did
serialPort1.NewLine = "\n";
serialPort1.NewLine = "\r";
serialPort1.NewLine = "\n\r";
serialPort1.NewLine = "\r\n";
And none of them format correctly, the only thing I was able to get working was that regex?
|
|
|
|
|
turbosupramk3 wrote: I think I tried
great piece of FUD.
you should know what will be received and program for that. Then if something else is received, cope with it.
What you must do is include logging in your code, so you can see what you are receiving.
and it makes sense to log each character in 2-digit hex, so everything is clearly identifiable.
Luc Pattyn [My Articles] Nil Volentibus Arduum
Fed up by FireFox memory leaks I switched to Opera and now CP doesn't perform its paste magic, so links will not be offered. Sorry.
|
|
|
|
|
I have nailed the carriage return portion down.
I am having a very hard time with the multi-threaded asynchronous circular buffer though. I cannot picture in my head how this should work from a mid level perspective.
|
|
|
|
|
You didn't mention any circular buffer before, and neither did I.
Luc Pattyn [My Articles] Nil Volentibus Arduum
Fed up by FireFox memory leaks I switched to Opera and now CP doesn't perform its paste magic, so links will not be offered. Sorry.
|
|
|
|
|
Ok, then scrap that, I thought that was where you were pointing me to, something like a stack. I'm glad it was not, because those looked very complicated!
In trying to write this down and conceptualize it first, I'm wondering if this is a good start?
uC -->
pc thread1 that does a serialport.readLine() into a string, then locks a global array1, copies into it, unlocks global array1 and then clears the string -->
pc thread2 that monitors global array1, locks it, copies data to local string or strings, clears and then unlocks global array1, parses data, adds unique single item delimiter and locks then copies to global array2, unlocks global array2, clears local string or strings -->
pc thread3 that monitors global array2, locks it, copies data to local string, clears global array2, unlocks global array2 and updates pc rich text boxes, then clears local strings
I believe asynchronous data transfer can be achieved if I use some do while loops, to wait for the locks/unlocks.
What do you think?
|
|
|
|
|
way too complex.
Luc Pattyn [My Articles] Nil Volentibus Arduum
Fed up by FireFox memory leaks I switched to Opera and now CP doesn't perform its paste magic, so links will not be offered. Sorry.
|
|
|
|
|
Ok, then don't look at the code I posted a few hours ago 
I'm trying to get it to work, but will try and simplify it also, if I can get it to work. I tried to google for some psuedocode, but didn't find anything.
|
|
|
|
|
Well Luc,
After 2 straight days, I've got it working and working fairly decent. If you want I will post the code, but I'm confident it is still to complex. This is my first attempt at data sharing amongst threads.
I'd love to see an efficient/less complex method if you are willing to share. I'd like to learn to do it the right/efficient way. If you're willing to guide me in the right direction, I'm all ears.
|
|
|
|
|
Why would one need all those threads? All they offer is overhead, bug risks, and performance loss.
You should have created one, not three. And please design before you code, not the other way around.
Luc Pattyn [My Articles] Nil Volentibus Arduum
Fed up by FireFox memory leaks I switched to Opera and now CP doesn't perform its paste magic, so links will not be offered. Sorry.
|
|
|
|
|
I chose multiple threads so that the data could be received asynchronously compared to when it was displayed and so no data was lost ... originally I had a single thread. I know you have serial experience, so if you think a single windows thread can handle the 115000 kbps, I will try and condense the 3 threads into 1.
|
|
|
|
|
You never provided info about baud rates, message length, message frequency, throughput.
On a single core, adding threads does not increase throughput; what it can do, using sufficient buffer space, is cope better with peak loads (it is the buffers who deal with the peaks, the threads are just an easy way to decouple the producer and consumer code).
On a multi-core, adding more threads may or may not help, it depends on a lot of circumstances.
I am guessing your real bottleneck is in the GUI, and then IIRC you are using a TextBox, which is a bad choice for displaying large amounts of line-oriented text. A ListBox is a far better choice, as it does not degrade quadratically under increasing data load. A ListBox holds a list of items, a TextBox concatenates all its data, which gets expensive.
I would use just one thread and a ListBox; some 20 lines of code should do it!
And if I were to need special operations, such as e.g. "save to file", I would add that functionality to the ListBox rather than paying a performance penalty all the time.
Luc Pattyn [My Articles] Nil Volentibus Arduum
Fed up by FireFox memory leaks I switched to Opera and now CP doesn't perform its paste magic, so links will not be offered. Sorry.
|
|
|
|
|
Thanks for the guidance Luc. I will try and condense the code as you've suggested. My code is about 200 lines long for the 3 threads, but it wasn't all in vain as I learned a lot. I was going to use a rich text box so that I could set thresholds and change background colors of each individual box, depending on the value. I would venture that would slow the gui down even more though, so listboxes it is.
|
|
|
|
|
You're welcome.
And you can beautify a ListBox as well, here[^] is an example.
And then, this[^] might interest you.
Luc Pattyn [My Articles] Nil Volentibus Arduum
Fed up by FireFox memory leaks I switched to Opera and now CP doesn't perform its paste magic, so links will not be offered. Sorry.
|
|
|
|
|
Awesome! I will be reading your article. This is great
|
|
|
|
|
It isn't 20 lines, but it's pretty fast compared to what I had before, I have the uC push serial data every 10mS and it has no problem keeping up.
What do you think?
public string[] produceParseConsumeSerial(byte serialByte)
{
try
{
startCharacters[0] = '@';
startCharacters[1] = '@';
startCharacters[2] = '@';
endCharacters[0] = '|';
endCharacters[1] = '|';
endCharacters[2] = '|';
valueDelimiter[0] = '\r';
valueDelimiter[1] = '\r';
readLine[readLineIndex] = serialByte;
if (readLine[0] == startCharacters[0])
{
if (readLine[1] == startCharacters[1])
{
if (readLine[2] == startCharacters[2])
{
if ((readLine[readLineIndex] != valueDelimiter[0]) && (readLine[readLineIndex] != endCharacters[0]) && (readLine[readLineIndex] != 0) && (readLine[readLineIndex] != startCharacters[0]))
{
serialPortReadBytes[serialPortReadBytesIndex] = readLine[readLineIndex];
serialPortReadBytesIndex = serialPortReadBytesIndex + 1;
}
if ((readLine[readLineIndex] == valueDelimiter[1]) && (readLine[readLineIndex - 1] == valueDelimiter[0]))
{
int nullCount = 0;
string readLineString = System.Text.ASCIIEncoding.ASCII.GetString(serialPortReadBytes).ToString();
foreach (byte singleByte in serialPortReadBytes)
{
if (singleByte != 0)
{
nullCount = nullCount + 1;
}
}
int serialPortReadBytesCount = serialPortReadBytes.Count();
Array.Clear(serialPortReadBytes, 0, serialPortReadBytesCount);
serialPortReadBytesIndex = 0;
readLineString = readLineString.Remove(nullCount);
if (readLineString != "")
{
completeLineByLineArray.Add(readLineString);
}
readLineString = "";
}
else if ((readLine[readLineIndex] == valueDelimiter[0]))
{
int nullCount = 0;
string readLineString = System.Text.ASCIIEncoding.ASCII.GetString(serialPortReadBytes).ToString();
foreach (byte singleByte in serialPortReadBytes)
{
if (singleByte != 0)
{
nullCount = nullCount + 1;
}
}
int serialPortReadBytesCount = serialPortReadBytes.Count();
Array.Clear(serialPortReadBytes, 0, serialPortReadBytesCount);
serialPortReadBytesIndex = 0;
readLineString = readLineString.Remove(nullCount);
if (readLineString != "")
{
completeLineByLineArray.Add(readLineString);
}
readLineString = "";
}
if ((readLine[readLineIndex] == endCharacters[2]) && (readLine[readLineIndex - 1] == endCharacters[1]) && (readLine[readLineIndex - 2] == endCharacters[0]))
{
string[] delimitedAndParsedArray = new string[completeLineByLineArray.Count];
Array.Copy(completeLineByLineArray.ToArray(), delimitedAndParsedArray, completeLineByLineArray.Count);
completeLineByLineArray.Clear();
int readLineLength = readLine.Count();
Array.Clear(readLine, 0, readLineLength);
readLineIndex = 0;
int serialPortReadBytesCountInitialClear = serialPortReadBytes.Count();
Array.Clear(serialPortReadBytes, 0, serialPortReadBytesCountInitialClear);
serialPortReadBytesIndex = 0;
return delimitedAndParsedArray;
}
}
}
readLineIndex = readLineIndex + 1;
}
else
{
int serialPortReadBytesCount = serialPortReadBytes.Count();
Array.Clear(serialPortReadBytes, 0, serialPortReadBytesCount);
int readLineLength = readLine.Count();
Array.Clear(readLine, 0, readLineLength);
readLineIndex = 0;
}
}
catch (Exception ex)
{
MessageBox.Show("produceParseConsumeSerial() failure " + ex.Message + " " + ex.Data + " " + ex.StackTrace);
}
return null;
}
|
|
|
|
|
turbosupramk3 wrote: What do you think?
I did not study that in any detail, it does look horrible, and I'm flabbergasted. IIRC all data is textual, so I was expecting something that reads one line (hence a string) at once (using SerialPort.ReadLine), then validates it and extracts data. Yours is fiddling with bytes, on and on, in incomprehensible ways; and it is not even complete, where is the serial port access itself?
It would have helped if you had shown an expected string.
It would have helped if your identifiers were shorter, how often does one need to see "read" or "readline"?
It also would have helped if your method had a meaningful name (normally one verb + one noun), I can only guess what produceParseConsumeSerial() is supposed to do, and I can't begin to imagine why it needs to return a string array. A functional comment on top would have been nice.
And I don't like that many nested if statements; it is too hard to understand, and it ruins the layout.
I suggest you scrap all of it, and come up with simple, clean code.
|
|
|
|
|
Ok,
I can try again. It's a method, not a class yet, that is why there is no serial port information as far as opening it, that is done in the method that calls upon produceParseConsumeSerial(). I guess the name is long, I combined the three threads and just named the single thread, the appended names of the three individual threads.
I used ReadByte() instead of ReadLine() and pass the method a byte, each time I receive one, I like doing it that way instead of ReadLine(). You are right, it does need more commenting, and an example line, version 3 will have that. I will probably combine the if statements with && in version 3 as well, you are right, that is something that would make it easier to read.
Thanks for the critique, the next version will be better, I promise. This version is at least functional, and that allows me to continue to improve it while testing functionality.
|
|
|
|
|
As OriginalGriff and Luc already pointed out it's a let's say tricky thing.
When I createed my first program that dealt with searial port I was like "What's up with all this nonsence about SerialPort headeaches?".
I was lucky in that I only recevied a "1", "2", "3" or "X" and I always recieved the whole message.
Later on with some more complex messaging I started to see the "headeaches".
The only way to deal with is corectly is to have and know the message start and end specifications known as preffix and suffix in the "serialport world".
Since that day on, I always create(if possible) the suffix and the prefix for devices let's say a RS232 scanner.
If I'm not in control of the suffix and prefix I ask for them and won't accept a situation with no clear prefix and suffix data.
One needs to know when the message starts and when it ends. How you deal with it(e.g event or background worker as Luc suggested) it's your choice.
But you need to know when a message starts and when it ends. Else it's a gamble. I mean it.
Ex: 3CXXXXXXXXXXX[CR][LF]
In this example 3C marks the start of the message and "\r\n" the end. XXXXXXX is the actual message.
All the best,
Dan
|
|
|
|
|
Hi Dan,
Thanks for the reply. I am fortunate enough to be able to create a prefix and suffix. I have my suffix right now as |||, and I could add a prefix, maybe @@@ ?
Does that sound sufficient, along with 3 threads working asynchronously (mapped out in my other reply) with 2 global arrays to parse the data?
|
|
|
|
|
Yes, sure. It should do the trick.
All the best,
Dan
|
|
|
|
|
If a total rewrite isn't needed, should I have the threads looping all of the time, with a small wait time at the end? Or have the receive thread spawn the other threads?
Also, is serialPort1.ReadByte(), or is serialPort1.ReadLine() a better option?
Thanks for the help everyone, I've learned a lot!
private void produceSerial()
{
try
{
List serialPortReadLine1 = new List(500);
byte[] readLine = new byte[500];
int readLineIndex = 0;
while(globalVariables.receivingSerial == true)
{
if (serialPort1.BytesToRead > 0)
{
readLine[readLineIndex] = Convert.ToByte(serialPort1.ReadByte());
readLineIndex = readLineIndex + 1;
}
if (readLine[0] == '@')
{
if (readLine[1] == '@')
{
if ((readLine[2] == '@') && (readLine[3] != 0))
{
serialPortReadLine1.Add(readLine[readLineIndex]);
if ((readLine[readLineIndex] == '|') && (readLine[readLineIndex - 1] == '|') && (readLine[readLineIndex - 2] == '|'))
{
lock (globalVariables.linesRead)
{
int elements = globalVariables.linesRead.Count;
for (int i = elements; i < (elements + 1); i++)
{
globalVariables.linesRead[i + 1] = serialPortReadLine1.ToString();
serialPortReadLine1.Clear();
readLineIndex = 0;
}
}
}
}
}
}
}
}
catch (Exception ex)
{
MessageBox.Show("produceSerial() failure " + ex.Message);
}
}
private void parseSerial()
{
try
{
string [] linesUnparsed = new string[500];
while (globalVariables.receivingSerial == true)
{
if (globalVariables.linesRead.Count != 0)
{
lock (globalVariables.linesRead)
{
globalVariables.linesRead.CopyTo(linesUnparsed);
globalVariables.linesRead.Clear();
}
List charactersReplaced = new List(500);
string characterReplaced = "";
foreach (string line in linesUnparsed)
{
characterReplaced = line.Replace("\r", "<>");
charactersReplaced.Add(characterReplaced);
}
char[] splitCharacters = new char[] { '<', '>' };
string[] linesParsed = new string[500];
foreach (string lineUnparsed in linesUnparsed)
{
linesParsed = lineUnparsed.Split(splitCharacters, StringSplitOptions.None);
}
Array.Clear(linesUnparsed, 0, linesUnparsed.Length);
lock (globalVariables.linesParsedArray)
{
foreach (string lineParsed in linesParsed)
{
globalVariables.linesParsedArray.Add(lineParsed);
}
Array.Clear(linesParsed, 0, linesParsed.Length);
}
}
}
}
catch (Exception ex)
{
MessageBox.Show("parseSerial() failure " + ex.Message);
}
}
private void consumeSerial()
{
try
{
string[] dataOutputQueue = new string[500];
while (globalVariables.receivingSerial == true)
{
if (globalVariables.linesParsedArray.Count != 0)
{
lock (globalVariables.lineRead)
{
globalVariables.linesParsedArray.CopyTo(dataOutputQueue);
globalVariables.linesParsedArray.Clear();
}
foreach (string dataOutputLine in dataOutputQueue)
{
if (dataOutputLine.Contains("ps1"))
{
}
if (dataOutputLine.Contains("ps2"))
{
}
}
Array.Clear(dataOutputQueue, 0, dataOutputQueue.Length);
}
}
}
catch (Exception ex)
{
MessageBox.Show("consumeSerial() failure " + ex.Message);
}
}
|
|
|
|
|
I developed some com addin for excel 2003. They did no work in excel 2010. Any one used com addin for UDF in excel 2010?ùThank for your time
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
