Message Structure Library

• Download demo - 14.8 KB
• Download MessageStructure (RAR) - 167.1 KB

Introduction

A message is a simple data structure, comprising typed fields. Fields can be primitive types like int, uint, float, double, byte, byte array, etc. Also a message can include arbitrarily nested messages. I wrote a simple message structure library for my needs. My problem was building message structures for a device simulator, what I tried to achieve is creating predefined message structs, filling in values of fields and sending them to the device as byte array. Then, receiving data from the device as byte array and casting it to a message to process values of fields.

Background

Marshalling is very useful for these kind of operations, it is not unsafe code and it is really fast. A sample conversion could be done like this:

byte[] GetBytes(Message msg) 
{
    int size = Marshal.SizeOf(msg);
    byte[] arr = new byte[size];
    IntPtr ptr = Marshal.AllocHGlobal(size);

    Marshal.StructureToPtr(msg, ptr, true);
    Marshal.Copy(ptr, arr, 0, size);
    Marshal.FreeHGlobal(ptr);

    return arr;
}

Message GetMessage(byte[] arr)
{
    Message msg = new Message();

    int size = Marshal.SizeOf(msg);
    IntPtr ptr = Marshal.AllocHGlobal(size);

    Marshal.Copy(arr, 0, ptr, size);

    msg = (Message)Marshal.PtrToStructure(ptr, msg.GetType());
    Marshal.FreeHGlobal(ptr);

    return msg;
}

Why can't I use this?

Because my fields are not always byte aligned. I mean, I can have a field with only 3 bits. I want to illustrate it with a figure:

As shown above, Field1 is 3 bits, then 2 bits reserved, Field2 is 3 bits, Field3 is 4 bits and Field4 is 4 bits.

My sample message is 16 bits long and I'm waiting a byte array with length two. I need to define all my fields as byte and it sums up total 4 bytes (without reserved field). And I can't use marshalling because my struct is 4 bytes and incoming data is 2 bytes. So, I need to parse data and allocate my fields manually.

Structure

The structure of a message is as follows:

Header message can consist of a "Message ID" and "Message Length" fields.

Header fields are used for identification, we can check its length and type to identify incoming data.

Footer message contains a "Checksum" field.

Generally, a message has a checksum field which is located in last byte(s) of message. It is an error-detecting code commonly used in digital networks and storage devices to detect accidental changes to raw data. Blocks of data entering these systems get a short check value attached, based on the remainder of a polynomial division of their contents; on retrieval the calculation is repeated, and corrective action can be taken against presumed data corruption if the check values do not match.

And here's the class diagram of the message structure library:

IMessage is an interface which must be implemented by Message class.

/// <summary>
/// Message interface.
/// </summary>
public interface IMessage
{
    /// <summary>
    /// Gets fields.
    /// </summary>
    List<Field> Fields { get; }

    /// <summary>
    /// Returns byte array representation of message.
    /// </summary>
    /// <returns>
    /// The <see cref="byte[]"/>.
    /// Byte array representation of message.
    /// </returns>
    byte[] GetMessage();

    /// <summary>
    /// Fills fields of message with input byte array.
    /// </summary>
    /// <param name="data">
    /// The byte array data.
    /// </param>
    void SetMessage(byte[] data);
}

Field is a class which holds all required data of a message field.

public class Field
{
    #region ctor

    /// <summary>
    /// Initializes a new instance of the <see cref="Field"/> class.
    /// </summary>
    /// <param name="name">
    /// The name.
    /// </param>
    /// <param name="length">
    /// The length.
    /// </param>
    public Field(string name, int length)
    {
        this.Name = name;
        this.Length = length;
        this.Resolution = 1;
        this.DataType = DataTypes.BYTE;
        var byteCount = (int)Math.Ceiling(length / 8f);
        this.Data = new byte[byteCount];
    }

    /// <summary>
    /// Initializes a new instance of the <see cref="Field"/> class.
    /// </summary>
    /// <param name="name">
    /// The name.
    /// </param>
    /// <param name="length">
    /// The length.
    /// </param>
    /// <param name="type">
    /// The type.
    /// </param>
    public Field(string name, int length, DataTypes type)
        : this(name, length)
    {
        this.DataType = type;
    }

    /// <summary>
    /// Initializes a new instance of the <see cref="Field"/> class.
    /// </summary>
    /// <param name="name">
    /// The name.
    /// </param>
    /// <param name="length">
    /// The length.
    /// </param>
    /// <param name="type">
    /// The type.
    /// </param>
    /// <param name="resolution">
    /// The resolution.
    /// </param>
    public Field(string name, int length, DataTypes type, int resolution)
        : this(name, length, type)
    {
        this.Resolution = resolution;
    }

    /// <summary>
    /// Initializes a new instance of the <see cref="Field"/> class.
    /// </summary>
    /// <param name="length">
    /// The length.
    /// </param>
    /// <param name="type">
    /// The type.
    /// </param>
    /// <param name="resolution">
    /// The resolution.
    /// </param>
    public Field(int length, DataTypes type, int resolution)
        : this(string.Empty, length, type, resolution)
    {
        this.Resolution = resolution;
    }

    /// <summary>
    /// Initializes a new instance of the <see cref="Field"/> class.
    /// </summary>
    /// <param name="length">
    /// The length.
    /// </param>
    /// <param name="type">
    /// The type.
    /// </param>
    public Field(int length, DataTypes type)
        : this(string.Empty, length, type)
    {
    }

    #endregion

    #region Properties

    /// <summary>
    ///  Gets or sets the value updated.
    /// </summary>
    public Action ValueUpdated { get; set; }

    /// <summary>
    /// Gets or sets the name.
    /// </summary>
    public string Name { get; set; }

    /// <summary>
    /// Gets or sets the length.
    /// </summary>
    public int Length { get; set; }

    /// <summary>
    /// Gets or sets the resolution.
    /// </summary>
    public double Resolution { get; set; }

    /// <summary>
    /// Gets or sets the byte array data.
    /// </summary>
    public byte[] Data { get; set; }

    /// <summary>
    /// Gets or sets the data type.
    /// </summary>
    public DataTypes DataType { get; set; }

    #endregion

    #region Public Methods

    /// <summary>
    /// Sets value of field.
    /// </summary>
    /// <param name="value">
    /// The value.
    /// </param>
    /// <exception cref="Exception">
    /// </exception>
    public void SetValue(object value)
    {
        switch (this.DataType)
        {
            case DataTypes.INT:
                this.SetValue((int)value);
                break;
            case DataTypes.UINT:
                this.SetValue((uint)value);
                break;
            case DataTypes.SHORT:
                this.SetValue((short)value);
                break;
            case DataTypes.USHORT:
                this.SetValue((ushort)value);
                break;
            case DataTypes.FLOAT:
                this.SetValue((float)value);
                break;
            case DataTypes.DOUBLE:
                this.SetValue((double)value);
                break;
            case DataTypes.BYTE:
                this.SetValue((byte)value);
                break;
            case DataTypes.BYTEARRAY:
                this.SetValue((byte[])value);
                break;
            default:
                throw new Exception("Hata");
        }
    }

    /// <summary>
    /// The set value.
    /// </summary>
    /// <param name="value">
    /// The value.
    /// </param>
    /// <typeparam name="T">
    /// Available types: int, uint, short, ushort, float, double, byte, char, byte[]
    /// </typeparam>
    public void SetValue<T>(T value)
    {
        var len = (int)Math.Ceiling(this.Length / 8f);

        if (typeof(T) == typeof(int))
        {
            var intvalue = (int)(Convert.ToInt32(value) / this.Resolution);
            this.Data = BitConverter.GetBytes(intvalue).Reverse().ToArray().PadTrimArray(len);
        }
        else if (typeof(T) == typeof(uint))
        {
            var uintvalue = (uint)(Convert.ToUInt32(value) / this.Resolution);
            this.Data = BitConverter.GetBytes(uintvalue).Reverse().ToArray().PadTrimArray(len);
        }
        else if (typeof(T) == typeof(short))
        {
            var shortvalue = (short)(Convert.ToInt16(value) / this.Resolution);
            this.Data = BitConverter.GetBytes(shortvalue).Reverse().ToArray();
        }
        else if (typeof(T) == typeof(ushort))
        {
            var ushortvalue = (ushort)(Convert.ToUInt16(value) / this.Resolution);
            this.Data = BitConverter.GetBytes(ushortvalue).Reverse().ToArray().PadTrimArray(len);
        }
        else if (typeof(T) == typeof(float))
        {
            var floatvalue = (float)(Convert.ToSingle(value) / this.Resolution);
            this.Data = BitConverter.GetBytes(floatvalue).PadTrimArray(len).Reverse().ToArray();
        }
        else if (typeof(T) == typeof(double))
        {
            double doublevalue = Convert.ToDouble(value) / this.Resolution;
            this.Data = BitConverter.GetBytes(doublevalue).Reverse().ToArray().PadTrimArray(len);
        }
        else if (typeof(T) == typeof(byte))
        {
            var bytevalue = (byte)(Convert.ToByte(value) / this.Resolution);
            this.Data = BitConverter.GetBytes(bytevalue).Reverse().ToArray().PadTrimArray(len);
        }
        else if (typeof(T) == typeof(char))
        {
            var charvalue = (char)Convert.ToChar(value);
            this.Data = BitConverter.GetBytes(charvalue).Reverse().ToArray().PadTrimArray(len);
        }
        else if (typeof(T) == typeof(byte[]))
        {
            this.Data = (byte[])(object)value;
        }
        else
        {
           throw new ArgumentException("value", "Invalid type.");
        }

        if (this.ValueUpdated != null)
        {
            this.ValueUpdated();
        }
    }

    /// <summary>
    /// The get value.
    /// </summary>
    /// <returns>
    /// The <see cref="string"/>.
    /// </returns>
    public string GetValue()
    {
        return this.GetValue(this.DataType);
    }

    /// <summary>
    /// The get value.
    /// </summary>
    /// <typeparam name="T">
    /// Available types: int, uint, short, ushort, float, double, byte, char, byte[]
    /// </typeparam>
    /// <returns>
    /// The <see cref="T"/>.
    /// Returns value after converted to selected type.
    /// </returns>
    public T GetValue<T>()
    {
        if (typeof(T) == typeof(int))
        {
            var arr = this.Data.PadTrimArray(4);
            var value = (int)(BitConverter.ToInt32(arr.Reverse().ToArray(), 0) * this.Resolution);
            return (T)Convert.ChangeType(value, typeof(T));
        }

        if (typeof(T) == typeof(uint))
        {
            var arr = this.Data.PadTrimArray(4);
            var value = (uint)(BitConverter.ToUInt32(arr.Reverse().ToArray(), 0) * this.Resolution);
            return (T)Convert.ChangeType(value, typeof(T));
        }

        if (typeof(T) == typeof(short))
        {
            var arr = this.Data.PadTrimArray(2);
            var value = (short)(BitConverter.ToInt16(arr.Reverse().ToArray(), 0) * this.Resolution);
            return (T)Convert.ChangeType(value, typeof(T));
        }

        if (typeof(T) == typeof(ushort))
        {
            var arr = this.Data.PadTrimArray(2);
            var value = (ushort)(BitConverter.ToUInt16(arr.Reverse().ToArray(), 0) * this.Resolution);
            return (T)Convert.ChangeType(value, typeof(T));
        }

        if (typeof(T) == typeof(float))
        {
            var arr = this.Data.PadTrimArray(4);
            var value = (float)(BitConverter.ToSingle(arr.Reverse().ToArray(), 0) * this.Resolution);
            return (T)Convert.ChangeType(value, typeof(T));
        }

        if (typeof(T) == typeof(double))
        {
            var arr = this.Data.PadTrimArray(4);
            var value = BitConverter.ToDouble(arr.Reverse().ToArray(), 0) * this.Resolution;
            return (T)Convert.ChangeType(value, typeof(T));
        }

        if (typeof(T) == typeof(byte))
        {
            var value = (byte)(this.Data[0] * this.Resolution);
            return (T)Convert.ChangeType(value, typeof(T));
        } 
            
        if (typeof(T) == typeof(char))
        {
            var value = (char)this.Data[0];
            return (T)Convert.ChangeType(value, typeof(T));
        }

        if (typeof(T) == typeof(byte[]))
        {
            return (T)Convert.ChangeType(this.Data, typeof(T));
        }

        return default(T);
    }

    /// <summary>
    /// The to string.
    /// </summary>
    /// <returns>
    /// The <see cref="string"/>.
    /// </returns>
    public override string ToString()
    {
        return this.Name;
    }

    #endregion

    #region Private Methods

    /// <summary>
    /// Returns string representation of value of field.
    /// </summary>
    /// <param name="dataType">
    /// The data type.
    /// </param>
    /// <returns>
    /// The <see cref="string"/>.
    /// String representation of field.
    /// </returns>
    private string GetValue(DataTypes dataType)
    {
        switch (dataType)
        {
            case DataTypes.INT:
                return this.GetValue<int>().ToString();
            case DataTypes.UINT:
                return this.GetValue<uint>().ToString();
            case DataTypes.SHORT:
                return this.GetValue<short>().ToString();
            case DataTypes.USHORT:
                return this.GetValue<ushort>().ToString();
            case DataTypes.FLOAT:
                return this.GetValue<float>().ToString();
            case DataTypes.DOUBLE:
                return this.GetValue<double>().ToString();
            case DataTypes.BYTE:
                return this.GetValue<byte>().ToString();
            case DataTypes.CHAR:
                return this.GetValue<char>().ToString();
            case DataTypes.BYTEARRAY:
                return BitConverter.ToString(this.GetValue<byte[]>());
            default:
                return null;
        }
    }

    #endregion
}

Message class holds all fields, header and footer messages.

public abstract class Message : IMessage
{
    #region Fields

    /// <summary>
    /// The checksum field.
    /// </summary>
    protected Field CheckSumField;

    /// <summary>
    /// Fields of message.
    /// </summary>
    private List<Field> fields;

    /// <summary>
    /// Fields of message without checksum field.
    /// </summary>
    private List<Field> fieldsWoChecksum;

    #endregion

    #region Properties

    /// <summary>
    /// Gets or sets the header message.
    /// </summary>
    public Message HeaderMessage { get; set; }

    /// <summary>
    /// Gets or sets the footer message.
    /// </summary>
    public Message FooterMessage { get; set; }

    /// <summary>
    /// Gets or sets a value indicating whether is checksum exists.
    /// </summary>
    public bool IsChecksumExists { get; set; }

    /// <summary>
    /// Gets the fields.
    /// </summary>
    public virtual List<Field> Fields
    {
        get
        {
            if (this.fields == null)
            {
                this.GatherFields();
                return this.fields;
            }

            return this.fields;
        }
    }

    #endregion

    #region Public Methods

    /// <summary>
    /// Fills fields of message with input byte array.
    /// </summary>
    /// <param name="data">
    /// The byte array data.
    /// </param>
    public virtual void SetMessage(byte[] data)
    {
        this.Fill(data);
    }

    /// <summary>
    /// Returns byte array representation of message.
    /// </summary>
    /// <returns>
    /// The <see cref="byte[]"/>.
    /// Byte array representation of message.
    /// </returns>
    public virtual byte[] GetMessage()
    {
        this.ReCalculateCheckSum();
        return this.ToByteArray();
    }

    /// <summary>
    /// Returns a string that represents the current object.
    /// </summary>
    /// <returns>
    /// A string that represents the current object.
    /// </returns>
    /// <filterpriority>2</filterpriority>
    public override string ToString()
    {
        if (this.fields == null)
        {
            this.GatherFields();
        }

        try
        {
            return string.Join(
                Environment.NewLine,
                this.fields.Select(p => p.Name + 
                    ":\t" + (p.GetValue() == "\0" ? 
                "Null Character" : p.GetValue())));
        }
        catch
        {
            return string.Empty;
        }
    }

    /// <summary>
    /// Initializes fields of message.
    /// </summary>
    protected void InitFields()
    {
        this.GatherFields();

        var fieldProperties =
            this.GetType()
                .GetProperties()
                .Where(p => p.PropertyType == typeof (Field))
                .ToList();

        foreach (var a in fieldProperties)
        {
            object o = a.GetValue(this, null);
            if (o == null)
            {
                continue;
            }

            var f = o as Field;
            if (f != null && f.Name == string.Empty)
            {
                f.Name = Utils.SplitCamelCase(a.Name);
            }
        }
    }

    /// <summary>
    /// Returns calculated checksum value.
    /// </summary>
    /// <returns>
    /// The <see cref="byte"/>.
    /// </returns>
    protected virtual byte GetCheckSum()
    {
        return Utils.CalculateChecksum(this.Fields.ToByteArray());
    }

    /// <summary>
    /// Calculates checksum and sets checksum field.
    /// </summary>
    protected void ReCalculateCheckSum()
    {
        if (this.fields == null)
        {
            this.GatherFields();
        }

        if (this.IsChecksumExists && this.CheckSumField != null)
        {
            this.CheckSumField.SetValue
                (Utils.CalculateChecksum(this.fieldsWoChecksum.ToByteArray()));
        }
    }

    #endregion

    #region Private Methods

    /// <summary>
    /// Gathers all fields.
    /// </summary>
    private void GatherFields()
    {
        var fieldProperties = this.GetType().GetProperties().ToList();
        this.fields = new List<Field>();

        foreach (var a in fieldProperties)
        {
            object o = a.GetValue(this, null);
            if (o == null)
            {
                continue;
            }

            if (o is Field)
            {
                fields.Add(o as Field);
            }
            else if (o is IEnumerable<Message>)
            {
                var y = o as IEnumerable<Message>;
                fields.AddRange(y.SelectMany(p => p.Fields));
            }
        }

        if (this.HeaderMessage != null)
        {
            this.fields.InsertRange(0, this.HeaderMessage.Fields);
        }

        if (this.FooterMessage != null)
        {
            this.fields.AddRange(this.FooterMessage.Fields);
        }

        if (this.IsChecksumExists)
        {
            this.CheckSumField = this.fields.Last();

            if (this.fieldsWoChecksum == null)
            {
                this.fieldsWoChecksum = this.Fields.Except
                    (new List<Field> { this.CheckSumField }).ToList();
            }

            this.fieldsWoChecksum.ForEach
                (p => p.ValueUpdated = this.ReCalculateCheckSum);
        }
    }

    #endregion
}

Using the Code

I prepared a sample project to use this library, I defined some Rx (incoming) and Tx (outgoing) messages with arbitrary fields. And wrote some methods to parse incoming data for identification.

A sample Rx message, WrapAcknowledge:

public class WrapAcknowledge : Message
{
    public Field Version { get; set; }
    public Field OutgoingData { get; set; }
    
    public WrapAcknowledge()
    {
        this.IsChecksumExists = true;
        this.HeaderMessage = new MyHeader(0x01, 0x01, 0x0F);
        this.FooterMessage = new MyFooter();
        this.Version = new Field(16, DataTypes.USHORT);
        this.OutgoingData = new Field(64, DataTypes.BYTEARRAY);

        this.InitFields();
    }
}

We need to tell class if we have checksum field and define our header and footer messages if exist. And we need to call InitFields base method to let message gather all own fields by reflection. All fields have a name property to print them out readable, and if we give declaration name to field properties clearly and camelcase, we don't need to worry about defining names for fields. We can parse them again with reflection.

A sample Tx message, WrapAround:

public enum WrapAroundType
{
    Type1 = 0,
    Type2
};

public class WrapAround : Message
{
    public Field Version { get; set; }
    public Field OutgoingData { get; set; }

    public WrapAround()
    {
        this.IsChecksumExists = true;
        this.HeaderMessage = new MyHeader(0x01, 0x01, 0x0F);
        this.FooterMessage = new MyFooter();
        this.Version = new Field(16, DataTypes.USHORT);
        this.OutgoingData = new Field(64, DataTypes.BYTEARRAY);
       
        this.InitFields();
    }

    public WrapAround(WrapAroundType type)
        : this()
    {
        if (type == WrapAroundType.Type1)
        {
            this.OutgoingData.SetValue(new byte[] 
                { 0x51, 0x45, 0xA0, 0x11, 0x00, 0x00, 0xFF, 0xFF });
        }
        else if (type == WrapAroundType.Type2)
        {
            this.OutgoingData.SetValue(new byte[] 
                { 0x3A, 0xBA, 0x02, 0x0F, 0x34, 0xA5, 0xF0, 0xF0 });
        }
    }
}

A more complicated message, LWRThreats:

public class LWRThreats : Message
{
    public Field Foo { get; set; }
    public List<Threat> Treats{ get; set; }

    public LWRThreats()
    { 
        this.IsChecksumExists = true;
        this.HeaderMessage = new MyHeader(0x01, 0x05, 0x36);
        this.FooterMessage = new MyFooter();
        this.Foo = new Field(8, DataTypes.BYTE);

        this.Treats = new List<Threat>().SetCapacity(3);
        this.InitFields();
    }
}

This message has sequentially:

• A Header Message (3 Fields)
• Its own field Foo (1 Field)
• List of Threats of Message with length 3 (Each threat message has 8 fields, total 24 Fields)
• A Footer Message (1 Field)

So this message has a total of 29 fields with this sequence.

Rx and Tx messages are defined exactly the same, so we can use a message as Rx and Tx message at the same time.

And here's the method to identify and message cast incoming data:

public static Message DetermineMessage(byte[] data)
{
    var header = new MyHeader();
    header.SetMessage(data);

    return ParseWithHeader(header, data);
}

private static Message ParseWithHeader(MyHeader header, byte[] data)
{
    var len = header.MessageLength.GetValue<ushort>();

    if (len != data.Length)
    {
        return null;
    }

    if (Utils.CalculateChecksum(data) != 0x00)
    {
        return null;
    }

    var tip = header.MessageType.GetValue<byte>();
    
    switch (tip)
    {
        case 0x01:
            Message mesaj = new WrapAcknowledge();
            mesaj.SetMessage(data);
            return mesaj;
        case 0x03:
            mesaj = new SystemStatus();
            mesaj.SetMessage(data);
            return mesaj;
        case 0x05:
            mesaj = new LWRThreats();
            mesaj.SetMessage(data);
            return mesaj;
        default:
            return null;
    }
}

First, I'm casting incoming data to header message, by this way I can retrieve its identification values. If value of length field is not equal to incoming data length, we need to eleminate this data.

Second check is checksum, we need to calculate checksum of message and check if it is equal to zero or not. If it fails, we need to eleminate this data.

And last check is finding available message type with a switch case like jump flow by using message type field of header. If there is no defined message with this message type value, we need to eleminate this data. Else, we can cast data to message and return it to caller.

Screenshots from GUI

In the first situation, I'm sending WrapAround message and receiving WrapAcknowledge message.

In the second situation, I'm converting a predefined byte array data to message. Parser identifies it and casts it to message which is LWRThreats message.

By my simple benchmark results; with an incoming data 100 byte array long, it can parse and cast message in less than 1 ms.

Points of Interest

I know using reflection and generics in this kind of situations is not healthy, but I used them to ease work of coder by providing a flexible structure and let them come to grips with their own challenging algorithmic problems. I will be glad if you fix my code and guide me for better techniques.

Hope you like it, thank you for reading!

History

• 01.01.2015 - Initial version