How to Write and Read Multidimensional Arrays of Blittable Types With BinaryWriter/Binary Reader - Short and Fast

Introduction

Here is real code which demonstrates a quick and not so dirty way in which to write and read multidimensional arrays of elements of blittable type in binary form.

Background

Often, it's required to write and read arrays of simple data in binary form. Most of the time, the elements are of simple blittable types. And most of the time, there are BinaryWriter or BinaryReader given as IO interface. So, this will be simple for one given array, but if you want to create a library which deals with multidimensional arrays of unknown element type and size, you will get an idea that this would not be such a simple task.

To write/read all elements, it's required to iterate over all elements of array with apriory unknown dimension - how to code this? Then, Read/Write of elements with BinaryWriter/BinaryReader of unknown type will require to use keyword dynamic for writing and switch-case for reading.

Combining of the two will require to use either strong generic typing, reflection, type tables or some other tricks - this all is a lot of work and in some cases, has huge performance penalties.

After that, the next idea will be... "Blittable types?" "How are they stored?", "YES!!! Write the underlied memory to stream. Simple!".

But if you take a look at BinaryWriter / BinaryReader, you will see that there are no members that consume IntPtr or other type of pointer, only byte[].

So, a simple try with search engine gets you a full nightmare: "How to create an interop library to use fwrite?", "PInvoke to FileWrite", "How to convert something to byte[]. Oh no! This is not possible, but you can copy memory..." - the whole program of expert tips and tricks including some sophisticated benchmarks.

But we can get this really very simple - only compile with /unsafe will be required, but no unmanaged code, no PInvoke, no buffer copy, no tricks - very small and simple code!

The idea besides is to "wrap a memory" to an instance of UnmanagedMemoryStream. Never heard of?

Using the Code

Here are two independent full working methods just to show the real code.

Method to Write an Array

static void testWriteArray()
{
 // Three-dimensional array.
            int[, ,] array3D = new int[,,] { { { 1, 2, 3 }, { 4, 5, 6 } }, 
                                 { { 7, 8, 9 }, { 10, 11, 12 } } };

 FileStream writeStream = new FileStream("c:\\test\\csharp.net-3darray.dat", FileMode.Create);
            BinaryWriter writeBinary = new BinaryWriter(writeStream);
            GCHandle arrHandle = GCHandle.Alloc(array3D, GCHandleType.Pinned);
            IntPtr arrPtr = arrHandle.AddrOfPinnedObject();

unsafe
            {
                int arrLen = array3D.Length;
                if (arrLen > 0)
                {
                    IEnumerator enumerator = array3D.GetEnumerator();
                    enumerator.MoveNext();
                    int arrSize = System.Runtime.InteropServices.Marshal.SizeOf(enumerator.Current) * arrLen;
                    // int elmSize = arrSize / arrLen;
                    using (UnmanagedMemoryStream arrStream = 
                    new UnmanagedMemoryStream((byte*)arrPtr.ToPointer(), arrSize))
                    {
                        arrStream.CopyTo(writeBinary.BaseStream, (int)arrStream.Length);
                    }
                }
            }

 arrHandle.Free();

 writeStream.Close();
}

Method to Read an Array

static Array testReadArray()
        {
            // Three-dimensional array.
            int[, ,] array3D = new int[2, 2, 3];

            FileStream readStream = new FileStream("c:\\test\\csharp.net-3darray.dat", FileMode.Open);
            BinaryWriter readBinary = new BinaryWriter(readStream);
            GCHandle arrHandle = GCHandle.Alloc(array3D, GCHandleType.Pinned);
            IntPtr arrPtr = arrHandle.AddrOfPinnedObject();

            unsafe
            {
                int arrLen = array3D.Length;
                if (arrLen > 0)
                {
                    IEnumerator enumerator = array3D.GetEnumerator();
                    enumerator.MoveNext();
                    int arrSize = 
			System.Runtime.InteropServices.Marshal.SizeOf(enumerator.Current) * arrLen;
                    //int elmSize = arrSize / arrLen;
                    using (UnmanagedMemoryStream arrStream = new UnmanagedMemoryStream
                    ((byte*)arrPtr.ToPointer(), arrSize, arrSize, FileAccess.Write))
                    {
                        readBinary.BaseStream.CopyTo(arrStream, (int)arrStream.Length);
                    }
                }
            }

            arrHandle.Free();
            readStream.Close();
            return array3D;
        }

What You Can Expand or How to Use in Your Library?

If you want to use the snippets in your real life library, simply write rank and dimensions to stream and use this information later together with Array.CreateInstance.

Points of Interest

I am always interested in ways to optimize my work. :)

History

• 27.01.2015 - First version