Click here to Skip to main content
15,867,686 members
Articles / Programming Languages / C#

Generic BinaryReader and BinaryWriter Extensions

Rate me:
Please Sign up or sign in to vote.
4.88/5 (17 votes)
26 Feb 2009CPOL5 min read 98.8K   639   30   34
Generics and extension methods in C++/CLI
A view from .NET Reflector

Introduction

A while ago, I was working on an application that needed to convert between structs and byte[]. I had many different kinds of structs that I needed to work with, and at first, I tried to maintain a large number of methods for each struct type. You could imagine how bloated the code was.

Eventually, I decided I would give a crack at making all of that conversion code generic, and this miniature project was born. The project provides a static class (like BitConverter) and extension methods for both BinaryReader and BinaryWriter to enable conversion between structs (of any kind) and a series of bytes as an Array or Stream.

Getting Started: The Impossible Way

Since I was somewhat familiar with C# already, naturally I would start the project in C# as well. To start it off, I wrote some code for one of the two conversion methods (shown below) and hit compile:

C#
public static unsafe byte[] GetBytes<T>(T value)
{
    // Check to see if the input is null.
    if (value == null)
    {
        throw new ArgumentNullException();
    }
 
    // Check to see if the input has value type semantics.
    if (!(value is ValueType))
    {
        throw new ArgumentException();
    }
 
    // Create an array of bytes the same size as the input.
    byte[] bytes = new byte[sizeof(T)];
 
    // Pin the byte array down so the garbage collector can't move it.
    fixed (byte* p1 = &bytes[0])
    {
        // Cast the pointer type from byte to T.
        T* p2 = (T*)p1;
 
        // Assign the value to the byte array.
        *p2 = value;
    }
 
    // Return the byte array.
    return bytes;
}

Instead of compiling successfully, Visual Studio gave me an error: “Cannot take the address of, get the size of, or declare a pointer to a managed type ('T').” This was logical. After all, T was not guaranteed to have value type semantics; you can't take the size of something that doesn't have a definite size, and a class can have any size during runtime unless it happens to derive from System.ValueType (e.g. Byte, Int32, UInt64, Single, Decimal, DateTime…). The next logical step would be to force the input to be a ValueType. I removed the error checking code and constrained the type parameter to ValueType:

C#
public static unsafe byte[] GetBytes<T>(T value) where T : ValueType
{
    // Create an array of bytes the same size as the input.
    byte[] bytes = new byte[sizeof(T)];
 
    // Pin the byte array down so the garbage collector can't move it.
    fixed (byte* p1 = &bytes[0])
    {
        // Cast the pointer type from byte to T.
        T* p2 = (T*)p1;
 
        // Assign the value to the byte array.
        *p2 = value;
    }
 
    // Return the byte array.
    return bytes;
}

After hitting the compile key, the error that Visual Studio gave me this time baffled me: “Constraint cannot be special class 'System.ValueType'.” Upon further research, I came upon an MSDN article that suggested constraining the type parameter to a struct. The constraint was accepted by the compiler, but again, it complained that taking the address, pointer, or size of a generic type was not allowed. (Smart compiler, eh?) This, to me, seemed like a dead end. How else was I supposed to write this method?

The C++/CLI Way

I also happened to be playing around with C++/CLI in Visual Studio at around the same time, when an idea hit me. If C++/CLI could do so many things that C# could not (like mixing managed code with unmanaged code), could it let me get around both restrictions in C#? There was only one way to find out, and after an hour of reading C++/CLI tutorials, I came up with this:

MC++
generic <typename T>
where T : System::ValueType
array<unsigned char> ^GenericBitConverter::GetBytes(T value)
{
       array<unsigned char> ^bytes = gcnew array<unsigned char>(sizeof(T));
       *reinterpret_cast<interior_ptr<T>>(&bytes[0]) = value;
       return bytes;
}

The method actually worked, and in just 3 lines of code!

Here is the expanded, more readable version (equivalent to the above, but split up to make the steps more obvious):

MC++
generic <typename T>
where T : System::ValueType
array<unsigned char> ^Iku::GenericBitConverter::GetBytes(T value)
{
       // Create an array of bytes the same size as the input.
       array<unsigned char> ^bytes = gcnew array<unsigned char>(sizeof(T));
 
       // Create a pointer to the byte array.
       interior_ptr<unsigned char> ptr1 = &bytes[0];
 
       // Cast the pointer to the same type as the input.
       interior_ptr<T> ptr2 = reinterpret_cast<interior_ptr<T>>(ptr1);
 
       // Assign the value to the byte array.
       *ptr2 = value;
 
       // Return the byte array.
       return bytes;
}

GenericBitConverter

… and that is the GenericBitConverter. It works with any type that is derived from ValueType (a struct), and thus will work with all the basic .NET data types like double and decimal (except string, which is really an object) as well as enumerations and other complex structures.

Making an Extension Method in C++/CLI

Now that the GenericBitConverter was in place, it would be nice if the same functionality were available to the BinaryReader and BinaryWriter classes, so I decided to extend those classes with the same functionality as well. This was not any easier than getting the GenericBitConverter to work properly. Writing the method was easy at first:

MC++
generic <typename T>
where T : System::ValueType
T Iku::IO::BinaryReaderExtension::ReadValue(BinaryReader ^binaryReader)
{
       return GenericBitConverter::ToValue<T>
		(binaryReader->ReadBytes(sizeof(T)), 0);
}

It just would not show up as an extension method in the C# code editor.

It was time to investigate why, so I wrote one in C# and I disassembled the assembly. While inspecting the output, I found the ExtensionAttribute attribute applied to three places: the assembly, the static class containing the extension method, and the extension method itself. Applying the attributes to the proper place in the C++/CLI project, I was able to get the extension method to show up:

Extension method for BinaryWriter

… and there are the generic BinaryReader and BinaryWriter extensions.

Now you can do stuff like this:

C#
DateTime timeStamp = binaryReader.ReadValue<DateTime>();
C#
static void Main(string[] args)
{
    SampleEnumeration sampEnum = SampleEnumeration.All ^ SampleEnumeration.Four;

    byte[] enumBytes = GenericBitConverter.GetBytes<SampleEnumeration>(sampEnum);

    // Writes "f0 ff ff ff ff ff ff ff"
    foreach (byte bite in enumBytes)
    {
        Console.Write("{0:x2} ", bite);
    }

    Console.WriteLine();
}

enum SampleEnumeration : long
{
    None = 0x0000000000000000,
    All = -1L,
    One = 0x0000000000000001,
    Four = 0x000000000000000f
}

Neat? Feel free to use this within your own C# or Visual Basic projects (by referencing the output of this project). It works perfectly!

Points of Interest

  • C++/CLI also lets you constrain generic parameters to the ValueType type; it does not complain. C# just demands struct in place of ValueType.
  • The C++/CLI unsigned char type is actually a synonym for the Byte type and not the Char type. The C++/CLI Char type is actually wchar_t.
  • Interior pointers (interior_ptr) point to their objects, but do not fix them in place. This is markedly different from C#’s fixed behavior, which fixes the object in memory and prevents the garbage collector from moving it (as long as the fixed statement is in scope).
  • Although C++/CLI does not offer any syntactic sugar for creating extension methods, it can be done by placing an ExtensionAttribute attribute on the method, its containing class, and its containing assembly. (ExtensionAttribute can be found under the namespace System::Runtime::CompilerServices) This nature of extension methods means it can be made from any language that supports attributes, static classes, and static methods.
  • The C# compiler will assume a generic type can be any regular object even if you constrain it to be a struct (or a ValueType). I have not found a way around it in C# (and thus this project was born).

Updates

  • Feb. 27, 2009: Added a download for the compiled assembly; not everyone has the ability to compile a C++/CLI project (users of the Express Editions come to mind). Now everyone can use it!

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
United States United States
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
Questionusing structs Pin
filmee2422-Feb-15 6:27
filmee2422-Feb-15 6:27 
AnswerRe: using structs Pin
Kevin Li (Li, Ken-un)22-Feb-15 6:39
Kevin Li (Li, Ken-un)22-Feb-15 6:39 
BugCLI runtime may rearrange fields. Pin
zergusik5-Feb-13 10:32
zergusik5-Feb-13 10:32 
GeneralRe: CLI runtime may rearrange fields. Pin
Kevin Li (Li, Ken-un)22-Feb-15 6:42
Kevin Li (Li, Ken-un)22-Feb-15 6:42 
QuestionIs there a way to achieve this generic byte converter in VB.NET? Pin
Shimmy Weitzhandler17-Jul-10 14:33
Shimmy Weitzhandler17-Jul-10 14:33 
AnswerRe: Is there a way to achieve this generic byte converter in VB.NET? Pin
Kevin Li (Li, Ken-un)22-Feb-15 6:43
Kevin Li (Li, Ken-un)22-Feb-15 6:43 
GeneralMy vote of 5 Pin
Shimmy Weitzhandler17-Jul-10 14:31
Shimmy Weitzhandler17-Jul-10 14:31 
Questionassembly attribute for c++/cli assembly Pin
Member 127498925-Feb-10 3:59
Member 127498925-Feb-10 3:59 
GeneralC# version (not exact, but equivalent) [modified] Pin
gunters1-Jun-09 5:29
gunters1-Jun-09 5:29 
GeneralRe: C# version (not exact, but equivalent) Pin
Daniel Grunwald31-Dec-09 10:49
Daniel Grunwald31-Dec-09 10:49 
GeneralPure C# Version? [modified] Pin
Richard Deeming5-Mar-09 6:53
mveRichard Deeming5-Mar-09 6:53 
GeneralRe: Pure C# Version? Pin
Kevin Li (Li, Ken-un)5-Mar-09 18:42
Kevin Li (Li, Ken-un)5-Mar-09 18:42 
GeneralEfficiency (VM and usage) [modified] Pin
User of Users Group1-Mar-09 4:17
User of Users Group1-Mar-09 4:17 
GeneralRe: Efficiency (VM and usage) Pin
Kevin Li (Li, Ken-un)2-Mar-09 11:00
Kevin Li (Li, Ken-un)2-Mar-09 11:00 
GeneralRe: Efficiency (VM and usage) [modified] Pin
User of Users Group3-Mar-09 12:56
User of Users Group3-Mar-09 12:56 
GeneralNote Pin
User of Users Group1-Mar-09 3:59
User of Users Group1-Mar-09 3:59 
GeneralRe: Note Pin
Kevin Li (Li, Ken-un)2-Mar-09 11:03
Kevin Li (Li, Ken-un)2-Mar-09 11:03 
GeneralRe: Note Pin
User of Users Group3-Mar-09 12:48
User of Users Group3-Mar-09 12:48 
JokeGood job. Pin
devace1-Mar-09 2:49
devace1-Mar-09 2:49 
GeneralRe: Good job. Pin
Kevin Li (Li, Ken-un)2-Mar-09 11:08
Kevin Li (Li, Ken-un)2-Mar-09 11:08 
Questionbroken link? Pin
Kenan E.K.27-Feb-09 7:25
professionalKenan E.K.27-Feb-09 7:25 
AnswerRe: broken link? Pin
Kevin Li (Li, Ken-un)27-Feb-09 7:58
Kevin Li (Li, Ken-un)27-Feb-09 7:58 
GeneralRe: broken link? Pin
Kenan E.K.27-Feb-09 12:47
professionalKenan E.K.27-Feb-09 12:47 
GeneralRe: broken link? Pin
Kevin Li (Li, Ken-un)27-Feb-09 17:01
Kevin Li (Li, Ken-un)27-Feb-09 17:01 
GeneralA very good one Pin
AnandChavali27-Feb-09 1:03
AnandChavali27-Feb-09 1:03 
Thanks for sharing (the power of C++ (CLI))

Thanks and Regards,
Anand.

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.