how to dynamically create properties (Data object instances) in C# for a dynamically varying number of Data objects?

So this isn't too hard but you will be neck deep in the reflection namespaces. The first problem is figuring out what a dynamic object returned from the server may look like. You can use reflection to get the properties that are on a given object then use PropInfo to find out the type of that property. From there, you can create a new type that has the properties you want and using some generic-method-foo, cast the values from your data object to your newly minted dynamic object.

I have a method for processing return object from a web service that may or may not have certain fields present. This allows me to eliminate a dependency on the service proxy class and to be flexible about what may be returned. That method inspects the object for a given property and attempts to assign the value to a field on the object. The class has a constructor which accepts the object returned from the web service (as type object) and calls this method to map the values onto the fields in the class.

That method looks like this:

public void MapDtoToFields(object serviceDto)
{
    try
    {
        // General properties mapped straight to base types
        AcknowledgeDeadline = PortableDtoHelpers.CastDtoToField<DateTime?>(serviceDto, "acknowledgeDeadline");
        AdmitDate = PortableDtoHelpers.CastDtoToField<DateTime?>(serviceDto, "admitDate");
        AssociatedRequest = PortableDtoHelpers.CastDtoToField<string>(serviceDto, "request");
        AssociatedRequestUnivId = PortableDtoHelpers.CastDtoToField<string>(serviceDto, "requestUniversalId");
        ClosedDate = PortableDtoHelpers.CastDtoToField<DateTime?>(serviceDto, "closedDate");
        ClosingComments = PortableDtoHelpers.CastDtoToField<string>(serviceDto, "closingComments");

        // Complex property mapped to another internal platform type
        RoutingRecipient = new CMPayerObject(PortableDtoHelpers.CastDtoToField<object>(serviceDto, "routingRecipientDto"));

        // Array property mapped to a collection
        object[] serviceDtoAttachedDiagnoses = PortableDtoHelpers.CastDtoToField<object[]>(serviceDto, "umAttachedDiagnosisDtos");
        if(serviceDtoAttachedDiagnoses != null)
            foreach (object item in serviceDtoAttachedDiagnoses)
                Diagnoses.AttachedDiagnoses.Add(new CMDiagnosisObject(item));
    }
    catch (Exception ex)
    {
        logger.Log(CommonLogLevel.Error, string.Format("There was an exception while mapping the server Dto to the fields. The exception is {0}", ex.ToString()));
    }
}

The gist of it is that each line utilizes the generic method CastDtoToField<t> to inspect the service object for the specified property then attempt to return a strongly-typed instance of the value. The CastDtoToFields method lives in a static class in a PCL so we have access to it from anywhere that uses that PCL (Portable Class Library).

The method looks like:

/// <summary>
/// Casts the field specified on the provided service Dto to the proper type and returns that to the caller.
/// </summary>
/// <typeparam name="T">The Type of the target data field</typeparam>
/// <param name="serviceDto">The dto returned from the web service</param>
/// <param name="dtoFieldName">The name of the field to map on the Dto.</param>
/// <returns>A typed value</returns>
public static T CastDtoToField<T>(object serviceDto, string dtoFieldName)
{
    try
    {
        logger.Log(CommonLogLevel.Trace, "Entering CastDtoToField<T> method.");
        // Defense first.
        if(serviceDto == null | string.IsNullOrWhiteSpace(dtoFieldName))
        {
            logger.Log(CommonLogLevel.Warn, "Returning default value because the service dto is null or the field name was empty.");
            return default(T);
        }

        // Test to see if the target property exists on the Dto
        if(!PortableClassHelpers.HasProperty(serviceDto, dtoFieldName))
        {
            // Property doesn't exist!
            logger.Log(CommonLogLevel.Warn, string.Format("The provided service dto (type {0}) does not have the property {1} on it.",
                serviceDto.GetType().ToString(), dtoFieldName));
            return default(T);
        }

        // Property exists on the service dto. Now see if there is a "specified" property.
        bool hasSpecified = false;
        string specifiedProperty = dtoFieldName + "Specified";
        hasSpecified = PortableClassHelpers.HasProperty(serviceDto, specifiedProperty);

        // If the "Specified" property exists, then the source type is not-nullable.
        if(hasSpecified)
        {
            // Source property is not nullable and we have to look to see...
            if(PortableClassHelpers.CastProperty<bool>(serviceDto, specifiedProperty))
            {
                // The specified property is set to TRUE so the attached property has a value and we should be able to
                // do a straight cast.
                // Wrap this in a try and toss back the default if it goes wrong.
                // (Yes... we have an outer try-catch but doing this here lets us log the specific error.
                try
                {
                    logger.Log(CommonLogLevel.Trace, string.Format("The property {0} has an attached 'Specified' property which is set to true. Returning the straight cast of the target property.", dtoFieldName));
                    return PortableClassHelpers.CastProperty<T>(serviceDto, dtoFieldName);
                }
                catch (Exception ex)
                {
                    logger.LogException(CommonLogLevel.Fatal, string.Format("There was an exception while trying to cast the property {0}. A 'specified' property was detected and set to true but the straight-cast failed.", dtoFieldName),ex);
                    return default(T);
                }
            }
            else
            {
                // The specified property is FALSE so the attached property doesn't have a valid value... even if it has a value.
                // So just return the default value.
                logger.Log(CommonLogLevel.Trace, string.Format("The property {0} has an attached 'Specified' property which is set to false. Returning the default of the target property.", dtoFieldName));
                return default(T);
            }
        }
        else
        {
            // No specified field so we try a straight cast.
            // Wrap this in a try and toss back the default if it goes wrong.
            // (Yes... we have an outer try-catch but doing this here lets us log the specific error.
            try
            {
                logger.Log(CommonLogLevel.Trace, string.Format("The property {0} does not have a 'Specified' property. Returning the straight cast of the target property.", dtoFieldName));
                return PortableClassHelpers.CastProperty<T>(serviceDto, dtoFieldName);
            }
            catch (Exception ex)
            {
                logger.LogException(CommonLogLevel.Fatal, string.Format("There was an exception while trying to cast the property {0}. No 'specified' property was detected and the straight-cast failed.", dtoFieldName),ex);
                return default(T);
            }
        }
    }
    catch (Exception oex)
    {
        logger.LogException(CommonLogLevel.Fatal, string.Format("There was an exception while trying to cast the property {0}. See the exception for details.", dtoFieldName), oex);
        return default(T);
    }
}

Whew! So that block of code looks to see if the defined property is present on the source object and returns a strongly typed value of that property. Note that there is no warranty on that code. Over the last year a bug here and a bug there has popped up where it won't properly cast something so if you use it, be aware that it may not correctly cast everything. Types that are arrays and don't implement IConvertable seem to be especially troublesome.

I don't go to the next step here but you should be able to extend it from this. To achieve what you want, you will need a base class that you will dynamically extend to hold your variable-sized response. I suggest also defining an interface for that base class and using the interface in all of your code. That way you can pass around the instance without all kinds of crappy type-casting. The important thing to know is that you CAN'T ADD PROPERTIES TO AN INSTANTIATED OBJECT. So you have to create a new object extending from your base type and add the additional properties to it before you create an instance of it. The System.Reflection.Emit namespace has the TypeBuilder class which is what you want to use. There is an example of building a type dynamically, creating an instance, and setting the value of a property on the MSDN page for TypeBuilder. You can see it here: https://msdn.microsoft.com/en-us/library/2sd82fz7.aspx[^] That should give you the other half of what you need.

So you should be able to tie it all together now.

Good luck!