Click here to Skip to main content
15,881,413 members
Articles / Programming Languages / C#

Cinchoo ETL - JSON Writer

Rate me:
Please Sign up or sign in to vote.
5.00/5 (7 votes)
30 Sep 2022CPOL27 min read 20.2K   9   4
Simple JSON writer for .NET
This article discusses how to use ChoJSONWriter component offered by ChoETL framework. It is a simple utility class used to save JSON data to a file or an external data source.

Contents

  1. Introduction
  2. Requirement
  3. "Hello World!" Sample
    1. Quick write - Data First Approach
    2. Code First Approach
    3. Configuration First Approach
  4. Writing All Records
  5. Write Records Manually
  6. Customize JSON Record
  7. Customize JSON Fields
    1. DefaultValue
    2. ChoFallbackValue
    3. Type Converters
      1. Declarative Approach
      2. Configuration Approach
      3. Custom Value Converter Approach
    4. Validations
    5. ChoIgnoreMember
    6. StringLength
    7. Display
    8. DisplayName
  8. Callback Mechanism
    1. Using JSONWriter events
    2. Implementing IChoNotifyRecordWrite interface
    3. BeginWrite
    4. EndWrite
    5. BeforeRecordWrite
    6. AfterRecordWrite
    7. RecordWriteError
    8. BeforeRecordFieldWrite
    9. AfterRecordFieldWrite
    10. RecordWriteFieldError
  9. Customization
  10. Using Dynamic Object
  11. Exceptions
  12. Using MetadataType Annotation
  13. Configuration Choices
    1. Manual Configuration
    2. Auto Map Configuration
    3. Attaching MetadataType class
  14. ToTextAll Helper Method
    1. ToText Helper Method
  15. Writing DataReader Helper Method
  16. Writing DataTable Helper Method
  17. Advanced Topics
    1. Override Converters Format Specs
    2. Currency Support
    3. Enum Support
    4. Boolean Support
    5. DateTime Support
  18. Fluent API
    1. NullValueHandling
    2. Formatting
    3. WithFields
    4. WithField
    5. IgnoreFieldValueMode
    6. ColumnCountStrict
    7. Configure
    8. Setup
  19. FAQ
    1. How to serialize an object?
    2. How to serialize collection of objects?
    3. How to serialize dynamic object?
    4. How to serialize anonymous object?
    5. How to serialize collection?
    6. How to serialize dictionary?
    7. How to serialize DataTable?
    8. How to serialize JSON to a file?
    9. How to serialize unindented JSON?
    10. How to serialize conditional property?
    11. How to serialize datetime in custom DateFormat?
    12. How to exclude property from Json Serialization?
    13. How to convert XML to JSON?
    14. How to convert CSV to JSON?
  20. History

1. Introduction

ChoETL is an open source ETL (extract, transform and load) framework for .NET. It is a code based library for extracting data from multiple sources, transforming, and loading into your very own data warehouse in .NET environment. You can have data in your data warehouse in no time.

This article talks about using ChoJSONWriter component offered by ChoETL framework. It is a simple utility class to save JSON data to a file / external data source.

Corresponding ChoJSONReader, a JSON reader article can be found here.

Features

  • Follows JSON standard file rules
  • Supports culture specific date, currency and number formats while generating files
  • Supports different character encoding
  • Provides fine control of date, currency, enum, boolean, number formats when writing files
  • Detailed and robust error handling, allowing you to quickly find and fix problems
  • Shortens your development time

2. Requirement

This framework library is written in C# using .NET 4.5 Framework / .NET core 2.x.

3. "Hello World!" Sample

  • Open VS.NET 2013 or higher
  • Create a sample VS.NET (.NET Framework 4.5) Console Application project
  • Install ChoETL via Package Manager Console using Nuget Command based on working .NET version:
    • Install-Package ChoETL.JSON
    • Install-Package ChoETL.JSON.NETStandard
  • Use the ChoETL namespace

Let's begin by looking into a simple example of generating the below JSON file having two columns.

Listing 3.1 Sample JSON Data File (emp.json)
JSON
[
  {
    "Id": 1,
    "Name": "Mark"
  },
  {
    "Id": 2,
    "Name": "Jason"
  }
]

There are a number of ways you can get the JSON file to be created with minimal setup.

3.1. Quick Write - Data First Approach

This is the zero-config and quickest approach to create JSON file in no time. No typed POCO object is needed. The sample code below shows how to generate a sample JSON file using dynamic objects.

Listing 3.1.1 Write List of Objects to JSON File
C#
List<ExpandoObject> objs = new List<ExpandoObject>();
dynamic rec1 = new ExpandoObject();
rec1.Id = 1;
rec1.Name = "Mark";
objs.Add(rec1);
 
dynamic rec2 = new ExpandoObject();
rec2.Id = 2;
rec2.Name = "Jason";
objs.Add(rec2);
 
using (var parser = new ChoJSONWriter("emp.json"))
{
    parser.Write(objs);
}

In the above sample, we give the list of dynamic objects to JSONWriter at one pass to write them to JSON file.

Listing 3.1.2 Write Each Object to JSON File
C#
using (var parser = new ChoJSONWriter("emp.json"))
{
    dynamic rec1 = new ExpandoObject();
    rec1.Id = 1;
    rec1.Name = "Mark";
    parser.Write(item);

    dynamic rec1 = new ExpandoObject();
    rec1.Id = 2;
    rec1.Name = "Jason";
    parser.Write(item);
}

In the above sample, we take control of constructing, passing each individual dynamic record to the JSONWriter to generate the JSON file using Write overload.

3.2. Code First Approach

This is another zeo-config way to generate JSON file using typed POCO class. First, define a simple POCO class to match the underlying JSON file layout.

Listing 3.2.1 Simple POCO Entity Class
C#
public partial class EmployeeRecSimple
{
    public int Id { get; set; }
    public string Name { get; set; } 
}

In the above, the POCO class defines two properties matching the sample JSON file template.

Listing 3.2.2 Saving to JSON File
C#
List<EmployeeRecSimple> objs = new List<EmployeeRecSimple>();

EmployeeRecSimple rec1 = new EmployeeRecSimple();
rec1.Id = 1;
rec1.Name = "Mark";
objs.Add(rec1);
 
EmployeeRecSimple rec2 = new EmployeeRecSimple();
rec2.Id = 2;
rec2.Name = "Jason";
objs.Add(rec2);
 
using (var parser = new ChoJSONWriter<EmployeeRecSimple>("emp.json"))
{
    parser.Write(objs);
}

The above sample shows how to create a JSON file from typed POCO class objects.

3.3. Configuration First Approach

In this model, we define the JSON configuration with all the necessary parameters along with JSON columns required to generate the sample JSON file.

Listing 3.3.1 Define JSON Configuration
C#
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Id"));
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name"));

In the above, the class defines two JSON properties matching the sample JSON file template.

Listing 3.3.2 Generate JSON File without POCO Object
C#
List<ExpandoObject> objs = new List<ExpandoObject>();

dynamic rec1 = new ExpandoObject();
rec1.Id = 1;
rec1.Name = "Mark";
objs.Add(rec1);
 
dynamic rec2 = new ExpandoObject();
rec2.Id = 2;
rec2.Name = "Jason";
objs.Add(rec2);
 
using (var parser = new ChoJSONWriter("emp.json", config))
{
    parser.Write(objs);
}

The above sample code shows how to generate JSON file from a list of dynamic objects using predefined JSON configuration setup. In the JSONWriter constructor, we specified the JSON configuration object to obey the JSON layout schema while creating the file. If there is any mismatch in the name or count of JSON columns, it will be reported as an error and stops the writing process.

Listing 3.3.3 Saving JSON File with POCO Object
C#
List<EmployeeRecSimple> objs = new List<EmployeeRecSimple>();

EmployeeRecSimple rec1 = new EmployeeRecSimple();
rec1.Id = 1;
rec1.Name = "Mark";
objs.Add(rec1);
 
EmployeeRecSimple rec2 = new EmployeeRecSimple();
rec2.Id = 2;
rec2.Name = "Jason";
objs.Add(rec2);
 
using (var parser = new ChoJSONWriter<EmployeeRecSimple>("emp.json", config))
{
    parser.Write(objs);
}

The above sample code shows how to generate JSON file from a list of POCO objects with JSON configuration object. In the JSONWriter constructor, we specified the JSON configuration object.

3.4. Code First with Declarative Configuration

This is the combined approach to define POCO entity class along with attaching JSON configuration parameters declaratively. id is required column and name is optional value column with default value "XXXX". If name is not present, it will take the default value.

Listing 3.4.1 Define POCO Object
C#
public class EmployeeRec
{
    [ChoJSONRecordField]
    [Required]
    public int? Id
    {
        get;
        set;
    }

    [ChoJSONRecordField]
    [DefaultValue("XXXX")]
    public string Name
    {
        get;
        set;
    }

    public override string ToString()
    {
        return "{0}. {1}.".FormatString(Id, Name);
    }
}

The code above illustrates about defining POCO object with necessary attributes required to generate JSON file. The first thing defines a property for each record field with ChoJSONRecordFieldAttribute to qualify for JSON record mapping. Id is a required property. We decorated it with RequiredAttribute. Name is given default value using DefaultValueAttribute. It means that if the Name value is not set in the object, JSONWriter spits the default value 'XXXX' to the file.

It is very simple and ready to save JSON data in no time.

Listing 3.4.2 Saving JSON File with POCO Object
C#
List<EmployeeRec> objs = new List<EmployeeRec>();

EmployeeRec rec1 = new EmployeeRec();
rec1.Id = 1;
rec1.Name = "Mark";
objs.Add(rec1);
 
EmployeeRec rec2 = new EmployeeRec();
rec2.Id = 2;
rec2.Name = "Jason";
objs.Add(rec2);
 
using (var parser = new ChoJSONWriter<EmployeeRec>("emp.json"))
{
    parser.Write(objs);
}

We start by creating a new instance of ChoJSONWriter object. That's all. All the heavy lifting of generating JSON data from the objects is done by the writer under the hood.

By default, JSONWriter discovers and uses default configuration parameters while saving JSON file. These can be overridable according to your needs. The following sections will give you in-depth details about each configuration attribute.

4. Writing All Records

It is as easy as setting up POCO object match up with JSON file structure, construct the list of objects and pass it to JSONWriter's Write method. This will write the entire list of objects into JSON file in one single call.

Listing 4.1 Write to JSON File
C#
List<EmployeeRec> objs = new List<EmployeeRec>();
//Construct and attach objects to this list
...

using (var parser = new ChoJSONWriter<EmployeeRec>("emp.json"))
{
    parser.Write(objs);
}

or:

Listing 4.2 Writer to JSON File Stream
C#
List<EmployeeRec> objs = new List<EmployeeRec>();
//Construct and attach objects to this list
...

using (var tx = File.OpenWrite("emp.json"))
{
    using (var parser = new ChoJSONWriter<EmployeeRec>(tx))
    {
        parser.Write(objs);
    }
}

This model keeps your code elegant, clean, easy to read and maintain.

5. Write Records Manually

This is an alternative way to write each and every individual record to JSON file in case when the POCO objects are constructed in a disconnected way.

Listing 5.1 Writing to JSON File
C#
var writer = new ChoJSONWriter<EmployeeRec>("emp.json");

EmployeeRec rec1 = new EmployeeRec();
rec1.Id = 1;
rec1.Name = "Mark";
 
writer.Write(rec1);

EmployeeRec rec2 = new EmployeeRec();
rec1.Id = 2;
rec1.Name = "Jason"; 

writer.Write(rec2);

6. Customize JSON Record

Using ChoJSONRecordObjectAttribute, you can customize the POCO entity object declaratively.

Listing 6.1 Customizing POCO Object for Each Record
C#
[ChoJSONRecordObject]
public class EmployeeRec
{
    [ChoJSONRecordField]
    public int Id { get; set; }
    [ChoJSONRecordField]
    [Required]
    [DefaultValue("ZZZ")]
    public string Name { get; set; }
}

Here are the available attributes to carry out customization of JSON load operation on a file:

  • Culture - The culture info used to read and write.
  • ColumnCountStrict - This flag indicates if an exception should be thrown if JSON field configuration mismatches with the data object members.
  • NullValue - Special null value text expects to be treated as null value from JSON file at the record level.
  • ErrorMode - This flag indicates if an exception should be thrown if writing and an expected field is failed to write. This can be overridden per property. Possible values are:
    • IgnoreAndContinue - Ignore the error, record will be skipped and continue with next.
    • ReportAndContinue - Report the error to POCO entity if it is of IChoNotifyRecordWrite type.
    • ThrowAndStop - Throw the error and stop the execution.
  • IgnoreFieldValueMode - N/A
  • ObjectValidationMode - A flag to let the reader know about the type of validation to be performed with record object. Possible values are:
    • Off - No object validation performed. (Default)
    • MemberLevel - Validation performed before each JSON property gets written to the file
    • ObjectLevel - Validation performed before all the POCO properties are written to the file

7. Customize JSON Fields

For each JSON column, you can specify the mapping in POCO entity property using ChoJSONRecordFieldAttribute.

Listing 7.1 Customizing POCO Object for JSON Columns
C#
public class EmployeeRec
{
    [ChoJSONRecordField]
    public int Id { get; set; }
    [ChoJSONRecordField]
    [Required]
    [DefaultValue("ZZZ")]
    public string Name { get; set; }
}

Here are the available members to add some customization to it for each property:

  • FieldName - JSON field name. If not specified, POCO object property name will be used as field name.
  • Size - Size of JSON column value
  • NullValue - Special null value text expect to be treated as null value from JSON file at the field level.
  • ErrorMode - This flag indicates if an exception should be thrown if writing and an expected field failed to convert and write. Possible values are:
    • IgnoreAndContinue - Ignore the error and continue to load other properties of the record.
    • ReportAndContinue - Report the error to POCO entity if it is of IChoRecord type.
    • ThrowAndStop - Throw the error and stop the execution.

7.1. DefaultValue

Any POCO entity property can be specified with default value using System.ComponentModel.DefaultValueAttribute. It is the value used to write when the JSON value null (controlled via IgnoreFieldValueMode).

7.2. ChoFallbackValue

Any POCO entity property can be specified with fallback value using ChoETL.ChoFallbackValueAttribute. It is the value used when the property is failed to writer to JSON. Fallback value only set when ErrorMode is either IgnoreAndContinue or ReportAndContinue.

7.3. Type Converters

Most of the primitive types are automatically converted to string/text and save them to JSON file. If the value of the JSON field aren't automatically converted into the text value, you can specify a custom / built-in .NET converters to convert the value to text. These can be either IValueConverter, IChoValueConverter or TypeConverter converters.

There are couple of ways in which you can specify the converters for each field:

  • Declarative Approach
  • Configuration Approach

7.3.1. Declarative Approach

This model is applicable to POCO entity object only. If you have POCO class, you can specify the converters to each property to carry out necessary conversion on them. The samples below show the way to do it.

Listing 7.3.1.1 Specifying Type Converters
C#
public class EmployeeRec
{
    [ChoJSONRecordField]
    [ChoTypeConverter(typeof(IntConverter))]
    public int Id { get; set; }
    [ChoJSONRecordField]
    [Required]
    [DefaultValue("ZZZ")]
    public string Name { get; set; }
}
Listing 7.3.1.2 IntConverter Implementation
C#
public class IntConverter : IValueConverter
{
    public object Convert
    (object value, Type targetType, object parameter, CultureInfo culture)
    {
        return value;
    }
 
    public object ConvertBack
    (object value, Type targetType, object parameter, CultureInfo culture)
    {
        int intValue = (int)value;
        return intValue.ToString("D4");
    }
}

In the example above, we defined custom IntConverter class. And showed how to format 'Id' JSON property with leading zeros.

7.3.2. Configuration Approach

This model is applicable to both dynamic and POCO entity object. This gives freedom to attach the converters to each property at runtime. This takes precedence over the declarative converters on POCO classes.

Listing 7.3.2.1 Specifying TypeConverters
C#
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();

ChoJSONRecordFieldConfiguration idConfig = new ChoJSONRecordFieldConfiguration("Id");
idConfig.AddConverter(new IntConverter());
config.JSONRecordFieldConfigurations.Add(idConfig);

config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name"));
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name1"));

In the above, we construct and attach the IntConverter to 'Id' field using AddConverter helper method in ChoJSONRecordFieldConfiguration object.

Likewise, if you want to remove any converter from it, you can use RemoveConverter on ChoJSONRecordFieldConfiguration object.

7.3.3. Custom Value Converter Approach

This approach allows to attach value converter to each JSON member using Fluent API. This is a quick way to handle any odd conversion process and avoid creating value converter class.

Listing 7.3.3.1 POCO Class
C#
public class EmployeeRec
{
    [ChoJSONRecordField]
    public int Id { get; set; }
    [ChoJSONRecordField(2, FieldName ="Name", QuoteField = true)]
    [Required]
    [DefaultValue("ZZZ")]
    public string Name { get; set; }
}

With the Fluent API, the sample below shows how to attach value converter to Id column.

Listing 7.3.3.2 Attaching Value Converter
C#
using (var dr = new ChoJSONWriter<EmployeeRec>(@"Test.json")
    .WithField(c => c.Id, valueConverter: (v) => 
              ((int)value).ToString("C3", CultureInfo.CurrentCulture))
    )
{
    Console.WriteLine(rec);
}

7.4. Validations

JSONWriter leverages both System.ComponentModel.DataAnnotations and Validation Block validation attributes to specify validation rules for individual fields of POCO entity. Refer to the MSDN site for a list of available DataAnnotations validation attributes.

Listing 7.4.1 Using Validation Attributes in POCO Entity
C#
[ChoJSONRecordObject]
public partial class EmployeeRec
{
    [ChoJSONRecordField(1, FieldName = "id")]
    [ChoTypeConverter(typeof(IntConverter))]
    [Range(1, int.MaxValue, ErrorMessage = "Id must be > 0.")]
    [ChoFallbackValue(1)]
    public int Id { get; set; }
 
    [ChoJSONRecordField]
    [Required]
    [DefaultValue("ZZZ")]
    [ChoFallbackValue("XXX")]
    public string Name { get; set; }
}

In the example above, I have used Range validation attribute for Id property. Required validation attribute to Name property. JSONWriter performs validation on them before saving the data to file when Configuration.ObjectValidationMode is set to ChoObjectValidationMode.MemberLevel or ChoObjectValidationMode.ObjectLevel.

In some cases, you may want to take control and perform manual self validation within the POCO entity class. This can be achieved by inheriting POCO object from IChoValidatable interface.

Listing 7.4.2 Manual Validation on POCO Entity
C#
[ChoJSONRecordObject]
public partial class EmployeeRec : IChoValidatable
{
    [ChoJSONRecordField]
    [ChoTypeConverter(typeof(IntConverter))]
    [Range(1, int.MaxValue, ErrorMessage = "Id must be > 0.")]
    [ChoFallbackValue(1)]
    public int Id { get; set; }
 
    [ChoJSONRecordField]
    [Required]
    [DefaultValue("ZZZ")]
    [ChoFallbackValue("XXX")]
    public string Name { get; set; }
 
    public bool TryValidate
    (object target, ICollection<ValidationResult> validationResults)
    {
        return true;
    }
 
    public bool TryValidateFor(object target, string memberName, 
                ICollection<ValidationResult> validationResults)
    {
        return true;
    }
}

The sample above shows how to implement custom self-validation in POCO object.

IChoValidatable interface exposes the below methods:

  • TryValidate - Validate entire object, return true if all validation passed. Otherwise, return false.
  • TryValidateFor - Validate specific property of the object, return true if all validation passed. Otherwise, return false.

7.5. ChoIgnoreMember

If you want to ignore a POCO class member from JSON parsing in OptOut mode, decorate them with ChoIgnoreMemberAttribute. The sample below shows Title member is ignored from JSON loading process.

Listing 7.5.1 Ignore a Member
C#
public class EmployeeRec
{
    public int Id { get; set; }
    public string Name { get; set; }
    [ChoIgnoreMember]
    public string Title { get; set; }
}

7.6. StringLength

In OptOut mode, you can specify the size of the JSON column by using System.ComponentModel.DataAnnotations.StringLengthAttribute.

Listing 7.6.1 Specifying Size of JSON Member
C#
public class EmployeeRec
{
    public int Id { get; set; }
    [StringLength(25)]
    public string Name { get; set; }
    [ChoIgnoreMember]
    public string Title { get; set; }
}

7.6. Display

In OptOut mode, you can specify the name of JSON column mapped to member using System.ComponentModel.DataAnnotations.DisplayAttribute.

Listing 7.6.1 Specifying Name of JSON Column
C#
public class EmployeeRec
{
    public int Id { get; set; }
    [Display(Name="FullName")]
    [StringLength(25)]
    public string Name { get; set; }
    [ChoIgnoreMember]
    public string Title { get; set; }
}

7.7. DisplayName

In OptOut mode, you can specify the name of JSON column mapped to member using System.ComponentModel.DataAnnotations.DisplayNameAttribute.

Listing 7.7.1 Specifying Name of JSON Column
C#
public class EmployeeRec
{
    public int Id { get; set; }
    [Display(Name="FullName")]
    [StringLength(25)]
    public string Name { get; set; }
    [ChoIgnoreMember]
    public string Title { get; set; }
}

8. Callback Mechanism

JSONWriter offers industry standard JSON data file generation out of the box to handle most of the needs. If the generation process is not handling any of your needs, you can use the callback mechanism offered by JSONWriter to handle such situations. In order to participate in the callback mechanism, you can use one of the following models:

  • Using event handlers exposed by JSONWriter via IChoWriter interface
  • Inheriting POCO entity object from IChoNotifyRecordWrite / IChoNotifyFileWrite / IChoNotifyRecordFieldWrite interfaces
  • Inheriting DataAnnotation's MetadataType type object by IChoNotifyRecordWrite / IChoNotifyFileWrite / IChoNotifyRecordFieldWrite interfaces.

In order to participate in the callback mechanism, either POCO entity object or DataAnnotation's MetadataType type object must be inherited by IChoNotifyRecordWrite interface.

Tip: Any exceptions raised out of these interface methods will be ignored.

IChoRecorder exposes the below methods:

  • BeginWrite - Invoked at the beginning of the JSON file write
  • EndWrite - Invoked at the end of the JSON file write
  • BeforeRecordWrite - Raised before the JSON record write
  • AfterRecordWrite - Raised after JSON record write
  • RecordWriteError - Raised when JSON record errors out while writing
  • BeforeRecordFieldWrite - Raised before JSON column value write
  • AfterRecordFieldWrite - Raised after JSON column value write
  • RecordFieldWriteError - Raised when JSON column value errors out while writing

IChoNotifyRecordWrite exposes the below methods:

  • BeforeRecordWrite - Raised before the JSON record write
  • AfterRecordWrite - Raised after JSON record write
  • RecordWriteError - Raised when JSON record write errors out

IChoNotifyFileWrite exposes the below methods:

  • BeginWrite - Invoked at the beginning of the JSON file write
  • EndWrite - Invoked at the end of the JSON file write

IChoNotifyRecordFieldWrite exposes the below methods:

  • BeforeRecordFieldWrite - Raised before JSON column value write
  • AfterRecordFieldWrite - Raised after JSON column value write
  • RecordFieldWriteError - Raised when JSON column value write errors out

IChoNotifyFileHeaderArrange exposes the below methods:

  • FileHeaderArrange - Raised before JSON file header is written to file, an opportunity to rearrange the JSON columns

IChoNotifyFileHeaderWrite exposes the below methods:

  • FileHeaderWrite - Raised before JSON file header is written to file, an opportunity to customize the header.

8.1 Using JSONWriter Events

This is more direct and the simplest way to subscribe to the callback events and handle your odd situations in parsing JSON files. The downside is that code can't be reusable as you do by implementing IChoNotifyRecordRead with POCO record object.

The sample below shows how to use the BeforeRecordLoad callback method to skip lines stating with '%' characters.

Listing 8.1.1 Using JSONWriter Callback Events
C#
static void IgnoreLineTest()
{
    using (var parser = new ChoJSONWriter("emp.json"))
    { 
        parser.BeforeRecordWrite += (o, e) =>
        {
            if (e.Source != null)
            {
                e.Skip = ((JObject)e.Source).Contains("name1");
            }
        };
        
        parser.Write(rec);
    }
}

Likewise, you can use other callback methods as well with JSONWriter.

8.2 Implementing IChoNotifyRecordWrite Interface

The sample below shows how to implement IChoNotifyRecordWrite interface to direct POCO class.

Listing 8.2.1 Direct POCO Callback Mechanism Implementation
C#
[ChoJSONRecordObject]
public partial class EmployeeRec : IChoNotifyrRecordWrite
{
    [ChoJSONRecordField]
    [ChoTypeConverter(typeof(IntConverter))]
    [Range(1, int.MaxValue, ErrorMessage = "Id must be > 0.")]
    [ChoFallbackValue(1)]
    public int Id { get; set; }
    
    [ChoJSONRecordField]
    [Required]
    [DefaultValue("ZZZ")]
    [ChoFallbackValue("XXX")]
    public string Name { get; set; }

    public bool AfterRecordWrite(object target, int index, object source)
    {
        throw new NotImplementedException();
    }

    public bool BeforeRecordWrite(object target, int index, ref object source)
    {
        throw new NotImplementedException();
    }

    public bool RecordWriteError(object target, int index, object source, Exception ex)
    {
        throw new NotImplementedException();
    }
}

The sample below shows how to attach Metadata class to POCO class by using MetadataTypeAttribute on it.

Listing 8.2.2 MetaDataType Based Callback Mechanism Implementation
C#
[ChoJSONRecordObject]
public class EmployeeRecMeta : IChoNotifyRecordWrite
{
    [ChoJSONRecordField]
    [ChoTypeConverter(typeof(IntConverter))]
    [Range(1, int.MaxValue, ErrorMessage = "Id must be > 0.")]
    [ChoFallbackValue(1)]
    public int Id { get; set; }

    [ChoJSONRecordField]
    [Required]
    [DefaultValue("ZZZ")]
    [ChoFallbackValue("XXX")]
    public string Name { get; set; }

    public bool AfterRecordWrite(object target, int index, object source)
    {
        throw new NotImplementedException();
    }

    public bool BeforeRecordWrite(object target, int index, ref object source)
    {
        throw new NotImplementedException();
    }

    public bool RecordWriteError(object target, int index, object source, Exception ex)
    {
        throw new NotImplementedException();
    }
} 

[MetadataType(typeof(EmployeeRecMeta))]
public partial class EmployeeRec
{
    public int Id { get; set; }
    public string Name { get; set; }
}

The sample below shows how to attach Metadata class for sealed or third party POCO class by using ChoMetadataRefTypeAttribute on it.

Listing 8.2.2 MetaDataType Based Callback Mechanism Implementation
C#
[ChoMetadataRefType(typeof(EmployeeRec))]
[ChoJSONRecordObject]
public class EmployeeRecMeta : IChoNotifyRecordWrite
{
    [ChoJSONRecordField]
    [ChoTypeConverter(typeof(IntConverter))]
    [Range(1, int.MaxValue, ErrorMessage = "Id must be > 0.")]
    [ChoFallbackValue(1)]
    public int Id { get; set; }

    [ChoJSONRecordField]
    [Required]
    [DefaultValue("ZZZ")]
    [ChoFallbackValue("XXX")]
    public string Name { get; set; }

    public bool AfterRecordWrite(object target, int index, object source)
    {
        throw new NotImplementedException();
    }

    public bool BeforeRecordWrite(object target, int index, ref object source)
    {
        throw new NotImplementedException();
    }

    public bool RecordWriteError(object target, int index, object source, Exception ex)
    {
        throw new NotImplementedException();
    }
} 

public partial class EmployeeRec
{
    public int Id { get; set; }
    public string Name { get; set; }
}

8.3 BeginWrite

This callback is invoked once at the beginning of the JSON file write. source is the JSON file stream object. In here, you have a chance to inspect the stream, return true to continue the JSON generation. Return false to stop the generation.

Listing 8.3.1 BeginWrite Callback Sample
C#
public bool BeginWrite(object source)
{
    StreamReader sr = source as StreamReader;
    return true;
}

8.4 EndWrite

This callback is invoked once at the end of the JSON file generation. source is the JSON file stream object. In here, you have a chance to inspect the stream, do any post steps to be performed on the stream.

Listing 8.4.1 EndWrite Callback Sample
C#
public void EndWrite(object source)
{
    StreamReader sr = source as StreamReader;
}

8.5 BeforeRecordWrite

This callback invoked before each POCO record object is written to JSON file. target is the instance of the POCO record object. index is the line index in the file. source is the JSON record line. In here, you have a chance to inspect the POCO object, and generate the JSON record line if needed.

Tip: If you want to skip the record from writing, set the source to null.

Return true to continue the load process, otherwise return false to stop the process.

Listing 8.5.1 BeforeRecordWrite Callback Sample
C#
public bool BeforeRecordWrite(object target, int index, ref object source)
{
    return true;
}

8.6 AfterRecordWrite

This callback invoked after each POCO record object is written to JSON file. target is the instance of the POCO record object. index is the line index in the file. source is the JSON record line. In here, you have a chance to do any post step operation with the record line.

Return true to continue the load process, otherwise return false to stop the process.

Listing 8.6.1 AfterRecordWrite Callback Sample
C#
public bool AfterRecordWrite(object target, int index, object source)
{
    return true;
}

8.7 RecordWriteError

This callback is invoked if error is encountered while writing POCO record object. target is the instance of the POCO record object. index is the line index in the file. source is the JSON record line. ex is the exception object. In here, you have a chance to handle the exception. This method is invoked only when Configuration.ErrorMode is ReportAndContinue.

Return true to continue the load process, otherwise return false to stop the process.

Listing 8.7.1 RecordWriteError Callback Sample
C#
public bool RecordLoadError(object target, int index, object source, Exception ex)
{
    return true;
}

8.8 BeforeRecordFieldWrite

This callback invoked before each JSON record column is written to JSON file. target is the instance of the POCO record object. index is the line index in the file. propName is the JSON record property name. value is the JSON column value. In here, you have a chance to inspect the JSON record property value and perform any custom validations, etc.

Return true to continue the load process, otherwise return false to stop the process.

Listing 8.8.1 BeforeRecordFieldWrite Callback Sample
C#
public bool BeforeRecordFieldWrite
       (object target, int index, string propName, ref object value)
{
    return true;
}

8.9 AfterRecordFieldWrite

This callback invoked after each JSON record column value is written to JSON file. target is the instance of the POCO record object. index is the line index in the file. propName is the JSON record property name. value is the JSON column value. Any post field operation can be performed here, like computing other properties, validations, etc.

Return true to continue the load process, otherwise return false to stop the process.

Listing 8.9.1 AfterRecordFieldWrite Callback Sample
C#
public bool AfterRecordFieldWrite
       (object target, int index, string propName, object value)
{
    return true;
}

8.10 RecordWriteFieldError

This callback is invoked when an error is encountered while writing JSON record column value. target is the instance of the POCO record object. index is the line index in the file. propName is the JSON record property name. value is the JSON column value. ex is the exception object. In here, you have a chance to handle the exception. This method is invoked only after the below two sequences of steps are performed by the JSONWriter:

  • JSONWriter looks for FallbackValue value of each JSON property. If present, it tries to use it to write.
  • If the FallbackValue value is not present and the Configuration.ErrorMode is specified as ReportAndContinue, this callback will be executed.

Return true to continue the load process, otherwise return false to stop the process.

Listing 8.10.1 RecordFieldWriteError Callback Sample
C#
public bool RecordFieldWriteError
(object target, int index, string propName, object value, Exception ex)
{
    return true;
}

9. Customization

JSONWriter automatically detects and loads the configuration settings from POCO entity. At runtime, you can customize and tweak these parameters before JSON generation. JSONWriter exposes Configuration property, it is of ChoJSONRecordConfiguration object. Using this property, you can perform the customization.

Listing 9.1 Customizing JSONWriter at Run-time
C#
class Program
{
    static void Main(string[] args)
    {        
        List<ExpandoObject> objs = new List<ExpandoObject>();
        dynamic rec1 = new ExpandoObject();
        rec1.Id = 1;
        rec1.Name = "Mark";
        objs.Add(rec1);

        dynamic rec2 = new ExpandoObject();
        rec2.Id = 2;
        rec2.Name = "Jason";
        objs.Add(rec2);

        using (var parser = new ChoJSONWriter("emp.json"))
        {
            parser.Configuration.ColumnCountStrict = true;
            parser.Write(objs);
        }
    }
}

10. Using Dynamic Object

So far, the article explained about using JSONWriter with POCO object. JSONWriter also supports generating JSON file without POCO entity objects. It leverages .NET dynamic feature. The sample below shows how to generate JSON stream using dynamic objects. The JSON schema is determined from first object. If there is mismatch found in the dynamic objects member values, an error will be raised and will stop the generation process.

The sample below shows it:

Listing 10.1 Generating JSON File from Dynamic Objects
C#
class Program
{
    static void Main(string[] args)
    {        
        List<ExpandoObject> objs = new List<ExpandoObject>();
        dynamic rec1 = new ExpandoObject();
        rec1.Id = 1;
        rec1.Name = "Mark";
        objs.Add(rec1);

        dynamic rec2 = new ExpandoObject();
        rec2.Id = 2;
        rec2.Name = "Jason";
        objs.Add(rec2);

        using (var parser = new ChoJSONWriter("emp.json"))
        {
            parser.Configuration.ColumnCountStrict = true;
            parser.Write(objs);
        }
    }
}

11. Exceptions

JSONWriter throws different types of exceptions in different situations.

  • ChoParserException - JSON file is bad and parser will not be able to recover.
  • ChoRecordConfigurationException - Any invalid configuration settings are specified, this exception will be raised.
  • ChoMissingRecordFieldException - A property is missing for a JSON column, this exception will be raised.

12. Using MetadataType Annotation

Cinchoo ETL works better with data annotation's MetadataType model. It is a way to attach MetaData class to data model class. In this associated class, you provide additional metadata information that is not in the data model. Its role is to add attribute to a class without having to modify this one. You can add this attribute that takes a single parameter to a class that will have all the attributes. This is useful when the POCO classes are auto generated (by Entity Framework, MVC, etc.) by an automatic tool. This is why the second class comes into play. You can add new stuff without touching the generated file. Also, this promotes modularization by separating the concerns into multiple classes.

For more information about it, please search on MSDN.

Listing 12.1 MetadataType Annotation Usage Sample
C#
[MetadataType(typeof(EmployeeRecMeta))]
public class EmployeeRec
{
    public int Id { get; set; }
    public string Name { get; set; }
}

[ChoJSONRecordObject]
public class EmployeeRecMeta : IChoNotifyRecordWrite, IChoValidatable
{
    [ChoJSONRecordField]
    [ChoTypeConverter(typeof(IntConverter))]
    [Range(1, 1, ErrorMessage = "Id must be > 0.")]
    [ChoFallbackValue(1)]
    public int Id { get; set; }

    [ChoJSONRecordField]
    [StringLength(1)]
    [DefaultValue("ZZZ")]
    [ChoFallbackValue("XXX")]
    public string Name { get; set; }

    public bool AfterRecordWrite(object target, int index, object source)
    {
        throw new NotImplementedException();
    }

    public bool BeforeRecordWrite(object target, int index, ref object source)
    {
        throw new NotImplementedException();
    }

    public bool RecordWriteError(object target, int index, object source, Exception ex)
    {
        throw new NotImplementedException();
    }
 
    public bool TryValidate
           (object target, ICollection<ValidationResult> validationResults)
    {
        return true;
    }
 
    public bool TryValidateFor
    (object target, string memberName, ICollection<ValidationResult> validationResults)
    {
        return true;
    }
}

In the above, EmployeeRec is the data class. It contains only domain specific properties and operations. Mark it as a very simple class to look at it.

We separate the validation, callback mechanism, configuration, etc. into metadata type class, EmployeeRecMeta.

13. Configuration Choices

If the POCO entity class is an auto-generated class or exposed via library or it is a sealed class, it limits you to attach JSON schema definition to it declaratively. In such a case, you can choose one of the options below to specify JSON layout configuration:

  • Manual Configuration
  • Auto Map Configuration
  • Attaching MetadataType class

I'm going to show you how to configure the below POCO entity class on each approach.

Listing 13.1 Sealed POCO Entity Class
C#
public sealed class EmployeeRec
{
    public int Id { get; set; }
    public string Name { get; set; }
}

13.1 Manual Configuration

Define a brand new configuration object from scratch and add all the necessary JSON fields to the ChoJSONConfiguration.JSONRecordFieldConfigurations collection property. This option gives you greater flexibility to control the configuration of JSON parsing. But the downside is the possibility of making mistakes and it is hard to manage them if the JSON file layout is large.

Listing 13.1.1 Manual Configuration
C#
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.ThrowAndStopOnMissingField = true;
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Id"));
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name"));

13.2 Auto Map Configuration

This is an alternative approach and very less error-prone method to auto map the JSON columns for the POCO entity class.

First, define a schema class for EmployeeRec POCO entity class as below:

Listing 13.2.1 Auto Map class
C#
public class EmployeeRecMap
{
    [ChoJSONRecordField]
    public int Id { get; set; }
 
    [ChoJSONRecordField]
    public string Name { get; set; } 
}

Then, you can use it to auto map JSON columns by using ChoJSONRecordConfiguration.MapRecordFields method.

Listing 13.2.2 Using Auto Map configuration
C#
ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
config.MapRecordFields<EmployeeRecMap>();

EmployeeRec rec1 = new EmployeeRec();
rec1.Id = 2;
rec1.Name = "Jason";

foreach (var e in new ChoJSONWriter<EmployeeRec>("emp.json", config)) 
    w.Write(rec1);

13.3 Attaching MetadataType Class

This is one another approach to attach MetadataType class for POCO entity object. The previous approach simply cares for auto mapping of JSON columns only. Other configuration properties like property converters, parser parameters, default/fallback values, etc. are not considered.

This model accounts for everything by defining MetadataType class and specifying the JSON configuration parameters declaratively. This is useful when your POCO entity is sealed and not partial class. Also, it is one of the favorable and less error-prone approaches to configure JSON parsing of POCO entity.

Listing 13.3.1 Define MetadataType Class
C#
[ChoJSONRecordObject]
public class EmployeeRecMeta : IChoNotifyRecordWrite, IChoValidatable
{
    [ChoJSONRecordField]
    [ChoTypeConverter(typeof(IntConverter))]
    [Range(1, 1, ErrorMessage = "Id must be > 0.")]
    public int Id { get; set; }

    [ChoJSONRecordField]
    [StringLength(1)]
    [DefaultValue("ZZZ")]
    [ChoFallbackValue("XXX")]
    public string Name { get; set; }

    public bool AfterRecordWrite(object target, int index, object source)
    {
        throw new NotImplementedException();
    }

    public bool BeforeRecordWrite(object target, int index, ref object source)
    {
        throw new NotImplementedException();
    }

    public bool RecordWriteError(object target, int index, object source, Exception ex)
    {
        throw new NotImplementedException();
    }
 
    public bool TryValidate
    (object target, ICollection<ValidationResult> validationResults)
    {
        return true;
    }
 
    public bool TryValidateFor(object target, string memberName, 
                               ICollection<ValidationResult> validationResults)
    {
        return true;
    }
}
Listing 13.3.2 Attaching MetadataType Class
C#
//Attach metadata 
ChoMetadataObjectCache.Default.Attach<EmployeeRec>(new EmployeeRecMeta());

using (var tx = File.OpenWrite("emp.json"))
{
    using (var parser = new ChoJSONWriter<EmployeeRec>(tx))
    {
        parser.Write(objs);
    }
}

14. ToTextAll Helper Method

This is a little nifty helper method to generate JSON formatted output from list of objects. It helps you to run and play with different options to see the JSON output quickly in test environment.

C#
static void ToTextTest()
{
    List<EmployeeRec> objs = new List<EmployeeRec>();
    EmployeeRec rec1 = new EmployeeRec();
    rec1.Id = 10;
    rec1.Name = "Mark";
    objs.Add(rec1);
 
    EmployeeRec rec2 = new EmployeeRec();
    rec2.Id = 200;
    rec2.Name = "Lou";
    objs.Add(rec2);
 
    Console.WriteLine(ChoJSONWriter.ToTextAll(objs));
}

14a. ToText Helper Method

This is a little nifty helper method to generate JSON formatted output from an object. It helps you to run and play with different options to see the JSON output quickly in test environment.

C#
static void ToTextTest()
{
    EmployeeRec rec1 = new EmployeeRec();
    rec1.Id = 10;
    rec1.Name = "Mark";
    objs.Add(rec1);
 
    Console.WriteLine(ChoJSONWriter.ToText(rec1));
}

15. Writing DataReader Helper Method

This helper method lets you create a JSON file / stream from ADO.NET DataReader.

C#
static void WriteDataReaderTest()
{
    SqlConnection conn = new SqlConnection(connString);
    conn.Open();
    SqlCommand cmd = new SqlCommand("SELECT * FROM Members", conn);
    IDataReader dr = cmd.ExecuteReader();

    StringBuilder json = new StringBuilder();
    using (var parser = new ChoJSONWriter(json))
    {
        parser.Write(dr);
    }
    Console.WriteLine(json.ToString());
}

16. Writing DataTable Helper Method

This helper method lets you to create JSON file / stream from ADO.NET DataTable.

C#
static void WriteDataTableTest()
{
    string connString = @"Data Source=(localdb)\v11.0;
                        Initial Catalog=TestDb;Integrated Security=True";

    SqlConnection conn = new SqlConnection(connString);
    conn.Open();
    SqlCommand cmd = new SqlCommand("SELECT * FROM Members", conn);
    SqlDataAdapter da = new SqlDataAdapter(cmd);
    DataTable dt = new DataTable();
    da.Fill(dt);
 
    StringBuilder json = new StringBuilder();
    using (var parser = new ChoJSONWriter(json)
        )
    {
        parser.Write(dt);
    }

    Console.WriteLine(json.ToString());
}

17. Advanced Topics

17.1 Override Converters Format Specs

Cinchoo ETL automatically parses and converts each JSON column values to the corresponding JSON column's underlying data type seamlessly. Most of the basic .NET types are handled automatically without any setup needed.

This is achieved through two key settings in the ETL system:

  1. ChoJSONRecordConfiguration.CultureInfo - Represents information about a specific culture including the names of the culture, the writing system, and the calendar used, as well as access to culture-specific objects that provide information for common operations, such as formatting dates and sorting strings. Default is 'en-US'.
  2. ChoTypeConverterFormatSpec - It is a global format specifier class that holds all the intrinsic .NET types formatting specs.

In this section, I'm going to talk about changing the default format specs for each .NET intrinsic data types according to parsing needs.

ChoTypeConverterFormatSpec is singleton class, the instance is exposed via 'Instance' static member. It is thread local, which means that there will be a separate instance copy kept on each thread.

There are two sets of format specs members given to each intrinsic type, one for loading and another one for writing the value, except for Boolean, Enum, DataTime types. These types have only one member for both loading and writing operations.

Specifying each intrinsic data type format specs through ChoTypeConverterFormatSpec will impact system wide, i.e., by setting ChoTypeConverterFormatSpec.IntNumberStyle = NumberStyles.AllowParentheses will impact all integer members of JSON objects to allow parentheses. If you want to override this behavior and take control of specific JSON data member to handle its own unique parsing of JSON value from global system wide setting, it can be done by specifying TypeConverter at the JSON field member level. Refer to section 13.4 for more information.

Listing 17.1.1 ChoTypeConverterFormatSpec Members
C#
public class ChoTypeConverterFormatSpec
{
    public static readonly ThreadLocal<ChoTypeConverterFormatSpec> Instance = 
    new ThreadLocal<ChoTypeConverterFormatSpec>(() => new ChoTypeConverterFormatSpec());
 
    public string DateTimeFormat { get; set; }
    public ChoBooleanFormatSpec BooleanFormat { get; set; }
    public ChoEnumFormatSpec EnumFormat { get; set; }
 
    public NumberStyles? CurrencyNumberStyle { get; set; }
    public string CurrencyFormat { get; set; }
 
    public NumberStyles? BigIntegerNumberStyle { get; set; }
    public string BigIntegerFormat { get; set; }
 
    public NumberStyles? ByteNumberStyle { get; set; }
    public string ByteFormat { get; set; }
 
    public NumberStyles? SByteNumberStyle { get; set; }
    public string SByteFormat { get; set; }
 
    public NumberStyles? DecimalNumberStyle { get; set; }
    public string DecimalFormat { get; set; }
 
    public NumberStyles? DoubleNumberStyle { get; set; }
    public string DoubleFormat { get; set; }
 
    public NumberStyles? FloatNumberStyle { get; set; }
    public string FloatFormat { get; set; }
 
    public string IntFormat { get; set; }
    public NumberStyles? IntNumberStyle { get; set; }
 
    public string UIntFormat { get; set; }
    public NumberStyles? UIntNumberStyle { get; set; }
 
    public NumberStyles? LongNumberStyle { get; set; }
    public string LongFormat { get; set; }
 
    public NumberStyles? ULongNumberStyle { get; set; }
    public string ULongFormat { get; set; }
 
    public NumberStyles? ShortNumberStyle { get; set; }
    public string ShortFormat { get; set; }
 
    public NumberStyles? UShortNumberStyle { get; set; }
    public string UShortFormat { get; set; }
}

The sample below shows how to load JSON data stream having 'se-SE' (Swedish) culture specific data using JSONWriter. Also, the input feed comes with 'EmployeeNo' values containing parentheses. In order to make the load successful, we have to set the ChoTypeConverterFormatSpec.IntNumberStyle to NumberStyles.AllowParenthesis.

Listing 17.1.2 Using ChoTypeConverterFormatSpec in Code
C#
static void FormatSpecDynamicTest()
{
    ChoTypeConverterFormatSpec.Instance.DateTimeFormat = "d";
    ChoTypeConverterFormatSpec.Instance.BooleanFormat = ChoBooleanFormatSpec.YOrN;
 
    List<ExpandoObject> objs = new List<ExpandoObject>();
    dynamic rec1 = new ExpandoObject();
    rec1.Id = 10;
    rec1.Name = "Mark";
    rec1.JoinedDate = new DateTime(2001, 2, 2);
    rec1.IsActive = true;
    objs.Add(rec1);
 
    dynamic rec2 = new ExpandoObject();
    rec2.Id = 200;
    rec2.Name = "Lou";
    rec2.JoinedDate = new DateTime(1990, 10, 23);
    rec2.IsActive = false;
    objs.Add(rec2);
 
    StringBuilder json = new StringBuilder();
    using (var parser = new ChoJSONWriter(json)
        )
    {
        parser.Write(objs);
    }

    Console.WriteLine(json.ToString());
}

17.2 Currency Support

Cinchoo ETL provides ChoCurrency object to read and write currency values in JSON files. ChoCurrency is a wrapper class to hold the currency value in decimal type along with support of serializing them in text format during JSON load.

Listing 17.2.1 Using Currency Members in Dynamic Model
C#
static void CurrencyDynamicTest()
{
    ChoTypeConverterFormatSpec.Instance.CurrencyFormat = "C2";
 
    List<ExpandoObject> objs = new List<ExpandoObject>();
    dynamic rec1 = new ExpandoObject();
    rec1.Id = 10;
    rec1.Name = "Mark";
    rec1.JoinedDate = new DateTime(2001, 2, 2);
    rec1.IsActive = true;
    rec1.Salary = new ChoCurrency(100000);
    objs.Add(rec1);
 
    dynamic rec2 = new ExpandoObject();
    rec2.Id = 200;
    rec2.Name = "Lou";
    rec2.JoinedDate = new DateTime(1990, 10, 23);
    rec2.IsActive = false;
    rec2.Salary = new ChoCurrency(150000);
    objs.Add(rec2);
 
    StringBuilder json = new StringBuilder();
    using (var parser = new ChoJSONWriter(json)
        )
    {
        parser.Write(objs);
    }

    Console.WriteLine(json.ToString());
}

The sample above shows how to output currency values using dynamic object model. As the currency output will have thousand comma separator, this will fail to generate JSON file. To overcome this issue, we specify the writer to quote all fields.

P.S.: The format of the currency value is figured by JSONWriter through ChoRecordConfiguration.Culture and ChoTypeConverterFormatSpec.CurrencyFormat.

The sample below shows how to use ChoCurrency JSON field in POCO entity class.

Listing 17.2.2 Using Currency Members in POCO Model
C#
public class EmployeeRecWithCurrency
{
    public int Id { get; set; }
    public string Name { get; set; }
    public ChoCurrency Salary { get; set; }
}
 
static void CurrencyPOCOTest()
{
    List<EmployeeRecWithCurrency> objs = new List<EmployeeRecWithCurrency>();
    EmployeeRecWithCurrency rec1 = new EmployeeRecWithCurrency();
    rec1.Id = 10;
    rec1.Name = "Mark";
    rec1.Salary = new ChoCurrency(100000);
    objs.Add(rec1);
 
    EmployeeRecWithCurrency rec2 = new EmployeeRecWithCurrency();
    rec2.Id = 200;
    rec2.Name = "Lou";
    rec2.Salary = new ChoCurrency(150000);
    objs.Add(rec2);
 
    StringBuilder json = new StringBuilder();
    using (var parser = new ChoJSONWriter<EmployeeRecWithCurrency>(json)
        )
    {
        parser.Write(objs);
    }

    Console.WriteLine(json.ToString());
}

17.3 Enum Support

Cinchoo ETL implicitly handles parsing/writing of enum column values from JSON files. If you want to fine control the parsing of these values, you can specify them globally via ChoTypeConverterFormatSpec.EnumFormat. Default is ChoEnumFormatSpec.Value.

FYI, changing this value will impact system wide.

There are three possible values that can be used:

  1. ChoEnumFormatSpec.Value - Enum value is used for parsing.
  2. ChoEnumFormatSpec.Name - Enum key name is used for parsing.
  3. ChoEnumFormatSpec.Description - If each enum key is decorated with DescriptionAttribute, its value will be used for parsing.
Listing 17.3.1 Specifying Enum Format Specs During Parsing
C#
public enum EmployeeType
{
    [Description("Full Time Employee")]
    Permanent = 0,
    [Description("Temporary Employee")]
    Temporary = 1,
    [Description("Contract Employee")]
    Contract = 2
}
 
static void EnumTest()
{
    ChoTypeConverterFormatSpec.Instance.EnumFormat = ChoEnumFormatSpec.Description;
 
    List<ExpandoObject> objs = new List<ExpandoObject>();
    dynamic rec1 = new ExpandoObject();
    rec1.Id = 10;
    rec1.Name = "Mark";
    rec1.JoinedDate = new DateTime(2001, 2, 2);
    rec1.IsActive = true;
    rec1.Salary = new ChoCurrency(100000);
    rec1.Status = EmployeeType.Permanent;
    objs.Add(rec1);
 
    dynamic rec2 = new ExpandoObject();
    rec2.Id = 200;
    rec2.Name = "Lou";
    rec2.JoinedDate = new DateTime(1990, 10, 23);
    rec2.IsActive = false;
    rec2.Salary = new ChoCurrency(150000);
    rec2.Status = EmployeeType.Contract;
    objs.Add(rec2);

    StringBuilder json = new StringBuilder();
    using (var parser = new ChoJSONWriter(json)
        )
    {
        parser.Write(objs);
    }

    Console.WriteLine(json.ToString());
}

17.4 Boolean Support

Cinchoo ETL implicitly handles parsing/writing of boolean JSON column values from JSON files. If you want to fine control the parsing of these values, you can specify them globally via ChoTypeConverterFormatSpec.BooleanFormat. Default value is ChoBooleanFormatSpec.ZeroOrOne:

FYI, changing this value will impact system wide.

There are four possible values that can be used:

  1. ChoBooleanFormatSpec.ZeroOrOne - '0' for false. '1' for true.
  2. ChoBooleanFormatSpec.YOrN - 'Y' for true, 'N' for false.
  3. ChoBooleanFormatSpec.TrueOrFalse - 'True' for true, 'False' for false.
  4. ChoBooleanFormatSpec.YesOrNo - 'Yes' for true, 'No' for false.
Listing 17.4.1 Specifying Boolean Format Specs During Parsing
C#
static void BoolTest()
{
    ChoTypeConverterFormatSpec.Instance.BooleanFormat = ChoBooleanFormatSpec.YOrN;
 
    List<ExpandoObject> objs = new List<ExpandoObject>();
    dynamic rec1 = new ExpandoObject();
    rec1.Id = 10;
    rec1.Name = "Mark";
    rec1.JoinedDate = new DateTime(2001, 2, 2);
    rec1.IsActive = true;
    rec1.Salary = new ChoCurrency(100000);
    rec1.Status = EmployeeType.Permanent;
    objs.Add(rec1);
 
    dynamic rec2 = new ExpandoObject();
    rec2.Id = 200;
    rec2.Name = "Lou";
    rec2.JoinedDate = new DateTime(1990, 10, 23);
    rec2.IsActive = false;
    rec2.Salary = new ChoCurrency(150000);
    rec2.Status = EmployeeType.Contract;
    objs.Add(rec2);
 
    StringBuilder json = new StringBuilder();
    using (var parser = new ChoJSONWriter(json)
        )
    {
        parser.Write(objs);
    }

    Console.WriteLine(json.ToString());
}

17.5 DateTime Support

Cinchoo ETL implicitly handles parsing/writing of datetime JSON column values from JSON files using system Culture or custom set culture. If you want to fine control the parsing of these values, you can specify them globally via ChoTypeConverterFormatSpec.DateTimeFormat. Default value is 'd'.

FYI, changing this value will impact system wide.

You can use any valid standard or custom datetime .NET format specification to parse the datetime JSON values from the file.

Listing 17.5.1 Specifying datetime Format Specs during Parsing
C#
static void DateTimeDynamicTest()
{
    ChoTypeConverterFormatSpec.Instance.DateTimeFormat = "MMM dd, yyyy";
 
    List<ExpandoObject> objs = new List<ExpandoObject>();
    dynamic rec1 = new ExpandoObject();
    rec1.Id = 10;
    rec1.Name = "Mark";
    rec1.JoinedDate = new DateTime(2001, 2, 2);
    rec1.IsActive = true;
    rec1.Salary = new ChoCurrency(100000);
    objs.Add(rec1);
 
    dynamic rec2 = new ExpandoObject();
    rec2.Id = 200;
    rec2.Name = "Lou";
    rec2.JoinedDate = new DateTime(1990, 10, 23);
    rec2.IsActive = false;
    rec2.Salary = new ChoCurrency(150000);
    objs.Add(rec2);
 
    StringBuilder json = new StringBuilder();
    using (var parser = new ChoJSONWriter(json)
        )
    {
        parser.Write(objs);
    }

    Console.WriteLine(json.ToString());
}

The sample above shows how to generate custom datetime values to JSON file.

Note: As the datetime values contains JSON separator, we instruct the writer to quote all fields.

18. Fluent API

JSONWriter exposes few frequent to use configuration parameters via Fluent API methods. This will make the programming of generating JSON files quicker.

18.1. NullValueHandling

Specifies null value handling options for the ChoJSONWriter:

  • Ignore - Ignore null values while writing JSON
  • Default - Include null values while writing JSON

18.2. Formatting

Specifies formatting options for the ChoJSONWriter:

  • None - No special formatting is applied. This is the default.
  • Indented - Causes child objects to be indented.

18.3 WithFields

This API method specifies the list of JSON fields to be considered for writing JSON file. Other fields will be discarded. Field names are case-insensitive.

C#
static void QuickDynamicTest()
{
    List<ExpandoObject> objs = new List<ExpandoObject>();
    dynamic rec1 = new ExpandoObject();
    rec1.Id = 10;
    rec1.Name = "Mark";
    rec1.JoinedDate = new DateTime(2001, 2, 2);
    rec1.IsActive = true;
    rec1.Salary = new ChoCurrency(100000);
    objs.Add(rec1);
 
    dynamic rec2 = new ExpandoObject();
    rec2.Id = 200;
    rec2.Name = "Lou";
    rec2.JoinedDate = new DateTime(1990, 10, 23);
    rec2.IsActive = false;
    rec2.Salary = new ChoCurrency(150000);
    objs.Add(rec2);
 
    StringBuilder json = new StringBuilder();
    using (var parser = new ChoJSONWriter(json)
        .WithFields("Id", "Name")
        )
    {
        parser.Write(objs);
    }

    Console.WriteLine(json.ToString());
}

18.4 WithField

This API method is used to add JSON column with specific date type, quote flag, and/or quote character. This method is helpful in dynamic object model, by specifying each and individual JSON column with appropriate datatype.

C#
static void QuickDynamicTest()
{
    List<ExpandoObject> objs = new List<ExpandoObject>();
    dynamic rec1 = new ExpandoObject();
    rec1.Id = 10;
    rec1.Name = "Mark";
    rec1.JoinedDate = new DateTime(2001, 2, 2);
    rec1.IsActive = true;
    rec1.Salary = new ChoCurrency(100000);
    objs.Add(rec1);
 
    dynamic rec2 = new ExpandoObject();
    rec2.Id = 200;
    rec2.Name = "Lou";
    rec2.JoinedDate = new DateTime(1990, 10, 23);
    rec2.IsActive = false;
    rec2.Salary = new ChoCurrency(150000);
    objs.Add(rec2);
 
    StringBuilder json = new StringBuilder();
    using (var parser = new ChoJSONWriter(json)
        .WithField("Id", typeof(int))
        .WithField("Name"))
        )
    {
        parser.Write(objs);
    }

    Console.WriteLine(json.ToString());
}

18.5. IgnoreFieldValueMode

Specifies ignore field value for the ChoJSONWriter:

  • None - Ignore field value is turned off. This is the default.
  • DbNull - DBNull value will be ignored.
  • Empty - Empty text value will be ignored.
  • WhiteSpace - Whitespace text will be ignored.

18.6 ColumnCountStrict

This API method used to set the JSONWriter to perform check on column countnness before writing JSON file.

C#
static void ColumnCountTest()
{
    List<ExpandoObject> objs = new List<ExpandoObject>();
    dynamic rec1 = new ExpandoObject();
    rec1.Id = 10;
    rec1.Name = "Mark";
    rec1.JoinedDate = new DateTime(2001, 2, 2);
    rec1.IsActive = true;
    rec1.Salary = new ChoCurrency(100000);
    objs.Add(rec1);
 
    dynamic rec2 = new ExpandoObject();
    rec2.Id = 200;
    rec2.Name = "Lou";
    rec2.JoinedDate = new DateTime(1990, 10, 23);
    rec2.IsActive = false;
    rec2.Salary = new ChoCurrency(150000);
    objs.Add(rec2);
 
    StringBuilder json = new StringBuilder();
    using (var parser = new ChoJSONWriter(json)
        .ColumnCountStrict()
        )
    {
        parser.Write(objs);
    }

    Console.WriteLine(json.ToString());
}

18.7. Configure

This API method is used to configure all configuration parameters which are not exposed via Fluent API.

C#
static void ConfigureTest()
{
    List<ExpandoObject> objs = new List<ExpandoObject>();
    dynamic rec1 = new ExpandoObject();
    rec1.Id = 10;
    rec1.Name = "Mark";
    objs.Add(rec1);

    dynamic rec2 = new ExpandoObject();
    rec2.Id = 200;
    rec2.Name = "Lou";
    objs.Add(rec2);

    StringBuilder json = new StringBuilder();
    using (var parser = new ChoJSONWriter(json)
        .Configure(c => c.ErrorMode = ChoErrorMode.ThrowAndStop)
        )
    {
        parser.Write(objs);
    }

    Console.WriteLine(json.ToString());
}

18.8. Setup

This API method is used to setup the writer's parameters / events via Fluent API.

C#
static void SetupTest()
{
    List<ExpandoObject> objs = new List<ExpandoObject>();
    dynamic rec1 = new ExpandoObject();
    rec1.Id = 10;
    rec1.Name = "Mark";
    objs.Add(rec1);

    dynamic rec2 = new ExpandoObject();
    rec2.Id = 200;
    rec2.Name = "Lou";
    objs.Add(rec2);

    StringBuilder json = new StringBuilder();
    using (var parser = new ChoJSONWriter(json)
        .Setup(r => r.BeforeRecordWrite += (o, e) =>
        {
        })
        )
    {
        parser.Write(objs);
    }

    Console.WriteLine(json.ToString());
}

19. FAQ

19.1. How to Serialize an Object?

This sample serializes object to JSON:

C#
string json = ChoJSONWriter.Serialize(new Account
{
    Email = "james@example.com",
    Active = true,
    Roles = new List<string>()
        {
            "DEV",
            "OPS"
        }
});

19.2. How to Serialize Collection of Objects?

This sample serializes collections of objects to JSON:

C#
string json = ChoJSONWriter.SerializeAll<Account>(new Account[] {
    new Account
    {
      Email = "james@example.com",
      Active = true,
      Roles = new List<string>()
      {
        "DEV",
        "OPS"
      }
    }
}
);

19.3. How to Serialize Dynamic Object?

This sample serializes dynamic object to JSON:

C#
dynamic obj = new ExpandoObject();
obj.Email = "james@example.com";
obj.Active = true;
obj.Roles = new List<string>()
{
    "DEV",
    "OPS"
};

string json = ChoJSONWriter.Serialize(obj);

Console.WriteLine(json);

19.4. How to Serialize Anonymous Object?

This sample serializes anonymous object to JSON:

C#
string json = ChoJSONWriter.Serialize(new
{
    Email = "james@example.com",
    Active = true,
    Roles = new List<string>()
    {
        "DEV",
        "OPS"
    }
});

Console.WriteLine(json);

19.5. How to Serialize Collection?

This sample serializes collection to JSON:

C#
string json = ChoJSONWriter.SerializeAll(new int[] { 1, 2, 3 });

Console.WriteLine(json);

19.6. How to Serialize Dictionary?

This sample serializes dictionary to JSON:

C#
string json = ChoJSONWriter.SerializeAll(new Dictionary<string, int>[] {
    new Dictionary<string, int>()
    {
        ["key1"] = 1,
        ["key2"] = 2
    }
    });

Console.WriteLine(json);

19.7. How to Serialize DataTable?

This sample serializes datatable to JSON:

C#
StringBuilder sb = new StringBuilder();
string connectionstring = @"Data Source=(localdb)\MSSQLLocalDB;
                          Initial Catalog=Northwind;Integrated Security=True";
using (var conn = new SqlConnection(connectionstring))
{
    conn.Open();
    var comm = new SqlCommand("SELECT TOP 2 * FROM Customers", conn);
    SqlDataAdapter adap = new SqlDataAdapter(comm);

    DataTable dt = new DataTable("Customer");
    adap.Fill(dt);

    using (var parser = new ChoJSONWriter(sb)
        .Configure(c => c.IgnoreRootName = true)
        )
        parser.Write(dt);
}

Console.WriteLine(sb.ToString());

19.8. How to Serialize JSON to a File?

This sample serializes JSON to a file.

C#
// serialize JSON to a string and then write string to a file
File.WriteAllText(@"c:\emp.json", ChoJSONWriter.Serialize(employee));

The sample below shows how to write directly to a file:

C#
using (StreamWriter file = File.CreateText(@"c:\emp.json"))
{
    using (var r = new ChoJSONWriter(file))
          r.Write(employee);
}

19.9. How to Serialize Unindented JSON?

This sample serializes an object to JSON without any formatting or indentation whitespace.

C#
string json = ChoJSONWriter.SerializeAll(new int[] { 1, 2, 3 },
    new ChoJSONRecordConfiguration().Configure(c => c.Formatting = Formatting.None));

Console.WriteLine(json);

19.10. How to Serialize Conditional Property?

This sample uses a conditional property to exclude a property from serialization.

C#
public class Employee
{
    public string Name { get; set; }
    public Employee Manager { get; set; }

    public bool ShouldSerializeManager()
    {
        // don't serialize the Manager property if an employee is their own manager
        return (Manager != this);
    }
}

public static void ConditionalPropertySerialize()
{
    Employee joe = new Employee();
    joe.Name = "Joe Employee";
    Employee mike = new Employee();
    mike.Name = "Mike Manager";

    joe.Manager = mike;

    // mike is his own manager
    // ShouldSerialize will skip this property
    mike.Manager = mike;

    string json = ChoJSONWriter.SerializeAll(new[] { joe, mike }, 
    new ChoJSONRecordConfiguration().Configure(c => c.UseJSONSerialization = true));

    Console.WriteLine(json);
}

19.11. How to Serialize datetime in Custom DateFormat?

This sample uses the DateFormatString setting to control how DateTime and DateTimeOffset are serialized.

C#
IList<DateTime> dateList = new List<DateTime>
{
    new DateTime(2009, 12, 7, 23, 10, 0, DateTimeKind.Utc),
    new DateTime(2010, 1, 1, 9, 0, 0, DateTimeKind.Utc),
    new DateTime(2010, 2, 10, 10, 0, 0, DateTimeKind.Utc)
};

string json = ChoJSONWriter.SerializeAll(dateList, new JsonSerializerSettings
{
    DateFormatString = "d MMMM, yyyy",
    Formatting = Formatting.Indented
});

Console.WriteLine(json);

19.12. How to Exclude Property from Json Serialization?

These samples show how to exclude property from JSON serialization.

Sample 1: Using ChoIgnoreMemberAttribute

C#
public class Account
{
    [ChoIgnoreMember]
    public string Email { get; set; }
    public bool Active { get; set; }
    public DateTime CreatedDate { get; set; }
    public IList<string> Roles { get; set; }
}

static void ExcludePropertyTest()
{
    string json = ChoJSONWriter.Serialize(new Account
    {
        Email = "james@example.com",
        Active = true,
        Roles = new List<string>()
            {
                "DEV",
                "OPS"
            }
    });

    Console.WriteLine(json);
}

Sample 2: Using JsonIgnoreAttribute

C#
public class Account
{
    [JsonIgnore]
    public string Email { get; set; }
    public bool Active { get; set; }
    public DateTime CreatedDate { get; set; }
    public IList<string> Roles { get; set; }
}

static void ExcludePropertyTest()
{
    string json = ChoJSONWriter.Serialize(new Account
    {
        Email = "james@example.com",
        Active = true,
        Roles = new List<string>()
            {
                "DEV",
                "OPS"
            }
    });

    Console.WriteLine(json);
}

Sample 3: Using IgnoreField on ChoJSONConfiguration

C#
public class Account
{
    public string Email { get; set; }
    public bool Active { get; set; }
    public DateTime CreatedDate { get; set; }
    public IList<string> Roles { get; set; }
}

static void ExcludePropertyTest()
{
    string json = ChoJSONWriter.Serialize(new Account
    {
        Email = "james@example.com",
        Active = true,
        Roles = new List<string>()
            {
                "DEV",
                "OPS"
            }

    }, new ChoJSONRecordConfiguration<Account>().Ignore(f => f.Email));

    Console.WriteLine(json);
}

Sample 4: Using IgnoreField on ChoJSONWriter

C#
public class Account
{
    public string Email { get; set; }
    public bool Active { get; set; }
    public DateTime CreatedDate { get; set; }
    public IList<string> Roles { get; set; }
}

static void ExcludePropertyTest()
{
    StringBuilder json = new StringBuilder();
    using (var w = new ChoJSONWriter<Account>(json)
        .IgnoreField(f => f.Email)
        )
    {
        w.Write(new Account
        {
            Email = "james@example.com",
            Active = true,
            Roles = new List<string>()
            {
                "DEV",
                "OPS"
            }
        });
    }

    Console.WriteLine(json);
}

19.13. How to Convert XML to JSON?

This sample shows converting XML file to JSON.

C#
string xml = @"<Employees xmlns=""http://company.com/schemas"">
    <Employee>
        <FirstName>name1</FirstName>
        <LastName>surname1</LastName>
    </Employee>
    <Employee>
        <FirstName>name2</FirstName>
        <LastName>surname2</LastName>
    </Employee>
    <Employee>
        <FirstName>name3</FirstName>
        <LastName>surname3</LastName>
    </Employee>
</Employees>";

StringBuilder json = new StringBuilder();
using (var r = ChoXmlReader.LoadText(xml))
{
    using (var w = new ChoJSONWriter(json))
        w.Write(r);
}

Console.WriteLine(json.ToString());

Output

JavaScript
[
  {
    "FirstName": "name1",
    "LastName": "surname1"
  },
  {
    "FirstName": "name2",
    "LastName": "surname2"
  },
  {
    "FirstName": "name3",
    "LastName": "surname3"
  }
]

19.14. How to Convert CSV to JSON?

This sample shows converting CSV file to JSON:

C#
string csv = @"Id, First Name
    1, Tom
    2, Mark";

StringBuilder json = new StringBuilder();
using (var r = ChoCSVReader.LoadText(csv)
    .WithFirstLineHeader()
    .WithMaxScanRows(2)
    )
{
    using (var w = new ChoJSONWriter(json))
    {
        w.Write(r.Take(2));
    }
}

Console.WriteLine(json.ToString());

20. History

  • 9th October, 2020: Initial version

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
United States United States
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
GeneralMy vote of 5 Pin
Franc Morales6-Oct-21 10:36
Franc Morales6-Oct-21 10:36 
GeneralRe: My vote of 5 Pin
Cinchoo30-Nov-21 4:28
Cinchoo30-Nov-21 4:28 
GeneralMy vote of 5 Pin
Olfello10-Jun-20 8:29
Olfello10-Jun-20 8:29 
GeneralRe: My vote of 5 Pin
Cinchoo30-Nov-21 4:27
Cinchoo30-Nov-21 4:27 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.