Making Application Source Agnostic

• Download source - 65.1 KB

Introduction

A simple approach based on Interface Segregation Principle to keep application completely decoupled from its source of data. It means application relies on interface to get raw-data, this interface gets resolved into a concrete implementation to provide data from a specific external source. So that way actually, we are keeping usage-of-data (application) and sources-of-data (providers) on different layers.

Now the second part of this model is that data providers should return data in the form of CLR objects but APIs should return Interface not the concrete types, e.g., say there is a Trade class implementing ITrade interface so the API GetTrades should not return Trade[] but ITade[].

That way, we define clear contracts for data to be consumed within an application which in turn enables the application to use a source as data provider if that could return data per contract.

Challenges

Theoretically, it appears to be simple but practically there are challenges to realize such concepts. Personally, I feel the primary challenge is to define various data-providers and that's supported APIs.

To give a quick background, an application with moderate complexity may have multiple sources of data which may provide data in various forms & formats. Conveniently, we make all such data to sink into a database so that it could become one-source of every type of data, required in application.

Which is fairly "ok" until we breach the line of data modeling and data processing. Eventually, we end-up putting data massaging and logic processing together into the database itself in this kind of model, which:

1. Hinders horizontal scaling of application in longer run and
2. Prevents you to consume a source directly into application even if that source has turned mature enough to provide realtime data on demand.

Now let's come back to segregation of data providers, one principle is to segregate data providers based on the nature of data they provide.

For example, a trade processing application may have data providers like ITradeDataProvider, ISecurityDataProvider and ISecurityPriceDataProvider to provider trade, security and pricing data respectively. We kept three providers there because these are logically related but different in nature of data and one can easily switch actual source of provider for any of these without impacting the others.

Wire Data Providers

Since we are keeping contract of data providers at application layer but concrete implementation on other layer, we need to have a factory to provide concrete data provider when application wants to have data.

In order to make it happen in a graceful fashion, we should use dependency injection to inject actual data providers to the application, I personally prefer Unity there but it can be any DI framework of your choice.

To realize it, first of all, we need to prepare mapping of interfaces with respective concrete types, design time configuration holds good here but again if you prefer runtime configuration even that's not going to break it anyway. Once we are done with mapping, then we know what concrete type is going to get resolved against an interface, when application demands.

  <register type="IDbConnection" mapTo="SqlConnection">
      <constructor>
        <param name="connectionString" value="Data Source=.\SQLEXPRESS;
Initial Catalog=TradeDb;Integrated Security=SSPI" />
      </constructor>
    </register>

    <register type="ITradeDataProvider" mapTo="TradeDataProvider" >
      <constructor>
        <param name="connection" dependencyType="IDbConnection" />
      </constructor>
      <property name="SProcGetTradeData" value="spAppGetTradeDate" />
    </register>

You can visit this article by Bert for a quick start on unity however, you should visit MSDN developer's guide for more details on how to use Unity DI, if required.

Resolving Data Provider

It's pretty much nothing to be done explicitly here to resolve a provider, Unity does it for us. All we need to do is to make a call to get container prepared (only once) and then a call to resolve type mapped with given interface.

public static class Resolver {
    private static IUnityContainer _container;
    static Resolver() {
       _container = new UnityContainer();
       _container.LoadConfiguration();
   }

   public static T Resolve<T>( ResolverRequest request ) where T: class {
        request = request ?? new ResolverRequest();
       switch ( request.ProviderType ) {
           case DataSourceProviderType.Default:
                return _container.Resolve<T>( );
            case DataSourceProviderType.Persistent:
                return _container.Resolve<T>("Persistent");
            case DataSourceProviderType.Specific:
                return _container.Resolve<T>( request.ProviderName );
            default:
                return default(T);
        }
    }
}

Here, we have written few extra lines of code for our Resolve routine, to meet the case where there may exist more than one mapping of given data provider. Modules may resolve to a specific type based on scenarios at runtime say, our default provider fails to return data then module should attempt to get data from persistent provider.

Based on above mappings & container, we can make a simple interface that should be used as factory to get a concrete instance of data provider in application.

public static class ProviderFactory {
   public static ITradeDataProvider GetTradeDataProvider(ResolverRequest request) {
       return Resolver.Resolve<ITradeDataProvider>( request );
   }
}

Using the Code

If we put everything in place, we can build a working model that would be source agnostic. Say there is a Trade Service to provide prepared objects of ITrade against given trade ids.

public class TradeService {

   public TradeEntity[] GetTrades( IEnumerable<Guid> tradeIds ) {

       ITradeDataContract[] tradeDataFromSource = GetTradeFromSource( tradeIds );
 
       IEnumerable<int> securityTraded = tradeDataFromSource
                                           .Select( item => item.SecurityIdTraded )
                                           .Distinct();       

       ISecurityDataContract[] securityDataFromSource = 
                                    GetSecurityDataFromSource( securityTraded );

       IPriceDataContract[] securityClosingPriceDataFromSource = 
                                   GetPriceDataFromSource(PriceStrategy.ClosingPrice, 
                                                          securityTraded );

       IPriceDataContract[] securityCurrentPriceDataFromSource = 
                                   GetPriceDataFromSource(PriceStrategy.CurrentPrice, 
                                                          securityTraded );

       ICountryDataContract[] countries = 
                            GetReferenceDataFromSource<ICountryDataContract> ( 
                                           ReferenceDataType.CountryData, 
                                           securityDataFromSource
                                             .Select( item => item.CountryCode )
                                             .Distinct() );

       ICurrencyDataContract[] currencies = 
                            GetReferenceDataFromSource<ICurrencyDataContract>( 
                                        ReferenceDataType.CurrencyData, 
                                        tradeDataFromSource
                                          .Select( item => item.TradeCurrency )
                                          .Distinct() );

        return new[] {
            new TradeEntity() {
                //* Consider you are preparing Domain Trade Entity 
                //* based on various source data received from different sources
            }
        };
    }
}

On trade service, we are just fetching data from different sources and doing massaging & association to return a prepared ITrade objects.

private static ITradeDataContract[] GetTradeFromSource( IEnumerable<Guid> tradeIds ) {           
     ITradeDataContract[] response = null;            
           
     var request = new ResolverRequest() {
                        ProviderType = DataSourceProviderType.Default
                   };

     try {
          using ( var provider = ProviderFactory.GetTradeDataProvider( request ) ) {
                    response = provider.GetTradeData( tradeIds );
          }
     } catch {
          request.ProviderType = DataSourceProviderType.Persistent;
          using ( var provider = ProviderFactory.GetTradeDataProvider( request ) ) {
               response = provider.GetTradeData( tradeIds );
          }
     }

    return response;
}

Points of Interest

You will find your application enabled for horizontal scaling to be able to add more computing power, less regression prone after modifications, better testable, ready for mocking and at least a better approach with minimum work.

History

• 2^nd March, 2015: Initial version