Application Integration through Microsoft Technologies

Introduction

Organizations have been using numerous applications that were built over many years. These applications probably are of different era, were written by using different languages and technologies, reside on different hardware platforms, use different operating systems, and provide very different functionality. In fact, many of the applications often have very little in common at all, resulting in isolated functionality and multiple instances of the same data. For your organization, these conditions can result in redundant activities, higher costs, and inefficient response to your customers.

Types of Application Integration

Application integration can be broadly categorized into three types:

· Manual application integration

· Semi-automated application integration

· Fully automated application integration

Application Integration type	Cost	Benefits
Manual	Higher labor costs that scale badly. Subject to human error.	Little change required from existing low-technology environment.
Semi-automated	Higher technology costs to implement. Subject to design-time and run-time errors.	Lower labor costs. Scales better. Less subject to human error at run time. Faster processing.
Fully automated	Highest technology costs to implement. Subject to design errors.	Lowest labor costs. Not subject to human error at run time. Lose human decision making on business processes, but faster processing.

Making Application Integration Scalable

An important part of making application integration scalable is to increase the number of automated steps and reduce the number of human steps. This generally involves creating interfaces between applications along with predefined logic that replaces human involvement. For automated application integration, you have two choices:

● Point-to-point model

● Integration hub model

Requirements for Application Integration

Typically, an environment that supports application integration meets at least the following requirements:

· Connectivity between different platforms

· Processing of complex business rules, including complex data transformation logic

· Support for business processes, from the very short to the very long, including processes that last weeks or months as data is passed and processed through different parts of the organization

· The ability to modify existing business processes or create new ones as business goals change

· The ability to adapt to changes in hardware, software, and business goals

To help meet these requirements, your application integration environment should:

· Expose a common interface through which applications can communicate, by using business semantics to request Web services.

· Allow service requests at the functional or data level for applications that do not support using business semantics.

· Use a common set of process and service rules to ensure consistency and reuse of integration services.

· Be capable of reusing the existing transport protocols that already exist in the enterprise.

· Insulate it from existing technologies by using interfaces.

Challenges of Application Integration

Technical Issues

When building an application integration environment, you are creating an environment that allows components to communicate with each other even though they were never designed to do so. In many cases, these components are not even aware of each other. They may include a variety of preexisting systems (some of which are essential), yet your environment needs to be noninvasive to existing applications.

In these conditions, complex architectural issues such as the following can arise:

· Control and coupling

· Data exchange and data formats

· Distribution and concurrency

· Scalability, reliability, and availability

You do not want to completely rebuild the application integration environment itself every few months. The aim, instead, is to have a relatively stable application integration environment, which is flexible enough to allow the addition of new applications or the alteration of relationships between applications.

Organizational Issues

Organizational issues in application integration arise mainly from the fact that the application integration environment will span multiple departments in the organization. Staff in different departments may choose to deploy applications that will then need to integrate with the rest of organization environment. To design a truly effective application integration environment, you need to isolate application integration as a separate management function, and determine who is responsible for making it happen. In larger organizations, you may have a group of people specifically responsible for this function, including application integration architects, application architects, and application owners, whereas in other cases, application integration may be one of many management functions that you are responsible for. Either way, by defining the application integration management function in this way, you help to ensure that it is properly considered whenever changes are made to the IT infrastructure.

Levels of Application Integration

Application integration can occur at many different levels, including:

· Presentation level

· Business process level

· Data level

· Communications level

Business Process – Level Integration

Business process – level integration often forms the starting point for defining application integration environment, because it is most closely related to the business requirements of the organization. Business process – level integration starts with defining your business processes and then specifying the logical integration that corresponds to those processes.

Defining Your Business Processes

Before you can design your application integration environment, it is important to understand the organization in logical terms, removed from the technology that underlies it. Business processes consist of a group of logical activities that model the activities in your organization. These activities typically represent the smallest units of logical work that can be done in the organization.

Mapping Business Requirements to Application Integration Requirements

After you have determined the business processes that the organization uses, you can begin to examine how they affect your application integration requirements. It is important to realize that a simple business process does not always correspond to simple requirements for application integration.

By modeling each of your business processes, you can define which applications must integrate with one another, and in what way. This enables you to make intelligent decisions about the specifics of application integration environment.

Data-Level Integration

You can define how applications communicate with each other at a business process level, but if they cannot understand the data they exchange, they cannot integrate successfully

There are two general ways to enable data-level integration:

· Add logic to enable each application to understand incoming data from other applications.

· Add logic to enable each application to interpret outgoing data to an intermediate data format and interpret incoming data from that format into a form the application understands.

To support the various complex data integration requirements, your application integration solution must often contain a considerable amount of logic that supports the access, interpretation, and transformation of data. You also need schematic models of enterprise data to describe data attributes. The definition and recognition of schemas enables the validation of data. Descriptions of how data elements may be related or mapped to each other allow for the transformation of data.

Communications-Level Integration

Enabling Communication between Applications

Because many applications are not designed to communicate directly with each other, it is likely that you will need to do some development work to ensure that your applications can be called by other applications.

There are two possible approaches to this problem:

· Rewrite your application to provide it with an API that other applications can call.

· Create a communications adaptor that acts as an intermediary between the application and other applications.

Important Considerations for Application Integration

There are few important considerations regarding Application integration and they are as follows

· Using Synchronous and Asynchronous Communication

· Increasing Automation

· Straight-Through Processing (STP)

· Ensuring Data Integrity

· Managing Latency

· Data Aggregation

· Combining Application Functionality

· Application Composition

· Managing Transactions

o Providing ACID Transactions

o Transactional Messaging

o Two-Phase Commit

o Long-Running Transactions

o Timed Transactions

Security and Operational Issue

Security Issue

Because different applications have different security requirements and features, it can be quite a challenge to ensure that your application integration environment functions properly without compromising your security requirements. Security is particularly important in application integration because a breach in an integration service may result in security breaches in other integrated systems.

Security Policy

The first step toward effective security in any environment is creating a written security policy. Many factors can affect the security policy, including the value of the assets you are protecting, the threats that your environment faces, and the vulnerabilities that are currently present

From an integration standpoint, your security policy should define:

• A mechanism for evaluating and classifying threats

• A mechanism for acting on threats

• A boundary for information security

• A plan for communication and enforcement.

• General security guidelines

• Reference to other documents

• A mechanism for modifying security policies

Defining Your Security Requirements

The precise security requirements of application integration environment depend on a number of factors, including the following:

· Security requirements of the organization

· Business requirements for application integration

· Technical requirements for application integration

· Capabilities of the applications which will be integrated

· Platforms of the applications

· Budgetary constraints

As a starting point for defining your security requirements, you should perform a risk analysis. Doing so allows you to identify the threats and vulnerabilities that you face and identify the countermeasures you can deploy to keep risk at an appropriate level.

Operational Management Considerations

Even after you build an application integration environment and it is running successfully, your work does not stop there. The environment will need to be monitored and maintained over time. This section discusses the various operational management considerations involved. Implementing the capabilities listed in this section should help to ensure that your architecture continues to function as smoothly as possible.

Defining an Operational Management Policy

Your operational management policy, at the minimum, should provide the following information:

· Clearly defined terminologies and target metrics

· Methodology and/or formulas for measuring the metrics to ensure that results are consistent

· Service-level prioritization to ensure that the most important policies are followed first

Defining Operational Requirements

The operational requirements of application integration environment depend on a number of factors, including the following:

· System Monitoring

· System and application health

· System and Application Performance Monitoring

· Security monitoring

· Service-level monitoring

· Business Activity Management

· Business transaction exception handling

· Contextual monitoring

· Rules-based alerting

· Historical data mining

· Change and Configuration Management

· Directory

Microsoft Technology/Tools Mapping on EAI Capabilities

Capabilities Required for EAI Environment

The capabilities are defined as follows:

· Business process integration capabilities

· Data integration capabilities

· Communications capabilities

· Security capabilities

· Operational management capabilities

Business Process Integration Capabilities

Capability	Provided By
Rules Processing	Custom .NET Framework components, SQL Server stored procedures, and BizTalk Server Business Rules Engine
Transaction Management and Compensation	Microsoft Distributed Transaction Coordinator (DTC) and BizTalk Server long-running transactions
Workflow	Windows Workflow Foundation and BizTalk Server Human Workflow Services
Orchestration	BizTalk Orchestration service
State Management	Common language runtime, IIS session management, and BizTalk Server orchestration state management
Event Processing	SQL Server triggers, MSMQ Events, and BizTalk Server messaging subscriptions
Schedule	Windows Server Schedule service and SQL Server Job Scheduler
Contract Management	Windows Communication Foundation and BizTalk Orchestration service

Data Integration Capabilities

Capability	Provided By
Data Validation	XML schemas
Data Access	ADO.NET and SQLXML
Schema Definition	Visual studio 2003/2005
Data Transformation	SQL Server DTS, XSLT and BizTalk Mapped
Schema Recognition	XML schemas and BizTalk Server parsers

Communication Capabilities

Capability	Provided By
Message Routing	Windows Communication Foundation and BizTalk Server Message Box and subscriptions
Message Delivery	BizTalk Server message ports
Message Queuing	Microsoft Message Queuing and BizTalk Message Queuing
Message Forwarding	Windows Communication Foundation and BizTalk Server Message Box and subscriptions
Message Correlation	BizTalk Orchestration correlation services
Message Instrumentation	BizTalk Orchestration
Addressing	BizTalk Server message ports
Transactional Delivery	Microsoft Message Queuing and BizTalk Server reliable messaging

Security Capabilities

Capability	Provided By
Authorization	Windows Card Space and Active Directory
Authentication	Windows Card Space and Active Directory and Share Point Portal Server Enterprise single sign-on
Information Protection	Windows Server
Identity Management	Windows Card Space and Enterprise single sign-on
No repudiation	Digital signatures
Profile Management	Enterprise single sign-on
Security Context Management	Active Directory and Enterprise single sign-on

Operational Management Capabilities

Capability	Provided By
Business Activity Management	BizTalk Server Business Activity Monitoring (BAM)
Event Handling	Windows Management Instrumentation (WMI) events, Event Viewer, Performance Monitor, and MOM
Configuration Management	System Management Server
Directory	Active Directory
Change Management	BizTalk Server Business Rules Engine and schema versioning
System Monitoring	Windows Server and MOM

References

http://msdn.microsoft.com/practices/Topics/appint/default.aspx?pull=/library/en-us/dnpag/html/eappint.asp

http://msdn.microsoft.com/practices/Topics/appint/default.aspx?pull=/library/en-us/dnpag/html/intpatt.asp

http://www.developers.net/intelisdshowcase/view/840

http://www.infoq.com/j+n/