Adapters and EAI

Technology, Tools and Trends

Presenter: Atul Apte

President and CEO, iConexio Technologies Canada Inc.

Copyright iConexio Technologies Canada Inc.

 Agenda

• Session 1:
– Background on software integration
– Background on adapters
– Overview of adapter technology
• Session 2:
– Overview of trends in adapter technology
– J2CA, Web Services, Semantic Adapters
– Overview of adapter development tools
• E-Fusion product architecture

• Q&A
 Background On Software Integration

• Software integration is the #1 IT priority today

• It is also the most complex and expensive problem to solve
• About 30% - 40% of all IT costs are integration-related
• Approximately 1/2 of all integration costs are adapter-
related
• Adapters represent the highest incremental cost of
software integration
• Each software system that needs to be integrated requires
an adapter
 Background On Software Integration

• The primary focus of software integration is

information systems (or business applications)
– The two main areas of software integration are:
• Data
• Function (Web Services)
– The two main types of software integration environments are:
• Point to Point (Application Server based)
• Brokered (Integration Platform based)
– The extended focus of software integration is business processes
• Integrating information systems with business processes
• Automating the integrated business processes
 Background On Software Integration

• Components of Software Integration Platform

– Foundation components
• Host (Application Server, Integration Server, others)
• Messaging bus (JMS, others)
• Object brokers (OMG)
• Database engines (RDBMS, ODBMS, others)
• Transaction engines (XA compliant, others)
– Value add integration components
• Transformer
– Mapping, Object aggregation, Object decomposition
• Broker
– Routing, Filters
• Adapters
 Current Perspective on EAI

• EAI is still a big, complex, expensive, tedious and

repeating problem faced universally by companies
of all sizes
– Approximately 1/3 of all IT costs are EAI related
– Approximately 1/2 of all EAI costs are adapter-related
• The definition of EAI is much broader today and includes
software systems within and outside the companies
firewalls
– The term “EAI” has been changed and misunderstood with the continuing
evolution of EAI technologies
 Current Perspective on EAI

• EAI is an ongoing initiative and requires strategic

thinking, strong risk management, and realistic
goal oriented implementation plans
• Industry studies have reported that the success
rates of EAI project is very low < 5%
• Existing methodologies, project management
practices & processes are not enough to deal with
the unique and specific risk management
requirements of EAI
 Current Perspective on EAI

• iConexio has a strong commitment to solve one of

the toughest problems of EAI namely
development of custom/customized adapters and
lowering the total cost of ownership
• The first step is to define EAI in a clear and
simple form easy to understand, comprehend, and
implement
 Definition of EAI

EAI is the process of integrating applications and

database systems into a unified software system
capable of supporting:
– On-demand data exchange
– Collaboration of shared functions
– Consolidation & Transformation of data into information
– Automating data synchronization
– Automating business process
 EAI Domains

• The EAI processes are grouped into a set of logical

processes (“Domains”) which simplifies the
definition of specific EAI processes
– Application To Application Domain (A2A - Domain)
• The process of integrating two or more business applications
– Application To Information Domain (A2I - Domain)
• The process of integrating a business application with information
producers (data warehouses), information access portals, etc.
– Application To Process Domain (A2P - Domain)
• The process of integrating a business application with one or more work
flows, EDI gateways, supply chain automation networks, mobile
workforce networks
 EAI Domain Reference Models

• Every EAI Domain is supported by a context

sensitive reference model that can be used to
define, design, deploy a customized EAI solution
• Each EAI domain reference model contains:
– A logical architecture that defines the components and
relationships of the EAI process
– A set of integration patterns that define the different behavior and
interactions of the EAI process
– A technology stack containing layers of technologies and
standards that enable the EAI process
 Application to Application (A2A) EAI

Business Application Transformer

Business Application

Function Database Database Function

Data
Integration
XML Broker
Adapter Adapter Adapter Adapter

SOAP

Web
UDDI
Services
Broker
 Application to Information (A2I) EAI

Business Application ETL

Information Portal

Function Database Database Function

Adapter Data Adapter

Warehouse

EII Portal
Server Server
 Application to Process (A2P) EAI

Business Application Business Application Partner / Supplier

Systems
Function Database Function Database Database Function

Adapter Adapter Adapter

XML
XML / EDIFACT

Firewall
WFMC

Workflow EDI
Engine Gateway
 A2A Patterns

Data Data Replication Pattern

Change
Detector

Data Source Data Data Data Data Target

Extractor Filter Loader

Other Patterns
Data Synchronization (Bi-directional Data Replication)
Composite Data Extraction
Collaboration of Shared Services
 A2I Patterns

Information Creation Pattern

Data
Retriever

Data Source Meta-Data Info Info Information Sink

Creator Aggregator Presenter

Other Patterns
Extract Transform Load (ETL)
Data Staging
 A2P Patterns

Task Delegation Pattern

Task Task to
Workflow Function Business
Assignment Function
Manager Application
Agent Map
Manager

Other Patterns
Business Process Management
Generic EAI Solution Architecture

Broker Transformer

Foundation (Host, JMS, CORBA, Transaction and Database Engines)

Adapter Adapter Adapter Adapter Adapter

Information Information Information Information Information

System System System System System
 Adapter Technology Status Quo

• Adapters represent the highest incremental cost of

integration
– Each applications that needs to be integrated requires a minimum
of one to three adapters (A2A, A2I, A2P)
• The average Fortune 1000 company1 has 48
applications and 14 databases for a total of 62
potential adapters
• There are few specific tools and platforms
available for development and maintenance of
adapters
1
IDC
 Definition of an Adapter

• There is no Single Definition of an Adapter

• But there is a common understanding of the
adapters behavior and role
• Adapters, Connectors, Plug-ins, Web Services,
Integration Components and other proprietary
terms are used in different context to mean
“Adapter Type Of Functionality”
• Adapter definitions tend to be context sensitive
 Generic Definition Of Adapter

• An Adapter is a software component which

integrates two different instances of software
systems
– Typically the software systems integrated are business applications
(information systems)
• In a brokered integration environment a business application is integrated
with the broker by using an adapter
– Adapter facilitates integration between the integrated software systems
• Provide connectivity between the systems
• Manage the interactions between the systems
• Enforce transformation and other business rules
 Background On Adapters

• The concept of adapters was first introduced by the

Enterprise Application Integration (EAI) vendors in the
later half of 1990’s
• Adapters have become the most important aspect of
software integration
– Every software that needs to be integrated requires an adapter
– The average Fortune 1000 company has 48 different business applications
and 14 different databases¹
– Adapters represent the highest incremental cost of software integration
¹ IDC
 Background On Adapters

• Until recently there were no standards for

developing and supporting adapters
– J2EE:JCA is the first system level standard designed to support
adapters on the Java Application Server
• Major vendors are now supporting J2EE:JCA in their Java application
servers

• Adapters are becoming a commodity

– System standards for adapters will result in more focus on
integration functionality
– Multiple providers of the same adapter will give users opportunity
to compare adapter features
 Background On Adapters

• Generally adapters are custom developed from

scratch
– Most information systems are proprietary or heavily customized
– The average cost of developing and maintaining an adapter for
packaged applications like SAP, PeopleSoft, etc. over a five year
period is over U$ 1 Million
• The adapter market has few tools available for
developers
– The biggest challenge in creating adapter tools is the expertise and
experience required to design, develop, and maintain complex
adapters
 Adapter Interactions

• An adapter use case defines a specific interaction between

the two software systems integrated by the adapter
– An adapter use case is a sequence of one or more functional steps
• There are two types of steps in an adapter use case
– Integration function (Connection, Parsing, Transformation, etc.)
– Support function (System Logging, Error handling, Audit Trail, etc.)
– Two types of use cases
• Simple Use Case
– Involves only one adapter (two software systems)
• Complex Use Case
– Involves two or more adapters (four or more software systems)
 Adapter Scenarios

• An adapter scenario is a specific instance of an

adapter use case and is bound to a specific runtime
environment
– Atomic Scenario (Derived from simple use case)
– Collaborative Scenario (Derived from complex use case)
• It is possible for adapter scenarios to change
between atomic and collaborative scenarios
depending on the runtime environment
Simple Adapter Use Case

Simple Use Case (Atomic Scenario)

SAP

API
Client
Adapter

•Check customer
SAP •Add new order
 Complex Adapter Use Case

Complex Use Case (Collaborative Scenario)

Simple Use Case

SAP Siebel
API

API
Client
Adapter Adapter

Add new Check

SAP order Siebel customer
 Adapter Technical Architecture
Technology Stack
Responsible for providing unidirectional or
Client Interface bidirectional access to adapter services
Responsible for defining and implementing service
Service Gateway contracts and access control
Responsible for parsing, filtering, and formatting
Document Manager data and documents
Responsible for data type conversion, data mapping,
Transformation Manager object aggregation and object decomposition
Responsible for managing single/multiple adapter
Scenario Manager scenarios involving one or more information systems
Responsible for managing connectivity to the
Information System Manager information systems
Responsible for managing the lifecycle and
Host Environment Manager collaborating with the operating platform
 Adapter Technical Architecture
Technology Stack
Client Interface

Service Gateway Optional Layer

Document Manager

Transformation Manager

Scenario Manager Optional

Layer
Information System Manager

Host Environment Manager

 Adapter Technical Architecture
Technology Stack
Client Interface

Service Gateway SOAP/UDDI

Document Manager
XML / XSLT
Transformation Manager

Scenario Manager

Information System Manager

Host Environment Manager J2EE:JCA

 Simplifying Adapter Complexity

• How to manage the technical complexity of

adapters?
– Use model based software development techniques
• Pre-defined software models based on integration and design patterns
will provide a robust framework for developing adapters
– Developing multiple adapters for one application
• Encapsulate fewer interfaces into one adapter instance
– Collaborate with external transformers instead of developing
transformer functionality in each adapter
 Adapter Models
Analysis, Architecture and Design Implementation
Concept
ual
Patterns

Function Structura
Integration
al l
Patterns
Patterns Patterns

Abstract
Adapter Model

Concrete
Adapter Model
 Trends In Adapter Technologies

• Adapters are more sophisticated in functionality

– Web services enable application function level collaboration
– J2EE Connector Architecture enables plug-and-play support for
adapters inside application servers
– Adapters can function in a point-to-point or broker based
integration environment
• Standard adapter models will provide a consistent
architecture and design for adapters
• Adapter tools will enable faster, cheaper, and
easier development and maintenance
 Impact of Web Services

• The most significant impact of web services is the

shift in software application architecture
• Service Oriented Architecture “SOA” is becoming
the foundation for new internet based business
applications
– The concept of “Composing” business applications from existing
software services
– Web services enables development of “SOA” compliant
applications
– Adapter have an even more critical role in fulfilling “Services”
 Impact of Web Services

• Where is the Synergy between adapters and web

services?
– Web services provide a standardized platform for defining,
locating, and invoking business services
• WSDL, UDDI, SOAP
– Adapter provide a standardized component mechanism for
fulfilling the web services by accessing external software systems
• J2CA
– Both web services and adapters are moving towards a common
semantic environment enabling rapid “composition”
• UDEF, RDF
 What is J2EE Connector Architecture?

• J2EE Connector Architecture (J2CA) is a sub-

specification of J2EE
• J2CA is a system level interface for hosting and
accessing connectors to non-J2EE applications
– J2CA connectors extend the J2EE servers ability to interface with
non-J2EE applications
– Most J2EE servers support J2CA specification
• IBM WebSphere, BEA WebLogic, Jboss, Oracle AS
– J2CA compliant adapters for popular applications like SAP,
PeopleSoft, are now available in the market place
 What is J2EE Connector Architecture?

• J2CA Specification (Ver 1.5) defines system

contracts for managing security, transactions, and
connections between J2EE application
components and legacy systems
• Support meta-data interfaces and bi-directional
access between adapters and legacy systems
• Message driven access to J2CA connectors is now
possible
 Moving Towards Semantic Adapters

• What is a Semantic Adapter?

– A Semantic Adapter provides an interface to the semantics of the
adapter
• Adapter Behavior (Services provided by the adapter)
• Meta-data of the adapted systems data structures
• Meta-data of the adapted systems service interfaces
• Configuration properties of the adapters runtime environment
– A Semantic Adapter uses an Ontology based approach to define
the semantics
• This involves associating commonly understood meaning to the
definition of adapter properties, behavior, configuration, and
associated meta-data
 Adapter Development Tools

• Fundamental Requirements
– Enable just in time adapter development and maintenance
• Customers do not purchase adapters ahead of requirement
• End points (information systems) change and therefore pre-built
adapters need customization too
– Extend the model based design (MBD) paradigm to adapters
• Model based development significantly reduces development time
and cost, and increases adapter quality
– Differentiate and support abstract adapter models and concrete
adapter models
– Enables multiple implementations of a common abstract adapter model
 Adapter Development Tools

• Fundamental Requirements
– Enable adapter developers to focus on the business requirements
of integrating two software systems
• Develop and maintain platform controllers which encapsulates the
operating platform system interfaces from adapter developers
– Eliminates the burden of learning and programming system level
interfaces and standards
– E.g: J2EE:JCA controller which encapsulates J2EE:JCA system contracts
from the adapter developer
• Generate adapter framework and component shell from pre-defined
adapter models
– Ensures design consistency, reduces bugs and enhances the quality of
adapters
 E-Fusion Adapter Technology

• Objectives:
– Solve the root cause of high adapter development and maintenance
costs
• Provide the best adapter development tools and runtime platform
– Define the core technology architecture for adapters
– Define new adapter standards and methodologies
• Support relevant industry standards applicable to adapters
– Develop a single adapter technology framework capable of
supporting different adapter interactions, programming languages
and operating platforms
• Reduces the total cost of ownership for adapters
 Overview of E-Fusion Adapter Tool

• iConexio has developed E-Fusion, a break-

through, patent pending technology for Adapter
development and maintenance
• E-Fusion reduces the time & cost to develop and
maintain software adapters from 41% to 75%
• Rapid progression towards holistic integration of
information systems for improved ROI
– A2A
– A2I
– A2P
 Benefits of E-Fusion

• E-Fusion extends the Model Based Development

(MBD) software paradigm and applies is to
adapters
• E-Fusion provides specific platform controllers
which eliminates 90% of the server/platform
coding
• E-Fusion’s Development and Testing
methodology significantly improves adapter
quality
 Benefits of E-Fusion

• E-Fusion sets new standards for adapters

– First to provide developers with design and implementation
standards for the business functions supported by adapters
– First full life cycle adapter development methodology
• E-Fusion provides a Adapter Platform which
includes all of the common adapter infrastructure
• E-Fusion enables developers to concentrate on the
elements unique to each adapter project,
independent of platform
 Positioning E-Fusion

• E-Fusion provides a common adapter tooling

environment across application platforms
– IBM platforms
• WebSphere Application Server
• WebSphere Business Integration Server (Crossworlds)
• MQSeries (Integrator, Workflow, Messaging)
• Mainframe and AS400
– J2EE Application Servers
– CORBA ORB’s
– .NET Platform
– Legacy Operating Systems (UNIX, Linux)
 Positioning E-Fusion

• E-Fusion provides a universal adapter hub across

enterprise integration environments
– Business Integration
• Process Integration
• Application Connectivity
– Enterprise Information Portals
• Information Integration
– EAI
• Data Integration
– Web Services
• Application Collaboration
 E-Fusion Product Architecture
Integrated Collaboration Environment (ICE)
Adapter Development Environment (ADE)
Extends
Adapter
Models
Adapter Generate Adapter
Development Tool Component Shell Java
From Models IDE
Assemble & Test
Adapter Adapter
Repository Register Components
Define Scenarios To E-Fusion
Code
Adapter Runtime Environment (ARE)

Configure Adapters Monitor Adapter

Adapter Adapter Extends

Deployment Tool Runtime Platform
Run Adapter Scenarios Application
Platform
 E-Fusion Adapter Technology

• Architecture:
– Adapter architecture based on the concept of functional layers
• Each layer encapsulates specific adapter functionality accessible as well
defined services
• Each layer can be implemented as an isolated/distributed component of the
adapter
– Architecture supports three types of interactions for an adapter
• Synchronization
– Keeping distributed data models in sync with each other
• Collaboration
– Enabling software systems to collaborate functions
• Delegation
– Enabling software systems to execute tasks on behalf of other systems
 E-Fusion Adapter Technology

• Adapter Model
– Is a collection of one or more adapter patterns managed by the E-
Fusion framework Integrated Collaboration Environment (ICE)
– E-Fusion supports two types of Adapter Models
• Abstract adapter model
– Represents the architecture and design of an adapter
– Adapters cannot be created directly from abstract adapter models as there
are no implementation details defined
• Concrete adapter model
– Represents the implementation reference model of an adapter
– Includes one or more run-time scenarios
– Adapters can be created from concrete adapter models
 Adapter Development Environment

• Adapter Development Environment (ADE) supports

developers throughout the full lifecycle of adapter
development and maintenance
– From design to maintenance
• Design, Develop, Assemble, Configure, Deploy, Maintain
• Develop pre-built adapter models and support custom adapter models
– Enable higher efficiencies in adapter development without changing
existing development workflow and processes
• Integrate E-Fusion with leading IDE tools and extend the tools capabilities
to support adapter development
• Define new adapter engineering and development methodologies which
can be applied with generic software development tools
 Adapter Runtime Environment

• Adapter Runtime Environment (ARE) supports

distributed and centralized adapter deployments
across multiple platforms
– Support dynamic distribution of adapter components across the
platforms in the adapter runtime environment
– Support centralized administration, monitoring, and debugging of
adapters at runtime
– Support dynamic language (locale) translation for multi-lingual
runtime environments
– Support dynamic component code download and side-by-side
runtime environments (production and QA)
 E-Fusion Adapter Runtime Platform

Control
Executio Executio
ler
n n
Manager Manager
Control
ler

Repository E-Fusion Adapter

Runtime Platform
E-Fusion Adapter
Runtime Platform Application Server
(J2EE / CORBA / Other)
CPU/OS
CPU/OS
 Summary

• Adapters are complex pieces of software

performing critical integration tasks
• The definition of adapters continues to expand and
support new application architectures (“SOA”)
• Adapter tools are now available for enabling rapid
adapter development and deployment
• The trend is towards developing semantically and
functionally rich adapters will continue to
strengthen the role of adapters
Q&A

Thank You