How enterprise integration challenges can be solved
Many approaches have been tried to make the integration simple without compromising on vital aspects of enterprise, such as security, transaction, availability, reliability, and so on. A few of the prominent methodologies used over time are Java Connector Architecture (JCA), RMI, RPC, and CORBA for platform-agnostic integration, message brokers with system adapter, and many more. Under the hood, they try to solve integration issues through one of the following techniques:
- Shared File: This is the simplest approach. Systems can write data in a predefined format to a file, which can be read by other endpoints. An adapter might be needed to convert a format between two different endpoints. Let's consider an example, a daily report used to be generated in a CSV file. Over time, the organization established a web presence and reports need to be pushed online now. How can this be achieved? The simplest way is to dump it in files that will be read by an adapter and fed into the CMS system. A filesystem is simple but not the best solution; it is not transactional. What if a particular file is corrupt, or what if at poll interval files are not available due to network failure? This necessitates the introduction of a complex system that has a retry mechanism, filter capabilities, and many more nonfunctional aspects such as secure access, archival, and so on.
- Shared database: This addresses a few of the challenges that are addressed by the filesystem, such as transactional behavior, role-based access, performance tuning, distributed support, and so on. The common mechanism is a set of join tables—one application writes data in a schema that is understood by others. On the flip side, this introduces tight coupling; if there is a change in schema, both the systems need to be updated accordingly. Evolving applications will become a challenge, as they will have to factor in the external system limitations. The integration effort might itself start with lot of hiccups, for example, compatibility issues between the SQL provided by database vendors of the application, data format, and types in their table. For example, if one system stores only the date while the other stores the date with time stamp, depending on need, at least one will have to change format.
- Remote procedure calls: This mechanism introduces a paradigm where each system can offer services by publishing the contract. These paradigms can be a method, parameters, result, and error. For example, an EJB service or a SOAP service can be exposed for providing raw data for a reporting module that renders it in multiple formats. The most limiting aspect is synchronous behavior, where systems have to wait for the result. There are other challenges such as serialization of data, network latency, performance issues of a module, which can bring down the whole application, and so on. From a security aspect, exposing the method name and parameter invites hackers to exercise their creativity.
- Messaging: This introduces the asynchronous model in which two heterogeneous modules can interact through data over a predefined connection. The greatest advantage is decoupling—none of the systems are dependent on the availability of the other and they can participate or withdraw from integration without impacting other components. JMS is an example of message-based integration. Spring Integration is based on this paradigm where multiple endpoints connect on to a channel, produce or consume messages, and perform further processing based on information in a message. We will deal with channel, endpoints, message payload, and other concepts in the upcoming chapters.
Even if we use one of the preceding techniques, enterprise systems are way outward from each other and all of them might not be working all the time. This necessitated the use of middleware that can orchestrate reliable communication between these disparate endpoints, typically called an Enterprise Service Bus (ESB). In layman's terms, ESB can be defined as the middle man who enables communication to and fro between heterogeneous interfaces.