May 10, 2016
Often times, at PointSource, we need to integrate with systems of record which are controlled by someone else. These systems may be third-party cloud services, legacy back-end data providers, existing private API’s, or even newly developed services. However, we are often looking to integrate, combine and refine these data services in order to support the best application experience possible. As a result, an integration tier is needed to support the mediation of these many systems of records into a cohesive API which can support the best digital and mobile experience across a multi-channel set of applications.
We’ve reached the first decision point – do we need an integration tier? Yes! Of course. However, at this point, we can choose from one of the many frameworks available – Node.js, Java, C#, etc. and start building point to point integrations between the client, API, and systems of records.
However, this is where I believe we can make it a bit more interesting and consider another level of decomposition in the design of a component-based integration tier.
Using BlueOak Server as the foundation, we set out, for one of our clients, to establish a well-defined set of components that would describe the different aspects of their business. The overall goal was to be able to compose and reuse these functional bits of their business and connectors to their systems of records into a set of API’s which support their mobile and digital experiences as well as outside consumers.
Designing with Composability in Mind
The design is intended to benefit from aspects of a micro-services architecture and modularization while attempting to mitigate some of the operational complexities.
Clear Component Boundaries
Each API could be decomposed into a set of components based on clear separations of concern. Each component should do one thing and do it well. The components should support reusability where applicable and define clear interfaces.
Each component should be testable in isolation. A component should contain a set of unit tests which validate the component’s interface and capabilities. The test cases support the definition a clear interface and enable identification of defect and regression issues during the Continuous Integration cycle.
The delivery of a newer version of a component should be independent of the consumption of the new feature. Using npm’s packaging and semver versioning, a new component can be developed, tested and delivered without requiring it to be consumed by the aggregation tier. When ready, the aggregation tier can develop a new or updated API by consuming the new version via updates to the package.json.
As a result, the design decomposed the API into the following high-level components:
An API Specification can be developed using the Open API (fna Swagger) support of BlueOak server. We leverage the ability to bind specification methods to API handlers using the x-handler, and operationId properties supported by BlueOak.
When a cross-cutting concern is necessary, like Authentication, Authorization, and Accounting concerns, we leverage the BlueOak middleware paradigm. For example, in the case of authentication and authorization, the x-* extensions available in the Open API specification were used to adorn the authentication and authorization requirements on the API specification. Using the runtime version of the API specification the middleware — our authentication.js component can secure and validate restrictions on any routes running on the server.
The API handler component deals with serializing the internal resource onto and off of the protocol wire and is the component which surfaces a mediation to the outside world via an API endpoint. The handler is implemented as a BlueOak handler and should be a very thin layer of code on top of the mediation it is making available. The only logic that should be necessary included aspects like content negotiation, response code mapping, or NLS translation support.
The mediation component is the heart of the integration tier and it describes a specific feature of the business that can be actioned on. The mediation encapsulates the business logic that is necessary to surface that feature as a succinct micro-business service available within the integration tier. The mediation should be sheltered from the details of where the request came from or who is providing the data in support of the request. These are the concerns of the API handler and service binding. The mediation’s goal is to aggregate, transform, and extend data from one or more service bindings. It should provide a clear, concise contract based on the semantics for the call and it should be defined in a manner, in which it may be reused over more than one use case.
Mediations are implemented as externalized BlueOak services. They reside in their own npm package. They are imported into a server based on BlueOak via the package.json and BlueOak’s dependency injection.
A separate npm package for the mediation provides a few necessary advantages:
- Supports the notion of developing a small, focused and well-tested component.
- The mediation can be versioned and managed via semver rules. This allows multiple version and multiple branch support around the core code logic that wraps the business feature.
A service binding component encapsulates the communication and interaction patterns necessary to deal with one system of record. This may a binding to a SQL database, a third party REST API, or possibly a SOAP Web Service.
The bindings, like mediations, are externalized BlueOak services and benefit from the same advantages of being a separate npm package. By encapsulating a system of record within a binding service we can abstract the interactions needed to query or build a resource from a specific data provider and generalize the resource contract made between a binding and mediation. This allows the mediation to maintain its focus on business logic and not tightly couple the mediation to the communication logic for a specific binding provider.
Composable BlueOak Services
As a result of this design and decomposition of the API implementation, we’re able to compose and target an API for the use case and experience needed. By leveraging the BlueOak Services feature, we’ve been able to build a robust collection of internal micro-services that describe and encapsulate the business functions the client requires. Defining strong component boundaries in the bindings and mediations have allowed for greater confidence in interactions and integrations with external systems of record as well as faster time to value and increased reusability of tested components.