Enhancing Applications with Application Services

Architecture

The architecture will need to support these paradigms

  • API First Approach resulting in OpenAPI specifications

  • Enable Parallel development streams

  • Managed API Management platform for securing and managing the backend services

  • AMQ Streams (Apache Kafka) to process user activity event streams

  • Preference for a managed cloud services for easy and rapid ramp up

1. Common challenges

There are however a number of challenges with the new requirements:

  • Adding new channels remains difficult, with a high risk of tight coupling to the existing services, which would slow down development productivity and time to market.

  • The existing services need to be managed and secured to allow access for external partners and development teams. Governance remains a challenge.

  • Adoption of new technologies such as event streaming requires time and new skills, which are not readily available inside the company.

In order to cope with these challenges, the development team decides for a new approach.

API First approach: before tackling the development of new services, the API contract is formalized in a OpenAPI spec document. This API design phase is done collaboratively with all stakeholders. Once a first version of the OpenAPI spec document is ready, it is pushed and managed in a service registry, which acts a the system of truth. Mocks are created for the API.

Parallel Development streams. The API first approach enables parallel development streams. UI development teams and other API consumers can start their development against the mocked APIs, without having to wait for an actual implementation. In parallel, backend development teams can implement the APIs using modern cloud-native frameworks. They continuously test the implementation against the OpenAPI spec to ensure that the implementation does not break the contract.

Manage and Secure the APIs. An API management platform allows to expose the APIs in a secure and managed manner for access by the mobile app and other third party applications.

Adoption of Apache Kafka as a streaming platform to ingest and process user activity event streams.

3. An in-depth look at the solution’s architecture

img-runtimearchitecture
Figure 1. Globex Runtime Architecture

Visitors engage with the Globex retail website in a number of ways

  • view list of products

  • search for products

  • like a product / add a product to favourites

  • view product details

  • add products to the shopping cart

Each of these activities generate a stream of events which are captured by the User Activity Tracking service. This service then pushes the events into the Kafka streaming platform. The events can then be consumed by other services such as the Product Recommendation service which powers the list of featured products. A new Featured Products section is created in the front-end web application to showcase the top featured products.

Since Globex does not want to allow direct access to these backend API services to other channels such as the mobile app, an API Management platform is introduced. The mobile app will access the services through the API Management platform.

4. More about the technology stack

4.1. Apicurio

As part of the API first approach, the first step is to design these APIs before actually implementing them. Apicurio Studio is an environment which allows one to collaboratively work on API specifications - today this is not a managed service - but will be offered as a managed service in the near future.

Even before the implementation starts, the various stakeholders come together to define the API specs. The API are defined for the existing catalogue service as well.

img-apicurio-landing
Figure 2. Apicurio studio landing page

This designer provides a graphical way of designing all the aspects of an OpenAPI - had different paths, datatypes and canned responses. For instance - <show example here> Which can then be later used to mock out those services.

img-apicurio
Figure 3. Apicurio studio API designer

It also allows you to work in both a graphical way of doing things and also with the source. The OpenAPI can be viewed as both a YAML and a JSON document.

4.2. Service Registry

Once the API design is complete, and we have the first version of the API, this can now be published in a schema registry. Red Hat OpenShift Service Registry allows us to publish the OpenAPIs, makes it accessible, and manages the OpenAPI from within the Red Hat’s cloud console

img-runtimearchitecture
Figure 4. Red Hat OpenShift Service Registry

You can upload new artifacts, new versions, view the metadata, download the specs, view documentation and view the content as well. Through Content rules one can validate new versions of the APIs against the existing specs to ensure validity and backward compatibility.

4.3. Microcks

We can also build mocks for the APIs using the same OpenAPIs.

Microcks is a tool which allows one to upload the same OpenAPI spec, and to build mocks for the APIs. This is for use of the teams who will actually consume the APIs and they can use these mocks to develop their pieces of code even before the APIs are completely implemented

img-microcks
Figure 5. Microcks

Each of the mocks that are created out of the OpenAPI’s examples has its own URL which can be invoked to provide a realistic response. This allows front end and mobile app developers to develop against the OpenAPIs specs without waiting for the final implementation of the backend services.

4.4. Red Hat OpenShift API Management

We use the managed Red Hat OpenShift API Management platform here to publish, manage and secure the backend APIs.

Each API can be configured to be secured using a number of ways. In this case, the APIs are secured with an API key which should be passed through http request header. The OpenShift API Management platform allows you to have various application plans. Developers can subscribe to those APIs and can access APIs through an assigned API key securely. You can monitor the APIs and also track usage

img-rhoam
Figure 6. Red Hat OpenShift API Management

As a developer, you would like to build functionality around the APIs. There is also a Development Portal which is currently under, well, development. You can sign in as a developer here. This developer has already subscribed to the API and is given an API key which should be used in all API calls to ensure the calls are authenticated by the API management platform.

img-rhoam-dev-portal
Figure 7. Red Hat OpenShift API Management Dev Portal

The devportal allows viewing Live documentation as well, which is another view of the OpenAPI specs. Developers can try it out to see what kind of responses they can get back. The developers can also view statistics for their account in a graph format

img-rhoam-dev-portal-statistics
Figure 8. Red Hat OpenShift API Management Dev Portal statistics

4.5. Kafka

Red Hat OpenShift Streams for Apache Kafka is a fully hosted and managed Kafka cloud service. This serves the requirement of Globex who did not want to install and maintain a streaming platform. This service can be very easily consumed and used by developers.

There are a number of topics created for Globex. The globex.tracking topic is one which ingests the activity events. This is then consumed by the Recommendation Engine which is a Kafka Streams application which then builds a list of Top 10 most popular products with most user interest.

img-kafka-topics
Figure 9. Red Hat OpenShift Streams for Apache Kafka

This is a managed service and security is very important. The service allows fine grained access controls based on user accounts and service accounts. You can define what each account is allowed to do.

img-kafka-topics
Figure 10. Red Hat OpenShift Streams for Apache Kafka - Access Control

With all this tooling in place as part of App modernization, GitOps, Continuous integration, deployment in staging. This means that Business can easily follow progress at different points of building the functionality which you can see here

4.6. Globex retail application

The new homepage of the Coolstuff store displays the top currently featured products.

img-globex-02-ui-landing
Figure 11. Globex App Home Page

The featured projects are displayed as a carousel in the page displaying the paginated list of products. Visitors can drill down to view product details, add products to the cart, view and edit catt, and also like their favourite products

img-globex-02-ui-pagination
Figure 12. Globex - Paginated products list with Featured products carousel

Each of the activities generates a continuous stream of events which is then used to recommend the top 10 products to be featured.

4.7. Kafdrop

<generate more records now> Kafdrop is a tool through which you can view this continuous stream of events. Kafdrop will point to the managed instance. We can view the topics and messaged easily as part of the development process

img-globex-02-kafdrop
Figure 13. Kafdrop

The Kafka streams application creates a number of other topics for data aggregation, and pushes the aggregated products list as events into the product-score-aggreated topic

img-globex-02-kafdrop-aggreated
Figure 14. Kafdrop - Aggregated topic with featured products

5. Deployment Architecture

The various components of the application run on various footprints.

  • Apicurio is a free hosted developer tool accessed through https://studio.apicur.io/

  • The various backend services are deployed as containers on Red Hat OpenShift Container Platform running on a public cloud. Microcks and Kafdrop tools are also deployed on OpenShift Container Platform. The UI itself is an Angular app which runs on NodeJS and deployed on the same OpenShift platform.

  • Managed Red Hat OpenShift Streams for Apache Kafka and Red Hat OpenShift Service Registry are deployed and accessed as fully managed services somewhere in the cloud and accessed through Red Hat Console

  • Red Hat OpenShift API Management is running on OpenShift Dedicated deployed on allows

  • The https://quay.io/ is a hosted containers repository which stores the container images for the backend and UI application.