Microservices and Standards; Request / Response

Standards are essential for implementing a Microservice Architecture. In this article, I will focus on the structure of requests and responses involved in such an architecture.

It's important to remember that a microservice does not expose its API to the outside world. These microservice APIs are intended to be consumed within the context of the overall application. In fact, a microservice architecture is a design pattern where the implementation of an application's sub-domains is distributed across isolated, independent services. Since we are still dealing with a single, overarching domain, it’s important to ensure that certain data attributes in both request and response types are used consistently across the system.

Typically, a request enters the application’s domain through an API gateway. This entry point functions similarly to the API interface in a monolithic system. Ideally, we aim to define a single API that, through the use of attributes, can handle various functionalities within the system.

The API request is then passed to an application orchestration layer, which is responsible for applying rules, performing validations, and determining the best route (i.e., the appropriate microservice) before forwarding the request. In some cases, the orchestration layer may enrich the request with additional attributes of its own.

Once the request reaches a given microservice, it is processed. This may result in another request being generated—either to another microservice or to an external entity or system. Responses are then generated by the called services and eventually returned to the API gateway, which sends a final response back to the original client.

In many cases, the initial microservice may forward the request to another microservice, forming a chain of services involved in the processing pipeline—second, third, fourth, and so on. This process continues until a final response is produced, traveling back through the same route—first to the orchestration layer and finally to the API gateway. In complex applications, this can involve several, even dozens, of microservices before the final response exits the system.

To accurately correlate the various requests and responses, it is crucial that they all share a unique transaction ID. Additionally, it’s important to identify the originating client that initiated the request. Therefore, having a standardized base structure for both requests and responses across all microservices is essential for maintaining consistent and reliable processing throughout the system.

Transitioning to a Microservices Architecture - Part 2

Microservices and the Development Organization

by Rubens Gomes

Microservice architecture is based on the principles of modular systems that align with the domain-driven design (DDD) paradigm. That is, the architecture is divided into sub-domains, each with specialized responsibilities, and delineated from other modules by what is known as a bounded context. To develop deep expertise and effectively address the concerns of these sub-domains, it is ideal to have development teams composed of specialists in each particular domain. These teams become the owners of the microservices within their respective sub-domains. This type of organizational structure is key to successfully implementing a microservices architecture.

As stated in Conway's Law:

“Organizations which design systems (in the broad sense used here) are constrained to produce designs which are copies of the communication structures of these organizations.”

The implications of Conway's Law are fundamental to the successful implementation of a microservices architecture. Because a microservice is a module with responsibility within a specific domain—and thus has a clear bounded context—assigning dedicated teams to work on specific sub-domains tends to naturally facilitate the development and maintenance of microservices.

In essence, what I’m emphasizing is that to achieve the best results from a Microservice Architecture, an organization must be structured around sub-domain expertise. Microservice architecture goes hand in hand with Conway’s Law. By structuring organizations around sub-domains, you naturally encourage the creation of applications that reflect the structure and purpose of the corresponding microservices. Communication within these expert teams becomes highly cohesive, focused, and aligned with the specific requirements of their sub-domain. This, in turn, supports the development of a clean and efficient Microservice Architecture.

In fact, one way to guide an organization toward a Microservice Architecture is to apply the Reverse Conway Maneuver (also known as Inverse Conway’s Law). This approach suggests that by organizing teams around sub-domain expertise, the system architecture will begin to reflect that structure—resulting in the natural emergence of a Microservice Architecture.

Why I Like Microservices

Why I Like the Microservices Architecture Style

By Rubens Gomes

In order to explain some of the reasons why I prefer the microservices architecture style, I’ve written this comparison between monoliths and microservices, based on over 30 years of practical, real-life experience in software development across both small and large companies.

---

Monolith Development Environment Setup

Setting up a development environment is one of the most important steps a developer must take when starting a new job. It involves not only configuring tools (e.g., IDEs, text editors), network access, databases, and source control, but—most importantly—creating an environment that facilitates the development and maintenance of the application.

This is, right off the bat, one of the issues I have with monolithic systems: monoliths are usually very complex to set up. I’ve had real-life experiences where it took over a week just to get a development environment ready for a single monolith application.

Monoliths are often tied to licensed platforms such as J2EE, include numerous libraries, have complicated builds involving multiple components, and use large, complex databases. Getting tests running and learning the entire business domain adds further complexity. Everything in a monolith is orders of magnitude more complicated than in a microservices setup.

In one job, it took me over two weeks to get my environment working for one of the department’s main monolith applications. The system used a J2EE backend server implemented on IBM WebSphere, which required a local installation and extensive configuration across multiple components: databases, system interfaces, and shared libraries. The build process itself was very complex, with different modules relying on various property configuration files.

---

Monoliths: Long Meetings and Long Releases

I remember, as a senior architect at a large enterprise, attending weekly meetings with project managers, QA leads, and department managers to review features being implemented for upcoming releases. The meeting room wall was covered with lists showing the lifecycle status of each feature.

As release dates approached, we had to coordinate what was ready, what was still in QA, and when we could potentially deploy. Everything took longer—development, testing, cross-team communication, orchestrating different projects, and aligning multiple parallel feature tracks. We also had to coordinate with the Operations team to schedule deployments.

This is, in my opinion, one of the greatest drawbacks of monoliths: how long it takes to get something into production. A single release could take months due to the number of steps and teams involved.

Often, we had to split features into different branches, each moving in parallel. While development was ongoing, production issues would arise, triggering the need for hotfix branches. To make things even more complicated, we had to manage operational logistics: when to deploy, who would deploy, at what time, and how to handle rollbacks if needed.

---

Microservice Development Environment Setup

Microservices, on the other hand, typically have smaller databases, fewer libraries, simpler business sub-domains, fewer tests, and significantly fewer lines of code. I’ve seen real-life cases where a contractor joined a team in the morning and by the afternoon had their development environment set up and was already coding on one or more microservices.

In my own experience, once we migrated from a monolith to microservices, everything became so much easier. Setting up a development environment in Eclipse or IntelliJ IDEA took only minutes. All I had to do was clone the microservice project, import it into Eclipse as a Maven project, and run a build via CLI or DevOps tools like Microsoft Azure. From the IDE, I could begin coding in no time.

---

Why Is It Easier to Work with Microservices?

Microservices are small, focused applications that are designed to do one thing—and do it well. You no longer have to juggle all the technologies, components, databases, and libraries that a monolith typically involves.

The build process becomes much simpler and faster since it doesn’t depend on various cross-cutting component libraries. Because microservices are focused on a specific sub-domain, the learning curve is much shorter. Our brains are better equipped to focus on one smaller, well-defined problem at a time—as opposed to having to understand a large, all-encompassing monolith.

Getting things up and running, writing tests, implementing features, delivering updates, fixing bugs—everything is so much faster and easier with microservices. You can stay focused on a specific business area and ensure features are delivered to production quickly. Testing is faster, and builds and deployments are simpler.

I’ve seen many cases where a pull request was approved in the morning and the code was in production the same day. Troubleshooting is also easier with microservices since logs can be scoped to individual applications, making it much simpler to trace issues.

Transitioning to a Microservices Architecture - Part 1

Transitioning to a Microservice Architecture – Part 1

By Rubens Gomes

I had the opportunity to serve as a technical lead during the implementation of the microservices architecture for the American Airlines Ticketing department from 2016 to 2023. The Ticketing department is responsible for the booking and payment processing of over 700,000 airline tickets daily. The company’s IT transformation began around 2017, and the ticketing team was among the first to have microservices running in production.

Transitioning to a microservice architecture involves significant changes—not only to the technical architecture and continuous integration/delivery pipelines, but also to the organizational culture and team dynamics. In a microservice architecture, teams become more independent and self-organized, each responsible for a specific part of the business domain. These teams develop deep expertise in their respective sub-domains and take full ownership of development, deployment, and production support.

In addition to these structural and organizational changes, a successful transition to microservices requires several critical foundational components. In future parts, I’ll elaborate on these elements and explain why they are essential for effectively implementing a microservices architecture.

Stay tuned for the next edition of "Transitioning to a Microservices Architecture – Part 2". I have much more to share on this topic, as I had the unique opportunity to witness—and help shape—a real-world microservices implementation from the ground up in a very large enterprise.

Installing LENS // The Kubernetes Platform IDE

Installing LENS // The Kubernetes IDE

• Download and install the latest version of LENS // The Kubernetes IDE . 
• Ensure the cluster "kubeconfig" configuration settings have been previously downloaded following the steps from Installing IBM Cloud CLI and Kubectl.
• All the cluster configurations should be stored in the KUBECONFIG file (e.g., C:\Users\rubens\.kube\config).

Running LENS

• Prior to running, LENS ensures you have previously configured the "Kube" cluster configurations as LENS will use those configs and the embedded certificates to authenticate itself to the different clusters.
• If issues with Authorization failed, you may have to update the IBM cloud bluemix certificate files (e.g.,  C:/Users/rubens/.bluemix/plugins/container-service/clusters/...) with the latest tokens.  In order to do that you need to download all the previous cluster configurations again.

• LENS WILL USE THE KUBECONFIG FILE LOCATED AT THE "KUBECONFIG" ENVIRONMENT VARIABLE FOLDER.  MAKE SURE YOU HAVE THE LATEST/UPDATED KUBECONFIG FILES/TOKENS IN THE KUBECONFIG FOLDER (E.G. C:\Users\rubens\.kube\config).  IF NOT YOU MAY RUN INTO AUTHORIZATION ERRORS.

Lens Authorization Errors

If you run into Authorization Errors when attempting to connect to a cluster within LENS follow these steps:

1. Ensure all previous steps have been followed and that you are running the latest version of IBM cloud, Kubernetes and IBM cloud Kubernetes service plugins
2. Ensure you have the KUBECONFIG environment variable correctly configured in your environment.  See previous notes.
3. Ensure that prior to running LENS you are currently logged in to IBM Cloud.
4. Ensure you have the latest updated configuration/tokens in your KUBECONFIG file.  You can run the previous steps to download the Cluster Configuration.  If you are on Linux you may consider using a script similar to the one below:

$ cat $HOME/bin/kubetoken.sh
#!/bin/sh -ahu
##
##       author : Rubens Gomes <Rubens.S.Gomes@gmail.com>
##
## written date : October 26, 2022
##
##      purpose : This script is used to update the IBM Cloud Cluster Configuration
##                tokens and settings in the KUBECONFIG ($HOME/.kube/config) file.
##

# local path to IBM Cloud tools
IBMCLOUD="ADD PATH TO IBM CLOUD CLI TOOLS"

# define a clean UNIX binary PATH
PATH=
PATH=${PATH}:/bin
PATH=${PATH}:/sbin
PATH=${PATH}:/usr/bin
PATH=${PATH}:/usr/sbin
PATH=${PATH}:/usr/local/bin
PATH=${PATH}:${IBMCLOUD}/bin
export PATH

# define a clean UNIX LD_LIBRARY_PATH
LD_LIBRARY_PATH=
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/lib
export LD_LIBRARY_PATH

# logging into IBMCloud using Single Sign On
\ibmcloud login --sso

if [ ${?} -ne 0 ]
then
 echo "Failed to login to IBM Cloud" 1>&2
 exit 1
fi

echo "-------------------------------------------------------------------"
echo "Downloading Cluster Configuration..."
\ibmcloud ks cluster config --cluster "<cluster ID>"

if [ ${?} -ne 0 ]
then
 echo "Failed to download IBM Cloud CLUSTER Configuration" 1>&2
 exit 2
fi

echo "Done"
exit 0

What Is a Service

In the context of software, service is a computer message invocation that is sent over a telecommunications networking protocol connection (e.g., TCP/IP or UDP/IP socket) to a remote computer. This message invocation has the intent of stimulating an application or multiple applications that are sitting on remote machines to perform some operation. That is, the remote application, upon receipt of the requested message, is expected to parse and process the message accordingly. And then it may either simply process the message and return no response, it could return a response message to the invoker.  Sometimes, the application may even forward that message to another systems for processing.

Based on the above definition of a service, we may have several different communication protocols that could used for the transfer of the request and response service messages between the remote computing ends.  Below are some common communication protocols that have been used in more recent computing:

• REST over HTTP
• SOAP over HTTP
• CORBA
• Some private messaging protocol over HTTP
• Some private messaging protocol over TCP/IP socket
• HL7 used in the healthcare arena
• SNMP used in the network management arena

CRUD

The essence of any service is based on the CRUD terminology. CRUD stands for Create, Read, Update and Delete. Most of the service APIs fall into one of the CRUD categories. For example:

• Create: AddAccount, CreateAccount
• Read: GetAccount, ReadAccount, RetrieveAccount
• Update: SaveAccount, UpdateAccount, ModifyAccount
• Delete: DeleteAccount, RemoveAccount

As we can see from the CRUD message APIs above, there are many different ways that the same service message can be named. The service API designer should therefore be careful to follow a consistent approach when naming the APIs. Otherwise, we may end up with several different names that mean the same thing (e.g. Create/Add or Get/Read or Delete/Remove).

Service Semantics and Behavior

A service should be well defined and understood, which means that its semantics and behavior should be clearly stated.

• Semantics: It consists of the all the wording that is used to define a service. For example, the service message name, message attributes names and types, and any type of errors (or exceptions) that might be incurred as a result of invoking that service. For example, when creating a customer login, the semantics might be defined as follows "createCustomerLogin(username, password, email), where the username, password and email are all text data defined by certain constraints; e.g., number of characters and character encoding scheme.

Notice, that in the above semantics we are not concerned about the underlying technical implementation of that service. In the "service contract" document, we would then add a section that provides specifics of the semantics implementation. In that case, we would define request/response messages using a WSDL language (SOAP), or use the IDL language for a CORBA service. For RESTFul, we would implement the service semantics in terms of its URI with HTTP header/POST body (request) and corresponding HTTP headers/body (response) along with the media type (e.g., JSON or XML).

• Behavior: It consists of the expected actions that might occur as a result of invoking a given service message. In this case, we want to elaborate on what activities might occur as the outcome of invoking the service. For example, in the case of creating a customer login, we would elaborate that an account with a unique user name is created on a given computer system. Or in multiple computer systems, depending on the business case being solved.

Characteristics of Good Services

A service should, foremost, address a specific business need. A service is therefore a means to an end; that is, it should address and solve a business requirement. Throughout the design and implementation of a service, the implementer should keep in mind the business audience who will consume that service, and aim the naming of the service API to that business audience.  That is, service request/response messages, attributes, and resources should have names that are meaningful and aimed at the business audience who will be consuming that service API.  Furthermore, the service API should be, whenever possible, validated with the business parties; that is, the programmers or business analysts  who will be ultimately using that service API.  The goal here is to ensure that the service API is not only well aligned with the business needs, but that it is easy to use and be programmed with.    The service API designer or implementer should refrain from trying to map a backend application API to a service API.

A side effect of using service enabler infrastructure computing (e.g., ESB, enterprise service buses) is  the use of what is called "technical services".  These type of services are used to collect technical information, such as performance analytic, logs, errors and other troubleshooting information from the underlying service enabler infrastructure.

Service Patterns

By service patterns, it is meant good practices and tips that are followed in the market to name a few of the service messages APIs and protocols.

Resource API Pattern

The Resource API Pattern is implemented by RESTFul web services. That is, the API is operating on a given resource (e.g. Account, User, Report, and so on). And the CRUD behaviour is leveraged by the underlying HTTP Protocol. That is, for HTTP the CRUD is as follows:

• Create: Post
• Read: Get
• Update: Put
• Delete: Delete

RPC API Pattern

The RPC (Remote Procedure Call) ..

Rubens Gomes

How to Set Up RESTClient for SSL Connections

The WizTools.org RESTClient 2.3.3 is a GUI application written in Java that helps with the testing of HTTP RESTful services.

Installation of RESTClient

Prerequisites

Prior to installing RESTClient, you should have the latest version of Java 6 JRE (or Java 7) installed on your computer. I recommend to install the java 6 JRE on the following folder "c:\java\jre6" (Windows) or "${HOME}/java/jre6" (UNIX). Ensure you have the latest version of java installed by running the following
comand from the system prompt. In my case, I verified below that my PC has a current version of Java 6 by entering the following command in my cygwin (Linux-Like) shell:

$ java -version
java version "1.6.0_30"
Java(TM) SE Runtime Environment (build 1.6.0_30-b12)
Java HotSpot(TM) 64-Bit Server VM (build 20.5-b03, mixed mode)

Installing RESTClient

In order to use the RESTClient GUI application you should first download the file "restclient-ui-2.3.3-jar-with-dependencies.jar" from the following page. I recommend you place the restclient jar file in a folder named "java\restclient" (Windows) or "java/restclient" (UNIX). For instance, "C:\java\restclient" (PC) or "${HOME}/java/restclient" (UNIX).

http://code.google.com/p/rest-client/downloads/list

Once you have the above jar file downloaded and saved in the above directory, you can run the RESTClient GUI by going into that folder, and running the command "java -jar restclient-ui-2.3.3-jar-with-dependencies.jar". For example, in my PC windows cygwin (Linux-like) shell, I run the following commands:

$ cd c:/java/restclient

$ pwd; ls -l
/cygdrive/c/java/restclient
total 8736
-rwx------+ 1 Administrators Domain Users 8944764 Jan 13 17:29 restclient-ui-2.3.3-jar-with-dependencies.jar

$ java -jar restclient-ui-2.3.3-jar-with-dependencies.jar

Importing SSL Certificate into a Local Keystore

Prior to being able to establish an SSL connection to your HTTP server from RESTClient, you must first have the server certificate exported from your browser to a local folder in your PC. Then, you need to import that certificate to a local SSL store file. After that, you will point the RESTClient to that SSL store file in order to be able to establish an SSL connection.

In the steps below, I am explaining how I exported and imported the server public certificate using Firefox version 3.6.3. I have only tested this procedure from Firefox.

Exporting an SSL certificate file from FireFox

First, you should hit the application server using an HTTPS scheme to upload and save the certificate in the browser. If your certificate was not generated from an authenticated SSL source (godaddy.com, for example), you will see a window with a message about "This Connection is Untrusted.... blah, blah, blah...." Simply select "I Understand the Risks", and "Confirm Security Exception".

Now, at the browser navigation bar (where you type the URL), you should see the left of the URL a little icon to indicate the SSL connection for that URL. Click on that icon to open up a window that contains information about that SSL certificate. Click on the "More Information ..." button to open up a window with a few tabs at the top. You should now be at the "Security" tab. Click on the "View Certificate" button to open up a window that contains your certificate information. Click on that window "Details" tab. After that, click on "Export...", and save that file to a temporary folder in your computer (e.g., "C:\tmp" or "/tmp"). Keep the Save as type "X.509 Certificate (PEM)" selected, and name the file however you like. For example, I named mine as "restclient", and Firefox automatically appended the extension "crt" to the filename.

That is it for exporting the SSL certificate to a local file. Now, the next step is to import the SSL certificate in that file to a local java keystore that will be used by the RESTClient application.

Importing PEM Firefox certificate into Java Keystore

In order to import the PEM certificate exported earlier into a keystore, you will need to have access to the java "keytool.exe" command from your system prompt. You will need to specify a "keystore" filename that you will later configure on the RESTClient application. The steps below illustrates the commands that I had to do on my cygwin (Linux-Like) shell to import this PEM certificate to a local java keystore.

$ pwd
/cygdrive/c/tmp
rugomes@rugomes-WS /cygdrive/c/tmp

$ ls -l restclient.crt
-rwx------+ 1 Administrators Domain Users 1336 Jan 25 18:15 restclient.crt

Now, type the command below to import the SSL certificate into a keystore called "c:/java/restclient.store". Notice that I am using cygwin (Linux-Like) shell, and the folders are separated by "/". If you use the Windows command shell, you would name that keystore as "C:\java\restclient.store" instead.

When prompted for the keystore password, simply type changeit.

$ keytool -noprompt -import -keystore "c:/java/restclient.store" -alias restclient -file restclient.crt
Enter keystore password: changeit
Re-enter new password: changeit
Certificate was added to keystore

To verify that your certificate was properly added to the above keystore (e.g., "C:\java\restclient.store"), type the following command. In my case, I am typing the following command from my cygwin (Linux-Like) shell. When prompted for the password, enter the same password typed previously "changeit".

The information about the certificate just imported should be displayed on your computer. I am not showing that entire screen here for security reasons (that is, I don't want to reveal the details of my certificate).

$ keytool -list -v -keystore "c:/java/restclient.store"
Enter keystore password: changeit
Keystore type: JKS
Keystore provider: SUN
Your keystore contains 1 entry
Alias name: restclient
Creation date: Jan 25, 2012
Entry type: trustedCertEntry
Owner: ...
Issuer: ...
Serial number: ...
Valid from: Fri Jan 20 17:00:01 EST 2012 until: Mon Jan 17 17:00:01 EST 2022
Certificate fingerprints:blah, blah, blah...

Configuring the SSL store on RESTClient

Now that we have the above certificate store created with the proper certificate, we can go to the step of configuring that store on the RESTClient GUI. For this step, you need to run the RESTClient GUI application, and go to the SSL tab window. Here is how I ran the RESTClient GUI from my Cygwin Linux-Like shell:

$ cd c:
rugomes@rugomes-WS /cygdrive/c
$ cd java/restclient
rugomes@rugomes-WS /cygdrive/c/java/restclient
$ java -jar restclient-ui-2.3.3-jar-with-dependencies.jar

Once the RESTClient GUI window shows up, select the "SSL tab". Then click the little button to the right of the "Trust store file: " prompt, and navigate thru the windows to select the above certificate keystore filename (e.g. C:\java\restclient.store).

Under the "Trust store password:", type the password "changeit" that was used during the above step when you created the keystore and imported the certificate into it. Then, you may want to select "Strict" for the Hostname verifier. Here is what each option under "Hostanem verifier" means:

ALLOW_ALL: The URL requested doesn't need to match the URL in the Certificate.
STRICT: The URL requested needs to match the URL in the Certificate.
BROWSER_COMPATIBLE: The URL requested must be in the same domain

Now, enter the https URL to the URL prompt under the same SSL tab window, and you should see the response from your HTTP/SSL connection. For example, in my case, I entered a URL similar to the following in the URL prompt of the RESTClient GUI:

https://{host}:{port}//rest/reports/test?outputType=csv

Please, note that you may need to configure other headers as required by your HTTP RESTful Web Service. For example, in my case I also had to configure Auth Type BASIC, and enter a Username and Password in the Auth tab window.

Happy SSL REST testing :)


Rubens.