Microservices and Standards; Request / Response

Standards are key to implementing a Microservice Architecture.  In this article, I want to focus on the Request and Response types.   A request usually enters a gateway into a microservice architecture system.  That request is processed and a response is generated.  Then, another request is generated from the second microservice which processes and generates a second response. This process goes on for several, sometimes hundreds of microservices until finally a response leaves the system.  To correlate the various requests and responses with a unique transaction all requests/responses must share a transaction ID.  In addition to a transaction, it is also important to identify the originating client that made the originating request.  Therefore, having a base request/response type shared among all microservices will help ensure that the processing of these requests and responses can be standardized across the system.

Rubens Gomes

Transitioning to a Microservices Architecture - Part 2

Microservices and the Development Organization

Microservice architecture is based on the principles of modular systems that comply with the domain-driven design paradigm.  That is, the architecture is broken into sub-domains with delegated specialized responsibilities that are delineated from other modules by what is called a boundary context.  Therefore, in order to have specialized knowledge that is best able to address the concerns of these sub-domains, it is best to have a development organization that has teams of experts in the particular sub-domain. The team becomes the owner of microservices in that sub-domain.  This type of development organization is key to implementing a microservices architecture.

As stated in Conway's Law:

"O]rganizations which design systems (in the broad sense used here) are constrained to produce designs which are copies of the communication structures of these organizations."

The implications of Conway's Law are fundamental to the successful implementation of microservices architecture.  Because a microservice is a module with a responsibility within a domain therefore with a clear boundary context, having teams that become delegated to work on that specific sub-domains will tend to facilitate the implementation of microservices.

Again, what I am trying to say is that in order to achieve the best results from a Microservice Architecture it is imperative that an organization is broken down into sub-domain knowledge expertise.  The Microservice Architecture goes hand in hand with Conway's Law.  That is, by breaking organizations into sub-domains, you force those organizations to create applications that more uniquely resemble the microservices of those sub-domains.  The communication within those expertise groups becomes very cohesive, unique, and very focused on the intents/requirements of those sub-domains.  This will help create a Microservice architecture. 

In fact, one way to force an organization to move to a Microservice Architecture is to use the reverse (or also known inverse) Conway's Law pattern, which basically states that by breaking an organization into groups of sub-domain experts you end up creating a Microservice Architecture.

Rubens Gomes

Why I Like Microservices

In order to explain some of my reasons why I like the microservices architecture style, I write and compare monoliths vs microservices based on practical real-life experiences of working with software development for more than 30 years in both small and large companies.

Monolith Development Environment Setup

The setup of a developer environment is one of the most important steps a developer must take when starting a job.  It entails not only the configuration of tools (e.g., IDEs, text editors), networking access, familiarity with databases, and source control, but most of all having an environment that facilitates the development and maintenance of the application.  And this is right off the bat one of the issues I have with monolith systems.  Monoliths are usually very complex to set up.  I have a few real-life examples of spending over a week just to get my development set up for a single monolith application.  It is sometimes licensed to use platforms like J2EE, all the libraries being used, the build itself that involves several components, and the database which is usually much larger.  Having the tests set up and running. Learning about the entire business domain. Everything in a monolith is orders of magnitude more complex than in microservices.

In one job it took me over two weeks to set up my development environment for one of the main monolith applications in that department. The monolith was a J2EE backend server implemented using IBM WebSphere.  It not only required installation of IBM WebSphere locally but lots of various configurations for the various components (database, interface to other systems, access to multiple libraries.) The build was very complex as there were different components being built with different property configuration files.

Monolith Long Meetings and Long Releases

I remember while being a senior architect at a very large enterprise when I used to attend weekly meetings with project managers, the QA manager, and the department manager to discuss the various features that were being implemented for the next release. The wall had all these different features and where each feature was in the development lifecycle phase.  As the timeline for the next release approached we had discussions on what was ready, and what was still in QA testing, and when we would think we could start planning a deployment date.  Everything took longer: development, testing, interaction with various teams, orchestrating the different projects, synchronizing the different features being developed in parallel, contacting the Operations teams to set up time for deployment, and on and on. This is to me one of the great drawbacks of monolith systems, the time it takes to get something in production. It usually encompasses so many steps that can take months just to get a single release in production.

Since there were various features, trying to get all of them in a single release was not always possible, we broke different projects into separate branches.  And those branches would move in parallel. At the same time, developers were cranking code, issues would arise in production therefore requesting hot fix branches to be created.  As if that was not enough we had all the operational issues to deal with.  When to deploy, who was going to deploy, at what time, and so on?

Microservice Development Environment Setup

On the other hand, a microservice typically has smaller databases, fewer libraries, a much less complex business sub-domain component, fewer tests, and of course much fewer lines of code to deal with. I have seen real-life experiences of a contractor joining a team in the morning, and in the afternoon having a development set up to work on a specific or few microservices.  And not only that being able to start coding on the first day!

From my previous experiences with monoliths, once we had converted it to microservices, things became so much easier.  Setting up a development environment in Eclipse or IntelliJIDEA only took a matter of minutes.  All I had to do was to clone the microservice project locally and import it to Eclipse as a Maven project.  Then, running builds was simply executing CLI commands or pressing a button in DevOps Microsoft Azure.  Bottom line from the IDE I was ready to start modifying the code in a matter of minutes.

Why Is It Easier to Work with Microservices?

First, microservices are very small applications that are focused on doing one thing and doing it well.  Therefore, there is no longer all of the monolith's different technologies, components, databases, and libraries to deal with.  Furthermore, the build of microservices becomes much simpler and faster since it does not depend on the various component libraries found in monoliths.  Since microservices are focused on a specific sub-domain the learning curve is much faster. Our brains are more capable of dealing with one smaller problem at a time; versus in a monolith that deals with everything at once.

Getting things up and running, writing a test, writing a feature, being focused, delivering features, fixing bugs everything is SO MUCH faster and easier when you are working on microservices compared to monoliths.  In a microservice, we can become very focused on a small area of the business, and we can ensure that features are delivered faster to production.  That's because testing is easier, and faster.  And builds and deployments are not only easier but much faster.  I have seen cases where a PR was approved in the morning, and the code was in production on the same day.  Also, tracking issues in logs with microservices is simpler because we can set up queries on specific applications.  And that means easier troubleshooting!

Rubens Gomes

Transitioning to a Microservices Architecture - Part 1

I had the opportunity to play a technical lead role in the implementation of  American Airlines Ticketing department microservices architecture from 2016-2023.  The AA Ticketing department is responsible for the booking and payment processing of over 700K airline tickets daily.  The company's IT department started its IT transformation around 2017, and the ticketing team was one of the first to have microservices in production. 

The transitioning to Microservices involves significant changes not only to the technical architecture, and continuous implementation/delivery but a social and cultural organizational change.  In a microservice architecture, the teams become more independent and self-organized around a specific piece of the domain.  That team maintains the expertise not only on a specific business sub-domain but is also responsible for its development, deployment, and production support.

In addition to the team organization structural changes, a transition to microservices requires fundamental components that must be in place for its successful implementation.  I would like to elaborate on some of these components next, and why they are essential in a microservices architecture.

Please see my next edition of "Transitioning to Microservices Architecture - Part 2" coming up.  I have so much to talk about this topic since I spent significant time seeing from the ground-up in a real and practical large microservices architecture implementation in a very large enterprise.  

Rubens Gomes

What Is a Service

In the context of software, service is a computer message invocation that is sent over a telecommunications networking protocol connection (e.g., TCP/IP or UDP/IP socket) to a remote computer. This message invocation has the intent of stimulating an application or multiple applications that are sitting on remote machines to perform some operation. That is, the remote application, upon receipt of the requested message, is expected to parse and process the message accordingly. And then it may either simply process the message and return no response, it could return a response message to the invoker.  Sometimes, the application may even forward that message to another systems for processing.

Based on the above definition of a service, we may have several different communication protocols that could used for the transfer of the request and response service messages between the remote computing ends.  Below are some common communication protocols that have been used in more recent computing:

• REST over HTTP
• SOAP over HTTP
• CORBA
• Some private messaging protocol over HTTP
• Some private messaging protocol over TCP/IP socket
• HL7 used in the healthcare arena
• SNMP used in the network management arena

CRUD

The essence of any service is based on the CRUD terminology. CRUD stands for Create, Read, Update and Delete. Most of the service APIs fall into one of the CRUD categories. For example:

• Create: AddAccount, CreateAccount
• Read: GetAccount, ReadAccount, RetrieveAccount
• Update: SaveAccount, UpdateAccount, ModifyAccount
• Delete: DeleteAccount, RemoveAccount

As we can see from the CRUD message APIs above, there are many different ways that the same service message can be named. The service API designer should therefore be careful to follow a consistent approach when naming the APIs. Otherwise, we may end up with several different names that mean the same thing (e.g. Create/Add or Get/Read or Delete/Remove).

Service Semantics and Behavior

A service should be well defined and understood, which means that its semantics and behavior should be clearly stated.

• Semantics: It consists of the all the wording that is used to define a service. For example, the service message name, message attributes names and types, and any type of errors (or exceptions) that might be incurred as a result of invoking that service. For example, when creating a customer login, the semantics might be defined as follows "createCustomerLogin(username, password, email), where the username, password and email are all text data defined by certain constraints; e.g., number of characters and character encoding scheme.

Notice, that in the above semantics we are not concerned about the underlying technical implementation of that service. In the "service contract" document, we would then add a section that provides specifics of the semantics implementation. In that case, we would define request/response messages using a WSDL language (SOAP), or use the IDL language for a CORBA service. For RESTFul, we would implement the service semantics in terms of its URI with HTTP header/POST body (request) and corresponding HTTP headers/body (response) along with the media type (e.g., JSON or XML).

• Behavior: It consists of the expected actions that might occur as a result of invoking a given service message. In this case, we want to elaborate on what activities might occur as the outcome of invoking the service. For example, in the case of creating a customer login, we would elaborate that an account with a unique user name is created on a given computer system. Or in multiple computer systems, depending on the business case being solved.

Characteristics of Good Services

A service should, foremost, address a specific business need. A service is therefore a means to an end; that is, it should address and solve a business requirement. Throughout the design and implementation of a service, the implementer should keep in mind the business audience who will consume that service, and aim the naming of the service API to that business audience.  That is, service request/response messages, attributes, and resources should have names that are meaningful and aimed at the business audience who will be consuming that service API.  Furthermore, the service API should be, whenever possible, validated with the business parties; that is, the programmers or business analysts  who will be ultimately using that service API.  The goal here is to ensure that the service API is not only well aligned with the business needs, but that it is easy to use and be programmed with.    The service API designer or implementer should refrain from trying to map a backend application API to a service API.

A side effect of using service enabler infrastructure computing (e.g., ESB, enterprise service buses) is  the use of what is called "technical services".  These type of services are used to collect technical information, such as performance analytic, logs, errors and other troubleshooting information from the underlying service enabler infrastructure.

Service Patterns

By service patterns, it is meant good practices and tips that are followed in the market to name a few of the service messages APIs and protocols.

Resource API Pattern

The Resource API Pattern is implemented by RESTFul web services. That is, the API is operating on a given resource (e.g. Account, User, Report, and so on). And the CRUD behaviour is leveraged by the underlying HTTP Protocol. That is, for HTTP the CRUD is as follows:

• Create: Post
• Read: Get
• Update: Put
• Delete: Delete

RPC API Pattern

The RPC (Remote Procedure Call) ..

Rubens Gomes

How to Set Up RESTClient for SSL Connections

The WizTools.org RESTClient 2.3.3 is a GUI application written in Java that helps with the testing of HTTP RESTful services.

Installation of RESTClient

Prerequisites

Prior to installing RESTClient, you should have the latest version of Java 6 JRE (or Java 7) installed on your computer. I recommend to install the java 6 JRE on the following folder "c:\java\jre6" (Windows) or "${HOME}/java/jre6" (UNIX). Ensure you have the latest version of java installed by running the following
comand from the system prompt. In my case, I verified below that my PC has a current version of Java 6 by entering the following command in my cygwin (Linux-Like) shell:

$ java -version
java version "1.6.0_30"
Java(TM) SE Runtime Environment (build 1.6.0_30-b12)
Java HotSpot(TM) 64-Bit Server VM (build 20.5-b03, mixed mode)

Installing RESTClient

In order to use the RESTClient GUI application you should first download the file "restclient-ui-2.3.3-jar-with-dependencies.jar" from the following page. I recommend you place the restclient jar file in a folder named "java\restclient" (Windows) or "java/restclient" (UNIX). For instance, "C:\java\restclient" (PC) or "${HOME}/java/restclient" (UNIX).

http://code.google.com/p/rest-client/downloads/list

Once you have the above jar file downloaded and saved in the above directory, you can run the RESTClient GUI by going into that folder, and running the command "java -jar restclient-ui-2.3.3-jar-with-dependencies.jar". For example, in my PC windows cygwin (Linux-like) shell, I run the following commands:

$ cd c:/java/restclient

$ pwd; ls -l
/cygdrive/c/java/restclient
total 8736
-rwx------+ 1 Administrators Domain Users 8944764 Jan 13 17:29 restclient-ui-2.3.3-jar-with-dependencies.jar

$ java -jar restclient-ui-2.3.3-jar-with-dependencies.jar

Importing SSL Certificate into a Local Keystore

Prior to being able to establish an SSL connection to your HTTP server from RESTClient, you must first have the server certificate exported from your browser to a local folder in your PC. Then, you need to import that certificate to a local SSL store file. After that, you will point the RESTClient to that SSL store file in order to be able to establish an SSL connection.

In the steps below, I am explaining how I exported and imported the server public certificate using Firefox version 3.6.3. I have only tested this procedure from Firefox.

Exporting an SSL certificate file from FireFox

First, you should hit the application server using an HTTPS scheme to upload and save the certificate in the browser. If your certificate was not generated from an authenticated SSL source (godaddy.com, for example), you will see a window with a message about "This Connection is Untrusted.... blah, blah, blah...." Simply select "I Understand the Risks", and "Confirm Security Exception".

Now, at the browser navigation bar (where you type the URL), you should see the left of the URL a little icon to indicate the SSL connection for that URL. Click on that icon to open up a window that contains information about that SSL certificate. Click on the "More Information ..." button to open up a window with a few tabs at the top. You should now be at the "Security" tab. Click on the "View Certificate" button to open up a window that contains your certificate information. Click on that window "Details" tab. After that, click on "Export...", and save that file to a temporary folder in your computer (e.g., "C:\tmp" or "/tmp"). Keep the Save as type "X.509 Certificate (PEM)" selected, and name the file however you like. For example, I named mine as "restclient", and Firefox automatically appended the extension "crt" to the filename.

That is it for exporting the SSL certificate to a local file. Now, the next step is to import the SSL certificate in that file to a local java keystore that will be used by the RESTClient application.

Importing PEM Firefox certificate into Java Keystore

In order to import the PEM certificate exported earlier into a keystore, you will need to have access to the java "keytool.exe" command from your system prompt. You will need to specify a "keystore" filename that you will later configure on the RESTClient application. The steps below illustrates the commands that I had to do on my cygwin (Linux-Like) shell to import this PEM certificate to a local java keystore.

$ pwd
/cygdrive/c/tmp
rugomes@rugomes-WS /cygdrive/c/tmp

$ ls -l restclient.crt
-rwx------+ 1 Administrators Domain Users 1336 Jan 25 18:15 restclient.crt

Now, type the command below to import the SSL certificate into a keystore called "c:/java/restclient.store". Notice that I am using cygwin (Linux-Like) shell, and the folders are separated by "/". If you use the Windows command shell, you would name that keystore as "C:\java\restclient.store" instead.

When prompted for the keystore password, simply type changeit.

$ keytool -noprompt -import -keystore "c:/java/restclient.store" -alias restclient -file restclient.crt
Enter keystore password: changeit
Re-enter new password: changeit
Certificate was added to keystore

To verify that your certificate was properly added to the above keystore (e.g., "C:\java\restclient.store"), type the following command. In my case, I am typing the following command from my cygwin (Linux-Like) shell. When prompted for the password, enter the same password typed previously "changeit".

The information about the certificate just imported should be displayed on your computer. I am not showing that entire screen here for security reasons (that is, I don't want to reveal the details of my certificate).

$ keytool -list -v -keystore "c:/java/restclient.store"
Enter keystore password: changeit
Keystore type: JKS
Keystore provider: SUN
Your keystore contains 1 entry
Alias name: restclient
Creation date: Jan 25, 2012
Entry type: trustedCertEntry
Owner: ...
Issuer: ...
Serial number: ...
Valid from: Fri Jan 20 17:00:01 EST 2012 until: Mon Jan 17 17:00:01 EST 2022
Certificate fingerprints:blah, blah, blah...

Configuring the SSL store on RESTClient

Now that we have the above certificate store created with the proper certificate, we can go to the step of configuring that store on the RESTClient GUI. For this step, you need to run the RESTClient GUI application, and go to the SSL tab window. Here is how I ran the RESTClient GUI from my Cygwin Linux-Like shell:

$ cd c:
rugomes@rugomes-WS /cygdrive/c
$ cd java/restclient
rugomes@rugomes-WS /cygdrive/c/java/restclient
$ java -jar restclient-ui-2.3.3-jar-with-dependencies.jar

Once the RESTClient GUI window shows up, select the "SSL tab". Then click the little button to the right of the "Trust store file: " prompt, and navigate thru the windows to select the above certificate keystore filename (e.g. C:\java\restclient.store).

Under the "Trust store password:", type the password "changeit" that was used during the above step when you created the keystore and imported the certificate into it. Then, you may want to select "Strict" for the Hostname verifier. Here is what each option under "Hostanem verifier" means:

ALLOW_ALL: The URL requested doesn't need to match the URL in the Certificate.
STRICT: The URL requested needs to match the URL in the Certificate.
BROWSER_COMPATIBLE: The URL requested must be in the same domain

Now, enter the https URL to the URL prompt under the same SSL tab window, and you should see the response from your HTTP/SSL connection. For example, in my case, I entered a URL similar to the following in the URL prompt of the RESTClient GUI:

https://{host}:{port}//rest/reports/test?outputType=csv

Please, note that you may need to configure other headers as required by your HTTP RESTful Web Service. For example, in my case I also had to configure Auth Type BASIC, and enter a Username and Password in the Auth tab window.

Happy SSL REST testing :)


Rubens.

What Is a Literal in Computer Programming?

In computer programming languages, a literal refers to a piece of information (such as a word, digit) that is written exactly as it is meant to be interpreted.

Below are some statements followed by comments (text starting with //). In Java a statement is a text ending with a ";". A statement is a complete sentence that provides a command (or instruction) to be executed by a computer. Comments is any text not interpreted nor executed by a computer. It is there solely for readability purposes. The computer skips everything that starts with "//" until the end of the line.

int x;

// int is a type

// x is a variable

x = 1; 

// 1 is an integer literal

String text = "abc";

// text is a variable

// = is an operator

// "abc" is a literal

Long nr = null;
// nr is a variable
// null is a literal

boolean var = true; 

// boolean is a type

// var is variable

// = is an operator

// true is a literal

As you can see, "1" is a integer number whose value is also 1. Therefore, 1 is a literal. "null" is a special kind of literal that has a value of 0 (zero). In the case above, assigning "null" literal to the variable "nr" implies that the computer should use the corresponding value of null (most of the times 0) and assign it to the variable. Other types of literal in java is true or false. In the case above the computer will assign "true" to the boolean typed variable called "var".

Rubens Gomes