Microservices: Hystrix dashboard – Detecting point of failures

This is the 4th part of the Developing Microservices with Spring Boot series. After being able to create an edge service, compose services and establish a fallback strategy, we will have a look to how to monitor the connections that the proxy does to other services. The full code is available on github.

When specifying callback strategies and proxying to an increasing number of microservices it would be good to have some kind of monitoring.

Using Zuul let us to create a stream of data related to the connections that are proxied. This data can be rendered by a Hystrix dashboard.

Hystrix dashboard

Hystrix dashboard

In this post we will create a Hystrix dashboard and see how this can help monitor our current microservices infrastructure. To reduce the complexity, we will take the architecture we have been working on the previous post.

Hands on code

We will have our hystrix dashboard server initialized through the configuration microservice. Bootstrapping the service will allow us to control all the initial configuration from one place.

Configuration service

File: configuration-service/src/main/resources/config/hystrix-dashboard.properties

Dashboard service

In order to build the dashboard service we will need to add the following dependencies to our pom.xml

As the server is bootstrapped, instead of application.properties we will have the bootstrap one (hystrix-dashboard/src/main/resources/bootstrap.properties)

On the main configuration file we will need to add the @EnableHystrixDashboard annotation:

After setting up the main class, we should be able to deploy and test it without further actions.

Deployment

After having done the modifications to the projects (or checked out the code), we can run each of the services with the following:

mvn spring-boot:run

For this example, to run everything, the recommended way to deploy is:

  • Deploy config-service
  • Deploy service a
  • Deploy service b
  • Deploy Eureka service
  • Deploy Zuul proxy service

Running jps should give us something like:

Test

After running the hystrix dashboard service, we should be able to test it hitting the url http://localhost:9010/hystrix.html

Hystrix dashboard main page

Hystrix dashboard main page

When running Zuul, by default, the hystrix stream is on http://localhost:9000/hystrix.stream. Opening this url on the browser should print the streamed data generated on real time (don’t ping that url!!, but it is ok to visualize it on the browser).

In order to see the stream with data flowing, you could hit the following urls:

This should make the stream to have some data. After having introduced the stream url on the hystrix page, we should be able to see a dashboard representing real time data of what is going on with the connections between the services:

Hystrix dashboard showcasing communication between services

Hystrix dashboard showcasing communication between services

Having a look to the hystrix dashboards there are some interesting things to highlight:

  • We could detect availability of the service quickly without having to go to logs or pinging
  • Looking to the percentiles makes easier to predict possible availability of the service.
  • Request graphs could help us to detect bottle neck and ‘God’ microservices

Summary

  • Hystrix dashboards could be part of our main dashboard for our microservices infrastructure. On a quick look we should be able to see if the system is right and how well each of the services are doing.
  • This, in combination with a distributed tracing monitoring (example Zipkin) could give us a good visibility about what is going on the microservices architecture.

One thought on “Microservices: Hystrix dashboard – Detecting point of failures

Leave a Reply

Your email address will not be published. Required fields are marked *