A key component to many architectures these days, especially microservice architectures, is a queue to separate services and provide a reliable communication mechanism. Azure has a few solutions that fit different needs for queue’s, but one often used for enterprise level reliable message queuing is Service Bus.
Another key component to software applications is monitoring. When building cloud native solutions (whether on Azure, AWS, GCP, whatever), this need is amplified due to the multiple services and their respective SLA’s (or lack thereof, in some cases) which in aggregate make up the solution and its SLA. It’s also important to monitor traffic on each service to determine where and when you need to scale, thus leveraging one of the great benefits cloud solutions gives you. In Azure, we have Azure Monitor built-in that we can use to pump metrics to a single location and do nice things like fire alerts, build reports, automate scale, or track information for regulatory needs.
With the above in mind, getting some detailed metrics from Service Bus into Monitor would be very beneficial when it comes to insights at the queue or topic level. The integration between Service Bus and Monitor currently provides metrics such as incoming, outgoing, size, connections, errors, and others. However, when looking at a topic that may have multiple subscriptions set up, it would be beneficial to have some metrics at that level, too, so we can monitor Service Bus and how incoming messages are being distributed to various topics and picked up by any listeners. Or not, in which case they become dead-lettered.
With Service Bus having a nice mature API and Monitor now providing the ability to send in custom metrics, building a solution to monitor topics and subscriptions is now within reach! To show how this can be done, I built a Python app that uses the Azure Python SDK to query Service Bus and send counts for both active messages and dead-letter messages for subscriptions within a topic (that’s a mouthful) to Monitor. There is a Dockerfile in the repo that packages it up so it can be run easily as a cron job in Kubernetes or any other platform that is container friendly.
I feel the code (hosted on GitHub) is self-explanatory (famous last words), so I’m not going into the technical detail in this post. Please comment here or on GitHub with any questions/comments/PR’s.