This is a blog post I wrote about monitoring tools and practices followed at Adxstudio in May 2012. The blog post was originally hosted on the Adxstudio community blog, but it is no longer accessible, and I am now hosting this post here for historical access.
Here at Adxstudio we’ve historically hosted websites for our clients on servers in our data centre when they didn’t have their own hosting infrastructure. With the adoption of Windows Azure over the last few years, we’ve been transitioning existing clients to use Windows Azure, as well as assisting new clients with hosting their websites in Windows Azure.
Part of our hosting offering in our own data centre was to use internally built tools that would run on the local servers to automatically monitor the health of websites, and restart the websites when they were detected to be in an unhealthy state. By an unhealthy state, what I really mean is that the website is completely down. The reasons for a website getting into such a state can vary greatly – and to be clear it is an extremely rare occurance – but sometimes it can happen, and when it does the monitoring tools compensate in an automated fashion.
Over the course of the last few weeks I’ve rewritten the monitoring tools (sometimes referred to as the ‘self-healing scripts’) to be easily installable in Azure deployments. The monitoring tools are simple yet effective – through a scheduled job, a monitoring program will try access a website on a recurring basis, and after several failed attempts it’ll restart the website. Recipients are notified by email when the website was failing and when it is restarted.
In single-website deployments the monitoring program is configured to know what endpoint name the site is running under so that it can connect to the local website by the machine’s IP address. In multi-website deployments, each website has to have a different hostname, and so along with the monitoring program being configured to connect to the sites by host name, a supplementary tool has been developed to add host entries to the web server’s hosts file based on the endpoints each website is using, so that the monitoring program can resolve the hostname locally.
The monitoring program, the host file editing program, and the scheduled job are all installed as startup tasks. This makes the deployment consistent and repeatable.
For all existing clients and our new and upcoming ones, the next time we’re working with you on your portal that’s hosted in Azure, know that we’ll be setting it up to include site monitoring, so that it’s always up and running.