Monitoring your AWS MarkLogic Cluster using CloudWatch

Use of Cloud Services to host data infrastructure is ever increasing. Several of MarkLogic’s most significant customers have made the move with many more considering and it’s a frequent topic of interest amongst prospects.

The attractions are cost, flexibility and ease of use. For many, it’s the best option.

Of course, if you’re running a business critical system you need to think beyond the system itself and consider enterprise features such as consistency, backup, resiliency, scalability, and security. Features such as these are key differentiators between MarkLogic and other NoSQL databases.

Another aspect of operation you will want to consider is your ability to monitor your system, and more importantly to be assured that you can pro-actively deal with any unforeseen events, and even better, mitigate problems before they occur.

MarkLogic has a built-in Monitoring API. All aspects of its internal operation can be securely externalized. What this means in practice is that you can monitor your system, and potentially be alerted should the unexpected happen.

The other part of the picture is an external tool to handle monitoring and alerting. AWS CloudWatch is a highly flexible service that allows collection of system metrics and creation of alerting rules based on those metrics.

What’s then needed is something to link those two things together. Although conceptually straightforward, this can, in practice, be daunting, especially when considering the question of what to monitor and where to set the trigger levels. To that end I’ve created a CloudWatch for MarkLogic project in GitHub. This project provides easy-to-use tools allowing configuration of MarkLogic metric collection and setup of alerting within AWS CloudWatch. It also contains a test application and JMeter driver so you can gain immediate hands-on experience of using MarkLogic and AWS CloudWatch together.

If you’d like to look at this in more detail, the project contains a walkthrough with screenshots, and a README.

Of course, a one-size-fits-all approach is not enough so this utility allows easy control of which metrics are collected, and what levels alarms are set at and for production use these should be reviewed based on expectations and experience. Nevertheless, this quick start should help you get off the ground.

Learn More

• Presentation: MarkLogic on AWS
• Video: Deploying MarkLogic to Amazon EC2
• Page: MarkLogic and Amazon Web Services