Update for all of those that are following the progress of Monolith. I’ve been busy this week coding the reporting engine - the code that analyses the hundreds of variables that are being collected - and I’m pleased to say that Perl was the correct choice for this application.Installation on the client servers will be a breeze, and relatively few non-standard modules are required for the server application.

I’ve been silently wondering for some time if the reporting engine equations should be part of a pluggable system (stored in the database) or if they should be hard coded. After writing up all of the equations, I’ve come to the conclusion that, for a quicker release, the equations and reporting engine will be hard-coded this time around for the default system alerts. Howeve -> user-defined alerts will be pluggable. Revisions ahead of 2.0 might feature pluggable main equations but I want v2.0 out asap.

Second bit of …

Monolith: MySQL server monitoring. I only have servers that run Redhat/Debian/Ubuntu Linux x86 and x86_64. I don’t have machines to test MySQL monitoring for the following OSes. Perhaps you would like to be a test candidate for the new version of Monolith? If so, let me know and you’ll be on the list, as well as get beta testing credit.

OSX Server: PPC and Intel

AIX

FreeBSD / OpenBSD / NetBSD

Windows Server

Solaris: Intel & Sparc

Linux servers that are NOT Intel based.

What this involves: testing the client script, perl modules, snmp stats, and other functions that may require a custom client script for that architecture.

Kontrollbase is a MySQL monitoring, analytics, reporting, and historical analysis webapp for MySQL database administrators and advanced users of MySQL databases. Its goal is to give you enterprise-level tools that are built on the open-source software model. We want your servers to perform at their best and these tools will help you get the most out of your investment.

For more information on kontrollbase please see this page.
To download kontrollbase please see this page.

Progress on the Open Source enterprise grade MySQL monitoring system;  the schema for Monolith version 2 has been designed. Due to the many suggestions for features and the interest it has received I’ve put this on the front burner. That said, here is some more info on the next steps I’ll be taking.

• Monitored servers will use a command line agent (called remotely) to pull information from both MySQL and the OS.
• Historical information will include all values from global status and global variables, as well as CPU/Memory/IO/Disk usage.
• Standard graphing functions - the ones in the list from the previous post - will gather information from various view tables that contain historical data that is collected from the agent script. User defined graphing will allow you to look at historical values over time for any of the various global status or global variable settings (integer based ones anyway). …

What would you like to see in a free enterprise-grade monitoring system for your daily MySQL needs?I’m rewriting Monolith - MySQL DBA Console from the ground up. This will be version 2 and I would like to get some input from the global MySQL community.So far I am going with the following; comment with any improvements/additions.

• Variable interval polling of server statistics
• Over 50 different alerts (see list below)
• Graphing of various server statistics (see list below)
• Tuning recommendations with cnf file changes to apply to server
• Change control documents for recommended performance/security tuning
• Threshold based alerting with multiple alert groups: info,warn,critical
• Sorting/ordering of servers via groups. ie: client -> dev,stage,prod
• RSS feeds for each alert group
• XML export with user defined fields for external applications …

Large databases, long mysqldump times, long waits for globally locked tables. These problems basically never go away when you rely on mysqldump with –all-databases or a list of databases, as it dumps schemas serially. I’m not going to explain serial vs parallel processing here since that’s a larger topic. Suffice to say that in these days of multi-core / multi-cpu servers we only make use of one processor’s core when we serially export databases using mysqldump. So, I have a new script that attempts to alleviate those issues and now I need testers to provide feedback/improvements.

In order to keep some sanity when dealing with hundreds of database servers, the script takes care of the following:

1. low global locking time requirements: solved by parallel tasks / forked processes
2. backup file checking: with mysqldump files; it checks for “–Dump completed” at the end of the sql file …

I’m attempting this is be as unbiased as possible, since I write the Monolith application. This will hopefully help one decide between a free MySQL server monitoring system and paying for a per-server based licensed product. Both have strengths and weaknesses that should be pointed out before making a decision. You can infer the weaknesses based on the strengths below. That said, let’s just get into it.

Strengths of each product over the other

Monolith - MySQL DBA Console

• presents overall database size, index size, data size, number of schema per server, and aggregate statistics for all monitored servers
• runs mysql server backups remotely and reports on backup state execution
• collects cnf files from each server during the talkback script execution for historical viewing
• built on the LAMP stack, no need for tomcat/jboss knowlege
• provides …

After the release of version 1.4 yesterday I was getting a lot of emails with questions so I decided to setup a wiki site for the application. It details the install process, the application design, and some other sections. You can see it here: http://www.monolith-mysql.com

Big changes in this release.

Features:

• Talkback report for local script backups to report back to monolith console
• Status page now summarizes all Codes and failed server connections for past 24 hours
• Install now has correct data import for user setup and system settings
• More GRAPHS!
• Improved logic for analytics and automated tuning recommendations, fixed thread_status bug
• Change request document generation
• Full HTML reporting for statistics gathering and tuning report
• XML API page for custom export of data with all fields available for output
• Integrates with VisualMining: see api-monitor-visualmining.php file
• Data table for index+data size reporting of all monitored databases
• Improved table information gathering via INFORMATION_SCHEMA - no more SHOW commands …

New features for the next release. As follows:

1. Status page now reports daily error code summarizations, connection failures to hosts
2. Talkback report page now reports on myback_talkback script version number
3. Talkback report page now tracks the my.cnf file during the backup process, and the cnf file contents for each host are stored as LONGTEXT in the monitor database
4. Multi-tier access levels, admins (rw) and general users (ro)
5. View tables for the status page summarizations speed up reporting
6. Daily summary email - similar to the status page, instead of singular emails for each backup success/fail notification

Look for 1.3 being released very soon! You’ll be able to find it here: http://sourceforge.net/projects/monolith-mysql