As many learning experiences do, this one also starts out “So I was working on a project at work and…”.  In this case, the end result is to try to run as many concurrent copies of MySQL on a single server as possible, maintaining real time replication each running differing data sets. To help with this, I sent out to do this on a server with 36 7200rpm 4GB SATA disks, giving me roughly 120TB of available space to work with.

This isn’t an abnormal type of machine for us. Sometimes you simply need a ton of disk space. There is a quirk with this particular machine that I’ve been told: the RAID controller has some issues with addressing very large virtual disks and I should create 2 60TB volumes and stitch them together with LVM. Easy enough: pvcreate both volumes, create a volume group and a logical volume out of it and viola: …

The multi master plugin for MySQL is here. MySQL Group Replication provides virtually synchronous updates on any node in a group of MySQL servers, with conflict handling and automatic group membership management and failure detection.

For a better understanding on how things work, we go under the hood in this post and will analyse the transaction life cycle on multi master and which components does it interact with. But before that we need to understand first what a group is.

Group Communication Toolkit

The multi master plugin is powered by a group communication toolkit. This is what decides which servers belong to the group, performs failure detection and orders server messages. Being the ordered messaging the magic thing that allows the data to be consistent across all nodes. You can check the details of the group communication toolkit at …

More MySQL releases than ever before. Check out the following that we just released:

– MySQL Connector Java 5.1.33 GA
– MySQL Fabric & MySQL Utilities 1.5.2 GA
– MySQL Connector Python 2.0.1 GA
– MySQL Fabric and Utilities 1.6.0 Alpha
– MySQL Connector C (Labs)
– MySQL Connector Python (Labs)
– MySQL Group Replication (Labs)
– MySQL HTTP Plugin (Labs)

Download from http://dev.mysql.com/downloads/ and http://dev.mysql.com/downloads/

The multi master plugin for MySQL is here “MySQL Group Replication“. It is a virtual synchronous solution for MySQL with conflict detection. It also supports automatic group membership management, failure detection and automatic distributed recovery.

With the introduction of this new feature there was a need to perform some good amount of testing as it involves complex functionalities like :

• Servers execute local transactions and broadcasts the update to the group.
• All servers in the group, even the sender, receive same transaction in the same order and check for conflicts.
• All servers, independently, decide to commit the transaction – no conflicts.
• A new node can join an existing group, so in this case we need distributed recovery to bring it at par with the other servers.

So a great deal of effort has been spent on testing these …

Multi-master update everywhere solution for MySQL is here. MySQL Group Replication ensures virtual synchronous updates on any node in a group of MySQL servers, with conflict handling and failure detection. Distributed recovery is also in the package to ease the process of adding new nodes.

Starting with MySQL 5.7.2 there has been a constant effort from the replication team to provide more fields to monitor the replication performance in the performance schema tables. This post gives a brief  overview of the Performance Schema tables that have been introduced for MySQL Group Replication.

TABLES INTRODUCED

There are two new tables introduced as a part of the MySQL Group Replication monitoring.

• performance_schema.replication_node_status
• performance_schema.replication_connection_nodes

Lets go through each table in detail :

REPLICATION NODE STATUS

With the introduction of Global Transaction Identifiers(GTID) in MySQL from mysql-5.6 GA a whole lot of different functionality have been developed around it so that GTID could be used in a much simpler and efficient way.

One of the interesting functionality with GTID in use is the function to sync a slave with its
master server using the WAIT_UNTIL_SQL_THREAD_AFTER_GTIDS. This is used to do a timed or an indefinite wait till the servers in contention are in sync with respect to the GTID executed set.

This function is used to address specific use cases in which transactions are applied using SQL/worker thread. In order to give more flexibility to the user a new function of syncing the servers with GTID has been introduced in mysql-5.7.5 called WAIT_FOR_EXECUTED_GTID_SET.

Limitations of WAIT_UNTIL_SQL_THREAD_AFTER_GITDS

The already existing WAIT_UNTIL_SQL_THREAD_AFTER_GITDS functions …

On MySQL-5.7.2, we released a new type of multi-threaded slave (MTS). It is called logical clock based MTS. Because it can parallel apply transactions in the same schema, it has the potential to improve slave throughput on almost any application, regardless of the schema layout. After it was released, we continued to work on the framework to improve it further. Therefore, 5.7.5 includes a few enhancements, in addition to those released as part of previous DMRs. This blog post introduces one of the new features in the latest DMR.

slave_preserve_commit_order

It is a system global variable and can be set dynamically.

• SET GLOBAL slave_perserve_commit_order = {ON|OFF};

Enabling this variable ensures that the order which transactions were committed on the master is preserved on the slave. The replication threads must be stopped before enabling this variable and this variable only affects logical clock based …

 I am happy to announce that slave_transaction_retries is available on multi-threaded slave now. You will no longer see the warning that slave_transaction_retries is not supported by multi-threaded slave since MySQL-5.7.5.

Multi-threaded slave and single-threaded slave share the system variable slave_transaction_retries and they have similar transaction retry logic. So this blog just brings you a little new knowledge and then refresh you with the transaction retry logic.

Transaction Retry In Worker Threads

For multi-threaded slave, Transaction retry is done in worker threads when they are applying their transactions. When a worker thread encounters a temporary error, it needs to read the transaction’s events from relay log by itself and then apply them again.

The only new thing is that worker threads need to read events from relay log by themselves. To support it, coordinator stores relay log position of …

The multi master plugin for MySQL is here. MySQL Group Replication ensures virtual synchronous updates on any node in a group of MySQL servers, with conflict handling and failure detection. Distributed recovery is also in the package to ease the process of adding new nodes to your server group.

How do you start? Just sit back, download MySQL Group Replication from http://labs.mysql.com/ and then let us begin this journey into the world of multi master MySQL.

Pre requisites

Under its hood, the multi master plugin is powered by a group communication toolkit. This is what decides which nodes belong to the server group, performs failure detection and orders server messages. This last being the magic thing that allows the data to be consistent across all nodes.

As of now we rely on the Corosync Cluster …

MySQL Group Replication is here and with it comes the need to install and configure the underlying group communication toolkit that supports it: Corosync. Corosync is a well known and reliable Group Communication System that is used in such applications as Pacemaker.

In term of support, we develop MySQL Group Replication based on Corosync version 1.4.6, so this tutorial is based on this version. Regardless of this, no known problems are know to exists when using newer versions, but no extensive testing has been done on those.

Along with Corosync we also encourage the use of NSS to better secure your data that is transmitted in the group.

Installing

To install Corosync, you can rely on your packet manager for most distributions or compile it from source.

==> From the package manager

• Debian distributions

$ sudo apt-get install corosync …