Overview The Skinny

In this blog post we will discuss how to use the Tungsten Connector keep-alive feature to ensure long-running MySQL & MariaDB/Percona Server client sessions stay connected in a Tungsten Cluster.

Agenda What’s Here?

• Briefly explore how the Tungsten Connector works
• Describe the Connector keep-alives – what are they and why do we use them?
• Discuss why the keep-alive feature is not available in Bridge mode and why
• Examine how to tune the keep-alive feature in the Tungsten Connector

Tungsten Connector: A Primer A Very Brief Summary

The Tungsten Connector is an intelligent MySQL database proxy located between the clients and the database servers, providing a single connection point, while routing queries to …

In this blog post, we demonstrate how to convert a single standalone cluster into a Composite Primary/DR topology running in two data centers.

Our example starting cluster has 5 nodes (1 master and 4 slaves) and uses service name alpha. Our target cluster will have 6 nodes (3 per cluster) in 2 member clusters alpha_east and alpha_west in composite service alpha.

This means that we will reuse the existing service name alpha as the name of the new composite service, and create two new service names, one for each cluster (alpha_east and alpha_west).

Below is an INI file extract example for our starting standalone cluster with 5 nodes:

[defaults]
...

[alpha]
connectors=db1,db2,db3,db4,db5
master=db1
members=db1,db2,db3,db4,db5
topology=clustered

To convert the above configuration to a Composite Primary/DR:

1. First you must stop all services on all existing nodes:

   shell> stopall
   

   …

In this blog post, we talk about what happened during an installation of the Tungsten Cluster into an environment with SELinux running and mis-configured.

An attempt to execute `tpm install` on v5.3.2 recently failed with the below error:

ERROR >> node3_production_customer_com >> Unable to run 'sudo systemctl status mysqld.service' or the database server is not running (DatasourceBootScriptCheck) 
Update the /etc/sudoers file or disable sudo by adding --enable-sudo-access=false 

Worse, this customer reported that this appeared as a WARNING only in Dev and Staging tests. So we checked, and it seemed we were able to access systemctl properly:

shell> sudo systemctl status mysqld.service
● mysqld.service - MySQL Percona Server
Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
Active: activating (start-post) since Tue 2018-06-19 17:46:19 BST; 1min 15s ago …

In this blog post, we talk about how to configure automatic start at boot time for the Tungsten Clustering components.

By default, Tungsten Clustering does not start automatically on boot. To enable Tungsten Clustering to start at boot time, use the deployall script provided to create the necessary boot scripts:

shell> sudo /opt/continuent/tungsten/cluster-home/bin/deployall

To disable automatic startup at boot time, use the undeployall command:

shell> sudo /opt/continuent/tungsten/cluster-home/bin/undeployall

For Multisite/Multimaster deployments in specific, there are separate cross-site replicators running. In this case, a custom startup script must be created, otherwise the replicator will be unable to start as it has been configured in a different directory.

1. Create a link from the Tungsten Replicator service startup script in the operating system …

In this blog post, we talk about the Automatic Reconnect feature in the Tungsten Connector.

Automatic reconnect enables the Connector to re-establish a connection in the event of a transient failure. Under specific circumstances, the Connector will also retry the query.

Connector automatic reconnect is enabled by default in Proxy and Smartscale modes.

Use the following tpm command option on the command line (remove the leading hyphens inside INI files):

--connector-autoreconnect=false

to disable automatic reconnect.

This feature is not available while running in Bridge Mode. Use the tpm command option --connector-bridge-mode=false to disable Bridge mode.

Automatic reconnect enables retries of statements under the following circumstances:

• not in bridge mode
• not inside a transaction
• no temp table has been created
…

In this blog post, we talk about how existing client connections are handled by the Tungsten Connector when a manual master role switch is invoked and how to adjust that behavior.

When a graceful switch is invoked via cctrl or the Tungsten Dashboard, by default the Connector will wait for five (5) seconds to allow in-flight activities to complete before forcibly disconnecting all active connections from the application side, no matter what type of query was in use.

If connections still exist after the timeout interval, they are forcibly closed, and the application will get back an error.

This configuration setting ONLY applies to a manual switch. During a failover caused by loss of MySQL availability, there is no wait and all connections are force-closed immediately.

This timeout is adjusted via the tpm option …

In this blog post, we talk about how query connections are handled by the Tungsten Connector, especially read-only connections.

There are multiple ways to configure session handling in the Connector. The three main modes are Bridge, Proxy/Direct and Proxy/SmartScale.

In Bridge mode, the data source to connect to is chosen ONCE for the lifetime of the connection, which means that the selection of a different node will only happen if a NEW connection is opened through the Connector.

So if your application reuses its connections, all traffic sent through that session will continue to land on the selected read slave, i.e., when using connection pooling.

http://docs.continuent.com/tungsten-clustering-6.0/connector-bridgemode.html

The key difference is in how the slave latency checking is handled: …

Mermaids have the same probability of fixing your permission problems, but people continue believing in the FLUSH PRIVILEGES myth.I see suggesting the usage of FLUSH PRIVILEGES every time someone writes a tutorial or a solution to a problem regarding creating a new account or providing different privileges. For example, the top post on /r/mysql as of the writing of these lines, “MySQL:The user specified as a definer does not exist (error 1449)-Solutions” has multiple guilty cases of this (Update: the user has corrected those lines after I posted this article).

It is not my intention to bash that post, but I have seen …

If you're setting up MySQL Cluster on Oracle Linux or another Linux such as Red Hat, CentOS or Fedora, you might have come across some problems getting the nodes to communicate. There are two ways you might bump into problems with network connectivity: The iptables firewall, and SELinux. These security mechanisms might prevent your various nodes—management, data, and API—from communicating with each other in various ways and with various symptoms.

Let's have a look at what you're likely to see. 

Data nodes stuck at "starting"

The first thing you might notice is that your data nodes get stuck in the "starting" state. Running show in the management client gives something like this:

[ndbd(NDB)]     2 node(s)
id=2    @192.168.56.211  (mysql-5.6.11 ndb-7.3.2, Nodegroup: 0, Master)
id=3    @192.168.56.212  (mysql-5.6.11 ndb-7.3.2, starting, Nodegroup: 0)

...and it never moves away from starting.

…

You probably know that mysql -h host_or_ip can connect you to a remote host.

But did you know that you can change the host you are connected to from within mysql?

The undocumented (as far as I can tell, in the MySQL manual and in the “help” on the mysql command line) CONNECT statement can help.
(more…)