Overview The Skinny

In this blog post, we explore the correct way to implement bi-directional Tungsten Replication between AWS Aurora and Tungsten Clustering for MySQL databases.

Background The Story

When we are approached by a prospect interested in using our solutions, we are proud of our pre-sales process by which that we engage at a very deep technical level to ensure the we provide the best possible solution to meet with the prospect’s requirements. This involves an in-depth hands-on POC, in addition to the significant time and effort we spend building and testing the solution architectures in our lab environment as part of the proposal process.

From time to time, we are presented with requirements that are not always quite so straight forward. Just recently we faced such a situation. A …

When Tungsten Replicator extracts data, the information that has been extracted is written down into the Tungsten History Log, or THL. These files are in a specific format and they are used to store all of the extracted information in a format that can easily be used to recreate and generate data in a target.

Each transaction from the source is written into the THL as an event, so within a single THL file there will be one or more events stored. For each event, we record information about the overall transaction, as well as then information about the transaction itself. That event can contain one or more statements, or rows, or both. Because we don’t want to get an ever increasing single file, the replicator will also divide up the THL into multiple files to tmake the data easier to manage.

We’ll get down into the details soon, until then, let’s start by looking at the basics of the THL, files and sequence numbers and how to …

The Tungsten Replicator is an extraordinarily powerful and flexible tool capable of moving vast volumes of data from source to target.

In this blog post we will discuss one specific aspect of the THL (Transaction History Log) – the METADATA unsafe_for_block_commit flag.

What do you mean, Unsafe?

In a recent customer support case, we were asked the meaning of the unsafe_for_block_commit flag. For example, list the event information for sequence number 3481394254:

[tungsten@tr2-mysql01 (sandbox) ~]$ thl list -seqno 3481394254 | more
SEQ# = 3481394254 / FRAG# = 0 (last frag)
- TIME = 2018-09-16 06:52:47.0
- EPOCH# = 3480364140
- EVENTID = mysql-bin.001068:0000000294739578;622252
- SOURCEID = tr2-mysql01.sandbox.yourdomain.com
- METADATA = [mysql_server_id=1;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true;service=brm;shard=shard_1736]
- TYPE = …