In this post, I provide an illustration of how to use Pentaho Data Integration (PDI) tool to set up external dictionaries in MySQL to support ClickHouse. Although I use MySQL in this example, you can use any PDI supported source.

ClickHouse

ClickHouse is an open-source column-oriented DBMS (columnar database management system) for online analytical processing. Source: wiki.

Pentaho Data Integration

Information from the Pentaho wiki: Pentaho Data Integration (PDI, also called Kettle) is the component of Pentaho responsible for the Extract, Transform and Load (ETL) processes. Though ETL tools are most frequently used in data warehouses environments, PDI can also be used for other purposes:

• Migrating data between …

This is the 5th blog in series of Maxscale blog, Below is the list of our previous blogs, Which provides deep insight for Maxscale and its use cases for different architectures.

• Scale with Maxscale part -1 (Intro & Galera Cluster)
• Scale with Maxscale part – 2 (Administration, Installation & operation)
• Scale with Maxscale part – 3 (Replication M-S)
• Scale with …

The schema changes in production can cause lock at time and makes the slave to lag. It is more tedious and troublesome with PXC ( Galera ) cluster which can be made smoother with Percona online schema change.

Image Courtesy : Photo by Andrew Ruiz on Unsplash

This is our second blog in the ProxySQL Series ( Blog I MySQL Replication Read-write Split up ). Will cover how to integrate ProxySQL with MHA to handle failover of Database servers.

We already have Master – Slave replication setup behind ProxySQL from previous blog [ProxySQL On MySQL Replication]

For this setup we have added one more node for MHA Manager , Which will keep eye on Master and Slave status.

• node5 (172.17.0.5) , MHA Manager

ProxySQL can be greatly configured with MHA for …

This presentation was made at LSPE event in Bangalore (India) held at Walmart labs on 10-03-2018. This presentation focuses how we have harnessed the power of Ansible at Mydbops.

In this post, We will learn how to handle online schema change if the table has triggers.

In PXC, an alter can be made directly ( TOI ) on tables with less than a 1G ( by default) , but on a 20GB or 200GB table we need some downtime to do ( RSU ).

Pt-osc is a good choice for Percona Cluster/Galera. By default percona toolkit’s pt-online-schema-change will create After “insert / update / delete” triggers for maintaining the sync between the shadow and the original table.

pt-online-schema-change process flow:

Check out the complete slides for effective MySQL administration here

If the tables has triggers already then pt-osc wont work well in …

This is part-4 of the Maxscale Blog series

1. Maxscale and Galera
2. Maxscale Basic Administration
3. Maxscale for Replication

Maxscale started supporting Amazon Aurora lately from its version 2.1 which comes with a BSL license, we are good until we use only 3 nodes, …

Introduction –

          Maxscale is an excellent Proxy from Mariadb Corporation, which providing the High Availability, Realtime RW split with replication Glaera cluster Amazon RDS Amazon Aurora, binlog streaming and many more advanced features, here in this blog we will discuss one such feature, 

In this blog post, i am going to share my recent activity with Maxscale. We had to help one of our client to archive only the DML ( CREATE & INSERT ) data into archive server from specific table.

Problem Statement –

Our client is having only one standalone ( Master ) setup and a Archive server , they need to archive one …

As Mydbops we are implementing Load Balancer using Maxscale or ProxySQL ( Our presentation ) for lot our client,  but these load balancers will become a SPOF (Single Point of failure) .  We have tried to explore services like HAProxy, Nginx, and Keepalived etc. Except Keepalived, all the services need to run on the standalone instance and  did not satisfy our exact need.

Keepalived does not requires any standalone instance, it can be deployed and configured with a minimal effort and provide the HA Solutions to the DB Infra. This approach not only fits for our DB setup, we can implement same …

Here's something that has puzzled me for several weeks.
Right after migrating  MySQL from 5.6 to 5.7, we started experiencing random xtrabackup failures on some, but not all, of our slaves.
The failures were only happening when taking an incremental backup, and it would always fail on the same table on each slave, with errors similar to the following:

171106 13:00:33 [01] Streaming ./gls/C_GLS_IDS_AUX.ibd
InnoDB: 262144 bytes should have been read. Only 114688 bytes read. Retrying for the remaining bytes.
InnoDB: 262144 bytes should have been read. Only 114688 bytes read. Retrying for the remaining bytes.
InnoDB: 262144 bytes should have been read. Only 114688 bytes read. Retrying for the remaining bytes.
InnoDB: 262144 bytes should have been read. Only 114688 bytes read. Retrying for the remaining bytes.
InnoDB: 262144 bytes should have been read. Only 114688 bytes read. Retrying for …