As Mydbops we have consulted many large scale MySQL deployments. This presentation is about one of our customer who is one of the largest retailer in North America. This is about their data migration to InnoDB Cluster ( MySQL ) from an enterprise database.
Please join Percona’s Chief Evangelist Colin Charles on Thursday, June 28th, 2018, as he presents Securing Database Servers From External attacks at 7:00 AM PDT (UTC-7) / 10:00 AM EDT (UTC-4).
A critical piece of your infrastructure is the database tier, yet people don’t pay enough attention to it judging by how many are bitten via poorly chosen defaults, or just a lack understanding of running a secure database tier. In this talk, I’ll focus on MySQL/MariaDB, PostgreSQL, and MongoDB, and cover external authentication, auditing, encryption, SSL, firewalls, replication, and more gems from over a decade of consulting in this space from Percona’s 4,000+ …
[Read more]I’ve been posting quite a bit about Docker as I’ve been working with it a lot as of late. I thought I would share something interesting I discovered a couple weeks ago while working on setting up a Docker container-based lab environment for training purposes here at Pythian, specifically when attempting to install the MySQL-python package inside the container.
I know what you’re thinking: why is he trying to install a package in a Docker container? Doesn’t that go against the “Docker run and forget about it” philosophy? Sure, but in this case, I’m looking to add orchestration via ansible, which I don’t think is completely out of the question in order to coordinate your Docker containers for something like replication. This requires using the …
[Read more]Nicolas Tobias has written an awesome guide to setting up replication from MariaDB to Hadoop/HDFS using Tungsten Replicator, in Spanish! He’s planning more of these so if you like what you see, please let him know!
Semana santa y yo con nuevas batallas que contar.
Me hayaba yo en el trabajo, pensando en que iba a invertir la
calma que acompa;a a los dias de vacaciones que libremente
podemos elegir trabajar y pense: No seria bueno terminar esa
sincronizacion entre los servidores de mariaDB y HIVE?
Ya habia buscado algo de info al respecto en Enero hasta tenia una PoC montada con unas VM que volvi a encender, pero estaba todo podrido: no arrancaba, no funcionba ni siquiera me acordaba como lo habia hecho y el history de la shell er un galimatias. Decidi que si lo rehacia todo desde cero iba a poder dejarlo escrito en un playbook y ademas, aprenderlo y automatizarlo hasta el limite de poder desplegar de forma automatica on …
[Read more]MySQL is the most widely used Relational Database Management System in the open source world. MySQL stepped into the NoSQL world by introducing the JSON Data Type in MySQL 5.7 release. In this blog post I am going to explain one of the major advantage of optimisation made in JSON Replication .
This was done from the MySQL 8.0.3 release.
What happened before 8.0.3 ?
Before MySQL 8.0.3, Whatever changes (Updates) made in JSON document, it will log as a full document in binary log & replicate the same into slave. The JSON data is stored as a blob internally. Below is an example of how it is logged as full document in binary log ?
Example –
Server version - 5.7.22-log MySQL Community Server (GPL) My Binlog …[Read more]
Recently I worked on one customer issue that I would describe as “slave performance too slow”. During a quick analysis, I’ve found that the replication slave SQL thread cannot keep up while processing row-based events from the master’s binary log.
For example:
mysql> SHOW SLAVE STATUSG
*************************** 1. row ***************************
...
Master_Log_File: binlog.0000185
Read_Master_Log_Pos: 86698585
...
Relay_Master_Log_File: binlog.0000185
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
...
Exec_Master_Log_Pos: 380
Relay_Log_Space: 85699128
...
Master_UUID: …[Read more]
This blog post describes how to keep sensitive data secure on slave servers in a MySQL async replication setup.
Almost every web application has a sensitive data: passwords,
SNN, credit cards, emails, etc. Splitting the database to
secure and “public” parts allows for restricting user and
application parts access to sensitive data.
Field encryption
This is based on MySQL encryption functions or on client-side encryption when the authorized user knows a secret, but encrypted data is distributed to all slaves.
MySQL 8.0 became Generally Available (GA) on April 19th, a great moment for us working on MySQL at Oracle. It is now a “fully grown adult” packed with new features, and improvements to existing features, as described here.
This blog post focuses on the impact of replication performance improvements that went into MySQL 8.0.…
In this blog post on Percona Live 2018 sessions, we’ll talk with Shuhoa Wu, Software Developer for Shopify, Inc. about how Ghostferry is the Swiss Army knife of live data migrations.
Existing tools like mysqldump and replication cannot migrate data between GTID-enabled MySQL and non-GTID-enabled MySQL – a common configuration across multiple cloud providers that cannot be changed. These tools are also cumbersome to operate and error-prone, thus requiring a DBA’s attention for each data migration. Shopify’s team introduced a tool that allows for easy migration of data between MySQL databases with constant downtime on the order of seconds.
Inspired by gh-ost, their tool is named Ghostferry and allows application developers at Shopify to migrate data without assistance …
[Read more]MySQL 8.0.2 introduced a small yet powerful per transaction metadata information containing the transaction length in binary logs.
MySQL binary logs are being used for many other things than MySQL replication or backup/recovery: replicate to Hadoop; replicate to other databases, such as Oracle; capture data change (CDC) and extract-transform-load (ETL); record change notification for cache invalidation; change tracking for differential backups; etc.…
