In this blog, I will discuss about warnings that
users might face if they are trying to purge active/in_use
binary log which are introduced in MySQL-5.6.12/MySQL-5.7.2
releases. Before jumping directly to that, I will be
covering few basic things about PURGE BINARY LOGS command, about
safe way to purge the binary logs on a replication forum. If you
are good at basics on "PURGE BINARY LOGS" , you can choose to
directly move to the section (at the end of the blog) where I was
talking about these new warnings.
Introduction:
==========
A binary log file contains “events” that describe database
changes occurred on a MySQL Server (typically
called Master) that enabled binary logging using --log-bin.
On a Master server, the files that are related to binary logging
are binary log files (files with binlog_base_name.6_digit
serial_number) and a binary log index file (file with …
Partitioning The advantage of partitioning is that we can drop /
truncate the partitions which is real quick and done in seconds.
It does lock the table (metadata lock) during this operation but
only for few seconds. That duration depends on the number of
pages in use in the buffer pool. With MySQL 5.6 it is possible to
exchange the partitions , which helps in moving archived data to
another table with similar structure.
Fully Automated archival / purge
Consider a fast growing table which is partitioned to help the
queries and help quick archival/purge. The queries if specify the
partitioning filter can limit the search to relevant partitions
and can considerably reduce the query time. Purging old data to
reclaim the storage space can be readily achieved by truncating
the old partitions. Archival has multiple techniques but with
MySQL 5.6 exchanging the partitions make archival a real …
Purpose
What does purge exactly do and why is it needed? If you have ever wondered then read on. It is really a type of garbage collector. When a user issues a DML like “DELETE FROM t WHERE c = 1;”, InnoDB doesn’t remove the matching record. This is what happens under the hood:
- It marks the record as deleted by setting a bit in the control bits of the record.
- Stores the before image of the modified columns to the UNDO log
- Updates the system columns DB_TRX_ID and DB_ROLL_PTR in the clustered index record. DB_TRX_ID identifies the transaction that made the last change, and DB_ROLL_PTR points to the new UNDO log record. This UNDO log record contains the old values of DB_TRX_ID and DB_ROLL_PTR, possibly pointing to an older transaction and undo log entry.
From …
[Read more]
Being a MySQL DBA , one faces a common issue in replication
environment -> Disk space issue on master, since the number of
binary logs have increased. Now, one of the solution to this
would be using expire_logs_days parameter in your mysql
config file. But what if, the slave is lagging by few
hours or if the slave is broken since few days and the binary
logs are removed due to the parameter set. Whenever the salve
comes up, it will go bonkers, knowing that the binary log where
it last stopped no more exists.
I faced this
issue a couple of times until I decided to automate it using a
script. Herewith I am attaching the URL to my python script which
can run regularly in cron. Features :
- Checks the slaves connected to the master (I have limit it to 3 for now.)
- Checks the last binary log file which is being used by the slave.
- …
But users are human, and an unpredictable amount of unplanned
events can happen everywhere. When I was consulting, the above
cases were quite common.
Before MySQL 5.1, the only method to clean up the process list
was by hand, or using a cron job to do it from time to
time.
MySQL 5.1 …