MySQL Bugs: #71516: MEB backup run with concurrent DDL can cause problems on windows source server.

Bug #71516	MEB backup run with concurrent DDL can cause problems on windows source server.
Submitted:	29 Jan 2014 21:37	Modified:	9 Jan 2015 16:23
Reporter:	Shane Bester (Platinum Quality Contributor)	Email Updates:
Status:	Unsupported	Impact on me:	None
Category:	MySQL Enterprise Backup	Severity:	S2 (Serious)
Version:	3.9.0, 3.10.0	OS:	Windows
Assigned to:	Priya Jayakumar	CPU Architecture:	Any

Description:
Manual says a backup could be corrupted if DDL is run during a backup.
http://dev.mysql.com/doc/mysql-enterprise-backup/3.9/en/bugs.backup.html

Indeed, MEB warns of this:

" mysqlbackup: WARNING: Change in the database schema has caused the copied 
frm files to become inconsistent. 
Please retry during low peak time or try a manual copy of the frm files"

However, the documentation fails to mention potential problems on the *source server* you're taking a backup from.

One example I saw on 5.5 now:

Version: '5.5.37'  socket: ''  port: 3307  MySQL Community Server (GPL)
140129 23:20:13  InnoDB: Operating system error number 5 in a file operation.
InnoDB: The error means mysqld does not have the access rights to
InnoDB: the directory. It may also be you have created a subdirectory
InnoDB: of the same name as a data file.
InnoDB: File name .\test\t4.ibd
InnoDB: File operation call: 'create'.
..
140129 23:22:02  InnoDB: error: space object of table 'test/t4',
InnoDB: space id 3423 did not exist in memory. Retrying an open.
140129 23:22:02  InnoDB: Operating system error number 2 in a file operation.

How to repeat:
Run concurrent DML and DDL workload while taking a MEB backup.
Observe the backup warning you:

Suggested fix:
Put warnings in manual that concurrent DDL + MEB backup [whichever server versions], can cause serious problems on the server as well as the backup.

I will note that I couldn't see much problem on 5.7.4 and 5.6.15  5.5.35 server typically had the problems.  I don't know if this is a server or MEB or documentation issue.

What I did was repeatedly took MEB backups in a loop for many hours, while some random workload was running against server.

Still seen when using meb 3.10.0 to backup a 5.5.36 server who runs DDL + DML during the backup, on windows.  I guess the mode in which MEB opens the files (using CreateFile?) causes either too much or not enough locking?

Version: '5.5.36'  socket: ''  port: 3306  MySQL Community Server (GPL)
140219  6:24:45  InnoDB: Operating system error number 183 in a file operation.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/operating-system-error-codes.html
InnoDB: File name .\test\t4.ibd
InnoDB: File operation call: 'rename'.
140219  6:28:28  InnoDB: Operating system error number 183 in a file operation.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/operating-system-error-codes.html
InnoDB: File name .\test\t2.ibd
InnoDB: File operation call: 'rename'.