Bug #105199 ndbcluster: Tuple corruption detected, exiting

• Submitted: 13 Oct 2021 6:41
• Modified: 19 Oct 2021 7:38
• Reporter: Hendrik Woltersdorf
• Status: Verified
• Category: MySQL Cluster: Cluster (NDB) storage engine
• Severity: S2 (Serious)
• Version: 7.5.22
• OS: CentOS (7.8.2003)
• CPU Architecture: x86

[13 Oct 2021 6:41] Hendrik Woltersdorf

Description:
Yesterday one datanode of one of our clusters shut itself down due to:
Tuple corruption detected, checksum: 0x1e788b8c, header_bits: 0x4820002, checksum word: 0x87654321
Exiting.

How to repeat:
no idea

[13 Oct 2021 6:42] Hendrik Woltersdorf

logs etc, collected by ndb_error_reporter

Attachment: ndb_error_report_20211013082443.zip (application/x-zip-compressed, text), 1.59 MiB.

[13 Oct 2021 6:43] Hendrik Woltersdorf

fixed typo in synopsis

[13 Oct 2021 9:18] MySQL Verification Team

Hi,

This seems to be a hardware error. Are you using registered/ecc RAM on your data nodes?

Thanks

[14 Oct 2021 11:56] Hendrik Woltersdorf

The machine is using "Multi-bit ECC".

[14 Oct 2021 13:47] MySQL Verification Team

Hi,

Thanks for the update. Anything comes to mind that was different when the crash happened? Any new procedures / applications? System updates?

So far I don't see any clues in the logs but let me look bit more.

thanks

[14 Oct 2021 13:51] Hendrik Woltersdorf

There was nothing new. Business as usual.

[14 Oct 2021 13:52] Hendrik Woltersdorf

Just for the records: We are using (a few) disk tables, containing BLOBs.