MySQL Bugs: #14636: Manual endian emulation - unneeded on some platforms

Bug #14636	Manual endian emulation - unneeded on some platforms
Submitted:	4 Nov 2005 12:30	Modified:	13 May 2010 16:04
Reporter:	Gunnar von Boehn	Email Updates:
Status:	Verified	Impact on me:	None
Category:	MySQL Server: InnoDB storage engine	Severity:	S4 (Feature request)
Version:	5.0.15	OS:	Any (any)
Assigned to:	Assigned Account	CPU Architecture:	Any

Description:
In some places manual endian emulation/convertion is done. 
For a number of architectures this is of course not needed.

One example is in extra/innochecksum.c

ulint mach_read_from_4(byte *b){
  return( ((ulint)(b[0]) << 24)
          + ((ulint)(b[1]) << 16)
          + ((ulint)(b[2]) << 8)
          + (ulint)(b[3])
          );
}

Architectures which support big endian and unaligned accesses
like for example the POWER and PowerPC could simply use
ulint=(unsigned long*)b;

How to repeat:
na

Suggested fix:
For better performance:
Defining one macro using just one memory access on architectures supporting it.

Gunnar,

yes, if we knew that a processor is using big-endian storage of integers and it is also insensitive to alignment errors, then we could store directly.

But since x86 and Sparc do not fulfill the above, there is little demand for an optimization like this. I will leave this to the 'Verified' state indefinitely.

Regards,

Heikki

Dear Heikki,

PowerPC use big-endian and can access unaligned 32bit words.

We build binaries for the following platforms
which would take advantage from non emulating. 
- Linux (IBM/Motorola/Freescale POWER/PowerPC)
- Mac OS X
- IBM AIX(IBM POWER)

I hope this info helps

Cheers
Gunnar

Gunnar,

Apple is moving to x86. That leaves AIX and Linux/PPC.

Regards,

Heikki

Dear Heikki,

The PowerPC/POWER architecture can savely access unaligned memory
and has native load/store commands which support both big-endian and
little-endian. So examples like the one mentioned here could be done in a single command.
And if innodb works somewhere with little-endian data then this data could
be directly accessed without the need for any convertion.

In short: on the PowerPC there is no need for any endian convertion at all.

To be honest I'm a bit confused by your answer.
Do you plan to optimize this or do you not care?
Please tell me so I know if it makes sense for me to continue to profile innodb.

Many thanks in advance
Kind regards

Gunnar

Hi,Heikki
  When a query needs to copy lost of data from InnoDB to MySQL, it cost much more to run the code bellow (in row0sel.c version 5.5)

                ptr = dest + len;

                for (;;) {
                        ptr--;
                        *ptr = *data;
                        if (ptr == dest) {
                                break;
                        }
                        data++;
                }

                

If InnoDB stores int data using the same format with MySQL, perhaps simple "memcpy" is enough, and more important, *quicker*.

Simplely modify few codes in row0mysql.c and row0sel.c can change it.
I wonder why InnoDB use the big-endian format?

I believe that the design decision for using the big-endian format is the ease of sorting and comparison. Comparisons may need to be executed more often than storage and retrieval. The big-endian format allows simple memcmp()-style sorting.

To make memcmp() work on signed integers, InnoDB even inverts the sign bit. For example, the 8-bit signed integer -128 would be represented as 0x80 in two's complement arithmetics. Because InnoDB flips the high-order bit, this becomes 0x00. Likewise, the 8-bit signed integer 127 (0x7f) would become 0xff. Thus, the bytes 0x00 through 0x7f would represent the numbers -128 through -1 and the bytes 0x80 through 0xff the numbers 0 through 127.

We could probably replace functions like mach_read_from_4() with library functions such as ntohl(), which would hopefully be implemented efficiently with what the platform has. For example, the Intel 80486 introduced the SWAB instruction. Because of unaligned access, some memcpy() might still be needed.