Bug #109335 Inconsistent UDF behavior with respect to `max_length`

• Submitted: 12 Dec 2022 1:17
• Modified: 12 Dec 2022 7:32
• Reporter: Trevor Gross
• Status: Verified
• Category: MySQL Server: User-defined functions ( UDF )
• Severity: S3 (Non-critical)
• Version: 8.0, 8.0.31, 5.7.40
• OS: Linux (mysql:8.0-debian)
• CPU Architecture: Any

[12 Dec 2022 1:17] Trevor Gross

Description:
Originally posted here: https://jira.mariadb.org/browse/MDEV-30200

From the UDF docs:

> MySQL passes a buffer to the xxx() function using the result parameter. This buffer is sufficiently long to hold 255 characters, which can be multibyte characters. The xxx() function can store the result in this buffer if it fits, in which case the return value should be a pointer to the buffer. If the function stores the result in a different buffer, it should return a pointer to that buffer.

If a `cfg.max_length` is set in `xxx_init()` that is less than 255, then the `result` buffer will have a length of 766 when `xxx()` is called (I am unsure why 766 and not `cfg.max_length`).

However, this behavior sometimes differs when `xxx` is called within another function or in some different contexts, e.g. `set @abc = (select xxx())`. For some reason, this pattern makes MySQL pass a `length` of 0 to `xxx()`, rather than the expected `max_length` or 755.

How to repeat:
Code:

```
bool udf_buf_bug_init(UDF_INIT * initid, UDF_ARGS * args, char * message) {
    initid->max_length = 100;
    fprintf(stderr, "UDF: set max length to 100\n");
    return false;
}
 
char * udf_buf_bug(UDF_INIT * initid, UDF_ARGS * args, char * result,
    unsigned long * length, unsigned char * is_null, unsigned char * error)
{
    fprintf(stderr, "UDF: got result buffer of length %ld\n", *length);
    *length = 0;
    return 0;
}
```

```
select udf_buf_bug();
set @xyz = (select udf_buf_bug());
```

Output:

```
UDF: set max length to 100
UDF: got result buffer of length 766
UDF: set max length to 100
UDF: got result buffer of length 0
```

Repo: https://github.com/tgross35/udf-buf-bug

Suggested fix:
Ensure a buffer of length 755 or 255 or max_length is passed in all contexts

[12 Dec 2022 2:29] Trevor Gross

It seems like the buffer is actually a longer length - I can disregard the initial length, write the buffer, and get the correct result. So it's just erroneously reporting zero length when that doesn't seem to be true.

I think just none of the examples check the provided length in their `xxx` function (metaphon or is_const) so it's easy to miss

[12 Dec 2022 7:32] MySQL Verification Team

Hello Trevor,

Thank you for the report and feedback.

regards,
Umesh