Bug #17123 fulltext plugin may not pass on allocated memory
Submitted: 4 Feb 2006 18:09 Modified: 1 Jun 2006 14:44
Reporter: Hartmut Holzgraefe Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: FULLTEXT search Severity:S2 (Serious)
Version:5.1 OS:
Assigned to: Sergei Golubchik CPU Architecture:Any

[4 Feb 2006 18:09] Hartmut Holzgraefe 
Description:
When creating a fulltext plugin that preprocesses data in a malloced buffer
the words to be put into the index are mangled as with_alloc is not set
for the custom parser (and is not settable from outside ./storage/myisam/ft_parser.c
either)

How to repeat:
static int dummy_parse(MYSQL_FTPARSER_PARAM *param)
{
  char *doc = malloc(param->length);
  int result;

  memcpy(doc, param->doc, param->length);
  result = param->mysql_parse(param->mysql_ftparam, doc, param->length);
  free(doc);   // works fine only when *not* freeing memory after call here
  return result;
}

Suggested fix:
either always set with_alloc to true when calling fulltext plugins 
or provide the needed structure definitions in plugin.h to make
this settable from within the plugins parse functuon using param->mysql_ftparam
[6 Feb 2006 14:14] Sergei Golubchik 
Not useless - an easy workaround is to store the pointer somewhere in the ftparser_state and do not free the memory until plugin->deinit() call.
[30 May 2006 16:10] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/7024
[31 May 2006 14:01] Sergei Golubchik 
Two changes worth documenting:

1. MYSQL_FTPARSER_PARAM structure got new field: flags.
   The only possible value for now is MYSQL_FTFLAGS_NEED_COPY.
   It means that mysql_add_word() needs to save a copy of the word, it cannot
   use a pointer to it, because the word is in the buffer that will be overwritten.

  The flag may be set or reset on any level - by MySQL before calling the parser
  plugin, by the parser plugin itself, or by mysql_parse() function.

2. mysql_parse() and mysql_add_word() now take MYSQL_FTPARSER_PARAM as
   the first argument, not MYSQL_FTPARSER_PARAM::mysql_ftparam as before.

As these changes are not backward compatible, the API version was changed.
[31 May 2006 18:35] Sergei Golubchik 
fixed in 5.1.12
[1 Jun 2006 14:44] Paul DuBois 
Noted in 5.1.12 changelog, and I've updated the
instructions on parser plugin writing to match.
[31 May 2007 1:21] Malvoj 
I tried this code in mysql 5.1.12

ptext = (char*)malloc(5);
strncpy(ptext, "king", 4);
param->mysql_parse(param, ptext, 4);

in fulltext parser and it still doesn't work. :(