Bug #52154 New does not get searched in Boolean mode Full text search
Submitted: 17 Mar 2010 18:39 Modified: 17 Mar 2010 20:30
Reporter: Naresh Bhagtani Email Updates:
Status: Not a Bug Impact on me:
None 
Category:MySQL Server: FULLTEXT search Severity:S2 (Serious)
Version:5.1 OS:Any
Assigned to: CPU Architecture:Any
Tags: boolean mode, reserved word

[17 Mar 2010 18:39] Naresh Bhagtani
Description:
Hello

I found out that MySQL FULL text search in Boolean Mode, does not count "new" as a word. I simply can't seem to get a result if my search term is "new". I tried this query for a database of properties in New York/ New Jersey

SELECT property_address, 
MATCH (property_address) 
AGAINST ('new*' IN BOOLEAN MODE) 
as score
FROM properties_master WHERE
MATCH (property_address) 
AGAINST ('new*' 
IN BOOLEAN MODE)
ORDER BY score DESC

There were absolutely no results. Whereas atleast 300 properties have the word New York in their address field. Then I tried something slightly different to prove my point.

SELECT property_address, 
MATCH (property_address) 
AGAINST ('new* york*' IN BOOLEAN MODE) 
as score
FROM properties_master WHERE
MATCH (property_address) 
AGAINST ('new* york*' 
IN BOOLEAN MODE)
ORDER BY score DESC

This time I got results. But when I checked the score, each record had a score of just 1. The boolean search clocked only York as the matching search term.

I wonder if there are any other such words that don't get searched? Has anyone noticed this? It seems "new" may be a reserved word! Any ideas how to get around this?

Thanks
Naresh

How to repeat:
I tried this query for a database of properties in New York/ New Jersey

SELECT property_address, 
MATCH (property_address) 
AGAINST ('new*' IN BOOLEAN MODE) 
as score
FROM properties_master WHERE
MATCH (property_address) 
AGAINST ('new*' 
IN BOOLEAN MODE)
ORDER BY score DESC

There were absolutely no results. Whereas atleast 300 properties have the word New York in their address field. Then I tried something slightly different to prove my point.

SELECT property_address, 
MATCH (property_address) 
AGAINST ('new* york*' IN BOOLEAN MODE) 
as score
FROM properties_master WHERE
MATCH (property_address) 
AGAINST ('new* york*' 
IN BOOLEAN MODE)
ORDER BY score DESC

This time I got results. But when I checked the score, each record had a score of just 1. The boolean search clocked only York as the matching search term.

Suggested fix:
There doesn't seem to be a documented work-around to this
[17 Mar 2010 20:30] MySQL Verification Team
Thank you for the bug report. Please read about stopwords:

http://dev.mysql.com/doc/refman/5.1/en/fulltext-boolean.html

"If the phrase contains no words that are in the index, the result is empty. For example, if all words are either stopwords or shorter than the minimum length of indexed words, the result is empty."

http://dev.mysql.com/doc/refman/5.1/en/fulltext-stopwords.html