MySQL Bugs: #30108: Query Browser incorrectly reporting out of memory errors

Bug #30108	Query Browser incorrectly reporting out of memory errors
Submitted:	27 Jul 2007 23:26	Modified:	15 Oct 2009 11:39
Reporter:	Jared Friedman	Email Updates:
Status:	Won't fix	Impact on me:	None
Category:	MySQL Query Browser	Severity:	S2 (Serious)
Version:	1.2.12	OS:	Windows
Assigned to:		CPU Architecture:	Any
Tags:	CHECKED

Description:
When using MySQL query browser, I sometimes get a dialog box which reads:

"The memory load of the system is extremly high.

In order to keep the system responsive record set retrieval has been stopped."

(sic) on the extremly, by the way.  I have 2GB of RAM on this client machine, 16GB on the server, I have virtually nothing else running, and I am executing a trivial query.  The problem occurs sporadically, but once it begins happening, the Query Browser is useless - it will complain on every query.  

Is there any way to disable this error message?

How to repeat:
Not clear - problem occurs sporadically

Suggested fix:
In the meantime, instructions for disabling the error message would be great.

Thank you for the bug report. Could you please describe exactly on which
Operational System you got that error message. I never saw it on Windows
and Linux with boxes with 512MB/2GB. Thanks in advance.

Hi Miguel,
  Thanks for the response.  I'm running MySQL Query Browser 1.2.10 on a Windows XP  machine with 2GB of RAM talking to a remote server.  There's more than 1GB of RAM free.  It's definitely a client-side problem.  Is there any other information I could provide that would be useful?  This bug is driving me crazy.

Thanks!
  Jared

Thank you for the feedback. Since I am not able to repeat could you please
provide a screen-shot of that error message for our GUI-Tool team to analyze.
Thanks in advance.

Error dialog

Attachment: MySQL1.gif (image/gif, text), 24.49 KiB.

Task list

Attachment: MySQL2.gif (image/gif, text), 85.76 KiB.

Performance tab

Attachment: mySQL3.gif (image/gif, text), 85.68 KiB.

Note: Terms like 'PhysicalMemory->Available' reference the Performance tab on Windows Task Manager

This happens on my system when the PhysicalMemory->Available falls to about 40,000KB.  It seems to me that there is an arbitrary check for this condition in the client program.  I don't know if this check is based on percentage of physical memory available or if it's set to an absolute number.  My system has 1GB of ram so if the threshold is based on a percentage it will be around 4%.  If you insist on making a check like this at least do these things:

1. Make the threshold based on something like this (this could easily be a single expression but I've expanded it for readability):
availableSwap = CommitCharge->Limit - CommitCharge->Total;
availableMemory = PhysicalMemory->Available + PhysicalMemory->SystemCache + availableSwap;
totalMemory = PhysicalMemory->Total + CommitCharge->Limit;
availablePercentage = usedMemory / totalMemory;
Basically meaning a threshold based on percentage of all available memory.  Technically you should not count the file system as used memory.  This is why I add it into the available memory.

2. Allow the user to modify the threshold and/or turn the check off altogether.  IMHO Determining whether you have enough resources to execute a task is better left to the host OS... let the OS tell your program 'no' :)

I have attached some screenshots for your reference.

For reference here is some relevant information about my system:
MySQL Query Browser version: 1.2.12
Physical Memory Total: 1039852KB (~1GB)
Page file limit: 3041488KB (~3GB)
OS: Windows XP Pro SP2

By the way bug #28930 is nearly identical to this one:
http://bugs.mysql.com/bug.php?id=28930

This happens to me and after reading the comments above I found that the query browser will also stop when the Physical Memory (K) drops to around 40000.  

My Virtual memory page settings are set to Initial size of 968 MB / Max 2536 MB with 968 MB currently allocated for all drives.

I can consistently repeat this error running queries multiple times in a row.. eg Hold down CTRL-ENTER to keep running a query.. you will see your memory usage go down the drain.  Closing email clients etc to free up more physical memory so that I can run a few more queries seems primitive..

Please fix this..  I run into this all the time when running Eclipse 3.2 and couple other development tools.

Thanks,
Jamie

In my original post was: "Technically you should not count the file system as used memory."

What I meant to write is: "Technically you should not count the file system cache as used memory."

Hi guys,
  Thanks much for the interest and assistance.  I agree with Starson - the check MySQL is doing does not work well at all, and it's not clear why MySQL should anticipate memory allocation failure.  Typically, you leave it to the OS to decide whether it has enough memory and return errors to the user only if a memory allocation actually fails.  So basically I'm advocating for possibility (2) in Starson's post.

  Starson, though, I respectfully disagree that this bug is nearly identical to bug #28930.  On my system, at least, I do not experience memory leaks with MySQL Query Browser.  Instead, I have a system which is running nearly empty including MySQL query browser where the application refuses to try a query because it thinks incorrectly that it won't be able to allocate enough memory.

  Do you guys agree that the check should just be disabled?

Yes, I think the check is unnecessary and should be either removed entirely.

I guess what I meant is not that this bug is nearly identical to bug #28930, but that it is related at the very least in allowing an easy way to reproduce this problem.

Please, try to repeat with a newer version, 1.2.12, and inform about the results.

Valeriy,
I realize that your last comment was aimed at Jared, but as my posts indicate I already confirmed the behavior in version 1.2.12.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

Query browser running on:
Windows 2000 Pro SP 4
256 mega of ram

Keeps giving me the above error when retrieving result sets.
I do realize that 256 megs of ram is not a lot, however, I receive this message when there are over 100 megs free. more than enough to produce my result set.

The memory check also does not appear very consistent.
Sometimes, I've just booted and right away I can not produce a result set while restarting the application (with no other changes) allows me to produce the result set.

While I know I've not provided much to help discover the cause, I am posting this comment to keep this bug alive.

PLEASE add a dialog or command-line option to either disable or configure the free memory check.

Hello,
  This bug has been quiet for some time now, but it hasn't been fixed.  In answer to Valeriy's earlier question, I upgraded to MySQL Query Browser 1.2.12 and I am experiencing exactly the same error message.  It is, frankly, driving me crazy which is why I am bothering to post here in the first place.

  I'd like to re-iterate my support for simply disabling the dialog completely.  Checking for a low memory condition is very difficult to do reliably across the multitude of environments that this software has to run on.  It is a task better left to the operating system, which will faithfully report to Query Browser if a requested memory allocation fails.  Trying to anticipate this will cause bugs without fixing any.

  What do you guys say?

I've also been experiencing this quite frequently. I'm running Windows XP on an Athlon64 3800+ with 1 GB RAM.

My resort has been to ssh to the server and running the query from there. But that's not really what one should be forced to do.

Error Message

Attachment: memory.PNG (image/png, text), 30.23 KiB.

Thank you for the bug report and feedback.

Experienced similar memory issues ranging from:

"The memory load of the system is extremly high.
In order to keep the system responsive record set retrieval has been stopped."

to 

rising memory usage > 1.4 GB used at one point (solely for Query Browser(QB)).

System is Vista Ultimate (32bit) on Satellite P200 w/2GB memory.

This behavior seems related to recently working with a newly created schema for handling SSUTA (streamlined sales tax) data which required insertion of tens of thousands of records. With that, QB frequently froze when manipulating the series of inserts in the query pane (finally I abandoned it and used notepad to change the inserts before opening and running in QB).

Since that time however, when working on simple queries (returning a handful of records) from this database QB begins to increase its memory use steadily over a session until it must be shutdown (or terminated via task manager). This is after having restarted the machine (not continuous running).

While the memory use seems tied to the volume of data being queried against, it also seems to rise without even touching that new schema (instead querying against a different schema).

The problem is annoying and expensive (time and lost work potential) given the need to restart the program. So if there's a work-around or fix it would be greatly appreciated!

QB version = 1.2.12
Queries run via QB query pane

I did shut down the database(XAMPP = mysql_stop.bat) and started it again. That increased the amount of physical memory available by 200KB(see the 40KB border mentioned). This is no real fix.. but perhaps will help to find a workaround. One would need a memory manager assuring some 40KB+ of memory is available.

http://www.analogx.com/contents/download/system/maxmem.htm

That small program ensures enough free physical memory. At least on a Windows system...

I'm having a similar problem.  I'm running on an 8core Windows XP-64bit box with 8gb of memory.  There's over 300M of memory free, but the query browser gives me the error message stating:

"The memory load of the system is extremely high.  In order to keep the system responsive record set retrieval has been stopped."

Please turn this "feature" of the query browser off!

I'm getting annoying message often on my Windows Vista x64. Looks like Vista shows available memory very differently from previous versions of Windows. I have 8GB RAM and 8GB swap, Vista doesn't include swap in available space size and trends to use all available memory, so task manager shows that used memory quickly reaches almost 100% and stays that way. When I close some programs, available bit increases, but not for long. So the Query Browser (I have 1.2.12) at some point stops working popping up the message "out of memory". The only solution is to reboot system because as I mentioned above closing programs doesn't help or help for a short time.

Please, please add a checkbox in options or something to turn this test off.  I get this message whenever my system starts to touch the page file (XP Pro SP2, Athlon 64 X2 2.2GHz, 1 GB RAM, Query Browser 1.2.12) which means with Eclipse, Firefox and Jetty running I get about 3 or 4 queries in before Query Browser essentially refuses to do what I say.  I know my result set is going to be ~50 rows, let me decide if I want to take the chance of my system become unresponsive.

Please.

Okay in my first text post I stated: "if the threshold is based on a
percentage it will be around 4%".  I figured I'd take a look at the source since this is taking so long for you guys to resolve.  It is based on %... anything less than 5% of free physical memory triggers it.  here you go:

Both snippets are from 'mysql-gui-tools-5.0r12\mysql-gui-common\source\windows\MySQLResultSet.pas'

lines 1468~1488
----------------------------------------------------------------
function progress_row_fetch(current_row_count: Longint; previous_row_count: Longint; result_set: PMYX_RESULTSET;
  user_data: Pointer):Integer;

var
  Thread: TExecuteQueryThread;
  S: WideString;

begin
  Thread := user_data;
  if GetMemoryLoad > 95  then
  begin
    Thread.CancelQuery;
    S := _('The memory load of the system is extremly high.');
    if current_row_count >= 100000 then
      S := S + ' ' + _('This is likely because of the current result being very large.') + ' ';
    S := S + #10#10#13#10 + _('In order to keep the system responsive record set retrieval has been stopped.');
    Thread.FErrorMessage := S;
    Thread.Synchronize(Thread.ShowErrorMessage);
    Result := 1;
  end
  else
----------------------------------------------------------------
and here's what it's calling...
lines 310~324:
----------------------------------------------------------------
procedure GlobalMemoryStatusEx(var lpBuffer: TMemoryStatusEx); stdcall; external 'Kernel32.dll' name 'GlobalMemoryStatusEx';

function GetMemoryLoad: Integer;

// Returns the current memory load of the system in % (0 .. 100).
var
  Status: TMemoryStatusEx;

begin
  ZeroMemory(@Status, SizeOf(Status));
  Status.dwLength := SizeOf(Status);
  GlobalMemoryStatusEx(Status);

  Result := Status.dwMemoryLoad;
end;
----------------------------------------------------------------

Anyways, it doesn't happen often enough for me to make it worth removing the if block and recompiling every time a new release is out, but I hope this helps anyone here that has the 'error' more frequently.

Maybe this will be fixed sooner now that the problem has been isolated in code?

P.S. There are many places where 'extremly' is used instead of 'extremely' in your code ;)

I would also like to vote for this very annoying issue. The typo makes searching online for it tricky too! Please please give us a target version.

I just downloaded latest build 1.2.15. The problem is still there. Now it's even worse - the query browser is crashes every time I try click "Edit table data" from MySQL Administrator, though it opens from start menu. My system is Vista x64 8GB RAM. The problem just killing me, my system has gigs of free RAM, but query browser don't let me open table of few lines. The only solution I have for now is to open some memory hungry application to push back some cache, then when I close the app Vista shows some memory free any query browser lets me run a query.

This issue is stopping me from using Query Browser.

Please can a fix for this please be prioritised.

As previous comments have suggested, this should be a fairly trivial fix - either drop the memory check and let the OS manage memory, or change the algorithm to calculate a reasonable limit

This needs re-verifying with actual version

Just tried with the latest version, 1.2.16.  I've upgraded to 2GB of RAM since my first run back in August of 2007.  Images of this run coming after this post.

Fortunately this is easy to reproduce.  I just start QB, enter the query:
select 1;
and hold ctrl+enter until QB eats enough memory.

Again 4% is the threshold which for my machine is 2088428 * .04 = 83537 and that's about where it gets triggered.

I'm no Pascal wizard but it seems to me the fix is trivial and here it is.  The latest source available for download right now is 1.2.14 so the line # might be a little off from 1.2.16.  Just change line 1471 in MySQLResultSet.pas from this:
  if GetMemoryLoad > 95  then
to this:
  if False then

p.s. can we get the source for 1.2.16?

Error dialog

Attachment: qbError.gif (image/gif, text), 20.59 KiB.

Showing version

Attachment: qbProperties.gif (image/gif, text), 47.91 KiB.

System information showing 81896KB free

Attachment: qbSysInfo.gif (image/gif, text), 47.08 KiB.

I am not able to reproduce this with MySQL Query Browser 1.2.17

I get the same behavior in 1.2.17 as I have in past versions.

I only have 1 GB Ram and can't repeat it. The difference from my system is that it is using service pack 3.

Please let me know if you will have this problem too by using SP 3.

This error still occurs for me on XP SP3 with version 1.2.17.

Microsoft Windows XP Professional
Version 2002
Service Pack 3

Athlon 64 X2 4200+
1GB RAM

yes i can confirm this problem occurs in XP SP3. 

i get this problem constantly while using a virtual machine which would use a fair chunk of memory.

SP3 here as well.

Susanne, to reproduce it try what I suggested in my last post... holding the keys down for at least a minute does it for me.  Your mileage may vary depending on your keyboard repeat rate and how much memory your system has.  If you open task manager to the performance tab you should notice it getting triggered when Physical Memory Available reaches ~40,000 because you have 1GB of RAM.  My threshold is ~80,000 because I have 2GB RAM.  If that doesn't work maybe you have some kind of memory manager running.  Try disabling it or running this on a fresh OS install... or maybe have a few colleagues try it out as well.  Another idea is that I always connect to a remote server.  I have not tried connecting to mysql on localhost via named pipe.  Perhaps those connections do not cause QB to eat as much memory on every query.  Just trying to help you reproduce.

For your reference my last post said:
start QB, enter the query:
select 1;
and hold ctrl+enter until QB eats enough memory.

Starson,

I did exactly what you once explained. I hold the key pressed around about half an hour ....

The hint with remote server is really good. Maybe that is the reason. I tested only with locale server.

I will give it a try.

Also instead of using the query
select 1;
it might be better to use one that provides a different result over time.  That way you can tell if you are actually querying the server or not by seeing if the result changes as you're holding down the keys.  If the result is static then you're not querying.  Better queries are:
select rand();
OR
select now();

I just did a fresh local install of mysql server and tried using a local connection in both TCP/IP mode and named pipe mode.  Both modes behaved the same as when I connect to a remote server.  I made sure I was really connecting using a named pipe by using Administrator to verify the connection did not show a port.

If you watch your Physical Memory->Available on the performance tab of task manager does it ever get close to dropping to 40000 while you continuously execute the query?  If not what's the lowest it gets to?

MySQL Server Essential 5.1.32 with named pipes enabled
MySQL Query Browser 1.2.17
MySQL Administrator 1.2.17
Windows XP SP3, up to date using windows update
2.0GB RAM

I have this issue on my machine (not such a problem now because I have upgraded my RAM) but I never had a problem connnecting to localhost, only remote servers. Also, I'm not sure that it's actually a memory use issue - I used to get it on the first query that I did and the memory usage of the process was reasonable. I think that the memory check may be flawed - even if it's not, I think it's not worth doing because I'm not convinced that the program should be dictating to the user what query they're allowed to run. Maybe it should be a warning at most?

Verified as described by using remote server.

Important here is that it only occurs when the server is remote.

I'm using a Windows 2003 x64 server and I'm using the query browser locally. 

The machine has 4GB of memory, approx. 2GB is used by System Cache, Commit charge is approx. 2GB, about 100M of physical memory is available. As stated earlier by another poster the system cache should be counted as usable memory.

When running the same query remotely I'm not experiencing any problems at all. Although this workaround is acceptable for me at the moment, the memory check performed by the Query Browser is annoying, not necessary and wrong. Please fix it by performing the correct check (taking the System Cache into account) and/or catch any memory problems when they occur.

I can confirm that this is *not* just a problem connecting to remote servers.  Using Query Browser 1.2.17 connecting to MySQL 5.0.67-community-nt on localhost under Windows XP SP3.

Please, just take this check out.  The only people who know of its existence are those who have been inconvenienced by it.

god fucking dammit, this bug is still there. fix it already! it's 2009!

Whilst I don't really think that last comment was helpful (!) it has been quite a long time and this bug has been marked as "Serious" with Risk and Effort as "Low" so it really should be fixed (or the test should be disabled) pretty quickly.

This bug has been annoying me for years and it's still there. Would somebody please disable this completely useless dialog box? 28% CPU with 134MB of physical RAM available does not constitute extremely high CPU usage.

I'm running Windows XP SP3 with 1GB of RAM on 1.4GHz Pentium III Sony VGN-S170B laptop.

Many thanks for writing a bug report. We are on the way to implement full functionality of MySQL Query Browser into MySQL Workbench. We won't fix this anymore.

More informations about MySQL Workbench you will find here:

http://dev.mysql.com/workbench/