MySQL Bugs: #50122: Configuration wizard cannot reconfigure instance, freezes and takes 100% CPU

Bug #50122	Configuration wizard cannot reconfigure instance, freezes and takes 100% CPU
Submitted:	6 Jan 2010 22:17	Modified:	31 Mar 2014 8:57
Reporter:	Abel Braaksma	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Config Wizard	Severity:	S1 (Critical)
Version:	5.1	OS:	Windows (Windows Server 2008 Web Edition R2 64 bit )
Assigned to:	Assigned Account	CPU Architecture:	Any
Tags:	configure, freeze, hang, innodb, myisam, reconfigure

Description:
After a default installation (using Microsoft Web Platform Installer) trying to reconfigure using the MySQL Server Instance Config Wizard fails on all accounts: it freezes ("not responding" in the caption bar) and eats 100% of CPU (separated over services.exe and MySQLnstanceConfig.exe).

It shows "Prepare configuration" and "Write configuration file" as successful. It stops on "Start service".

Killing MySQLInstanceConfig.exe is the only way out (and subsequently eases the stress on services.exe, which goes to 0% CPU). Trying to start the service manually does *not* solve the problem, it fails. The event viewer logged the following errors when using the wizard or when starting the service:

Faulting application name: mysqld.exe, version: 0.0.0.0, time stamp: 0x4af32fcc
Faulting module name: mysqld.exe, version: 0.0.0.0, time stamp: 0x4af32fcc
Exception code: 0xc0000005
Fault offset: 0x000000000025ebc4
Faulting process id: 0xa64
Faulting application start time: 0x01ca8efdc9d8440c
Faulting application path: C:\Program Files\MySQL\MySQL Server 5.1\bin\mysqld.exe
Faulting module path: C:\Program Files\MySQL\MySQL Server 5.1\bin\mysqld.exe
Report Id: 082bb482-faf1-11de-9bb1-020053dee29d

It is remarkable that when you select InnoDB (i.e., Transactional) and change the path, the process *does* create a new ibdata1 file in the new directory.

NOTE: there's one exception to the freezing/100% CPU: if you select Transactional + deselect "Enable TCP/IP" + deselect "Modify security settings". It says "failed to start service". The error in the event log is equal to the errors above. Strange fact: doing this twice (hit back and "Execute", no changes) then it freezes again.

How to repeat:
Logged in as Administrator, follow the following steps on a freshly installed Windows 2008 Web Server R2 64 bit (possibly any other Windows version, considering scattered reports on the internet):

1. Install MySQL 5.1 using MSI install or MS Web Platform Installer
2. Start MySQL Server Instance Config Wizard and choose "reconfigure instance"
3. Change anything (except see NOTE above)
4. Hit "execute" on the last step

Result: when the wizard tries to start the service, it stops responding and brings the CPU to 100%.

Suggested fix:
Use a separate thread that creates the service to prevent freeze or lockup. If it is not created within X seconds or so, give an information message to the user. An error (as apparently appears here) should be presented instead of causing 100% CPU and freeze-up.

Apparently some access violation occurs (0xc0000005), this shouldn't cost 100% CPU.

Alternatively, but rather a feature request: changing configuration should backup the previous configuration. When (re)starting fails, rollback to the previous working situation can be a huge feat.

Something that may help solve the issue: when I turn on Process Monitor from Sysinternals to try to find out what goes wrong, the "Start service" becomes read after hitting Execute and it says "The service cannot be started" as it should (while still not helpful, it is much better then having a frozen application).

ProcMon heavily slows down certain processes. I don't know the inner workings, but something tells me we're looking at a spin-lock situation in a thread-to-thread wait state or something.

Thank you for the bug report. Could you please open a command prompt running it as administrator, then go to the MySQL's install folder, verify if there is a file called my.ini otherwise search to find it a keep its path, move to the bin folder and issue:

mysqld --defaults-file="path_for_my_ini\my.ini" --standalone --console

check if there are problems to start the server as standalone and print here the probably error messages. Thanks in advance.

Having the same problem on a Windows 7 Ultimate with mySQL-5.1.42-win32.msi
Tried everything possible I could imagine, including all of the above.
Here a dump starting from the resulting errors following this command:
mysqld --defaults-file="path_for_my_ini\my.ini" --standalone --console

---------Start Dump-----------------------------
                                                      ;InnoDB: End of page dump
100111 20:33:28  InnoDB: Page checksum 1575996416, prior-to-4.0.14-form checksum
 1371122432
InnoDB: stored checksum 0, prior-to-4.0.14-form stored checksum 0
InnoDB: Page lsn 0 0, low 4 bytes of lsn at page end 0
InnoDB: Page number (if stored to page already) 0,
InnoDB: space id (if created with >= MySQL-4.1.1 and stored already) 0
InnoDB: Page may be a freshly allocated page
100111 20:33:28InnoDB: Error: trying to access a stray pointer 83B53FF8
InnoDB: buf pool start is at 03B44000, end at 06A44000
InnoDB: Probable reason is database corruption or memory
InnoDB: corruption. If this happens in an InnoDB database recovery, see
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: how to force recovery.
100111 20:33:28  InnoDB: Assertion failure in thread 3092 in file G:\mysql-5.1.4
2-winbuild\mysql-community-nt-5.1.42-build\storage\innobase\include\buf0buf.ic l
ine 264
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100111 20:33:28 - mysqld got exception 0xc0000005 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=26214400
read_buffer_size=65536
max_used_connections=0
max_threads=100
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 58230 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
005EC820    mysqld.exe!buf_frame_align()[buf0buf.ic:264]
005D207C    mysqld.exe!page_dir_slot_get_rec()[page0page.ic:503]
0061CFF6    mysqld.exe!page_cur_search_with_match()[page0cur.c:351]
005D834C    mysqld.exe!btr_cur_search_to_nth_level()[btr0cur.c:521]
00601DA0    mysqld.exe!btr_pcur_open()[btr0pcur.ic:494]
00602A76    mysqld.exe!btr_pcur_open_on_user_rec()[btr0pcur.c:550]
005F84FA    mysqld.exe!dict_load_indexes()[dict0load.c:610]
005F8867    mysqld.exe!dict_load_sys_table()[dict0load.c:1015]
0060DC98    mysqld.exe!dict_boot()[dict0boot.c:389]
005C0A3A    mysqld.exe!innobase_start_or_create_for_mysql()[srv0start.c:1521]
005AEAF8    mysqld.exe!innobase_init()[ha_innodb.cc:1979]
0047DEC8    mysqld.exe!ha_initialize_handlerton()[handler.cc:435]
00429103    mysqld.exe!plugin_initialize()[sql_plugin.cc:1014]
0042CCD4    mysqld.exe!plugin_init()[sql_plugin.cc:1238]
00417F7F    mysqld.exe!init_server_components()[mysqld.cc:3951]
00418631    mysqld.exe!win_main()[mysqld.cc:4420]
0041897D    mysqld.exe!mysql_service()[mysqld.cc:4598]
00418C10    mysqld.exe!main()[mysqld.cc:4784]
004011F6    mysqld.exe!__tmainCRTStartup()[crt0.c:327]
01961943
2D00646C
6665642D
746C7561
69662D73
2E3D656C
796D5C2E
696E692E
732D2D00
646E6174
6E6F6C61
2D2D0065
736E6F63
00656C6F    rtree_page_mbr()[rt_mbr.c:788]
73726573
676E696D
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

C:\Program Files\MySQL\MySQL Server 5.1\bin>
---------End Dump-------------------------------

I hope you guys can find the problem.

The issue itself is solved and it's (easily) repeatable on any system:

1. Install using the defaults
2. Use the Windows Config Wizard, select Transact and another location for the DB
3. Finish the wizard

Now, depending on some factors, the following happens:

a) a new file is created
b) the db log files are not moved along
c) on restart of the service, MySQL complains about this, but this is not clearly communicated to either Event Log or the Config Wizard
d) MySQL tries to build the DB, but fails and stops. The Wizard sometimes does, sometimes doesn't notice the stopping.

Alternatively, in particular if you did the move as above and then try again:

d) MySQL notices an unfinished build of the DB and tries to rebuild it.
e) It fails and dumps all contents to the console.

I'm under the impression that the last point (e), creates the 100% CPU usage. It's dumping many NULL and other characters and the service has no way to deal with that. No idea what exactly happens, but when running from the console, it already took a long time before the dump ended.

RESOLUTION

It is possible to solve this by hand, but that's not the idea of using the Wizard, which is supposed to take care of this for you (a simple move of the original files in the original location suffices):

I. Prior to using the wizard, go to the old location
II. Move or copy all files to the new location
III. If you ran the wizard and had this issue: remove the wrongly created DB in the new location by hand.
IV. Edit the my.ini and adjust the location of the DB LOG files
V. Run the wizard again.

CONCLUSION

While it is now clear what has happened, it is still clearly a bug and a very hard one to understand or troubleshoot as a user. You use the wizard to make your life easier and because you don't understand the more spartan by-hand interface. Either remove this functionality (moving of db location) from the wizard (if it's used, it's guaranteed to fail) or fix it such that it works. Fixing includes: changing both the DB Log path and the DB path (MySQL needs them in one location?) in my.ini and moving the files of the old location to the new one.

NOTE: this currently fails on any installation, even a completely new one!

Abel, unfortunately, I was not able to get the error following *exact* steps you posted...

My environment:
  Win7 Ultimate x64 in VBox.
  MySQL 5.1.42 x64 msi

The only thing I was careful about was the place I put ibdata in. I used \Users\MyAccount and all was well.

I did noticed (Not Responding) in wizard but it lasted for about 5 seconds. Wizard returned and finished correctly and completely.

I do not say there is a problem here but it should be repeatable on my box for me to do anything.

Same (correct) behavior observed on Win 2008SE x64, my primary (physical) box. Please check on Bug#49233.

Finally, there was a problem removing the service from the list (Bug#45050).

This crash is very prominently listed on Windows Error reporting site , there is about 4000 crashes our customers got so far

Wlad, if so, please try to get to the bottom of the problem. Assigning you.

The crash callstack from Wndows Error reporting data (5.1.41, 32 bit):
 
	mysqld.exe!buf_frame_align(unsigned char * ptr=0x824a3ff8)  Line 264 + 0x19 bytes	C
 	mysqld.exe!page_dir_slot_get_rec(unsigned char * slot=0x824a3ff8)  Line 503 + 0xb bytes	C
 	mysqld.exe!page_cur_search_with_match(unsigned char * page=0x024a0000, dict_index_struct * index=0x0208b080, dtuple_struct * tuple=0x0208d080, unsigned long mode=0x00000002, unsigned long * iup_matched_fields=0x0121e970, unsigned long * iup_matched_bytes=0x0121e974, unsigned long * ilow_matched_fields=0x0121e984, unsigned long * ilow_matched_bytes=0x0121e97c, page_cur_struct * cursor=0x0121ec1c)  Line 351	C
 	mysqld.exe!btr_cur_search_to_nth_level(dict_index_struct * index=0x0208b080, unsigned long level=0x00000000, dtuple_struct * tuple=0x0208d080, unsigned long mode=0x00000002, unsigned long latch_mode=0x00000001, btr_cur_struct * cursor=0x0121ec18, unsigned long has_search_latch=0x00000000, mtr_struct * mtr=0x0121ec88)  Line 521	C
 	mysqld.exe!btr_pcur_open(dict_index_struct * index=0x0208b080, dtuple_struct * tuple=0x0208d080, unsigned long mode=0x00000002, unsigned long latch_mode=0x00000001, btr_pcur_struct * cursor=0x0121ec18, mtr_struct * mtr=0x0121ec88)  Line 494 + 0x35 bytes	C
 	mysqld.exe!btr_pcur_open_on_user_rec(dict_index_struct * index=0x0208b080, dtuple_struct * tuple=0x0208d080, unsigned long mode=0x00000002, unsigned long latch_mode=0x00000001, btr_pcur_struct * cursor=0x0121ec18, mtr_struct * mtr=0x0121ec88)  Line 550 + 0x26 bytes	C
 	mysqld.exe!dict_load_indexes(dict_table_struct * table=0x02088880, mem_block_info_struct * heap=0x00000000)  Line 610 + 0x1b bytes	C
 	mysqld.exe!dict_load_sys_table(dict_table_struct * table=0x02088880)  Line 1015	C
 	mysqld.exe!dict_boot()  Line 389	C
 	mysqld.exe!innobase_start_or_create_for_mysql()  Line 1521	C
 	mysqld.exe!innobase_init(void * p=0x01519c10)  Line 1924	C++
 	mysqld.exe!ha_initialize_handlerton(st_plugin_int * plugin=0x01506dd8)  Line 435 + 0x13 bytes	C++
 	mysqld.exe!plugin_initialize(st_plugin_int * plugin=0x00000000)  Line 1014 + 0x3 bytes	C++
 	mysqld.exe!plugin_init(int * argc=0x009c3e00, char * * argv=0x00e15d90, int flags=0x00000000)  Line 1238 + 0x5 bytes	C++
 	mysqld.exe!init_server_components()  Line 3930 + 0x2d bytes	C++
 	mysqld.exe!win_main(int argc=0x00000002, char * * argv=0x00e11930)  Line 4399 + 0x5 bytes	C++
 	mysqld.exe!mysql_service(void * p=0x009c5438)  Line 4577	C++
>	mysqld.exe!_callthreadstart()  Line 293 + 0x6 bytes	C
 	mysqld.exe!_threadstart(void * ptd=)  Line 275 + 0x5 bytes	C

minidump

Attachment: minidump.7z (application/octet-stream, text), 369.73 KiB.

Verified with Windows Error reporting data (s. attached minidump)

Vladislav, Tonci,

Not sure if the system is causing this. While the error may appear on any system, it is possible that only Windows 2008 R2 64 bit is the environment where the 100% CPU came from.

While it was mentioned in my original report, this may be of (great) influence to how the Config Wizard behaves: I installed it using Microsoft Web Platform Installer (http://www.microsoft.com/web). Afaik, the PI only downloads and starts the MySQL default installer, but I may be wrong there. 

Meanwhile I'll try to test on some other systems as well to fine-tune the step-by-step instruction (can't currently do it on Win2008, that system went to Production the other night).

EDIT: I meant, I used the PI, and after installation I changed the defaults.

In case it matters: the wizard screen where you select the location of the MySQL DB files is always empty, even if my.ini contains correct and existing locations.

I've seen this moved a few times from Config Wizard to InnoDB. But while InnoDB reports the error and crashes, the reason it doing so is a corrupt and wrongly placed DB file as the result of using the wizard. If the wizard did:

- move the files correctly
- prevent the eternal loop (100% CPU)
- or removes this "feature"

everything would be fine. I suggest to stick it back to the category Config Wizard.

Duplicate of bug: http://bugs.mysql.com/bug.php?id=44416.

Bug: http://bugs.mysql.com/bug.php?id=44416 it was marked as duplicate of this one.

The same bug on a Windows 7 Enterprise x64 with mysql-5.1.43-winx64.msi

http://dpotter.net/Technical/2009/11/mysql-install-hangs/

This fixed it for me.

Hello.

I have the "same problem" in windows xp x64 sp2
The wizard gets stuck while starting the service.

But i got maybe an option.

*delete innodb folder and reinstall mysql
*run the setup wizard as you use to do
*UNCHECK install as a windows service
*install mysql as a service from a win shell
*set user and password from phpmyadmin or your favorite app

done, i hope it works for you.

Jordan Loehr fix worked for me too!

I've run into the same issue using Web Platform Installer on Windows Standard Server 2008 SP2. 

I downloaded the 64bit MSI package listed in the Web Platform Installer and got the same hang. 

Then following the steps outlined in the last post I installed it without installing it as a service. This worked.

Then I used mysqld --install to install it as a Windows Service. This worked, but I got File Not Found error when attempting to start the service.

Using sc.exe to query the Windows Service catalog, I see that BinPath looks incorrect.

On my machine, MySQL is installed at c:\program files\MySQL\MySQL Server 5.1 with the executables being in the \bin directory, but mysqld sets the bin path to "c:\Program Files\MySQL MySQL". This doesn't look like a valid path. 

Running the following allows me to start and stop the service:

sc config MySQL binPath= "c:\program files\mysql\mysql server 5.1\bin\mysqld.exe"

http://dpotter.net/Technical/2009/11/mysql-install-hangs/

worked like a charm

Bug #42860 was marked as duplicate of this one.

Folks, did anyone try latest, 5.1.alot (where alot is meanwhile == 48). The binary you try is meanwhile 1.5 years old, and there are 15 update releases since.  Chances are this is fixed, since I all reports about this bug I see reference 5.1.31 version or near it.

Ok, I seem to be able to reproduce it now. Not sure what InstallWizard does wrongs (and it can do  many things wrongly:), but I get the mentioned crash in innodb with the latest 5.1.48.

So the sequence that leads to the crash is following, I'm not sure if it can be simplified.

1) Install latest MySQL Server (currently 5.1.48) from community downloads.
 Use config wizard at the end of installation to create a database with default settings
3) Uninstall MySQL Server.
4) Reboot.
5) as administrator, issue "sc delete mysql" (ConfigWizard does not cleanup after itself on some odd reason)
6) Install latest MySQL Server (currently 5.1.48) from community downloads.
 Use config wizard at the end of installation to create a database and (IMPORTANT) change path to Innodb datafiles to be something else (I change drive to D:\ and use the proposed location D:\MySQL Datafiles).

Service registration will fail with the mentioned crash. No hang of ConfigWizard is observed (just service registration failed).

log file

Attachment: WIN2008.err (application/octet-stream, text), 103.19 KiB.

the full datadir + my.ini file

Attachment: data.7z (application/octet-stream, text), 140.24 KiB.

Maybe, attached iblogfiles can be useful

Try this guide, it helped me with MySQL 5.5.19 / 5.5.20.
At step 4, the service started already!
http://specifysoftware.org/content/i-get-error-when-trying-install-mysql-error0-cannot-cre...

Hi, this does not seem to work for me: http://specifysoftware.org/content/i-get-error-when-trying-install-mysql-error0-cannot-cre...

machine: win7 ultimate, my sql community server 5.5.22, wizard hangs after write config file and my.ini is created, also inno db file): 

This problem was not present in 5.5.20 and after applying update to 5.5.21 everything was ok. Then I updated to 5.5.22 and version would not change (new update was always found in web installer: 5.5.22). So I uninstalled the server and after that i cannot get it to run the service instance again...

Before reinstalling, i deleted all MySql related software from disk (workbench, .net connector, server) and rebooted the system. After that, i downloaded 5.5.22 server and attempted to install: same problem occurs

Tried installing 5.6.4m7-win64.msi, unchecked 'modify security' checkbox and instance config was successful. Issue is fixed for me.

This problem still exists in 5.5.27-winx64.msi.   I think to reproduce it you need to uninstall once, with your data location changed from the default.  Then if you don't follow all the steps in 
http://dpotter.net/Technical/2009/11/mysql-install-hangs/ (thanks jotun jotun) you will see the problem.  For me the issue was removing the c:\ProgramData\MySql folder.   Win7 Ultimate 64 bit.   I believe the actual bug is in the uninstall, which causes the Wizard to fall over second time through.

I was just installing on Windows Web Server 2008 R2 w/ SP1.  Had the same issue with 5.2.29-winx64. 

After following the steps for deleting the folders noted on dpotter.net which was linked earlier I looked around a bit more and noticed there was a MySQL folder in the C:\ProgramData folder.  Deleting this in addition to the other locations allowed me to successfully install.

Typo'd previously, I had meant 5.5.x not 5.2.x

Posted by developer:
 
Thank you for the bug report. We are closing this report now for two related reasons:

* MySQL Server 5.1 is an obsolete version of MySQL Server
* For more recent MySQL Server versions, the config wizard is no longer available, and the MySQL Installer for Windows is the recommended way to install MySQL Server on Windows