Bug #22879 Server Crashed down, i don't find why
Submitted: 2 Oct 2006 7:57 Modified: 23 Dec 2006 17:24
Reporter: Thomas Goik Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Server Severity:S2 (Serious)
Version:Ver 4.1.11-Debian_4sarge7-log OS:Linux (Linux 2.4.22 #3 SMP )
Assigned to: CPU Architecture:Any

[2 Oct 2006 7:57] Thomas Goik
Description:
mysqld got signal 11;
....
key_buffer_size=824180736
read_buffer_size=4190208
max_used_connections=68
max_connections=100
threads_connected=17
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 1623663 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
[INFO]
This means that mysql uses max of 1,6 GB RAM, this is not correct, it should use up to 2,4 GB, i gues i have to increas some values

thd=0x7a9e3580
...
Cannot determine thread, fp=0xbcdf91f8, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x818935f
0x4005b825
0x84bb5ed
0x84bc87c
0x84bc217
0x84bb70d
0x849e5da
0x849f355
0x82121e5
0x821269d
0x8212934
0x81dd5df
0x81ce6c3
0x81c5f6f
0x81c68c2
0x81c3252
0x819f6f0
0x81a2bdc
0x819c0cf
0x819bce6
0x819b448
0x40055e51
0x402a08aa
New value of fp=(nil) failed sanity check, terminating stack trace!

This I found in my error logs but i can not figure out how this came up, i did the stacktrace and got this:
0x818935f _ZNK11Gis_polygon13exterior_ringEP6String + 175
0x4005b825 _end + 934815841
0x84bb5ed _ufc_foobar + 10605
0x84bc87c _ufc_foobar + 15356
0x84bc217 _ufc_foobar + 13719
0x84bb70d _ufc_foobar + 10893
0x849e5da __pthread_handles + 153978
0x849f355 __pthread_handles + 157429
0x82121e5 btr_search_move_or_delete_hash_entries + 21
0x821269d btr_search_move_or_delete_hash_entries + 1229
0x8212934 btr_search_move_or_delete_hash_entries + 1892
0x81dd5df row_upd_build_sec_rec_difference_binary + 703
0x81ce6c3 row_rename_table_for_mysql + 4851
0x81c5f6f row_ins_cascade_n_ancestors + 127
0x81c68c2 row_ins_dupl_error_with_rec + 18
0x81c3252 row_ins_clust_index_entry_by_modify + 194
0x819f6f0 dict_index_add_to_cache + 640
0x81a2bdc dict_print_info_on_foreign_key_in_create_format + 908
0x819c0cf dict_create_index_tree_step + 623
0x819bce6 dict_create_sys_fields_tuple + 230
0x819b448 dict_create_sys_columns_tuple + 680
0x40055e51 _end + 934792845
0x402a08aa _end + 937195750
s-esl4:/var/log/mysql# gunzip < /usr/share/doc/mysql-server-4.1/mysqld.sym.gz > /tmp/mysqld.sym
You have new mail in /var/mail/root
s-esl4:/var/log/mysql# resolve_stack_dump -s /tmp/mysqld.sym -n ./mysql.stack
0x818935f handle_segfault + 703
0x4005b825 _end + 933038613
0x84bb5ed free_block + 173
0x84bc87c flush_cached_blocks + 380
0x84bc217 flush_key_blocks_int + 263
0x84bb70d flush_key_blocks + 61
0x849e5da flush_blocks + 42
0x849f355 mi_repair_by_sort + 565
0x82121e5 _ZN9ha_myisam6repairEP3THDR17st_mi_check_paramb + 1749
0x821269d _ZN9ha_myisam14enable_indexesEj + 285
0x8212934 _ZN9ha_myisam15end_bulk_insertEv + 100
0x81dd5df _ZN13select_insert8send_eofEv + 31
0x81ce6c3 _Z9do_selectP4JOINP4ListI4ItemEP8st_tableP9Procedure + 579
0x81c5f6f _ZN4JOIN4execEv + 3759
0x81c68c2 _Z12mysql_selectP3THDPPP4ItemP13st_table_listjR4ListIS1_ES2_jP8st_orderSB_S2_SB_mP13select_resultP18st_select_lex_unitP13st_sel + 178
0x81c3252 _Z13handle_selectP3THDP6st_lexP13select_result + 242
0x819f6f0 _Z21mysql_execute_commandP3THD + 9552
0x81a2bdc _Z11mysql_parseP3THDPcj + 204
0x819c0cf _Z16dispatch_command19enum_server_commandP3THDPcj + 927
0x819bce6 _Z10do_commandP3THD + 134
0x819b448 handle_one_connection + 856
0x40055e51 _end + 933015617
0x402a08aa _end + 935418522

However, I do not know what this means or how to interpret it.

I using the BIN-LOG, slow-log, error-log, my system is serving in the main dbase around 700- 800 tables and up to 6 GB data. The server runs mainly mysql, a little apache to serve some stats and this is all.

The mysqld uses the cache function and ft_word is change to 3 from 4 

That all, if there is any needing of further information, please contact me

Best regards 
Thomas

How to repeat:
none

Suggested fix:
none
[2 Oct 2006 8:06] Valeriy Kravchuk
Thank you for a problem report. Please, send your my.cnf content and, if possible, probplematic query that crashed your server. How much RAM do you have?
[2 Oct 2006 8:40] Thomas Goik
>Thank you for a problem report. Please, send your my.cnf content and, if
>possible, probplematic query that crashed your server. How much RAM do
>you have?

I did'n find the query who crashed the server down, the RAM question is:
3 GB

I send the my.cnf file at the file register here!
[2 Oct 2006 8:46] Valeriy Kravchuk
I still do not see any my.cnf uploaded/sent. I am almost sure that we have out of memory issue bacause of improper configuration, but let me double check...
[2 Oct 2006 8:58] Thomas Goik
u see it now?

Thanks?
[2 Oct 2006 9:12] MySQL Verification Team
hi thomas, did you "set global key_buffer_size=.." while server was running? Do you have any alter table statements running at same time?
[2 Oct 2006 9:29] Thomas Goik
hi Shane;

i didn't set anything meanwhile the server was running.
Yes the system does alter tables on temp search tables, but this since a long time ago and never problems.

what says Valeriy: out of memory 
could be, this day i had more client requests

Ciao
[2 Oct 2006 12:30] Heikki Tuuri
Hi!
The InnoDB stack trace above is nonsensical. Are you sure that you used the right mysqld.sym to resolve the stack trace?
Regards,
Heikki
[15 Oct 2006 18:32] Thomas Goik
Hi, today i got another crash down by the server
Here is the error code with the stack trace:

ey_buffer_size=824180736
read_buffer_size=4190208
max_used_connections=101
max_connections=100
threads_connected=21
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 1623663 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x8625ea38
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0xbd1ff368, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x818935f
0x4005b825
0x8184e5f
0x8184993
0x81bacda
0x819be7f
0x819bce6
0x819b448
0x40055e51
0x402a08aa
New value of fp=(nil) failed sanity check, terminating stack trace!
...
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x7f900018  is invalid pointer
thd->thread_id=1562976

s-esl4:/tmp# resolve_stack_dump -s /tmp/mysqld.sym -n mysqld.stack
0x818935f _ZNK11Gis_polygon13exterior_ringEP6String + 175
0x4005b825 _end + 934815841
0x8184e5f mysql_free_result + 159
0x8184993 mysql_fetch_row + 179
0x81bacda ibuf_build_entry_from_ibuf_rec + 490
0x819be7f dict_create_index_tree_step + 31
0x819bce6 dict_create_sys_fields_tuple + 230
0x819b448 dict_create_sys_columns_tuple + 680
0x40055e51 _end + 934792845
0x402a08aa _end + 937195750

This is realy wired because the server is runnig for a long time with no errors
[23 Nov 2006 17:24] Valeriy Kravchuk
This is either out of memory issue, or, based on initial stack trace, a duplicate of bug #17332. Please, check with the patch (or new version with a patch) for bug #17332 (not fixed yet!), and inform about the results.
[24 Dec 2006 0:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".