| Bug #70403 | No useful error message when memcached fails to start due to lack of memory | ||
|---|---|---|---|
| Submitted: | 23 Sep 2013 21:40 | Modified: | 17 Oct 2013 11:11 |
| Reporter: | Hartmut Holzgraefe | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Cluster: Memcached | Severity: | S3 (Non-critical) |
| Version: | MySQL Cluster 7.2.12, MySQL Cluster 7.2.13 | OS: | Any |
| Assigned to: | CPU Architecture: | Any | |
[25 Sep 2013 14:05]
MySQL Verification Team
Hello Hartmut, Thank you for the bug report. Verified as described. Thanks, Umesh
[10 Oct 2013 17:52]
John Duncan
Yes, the "Failed to initialize instance" message comes from engine_loader.c. I think the current state is this: InnoDB memcached and NDB memcached trees have diverged (mostly because InnoDB memcached runs as a plugin daemon inside mysqld rather than as an independent process). Couchbase is separately making changes to their own product based on Memcached 1.6. Mainline memcached people still feel that 1.4 offers better performance. And nobody has recently maintained the "common" 1.6 tree on github. I will try to improve a few of these startup-time error messages on the NDB side. (And hope to make some headway on the other issues, outside of this bug).
[17 Oct 2013 11:11]
Jon Stephens
Documented fix int he NDB 7.2.15 and 7.3.3 changelogs, as follows:
When attempting to start memcached with a cache_size larger than
that of the available memory and with preallocate=true failed,
the error message provided only a numeric code, and did not
indicate what the actual source of the error was.
Closed.
Description: Start memcached with a cache_size larger than the available memory and preallocate=true, e.g.: ./bin/memcached -E `pwd`/lib/ndb_engine.so -u mysql -e "cache_size=4096m;preallocate=true" Startup will fail with "Failed to initialize instance. Error code: 3" without any useful hint towards what "error code 3" is (it is actually the memcache specific ENGINE_NOMEM error): 23-Sep-2013 23:29:17 CEST NDB Memcache 5.5.30-ndb-7.2.12 started [NDB 7.2.12; MySQL 5.5.30] Contacting primary management server (localhost:1186) ... Connected to "localhost:1186" as node id 7. Retrieved 4 key prefixes for server role "default_role". The default behavior is that: GET uses NDB only SET uses NDB only DELETE uses NDB only. The 3 explicitly defined key prefixes are "b:" (demo_table_large), "mc:" () and "t:" (demo_table_tabs) Server started with 4 threads. Priming the pump ... Connected to "" as node id 8. Scheduler: using 2 connections to cluster 0 Scheduler: starting for 1 cluster; c0,f0,g1,t1 done [0.733 sec]. Failed to initialize instance. Error code: 3 Sometimes, but not always, this is also followed by a segmentation fault How to repeat: See above ... Suggested fix: Produce useful error message instead of just an (undocumented?) numeric error code. The error output is generated within the bundled memcache code so this is actually an upstream issue, but as development of the memcached 1.6 beta branch seems to have stalled an upstream fix doesn't seem likely any time soon? PS: as the InnoDB memcached implementation also relies on its own bundled memcached 1.6 source it is probably affected in a similar way ...