| Bug #69935 | ndbmemcached built with bundled memcached but not with bundled libevent fails | ||
|---|---|---|---|
| Submitted: | 6 Aug 2013 8:09 | Modified: | 26 Sep 2014 21:54 |
| Reporter: | Hartmut Holzgraefe | Email Updates: | |
| Status: | Verified | Impact on me: | |
| Category: | MySQL Cluster: Memcached | Severity: | S3 (Non-critical) |
| Version: | MySQL Cluster 7.3.0, 7.3.1, 7.3.2, 7.4.1 | OS: | Linux (Ubuntu 13.04) |
| Assigned to: | CPU Architecture: | Any | |
[18 Sep 2013 8:57]
MySQL Verification Team
Hello Hartmut, Thank you for the bug report. Verified as described. Thanks, Umesh
[18 Sep 2013 8:58]
MySQL Verification Team
// Build steps
sudo cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/home/ushastry/mybuild/mysql-cluster-com-7.3.2 -DWITH_NDBCLUSTER_STORAGE_ENGINE=1 -DWITH_BUNDLED_MEMCACHED=1 -DWITH_BUNDLED_LIBEVENT=0 -DWITH_NDB_JAVA=OFF -DENABLE_DOWNLOADS=1
sudo make
sudo make install
// Started cluster ( 1MGM, 2 Storage and 1 active mysqld api and couple of vacant api slots)
// Started memchaced
ushastry@ushastry:~/mybuild/mysql-cluster-com-7.3.2$ bin/memcached -E lib/ndb_engine.so
18-Sep-2013 13:57:23 IST NDB Memcache 5.6.11-ndb-7.3.2 started [NDB 7.3.2; MySQL 5.6.11]
Contacting primary management server (localhost:1186) ...
Connected to "localhost:1186" as node id 5.
Retrieved 4 key prefixes for server role "default_role".
The default behavior is that:
GET uses NDB only
SET uses NDB only
DELETE uses NDB only.
The 3 explicitly defined key prefixes are "b:" (demo_table_large), "mc:" () and "t:" (demo_table_tabs)
Connected to "127.0.0.1" as node id 6.
Server started with 4 threads.
Priming the pump ...
Connected to "127.0.0.1" as node id 7.
Scheduler: using 2 connections to cluster 0
Scheduler: starting for 1 cluster; c0,f0,g1,t1
done [0.772 sec].
Failed to notify thread: Socket operation on non-socket
Failed to notify thread: Socket operation on non-socket
Failed to notify thread: Socket operation on non-socket
Failed to notify thread: Socket operation on non-socket
Failed to notify thread: Socket operation on non-socket
Failed to notify thread: Socket operation on non-socket
Failed to notify thread: Socket operation on non-socket
Failed to notify thread: Socket operation on non-socket
memcached: /home/ushastry/mybuild/mysql-cluster-com-7.3.2/storage/ndb/memcache/extra/memcached/daemon/memcached.c:4968: unregister_event: Assertion `c->registered_in_libevent' failed.
Aborted (core dumped)
Only once noticed below warnings, but later it just aborts with assertion..
[warn] event_del: event has no event_base set.
[...]
// No matter whether I use telnet 127.0.0.1 11211, or libmemchached clients - clients just hangs but the memcached aborts with ..
ushastry@ushastry:~/mybuild/mysql-cluster-com-7.3.2$ uname -an
Linux ushastry 3.5.0-23-generic #35~precise1-Ubuntu SMP Fri Jan 25 17:13:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
MySQL Cluster version 7.3.2
libevent-2.0.21
libmemcached-0.44
memcached - whatever bundled with MySQL Cluster version 7.3.2
PS: Just to inform that on OL with default libevent, I don't see this issue..
[18 Sep 2013 9:00]
MySQL Verification Team
Stacktrace etc details..
Attachment: 69935_stacktrace.txt (text/plain), 32.54 KiB.
[26 Sep 2014 21:54]
Hartmut Holzgraefe
Still reproducible with 7.4.1
Description: Cluster built with -DWITH_BUNDLED_MEMCACHED=1 -DWITH_BUNDLED_LIBEVENT=0 Building succeeds, built memcached binary can be started with bin/memcached -E lib/ndb_engine.so but the memcached service is not functional Startup messages look like this: mysql-ndb-7.3.1 > bin/memcached -E lib/ndb_engine.so -u root 06-Aug-2013 09:56:25 CEST NDB Memcache 5.6.10-ndb-7.3.1 started [NDB 7.3.1; MySQL 5.6.10] Contacting primary management server (localhost:1186) ... Connected to "localhost:1186" as node id 7. Retrieved 4 key prefixes for server role "default_role". The default behavior is that: GET uses NDB only SET uses NDB only DELETE uses NDB only. The 3 explicitly defined key prefixes are "b:" (demo_table_large), "mc:" () and "t:" (demo_table_tabs) Server started with 4 threads. Priming the pump ... Connected to "" as node id 8. Scheduler: using 2 connections to cluster 0 Scheduler: starting for 1 cluster; c0,f0,g1,t1 done [0.857 sec]. Failed to notify thread: Socket operation on non-socket Failed to notify thread: Socket operation on non-socket Failed to notify thread: Socket operation on non-socket Failed to notify thread: Socket operation on non-socket Failed to notify thread: Socket operation on non-socket Failed to notify thread: Socket operation on non-socket Failed to notify thread: Socket operation on non-socket Failed to notify thread: Socket operation on non-socket and when connecting on port 11211 the memcached server just emits [warn] event_del: event has no event_base set. [warn] event_del: event has no event_base set. [warn] event_del: event has no event_base set. [...] repeatedly without even replying to the simplest "stats" command. Pressing CTRL-C finally makes it stop printing warnings: [...] [warn] event_del: event has no event_base set.^C Failed to notify thread: Socket operation on non-socket but the client still hangs. This is probably a libevent version issue as the bundled libevent code apparently is "1.3.99-trunk" (found in ./libevent/WIN32-Code/event-config.h:#define _EVENT_VERSION "1.3.99-trunk", all other version info was unfortunately lost when autotools files were purged and replaced by CMake ones) Current Ubuntu comes with libevent 2.0.5 though ... How to repeat: * Build with -DWITH_BUNDLED_MEMCACHED=1 -DWITH_BUNDLED_LIBEVENT=0 * set up and start cluster and the build memcached * watch memcached output on restart and when trying to connect to it with telnet hostname 11211 Suggested fix: Either make the bundled memcached code work with more current libevent versions, or at least have CMake check whether a compatible version was found when WITH_BUNDLED_LIBEVENT is off