InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 241.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:51:57 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 69 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532709, signal count 3541680 --Thread 1290443072 has waited at row0upd.c line 2168 for 251.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 175.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16680080, rounds 87640599, OS waits 1329376 RW-shared spins 2088511, rounds 59696619, OS waits 1983752 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D44618 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 4325 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 34D44617, not started MySQL thread id 2932547745, OS thread handle 0x4dfac940, query id 57252940896 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 tlmsrvfsnpusr ---TRANSACTION 0, not started MySQL thread id 2932547744, OS thread handle 0x4e1f5940, query id 57252940892 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932547743, OS thread handle 0x4dee9940, query id 57252940891 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932547742, OS thread handle 0x4d070940, query id 57252940883 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932547741, OS thread handle 0x5257f940, query id 57252940872 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932547740, OS thread handle 0x50b58940, query id 57252940837 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932547739, OS thread handle 0x510ee940, query id 57252940894 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932547738, OS thread handle 0x4dce1940, query id 57252940873 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932547736, OS thread handle 0x4e3bc940, query id 57252940815 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932547734, OS thread handle 0x4b33d940, query id 57252940817 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932547732, OS thread handle 0x520ac940, query id 57252940848 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932547730, OS thread handle 0x4b504940, query id 57252940877 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932547729, OS thread handle 0x4e06f940, query id 57252940867 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932547727, OS thread handle 0x51f67940, query id 57252940879 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57252792953 3.156.13.68 monyog ---TRANSACTION 34D43B7D, ACTIVE 175 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 251 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272820346 OS file writes, 159811986 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 8.48 writes/s, 8.48 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 8.74 hash searches/s, 5.29 non-hash searches/s --- LOG --- Log sequence number 74495216112 Log flushed up to 74495216112 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151378198 log i/o's done, 8.48 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032987 Database pages 1256011 Old database pages 463624 Modified db pages 1053 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185742, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256011, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 2 queries inside InnoDB, 0 queries in queue 1 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172513, updated 77598106, deleted 2370139, read 7760460690 0.00 inserts/s, 4.25 updates/s, 0.00 deletes/s, 8.75 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 272.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:52:37 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 40 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532714, signal count 3541684 --Thread 1290443072 has waited at row0upd.c line 2168 for 291.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 215.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 20.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 Mutex spin waits 16685720, rounds 87652635, OS waits 1329380 RW-shared spins 2088512, rounds 59696649, OS waits 1983753 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D449C2 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 4512 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932550832, OS thread handle 0x513b9940, query id 57253001915 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 tlmsrvfsnpusr ---TRANSACTION 0, not started MySQL thread id 2932550831, OS thread handle 0x4cfee940, query id 57253001924 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932550830, OS thread handle 0x4b1b7940, query id 57253001918 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932550823, OS thread handle 0x510ad940, query id 57253001927 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932550820, OS thread handle 0x4aca3940, query id 57253001921 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57252792953 3.156.13.68 monyog ---TRANSACTION 34D4482A, ACTIVE 20 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 215 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 291 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272820792 OS file writes, 159812432 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 11.15 writes/s, 11.15 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 11.85 hash searches/s, 12.37 non-hash searches/s --- LOG --- Log sequence number 74495686109 Log flushed up to 74495686109 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151378643 log i/o's done, 11.12 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032967 Database pages 1256031 Old database pages 463632 Modified db pages 1081 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185762, written 117937398 0.00 reads/s, 0.50 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256031, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 3 queries inside InnoDB, 0 queries in queue 1 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172549, updated 77598291, deleted 2370141, read 7760461220 0.90 inserts/s, 4.62 updates/s, 0.05 deletes/s, 13.25 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 303.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:52:57 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532715, signal count 3541684 --Thread 1290443072 has waited at row0upd.c line 2168 for 311.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 235.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 40.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 10.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 Mutex spin waits 16687098, rounds 87654881, OS waits 1329380 RW-shared spins 2088513, rounds 59696679, OS waits 1983754 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D44AC2 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 4569 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932552323, OS thread handle 0x510ad940, query id 57253038322 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932552321, OS thread handle 0x50130940, query id 57253038317 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932552320, OS thread handle 0x525c0940, query id 57253038316 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932552322, OS thread handle 0x4ba18940, query id 57253038314 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932552319, OS thread handle 0x510ee940, query id 57253038299 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932552318, OS thread handle 0x4e84e940, query id 57253038295 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932552316, OS thread handle 0x4ea15940, query id 57253038292 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932552317, OS thread handle 0x51337940, query id 57253038293 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932552315, OS thread handle 0x4d43f940, query id 57253038321 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932552312, OS thread handle 0x50234940, query id 57253038281 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932552309, OS thread handle 0x4bc20940, query id 57253038283 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932552308, OS thread handle 0x4fb59940, query id 57253038315 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932552307, OS thread handle 0x4dc1e940, query id 57253038302 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57252792953 3.156.13.68 monyog ---TRANSACTION 34D44A6B, ACTIVE 10 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 40 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 235 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 311 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272820908 OS file writes, 159812548 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 5.80 writes/s, 5.80 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 7.55 hash searches/s, 4.35 non-hash searches/s --- LOG --- Log sequence number 74495715629 Log flushed up to 74495715629 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151378759 log i/o's done, 5.80 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032967 Database pages 1256031 Old database pages 463632 Modified db pages 1084 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185762, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256031, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 4 queries inside InnoDB, 0 queries in queue 1 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172550, updated 77598348, deleted 2370141, read 7760461385 0.05 inserts/s, 2.85 updates/s, 0.00 deletes/s, 8.25 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ ===================================== 141212 13:53:17 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532718, signal count 3541687 --Thread 1290443072 has waited at row0upd.c line 2168 for 331.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 255.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 60.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 30.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 Mutex spin waits 16688806, rounds 87658569, OS waits 1329383 RW-shared spins 2088513, rounds 59696679, OS waits 1983754 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D44C19 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 4646 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932553995, OS thread handle 0x51de1940, query id 57253076276 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553993, OS thread handle 0x4eda3940, query id 57253076231 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 34D44C17, not started MySQL thread id 2932553992, OS thread handle 0x4afaf940, query id 57253076262 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 tlmsrvfsnpusr ---TRANSACTION 0, not started MySQL thread id 2932553991, OS thread handle 0x4c70b940, query id 57253076273 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553990, OS thread handle 0x51bd9940, query id 57253076274 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553989, OS thread handle 0x4f78a940, query id 57253076239 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553988, OS thread handle 0x4bf6d940, query id 57253076201 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553986, OS thread handle 0x4cfee940, query id 57253076257 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553985, OS thread handle 0x5088d940, query id 57253076253 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553984, OS thread handle 0x51170940, query id 57253076221 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553983, OS thread handle 0x50991940, query id 57253076189 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553982, OS thread handle 0x4aeab940, query id 57253076266 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553981, OS thread handle 0x50234940, query id 57253076265 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553978, OS thread handle 0x4aff0940, query id 57253076260 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553977, OS thread handle 0x4cc60940, query id 57253076251 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553976, OS thread handle 0x4d37c940, query id 57253076227 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553975, OS thread handle 0x4dfed940, query id 57253076214 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553974, OS thread handle 0x4b0f4940, query id 57253076206 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553973, OS thread handle 0x4e1f5940, query id 57253076188 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932553972, OS thread handle 0x50ea5940, query id 57253076279 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57252792953 3.156.13.68 monyog ---TRANSACTION 34D44A6B, ACTIVE 30 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 60 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 255 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 331 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272821062 OS file writes, 159812702 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 7.70 writes/s, 7.70 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 9.40 hash searches/s, 6.00 non-hash searches/s --- LOG --- Log sequence number 74495752815 Log flushed up to 74495752815 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151378913 log i/o's done, 7.70 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032967 Database pages 1256031 Old database pages 463632 Modified db pages 1086 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185762, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256031, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 4 queries inside InnoDB, 0 queries in queue 1 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172550, updated 77598425, deleted 2370141, read 7760461573 0.00 inserts/s, 3.85 updates/s, 0.00 deletes/s, 9.40 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 334.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 258.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:53:37 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532718, signal count 3541687 --Thread 1290443072 has waited at row0upd.c line 2168 for 351.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 275.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 80.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 50.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 Mutex spin waits 16690215, rounds 87661228, OS waits 1329383 RW-shared spins 2088513, rounds 59696679, OS waits 1983754 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D44D2D Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 4726 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932556088, OS thread handle 0x4dee9940, query id 57253123138 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556087, OS thread handle 0x4c80f940, query id 57253123064 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556086, OS thread handle 0x510ad940, query id 57253123133 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556085, OS thread handle 0x522f5940, query id 57253123131 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556084, OS thread handle 0x50130940, query id 57253123125 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556083, OS thread handle 0x4cb5c940, query id 57253123119 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556082, OS thread handle 0x4cada940, query id 57253123114 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556080, OS thread handle 0x506c6940, query id 57253123106 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556079, OS thread handle 0x508ce940, query id 57253123103 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556078, OS thread handle 0x4bf2c940, query id 57253123102 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556076, OS thread handle 0x51ad5940, query id 57253123078 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556072, OS thread handle 0x4dc1e940, query id 57253123096 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556071, OS thread handle 0x51ea4940, query id 57253123088 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556070, OS thread handle 0x50b58940, query id 57253123144 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556068, OS thread handle 0x52705940, query id 57253123127 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556067, OS thread handle 0x52746940, query id 57253123120 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556065, OS thread handle 0x4f645940, query id 57253123111 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556064, OS thread handle 0x4ade8940, query id 57253123094 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556063, OS thread handle 0x4ca58940, query id 57253123087 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556062, OS thread handle 0x4a811940, query id 57253123070 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556061, OS thread handle 0x4ae29940, query id 57253123090 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556060, OS thread handle 0x4b68a940, query id 57253123109 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556057, OS thread handle 0x4efab940, query id 57253123097 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932556056, OS thread handle 0x4bbdf940, query id 57253123145 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57252792953 3.156.13.68 monyog ---TRANSACTION 34D44A6B, ACTIVE 50 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 80 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 275 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 351 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272821222 OS file writes, 159812862 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 8.00 writes/s, 8.00 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 9.30 hash searches/s, 2.00 non-hash searches/s --- LOG --- Log sequence number 74495789696 Log flushed up to 74495789696 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151379073 log i/o's done, 8.00 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032967 Database pages 1256031 Old database pages 463632 Modified db pages 1086 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185762, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256031, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 4 queries inside InnoDB, 0 queries in queue 1 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172550, updated 77598505, deleted 2370141, read 7760461759 0.00 inserts/s, 4.00 updates/s, 0.00 deletes/s, 9.30 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 365.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 289.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:53:57 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532718, signal count 3541687 --Thread 1290443072 has waited at row0upd.c line 2168 for 371.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 295.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 100.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 70.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 Mutex spin waits 16691596, rounds 87664533, OS waits 1329383 RW-shared spins 2088513, rounds 59696679, OS waits 1983754 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D44E50 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 4774 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932557899, OS thread handle 0x51337940, query id 57253164702 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557898, OS thread handle 0x52601940, query id 57253164683 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557896, OS thread handle 0x4cc1f940, query id 57253164720 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557893, OS thread handle 0x50cde940, query id 57253164697 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557891, OS thread handle 0x4b955940, query id 57253164685 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557890, OS thread handle 0x4ffaa940, query id 57253164715 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557889, OS thread handle 0x4ab5e940, query id 57253164706 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557888, OS thread handle 0x5080b940, query id 57253164690 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557887, OS thread handle 0x4bc61940, query id 57253164726 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557886, OS thread handle 0x4c4c2940, query id 57253164712 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557885, OS thread handle 0x4e5c4940, query id 57253164709 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557884, OS thread handle 0x522f5940, query id 57253164704 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557883, OS thread handle 0x4da98940, query id 57253164696 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557882, OS thread handle 0x4b33d940, query id 57253164688 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557880, OS thread handle 0x51378940, query id 57253164732 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557879, OS thread handle 0x5106c940, query id 57253164729 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557878, OS thread handle 0x4b70c940, query id 57253164723 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932557877, OS thread handle 0x4d8d1940, query id 57253164718 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57252792953 3.156.13.68 monyog ---TRANSACTION 34D44A6B, ACTIVE 70 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 100 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 295 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 371 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272821318 OS file writes, 159812958 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 4.80 writes/s, 4.80 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 5.70 hash searches/s, 7.55 non-hash searches/s --- LOG --- Log sequence number 74495812843 Log flushed up to 74495812843 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151379169 log i/o's done, 4.80 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032967 Database pages 1256031 Old database pages 463632 Modified db pages 1087 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185762, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256031, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 4 queries inside InnoDB, 0 queries in queue 1 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172550, updated 77598553, deleted 2370141, read 7760461873 0.00 inserts/s, 2.40 updates/s, 0.00 deletes/s, 5.70 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ ===================================== 141212 13:54:17 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532724, signal count 3541692 --Thread 1290443072 has waited at row0upd.c line 2168 for 391.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 315.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 120.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 90.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 8.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 Mutex spin waits 16694905, rounds 87671187, OS waits 1329388 RW-shared spins 2088514, rounds 59696709, OS waits 1983755 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D4504B Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 4873 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932559612, OS thread handle 0x4cada940, query id 57253201842 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559611, OS thread handle 0x516c5940, query id 57253201825 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559610, OS thread handle 0x4d43f940, query id 57253201786 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559609, OS thread handle 0x4e5c4940, query id 57253201720 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559608, OS thread handle 0x4ea97940, query id 57253201732 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559607, OS thread handle 0x4d74b940, query id 57253201839 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559606, OS thread handle 0x4da98940, query id 57253201805 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559605, OS thread handle 0x51170940, query id 57253201756 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559604, OS thread handle 0x4c995940, query id 57253201740 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559603, OS thread handle 0x5294e940, query id 57253201725 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559602, OS thread handle 0x4ec1d940, query id 57253201844 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559601, OS thread handle 0x518cd940, query id 57253201817 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559600, OS thread handle 0x51274940, query id 57253201795 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559599, OS thread handle 0x4d0f2940, query id 57253201750 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559595, OS thread handle 0x513b9940, query id 57253201735 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559591, OS thread handle 0x50cde940, query id 57253201813 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559590, OS thread handle 0x4bc20940, query id 57253201766 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559589, OS thread handle 0x5090f940, query id 57253201733 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559588, OS thread handle 0x4f0af940, query id 57253201845 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559584, OS thread handle 0x4bbdf940, query id 57253201764 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559582, OS thread handle 0x4e132940, query id 57253201826 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559577, OS thread handle 0x4c585940, query id 57253201746 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559575, OS thread handle 0x4d80e940, query id 57253201731 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932559574, OS thread handle 0x4af6e940, query id 57253201736 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57252792953 3.156.13.68 monyog ---TRANSACTION 34D44FB1, ACTIVE 8 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 90 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 120 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 315 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 391 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272821574 OS file writes, 159813214 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 12.80 writes/s, 12.80 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 12.80 hash searches/s, 16.25 non-hash searches/s --- LOG --- Log sequence number 74495926953 Log flushed up to 74495926953 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151379421 log i/o's done, 12.60 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1092 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.20 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 5 queries inside InnoDB, 0 queries in queue 1 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172577, updated 77598652, deleted 2370141, read 7760462202 1.35 inserts/s, 4.95 updates/s, 0.00 deletes/s, 16.45 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 396.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 320.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:54:37 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532724, signal count 3541692 --Thread 1290443072 has waited at row0upd.c line 2168 for 411.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 335.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 140.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 110.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 28.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 Mutex spin waits 16696087, rounds 87673644, OS waits 1329388 RW-shared spins 2088514, rounds 59696709, OS waits 1983755 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D45137 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 4934 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932561351, OS thread handle 0x4f8cf940, query id 57253240092 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932561349, OS thread handle 0x50b17940, query id 57253240106 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932561347, OS thread handle 0x520ac940, query id 57253240104 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932561348, OS thread handle 0x4f7cb940, query id 57253240102 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932561346, OS thread handle 0x503ba940, query id 57253240096 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932561345, OS thread handle 0x4b914940, query id 57253240129 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932561343, OS thread handle 0x513b9940, query id 57253240100 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932561341, OS thread handle 0x4c70b940, query id 57253240128 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932561340, OS thread handle 0x4aff0940, query id 57253240094 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932561338, OS thread handle 0x50581940, query id 57253240130 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932561337, OS thread handle 0x4bf6d940, query id 57253240127 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932561335, OS thread handle 0x4e84e940, query id 57253240115 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932561334, OS thread handle 0x5257f940, query id 57253240111 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932561333, OS thread handle 0x4ec1d940, query id 57253240112 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932561332, OS thread handle 0x5106c940, query id 57253240118 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57252792953 3.156.13.68 monyog ---TRANSACTION 34D44FB1, ACTIVE 28 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 110 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 140 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 335 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 411 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272821696 OS file writes, 159813336 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 6.10 writes/s, 6.10 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 6.35 hash searches/s, 2.75 non-hash searches/s --- LOG --- Log sequence number 74495956619 Log flushed up to 74495956619 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151379543 log i/o's done, 6.10 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1094 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 5 queries inside InnoDB, 0 queries in queue 1 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172577, updated 77598713, deleted 2370141, read 7760462329 0.00 inserts/s, 3.05 updates/s, 0.00 deletes/s, 6.35 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 427.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 351.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:54:57 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532727, signal count 3541693 --Thread 1290443072 has waited at row0upd.c line 2168 for 431.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 355.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 0, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 160.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 130.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 48.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 8.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 7.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 Mutex spin waits 16698713, rounds 87679617, OS waits 1329389 RW-shared spins 2088516, rounds 59696769, OS waits 1983757 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D452CE Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5011 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932562997, OS thread handle 0x510ad940, query id 57253273064 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932562995, OS thread handle 0x4d74b940, query id 57253273089 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932562996, OS thread handle 0x51c5b940, query id 57253273088 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932562994, OS thread handle 0x4ae29940, query id 57253273084 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932562993, OS thread handle 0x4b1b7940, query id 57253273083 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932562992, OS thread handle 0x4c995940, query id 57253273085 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932562989, OS thread handle 0x4b4c3940, query id 57253273031 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932562987, OS thread handle 0x50c1b940, query id 57253273066 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932562986, OS thread handle 0x4dce1940, query id 57253273048 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932562984, OS thread handle 0x4e3bc940, query id 57253273033 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932562979, OS thread handle 0x4ff28940, query id 57253273026 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932562978, OS thread handle 0x4b33d940, query id 57253273022 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57252792953 3.156.13.68 monyog ---TRANSACTION 34D45269, ACTIVE 7 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 8 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 48 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 130 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 160 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 355 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 431 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272821880 OS file writes, 159813520 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 9.20 writes/s, 9.20 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 69.55 hash searches/s, 15.65 non-hash searches/s --- LOG --- Log sequence number 74496013899 Log flushed up to 74496013899 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151379727 log i/o's done, 9.20 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1103 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 7 queries inside InnoDB, 0 queries in queue 1 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77598790, deleted 2370141, read 7760463822 0.80 inserts/s, 3.85 updates/s, 0.00 deletes/s, 74.65 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ ===================================== 141212 13:55:17 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532729, signal count 3541694 --Thread 1290443072 has waited at row0upd.c line 2168 for 451.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 375.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 180.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 150.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 68.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 28.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 27.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 15.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16700050, rounds 87682057, OS waits 1329390 RW-shared spins 2088517, rounds 59696799, OS waits 1983758 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D453BD Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5076 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932564734, OS thread handle 0x50fa9940, query id 57253314491 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932564733, OS thread handle 0x4b0b3940, query id 57253314489 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932564731, OS thread handle 0x5294e940, query id 57253314474 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932564730, OS thread handle 0x4b649940, query id 57253314502 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932564729, OS thread handle 0x4c585940, query id 57253314463 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932564728, OS thread handle 0x50b58940, query id 57253314500 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932564727, OS thread handle 0x4cd64940, query id 57253314467 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932564725, OS thread handle 0x4b70c940, query id 57253314493 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932564724, OS thread handle 0x4dce1940, query id 57253314483 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932564723, OS thread handle 0x50cde940, query id 57253314499 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932564721, OS thread handle 0x4ea97940, query id 57253314481 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932564624, OS thread handle 0x4bbdf940, query id 57253312242 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57252792953 3.156.13.68 monyog ---TRANSACTION 34D45303, ACTIVE 15 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 27 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 28 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 68 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 150 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 180 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 375 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 451 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272822010 OS file writes, 159813650 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 6.50 writes/s, 6.50 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 6.70 hash searches/s, 2.35 non-hash searches/s --- LOG --- Log sequence number 74496045530 Log flushed up to 74496045530 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151379857 log i/o's done, 6.50 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1105 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 8 queries inside InnoDB, 0 queries in queue 2 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77598855, deleted 2370141, read 7760463962 0.00 inserts/s, 3.25 updates/s, 0.00 deletes/s, 7.00 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 458.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 382.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:55:37 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532730, signal count 3541694 --Thread 1290443072 has waited at row0upd.c line 2168 for 471.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 395.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 200.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 170.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 88.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 48.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 47.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 35.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 12.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16701627, rounds 87685452, OS waits 1329390 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D454F1 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5139 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932566487, OS thread handle 0x525c0940, query id 57253355080 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932566486, OS thread handle 0x4ed62940, query id 57253355101 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932566485, OS thread handle 0x4ba18940, query id 57253355097 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932566484, OS thread handle 0x51b16940, query id 57253355079 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932566483, OS thread handle 0x4a893940, query id 57253355067 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932566480, OS thread handle 0x4c891940, query id 57253355085 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932566479, OS thread handle 0x508ce940, query id 57253355113 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932566478, OS thread handle 0x4f172940, query id 57253355086 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932566477, OS thread handle 0x4f80c940, query id 57253355073 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932566468, OS thread handle 0x520ac940, query id 57253355069 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57252792953 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 12 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 35 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 47 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 48 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 88 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 170 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 200 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 395 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 471 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272822136 OS file writes, 159813776 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 6.30 writes/s, 6.30 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 8.00 hash searches/s, 6.35 non-hash searches/s --- LOG --- Log sequence number 74496077146 Log flushed up to 74496077146 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151379983 log i/o's done, 6.30 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1109 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77598918, deleted 2370141, read 7760464122 0.00 inserts/s, 3.15 updates/s, 0.00 deletes/s, 8.00 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ ===================================== 141212 13:55:53 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 4 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532730, signal count 3541694 --Thread 1290443072 has waited at row0upd.c line 2168 for 487.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 411.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 216.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 186.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 104.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 64.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 63.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 51.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 28.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16702523, rounds 87687340, OS waits 1329390 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D4559B Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5184 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932567961, OS thread handle 0x520ac940, query id 57253393162 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567958, OS thread handle 0x4b851940, query id 57253393128 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567957, OS thread handle 0x4e7cc940, query id 57253393100 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567952, OS thread handle 0x4cada940, query id 57253393163 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567951, OS thread handle 0x50707940, query id 57253393073 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567950, OS thread handle 0x510ee940, query id 57253393118 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567949, OS thread handle 0x508ce940, query id 57253393116 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567948, OS thread handle 0x4fad7940, query id 57253393016 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567947, OS thread handle 0x4dad9940, query id 57253393124 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567946, OS thread handle 0x4b176940, query id 57253393064 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567944, OS thread handle 0x4eea7940, query id 57253393009 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567943, OS thread handle 0x518cd940, query id 57253392947 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567942, OS thread handle 0x4be28940, query id 57253393157 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567941, OS thread handle 0x50685940, query id 57253393150 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567940, OS thread handle 0x4aca3940, query id 57253393145 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567939, OS thread handle 0x4bf6d940, query id 57253393132 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567938, OS thread handle 0x4e3bc940, query id 57253393130 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567937, OS thread handle 0x50379940, query id 57253393111 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567930, OS thread handle 0x4d6c9940, query id 57253393070 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567931, OS thread handle 0x525c0940, query id 57253393048 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567929, OS thread handle 0x4f172940, query id 57253393060 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567928, OS thread handle 0x50a54940, query id 57253393037 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567927, OS thread handle 0x51ad5940, query id 57253393039 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567926, OS thread handle 0x521f1940, query id 57253393014 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567925, OS thread handle 0x4e501940, query id 57253393007 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567923, OS thread handle 0x4f47e940, query id 57253393008 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567919, OS thread handle 0x4b0b3940, query id 57253392982 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567920, OS thread handle 0x4d37c940, query id 57253392981 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567918, OS thread handle 0x4c4c2940, query id 57253392976 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567917, OS thread handle 0x521b0940, query id 57253392962 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567916, OS thread handle 0x4e43e940, query id 57253393170 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567915, OS thread handle 0x4b072940, query id 57253393122 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567914, OS thread handle 0x4c70b940, query id 57253393080 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567913, OS thread handle 0x5288b940, query id 57253393027 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567912, OS thread handle 0x4f80c940, query id 57253392999 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567911, OS thread handle 0x50ea5940, query id 57253392987 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567909, OS thread handle 0x4ea15940, query id 57253392961 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567907, OS thread handle 0x4f0f0940, query id 57253393169 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567908, OS thread handle 0x4cc1f940, query id 57253393168 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932567703, OS thread handle 0x4f4bf940, query id 57253393065 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 28 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 51 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 63 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 64 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 104 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 186 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 216 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 411 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 487 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272822226 OS file writes, 159813866 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 6.00 writes/s, 6.00 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 7.00 hash searches/s, 3.50 non-hash searches/s --- LOG --- Log sequence number 74496099377 Log flushed up to 74496099377 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151380073 log i/o's done, 6.00 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1111 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77598963, deleted 2370141, read 7760464231 0.00 inserts/s, 3.00 updates/s, 0.00 deletes/s, 7.00 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 489.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 413.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:56:13 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532732, signal count 3541696 --Thread 1290443072 has waited at row0upd.c line 2168 for 507.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 431.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 236.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 206.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 124.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 84.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 83.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 71.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 48.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16704095, rounds 87690885, OS waits 1329392 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D456E0 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5253 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932569982, OS thread handle 0x4d3bd940, query id 57253443828 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569981, OS thread handle 0x51ea4940, query id 57253443768 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569980, OS thread handle 0x522b4940, query id 57253443754 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569978, OS thread handle 0x4c585940, query id 57253443788 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569977, OS thread handle 0x5194f940, query id 57253443770 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569976, OS thread handle 0x4ece0940, query id 57253443831 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569975, OS thread handle 0x52601940, query id 57253443802 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569974, OS thread handle 0x512b5940, query id 57253443794 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569973, OS thread handle 0x4d0f2940, query id 57253443792 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569971, OS thread handle 0x4e5c4940, query id 57253443829 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569970, OS thread handle 0x4de67940, query id 57253443819 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569969, OS thread handle 0x4fda2940, query id 57253443812 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569968, OS thread handle 0x50b58940, query id 57253443791 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569966, OS thread handle 0x4c995940, query id 57253443832 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569965, OS thread handle 0x4e646940, query id 57253443737 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569964, OS thread handle 0x50275940, query id 57253443822 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569962, OS thread handle 0x527c8940, query id 57253443800 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569960, OS thread handle 0x517c9940, query id 57253443751 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569958, OS thread handle 0x4f500940, query id 57253443749 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569957, OS thread handle 0x4f8cf940, query id 57253443776 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932569954, OS thread handle 0x5247b940, query id 57253443779 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 48 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 71 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 83 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 84 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 124 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 206 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 236 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 431 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 507 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272822364 OS file writes, 159814004 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 6.90 writes/s, 6.90 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 9.85 hash searches/s, 5.90 non-hash searches/s --- LOG --- Log sequence number 74496132349 Log flushed up to 74496132349 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151380211 log i/o's done, 6.90 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1113 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77599032, deleted 2370141, read 7760464428 0.00 inserts/s, 3.45 updates/s, 0.00 deletes/s, 9.85 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 520.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 444.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1320794432 has waited at row0sel.c line 3872 for 249.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:56:33 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532732, signal count 3541696 --Thread 1290443072 has waited at row0upd.c line 2168 for 527.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 451.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 256.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 226.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 144.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 104.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 103.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 91.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 68.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16704989, rounds 87692541, OS waits 1329392 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D4578E Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5292 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932572121, OS thread handle 0x4ca17940, query id 57253491944 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932572119, OS thread handle 0x512b5940, query id 57253492002 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932572117, OS thread handle 0x51337940, query id 57253491967 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932572116, OS thread handle 0x50581940, query id 57253491979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932572115, OS thread handle 0x4a915940, query id 57253491953 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932572113, OS thread handle 0x4f0af940, query id 57253491941 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932572112, OS thread handle 0x4cc1f940, query id 57253491980 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932572110, OS thread handle 0x4c891940, query id 57253491955 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932572109, OS thread handle 0x4e5c4940, query id 57253491955 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932572108, OS thread handle 0x51e63940, query id 57253491960 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932572102, OS thread handle 0x4b504940, query id 57253492000 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932572098, OS thread handle 0x510ee940, query id 57253491985 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932572097, OS thread handle 0x4e277940, query id 57253491937 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 68 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 91 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 103 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 104 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 144 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 226 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 256 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 451 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 527 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272822442 OS file writes, 159814082 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 3.90 writes/s, 3.90 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 5.30 hash searches/s, 3.15 non-hash searches/s --- LOG --- Log sequence number 74496151885 Log flushed up to 74496151885 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151380289 log i/o's done, 3.90 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1115 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77599071, deleted 2370141, read 7760464534 0.00 inserts/s, 1.95 updates/s, 0.00 deletes/s, 5.30 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ ===================================== 141212 13:56:53 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532732, signal count 3541696 --Thread 1290443072 has waited at row0upd.c line 2168 for 547.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 471.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 276.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 246.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 164.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 124.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 123.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 111.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 88.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16706150, rounds 87695076, OS waits 1329392 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D45872 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5348 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932574002, OS thread handle 0x4b914940, query id 57253537354 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932574001, OS thread handle 0x5190e940, query id 57253537338 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932573999, OS thread handle 0x4bc61940, query id 57253537329 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932573998, OS thread handle 0x51e63940, query id 57253537352 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932573997, OS thread handle 0x5247b940, query id 57253537346 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932573996, OS thread handle 0x4c70b940, query id 57253537336 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932573993, OS thread handle 0x510ee940, query id 57253537315 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932573992, OS thread handle 0x4f47e940, query id 57253537348 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932573987, OS thread handle 0x4d070940, query id 57253537341 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932573985, OS thread handle 0x51233940, query id 57253537298 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 88 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 111 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 123 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 124 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 164 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 246 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 276 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 471 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 547 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272822553 OS file writes, 159814193 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 5.55 writes/s, 5.55 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 6.50 hash searches/s, 3.20 non-hash searches/s --- LOG --- Log sequence number 74496178670 Log flushed up to 74496178670 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151380400 log i/o's done, 5.55 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1116 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77599127, deleted 2370141, read 7760464664 0.00 inserts/s, 2.80 updates/s, 0.00 deletes/s, 6.50 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 551.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 475.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1320794432 has waited at row0sel.c line 3872 for 280.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1358334272 has waited at row0sel.c line 3872 for 250.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:57:13 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532733, signal count 3541697 --Thread 1290443072 has waited at row0upd.c line 2168 for 567.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 491.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 296.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 266.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 184.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 144.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 143.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 131.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 108.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16707156, rounds 87696953, OS waits 1329393 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D45940 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5395 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932576378, OS thread handle 0x51233940, query id 57253593695 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576376, OS thread handle 0x4cc1f940, query id 57253593681 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576375, OS thread handle 0x50fea940, query id 57253593676 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576374, OS thread handle 0x4f0af940, query id 57253593671 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576373, OS thread handle 0x4b33d940, query id 57253593643 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576372, OS thread handle 0x4c2fb940, query id 57253593640 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576370, OS thread handle 0x522f5940, query id 57253593620 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576369, OS thread handle 0x50da1940, query id 57253593693 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576365, OS thread handle 0x51274940, query id 57253593638 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576362, OS thread handle 0x51ad5940, query id 57253593685 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576360, OS thread handle 0x4f500940, query id 57253593670 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576359, OS thread handle 0x5088d940, query id 57253593668 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576352, OS thread handle 0x50379940, query id 57253593634 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576349, OS thread handle 0x51b16940, query id 57253593673 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576347, OS thread handle 0x4be28940, query id 57253593688 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576346, OS thread handle 0x4d0f2940, query id 57253593660 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576345, OS thread handle 0x4ae29940, query id 57253593657 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932576343, OS thread handle 0x50581940, query id 57253593627 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 108 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 131 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 143 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 144 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 184 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 266 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 296 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 491 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 567 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272822647 OS file writes, 159814287 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 4.70 writes/s, 4.70 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 4.70 hash searches/s, 3.25 non-hash searches/s --- LOG --- Log sequence number 74496202798 Log flushed up to 74496202798 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151380494 log i/o's done, 4.70 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1119 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77599174, deleted 2370141, read 7760464758 0.00 inserts/s, 2.35 updates/s, 0.00 deletes/s, 4.70 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 582.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 506.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1320794432 has waited at row0sel.c line 3872 for 311.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1358334272 has waited at row0sel.c line 3872 for 281.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:57:33 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532734, signal count 3541698 --Thread 1290443072 has waited at row0upd.c line 2168 for 587.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 511.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 316.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 286.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 204.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 164.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 163.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 151.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 128.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16708392, rounds 87699691, OS waits 1329394 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D45A34 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5443 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932578109, OS thread handle 0x4e6c8940, query id 57253629679 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578108, OS thread handle 0x522b4940, query id 57253629671 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578107, OS thread handle 0x51b16940, query id 57253629639 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578106, OS thread handle 0x4f02d940, query id 57253629632 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578104, OS thread handle 0x4ae6a940, query id 57253629682 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578100, OS thread handle 0x4b33d940, query id 57253629593 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578098, OS thread handle 0x4bf6d940, query id 57253629563 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578097, OS thread handle 0x4cca1940, query id 57253629691 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578094, OS thread handle 0x4a811940, query id 57253629668 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578093, OS thread handle 0x5106c940, query id 57253629650 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578090, OS thread handle 0x4c544940, query id 57253629609 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578087, OS thread handle 0x4e37b940, query id 57253629636 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578083, OS thread handle 0x51b57940, query id 57253629601 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578082, OS thread handle 0x4e84e940, query id 57253629561 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578080, OS thread handle 0x4bbdf940, query id 57253629672 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578079, OS thread handle 0x50fa9940, query id 57253629656 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578078, OS thread handle 0x4c37d940, query id 57253629661 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578076, OS thread handle 0x4fde3940, query id 57253629582 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578075, OS thread handle 0x51170940, query id 57253629646 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578074, OS thread handle 0x50fea940, query id 57253629618 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578073, OS thread handle 0x4d0f2940, query id 57253629605 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578072, OS thread handle 0x50581940, query id 57253629598 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578071, OS thread handle 0x51fa8940, query id 57253629592 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578070, OS thread handle 0x52683940, query id 57253629577 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578068, OS thread handle 0x51a53940, query id 57253629690 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578067, OS thread handle 0x4f0af940, query id 57253629692 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578066, OS thread handle 0x50bda940, query id 57253629678 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578065, OS thread handle 0x4c2fb940, query id 57253629643 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578064, OS thread handle 0x52746940, query id 57253629638 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578063, OS thread handle 0x5257f940, query id 57253629626 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578061, OS thread handle 0x51de1940, query id 57253629579 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578058, OS thread handle 0x4b914940, query id 57253629565 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578057, OS thread handle 0x50b17940, query id 57253629689 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932578055, OS thread handle 0x4ea15940, query id 57253629688 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 128 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 151 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 163 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 164 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 204 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 286 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 316 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 511 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 587 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272822743 OS file writes, 159814383 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 4.80 writes/s, 4.80 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 5.80 hash searches/s, 5.15 non-hash searches/s --- LOG --- Log sequence number 74496226701 Log flushed up to 74496226701 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151380590 log i/o's done, 4.80 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1121 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77599222, deleted 2370141, read 7760464874 0.00 inserts/s, 2.40 updates/s, 0.00 deletes/s, 5.80 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ ===================================== 141212 13:57:53 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532734, signal count 3541698 --Thread 1290443072 has waited at row0upd.c line 2168 for 607.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 531.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 336.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 306.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 224.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 184.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 183.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 171.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 148.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16710896, rounds 87705779, OS waits 1329394 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D45C51 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5514 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932579832, OS thread handle 0x4aff0940, query id 57253666015 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932579830, OS thread handle 0x51ea4940, query id 57253665968 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932579826, OS thread handle 0x4c544940, query id 57253665980 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932579824, OS thread handle 0x4aeab940, query id 57253666032 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932579823, OS thread handle 0x4c80f940, query id 57253665989 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932579822, OS thread handle 0x4f0f0940, query id 57253666030 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932579821, OS thread handle 0x50fa9940, query id 57253666023 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932579819, OS thread handle 0x4aeec940, query id 57253666004 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932579816, OS thread handle 0x4cca1940, query id 57253666045 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932579815, OS thread handle 0x50644940, query id 57253666018 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932579813, OS thread handle 0x522b4940, query id 57253666046 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 148 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 171 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 183 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 184 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 224 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 306 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 336 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 531 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 607 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272822885 OS file writes, 159814525 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 7.10 writes/s, 7.10 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 8.25 hash searches/s, 17.00 non-hash searches/s --- LOG --- Log sequence number 74496260222 Log flushed up to 74496260222 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151380732 log i/o's done, 7.10 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1122 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77599293, deleted 2370141, read 7760465040 0.00 inserts/s, 3.55 updates/s, 0.00 deletes/s, 8.30 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 613.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 537.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1320794432 has waited at row0sel.c line 3872 for 342.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1358334272 has waited at row0sel.c line 3872 for 312.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:58:13 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532735, signal count 3541699 --Thread 1290443072 has waited at row0upd.c line 2168 for 627.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 551.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 356.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 326.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 244.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 204.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 203.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 191.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 168.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16712654, rounds 87709332, OS waits 1329395 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D45DA6 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5596 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932581462, OS thread handle 0x4d0f2940, query id 57253700291 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581460, OS thread handle 0x4cc60940, query id 57253700232 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581458, OS thread handle 0x51b16940, query id 57253700189 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581457, OS thread handle 0x506c6940, query id 57253700268 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581456, OS thread handle 0x4bf6d940, query id 57253700208 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581455, OS thread handle 0x522b4940, query id 57253700202 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581454, OS thread handle 0x4bbdf940, query id 57253700276 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581453, OS thread handle 0x4bde7940, query id 57253700256 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581452, OS thread handle 0x4f43d940, query id 57253700229 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581451, OS thread handle 0x4ca99940, query id 57253700219 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581450, OS thread handle 0x4ece0940, query id 57253700196 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581449, OS thread handle 0x4b996940, query id 57253700297 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581448, OS thread handle 0x4b892940, query id 57253700282 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581447, OS thread handle 0x51f67940, query id 57253700264 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581446, OS thread handle 0x50da1940, query id 57253700274 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581445, OS thread handle 0x4c70b940, query id 57253700267 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581444, OS thread handle 0x4e74a940, query id 57253700234 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581443, OS thread handle 0x50cde940, query id 57253700302 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581442, OS thread handle 0x5290d940, query id 57253700293 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581441, OS thread handle 0x4e84e940, query id 57253700290 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581440, OS thread handle 0x51fe9940, query id 57253700216 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581438, OS thread handle 0x4c37d940, query id 57253700213 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581437, OS thread handle 0x4b33d940, query id 57253700273 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581436, OS thread handle 0x4de67940, query id 57253700242 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581435, OS thread handle 0x4d78c940, query id 57253700225 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581434, OS thread handle 0x4e06f940, query id 57253700223 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581433, OS thread handle 0x4ada7940, query id 57253700206 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581432, OS thread handle 0x4e646940, query id 57253700209 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581431, OS thread handle 0x4ebdc940, query id 57253700201 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581430, OS thread handle 0x526c4940, query id 57253700197 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932581427, OS thread handle 0x4b0f4940, query id 57253700303 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 168 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 191 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 203 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 204 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 244 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 326 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 356 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 551 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 627 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272823049 OS file writes, 159814689 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 8.20 writes/s, 8.20 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 10.10 hash searches/s, 5.05 non-hash searches/s --- LOG --- Log sequence number 74496299483 Log flushed up to 74496299483 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151380896 log i/o's done, 8.20 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1124 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77599375, deleted 2370141, read 7760465242 0.00 inserts/s, 4.10 updates/s, 0.00 deletes/s, 10.10 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 644.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 568.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1320794432 has waited at row0sel.c line 3872 for 373.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1358334272 has waited at row0sel.c line 3872 for 343.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1260624192 has waited at row0sel.c line 3872 for 261.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:58:33 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532737, signal count 3541701 --Thread 1290443072 has waited at row0upd.c line 2168 for 647.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 571.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 376.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 346.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 264.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 224.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 223.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 211.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 188.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16714425, rounds 87712668, OS waits 1329397 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D45F14 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5667 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932583022, OS thread handle 0x4d5c5940, query id 57253738654 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932583021, OS thread handle 0x4e132940, query id 57253738643 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932583020, OS thread handle 0x4f78a940, query id 57253738595 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932583019, OS thread handle 0x4f645940, query id 57253738580 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932583017, OS thread handle 0x4dfac940, query id 57253738601 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932583016, OS thread handle 0x516c5940, query id 57253738586 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932583015, OS thread handle 0x4eee8940, query id 57253738581 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932583013, OS thread handle 0x4b4c3940, query id 57253738651 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932583012, OS thread handle 0x4be69940, query id 57253738636 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932583011, OS thread handle 0x50b17940, query id 57253738644 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932583010, OS thread handle 0x50789940, query id 57253738613 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932583009, OS thread handle 0x522f5940, query id 57253738611 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932583008, OS thread handle 0x4f8cf940, query id 57253738605 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932583006, OS thread handle 0x4f500940, query id 57253738650 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932583005, OS thread handle 0x4d584940, query id 57253738645 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 188 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 211 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 223 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 224 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 264 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 346 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 376 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 571 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 647 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272823191 OS file writes, 159814831 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 7.10 writes/s, 7.10 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 8.25 hash searches/s, 7.85 non-hash searches/s --- LOG --- Log sequence number 74496332267 Log flushed up to 74496332267 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151381038 log i/o's done, 7.10 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1124 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77599446, deleted 2370141, read 7760465407 0.00 inserts/s, 3.55 updates/s, 0.00 deletes/s, 8.25 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ ===================================== 141212 13:58:53 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532738, signal count 3541702 --Thread 1290443072 has waited at row0upd.c line 2168 for 667.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 591.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 396.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 366.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 284.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 244.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 243.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 231.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 208.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16715741, rounds 87715217, OS waits 1329398 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D46014 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5741 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932584586, OS thread handle 0x526c4940, query id 57253771501 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932584578, OS thread handle 0x4f0f0940, query id 57253771473 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932584577, OS thread handle 0x4d543940, query id 57253771487 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932584576, OS thread handle 0x4d84f940, query id 57253771462 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932584573, OS thread handle 0x4f7cb940, query id 57253771478 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932584571, OS thread handle 0x51d5f940, query id 57253771470 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932584566, OS thread handle 0x4ade8940, query id 57253771481 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932584564, OS thread handle 0x4e0b0940, query id 57253771468 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932584563, OS thread handle 0x52746940, query id 57253771466 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932584562, OS thread handle 0x4b68a940, query id 57253771458 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932584561, OS thread handle 0x51ad5940, query id 57253771455 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932584560, OS thread handle 0x4ece0940, query id 57253771452 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 208 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 231 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 243 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 244 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 284 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 366 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 396 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 591 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 667 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272823339 OS file writes, 159814979 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 7.40 writes/s, 7.40 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 9.05 hash searches/s, 2.05 non-hash searches/s --- LOG --- Log sequence number 74496366372 Log flushed up to 74496366372 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151381186 log i/o's done, 7.40 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1124 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77599520, deleted 2370141, read 7760465588 0.00 inserts/s, 3.70 updates/s, 0.00 deletes/s, 9.05 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 675.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 599.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1320794432 has waited at row0sel.c line 3872 for 404.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1358334272 has waited at row0sel.c line 3872 for 374.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1260624192 has waited at row0sel.c line 3872 for 292.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1383094592 has waited at row0sel.c line 3872 for 252.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1350613312 has waited at row0sel.c line 3872 for 251.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:59:13 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532739, signal count 3541703 --Thread 1290443072 has waited at row0upd.c line 2168 for 687.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 611.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 416.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 386.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 304.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 264.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 263.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 251.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 228.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16717050, rounds 87718003, OS waits 1329398 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D46108 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5813 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932586255, OS thread handle 0x4ad66940, query id 57253806246 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 34D46101, not started MySQL thread id 2932586252, OS thread handle 0x50cde940, query id 57253806232 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 tlmsrvfsnpusr ---TRANSACTION 34D46102, not started MySQL thread id 2932586250, OS thread handle 0x52683940, query id 57253806240 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 tlmsrvfsnpusr ---TRANSACTION 0, not started MySQL thread id 2932586251, OS thread handle 0x4c80f940, query id 57253806239 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932586248, OS thread handle 0x4b996940, query id 57253806255 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932586247, OS thread handle 0x4c544940, query id 57253806265 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932586244, OS thread handle 0x4e06f940, query id 57253806238 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932586242, OS thread handle 0x4b1b7940, query id 57253806275 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932586241, OS thread handle 0x50234940, query id 57253806258 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932586234, OS thread handle 0x4d80e940, query id 57253806273 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932586232, OS thread handle 0x4ff28940, query id 57253806253 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 228 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 251 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 263 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 264 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 304 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 386 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 416 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 611 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 687 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272823483 OS file writes, 159815123 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 7.20 writes/s, 7.20 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 8.85 hash searches/s, 1.70 non-hash searches/s --- LOG --- Log sequence number 74496399620 Log flushed up to 74496399620 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151381330 log i/o's done, 7.20 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1124 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77599592, deleted 2370141, read 7760465765 0.00 inserts/s, 3.60 updates/s, 0.00 deletes/s, 8.85 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 706.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 630.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1320794432 has waited at row0sel.c line 3872 for 435.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1358334272 has waited at row0sel.c line 3872 for 405.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1260624192 has waited at row0sel.c line 3872 for 323.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1383094592 has waited at row0sel.c line 3872 for 283.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1350613312 has waited at row0sel.c line 3872 for 282.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1332775232 has waited at row0sel.c line 3872 for 270.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1337301312 has waited at row0sel.c line 3872 for 247.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 13:59:33 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532741, signal count 3541705 --Thread 1290443072 has waited at row0upd.c line 2168 for 707.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 631.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 436.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 406.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 324.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 284.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 283.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 271.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 248.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16718797, rounds 87721867, OS waits 1329399 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D46257 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5895 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932588304, OS thread handle 0x4be28940, query id 57253844661 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932588303, OS thread handle 0x51c5b940, query id 57253844647 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932588302, OS thread handle 0x513b9940, query id 57253844642 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932588300, OS thread handle 0x4dfac940, query id 57253844663 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932588297, OS thread handle 0x50c1b940, query id 57253844641 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932588294, OS thread handle 0x4f4bf940, query id 57253844657 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932588292, OS thread handle 0x50275940, query id 57253844643 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932588291, OS thread handle 0x51378940, query id 57253844628 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932588290, OS thread handle 0x4bd65940, query id 57253844697 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932588288, OS thread handle 0x5288b940, query id 57253844665 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932588287, OS thread handle 0x4aca3940, query id 57253844659 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932588282, OS thread handle 0x4df6b940, query id 57253844702 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932588271, OS thread handle 0x51233940, query id 57253844649 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932588268, OS thread handle 0x4bde7940, query id 57253844703 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932588264, OS thread handle 0x4e37b940, query id 57253844634 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 248 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 271 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 283 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 284 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 324 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 406 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 436 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 631 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 707 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272823647 OS file writes, 159815287 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 8.20 writes/s, 8.20 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 10.25 hash searches/s, 4.90 non-hash searches/s --- LOG --- Log sequence number 74496438713 Log flushed up to 74496438713 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151381494 log i/o's done, 8.20 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1128 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77599674, deleted 2370141, read 7760465970 0.00 inserts/s, 4.10 updates/s, 0.00 deletes/s, 10.25 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ ===================================== 141212 13:59:53 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532743, signal count 3541707 --Thread 1290443072 has waited at row0upd.c line 2168 for 727.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 651.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 456.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 426.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 344.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 304.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 303.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 291.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 268.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16720977, rounds 87726766, OS waits 1329400 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D46412 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 5972 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932590054, OS thread handle 0x516c5940, query id 57253880817 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590052, OS thread handle 0x4d6c9940, query id 57253880867 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590050, OS thread handle 0x509d2940, query id 57253880794 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590049, OS thread handle 0x4f4bf940, query id 57253880777 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590048, OS thread handle 0x4dfac940, query id 57253880783 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590046, OS thread handle 0x51233940, query id 57253880871 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590045, OS thread handle 0x4c80f940, query id 57253880846 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590044, OS thread handle 0x4ebdc940, query id 57253880793 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590043, OS thread handle 0x4be69940, query id 57253880784 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590042, OS thread handle 0x4e646940, query id 57253880862 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590041, OS thread handle 0x4d543940, query id 57253880835 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590040, OS thread handle 0x51b57940, query id 57253880825 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590039, OS thread handle 0x4ab5e940, query id 57253880819 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590038, OS thread handle 0x50b99940, query id 57253880878 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590036, OS thread handle 0x4f37a940, query id 57253880877 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590035, OS thread handle 0x506c6940, query id 57253880805 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590034, OS thread handle 0x4e1f5940, query id 57253880874 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590033, OS thread handle 0x512f6940, query id 57253880815 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590032, OS thread handle 0x50a54940, query id 57253880810 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590031, OS thread handle 0x4bbdf940, query id 57253880801 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590030, OS thread handle 0x5288b940, query id 57253880790 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932590029, OS thread handle 0x4b8d3940, query id 57253880778 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 268 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 291 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 303 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 304 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 344 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 426 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 456 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 651 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 727 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272823801 OS file writes, 159815441 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 7.70 writes/s, 7.70 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 8.60 hash searches/s, 10.80 non-hash searches/s --- LOG --- Log sequence number 74496475010 Log flushed up to 74496475010 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151381648 log i/o's done, 7.70 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032963 Database pages 1256035 Old database pages 463633 Modified db pages 1129 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185766, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256035, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172593, updated 77599751, deleted 2370141, read 7760466142 0.00 inserts/s, 3.85 updates/s, 0.00 deletes/s, 8.60 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 737.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 661.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1320794432 has waited at row0sel.c line 3872 for 466.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1358334272 has waited at row0sel.c line 3872 for 436.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1260624192 has waited at row0sel.c line 3872 for 354.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1383094592 has waited at row0sel.c line 3872 for 314.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1350613312 has waited at row0sel.c line 3872 for 313.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1332775232 has waited at row0sel.c line 3872 for 301.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1337301312 has waited at row0sel.c line 3872 for 278.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 141212 14:00:07 [Warning] Access denied for user 'sitescope'@'localhost' (using password: YES) ===================================== 141212 14:00:13 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 12 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532745, signal count 3541709 --Thread 1290443072 has waited at row0upd.c line 2168 for 747.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 671.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 476.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 446.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 364.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 324.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 323.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 311.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 288.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16723071, rounds 87730724, OS waits 1329401 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D4657A Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 6056 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932591844, OS thread handle 0x4f645940, query id 57253925355 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932591842, OS thread handle 0x5194f940, query id 57253925351 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932591840, OS thread handle 0x4bb9e940, query id 57253925345 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932591839, OS thread handle 0x4c70b940, query id 57253925336 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932591831, OS thread handle 0x4bd65940, query id 57253925350 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 34D464B5, not started MySQL thread id 2932590820, OS thread handle 0x4fc5d940, query id 57253925356 216.178.99.67 atcmsnbcpusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 288 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 311 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 323 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 324 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 364 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 446 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 476 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 671 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 747 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272824003 OS file writes, 159815643 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 10.83 writes/s, 10.83 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 15.58 hash searches/s, 6.25 non-hash searches/s --- LOG --- Log sequence number 74496530270 Log flushed up to 74496530270 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151381850 log i/o's done, 10.83 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032962 Database pages 1256036 Old database pages 463633 Modified db pages 1144 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185767, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256036, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172610, updated 77599835, deleted 2370141, read 7760466328 1.17 inserts/s, 4.25 updates/s, 0.00 deletes/s, 10.00 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ ===================================== 141212 14:00:33 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532745, signal count 3541709 --Thread 1290443072 has waited at row0upd.c line 2168 for 767.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 691.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 496.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 466.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 384.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 344.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 343.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 331.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 308.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16725148, rounds 87734769, OS waits 1329401 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D466DF Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 6143 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932593284, OS thread handle 0x4b851940, query id 57253953153 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932593283, OS thread handle 0x4dce1940, query id 57253953136 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932593281, OS thread handle 0x4d74b940, query id 57253953126 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932593279, OS thread handle 0x4b1b7940, query id 57253953147 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932593278, OS thread handle 0x523f9940, query id 57253953151 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932593275, OS thread handle 0x4d43f940, query id 57253953122 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932593273, OS thread handle 0x4ca17940, query id 57253953142 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932593272, OS thread handle 0x52746940, query id 57253953132 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932593270, OS thread handle 0x51a53940, query id 57253953140 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253384643 3.156.13.68 monyog ---TRANSACTION 34D466DE, ACTIVE (PREPARED) 0 sec preparing 2 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2932593280, OS thread handle 0x50707940, query id 57253953145 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 tlmsrvfsnpusr COMMIT ---TRANSACTION 34D45460, ACTIVE 308 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 331 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 343 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 344 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 384 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 466 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 496 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 691 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 767 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 1; buffer pool: 0 72215 OS file reads, 272824210 OS file writes, 159815849 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 10.35 writes/s, 10.30 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 13.95 hash searches/s, 4.35 non-hash searches/s --- LOG --- Log sequence number 74496587390 Log flushed up to 74496587030 Last checkpoint at 74486369239 1 pending log writes, 0 pending chkp writes 151382057 log i/o's done, 10.35 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032961 Database pages 1256037 Old database pages 463634 Modified db pages 1153 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185768, written 117937398 0.00 reads/s, 0.05 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256037, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172626, updated 77599923, deleted 2370141, read 7760466530 0.80 inserts/s, 4.40 updates/s, 0.00 deletes/s, 10.10 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 768.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 692.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1320794432 has waited at row0sel.c line 3872 for 497.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1358334272 has waited at row0sel.c line 3872 for 467.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1260624192 has waited at row0sel.c line 3872 for 385.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1383094592 has waited at row0sel.c line 3872 for 345.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1350613312 has waited at row0sel.c line 3872 for 344.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1332775232 has waited at row0sel.c line 3872 for 332.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1337301312 has waited at row0sel.c line 3872 for 309.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 14:00:53 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 4 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532748, signal count 3541712 --Thread 1290443072 has waited at row0upd.c line 2168 for 787.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 711.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 516.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 486.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 404.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 364.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 363.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 351.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 328.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16727037, rounds 87738322, OS waits 1329403 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.25 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D46837 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 6244 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 34D46836, not started MySQL thread id 2932595068, OS thread handle 0x4b6cb940, query id 57253993030 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 tlmsrvfsnpusr ---TRANSACTION 0, not started MySQL thread id 2932595064, OS thread handle 0x4be69940, query id 57253992992 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595063, OS thread handle 0x4e3bc940, query id 57253993020 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595061, OS thread handle 0x51ea4940, query id 57253992991 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595059, OS thread handle 0x50234940, query id 57253993029 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595058, OS thread handle 0x510ee940, query id 57253992994 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595057, OS thread handle 0x4e173940, query id 57253993038 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595056, OS thread handle 0x4e06f940, query id 57253993024 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595055, OS thread handle 0x51e63940, query id 57253993018 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595052, OS thread handle 0x4c2fb940, query id 57253992986 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595051, OS thread handle 0x4dc1e940, query id 57253992962 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595050, OS thread handle 0x4ade8940, query id 57253992949 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595048, OS thread handle 0x4c689940, query id 57253993034 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595044, OS thread handle 0x4b649940, query id 57253993028 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595043, OS thread handle 0x4c995940, query id 57253993004 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595041, OS thread handle 0x4e84e940, query id 57253992970 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595040, OS thread handle 0x5257f940, query id 57253992976 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595039, OS thread handle 0x4cc1f940, query id 57253992961 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595037, OS thread handle 0x4e1f5940, query id 57253993039 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595034, OS thread handle 0x4dad9940, query id 57253992980 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932595033, OS thread handle 0x4c891940, query id 57253992975 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932594868, OS thread handle 0x4b8d3940, query id 57253993001 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253985532 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 328 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 351 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 363 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 364 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 404 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 486 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 516 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 711 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 787 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272824411 OS file writes, 159816051 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 10.00 writes/s, 10.00 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 10.25 hash searches/s, 3.25 non-hash searches/s --- LOG --- Log sequence number 74496634037 Log flushed up to 74496634037 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151382258 log i/o's done, 10.00 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032961 Database pages 1256037 Old database pages 463634 Modified db pages 1154 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185768, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256037, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172626, updated 77600023, deleted 2370141, read 7760466761 0.00 inserts/s, 5.00 updates/s, 0.00 deletes/s, 10.25 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 799.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 723.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1320794432 has waited at row0sel.c line 3872 for 528.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1358334272 has waited at row0sel.c line 3872 for 498.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1260624192 has waited at row0sel.c line 3872 for 416.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1383094592 has waited at row0sel.c line 3872 for 376.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1350613312 has waited at row0sel.c line 3872 for 375.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1332775232 has waited at row0sel.c line 3872 for 363.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1337301312 has waited at row0sel.c line 3872 for 340.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 14:01:13 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532755, signal count 3541719 --Thread 1290443072 has waited at row0upd.c line 2168 for 807.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 731.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 536.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 506.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 424.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 384.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 383.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 371.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 348.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16756822, rounds 87794476, OS waits 1329407 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.24 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D46A19 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 6365 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932597044, OS thread handle 0x4cfee940, query id 57254035638 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597043, OS thread handle 0x4ca99940, query id 57254035628 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597042, OS thread handle 0x526c4940, query id 57254035608 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597041, OS thread handle 0x4a915940, query id 57254035592 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597040, OS thread handle 0x4a956940, query id 57254035674 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597038, OS thread handle 0x4cca1940, query id 57254035662 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597039, OS thread handle 0x4de67940, query id 57254035658 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597037, OS thread handle 0x4b8d3940, query id 57254035654 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597036, OS thread handle 0x50ea5940, query id 57254035648 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597035, OS thread handle 0x4cada940, query id 57254035639 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597034, OS thread handle 0x51b57940, query id 57254035624 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597033, OS thread handle 0x525c0940, query id 57254035600 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597032, OS thread handle 0x4e646940, query id 57254035593 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597031, OS thread handle 0x4c80f940, query id 57254035594 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597030, OS thread handle 0x4ffeb940, query id 57254035582 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597028, OS thread handle 0x5184b940, query id 57254035649 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597026, OS thread handle 0x4eee8940, query id 57254035673 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597023, OS thread handle 0x4c3be940, query id 57254035652 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597018, OS thread handle 0x50581940, query id 57254035672 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597017, OS thread handle 0x51c5b940, query id 57254035666 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597016, OS thread handle 0x4dee9940, query id 57254035671 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597014, OS thread handle 0x523f9940, query id 57254035644 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597013, OS thread handle 0x4f582940, query id 57254035613 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597010, OS thread handle 0x4ca17940, query id 57254035604 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932597009, OS thread handle 0x4f604940, query id 57254035599 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 34D4692A, not started MySQL thread id 2932590824, OS thread handle 0x4a997940, query id 57254035584 216.178.99.67 atcnbcpusr ---TRANSACTION 34D4692B, not started MySQL thread id 2932590807, OS thread handle 0x4f8cf940, query id 57254035581 216.178.99.67 atcnbcpusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253985532 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 348 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 371 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 383 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 384 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 424 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 506 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 536 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 731 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 807 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272824661 OS file writes, 159816301 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 12.50 writes/s, 12.50 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 13.20 hash searches/s, 5.70 non-hash searches/s --- LOG --- Log sequence number 74496694392 Log flushed up to 74496694392 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151382508 log i/o's done, 12.50 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032960 Database pages 1256038 Old database pages 463634 Modified db pages 1166 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185769, written 117937398 0.00 reads/s, 0.05 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256038, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172630, updated 77600144, deleted 2370141, read 7761066728 0.20 inserts/s, 6.05 updates/s, 0.00 deletes/s, 29996.85 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ ===================================== 141212 14:01:33 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532756, signal count 3541720 --Thread 1290443072 has waited at row0upd.c line 2168 for 827.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 751.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 556.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 526.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 444.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 404.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 403.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 391.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 368.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16758686, rounds 87798095, OS waits 1329408 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.24 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D46B79 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 6469 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932598759, OS thread handle 0x4c891940, query id 57254072836 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932598758, OS thread handle 0x4f37a940, query id 57254072808 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932598755, OS thread handle 0x4bc20940, query id 57254072814 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932598753, OS thread handle 0x4d8d1940, query id 57254072825 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932598752, OS thread handle 0x4cada940, query id 57254072811 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932598749, OS thread handle 0x4e173940, query id 57254072829 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253985532 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 368 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 391 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 403 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 404 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 444 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 526 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 556 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 751 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 827 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272824869 OS file writes, 159816509 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 10.40 writes/s, 10.40 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 11.25 hash searches/s, 2.25 non-hash searches/s --- LOG --- Log sequence number 74496743200 Log flushed up to 74496743200 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151382716 log i/o's done, 10.40 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032960 Database pages 1256038 Old database pages 463634 Modified db pages 1167 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185769, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256038, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172630, updated 77600248, deleted 2370141, read 7761066953 0.00 inserts/s, 5.20 updates/s, 0.00 deletes/s, 11.25 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 830.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 754.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1320794432 has waited at row0sel.c line 3872 for 559.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1358334272 has waited at row0sel.c line 3872 for 529.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1260624192 has waited at row0sel.c line 3872 for 447.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1383094592 has waited at row0sel.c line 3872 for 407.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1350613312 has waited at row0sel.c line 3872 for 406.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1332775232 has waited at row0sel.c line 3872 for 394.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1337301312 has waited at row0sel.c line 3872 for 371.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 14:01:53 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532758, signal count 3541721 --Thread 1290443072 has waited at row0upd.c line 2168 for 847.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 771.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 576.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 546.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 464.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 424.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 423.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 411.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 388.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16760693, rounds 87802470, OS waits 1329410 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.24 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D46CF7 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 6590 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932600373, OS thread handle 0x4da98940, query id 57254107343 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932600372, OS thread handle 0x51b16940, query id 57254107339 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932600371, OS thread handle 0x4dee9940, query id 57254107326 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932600370, OS thread handle 0x4d5c5940, query id 57254107334 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932600369, OS thread handle 0x4fda2940, query id 57254107293 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932600368, OS thread handle 0x4bc61940, query id 57254107325 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932600365, OS thread handle 0x4d953940, query id 57254107346 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932600363, OS thread handle 0x52601940, query id 57254107331 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932600362, OS thread handle 0x516c5940, query id 57254107310 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932600361, OS thread handle 0x4cada940, query id 57254107301 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932600352, OS thread handle 0x4e1f5940, query id 57254107317 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932600342, OS thread handle 0x4f4bf940, query id 57254107355 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253985532 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 388 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 411 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 423 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 424 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 464 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 546 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 576 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 771 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 847 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272825110 OS file writes, 159816750 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 12.05 writes/s, 12.05 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 12.95 hash searches/s, 1.20 non-hash searches/s --- LOG --- Log sequence number 74496801543 Log flushed up to 74496801543 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151382957 log i/o's done, 12.05 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032960 Database pages 1256038 Old database pages 463634 Modified db pages 1170 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185769, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256038, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172630, updated 77600369, deleted 2370141, read 7761067212 0.00 inserts/s, 6.05 updates/s, 0.00 deletes/s, 12.95 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 861.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 785.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1320794432 has waited at row0sel.c line 3872 for 590.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1358334272 has waited at row0sel.c line 3872 for 560.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1260624192 has waited at row0sel.c line 3872 for 478.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1383094592 has waited at row0sel.c line 3872 for 438.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1350613312 has waited at row0sel.c line 3872 for 437.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1332775232 has waited at row0sel.c line 3872 for 425.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1337301312 has waited at row0sel.c line 3872 for 402.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 14:02:13 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532758, signal count 3541721 --Thread 1290443072 has waited at row0upd.c line 2168 for 867.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 791.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 596.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 566.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 484.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 444.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 443.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 431.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 408.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16763098, rounds 87807571, OS waits 1329410 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.24 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D46ED0 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 6727 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932601980, OS thread handle 0x4b914940, query id 57254138369 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601976, OS thread handle 0x4ea15940, query id 57254138383 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601975, OS thread handle 0x4df2a940, query id 57254138378 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601974, OS thread handle 0x4fc9e940, query id 57254138362 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601973, OS thread handle 0x4fda2940, query id 57254138359 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601972, OS thread handle 0x4ab5e940, query id 57254138393 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601970, OS thread handle 0x4dfac940, query id 57254138376 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601969, OS thread handle 0x510ad940, query id 57254138339 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601968, OS thread handle 0x500ae940, query id 57254138338 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601966, OS thread handle 0x4f8cf940, query id 57254138380 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601965, OS thread handle 0x51fa8940, query id 57254138335 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601964, OS thread handle 0x50da1940, query id 57254138391 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601962, OS thread handle 0x4e132940, query id 57254138351 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601960, OS thread handle 0x4b892940, query id 57254138392 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601959, OS thread handle 0x5190e940, query id 57254138385 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601958, OS thread handle 0x4b851940, query id 57254138373 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601957, OS thread handle 0x4d43f940, query id 57254138366 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601955, OS thread handle 0x4ebdc940, query id 57254138358 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932601954, OS thread handle 0x5106c940, query id 57254138346 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253985532 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 408 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 431 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 443 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 444 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 484 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 566 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 596 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 791 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 867 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272825384 OS file writes, 159817024 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 13.70 writes/s, 13.70 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 14.55 hash searches/s, 3.35 non-hash searches/s --- LOG --- Log sequence number 74496866501 Log flushed up to 74496866501 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151383231 log i/o's done, 13.70 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032960 Database pages 1256038 Old database pages 463634 Modified db pages 1172 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185769, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256038, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172630, updated 77600506, deleted 2370141, read 7761067503 0.00 inserts/s, 6.85 updates/s, 0.00 deletes/s, 14.55 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ ===================================== 141212 14:02:33 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532762, signal count 3541725 --Thread 1290443072 has waited at row0upd.c line 2168 for 887.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 811.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 616.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 586.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 504.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 464.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 463.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 451.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 428.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16765832, rounds 87813139, OS waits 1329414 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.24 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D470D5 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 6857 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932603806, OS thread handle 0x4ceea940, query id 57254179223 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932603802, OS thread handle 0x4ab5e940, query id 57254179222 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932603798, OS thread handle 0x508ce940, query id 57254179228 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932603795, OS thread handle 0x51b16940, query id 57254179213 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932603793, OS thread handle 0x4f4bf940, query id 57254179226 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932603792, OS thread handle 0x5247b940, query id 57254179217 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932603790, OS thread handle 0x4e06f940, query id 57254179237 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253985532 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 428 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 451 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 463 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 464 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 504 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 586 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 616 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 811 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 887 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272825644 OS file writes, 159817284 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 13.00 writes/s, 13.00 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 13.90 hash searches/s, 6.55 non-hash searches/s --- LOG --- Log sequence number 74496928110 Log flushed up to 74496928110 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151383491 log i/o's done, 13.00 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032960 Database pages 1256038 Old database pages 463634 Modified db pages 1174 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185769, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256038, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172630, updated 77600636, deleted 2370141, read 7761067781 0.00 inserts/s, 6.50 updates/s, 0.00 deletes/s, 13.90 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 892.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 816.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1320794432 has waited at row0sel.c line 3872 for 621.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1358334272 has waited at row0sel.c line 3872 for 591.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1260624192 has waited at row0sel.c line 3872 for 509.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1383094592 has waited at row0sel.c line 3872 for 469.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1350613312 has waited at row0sel.c line 3872 for 468.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1332775232 has waited at row0sel.c line 3872 for 456.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1337301312 has waited at row0sel.c line 3872 for 433.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 14:02:53 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532764, signal count 3541727 --Thread 1290443072 has waited at row0upd.c line 2168 for 907.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 831.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 636.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 606.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 524.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 484.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 483.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 471.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 448.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16768324, rounds 87818282, OS waits 1329415 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.24 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D472C0 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 6997 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932605263, OS thread handle 0x4e5c4940, query id 57254204283 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932605262, OS thread handle 0x5194f940, query id 57254204279 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253985532 3.156.13.68 monyog ---TRANSACTION 34D472BF, ACTIVE (PREPARED) 0 sec preparing 2 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2932605267, OS thread handle 0x4df2a940, query id 57254204286 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 tlmsrvfsnpusr COMMIT ---TRANSACTION 34D45460, ACTIVE 448 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 471 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 483 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 484 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 524 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 606 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 636 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 831 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 907 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 1; buffer pool: 0 72215 OS file reads, 272825924 OS file writes, 159817563 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 14.00 writes/s, 13.95 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 14.00 hash searches/s, 3.55 non-hash searches/s --- LOG --- Log sequence number 74496995449 Log flushed up to 74496995088 Last checkpoint at 74486369239 1 pending log writes, 0 pending chkp writes 151383771 log i/o's done, 14.00 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032960 Database pages 1256038 Old database pages 463634 Modified db pages 1177 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185769, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256038, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172630, updated 77600777, deleted 2370141, read 7761068061 0.00 inserts/s, 7.05 updates/s, 0.00 deletes/s, 14.00 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ 141212 14:03:01 [Warning] Aborted connection 2932590815 to db: 'ATCPBRAV01' user: 'atcbravpusr' host: '216.178.99.67' (Got timeout reading communication packets) 141212 14:03:01 [Warning] Aborted connection 2932590819 to db: 'ATCPSYFY02' user: 'atcsyfypusr' host: '216.178.99.67' (Got timeout reading communication packets) InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 1290443072 has waited at row0upd.c line 2168 for 923.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1355938112 has waited at row0upd.c line 2168 for 847.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1320794432 has waited at row0sel.c line 3872 for 652.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1358334272 has waited at row0sel.c line 3872 for 622.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1260624192 has waited at row0sel.c line 3872 for 540.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1383094592 has waited at row0sel.c line 3872 for 500.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1350613312 has waited at row0sel.c line 3872 for 499.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 InnoDB: Warning: a long semaphore wait: --Thread 1332775232 has waited at row0sel.c line 3872 for 487.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: Warning: a long semaphore wait: --Thread 1337301312 has waited at row0sel.c line 3872 for 464.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 141212 14:03:13 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532766, signal count 3541729 --Thread 1290443072 has waited at row0upd.c line 2168 for 927.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 851.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 656.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 626.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 544.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 504.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 503.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 491.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 468.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16769940, rounds 87821640, OS waits 1329417 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.24 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D473F3 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 7087 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932606933, OS thread handle 0x5194f940, query id 57254243457 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606932, OS thread handle 0x50de2940, query id 57254243441 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606931, OS thread handle 0x4cada940, query id 57254243434 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606930, OS thread handle 0x5257f940, query id 57254243432 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606929, OS thread handle 0x4e911940, query id 57254243485 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606928, OS thread handle 0x4c37d940, query id 57254243470 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606927, OS thread handle 0x4ae29940, query id 57254243460 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606926, OS thread handle 0x4cca1940, query id 57254243450 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606925, OS thread handle 0x4eee8940, query id 57254243447 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606924, OS thread handle 0x4e74a940, query id 57254243480 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606923, OS thread handle 0x4ca17940, query id 57254243477 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606922, OS thread handle 0x4ffaa940, query id 57254243473 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606921, OS thread handle 0x52809940, query id 57254243467 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606920, OS thread handle 0x4b6cb940, query id 57254243455 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606918, OS thread handle 0x51fa8940, query id 57254243442 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606917, OS thread handle 0x50bda940, query id 57254243490 uszmpwslp007lb.west.dmz-nbcuni.com 216.178.99.109 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606915, OS thread handle 0x4d3bd940, query id 57254243489 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932606913, OS thread handle 0x4f172940, query id 57254243446 uszmpwslp007la.west.dmz-nbcuni.com 216.178.99.108 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253985532 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 468 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 491 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 503 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 504 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 544 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 626 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 656 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 851 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 927 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272826103 OS file writes, 159817743 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 8.95 writes/s, 9.00 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4761 buffer(s) 9.00 hash searches/s, 2.10 non-hash searches/s --- LOG --- Log sequence number 74497037589 Log flushed up to 74497037589 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151383950 log i/o's done, 8.95 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032960 Database pages 1256038 Old database pages 463634 Modified db pages 1178 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185769, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256038, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172630, updated 77600866, deleted 2370141, read 7761068241 0.00 inserts/s, 4.45 updates/s, 0.00 deletes/s, 9.00 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ ===================================== 141212 14:03:33 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 20 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 21341105 1_second, 21341081 sleeps, 2128299 10_second, 59713 background, 59708 flush srv_master_thread log flush and writes: 21396089 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 3532771, signal count 3541734 --Thread 1290443072 has waited at row0upd.c line 2168 for 947.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1355938112 has waited at row0upd.c line 2168 for 871.00 seconds the semaphore: X-lock (wait_ex) on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1320794432 has waited at row0sel.c line 3872 for 676.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1358334272 has waited at row0sel.c line 3872 for 646.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1260624192 has waited at row0sel.c line 3872 for 564.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1383094592 has waited at row0sel.c line 3872 for 524.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1350613312 has waited at row0sel.c line 3872 for 523.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f76540 created in file buf0buf.c line 916 a writer (thread id 1290443072) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0row.c line 704 --Thread 1332775232 has waited at row0sel.c line 3872 for 511.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 --Thread 1337301312 has waited at row0sel.c line 3872 for 488.00 seconds the semaphore: S-lock on RW-latch at 0x2aaaf7f763c0 created in file buf0buf.c line 916 a writer (thread id 1355938112) has reserved it in mode wait exclusive number of readers 1, waiters flag 1, lock_word: ffffffffffffffff Last time read locked in file row0sel.c line 3872 Last time write locked in file /pb2/build/sb_0-5135321-1330721177.97/mysqlcom-pro-5.5.22/storage/innobase/row/row0upd.c line 2168 Mutex spin waits 16773203, rounds 87828997, OS waits 1329419 RW-shared spins 2088518, rounds 59696829, OS waits 1983759 RW-excl spins 26155, rounds 6243538, OS waits 201064 Spin rounds per wait: 5.24 mutex, 28.58 RW-shared, 238.71 RW-excl ------------------------ LATEST FOREIGN KEY ERROR ------------------------ 141208 7:18:20 Transaction: TRANSACTION 346CB925, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 1 row lock(s), undo log entries 1 MySQL thread id 2902449886, OS thread handle 0x4b955940, query id 56608342979 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr update INSERT INTO url_rules (url_id,device_class_id,device_os_id,device_os_version_id,device_specific_id,rule_type,created,target_url) VALUES (-1062,2,NULL,NULL,NULL,'passthru',NOW(),'') Foreign key constraint fails for table `NBCPPOINTS01`.`url_rules`: , CONSTRAINT `url_rules_ibfk_1` FOREIGN KEY (`url_id`) REFERENCES `urls` (`id`) Trying to add in child table, in index `one_rule_to_insert_lotr_joke_here` tuple: DATA TUPLE: 6 fields; 0: len 4; hex 7ffffbda; asc ;; 1: len 4; hex 80000002; asc ;; 2: SQL NULL; 3: SQL NULL; 4: SQL NULL; 5: len 4; hex 8000734a; asc sJ;; But in parent table `NBCPPOINTS01`.`urls`, in index `PRIMARY`, the closest match we can find is record: PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 8000004f; asc O;; 1: len 6; hex 0000000773ec; asc s ;; 2: len 7; hex 98000001490110; asc I ;; 3: len 4; hex 80000001; asc ;; 4: len 1; hex 2f; asc /;; ------------------------ LATEST DETECTED DEADLOCK ------------------------ 140523 21:10:05 *** (1) TRANSACTION: TRANSACTION 11CC6C37, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321565, OS thread handle 0x522b4940, query id 6415881803 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '8272798653801b9d86e634.48309159', '1400904606.5505') *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C37 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) TRANSACTION: TRANSACTION 11CC6C38, ACTIVE 0 sec inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s) MySQL thread id 325321564, OS thread handle 0x5106c940, query id 6415881802 uszmpwslp020lb.west.dmz-nbcuni.com 216.178.102.38 drupisaf01usr update INSERT INTO semaphore (name, value, expire) VALUES ('variable_init', '2726299353801b9d8708d4.52141671', '1400904606.5504') *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock mode S locks rec but not gap Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1801 page no 3 n bits 96 index "PRIMARY" of table "DRUPISAFLET01"."semaphore" trx id 11CC6C38 lock_mode X locks rec but not gap waiting Record lock, heap no 26 PHYSICAL RECORD: n_fields 5; compact format; info bits 32 0: len 13; hex 7661726961626c655f696e6974; asc variable_init;; 1: len 6; hex 000011cc6bae; asc k ;; 2: len 7; hex 2a000001b303cb; asc * ;; 3: len 30; hex 3139373336393038303935333830316239613931313264302e3839323336; asc 197369080953801b9a9112d0.89236; (total 33 bytes); 4: len 8; hex 0de0e5e606e0d441; asc A;; *** WE ROLL BACK TRANSACTION (2) ------------ TRANSACTIONS ------------ Trx id counter 34D475F9 Purge done for trx's n:o < 34D43287 undo n:o < 0 History list length 7226 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 0, not started MySQL thread id 2932608766, OS thread handle 0x512f6940, query id 57254278090 uszmpwslp007lc.west.dmz-nbcuni.com 216.178.99.110 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2932608761, OS thread handle 0x4b851940, query id 57254278067 uszmpwslp007ld.west.dmz-nbcuni.com 216.178.99.111 nbcptspusr ---TRANSACTION 0, not started MySQL thread id 2901087953, OS thread handle 0x51da0940, query id 57253985532 3.156.13.68 monyog ---TRANSACTION 34D45460, ACTIVE 488 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932565232, OS thread handle 0x4fb59940, query id 57253328634 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45461, sees < 34D4364E ---TRANSACTION 34D45303, ACTIVE 511 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 0 MySQL thread id 2932563032, OS thread handle 0x4f708940, query id 57253278938 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 statistics SELECT 1 AS expression FROM variable variable WHERE ( (name = 'css_js_query_string') ) Trx read view will not see trx with id >= 34D45304, sees < 34D4364E ---TRANSACTION 34D45269, ACTIVE 523 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562481, OS thread handle 0x5080b940, query id 57253264056 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D451F9, ACTIVE 524 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932562392, OS thread handle 0x52705940, query id 57253262069 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44FB1, ACTIVE 564 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932558590, OS thread handle 0x4b239940, query id 57253186611 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D44A6B, ACTIVE 646 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932551453, OS thread handle 0x50f68940, query id 57253018991 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D4482A, ACTIVE 676 sec starting index read, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 1 lock struct(s), heap size 376, 0 row lock(s) MySQL thread id 2932549140, OS thread handle 0x4eb9b940, query id 57252973184 uszmpwslp020lc.west.dmz-nbcuni.com 216.178.102.39 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) ---TRANSACTION 34D43B7D, ACTIVE 871 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932531957, OS thread handle 0x50d1f940, query id 57252604040 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='s:6:\"ngholl\";' WHERE ( (name = 'css_js_query_string') ) ---TRANSACTION 34D4364E, ACTIVE 947 sec updating or deleting, thread declared inside InnoDB 499 mysql tables in use 1, locked 1 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 2932525255, OS thread handle 0x4cea9940, query id 57252478512 uszmpwslp020la.west.dmz-nbcuni.com 216.178.102.37 drubwdpusr01 Updating UPDATE variable SET value='a:0:{}' WHERE ( (name = 'modernizr_load') ) -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 72215 OS file reads, 272826425 OS file writes, 159818065 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 16.10 writes/s, 16.10 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 0, seg size 2, 405 merges merged operations: insert 473, delete mark 0, delete 0 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 74374243, node heap has 4757 buffer(s) 14.15 hash searches/s, 8.85 non-hash searches/s --- LOG --- Log sequence number 74497161346 Log flushed up to 74497161346 Last checkpoint at 74486369239 0 pending log writes, 0 pending chkp writes 151384272 log i/o's done, 16.10 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 38461767680; in additional pool allocated 0 Dictionary memory allocated 15447592 Buffer pool size 2293759 Free buffers 1032964 Database pages 1256038 Old database pages 463634 Modified db pages 1203 Pending reads 0 Pending writes: LRU 0, flush list 5, single page 0 Pages made young 246, not young 0 0.00 youngs/s, 0.00 non-youngs/s Pages read 70269, created 1185769, written 117937398 0.00 reads/s, 0.00 creates/s, 0.00 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 1256038, unzip_LRU len: 0 I/O sum[0]:cur[0], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 9 queries inside InnoDB, 0 queries in queue 3 read views open inside InnoDB Main thread process no. 27192, id 1217706304, state: flushing buffer pool pages Number of rows inserted 124172652, updated 77601003, deleted 2370143, read 7761068545 1.10 inserts/s, 6.85 updates/s, 0.10 deletes/s, 15.20 reads/s ---------------------------- END OF INNODB MONITOR OUTPUT ============================ InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Error: semaphore wait has lasted > 600 seconds InnoDB: We intentionally crash the server, because it appears to be hung. 141212 14:03:39 InnoDB: Assertion failure in thread 1238686016 in file srv0srv.c line 2455 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html InnoDB: about forcing recovery. 22:03:39 UTC - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=268435456 read_buffer_size=2097152 max_used_connections=513 max_threads=1000 thread_count=14 connection_count=14 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 3345550 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0 thread_stack 0x40000 /mysql/mysql/bin/mysqld(my_print_stacktrace+0x35)[0x79ada5] /mysql/mysql/bin/mysqld(handle_fatal_signal+0x403)[0x669f53] /lib64/libpthread.so.0[0x3b3160eb70] /lib64/libc.so.6(gsignal+0x35)[0x3b30a30265] /lib64/libc.so.6(abort+0x110)[0x3b30a31d10] /mysql/mysql/bin/mysqld[0x828a8e] /lib64/libpthread.so.0[0x3b3160673d] /lib64/libc.so.6(clone+0x6d)[0x3b30ad44bd] The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 141212 14:03:41 mysqld_safe Number of processes running now: 0 141212 14:03:41 mysqld_safe mysqld restarted 141212 14:03:41 [Note] Plugin 'FEDERATED' is disabled. 141212 14:03:41 InnoDB: The InnoDB memory heap is disabled 141212 14:03:41 InnoDB: Mutexes and rw_locks use GCC atomic builtins 141212 14:03:41 InnoDB: Compressed tables use zlib 1.2.3 141212 14:03:41 InnoDB: Using Linux native AIO 141212 14:03:41 InnoDB: Initializing buffer pool, size = 35.0G 141212 14:03:43 InnoDB: Completed initialization of buffer pool 141212 14:03:43 InnoDB: highest supported file format is Barracuda. InnoDB: Log scan progressed past the checkpoint lsn 74486369239 141212 14:03:44 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: Doing recovery: scanned up to log sequence number 74491611648 InnoDB: Doing recovery: scanned up to log sequence number 74496854528 InnoDB: Doing recovery: scanned up to log sequence number 74497179302 141212 14:03:44 InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percents: 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Last MySQL binlog file position 0 517554867, file name /mysql/log/mysql-bin.001345 141212 14:03:47 InnoDB: Waiting for the background threads to start 141212 14:03:48 InnoDB: 1.1.8 started; log sequence number 74497179302 141212 14:03:48 [Note] Recovering after a crash using /mysql/log/mysql-bin 141212 14:03:49 [Note] Starting crash recovery... 141212 14:03:49 [Note] Crash recovery finished. 141212 14:03:49 [ERROR] /mysql/mysql/bin/mysqld: Table './mysql/user' is marked as crashed and should be repaired 141212 14:03:49 [Warning] Checking table: './mysql/user' 141212 14:03:49 [ERROR] 1 client is using or hasn't closed the table properly 141212 14:03:49 [ERROR] /mysql/mysql/bin/mysqld: Table './mysql/db' is marked as crashed and should be repaired 141212 14:03:49 [Warning] Checking table: './mysql/db' 141212 14:03:49 [ERROR] 1 client is using or hasn't closed the table properly 141212 14:03:49 [Note] Event Scheduler: Loaded 0 events 141212 14:03:49 [Note] /mysql/mysql/bin/mysqld: ready for connections. Version: '5.5.22-enterprise-commercial-advanced-log' socket: '/mysql/mysql.sock' port: 3306 MySQL Enterprise Server - Advanced Edition (Commercial)