| Bug #72181 | mysqldbcompare crashes with python error | ||
|---|---|---|---|
| Submitted: | 1 Apr 2014 9:54 | Modified: | 5 Sep 2014 5:21 |
| Reporter: | Heiko Miersch | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Utilities | Severity: | S2 (Serious) |
| Version: | 1.3.6, 1.4.2 | OS: | Linux (Ubuntu 12.04 LTS) |
| Assigned to: | CPU Architecture: | Any | |
| Tags: | mysql utilities, mysqldbcompare | ||
[21 Apr 2014 18:52]
Sveta Smirnova
Thank you for the report. I cannot repeat described behavior with version 1.4.2. Please upgrade, try in your environment and inform us if the issue still exists.
[29 Apr 2014 10:25]
Heiko Miersch
I've tried out mysql utilities v1.4.2 and mysql-python-connector v1.2.1, but same result
-- snip --
Traceback (most recent call last):
File "/usr/bin/mysqldbcompare", line 283, in <module>
db1, db2, options)
File "/usr/lib/python2.7/dist-packages/mysql/utilities/command/dbcompare.py", line 486, in database_compare
reporter, options)
File "/usr/lib/python2.7/dist-packages/mysql/utilities/command/dbcompare.py", line 318, in _check_data_consistency
diag_msgs=debug_msgs)
File "/usr/lib/python2.7/dist-packages/mysql/utilities/common/dbcompare.py", line 1170, in check_consistency
(tbl1_rows, tbl2_rows)))
File "/usr/lib/python2.7/dist-packages/mysql/utilities/common/sql_transform.py", line 278, in transform_data
sql_str += " %s" % build_set_clauses(source, src_cols, row1, row2)
File "/usr/lib/python2.7/dist-packages/mysql/utilities/common/sql_transform.py", line 226, in build_set_clauses
and not col_metadata[col_idx]['is_text']:
TypeError: 'NoneType' object has no attribute '__getitem__'
-- snap --
[29 Apr 2014 16:56]
Joseph Presley
I get the same error with mysql-connector-python 1.1.6 and Utilities version 1.4.2 on Mac OS X Mavericks.
Traceback (most recent call last):
File "/bin/mysqldbcompare", line 283, in <module>
db1, db2, options)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/mysql/utilities/command/dbcompare.py", line 486, in database_compare
reporter, options)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/mysql/utilities/command/dbcompare.py", line 318, in _check_data_consistency
diag_msgs=debug_msgs)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/mysql/utilities/common/dbcompare.py", line 1170, in check_consistency
(tbl1_rows, tbl2_rows)))
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/mysql/utilities/common/sql_transform.py", line 278, in transform_data
sql_str += " %s" % build_set_clauses(source, src_cols, row1, row2)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/mysql/utilities/common/sql_transform.py", line 226, in build_set_clauses
and not col_metadata[col_idx]['is_text']:
TypeError: 'NoneType' object is unsubscriptable
Macintosh-4:~ jpresley$ mysqldbcompare --server1=root:un1c0rn23@localhost '`wordpressunified-pre-thf-pingback-settings`:`wordpressunified`' --difftype=sql --run-all-tests > wordpressunified_thf_pingback_settings.sql
Traceback (most recent call last):
File "/bin/mysqldbcompare", line 283, in <module>
db1, db2, options)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/mysql/utilities/command/dbcompare.py", line 486, in database_compare
reporter, options)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/mysql/utilities/command/dbcompare.py", line 318, in _check_data_consistency
diag_msgs=debug_msgs)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/mysql/utilities/common/dbcompare.py", line 1170, in check_consistency
(tbl1_rows, tbl2_rows)))
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/mysql/utilities/common/sql_transform.py", line 278, in transform_data
sql_str += " %s" % build_set_clauses(source, src_cols, row1, row2)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/mysql/utilities/common/sql_transform.py", line 226, in build_set_clauses
and not col_metadata[col_idx]['is_text']:
TypeError: 'NoneType' object is unsubscriptable
[29 Apr 2014 17:47]
Joseph Presley
To clarify I had the original error cited with the __getitem__ but it switched to the TypeError: 'NoneType' object is unsubscriptable when I switched python versions. The original bug would appear intermittently, but I think I figured out why I got it consistently this time. There were some entries in the table I was comparing which should have produced an SQL entry such as: UPDATE `wordpressunified`.`ms1_5_options` SET `option_value` = '' WHERE `option_id` = '48'; When I compared databases where the updated 'option_value' entry was not empty, mysqldbcompare worked. I ended up reversing the comparison and then manually updating the sql import file to make the comparison work the way I intended it to.
[30 Apr 2014 11:49]
Heiko Miersch
I've run some test with multiple outputtypes and mysqldbcompare only crashed with difftype=sql. The other difftypes run normal.
[1 May 2014 20:46]
k w
I was also experiencing the same issue. I updated line 226 in "/usr/lib/python2.7/dist-packages/mysql/utilities/common/sql_transform.py": and col_metadata is not None and not col_metadata[col_idx]['is_text']: It probably makes sense to fix the `build_set_clauses` function, but for now this works for me.
[29 May 2014 17:24]
Joseph Presley
I can confirm the fix that k w posted worked for me.
[8 Jul 2014 19:25]
Sveta Smirnova
Thank you for the feedback. Verified as described. To repeat: load attached dump, then run mysqldbcompare --server1=root:@127.0.0.1:13000 --difftype=sql 'test:test1' -a
[8 Jul 2014 19:26]
Sveta Smirnova
test data
Attachment: bug72181.sql (text/x-sql), 5.24 KiB.
[11 Aug 2014 23:06]
Kevin Andrews
Applied fix described by k w, working perfectly now. Error seems to only occurs when using the difftype of sql. All other formatted output works fine without the fix.
[5 Sep 2014 5:21]
Philip Olson
Fixed as of the upcoming MySQL Utilities 1.5.2 release, and here's the changelog entry: The "mysqldbcompare" utility failed when comparing two databases while using the SQL difftype. Thank you for the bug report.
Description: while comparing 2 databases, mysqldbcompare exits with a python error -- snip -- Traceback (most recent call last): File "/usr/bin/mysqldbcompare", line 250, in <module> db1, db2, options) File "/usr/lib/python2.7/dist-packages/mysql/utilities/command/dbcompare.py", line 480, in database_compare reporter, options) File "/usr/lib/python2.7/dist-packages/mysql/utilities/command/dbcompare.py", line 315, in _check_data_consistency obj1, obj2, options) File "/usr/lib/python2.7/dist-packages/mysql/utilities/common/dbcompare.py", line 1009, in check_consistency (tbl1_rows, tbl2_rows))) File "/usr/lib/python2.7/dist-packages/mysql/utilities/common/sql_transform.py", line 278, in transform_data sql_str += " %s" % build_set_clauses(source, src_cols, row1, row2) File "/usr/lib/python2.7/dist-packages/mysql/utilities/common/sql_transform.py", line 226, in build_set_clauses and not col_metadata[col_idx]['is_text']: TypeError: 'NoneType' object has no attribute '__getitem__' -- snap -- The crash is always on the same table while comparing the data. table-layout... -- snip -- CREATE TABLE IF NOT EXISTS `tx_artecroominfo_objects` ( `uid` int(11) NOT NULL AUTO_INCREMENT, `pid` int(11) NOT NULL DEFAULT '0', `tstamp` int(11) NOT NULL DEFAULT '0', `crdate` int(11) NOT NULL DEFAULT '0', `cruser_id` int(11) NOT NULL DEFAULT '0', `t3ver_oid` int(11) NOT NULL DEFAULT '0', `t3ver_id` int(11) NOT NULL DEFAULT '0', `t3ver_label` varchar(30) NOT NULL DEFAULT '', `sys_language_uid` int(11) NOT NULL DEFAULT '0', `l18n_parent` int(11) NOT NULL DEFAULT '0', `l18n_diffsource` mediumblob NOT NULL, `deleted` tinyint(4) NOT NULL DEFAULT '0', `hidden` tinyint(4) NOT NULL DEFAULT '0', `title` tinytext NOT NULL, `target_pid` tinytext NOT NULL, `type_uid` int(11) NOT NULL DEFAULT '0', `pos_x` tinytext NOT NULL, `pos_y` tinytext NOT NULL, `pos_rot` tinytext NOT NULL, `type_title` tinytext NOT NULL, `type_libname` tinytext NOT NULL, `signature_type` int(11) NOT NULL DEFAULT '0', `stock_title` tinytext NOT NULL, `color` tinytext NOT NULL, `inherited_target` tinytext NOT NULL, PRIMARY KEY (`uid`), KEY `parent` (`pid`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=14580 ; -- snap -- system: using python v2.7.3 mysql-connector-python v1.1.6 - installed from mysql provided deb-package mysql-utilities v1.3.6 - installed from mysql provided deb-package How to repeat: I call the tool from console -- snip -- mysqldbcompare --server1=<user>:<pw>@localhost:3306 --difftype=sql 'vscout:`vscout-test`' -a > compare.sql -- snap --