Bug #118940 libmyodbc9a.so segfault when trying to update data in external MySQL table from Filemaker
Submitted: 4 Sep 18:18 Modified: 1 Oct 15:54
Reporter: Chris Windle Email Updates:
Status: Verified Impact on me:
None 
Category:Connector / ODBC Severity:S1 (Critical)
Version:9.4.0 OS:Ubuntu (24.04.3 LTS)
Assigned to: CPU Architecture:x86

[4 Sep 18:18] Chris Windle
Description:
Here are the two entries it created in the system log for the segfault:

Sep 04 12:00:31 <server name> kernel: fmserverd[608759]: segfault at 73e74616ea66 ip 000073e01ef66ccb sp 000073e0987e1a30 error 4 in libmyodbc9a.so[73e01ee9c000+1b6000] likely on CPU 1 (core 0, socket 0)
Sep 04 12:00:31 <server name> kernel: Code: eb a3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 f3 0f 1e fa 48 85 ff 74 27 55 48 89 e5 53 48 89 fb 48 83 ec 08 0f 1f 00 48 89 df <48> 8b 1b e8 dd 98 f3 ff 48 85 db 75 f0 48 8b 5d f8 c9 c3 66 90 c3

How to repeat:
Attach to an external MySQL database from Filemaker, and create a form with any of the fields displayed in a text box, view the live data and then change the text field and leave the field to cause it to update the data and it immediately gets an exception in the filemaker server.
[8 Sep 6:56] MySQL Verification Team
Hi,
I do not have access to filemaker. Can you reproduce this every time?
[8 Sep 13:13] Chris Windle
Yes, it happens every time accessing any field of any table that we have tried.  The database is a MySQL version 8.0.42 database that is running in Azure, so I also tried downgrading to 8.0.42 odbc driver using 8 I also downgraded to 8.0.42 community client drivers and the 8.4.0 odbc connector and saw the same issue.  It did not have any problem doing the same test of connecting and updating using the isql app.  If there is something we can do to assist you with debugging, please let me know. Thanks
[8 Sep 13:14] Chris Windle
Also, the version of UnixODBC we are using is 2.3.12
[8 Sep 18:48] Chris Windle
You can download a 45-day free trial of Filemaker with this link: https://www.claris.com/trial/
[9 Sep 11:23] MySQL Verification Team
Thanks for the report
[10 Sep 6:22] Rafal Somla
Posted by developer:
 
What makes us think it is a problem in the ODBC driver? The provided entry from the system log points at crash in `fmserverd` executable, i.e., the Filemaker server as I understand, but we have no trace confirming that the crash happened inside `libmyodbc9a.so` code. Do we have any evidence that the ODBC driver contributed to that crash?

As I understand the issue was first observed when using MySQL ODBC driver version 9.4.0 and also repeated with 8.4.0 and 8.0.42 drivers. Is that correct?
[11 Sep 18:31] Chris Windle
What made me think it was in the ODBC driver is in the log entry I posted that shows it was in fmserverd in the file libmyodbc9a.so, which starts at address 0x73e01ee9c000+1b6000 (and more specifically at ip = 0x73e01ef66ccb), which I believe would be offset 0x000caccb into the libmyodbc9a.so.

Sep 04 12:00:31 <server name> kernel: fmserverd[608759]: segfault at 73e74616ea66 ip 000073e01ef66ccb sp 000073e0987e1a30 error 4 in libmyodbc9a.so[73e01ee9c000+1b6000] likely on CPU 1 (core 0, socket 0)

A objdump shows that it looks like it was probably a stack overflow, maybe, because it seems to be at an offset that is wrong,  from the dump:

cab83:       4c 89 f8                mov    %r15,%rax
   cab86:       e9 ee 0a 00 00          jmp    cb679 <_Z12my_SQLSetPosPvmtt+0x175f>
   cab8b:       c7 85 90 fe ff ff 00    movl   $0x0,-0x170(%rbp)
   cab92:       00 00 00
   cab95:       48 c7 85 98 fe ff ff    movq   $0x0,-0x168(%rbp)
   cab9c:       00 00 00 00
   caba0:       48 c7 85 a0 fe ff ff    movq   $0x0,-0x160(%rbp)
   caba7:       00 00 00 00
   cabab:       c7 85 a8 fe ff ff 00    movl   $0x0,-0x158(%rbp)
   cabb2:       00 00 00
   cabb5:       48 c7 85 b0 fe ff ff    movq   $0x0,-0x150(%rbp)
   cabbc:       00 00 00 00
   cabc0:       66 c7 85 b8 fe ff ff    movw   $0x0,-0x148(%rbp)
   cabc7:       00 00
   cabc9:       48 c7 85 c0 fe ff ff    movq   $0x0,-0x140(%rbp)
   cabd0:       00 00 00 00
   cabd4:       66 c7 85 c8 fe ff ff    movw   $0x0,-0x138(%rbp)
   cabdb:       00 00
   cabdd:       c7 85 cc fe ff ff 00    movl   $0x0,-0x134(%rbp)
   cabe4:       00 00 00
   cabe7:       48 c7 85 d0 fe ff ff    movq   $0x0,-0x130(%rbp)
   cabee:       00 00 00 00
   cabf2:       66 c7 85 d8 fe ff ff    movw   $0x0,-0x128(%rbp)
   cabf9:       00 00
   cabfb:       48 c7 85 e0 fe ff ff    movq   $0x0,-0x120(%rbp)
   cac02:       00 00 00 00
   cac06:       48 c7 85 e8 fe ff ff    movq   $0x0,-0x118(%rbp)
   cac0d:       00 00 00 00
   cac11:       48 c7 85 f0 fe ff ff    movq   $0x0,-0x110(%rbp)
   cac18:       00 00 00 00
   cac1c:       48 c7 85 f8 fe ff ff    movq   $0x0,-0x108(%rbp)
   cac23:       00 00 00 00
   cac27:       48 c7 85 00 ff ff ff    movq   $0x0,-0x100(%rbp)
   cac2e:       00 00 00 00
   cac32:       48 c7 85 08 ff ff ff    movq   $0x0,-0xf8(%rbp)
   cac39:       00 00 00 00
   cac3d:       48 c7 85 10 ff ff ff    movq   $0x0,-0xf0(%rbp)
   cac44:       00 00 00 00
   cac48:       66 c7 85 18 ff ff ff    movw   $0x0,-0xe8(%rbp)
   cac4f:       00 00
   cac51:       c7 85 1c ff ff ff 00    movl   $0x0,-0xe4(%rbp)
   cac58:       00 00 00
   cac5b:       48 c7 85 20 ff ff ff    movq   $0x0,-0xe0(%rbp)
   cac62:       00 00 00 00
   cac66:       48 c7 85 28 ff ff ff    movq   $0x0,-0xd8(%rbp)
   cac6d:       00 00 00 00
   cac71:       66 c7 85 30 ff ff ff    movw   $0x0,-0xd0(%rbp)
   cac78:       00 00
   cac7a:       66 c7 85 32 ff ff ff    movw   $0x0,-0xce(%rbp)
   cac81:       00 00
   cac83:       66 c7 85 34 ff ff ff    movw   $0x0,-0xcc(%rbp)
   cac8a:       00 00
   cac8c:       66 c7 85 36 ff ff ff    movw   $0x0,-0xca(%rbp)
   cac93:       00 00
   cac95:       48 c7 85 38 ff ff ff    movq   $0x0,-0xc8(%rbp)
   cac9c:       00 00 00 00
   caca0:       66 c7 85 40 ff ff ff    movw   $0x0,-0xc0(%rbp)
   caca7:       00 00
   caca9:       48 c7 85 48 ff ff ff    movq   $0x0,-0xb8(%rbp)
   cacb0:       00 00 00 00
   cacb4:       66 c7 85 50 ff ff ff    movw   $0x0,-0xb0(%rbp)
   cacbb:       00 00
   cacbd:       48 c7 85 58 ff ff ff    movq   $0x0,-0xa8(%rbp)
   cacc4:       00 00 00 00
   cacc8:       66 c7 85 60 ff ff ff    movw   $0x0,-0xa0(%rbp)
   caccf:       00 00
[12 Sep 16:37] Rafal Somla
Posted by developer:
 
Indeed, the log entry points to `libmyodbc9a.so` -- I was blind apparently -- sorry for that.
[17 Sep 12:40] Chris Windle
Do you think that this is something that you might be able to fix quickly or will it take awhile? If so, do you have any rough ideas of when it might be available?  We are currently trying to connect a Filemaker database to external tables we have in a MySQL database and it is working as a read-only source, but we can't update anything because of this bug.  So, we are currently in a holding pattern trying to determine if we want to wait for this to be fixed, so we can go ahead with our plans, or if we should look at other alternatives to try and accomplish the same goal.  So, understanding the timeframe on this issue being addressed is very important to us.  Thanks.
[17 Sep 13:25] Rafal Somla
Posted by developer:
 
Hi,

No, I don't think it can happen quickly. Main reason is that it is unlikely we can reproduce this issue locally. Given that, the only way forward would be to try to collect more data from your side and do the "blind guess" game which might or might not lead us to the root cause. We can always try but without an ability to reproduce locally there are slim chances it will work. Things you can do to start the process:

1. Enable ODBC tracing -- maybe some hints can be found in the ODBC trace provided you are able to isolate trace entries related to the crash (or just before the crash).

2. Given all the information you can find try to reproduce the same issue *without* Filemaker or any other complex environment so that we have a chance to reproduce it on our side.
[17 Sep 13:49] Chris Windle
I think if you obtain the free 30-day evaluation copy of Filemaker that I provided the link for, you should be able to easily reproduce locally, by just creating a single form and put any field from a MySQL database on that form, then view it and try to update the field. It has happened on every field we have tried. 

Being a software developer, I totally understand the dilemma of not being able to make much progress when you can't reproduce it consistently or easily.  But, in this case, it seems pretty easy and consistent, if that helps.
[24 Sep 22:13] Chris Windle
I was just curious, if anybody had tried to reproduce the issue with Filemaker using an evaluation copy from the provided link. We are trying to plan what direction we need to go in, as a company to integrate data from our Redmine system for project and bug tracking, which is currently using postgreSQL, and integrating with our courporate data in Filemaker Pro. The timeframe and availability of MySQL as an option, is a key factor in the direction of our plans.
[25 Sep 17:14] MySQL Verification Team
Hi Chris,

As you can see bug has bin verified so yes, we reproduced the problem with eval version of Filemaker.
[30 Sep 6:41] Rafal Somla
Posted by developer:
 
Hi Chris,

Sadly, further working with the Filemaker trial is not an option for us because it can be used only for "personal and non-commercial purposes". Therefore, as I see it, the only option left on the table is this "blind guess" game I mentioned where you perform some experiments on your side and share results with us. But again, the mileage might vary...
[1 Oct 15:54] Chris Windle
I understand the using Filemaker trial only for personal and non-commercial purposes point.  But, I am not sure why that would preclude you from using it to assist in fixing issues working with driver, since you won't be the ones using Filemaker at all.  It would be your customers, which most likely, like us, have a license for using Filemaker for commercial use.  But, I also understand that you have to abide with what your legal department says.  So, I am open to providing you anything you need that would be helpful in resolving this.  We are currently using the driver as a one-way only solution to allow us to access MySQL database data to Filemaker, but we are having to export data from Filemaker and then import it into MySQL using external scripts, as a work around of the limitation that we can't directly write to MySQL records.  But, this puts duplicated data in both datbases, which causes potential out-of-sync situations, etc. So, please let us know what we can do to assist you in making any progress.
[2 Oct 7:14] Rafal Somla
Posted by developer:
 
I see two possible directions of investigation.

1. Since it is a crash, maybe we can get a core dump and a stack trace that would point at the place inside the connector where the crash happens. Getting meaningful stack trace might involve using a debug build of the driver.

2. Another source of insight would be to learn what was the last query/thing done by the driver before the crash -- one could try to get that information from ODBC trace logs.