MySQL Bugs: #53240: parallel windows compile flakey on sql

Bug #53240	parallel windows compile flakey on sql_yacc.cc
Submitted:	28 Apr 2010 9:04	Modified:	16 Jan 2014 12:40
Reporter:	Bernd Ocklin	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Server: Compiling	Severity:	S3 (Non-critical)
Version:	mysql-5.1 ff	OS:	Any
Assigned to:	Bernd Ocklin	CPU Architecture:	Any

Description:
WIndows parallel compile fails sporadically due to missing/unclean target dependencies in sql/CMakeLists.txt. From the logs:

347>Generating sql_yacc.h, sql_yacc.cc
351>Build started: Project: sql, Configuration: RelWithDebInfo|x64
...
351>c1xx : fatal error C1083: Cannot open source file: '.\sql_yacc.cc': Permission denied

Custom target "GenServerSource" does only depend on the files sql_yacc.*. Bison creates this but still might write to it/lock it when a second thread starts compiling it.

How to repeat:
Compile in parallel and have bad luck.

Suggested fix:
General server custom target is wrongly depended on the sql_yacc.cc *file* and not on the actual bison *target* (which does not even exit). Fix is to introduce a bison target by e.g. using the 

ADD_CUSTOM_COMMAND(TARGET sql_yacc ... )

ADD_CUSTOM_TARGET(
        GenServerSource ALL
        DEPENDS sql_yacc ...)

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/106896

3581 Bernd Ocklin	2010-04-29
      Bug#53240: introduce bison target for fixing compile dependencies

Lars, feel free set proper lead. We would really like to get this into 5.1 mainline.

@Bernd, can you attach the full log to the bug report? I never have seen this (I routinely compile on 4 and 8 cores), so I'm trying to understand what is going on in parallel.

And depending on the file is just fine. CMake is smart enough to recognize that if file is missing and there is a rule to generate it, it is generated before target is built. If it were not so, you would get a missing sql_yacc.cc, which is not the case here.

Ok, I can guess what happens here.
Say embedded  and sql are compiled in parallel. 
a ) embedded depends on GenServerSource which depends on sql_yacc.cc, and now bison starts generating its output.
b ) sql starts compiling, sees sql_yacc.cc is already there and  there is no need to generate it.  Tries to  compile sql_yacc.cc . However bison is not yet finished, so sql_yacc.cc is locked.

If you want to fix it cleanly, there is no need to create an extra sql_yacc target. 
Instead, make sql depend on GenerateServerSource using ADD_DEPENDENCIES. It will prevent parallel execution. Also it will ensure that other generated files cannot be created and used at the same time.