News:

--

Main Menu

Fix ffmpeg parallel builds

Started by hobbes1069, January 26, 2012, 08:15:04 PM

Previous topic - Next topic

hobbes1069

After a long and drawn out process (and even trying to switch ffmpeg to using the ExternalProject cmake module) I finally found a simple fix for parallel building of ffmpeg.

Basically make doesn't like the way it's being called and can't connect to the job sever. This was fixes in cmake 2.8.4 or so but only in ExternalProject. The fix is to use the shell environment variable MAKE instead of the cmake variable CMAKE_BUILD_TOOL.


diff -Naur avidemux_2.5.6.orig/cmake/admFFmpegBuild.cmake avidemux_2.5.6/cmake/admFFmpegBuild.cmake
--- avidemux_2.5.6.orig/cmake/admFFmpegBuild.cmake      2011-12-28 08:51:49.000000000 -0600
+++ avidemux_2.5.6/cmake/admFFmpegBuild.cmake   2012-01-26 14:07:41.928167845 -0600
@@ -167,7 +167,7 @@

# Build FFmpeg
add_custom_command(OUTPUT "${FFMPEG_BINARY_DIR}/ffmpeg${CMAKE_EXECUTABLE_SUFFIX}"
-                                  COMMAND ${CMAKE_COMMAND} -DCMAKE_BUILD_TOOL=${CMAKE_BUILD_TOOL} -P "${CMAKE_SOURCE_DIR}/cmake/admFFmpegMake.cmake"
+                                  COMMAND ${CMAKE_COMMAND} -DCMAKE_BUILD_TOOL=$(MAKE) -P "${CMAKE_SOURCE_DIR}/cmake/admFFmpegMake.cmake"
                                   WORKING_DIRECTORY "${FFMPEG_BINARY_DIR}")

add_custom_target(ffmpeg ALL


Richard

hobbes1069

Thinking about it, this probably won't work for non-*nix builds so it may be best to put it in a if(UNIX)... conditional.

Richard

mean

From my understanding the problem is that several libs where depending upon ffmpeg
so ffmpeg build was started several times in parallel due to the lack of support of such deps in cmake

That's why i dont get why this fix solves the issue

hobbes1069

WARNING: Complete guess :)

Perhaps since using the $(MAKE) environment variable properly connects it to the job server then make knows about this redundancy and prevents the multiple jobs?

Richard

FeRD_NYC

#4
make's definitely not that smart on its own. I just tried a stupid test (cleaned the fontforge builddir I still had laying around from earlier in the week, then set off two instances of "export CCACHE_DISABLE=1; make" in the directory a few seconds apart) and it trampled all over itself.

It *could* be that CMake is recognizing that an external program has control over the build process, and temporarily suspends all of its own concurrency to wait for the completion of that job. Possibly because it sees the make invocation as an atomic operation. But that's also puuure conjecture.