build 'avidemux_core' with multithreading is broken

[Jan Gruuthuse]

jan@jan-Z77-Extreme3:~$ grep -c ^core /proc/cpuinfo
8
jan@jan-Z77-Extreme3:~$ nproc
8

both include the virtual cores (HT)

[eumagga0x2a]

use 'nproc' instead for better num of cores detection

Thanks! Filling in gaps in knowledge is the best part of being a Linux user 

This will do it:

Code Select Expand

diff --git a/bootStrap.bash b/bootStrap.bash
index d4ea659..ce4e8d9 100644
--- a/bootStrap.bash
+++ b/bootStrap.bash
@@ -16,7 +16,7 @@ debug=0
qt_ext=Qt5
QT_FLAVOR="-DENABLE_QT5=True"
export QT_SELECT=5 # default for ubuntu, harmless for others
-export O_PARAL="-j 2"
+export O_PARAL="-j $(nproc)"
fail()
{
         echo "** Failed at $1**"

[mean]

First part committed, but done differently

[eumagga0x2a]

Code Select Expand

gmake[3]: warning: -jN forced in submake: disabling jobserver mode.

Why hardcoding "-j 4"? I have only two and sl1pkn07 has 24.

[mean]

Portable way to do it ?

[eumagga0x2a]

Is $(MAKE) as in

Code Select Expand

diff --git a/cmake/admFFmpegBuild.cmake b/cmake/admFFmpegBuild.cmake
index 317c7dc..43a3038 100644
--- a/cmake/admFFmpegBuild.cmake
+++ b/cmake/admFFmpegBuild.cmake
@@ -307,7 +307,7 @@ registerFFmpeg("${FFMPEG_SOURCE_DIR}" "${FFMPEG_BINARY_DIR}" 0)

if(CMAKE_HOST_UNIX)
         add_custom_target(         libavutil_dummy
-                                   COMMAND ${CMAKE_BUILD_TOOL}  -j 4 # We assume make or gnumake when host is unix
+                                   COMMAND $(MAKE) # We assume make or gnumake when host is unix
                                    WORKING_DIRECTORY "${FFMPEG_BINARY_DIR}")
else(CMAKE_HOST_UNIX)
         add_custom_target(         libavutil_dummy

not portable? Would it fail on Mac?

[mean]

1- ${MAKE} is not a "true" cmake variable. The true one is CMAKE_BUILD_TOOL or its newer name. Its value  can be ninja, make, devenv,.. depending on the platform
2- -j 4 only works if you use unix style make -gnumake)

[eumagga0x2a]

1- ${MAKE} is not a "true" cmake variable.

Actually, FFmpeg compilation with ${MAKE} fails. $(MAKE) works.

The true one is CMAKE_BUILD_TOOL or its newer name.

Yes, CMAKE_MAKE_PROGRAM, as CMAKE_BUILD_TOOL is deprecated.

Its value  can be ninja, make, devenv,.. depending on the platform
2- -j 4 only works if you use unix style make -gnumake)

The issue is that this 1) knocks out the make jobserver and 2) hardcodes a concurrency level which may or may not suit the build machine. This is why I would like to find out where the working for me $(MAKE) really fails, because the benefits of using $(MAKE) are obvious while the disadvantages remain sort of abstract.

[eumagga0x2a]

The following patch doesn't help with the jobserver (seems to be an old cmake bug) but at least sets -jN to the value matching the number of CPU cores:

Code Select Expand

diff --git a/cmake/admFFmpegBuild.cmake b/cmake/admFFmpegBuild.cmake
index 317c7dc..32cf4b8 100644
--- a/cmake/admFFmpegBuild.cmake
+++ b/cmake/admFFmpegBuild.cmake
@@ -305,9 +305,15 @@ convertPathToUnix(ffmpeg_gnumake_executable ${BASH_EXECUTABLE})
configure_file("${AVIDEMUX_TOP_SOURCE_DIR}/cmake/ffmpeg_make.sh.cmake" "${FFMPEG_BINARY_DIR}/ffmpeg_make.sh")
registerFFmpeg("${FFMPEG_SOURCE_DIR}" "${FFMPEG_BINARY_DIR}" 0)

+include(ProcessorCount)
+ProcessorCount(N)
+if(NOT N EQUAL 0)
+  set(JOBS_IN_PARALLEL -j${N})
+endif(NOT N EQUAL 0)
+
if(CMAKE_HOST_UNIX)
         add_custom_target(         libavutil_dummy
-                                   COMMAND ${CMAKE_BUILD_TOOL}  -j 4 # We assume make or gnumake when host is unix
+                                   COMMAND ${CMAKE_MAKE_PROGRAM} ${JOBS_IN_PARALLEL} # We assume make or gnumake when host is unix
                                    WORKING_DIRECTORY "${FFMPEG_BINARY_DIR}")
else(CMAKE_HOST_UNIX)
         add_custom_target(         libavutil_dummy

It is still a hack in comparison to $(MAKE) which propagates concurrency level automagically.