Environment setup for cross-compiling on Linux for Windows

Started by eumagga0x2a, November 15, 2016, 12:08:38 PM

Previous topic - Next topic

eumagga0x2a

It happens from time to time that I do need an ability to check whether my changes don't have detrimental effects on Windows. Currently, an attempt to cross compile ends in

Run Build Command:"/usr/bin/gmake" "cmTC_f177f/fast"
/usr/bin/gmake -f CMakeFiles/cmTC_f177f.dir/build.make CMakeFiles/cmTC_f177f.dir/build
gmake[1]: Entering directory '/home/eumagga/adm/git/avidemux2/buildMingwCore/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_f177f.dir/execinfo.cpp.obj
/usr/bin/x86_64-w64-mingw32-g++    @CMakeFiles/cmTC_f177f.dir/includes_CXX.rsp  -o CMakeFiles/cmTC_f177f.dir/execinfo.cpp.obj -c /home/eumagga/adm/git/avidemux2/cmake/cmake_compile_check/execinfo.cpp
In file included from /home/eumagga/adm/git/avidemux2/cmake/cmake_compile_check/execinfo.cpp:2:0:
/usr/include/stdio.h:102:19: error: conflicting declaration 'typedef __ssize_t ssize_t'
typedef __ssize_t ssize_t;
                   ^~~~~~~
In file included from /usr/x86_64-w64-mingw32/sys-root/mingw/include/stddef.h:7:0,
                 from /usr/lib/gcc/x86_64-w64-mingw32/6.2.0/include/stddef.h:1,
                 from /usr/include/stdio.h:33,
                 from /home/eumagga/adm/git/avidemux2/cmake/cmake_compile_check/execinfo.cpp:2:
/usr/x86_64-w64-mingw32/sys-root/mingw/include/crtdefs.h:45:35: note: previous declaration as 'typedef long long int ssize_t'
__MINGW_EXTENSION typedef __int64 ssize_t;
                                   ^~~~~~~
In file included from /usr/include/sys/types.h:132:0,
                 from /usr/include/stdlib.h:314,
                 from /home/eumagga/adm/git/avidemux2/cmake/cmake_compile_check/execinfo.cpp:3:
/usr/include/time.h:75:18: error: conflicting declaration 'typedef __time_t time_t'
typedef __time_t time_t;
                  ^~~~~~
In file included from /usr/x86_64-w64-mingw32/sys-root/mingw/include/stddef.h:7:0,
                 from /usr/lib/gcc/x86_64-w64-mingw32/6.2.0/include/stddef.h:1,
                 from /usr/include/stdio.h:33,
                 from /home/eumagga/adm/git/avidemux2/cmake/cmake_compile_check/execinfo.cpp:2:
/usr/x86_64-w64-mingw32/sys-root/mingw/include/crtdefs.h:138:20: note: previous declaration as 'typedef __time64_t time_t'
typedef __time64_t time_t;
                    ^~~~~~
CMakeFiles/cmTC_f177f.dir/build.make:66: recipe for target 'CMakeFiles/cmTC_f177f.dir/execinfo.cpp.obj' failed
gmake[1]: *** [CMakeFiles/cmTC_f177f.dir/execinfo.cpp.obj] Error 1


configure error or, with the execinfo part in cmake/admCheckMiscLibs.cmake commented out, with

cd /home/eumagga/adm/git/avidemux2/buildMingwCore/ADM_coreUtils/src/Source && /usr/bin/x86_64-w64-mingw32-g++   -DNDEBUG -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES @CMakeFiles/ADM_json.dir/includes_CXX.rsp -fmessage-length=0 -O2 -DNDEBUG   -Werror=attributes -o CMakeFiles/ADM_json.dir/internalJSONNode.cpp.obj -c /home/eumagga/adm/git/avidemux2/avidemux_core/ADM_coreUtils/src/Source/internalJSONNode.cpp
In file included from /usr/x86_64-w64-mingw32/sys-root/mingw/include/c++/cstdint:41:0,
                 from /usr/x86_64-w64-mingw32/sys-root/mingw/include/c++/bits/char_traits.h:420,
                 from /usr/x86_64-w64-mingw32/sys-root/mingw/include/c++/string:40,
                 from /home/eumagga/adm/git/avidemux2/avidemux_core/ADM_coreUtils/src/Source/JSONDefs/Strings_Defs.h:26,
                 from /home/eumagga/adm/git/avidemux2/avidemux_core/ADM_coreUtils/src/Source/JSONDefs.h:14,
                 from /home/eumagga/adm/git/avidemux2/avidemux_core/ADM_coreUtils/src/Source/JSONDebug.h:4,
                 from /home/eumagga/adm/git/avidemux2/avidemux_core/ADM_coreUtils/src/Source/internalJSONNode.h:4,
                 from /home/eumagga/adm/git/avidemux2/avidemux_core/ADM_coreUtils/src/Source/internalJSONNode.cpp:1:
/usr/include/stdint.h:122:27: error: conflicting declaration 'typedef long unsigned int uintptr_t'
typedef unsigned long int uintptr_t;
                           ^~~~~~~~~


build error, which lets me wonder why it picks headers from /usr/include instead of /usr/x86_64-w64-mingw32/sys-root/mingw/include.

What am I missing?

My patch for bootStrapCrossMingw_w64Qt5.sh to reflect the paths used in Fedora:

diff --git a/bootStrapCrossMingw_w64Qt5.sh b/bootStrapCrossMingw_w64Qt5.sh
index 7d18d85..7a77862 100644
--- a/bootStrapCrossMingw_w64Qt5.sh
+++ b/bootStrapCrossMingw_w64Qt5.sh
@@ -1,15 +1,15 @@
#!/bin/bash
# ** Put your config here **
export QT_SELECT=5
-export PATH=$PATH:/mingw/bin
+export PATH=$PATH:/usr/x86_64-w64-mingw32
export CROSS_PREFIX=x86_64-w64-mingw32
-export SDLDIR=/mingw
-export MINGW=/mingw
-export MINGWDEV=/mingw_dev
-export QT_HOME=/mingw/Qt/current
-export CFLAGS="-I/mingw/include -L/mingw/lib"
+export SDLDIR=/usr/x86_64-w64-mingw32/
+export MINGW=/usr/x86_64-w64-mingw32/sys-root/mingw
+export MINGWDEV=/home/eumagga/adm/cross
+export QT_HOME=/usr/x86_64-w64-mingw32/sys-root/mingw
+export CFLAGS="-I/usr/x86_64-w64-mingw32/sys-root/mingw/include -L/usr/x86_64-w64-mingw32/sys-root/mingw/lib"
export O_PARAL="-j 2"
-export TOOLCHAIN_LOCATION=/mingw
+export TOOLCHAIN_LOCATION=/usr
export CROSS_C_COMPILER=gcc
export CROSS_CXX_COMPILER=g++


By the way, does Avidemux need a mingw64 build of x264 to be able to encode in h264 as well?

Any hints or help would be greatly appreciated.

mean

In my setup
/mingw contains everything needed to cross compile
(the compilers & friends are there as symlinks to their real location)
Specifically to avoid these problems, i.e. having a sysroot clearly separated from host system

Yes, you need a mingw64 build

Funnily enough, the resulting binaries works great with wine, including sound and D3D display
Even dxva2 should work (does not in my case)



eumagga0x2a

Thank you for shedding some light on specifics of your setup.

Quote from: mean on November 15, 2016, 12:43:58 PM
Yes, you need a mingw64 build

This was the only step where I had success with

--host=x86_64-w64-mingw32 --cross-prefix="x86_64-w64-mingw32-" --prefix=/usr/x86_64-w64-mingw32/sys-root/mingw --enable-shared --enable-static

as options for ./configure (the mingw64-configure wrapper turned out to be unusable with x264).

mean

The 32 bits mxe is the simpler way to compile it
As for x264, this is what i use

PFX="i686-w64-mingw32.shared"
#
export CC=$PFX-gcc
export LD=$PFX-ld
export AR=$PFX-ar
export RANLIB=$PFX-ranlib
export STRIP=$PFX-strip
export CFLAGS="-I/mingw/include"
export LDFLAGS="-L/mingw/bin -L/mingw/lib"

./configure \
   --enable-shared=yes \
   --disable-static \
--enable-pthread \
   --host=$PFX \
   --prefix=/mingw \
   --disable-static --enable-shared \
  --disable-avs --disable-lavf --disable-gpac --disable-swscale


AQUAR

This topic is of interest to me.
Could you please maintain this discussion live on the forum.
(As opposed to going private - has happened before with code contributing members).


mean

To give a bit more detail

I'm using a LXC container, based on recent debian with MXE compiled in it
As a result, i have minimal host contamination, i.e. no host Qt etc...

With ccache, it takes ~ 3 mn to rebuild the 32 bits windows binary


eumagga0x2a

Thank you, I suspect cmake issues behind the problems I faced trying to cross-compile Avidemux in Fedora using the tools provided by the distribution. I'll try to replicate your setup once I find spare time again (but still ASAP).

By the way, building Qt 5.7.0 from source on Windows in MSYS2 (I tried to setup a build environment on Windows first, but this turned out to be a no-go) took just short of 6 hours on my hardware  ;D

mean

you could try win build, it pulls binaries by default
But it is tricky, you need the right setup for cmake & gcc, else it will not work at all

eumagga0x2a

Quote from: mean on November 16, 2016, 07:09:02 AM
I'm using a LXC container, based on recent debian with MXE compiled in it

Where do you actually put the mxe directory? Into /mingw?

mean

It is in /opt/mxe
but there are symlinks to the bare minimum from /mingw to /opt/mxe (i.e. binutils etc...)

All the other depenedancies are copied or installed to /mingw also


Core is pretty easy to crosscompile

The run starts with the Qt version


eumagga0x2a

Thank you very much, do you use some kind of LOCAL_PKG_LIST in settings.mk to avoid quite wasteful compiling of packages not needed for Avidemux like oce or openscenegraph, and if yes, could you please share it?

Do you use i686-w64-mingw32.shared as MXE_TARGETS to build the toolchain for a 32 bit Avidemux and x86_64-w64-mingw32.shared for 64 bit? (MXE defaults to i686-w64-mingw32.static)


mean

From memory, i targeted Qt and it pulled pretty much all was needed for base system

eumagga0x2a

Thank you very much, will try out this configuration later today.

eumagga0x2a

I think, the cross build problems on Fedora which I experienced in the Debian container as well are related to CMAKE_FIND_ROOT_PATH not set to the target mingw environment root. With the following modifications

diff --git a/foreignBuilds/cross_mingw64_core b/foreignBuilds/cross_mingw64_core
index 0c1a855..04a3e23 100644
--- a/foreignBuilds/cross_mingw64_core
+++ b/foreignBuilds/cross_mingw64_core
@@ -1,11 +1,12 @@
#!/bin/bash
export PKG_CONFIG_PATH=$MINGW/lib/pkgconfig
export PKG_CONFIG_LIBDIR=$MINGW/lib/pkgconfig
-export PATH=$MINGW/bin:$PATH
+#export PATH=$MINGW/bin:$PATH
pkg-config --list-all
#export DEBUG="-DVERBOSE=1 -DDEBUG=1  -DCMAKE_BUILD_TYPE=Debug    "
echo "<<<<<<<<<<<<Cross compiling core ($MINGW)>>>>>>>>>>>>>>>>"
cmake   -DCROSS=$MINGW \
+        -DCMAKE_FIND_ROOT_PATH=$MINGW \
-DTOOLCHAIN_LOCATION=$TOOLCHAIN_LOCATION \
         -DCMAKE_INSTALL_PREFIX=$MINGW/Release \
-DCMAKE_CROSS_PREFIX=$CROSS_PREFIX \
@@ -14,7 +15,7 @@ cmake   -DCROSS=$MINGW \
         -DCMAKE_LINKER:STRING=$CROSS_PREFIX-ld \
         -DCMAKE_AR:STRING=$CROSS_PREFIX-ar \
         -DCMAKE_SYSTEM_NAME:STRING=Windows \
-        -DCMAKE_RC_COMPILER:STRING=$MINGW/bin/$CROSS_PREFIX-windres \
+        -DCMAKE_RC_COMPILER:STRING=$CROSS_PREFIX-windres \
         ${EXTRA_CMAKE_ARG} \
         -DAVIDEMUX_TOP_SOURCE_DIR="$TOP"  \
         -G "Unix Makefiles" \
diff --git a/foreignBuilds/cross_mingw64_plugins b/foreignBuilds/cross_mingw64_plugins
index 8b4768f..f01ed14 100644
--- a/foreignBuilds/cross_mingw64_plugins
+++ b/foreignBuilds/cross_mingw64_plugins
@@ -1,20 +1,21 @@
#!/bin/bash
export PKG_CONFIG_PATH=$MINGW/lib/pkgconfig
export PKG_CONFIG_LIBDIR=$MINGW/lib/pkgconfig
-export PATH=$MINGW/bin:$PATH
+#export PATH=$MINGW/bin:$PATH
pkg-config --list-all
#export DEBUG="-DVERBOSE=1   -DCMAKE_BUILD_TYPE=Debug -G \"CodeBlocks - Unix Makefiles\"   "
export DEBUG="-G \"Unix Makefiles\""
echo "<<<<<<<<<<<<Cross compiling plugins>>>>>>>>>>>>>>>"
cmake   -DCROSS=$MINGW \
-DTOOLCHAIN_LOCATION=$TOOLCHAIN_LOCATION \
+ -DCMAKE_FIND_ROOT_PATH=$MINGW \
         -DCMAKE_INSTALL_PREFIX=$MINGW/Release \
-DCMAKE_CROSS_PREFIX=$CROSS_PREFIX \
         -DCMAKE_C_COMPILER:STRING=$CROSS_PREFIX-${CROSS_C_COMPILER} \
         -DCMAKE_CXX_COMPILER:STRING=$CROSS_PREFIX-${CROSS_CXX_COMPILER} \
         -DCMAKE_LINKER:STRING=$CROSS_PREFIX-ld \
         -DCMAKE_AR:STRING=$CROSS_PREFIX-ar \
-DCMAKE_RC_COMPILER:STRING=$MINGW/bin/$CROSS_PREFIX-windres \
-DCMAKE_RC_COMPILER:STRING=$CROSS_PREFIX-windres \
         -DCMAKE_SYSTEM_NAME:STRING=Windows \
         -DQT_HOME:STRING=$QT_HOME \
         ${EXTRA_CMAKE_ARG} \
diff --git a/foreignBuilds/cross_mingw64_qt5 b/foreignBuilds/cross_mingw64_qt5
index 4356362..13185bf 100644
--- a/foreignBuilds/cross_mingw64_qt5
+++ b/foreignBuilds/cross_mingw64_qt5
@@ -2,24 +2,25 @@
#
export PKG_CONFIG_PATH=$MINGW/lib/pkgconfig
export PKG_CONFIG_LIBDIR=$MINGW/lib/pkgconfig
-export PATH=$MINGW/bin:$PATH
+export PATH=$PATH:$QT_HOME/bin
pkg-config --list-all
#export DEBUG="-DVERBOSE=1   -DCMAKE_BUILD_TYPE=Debug -G \"CodeBlocks - Unix Makefiles\"   "
export DEBUG="-G \"Unix Makefiles\""
echo "<<<<<<<<<<<<Cross compiling core>>>>>>>>>>>>>>>"
cmake   -DCROSS=$MINGW \
-DENABLE_QT5=True \
+ -DCMAKE_FIND_ROOT_PATH=$MINGW \
-DTOOLCHAIN_LOCATION=$TOOLCHAIN_LOCATION  \
         -DCMAKE_INSTALL_PREFIX=$MINGW/Release \
         -DQT_HOME:STRING=$QT_HOME \
-DCMAKE_CROSS_PREFIX=$CROSS_PREFIX \
         -DCMAKE_SYSTEM_NAME:STRING=Windows \
         -DAVIDEMUX_TOP_SOURCE_DIR="$TOP"  \
- -DCMAKE_RC_COMPILER:STRING=$MINGW/bin/$CROSS_PREFIX-windres \
-        -DCMAKE_C_COMPILER:STRING=$MINGW/bin/$CROSS_PREFIX-${CROSS_C_COMPILER} \
-        -DCMAKE_CXX_COMPILER:STRING=$MINGW/bin/$CROSS_PREFIX-${CROSS_CXX_COMPILER} \
-        -DCMAKE_LINKER:STRING=$MINGW/bin/$CROSS_PREFIX-ld \
+ -DCMAKE_RC_COMPILER:STRING=$CROSS_PREFIX-windres \
+        -DCMAKE_C_COMPILER:STRING=$CROSS_PREFIX-${CROSS_C_COMPILER} \
+        -DCMAKE_CXX_COMPILER:STRING=$CROSS_PREFIX-${CROSS_CXX_COMPILER} \
+        -DCMAKE_LINKER:STRING=$CROSS_PREFIX-ld \
         ${EXTRA_CMAKE_ARG} \
-        -DCMAKE_AR:STRING=$MINGW/bin/$CROSS_PREFIX-ar \
+        -DCMAKE_AR:STRING=$CROSS_PREFIX-ar \
         -G "Unix Makefiles" \
         $TOP/avidemux/qt4
diff --git a/foreignBuilds/cross_mingw64_qt5_plugins b/foreignBuilds/cross_mingw64_qt5_plugins
index 662ad7f..a7ceb2b 100644
--- a/foreignBuilds/cross_mingw64_qt5_plugins
+++ b/foreignBuilds/cross_mingw64_qt5_plugins
@@ -1,13 +1,14 @@
#!/bin/bash
export PKG_CONFIG_PATH=$MINGW/lib/pkgconfig
export PKG_CONFIG_LIBDIR=$MINGW/lib/pkgconfig
-export PATH=$MINGW/bin:$PATH
+#export PATH=$MINGW/bin:$PATH
pkg-config --list-all
#export DEBUG="-DVERBOSE=1   -DCMAKE_BUILD_TYPE=Debug -G \"CodeBlocks - Unix Makefiles\"   "
export DEBUG="-G \"Unix Makefiles\""
echo "<<<<<<<<<<<<Cross compiling plugins>>>>>>>>>>>>>>>"
cmake   -DCROSS=$MINGW \
-DTOOLCHAIN_LOCATION=$TOOLCHAIN_LOCATION \
+ -DCMAKE_FIND_ROOT_PATH=$MINGW \
         -DCMAKE_INSTALL_PREFIX=$MINGW/Release \
-DENABLE_QT5=True \
-DCMAKE_CROSS_PREFIX=$CROSS_PREFIX \
@@ -15,7 +16,7 @@ cmake   -DCROSS=$MINGW \
         -DCMAKE_CXX_COMPILER:STRING=$CROSS_PREFIX-${CROSS_CXX_COMPILER} \
         -DCMAKE_LINKER:STRING=$CROSS_PREFIX-ld \
         -DCMAKE_AR:STRING=$CROSS_PREFIX-ar \
-DCMAKE_RC_COMPILER:STRING=$MINGW/bin/$CROSS_PREFIX-windres \
-DCMAKE_RC_COMPILER:STRING=$CROSS_PREFIX-windres \
         -DCMAKE_SYSTEM_NAME:STRING=Windows \
         -DQT_HOME:STRING=$QT_HOME \
         ${EXTRA_CMAKE_ARG} \
diff --git a/foreignBuilds/cross_mingw_cli b/foreignBuilds/cross_mingw_cli
index 8933f10..40a85b4 100644
--- a/foreignBuilds/cross_mingw_cli
+++ b/foreignBuilds/cross_mingw_cli
@@ -2,15 +2,16 @@
#
export PKG_CONFIG_PATH=$MINGW/lib/pkgconfig
export PKG_CONFIG_LIBDIR=$MINGW/lib/pkgconfig
-export PATH=$MINGW/bin:$PATH
+#export PATH=$MINGW/bin:$PATH
pkg-config --list-all
#export DEBUG="-DVERBOSE=1   -DCMAKE_BUILD_TYPE=Debug -G \"CodeBlocks - Unix Makefiles\"   "
export DEBUG="-G \"Unix Makefiles\""
echo "<<<<<<<<<<<<Cross compiling core>>>>>>>>>>>>>>>"
cmake   -DCROSS=$MINGW \
+        -DCMAKE_FIND_ROOT_PATH=$MINGW \
         -DCMAKE_INCLUDE_PATH=$MINGW/include -DCMAKE_LIBRARY_PATH=$MINGW/lib \
         -DCMAKE_INSTALL_PREFIX=$MINGW/Release \
- -DCMAKE_RC_COMPILER:STRING=$MINGW/bin/$CROSS_PREFIX-windres \
+ -DCMAKE_RC_COMPILER:STRING=$CROSS_PREFIX-windres \
         -DCMAKE_C_COMPILER:STRING=$CROSS_PREFIX-${CROSS_C_COMPILER} \
         -DCMAKE_CXX_COMPILER:STRING=$CROSS_PREFIX-${CROSS_CXX_COMPILER} \
         -DCMAKE_LINKER:STRING=$CROSS_PREFIX-ld \


I was able to cross compile 32bit Avidemux core successfully. Qt5 fails ATM because it doesn't find Qt headers despite QT_HEADERS_DIR and QT_INCLUDES from admCheckQt5.cmake being set correcty. Investigating.