VideoToolbox H.264 doesn't work on Avidemux 2.8

[puckluck]

New user here and please excuse my lack of knowledge regarding Avidemux.

Encoding videos using VideoToolbox H.264 output last worked for me with version v2.7.7 on my M1 Mac (Monterey). However, encoding videos using the same codec no longer works on v2.8 including nightly builds as it'll race right through the process, instantaneously resulting in an extremely small file size. Any idea what might be causing this? I'd love to use the latest version of Avidemux instead of v2.7.7.

Thank you

[eumagga0x2a]

However, encoding videos using the same codec no longer works on v2.8 including nightly builds as it'll race right through the process, instantaneously resulting in an extremely small file size. Any idea what might be causing this?

Could you please reproduce the issue with the latest nightly, close Avidemux and either attach /tmp/admlog.txt to your reply (compress it when necessary) or provide it via WeTransfer (no email address required despite the website suggesting otherwise!), Mega, Dropbox or Google Drive?

The VideoToolbox H264 encoder works fine both with my own local builds as well with the latest official nightly on my late 2016 Intel MBP with Monterey. The problem might be related either to some specific properties of the input or to Rosetta 2 / M1 as Avidemux for macOS is x86_64-only.

[eumagga0x2a]

Thank you for the log file. It looks like VideoToolbox doesn't like when input width or height is too small. The lower limit seems to be 640 x 352. The main difference is that in my case the setup of the encoder fails (I get an error message), but in your case it deceitfully succeeds and only the encoding itself fails:

Code Select Expand

[printLavError] 00:56:05-721  Error -542398533 encoding video (Generic error in an external library)
 [encode] 00:56:05-721  [ffvtenc] Error -542398533 encoding video
 [saveLoop] 00:56:05-721  [FF] Wrote 0 frames, nb audio streams 1
Does VideoToolbox H.264 encoder fail for you also with usual 720p sources?

[puckluck]

Thank you for the log file. It looks like VideoToolbox doesn't like when input width or height is too small. The lower limit seems to be 640 x 352. The main difference is that in my case the setup of the encoder fails (I get an error message), but in your case it deceitfully succeeds and only the encoding itself fails:

Code Select Expand

[printLavError] 00:56:05-721  Error -542398533 encoding video (Generic error in an external library)
 [encode] 00:56:05-721  [ffvtenc] Error -542398533 encoding video
 [saveLoop] 00:56:05-721  [FF] Wrote 0 frames, nb audio streams 1
Does VideoToolbox H.264 encoder fail for you also with usual 720p sources?

Unfortunately 720p sources fail. I also tested 1920x1080p and 1024x576p just for the heck of it but they failed too. All formats work in 2.7.7 and versions before that using VideoToolbox H.264. Please let me know if you'd like me to try anything else. Thank you.

[eumagga0x2a]

Would you have disk space and patience to build x86_64 Avidemux on your M1 Mac from source? I have no idea whether it will work, but you will need Homebrew for Intel

https://www.wisdomgeek.com/development/installing-intel-based-packages-using-homebrew-on-the-m1-mac/

then follow macOS build instructions on https://github.com/mean00/avidemux2

[puckluck]

Would you have disk space and patience to build x86_64 Avidemux on your M1 Mac from source? I have no idea whether it will work, but you will need Homebrew for Intel

https://www.wisdomgeek.com/development/installing-intel-based-packages-using-homebrew-on-the-m1-mac/

then follow macOS build instructions on https://github.com/mean00/avidemux2

I might be able to do this in the future and if so, I'll certainly let you know about the results. Thanks again.

[eumagga0x2a]

The problem might be related either to some specific properties of the input or to Rosetta 2 / M1 as Avidemux for macOS is x86_64-only.

Meanwhile I was able to build Avidemux on an M1 Mac natively. I can confirm your report and also rule out both input and x86_64 to arm64 translation as the cause. Will try to look into it.

[eumagga0x2a]

Should be fixed by [ffmpeg/patches] Add upstream patch to fix VideoToolbox encoder broken if B-frames were disabled, thank you very much for your report.

[puckluck]

Should be fixed by [ffmpeg/patches] Add upstream patch to fix VideoToolbox encoder broken if B-frames were disabled, thank you very much for your report.

Thank you very much for the patch and sorry, I just didn't have enough time to test out your previous suggestion.

Also, this is probably an amateur question, but can you tell me how to apply the patch?

[eumagga0x2a]

can you tell me how to apply the patch?

You would need to compile Avidemux off the current git master or wait for an official nightly build (x86_64) to happen. I didn't try to cross-compile for Intel, it really doesn't make sense on M1 as the performance of a natively built Avidemux is so much better.

If you decide to create a build environment, please note that the APFS filesystem to store the clone of the Avidemux repository must be case sensitive, else the build fails. I was shocked to discover that Apple ships devices with file system in case-insensitive mode, apparently due to incompatibilities with some software by Adobe For me, I solved this problem by creating a subvolume and formatting it with a case-sensitive APFS.

However, there are still unsolved issues on Apple/arm64 regarding packaging of libraries from Homebrew in Avidemux app bundle because doing so invalidates their code signatures which in turn means that macOS immediately kills Avidemux process when it tries to load these libraries. Running a native Avidemux build on an M1 Mac from the intermediate install directory "Avidemux2.8.app" works.

[puckluck]

I'll probably wait for the official nightly build as mentioned. Thanks for the additional info regarding M1, etc. and all your help with this is very much appreciated.