Combining videos corrupts appended videos

[Avidummy]

When combining two mp4s on copy mode, the second mp4 is garbled.  The codec settings should be precisely the same, and if they are not I'd like to know why.

Version 2.8.2

I'm joining 20-second trailcam AVIs by encoding with h264.  That part works fine.

I drag first AVI into window, then next AVI, etc.  Then 2-pass encode with h264.  Result is video1.

Then go to file, close, and then start dragging more AVIs to join and encode.  Result is video2.  All is well.

Then I change video and audio output to copy and attempt to join video1 and video2.  The result is video2 is corrupted when appended to video1.  How is that possible if both mp4s were encoded by the same session of avidemux without making any changes?


Size:            47
Extradata:         01 64 00 28 FF E1 00 1B 67 64 00 28 AC D9 40 78 02 27 E5 C0 44 00 00 03 00 04 00 00 03 00 CA 3C 60 C6 58 01 00 05 68 EB E5 2C 8B FD F8 F8 00

=====================================================
Size:            48
Extradata:         01 64 00 28 FF E1 00 1B 67 64 00 28 AC D9 40 78 02 27 E5 C0 44 00 00 03 00 04 00 00 03 00 CA 3C 60 C6 58 01 00 06 68 EB E3 4B 22 C0 FD F8 F8 00

When using 2.8.1 I noticed the frame rate was changed to fractional for no reason.  2.8.2 fixed that.  Now it appears codec settings are changing for no reason?  Is that a known issue?

Now that the damage is done, how can I join those videos without increasing the size or decreasing the quality?

[eumagga0x2a]

As codec extradatas don't match (even extradata sizes don't match), these videos cannot be appended to each other in copy mode. I wasn't aware of 2-pass encoding mode producing varying extradata depending on video content at least in case of x264, but actually this looks logical as the point of multi-pass encoding is to tweak encoding parameters to stay within specified bitrate and quality bounds when possible. Please resort to CRF, CQ or CBR modes when you need fully compatible output.

However, the fact that Avidemux doesn't always warn users about H.264 extradata mismatch is definitely a bug which must be fixed for 2.8.2. Thank you for raising the flag.

[Avidummy]

The extradatas used to match with the older versions on 2-pass.  I've been encoding and joining short trailcam AVIs into long videos since 2017.  Every old video I checked had identical extradata with a size of 41.  The 2.8.x versions increased the extradata size and also introduced the variability.

AVIs encoded and joined using 2.8.0,1,2 will result in extradata size of 47 or 48.

AVIs encoded and joined using 2.7.8 have so far resulted in 44 without deviation.

The exception is a video encoded with 2.8.2 resulting in an extradata size of 47, then re-encoded using 2.7.8 will result in 43 extradata size.  4 is simply subtracted.

A warning about the mismatch would be ok, but I really needed to be warned before processing 100s of AVIs with the expectation I could easily join them later.  Unfortunately I deleted the original AVIs because I had faith avidemux functioned as it always has.  After so many years of doing this I didn't know I needed to check the extradata.

An option to enforce a specified extradata size on 2-pass would be a good thing.  I'd rather it miss the target bitrate than make it impossible to join videos.

MP4joiner will join the mismatched videos and playback is good, but sometimes moving the playback across the joined line results in garbled output, but moving back across the line fixes it.  The joining process is instant so no re-encoding is happening.  The resultant video has the same extradata as the first video in the list to be joined.

The AVIs are not very complex.  Most are black and white nighttime videos of animals eating corn.  There is no obvious reason why extradata would need to be so different.

[eumagga0x2a]

H.264 extradata is generated by libx264 absolutely out of our control regarding its internal decisions.

[Avidummy]

I found that joining the videos using the same 2-pass settings that originally encoded them results in a video almost exactly the same size with no discernable difference in quality, and it plays flawlessly.  The downside is obviously using lots of electricity and generating heat.

I will join the mp4s I've already processed using that method and for future AVIs I will revert back to 2.7.8 until I notice it too produces mismatched extradata.  So far I have not.

Hopefully an option to enforce a specific extradata size will be offered in future versions.  Or maybe some other tweak could be implemented without sacrificing the 2-pass option.  Fingers crossed 

H.264 extradata is generated by libx264 absolutely out of our control regarding its internal decisions.

It looks like we were posting at the same time.  My reply to that is the extradata size has been increasing with the versions of avidemux, and the mismatches never occurred before 2.8.0.  At least, I have not noticed a mismatch yet with versions prior to 2.8.0.

It can be demonstrated by encoding some AVIs on 2.8.0 and 2.7.8 with the same codec settings then comparing the extradata size.  Or a video encoded on 2.8.0 then reencoded on 2.7.8 will have its extradata reduced by 4.

I don't know enough about programming or codecs to say for sure, but it appears something the programmers are doing is influencing the extradata size.

[eumagga0x2a]

x264 (libx264) is a different free software project, Avidemux is just one of its many users.

Avidemux has no say in case of H.264 extradata, we only supply configuration to this (external) library. On my quick test, libx264 generated consistent extradata encoding different videos with same properties (width, height, fps, progressive / interlaced etc.) when in CRF (constant rate factor = constant quality) mode rather than using 2-pass, where the content of videos plays a role.

In short, don't use 2-pass mode when encoding with libx264, use the CRF one instead (2-pass is only beneficial if you need to stay within some fixed bandwidth bounds). I expect for now that this avoids the problem, given that input videos have exactly the same properties, i.e. same dimensions and frame rate.

If in reality CRF doesn't help, things may become interesting (does Avidemux end up filling some of encoder parameters with random garbage? If so, this would be a fixable bug).