Audio sync issues on 2.7.6

[jr]

Worked in 2.7.5, but now in 2.7.6 the audio doesn't match.

Webcam recordings from Quicktime on MacOS that are a number of hours long often result in files where video frames don't automatically advance when played in Windows on MPC-HC or VLC (works fine in iTunes and Windows Media Player though). Avidemux 2.7.5 fixed this well:

• Open original MOV in Avidemux
• Save desired segment in AVI container, acknowledging the Avidemux AVI container warning
• Open the AVI in Avidemux. Avidemux produces the prompt "This video contains B-frames, but presentation time stamps (PTS) are either missing or monotonically increasing. Avidemux can try to reconstruct correct PTS by decoding the entire video. This may take a lot of time. Proceed?"
• Save the file (can be any container - MP4, etc.)

The resulting file sizes also differ between versions. Using a segment that is 7:43 long:

• AVI 2.7.5: 610 MB, 84:13:58 long (per Windows, because of bad timestamps?)
• MP4 2.7.5: 609 MB, 7:43 long
• AVI 2.7.6: 613 MB, 52:19:57 long (per Windows, because of bad timestamps?)
• MP4 2.7.6: 612 MB, 9:36 long (per Windows)

If you want more details or something rewritten, let me know.

The only thing possibly related that I found was https://avidemux.org/smif/index.php/topic,19190.0.html but it also doesn't have a ton of detail.

Hopefully this is somewhat helpful, although I realize it might not be enough, apologies.

Avidemux tested on Windows x64. I love Avidemux - thank you all for your work on it! Last year I thought to see if I could contribute to the 2.7.5 codebase, but the Windows build setup process was daunting for me, sorry. 

Detail if interested:

• When these non-advancing-video files are opened in Avidemux, the first frame is listed at between the 6 and 7 minute mark, presumably because the timestamps are messed up.
• The issue with long Quicktime webcam records often having issues has been happening since I started using it last year.
• I presume that Avidemux's 'magic' is happening in the reconstruct PTS step, but for whatever reason, the only way Avidemux notices the errors is when opening an AVI file, not other container types.
• Quicktime webcam records are supposed to be 720P, 30 FPS. They use the MOV container, with H264 encoding.

[jr]

None of these are the core fault of Avidemux, but just as side notes in case anyone also doing long MacOS webcam recordings stumbles across this thread, I've also seen two other issues with Mac webcam recordings:

• Video recording (but not audio) will sometimes cut out for 5 or 10 minutes, then resume. This seems to happen once every hour or two. Advancing frame by frame will show perhaps a gap, then another gap, then video resumes.
• There seems to be perhaps a 5-10% chance per hour of what I call a recording corruption point. If this occurs, trying to seek to a point beyond the corruption point will result in Quicktime freezing / sucking up CPU cycles. In Avidemux: 1) clicking on a time beyond the corruption point simply shows the last non-corrupted time in the time status panel. 2) Typing a time beyond the corruption point results in Avidemux freezing - perhaps it would resume if I let it continue for many minutes or hours. 3) Advancing frame by frame beyond the corruption point is possible, and does show valid frames. Choosing a start point beyond the corruption point in Handbrake is possible. To obtain video from after the corrupted point, best approach seems to be: Lossless export from last selectable frame in Avidemux (to reduce "Seeking to start point" encode delays in Handbrake), then re-encode from the real desired start time in Handbrake.

[eumagga0x2a]

Worked in 2.7.5, but now in 2.7.6 the audio doesn't match.

What does "doesn't match" mean? A constant offset (easily correctable)? A growing desync (not really correctable)?

Webcam recordings from Quicktime on MacOS that are a number of hours long often result in files where video frames don't automatically advance when played in Windows on MPC-HC or VLC (works fine in iTunes and Windows Media Player though).

For starters, please provide admlog.txt located at %localappdata%\avidemux\ from just starting Avidemux, loading the broken MOV file produced by QuickTime and closing the application using the latest available official 2.7.7 nightly:

https://avidemux.org/nightly/win64/ (builds cross-compiled for Windows on Linux with MinGW)
https://avidemux.org/nightly/vsWin64/ (builds compiled on Windows with Microsoft Visual Studio)

Last year I thought to see if I could contribute to the 2.7.5 codebase, but the Windows build setup process was daunting for me, sorry. 

I would recommend just using Linux for Avidemux development (and for generating builds for Windows), it is both simple and documented. It is also easy to build Avidemux from source on macOS, but no cross-compilation for Windows is possible.

When these non-advancing-video files are opened in Avidemux, the first frame is listed at between the 6 and 7 minute mark, presumably because the timestamps are messed up.

I agree, but please retest with the latest nightly to be sure.

The issue with long Quicktime webcam records often having issues has been happening since I started using it last year.

Are there any alternatives? Of course, it is possible that errors are generated by underlying HW accel. layers.

I presume that Avidemux's 'magic' is happening in the reconstruct PTS step, but for whatever reason, the only way Avidemux notices the errors is when opening an AVI file, not other container types.

Saving as AVI discards all presentation timestamps, so that the only way to reconstruct the correct display order of frames is to decode the stream (and because libavcodec doesn't provide means to decode just slice headers – which would be very fast, – we have to decode macroblocks as well, which is damn slow, throwing out the output). To trigger timestamp invalidation and reconstruction prompt when loading a MOV / MP4, the stream would need to have a timestamp irregularity (a disagreement between frame order derived from POC vs derived from PTS) within the first 100 frames.

[eumagga0x2a]

NB: In case of CleanTalk external antispam service blocking replies for no reason, please don't hesitate to send the reply as PM. There is no reliable remedy for CleanTalk false-positives. Posting from a private browser window and/or from a different IP address reportedly helps sometimes.

[jr]

Thank you so much for your very kind and helpful reply, including explaining how some things work.

For starters, please provide admlog.txt located at %localappdata%\avidemux\ from just starting Avidemux, loading the broken MOV file produced by QuickTime and closing the application using the latest available official 2.7.7 nightly:

https://avidemux.org/nightly/win64/ (builds cross-compiled for Windows on Linux with MinGW)
https://avidemux.org/nightly/vsWin64/ (builds compiled on Windows with Microsoft Visual Studio)

I wasn't sure if you wanted me to try both versions but I did, and the issue seemed to persist in both. I tried avidemux_2.7.7 r200813_win64.exe and Avidemux_2.7.7 VC++ 64bits 200813.exe. I'm providing the log file from one of them. I'm uncertain how much is normal, but the following lines stood out to me as likely at least being related to the time in Avidemux starting between 6 and 7 minutes on all files that are broken like this:

Code Select Expand

...
[addReferenceVideo] 20:00:08-362 max increment 00:06:52,350 = 412350200 us for frame 74200
...
[addReferenceVideo] 20:00:08-362 The first frame has a PTS > 0, adjusting to 412350 ms
[updateStartTime] 20:00:08-363 Using pts2=00:06:52,350 to get dts, as this video does not start at zero
...


What does "doesn't match" mean? A constant offset (easily correctable)?

I did some more testing and it does not seem to be a constant offset. When I save a 13 minute section, the audio seems about in sync at the beginning but seems to off by about a minute towards the end.

The issue seems to occur in the MOV > AVI step, as I tried:

• The MOV > AVI step in 2.7.7, then the AVI > MP4 step in 2.7.5, but the audio was out of sync.
  
• The MOV > AVI step in 2.7.5, then the AVI > MP4 step in 2.7.6 and 2.7.7 and the audio seemed fine. (The 2.7.7 MP4 was 162 KB smaller than the 1 GB 2.7.6 32-bit MP4, so the files weren't exactly the same. (32-bit so that I could install both on the same Windows.))

Are there any alternatives? Of course, it is possible that errors are generated by underlying HW accel. layers.

Last year I tried one other app for recording, and it would occassionally randomly stop recording, which I presumed was that app's response to the same cause as video temporarily cutting out in QuickTime.

But it's still rather interesting to me that it happens on both my 2017 and 2018 MacBook Pro, including after a fresh MacOS installation.

Again, the underlying root cause isn't the fault of Avidemux, and Avidemux 2.7.5 can already 'fix' it, but if you happen to recommend I try a different solution off the top of your head, let me know.

Regardless, thank you!

You cannot view this attachment.

[eumagga0x2a]

I wasn't sure if you wanted me to try both versions but I did

While it for sure didn't harm, I simply pointed to both possible sources of recent official Avidemux builds for Windows. Sometimes only one is refreshed (or none like now, with latest builds being two weeks old).

the issue seemed to persist in both. I tried avidemux_2.7.7 r200813_win64.exe and Avidemux_2.7.7 VC++ 64bits 200813.exe. I'm providing the log file from one of them.

Thank you, the log is from the former, which is fine.

I'm uncertain how much is normal, but the following lines stood out to me as likely at least being related to the time in Avidemux starting between 6 and 7 minutes on all files that are broken like this:

Code Select Expand

...
[addReferenceVideo] 20:00:08-362 max increment 00:06:52,350 = 412350200 us for frame 74200...
[addReferenceVideo] 20:00:08-362 The first frame has a PTS > 0, adjusting to 412350 ms
[updateStartTime] 20:00:08-363 Using pts2=00:06:52,350 to get dts, as this video does not start at zero
...

Yes, this is the end result. Peculiarities start at line 458 at the latest:

Code Select Expand

[indexify] 19:59:29-244 Build Track index, track timescale: 30000
[indexify] 19:59:29-252 Histogram map has 2 elements.
Frame duration 1000 count: 266342
Frame duration 12370506 count: 2
[indexify] 19:59:29-255 Video index done.
[indexify] 19:59:29-255 Setting video timebase to 2 / 30000
We have an almost perfectly regular timing with two incidents of a huge gap of exactly 12370506 ticks at 30 kHz (that 6 minutes ~52 seconds) which would not pose a problem unless all composition to sample entries (ctts) were correct, but it looks like one of these entries would specify a presentation timestamp far earlier as the decode timestamp (probably for the first frame after the gap).

As pts cannot be less than dts, we end up with a huge first frame delay (line 1222):

Code Select Expand

[shiftTimeBy] 20:00:08-321 MP4, Must increase pts by 412350200 us
It might be worth exploring a strategy of invalidating extreme outlier ctts entries if only it were easily possible to draw a line.The reason for repeating messages

Code Select Expand

[adm_lavLogCallback] 20:00:05-520 [lavc] Invalid UE golomb code
likely from decoding slice headers might be that non-IDR frames are marked as IDR in the H.264 stream.

What does "doesn't match" mean? A constant offset (easily correctable)?

I did some more testing and it does not seem to be a constant offset. When I save a 13 minute section, the audio seems about in sync at the beginning but seems to off by about a minute towards the end.

Looking at line 436

Code Select Expand

[H264] No timing information
[extractSPSInfo_mp4Header] 19:59:29-157 Width2 : 1280
[extractSPSInfo_mp4Header] 19:59:29-157 Height2: 720
[extractSPSInfo] 19:59:29-157 width:1280
[extractSPSInfo] 19:59:29-157 height:720
[extractSPSInfo] 19:59:29-157 fps1000:25000
(1000/25000 timebase = 25 FPS due to missing timebase info in codec extradata is a fallback value, wrong for this stream) and at the average FPS very far off from the actual frame increment, using AVI as a workaround is definitely a bad idea.

Would it be possible for you to provide an innocuous similarly broken MOV file for me to play with, please? If it is too large for WeTransfer or similar services, it should be fine to split it with a file cutter into 1 GiB sized chunks which I would concatenate after download.

I hesitate to try something out without the ability to test the actual effect of my changes.

The 2.7.7 MP4 was 162 KB smaller than the 1 GB 2.7.6 32-bit MP4, so the files weren't exactly the same. (32-bit so that I could install both on the same Windows.)

You can have as many Avidemux installations (Avidemux is always a portable app, so just unpack the installer or use the ZIP version and copy the directory whenever you want within local storage) on one system as you like, just never run them simultaneously.

The 32-bit legacy compatibility builds are significantly different code-wise from the normal ones starting with an ancient FFmpeg version, the last one still compatible with Windows XP, don't use them unless you have no other choice.

Are there any alternatives? Of course, it is possible that errors are generated by underlying HW accel. layers.

Last year I tried one other app for recording, and it would occassionally randomly stop recording, which I presumed was that app's response to the same cause as video temporarily cutting out in QuickTime.

But it's still rather interesting to me that it happens on both my 2017 and 2018 MacBook Pro, including after a fresh MacOS installation.

Different MBPs, but the same faulty webcam hardware? Did you use the built-in one or an external solution?

[jr]

Different MBPs, but the same faulty webcam hardware? Did you use the built-in one or an external solution?

The built-in webcams on both of my MBPs, yes ('17 and '18 model years). I assumed it affected all MBPs of at least those years.

We have an almost perfectly regular timing with two incidents of a huge gap

It is interesting that the amount is always 6:52.

(1000/25000 timebase = 25 FPS due to missing timebase info in codec extradata is a fallback value, wrong for this stream) and at the average FPS very far off from the actual frame increment, using AVI as a workaround is definitely a bad idea.

I hesitate to try something out without the ability to test the actual effect of my changes.

I'm guessing there was some change related to this behavior between 2.7.5 and 2.7.6, but as you say, best for me just to give you a file for now.

I just recorded a test video that has the first issue at https://drive.google.com/drive/folders/1hRlFNz9tmCxw5zL3l2a_MSoOwSpJ6j_0
I recorded a talking clock to hopefully make it easiest to see issues. (I didn't see a way to make it smaller offhand, sorry.)

After having had time to look at my video if you want, if you have a suggestion for a different workaround than MOV>AVI and AVI>MP4, let me know, thanks!

The rest of my reply is just sidenotes; you don't need to read or reply.

1000/25000 timebase = 25 FPS due to missing timebase info in codec extradata is a fallback value, wrong for this stream

I was able to cross-compile a Windows exe to play around with from Windows Subsystem for Linux (Ubuntu 16.04), thanks. (I tried figuring out where the fallback value was specified in the code but couldn't in a little bit of searching haha.)

The 32-bit legacy compatibility builds are significantly different code-wise from the normal ones starting with an ancient FFmpeg version, the last one still compatible with Windows XP, don't use them unless you have no other choice.

Ah, thank you for letting me know! I had processed a few final files with it before your reply, but I still kept the source files so I can re-process the originals, thank you!

You can have as many Avidemux installations (Avidemux is always a portable app, so just unpack the installer or use the ZIP version and copy the directory whenever you want within local storage) on one system as you like...

Sorry, I was unclear. Yes, I saw a 2.7.7 zip, although it isn't even necessary since the Windows-exe-built-within-Linux out of the box will do a side-by-side installation with stable Windows releases. While I am providing you the files you need to investigate, I can use a temporary workflow of 2.7.5 for the MOV>AVI step, and 2.7.6 for the AVI>MP4 step (although I suppose I could also just do everything in 2.7.5).
I didn't see a ZIP for 2.7.6, and I couldn't get of the 3 methods from https://www.codetwo.com/kb/msi-from-exe/ to work to unpack the EXE installer, but just copying the files from /Program Files/ to a new directory worked to get multiple versions on the same system, thanks.

[eumagga0x2a]

I just recorded a test video that has the first issue at https://drive.google.com/drive/folders/1hRlFNz9tmCxw5zL3l2a_MSoOwSpJ6j_0
I recorded a talking clock to hopefully make it easiest to see issues. (I didn't see a way to make it smaller offhand, sorry.)

Thank you very much for the admittedly huge sample (probably due to a very high noise level in low light conditions). I've looked at it and I'm not optimistic that there is a way to achieve A/V sync. If a detour via AVI works in 2.7.5, it can be only by accident (in 2.7.6 it can't work because it calculates AVI timebase based on the true average FPS instead of the minimum time increment, which doesn't fit when there are significant gaps in timestamps).

First of all, the video track is not 30 fps as it claims to be. Making it standard 30000/1001 (29.97) reduces the rate of A/V desync already quite a lot, but unfortunately it is still off by a small amount (about 2 seconds per hour), maybe 30000/1002 might be a better approximation.

1000/25000 timebase = 25 FPS due to missing timebase info in codec extradata is a fallback value, wrong for this stream

I was able to cross-compile a Windows exe to play around with from Windows Subsystem for Linux (Ubuntu 16.04), thanks. (I tried figuring out where the fallback value was specified in the code but couldn't in a little bit of searching haha.)

This is great, 16.04 is a bit old, but as long as the current MXE can be built on top of it, it doesn't matter.

The fallback value is specified in avidemux_core/ffmpeg_package/patches/libavcodec_h264_parser.c.patch:72 awaiting a better solution.

I didn't see a ZIP for 2.7.6, and I couldn't get of the 3 methods from https://www.codetwo.com/kb/msi-from-exe/ to work to unpack the EXE installer

7zip is capable of extracting the payload of the NSIS installer used by MinGW ("win64") builds. Regarding VC++ QtIFW installer, their payload is stored as a few concatenated(?) 7zip archives, so it should not be too complicated to look for the magic number 37 7A BC AF 27 1C in the binary and feed the data to 7z.

Unless the existing recordings are of great value and need to be rescued, I would strongly recommend to aquire a different piece of hardware (and software, if existing MBPs have to remain involved) for the task. I think that already budget cameras would surpass MBP by far here.