Consistent Invalid timestamps error while muxing from mkv to mp4.

[Amey8a]

Hi, I have been facing a serious issue which is not mentioned anywhere on this forum.
let me explain, I have multiple files in mkv container with avc/aac. I want them to be in mp4 container (just because I like it that way). So, I was using avidemux 2.7.5 release for a long time, but it has timestamps rounding off issues that makes all constant frame rate videos to variable frame rates whenever using copy mode and this issue is consistent with all 23.976 fps videos (most probably due to the way avidemux handles timestamps instead of using timescale). Now, I am using the latest nightly builds of avidemux from past week to handle mkv to mp4 muxing which for some videos work nicely but for most of the videos I get this error to be specific (exactly this error with all the video files with 23.976 fps).

"The saved video is incomplete. The error occured at 00:43:29,335 ("some two digit number"). This may happen as result of invalid time stamps in the video."

This happens when using mp4 container. If I use mp4v2, muxing goes normally but the result is again variable frame rate with min and max delta values in mediainfo and non monotonous increasing decoding timestamp error in ffmpeg. The error is consistent with h264 and h265 codecs. Encoding the video with resampling sure solves this issue but it is a very time consuming option.

In ffmpeg I am using vfrdet option to verify variable frames in a video file. Any help is appreciable. I am sending avidemux error, mediainfo output and ffmpeg output for the same video file.

[eumagga0x2a]

I'm going to investigate further improvements for the reconstruction of the original time base of the video stream in the MKV demuxer during the next release cycle. For now, please try to force the 1/24000 time base in the settings of the MP4 muxer. This won't make the video constant fps and it won't help if you saw "sorting error" and "duplicated timestamps" in admlog.txt when loading the MKV, however.

In general, admlog.txt from just loading the source MKV would show what is actually going on here. Looking at the long time before rounding to the multiple of time base leads to a timestamp collision, this might happen when the demuxer thinks that the video stream matches a 1000/23976 time base better than the standard 1001/24000 and/or (mis)detects that recalculating all timestamps from a time base is not viable.

[Amey8a]

Yes, forcing timescale behaves just as mp4v2 i.e. avidemux bypasses the timestamp error keeping invalid dts values.

One more thing, if I remux mkv made by mkvtoolnix into mp4 using avidemux forcing timescale the error come back as before and if I remux mkv made by avidemux then forcing timescale bypasses the timestamp error.

I checked admlog for the good mkv files that gave sum of error as zero for proper 24000/1001,  while bad mkv throws multiple errors for all the options available for the given fps. Majority of videos are in mkv container which deal with pts only, making mediainfo shows constant fps everytime. I think I have to do experiments with avidemux, ffmpeg and mediainfo to improve my understanding on these topics.

Thanks for pointing me to admlog just after loading the video, it will be helpful for better debugging.

[eumagga0x2a]

Your nightly is not up-to-date, it doesn't include some changes to the Matroska demuxer targeted precisely at the problem you experience. Please update and re-test.

[Amey8a]

So, I did some testing using everything up-to-date (avidemux, mkvtoolnix, ffmpeg). For testing I have selected the same previous video file. Here's what I did.

Source video file :- sample.mkv (Random video downloaded from internet)

1. Extracted mkv to remove whatever container header there was. Muxed raw codec files to mkv using mkvtoolnix.
2. Muxed to mkv with avidemux.

Source video file :- sample.mp4 (changed container from sample.mkv using ffmpeg)

3. Muxed to mkv with mkvtoolnix.
4. Muxed to mkv with avidemux.

Source video file :- sample.mp4 (changed container from sample.mkv using avidemux)

5. Muxed to mkv with mkvtoolnix
6. Muxed to mkv with avidemux

At this point I have 6 mkv file variants of the same source and these are my results.

a) No. 1 to 4  selected 24000/1001 as viable fps with sum of error=0, No. 5 to 6 selected 23976/1000 as viable fps with sum of error not equal to zero for any fps option.

b) Out of all six variants no.5 muxed correctly without any avidemux or ffmpeg error with variable frame rate instead of constant frame rate of source video file.

c) No. 1, 2, 3, 4, 6 all had non monotonous dts error in ffmpeg and muxed without avidemux error using mp4v2 container.

Only attaching admlog for no.1, 3, 5 as more than 4 attachments are not allowed.

[eumagga0x2a]

In samples 1 and 3 Avidemux finds the standard 1001/24000 time base applicable and forces all timestamps to a multiple of this timebase. There should not be any problems saving 1 and 3 in copy mode using the MKV or MP4 muxer (unless there is some hidden bug in mkvHeader::enforceFixedFrameRate). Have I understood correctly that saving these files in copy mode fails?

In sample 5, Avidemux detects too high deviation from the standard 1001/24000 time base while the deviation for 1000/23976 turns out to be lower, so that it forces timestamps to multiple of 1000/23976 and sets DTS from sorted list of PTS.

I would have reduced all the testing setup to remuxing the source mkv file with ffmpeg to mp4, loading this mp4 in Avidemux and checking admlog.txt to verify that ffmpeg has used the 1001/24000 time base* and how regular the file is (Avidemux prints a histogramm if time base numerator is >= 100 ticks). If it turns out to be perfectly regular, this source mkv is usable for further testing, especially Avidemux should also force timestamps to multiple of time base. Saving this file in copy mode in Avidemux should also succeed.

Only attaching admlog for no.1, 3, 5 as more than 4 attachments are not allowed.

I'd put them all into a single archive (zip, gzip or 7zip).

*) Time base is not necessarily equal inverted fps, as it only means that all timestamps can be expressed as a multiple of the time base: fps <= (1/timebase)

[Amey8a]

No. 4 exactly does what you mentioned for test setup. I have included the zip file with all test admlog. Now, I am starting to think that the source file is badly muxed,  since some other mkv files gave clean mp4 file with no issue if not all. Even after correct fps calculation, there might be duplicate timestamps created by old muxers which were not using timescale but using rounding of close frame timings.

For example, old avidemux used rounding of to get correct ratio like 23.976 by considering every other frame timing in calculation and that leads to variable frame rate in case of mp4 files. When these mp4 files are muxed back to mkv, the mkv container will show constant frame rate due to only using pts. Now, if these mkv files are muxed back to mp4 using latest avidemux builds the result will be dts collision and thus the error.

[eumagga0x2a]

What is missing is admlog.txt from loading the mp4 generated by ffmpeg from the source mkv.

[Amey8a]

What is missing is admlog.txt from loading the mp4 generated by ffmpeg from the source mkv.

Here it is.

[eumagga0x2a]

I am starting to think that the source file is badly muxed,  since some other mkv files gave clean mp4 file with no issue if not all. Even after correct fps calculation, there might be duplicate timestamps created by old muxers which were not using timescale but using rounding of close frame timings.

I don't think so because in such a case ffmpeg would have failed to save it in stream copy mode to mp4.

For example, old avidemux used rounding of to get correct ratio like 23.976 by considering every other frame timing in calculation and that leads to variable frame rate in case of mp4 files.

I have no idea about versions prior to 2.6, around the time of ~ 2.7.1 there were two independent steps of rounding:

one was default rounding to multiple of time base in libavformat-based muxers, which caused frequent failures to save videos. I disabled this rounding for this reason.

The other was rescaling timestamps in microseconds as used by Avidemux into ticks of time base as used by ffmpeg. This function had a rounding bug which made pseudo-variable fps from perfectly regular files. I identified and fixed it only recently, in late fall of 2019.

Subsequently a capability to track the time base all the way from demuxer to muxer was added to Avidemux in order to choose time base better matching the properties of the stream. This allowed to re-enable the default rounding to multiple of time base, which gave back a constant fps output for constant fps video streams but (mostly) without collisions.

At least this is how it now works for mp4 in- and output. Matroska is much more difficult as it uses just a 1/1000 time base and the demuxer has to decide whether it actually should represent a constant fps video with a standard time base.

When these mp4 files are muxed back to mkv, the mkv container will show constant frame rate due to only using pts. Now, if these mkv files are muxed back to mp4 using latest avidemux builds the result will be dts collision and thus the error.

There should not be any dts collisions if there were no pts collisions in the source mkv. Anything else is our bug.

Code Select Expand

[indexify] 14:48:47-245 Build Track index, track timescale: 16000
[indexify] 14:48:47-250 Histogram map has 3 elements.
Frame duration 656 count: 47605
Frame duration 672 count: 115654
Frame duration 679 count: 1
[indexify] 14:48:47-250 Checking whether we need to fix jitter from rounding errors...
[indexify] 14:48:47-250 No, nothing we can do.
[indexify] 14:48:47-252 Video index done.
[indexify] 14:48:47-252 Setting video timebase to 1 / 16000

The source mkv is not regular.

[eumagga0x2a]

The last thing to try* would be loading the source mkv in Avidemux and saving it in copy mode to mkv or mp4 (using the MP4 muxer).

*) In sense looking into admlog.txt from these steps.

[Amey8a]

The last thing to try* would be loading the source mkv in Avidemux and saving it in copy mode to mkv or mp4 (using the MP4 muxer).

*) In sense looking into admlog.txt from these steps.

I have tried that as my first step, the source mkv was also made with mkvtoolnix (to be specific 38.0.0 64bit)

The source mkv is not regular.

Thats what I think. May be I will download another copy from different source for testing purpose. Thanks for clarifying many things.

One question though, do we have a pts/dts recreation option by discarding the one that came with the video to fix the pts/dts collisions without encoding i.e in copy codec mode?, is it even possible?

In ffmpeg I have tried -fflags +ignpts, -fflags +igndts, but no effect whatsoever. Best option always seems like encoding the video in all the cases.

[eumagga0x2a]

The last thing to try* would be loading the source mkv in Avidemux and saving it in copy mode to mkv or mp4 (using the MP4 muxer).

I have tried that as my first step, the source mkv was also made with mkvtoolnix (to be specific 38.0.0 64bit)

Oh, I see, but the log says that Avidemux has successfully forced timestamps into a perfectly regular 1001/24000 constant fps pattern. Does subsequent saving in copy mode as MKV or MP4 fail?

The source mkv is not regular.

Thats what I think. May be I will download another copy from different source for testing purpose.

Let's say, at least ffmpeg doesn't try to make it regular.

One question though, do we have a pts/dts recreation option by discarding the one that came with the video to fix the pts/dts collisions without encoding i.e in copy codec mode?, is it even possible?

We are doing exactly that when forcing constant fps in the MKV demuxer. Just not arbitrarily as it would result in loss of A/V sync. If saving a MKV video which doesn't have duplicated pts in copy mode ends with a dts collision, there is a bug in our code. It means that we propagate a time base down the chain which is not valid for this video. Or the code responsible for creation of dts in the MKV demuxer is faulty.

[Amey8a]

Oh, I see, but the log says that Avidemux has successfully forced timestamps into a perfectly regular 1001/24000 constant fps pattern. Does subsequent saving in copy mode as MKV or MP4 fail?

Mkv to Mkv muxing never fails, and if avidemux detects input mkv as 1000/23976 then muxing to mp4 also does not fail in some cases. The results are not consistent though, they vary from video to video like some 1001/24000 videos came out clean in mkv to mp4 muxing while others shows timestamp issue.

My biggest question is why all the videos shows timestamp error on exactly same time interval that I have shows in my first post i.e.
00:43:29,335 exact?. I remember that I muxed many of my old videos from mkv to mp4 long ago with avidemux 2.7.5, and that is the only link common between them.

We are doing exactly that when forcing constant fps in the MKV demuxer. Just not arbitrarily as it would result in loss of A/V sync. If saving a MKV video which doesn't have duplicated pts in copy mode ends with a dts collision, there is a bug in our code. It means that we propagate a time base down the chain which is not valid for this video. Or the code responsible for creation of dts in the MKV demuxer is faulty.

I will try extracting the timestamps from the source video using InviskaMKV extract to check if the pts are duplicated there or not.

[eumagga0x2a]

Oh, I see, but the log says that Avidemux has successfully forced timestamps into a perfectly regular 1001/24000 constant fps pattern. Does subsequent saving in copy mode as MKV or MP4 fail?

Mkv to Mkv muxing never fails, and if avidemux detects input mkv as 1000/23976 then muxing to mp4 also does not fail in some cases. The results are not consistent though, they vary from video to video like some 1001/24000 videos came out clean in mkv to mp4 muxing while others shows timestamp issue.

In other words, muxing as MP4 fails, right? I'll try to generate a 1001/24000 video myself any see whether I can reproduce the failure.

My biggest question is why all the videos shows timestamp error on exactly same time interval that I have shows in my first post i.e.
00:43:29,335 exact?. I remember that I muxed many of my old videos from mkv to mp4 long ago with avidemux 2.7.5, and that is the only link common between them.

I don't know yet, will need to investigate when I get a suitable sample mkv. This is too soon for the difference between 1001/24000 and 1000/23976 to get near the half of frame duration.

I will try extracting the timestamps from the source video using InviskaMKV extract to check if the pts are duplicated there or not.

PTS are not duplicated, else it would have been noted in Avidemux log.