Too short error when saving as mp4 (copy mode)

[Garfield]

Hi

When I try to save as mp4, I hit the following error (no issue with mkv).
You cannot view this attachment.

You cannot view this attachment.

You cannot view this attachment.

It is the same for all 3 optimisation options.

System settings

*************************
  Avidemux v2.7.8
*************************
 http://www.avidemux.org
 Code      : Mean, JSC, Grant Pedersen,eumagga0x2a
 GFX       : Nestor Di, nestordi@augcyl.org
 Design    : Jakub Misak
 FreeBSD   : Anish Mistry, amistry@am-productions.biz
 Audio     : Mihail Zenkov
 Mac OS X  : Kuisathaverat, Harry van der Wolf
 Win32     : Grant Pedersen

Build Target: Microsoft Windows (x86-64)
User Interface: Qt (5.12.5)
Operating System: Microsoft Windows (6.2.9200; 64-bit)
Time: Sun Nov 14 15:08:08 2021

0: C:\Program Files\Avidemux 2.7 VC++ 64bits\avidemux.exe

[eumagga0x2a]

Does the problem persist with the latest 2.8.0 nightly? Streaming optimisations have nothing to do with decode timestamps colliding.

[danielrg]

Okay I am resurrecting this because I'm having the _exact_ same issue with the Windows 2.8.1 r220617 (220617_00c978ce708-fflibs 4.4.2 64 bit), and I also had it with 2.8.0 (regular release) - which was why I downloaded the nightly build to try.

With 2.7.5, I didn't have any issues.  So I actually went back to Windows avidemux 2.7.5 to process this batch of videos.

Here's the problem.  I have mp4 files, and they have three AAC tracks of different qualities.  I only want to keep the highest quality AAC track.  So I open the video, select COPY on the highest bitrate AAC track, deselect the other AAC tracks, and also COPY on the H.264 video track.  So this is just a copy into a new mp4 container.  When I save the video, I get this error in an "info" dialog box:

Code Select Expand

Too Short
The saved video is incomplete.  The error occured at 00:08:05,918 (28%). This may happen as a result of invalid time stamps in the video.
I indeed only get 28% of the file saved, then this error.  If I use MP4v2, it works.  If I use MKV, it works.  But MP4, doesn't work.

[danielrg]

I have attached the log for the run that DIDN'T work (2.8.1 6/17 build 64 bit on Windows).  (errorAttemptLog.txt)

ALSO, I WAS able to get it to work with 2.8.1 6/17 build 64 bit on Windows.  In the MP4 configure, I set "Timescale" to 30 kHz, (it WAS on Auto for the failed attempt).  I actually thought it was "30 Hz" at first and thought it had to do with frame rate or something, because the video I'm operating on is 29.97 FPS.  Voila!  It worked just fine, no error.  That log is attached as (successAttemptLog.txt).

I didn't try other timescale settings.  Well, I have 2 workarounds for the error - use 2.7.5, or use 2.8.1 6/17 with the timescale set to 30 kHz.  So I'm probably good.  But there may be a bug buried in there somewhere, hopefully this can help find it!

I have the video files available too if that's helpful.

[eumagga0x2a]

ALSO, I WAS able to get it to work with 2.8.1 6/17 build 64 bit on Windows.  In the MP4 configure, I set "Timescale" to 30 kHz,

This is exactly what this option in the MP4 muxer is for, to force the timebase to 1/timescale to avoid timestamps collisions when the timestamps in the source video doesn't match the declared or calculated timebase. 29.97 fps is (should be) actually 30000/1001 (timebase 1001/30000). But if timing in the source doesn't match that, we end up with duplicated timestamps after rounding to a multiple of timebase in the muxer.

I'll have a look at the logs, thanks.

edit: Downloading attachments doesn't work on this board over HTTP/2 ("spdy") protocol, I need to switch to a browser which allows to disable HTTP/2 for that.

[eumagga0x2a]

Too bad, now even disabling HTTP/2 (network.http.spdy.enabled.http2 --> false in Firefox) doesn't help. Would it be possible for you to use Pastebin, WeTransfer, Mega, Dropbox or Google Drive to share the logs, please?

[danielrg]

Oh bummer on the logs.  At first I tried just putting them in a code block, but it exceeded the post character limit, so I attached them instead.  But I couldn't download the attachments either now that I tried.

To the point, I have placed them on Google docs:

https://drive.google.com/drive/folders/1sBgHahdYCJkge3_eU_4T1JQ4Zukx7YYC?usp=sharing

Let me know if you have any trouble getting them.  I compared the logs of the before and after and there were only a handful of lines differing, pretty much having to do with timebase...

Thanks for taking a look!

[eumagga0x2a]

Thank you very much for the logs. The cause of the problems is an insanely high clock (timebase denominator) in the source MP4 file which should have been a standard, constant 30000/1001 fps but instead was created as a variable fps video with frame duration oscillating between 33366 and 33367 microseconds, thus approximating the target:

Code Select Expand

[indexify] 03:08:36-582 Build Track index, track timescale: 1000000
[indexify] 03:08:36-584 Histogram map has 2 elements.
Frame duration 33366 count: 17109
Frame duration 33367 count: 34218
[indexify] 03:08:36-585 Video index done.
[indexify] 03:08:36-585 Setting video timebase to 1 / 1000000
[indexify] 03:08:36-585 Variable frame rate, 33367 us per frame on average.
Starting with this point, things start to fall apart:

Code Select Expand

[getTimeBase] 03:09:00-385 Ref video 0 is frame-encoded, copy mode: 1
[usSecondsToFrac] 03:09:00-386 1 us -> 0 / 1 (max: 90000)
[getTimeBase] 03:09:00-386 Timebase set to 0 / 1
which is invalid and replaced by

Code Select Expand

[rescaleFps] 03:09:00-394  TimeBase for video 1000/29969
[open] 03:09:00-394 Video stream time base :1000,29969
[FF] Bitrate 127
[initAudio] 03:09:00-394 Using default channel layout 0x3 for 2 channels
[initAudio] 03:09:00-395 Audio has extradata and muxer requires globalHeader, assuming it is done so.
[initAudio] 03:09:00-395 Language for track 0 is eng
[FF] Audio initialized
[open] 03:09:00-395 Timebase In  = 1000/29969
[open] 03:09:00-396 Timebase codec = 1000/29969
[open] 03:09:00-396 Timebase stream = 1/29969
[open] 03:09:00-396 Using 1000 as timebase roundup.
which is destined to fail after approx. (1001/(30000*2))/(1000/29969 - 1001/30000) = (1001*29969)/(1031*2) =~ 14548 frames when the timing becomes half a frame duration off. This is pretty close to the reality where the collision occurs after 14563 frames.

I have the video files available too if that's helpful.

Yes, please. If the source video with the insanely high clock frequency is innocuous enough, it would be great to obtain it as a sample to be able to contemplate a solution.

[danielrg]

Wow that's fascinating.  Thanks for the background on why this happened.

I have placed the source video file in the same location as the logs, let me know if you don't see it.

The file was created by VLC media player using the convert/save function on a network source.

[eumagga0x2a]

Thank you, I've got the sample.

The file was created by VLC media player using the convert/save function on a network source.

Could you please give a hint what kind of network source was involved? The problem may be specific to that source, not to VLC as such.

[danielrg]

Network source is an ".m3u8" URL is derived from the videos from the CDN servicing this website:  https://www.misterrogers.org/watch/

[eumagga0x2a]

I see, thanks. Unfortunately, VLC is at fault here. The source delivers video and audio as MPEG-TS chunks, i.e. it uses by definition (as any MPEG-TS) 90 kHz clock which Avidemux automatically replaces with 30 kHz with a 1001/30000 timebase. The only unusual thing with this stream is each second IDR slice not prepended by SPS and PPS NAL units, not relevant to the issue.

Will look into improving the way we deal with such MP4 files, probably as a special case.

[danielrg]

Interesting, thanks for the background.  I'm definitely a novice when it comes to this stuff.

Is there a better alternative tool for converting streaming MPEG-TS content than VLC?

I'm glad I could get you enough information for the issue.

[eumagga0x2a]

An application needs just to concatenate MPEG-TS fragments. It looks slightly more complicated with audio as the audio track is delivered as another MPEG-TS stream (they are not muxed), so once both streams are written to a pair of MPEG-TS files, ffmpeg should be used to mux them together.

After that, Avidemux would be able to remux the file into e.g. MP4 using a proper timebase.

I mean, if VLC is capable of writing both video and audio streams to a single MPEG-TS file, you should feed it to Avidemux for further cutting or processing.