I've grown to love a really slick encoding tool. It's part of the Replay suite from Applian. The tool is Replay Converter. It's up to version 6 now. In general, I'm using it with a slightly customized template which includes the "same quality" switch. In my opinion, this program does a very fine job packing all possible quality into a video and still making it the smallest possible size. Part of that ability is, I believe due to them setting large GOP sizes, which I've chaffed against in the past. But now that I don't try to edit the resulting files, I don't mind.
What I mind is that sometimes a file I feed into it has the audio sync off a bit. No problem, I just load it up into Avidemux and do some plus or minus shifting in ms. That's not what I mind. What I mind terribly is that probably 1/3 to 1/2 of the very finely encoded files I try to audio-shift in Avidemux, Avidemux barfs with this disgusting and really useless message, "Too short - The video has been saved but seems to be incomplete." I have a gripe at two levels.
1. I can find the GOP that was allegedly "bad", and can cut it out. However, without cutting it out, the video will play fine in the timeline, both at normal speed and even single frame stepping. I can see no flaw in any frame that RVC created. I can (and have at times) cut out the one GOP just to save the rest of it AND accomplish my audio timeshift. I can think of no excuse for such an otherwise fine program as Avidemux to EVER give this "Too short" message, at least in the context of a perfectly good file. Do you say, "there's something flawed in it"? Really then TELL ME WHAT THAT FLAW IS!!! If you're going to abort a simple remuxing (and I've never seen other remuxers from my Mpeg2 days that EVER blew up), then you have a moral obligation to at least inform the user as to what the problem was! I realize I'm being dogmatic here, but I believe I am in the right. Back when I had a real life and career, I was a programmer and considered by many who worked with me to be a good one. I learned you ALWAYS give some kind of meaningful information to the user when there's a problem. ALWAYS. Sometimes to not overwhelm them with tech-talk, you may have to give an error code and tell them to look it up or talk to customer support. This "too short" message HAS to give information and it does not. It's a serious flaw and no real excuse for it. We sometimes work hard to create content and to have to throw it away because either the program can't handle "good data" and won't even tell us why, is inexcusable.
2. The broader gripe I have, I have had for a long time. I feel the ONLY time "too short" should EVER come out at us is when the storage space has run out. There WILL be damaged files out there. There just will be. Granted, most of them will come from something other than one's own video camera. But still. A tool like this ought to ALWAYS go to the end. If there are garbage chunks, the program should find a way to handle it, of course giving warnings about it to the user, along with relevant data items (for example, "Garbage data found at 1:03:39.027; dropping 13 frames") or perhaps giving an option, either in preferences or an interruption during the muxing itself, to replace lost frames with black. I think most of the video streams have some kind of presentation time stamps that correlate video frames to audio packets. This should be usable by Avidemux to find the next good data after a bad spot and resume normal muxing/copying. I realize garbage detection and processing is not a pleasant thing to program for, but it is something that should be a high priority.
I did do searching for "too short" to see how this has been discussed before, but it appears to hardly been discussed at all. Yet I cannot believe that there are not hundreds of people who would echo my concerns. I do not have a particular propensity to acquired garbaged files (but I do catch a lot that is streamed). I'm in the middle of a crunch to get out a lot of files for a project I'm trying to do to stay afloat, and I need to not keep running into these inappropriate brick walls. Every time I have to adjust audio sync, I dread that "too short" will appear in a perfectly good file, usually (but not always) encoded cleanly and excellently by Replay Converter. What do I do now? Do I release the video with the audio needlessly delayed by 167 ms, or do I compromise content by cutting out a 4-sec. GOP? I hate both options. Please consider addressing at least my 1st gripe. It should be EASY to fix. Admittedly #2 will be harder.
I cannot find a donate button, but if there was, and if I could FULLY trust Avidemux to "copy/copy" to the end ALWAYS (except for out of storage space), I'd make a donation. It has become a VERY fine program, but this serious flaw greatly hurts its usefulness. I frankly am very surprised it's gone this far and long without someone bring these things up.
Well, if the program can't handle PTS issues in its current form, it should have 2 extras options:
1. Diagnose
2. Repair
If Diagnose found one or more issues it would offer to run a Repair pass. If you did not reply within 10s, (say as an option under Preferences), it would automatically run Repair for you. It would then write out a copy of the original, repackaged as MKV/MP4, if necessary, which it would then happily accept for A/B editing.
BTW, have you tried changing the timing offset using MKVToolNix GUI?
Dan.
This is MP4 I don't use MKVs. But I do like your suggestions for correcting Avidemux. It's a start.
I presume you know the timing offset for an affected clip. Is it consistent for all files from this source? Or does it vary depending on how many "bad bits" are in the encode?
If it's consistent, you could write a batchfile to run FFMPEG and just drag-and-drop an affected file onto the icon of say -300mS.bat.
Look at the batchfile here as a possible drag-and-drop template to run FFMPEG. Change the instances of .mkv to .mp4 so FFMPEG knows what container to use: http://avidemux.org/smif/index.php/topic,17390.msg78865.html#msg78865
Look here for a description of altering the timing offset using FFMPEG: https://wjwoodrow.wordpress.com/2013/02/04/correcting-for-audiovideo-sync-issues-with-the-ffmpeg-programs-itsoffset-switch/
Quote from: dosdan on October 10, 2017, 05:24:57 AM
I presume you know the timing offset for an affected clip. Is it consistent for all files from this source? Or does it vary depending on how many "bad bits" are in the encode?
If it's consistent, you could write a batchfile to run FFMPEG and just drag-and-drop an affected file onto the icon of say -300mS.bat.
Yes, I have estimated (using playback of video with shifted audio in Avidemux which works well at that point) that my audio needs to come 133 ms sooner, therefore I set the delay to -133ms. No it's neither. This file as originally supplied to the RVC encoder was off by that much. I don't think there are "bad bits". But I'll still consider your bat file idea. I can change the shift value with each file I need to shift.
But this is still a kludgy workaround of the severe and unjustified weakness of Avidemux as described. (I say kludgy as I feel all bat file operations are kludgy at best, esp. when it's more than 1 or 2 parameters for a command, which I realize this case has to be.) I didn't know that we could drag/drop onto an icon (shortcut) for a bat file. Cool.
Quote from: TCmullet on October 10, 2017, 06:07:55 AM
>8>8 of the severe and unjustified weakness of Avidemux as described. (I say kludgy as I feel all bat file operations are kludgy at best, esp. when it's more than 1 or 2 parameters for a command, which I realize this case has to be.) >8 >8
A harsh comment for
QuoteAvidemux is a simple tool for simple video processing tasks. The keyword here is simple: it does not offer tools like a timeline, multitrack editing, you cannot freely move or splice audio and video clips from various sources. However, Avidemux allows you to do elementary things in a very straightforward way.
source: Avidemux Wiki (http://www.avidemux.org/admWiki/doku.php?id=using:quickstart)
My user opinion of course.
Quote from: TCmullet on October 10, 2017, 06:07:55 AM
But this is still a kludgy workaround of the severe and unjustified weakness of Avidemux as described. (I say kludgy as I feel all bat file operations are kludgy at best, esp. when it's more than 1 or 2 parameters for a command, which I realize this case has to be.) I didn't know that we could drag/drop onto an icon (shortcut) for a bat file.
If the timing offset is consistent and the mapping is the same for all the affected files, then there is no real effort involved. You just drag-and-drop one clip or a whole bunch of them onto -133.bat, and then look in the "converted" sub-directory beneath where the original files were located. Alternatively, it would be trivial to leave the converted files in the same directory, but ending then in "_converted.MP4" to distinguish them for the original files. Or, if there was no higher errorlevel reported on an offset conversion pass (I think errorlevel 0 means "OK"), you could automatically delete the original.
Yes, in a perfect world ADM would be bullet-proof. But until perfection arrives, we either work with what we've got and work-around problems when we can, while hoping the developer(s) aren't burnt out yet and will get around to fixing it, (even if the problem is unlikely to affect them directly), when they can muster up the time and the motivation.
Dan.
Quote from: dosdan on October 10, 2017, 06:53:12 AM
If the timing offset is consistent and the mapping is the same for all the affected files, then there is no real effort involved. You just drag-and-drop one clip or a whole bunch of them onto -133.bat, and then look in the "converted" sub-directory beneath where the original files were located. Alternatively, it would be trivial to leave the converted files in the same directory, but ending then in "_converted.MP4" to distinguish them for the original files. Or, if there was no higher errorlevel reported on an offset conversion pass (I think errorlevel 0 means "OK"), you could automatically delete the original.
No, each file is uniquely "off" if at all. Have to tune each one. Guess I'll have to research how to install ffmpeg and hope installing it doesn't mess up something else I already have. It's funny, I can remember the days I was averse to Windows at all, not wanting to give up my command line. And now for years, probably due to pathing issues, I dislike anything not gui, and dislike messing with path commands. But will consider your suggestions.
Quote from: Jan Gruuthuse on October 10, 2017, 06:16:38 AM
QuoteAvidemux is a simple tool for simple video processing tasks. The keyword here is simple: it does not offer tools like a timeline, multitrack editing, you cannot freely move or splice audio and video clips from various sources. However, Avidemux allows you to do elementary things in a very straightforward way.
source: Avidemux Wiki (http://www.avidemux.org/admWiki/doku.php?id=using:quickstart)
My user opinion of course.
I'm not expecting "timeline, multitrack editing, ... from various sources, etc." Yes, the simple elementary things only. BUT, it's not unreasonable to expect those simple elementary things to work and to handle exception conditions, or at least report what the exception condition IS. This is with regard to my 1st point... I shouldn't be left to wonder for all eternity what ADM thought was wrong with that GOP. It would be better knowing what the alleged error was even if not corrected by ADM, rather than not even being told what it was.
In many cases over the years, the "too short" message occurs on files way too big to send. Often the whole point of "sending a sample" is thwarted by the very problem. "I can't send a sample as the tool I have to extract a sample is failing to copy/copy the sample", is often the refrain.
However, in this case, the entire file is small enough (1.4 GB) that for a short time, I could upload it to and host it on my web server and ones here could download it to examine personally. This is in case a programmer wants to see what's happening in this case to trigger the error. If I was a programmer again, this is what I'd ask the one reporting an error. Yes, I'd say, "please send me the file".
But I also think that the program should already be reporting whatever condition it CAN detect. (Obviously, it's already detecting a condition and choosing to stop. It's just not telling us what that condition is.)
That error is usually the result of invalid timestamps
The muxer bails out
You have a message such as "DTS going backward" in the log or similar
I would like to see the errors. I can then report them to the author of the conversion utility that I used. But I have looked everywhere I can think of to find such a log. Would you please tell me where the file is located?
- Avidemux menu: Help
- Open the application data folder
- Attach compressed admlog.txt
Quote from: eumagga0x2a on October 06, 2017, 06:02:03 AM
Please reproduce the issue, open the application data folder from the "Help" menu in Avidemux, compress and attach the Avidemux log file (admlog.txt) to your reply.
Quote from: Jan Gruuthuse on October 11, 2017, 06:51:00 AM
- Avidemux menu: Help
- Open the application data folder
- Attach compressed admlog.txt
Thanks. (Yes, I never woulda found it poking around in Windows. Not unless I thought to do a programmed search for a folder named avidemux.)
Please see the attached zip file of admlog.txt.
If anyone wants to see it, I can upload the whole 1.4GB file (which looks to eye to be beautifully encoded, even when playing the trouble spot) to the host of my website. Let me know. It's only 1.4GB, which I can host there temporarily. That small, yet holds 1 hr 8 min. of 720p60 sporting action.
I just ran into another encoded file with the symptom. Fails very soon into the copy/copy. I can't upload the video for this as it's over 3GB, but here attached is the log for IT.
But it seems I'm creating more files WITH this problem than without. Oddly, it never seems to happen more than once throughout a 9-150 min. video.
Percent:7
[adm_lavLogCallback] 19:08:25-205 [lavc] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 31882000 >= 31882000
[FF]Error writing video packet
As expected, duplicated decode timestamps, caught by the bundled FFmpeg which gets actually used when you select MP4 as muxer.
By the way, it doesn't look easy to communicate relevant FFmpeg messages to the user, at least without further going into patching the bundled FFmpeg.
I have samples few MiB in size exhibiting the same problem.
Not directly related to the issue, but why do you use 32 bit Avidemux on a 64 bit Windows? With 64 bit, you could use DXVA2 hardware accelerated display, on Windows > 7 with some luck also decoding.
Quote from: eumagga0x2a on October 11, 2017, 08:00:19 PM
Not directly related to the issue, but why do you use 32 bit Avidemux on a 64 bit Windows? With 64 bit, you could use DXVA2 hardware accelerated display, on Windows > 7 with some luck also decoding.
Because (1) up til a few years ago when I gained one PC with 64-bit i5 and 64-bit Windows, all my other systems were WinXP-32bit whether 32-bit PC or not. And (2), I got advice from various boards suggesting to keep all video tools 32-bit, as many of the 64-bit ones (filters, programs, etc.) were not as bugfree as the 32-bit.
I don't know what DVXA2 is, but I don't see why accelerating the display is relevant to an editing program such as Avidemux. The display window doesn't have to look like a hi-res TV; only has to show me what frame of video I'm on.
Quote from: eumagga0x2a on October 11, 2017, 07:37:18 PM
Percent:7
[adm_lavLogCallback] 19:08:25-205 [lavc] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 31882000 >= 31882000
[FF]Error writing video packet
As expected, duplicated decode timestamps, caught by the bundled FFmpeg which gets actually used when you select MP4 as muxer.
I don't begin to claim to know much about FFMpeg, only that it's been around a long time, is a great standalone tool for those hardy enough to get buried in the muck of specifying parameters in an age of GUIs, AND that it's the core driver (or maybe core functionality would be a better term) underlying many GUI-based programs.
I didn't know that AVIDemux was using it for things. But I DO know that Replay Video Converter (from Applian), the program that created my file, DOES use FFMpeg to generate/encode to H.264 this file. So on the surface, I want to ask, how is it that FFMpeg could generate encoded video that would cause an error when the same program (FFMpeg) is used to read the file??
I'd like your thoughts on that, please. I did google much of the big phrase and found that this has been going on a long time, and that supposedly a bug-ticket has long since been filed with FFMpeg. But I'll make one more observation...
In this thread,
https://superuser.com/questions/978888/application-provided-invalid-non-monotonically-increasing-dts-in-ffmpeg
(https://superuser.com/questions/978888/application-provided-invalid-non-monotonically-increasing-dts-in-ffmpeg)
the answer given is, "If the files play fine, you don't have to worry about it. It means that in the input file, the timestamps associated with samples are not increasing monotonically. That shouldn't be the case, but I think that ffmpeg will usually correct these problems on its own. If you are using an outdated ffmpeg version, updating to a newer one may help resolve these issues, in case it's a problem with the decoder rather than the actual input files."
What I want to observe is this: This reminds me of what I read about Mpeg encoding years ago with regard to B-frames. I thought I read it in connection with Mpeg4 (H.264, et al), but it may have been referring to Mpeg2 as well. The B-frames (as I'm sure you know) are bi-directional predictors, not just prior-frame predictors like the P frames. Therefore they can point FORWARD in time as well as backward. This is what makes them so efficiency-producing. But one of the points made years ago was that the encoder is not required to always write out the frames in chronological order. As long as the PTSes (presentation timestamps) or DTSes (display timestamps, apparently they are synonyms) are properly attached to the frames to which they belong, it shouldn't matter what order they are written out in. It's the job of the decoder to decode all the frames within a GOP and then SORT them into timestamp order for output to the player. So, given this concept, why does any player or muxer care that the frames are not in order? With regard to a muxer, I would think it's job is to simple shove the GOP out the door. I'll appreciate any explanation you could give. (And that's enough questions for one posting.)
Quote from: eumagga0x2a on October 11, 2017, 07:37:18 PM
I have samples few MiB in size exhibiting the same problem.
Could you please tell how you solve that problem? That is, how do you reclaim the file, shuffling in whatever way necessary BUT WITHOUT reencoding? Is there an FFMpeg command that (again, without reenoding) will shuffle things back into proper order so that FFMpeg inside of AVD will not barf at it??
Your sample would still be warmly welcome. I don't have a full solution for now. In my sample, the DTS collision happens due to rounding by Avidemux (timestamps are still increasing in the sample, but the values are very close; when rounded, they become identical).
With the rounding logic commented out, saving the sample in copy mode still fails later due to a real DTS collision, which mostly points at a too high frame increment value got by the demuxer.
Quote from: TCmullet on October 12, 2017, 12:33:01 AM
I don't know what DVXA2 is, but I don't see why accelerating the display is relevant to an editing program such as Avidemux.
This massively saves CPU cycles for stuff like scaling. I'm not sure whether this works with the DirectX video acceleration version 2.0, but with decoding in hardware enabled, it should short-circuit decoding and display, avoiding downloading vast amount of data from the graphics card to the RAM and uploading it back.
Quote from: eumagga0x2a on October 12, 2017, 07:07:18 AM
Quote from: TCmullet on October 12, 2017, 12:33:01 AM
I don't know what DVXA2 is, but I don't see why accelerating the display is relevant to an editing program such as Avidemux.
This massively saves CPU cycles for stuff like scaling. I'm not sure whether this works with the DirectX video acceleration version 2.0, but with decoding in hardware enabled, it should short-circuit decoding and display, avoiding downloading vast amount of data from the graphics card to the RAM and uploading it back.
Again, I don't see what this has to do with Avidemux when in "copy/copy" mode. It sounds like you're talking about when ADM is H264 encoding or other number crunching. But in copy/copy, the video and audio streams are not being decoded or encoded; just mixed (muxed) together on output. No video is being displayed while muxing.
Sure, hardware accel is relevant for tasks like identifying suitable positions for the markers, not for saving in copy mode.
It would be beneficial for solving the issue you raised here if you finally provide a sample video.
Thanks.
@TCmullet, you could test the latest win64 nightly (http://avidemux.org/nightly/win64/). The latest changes in the code should have made it more tolerant to timestamp errors and the "The video has been saved but seems to be incomplete" error has been altered to provide at least the linear time of the location where the muxer had to bail out.
Hey, I want to say that I just recently changed from using 2.7.0 32-bit to 2.7.1 64-bit. I am THRILLED that "too short" no longer appears! This has opened worlds to me in what I can do now! Thank you to whoever fixed that! Notorious for creating files that would trigger the error was Applian's Replay Media Converter, which I wanted to use a lot. I wanted to create files with it, THEN use ADM for various things. Now I can! Also nifty is the mild "deep six" it goes into to rebuild PTSes from defective videos. Ran into a bunch of these from a source long ago. Interestingly it warns "this could take a while". For me it has never taken more than about 6 minutes for about 2 hr HD video. But the big issue is the "too short". NOW, the only time that should happen is when you run out of output disk space, ha ha!