Avidemux Forum

Avidemux => Windows => Topic started by: mederi on July 22, 2019, 01:58:44 PM

Title: Issue with Xvid video in Avideumx 2.7.x
Post by: mederi on July 22, 2019, 01:58:44 PM
Recently I wanted to play some old videos in Panasonic Smart TV, but it had problem with the beginning of videos and I had to jump ahead a bit. Then I wanted to use Avidemux to investigate the issue with problematic files and to try to remux them or something. It seems that Avidemux 2.7.x win32 (tried 2.7.1 and 2.7.2 on Windows Vista) suffers the same issue and cannot properly decode older Xvid files. If I load the video and use right-arrow key to move 1 frame to the right, it jumps to a next key frame at 12 seconds. Older Avidemux 2.6.8 can move frame by frame properly so I used it to cut a sample. VLC media player can play these video files very well in PC.
Please could somebody help me to identify and name the cause of the issue of the file, what module/decoder and where should I report it so it can be fixed?
My investigation so far:
MediaInfo:
General
Complete name                            : D:\Xvid (Simple Profile L3), mp3.avi
Format                                   : AVI
Format/Info                              : Audio Video Interleave
File size                                : 926 KiB
Duration                                 : 13 s 240 ms
Overall bit rate                         : 573 kb/s

Video
ID                                       : 0
Format                                   : MPEG-4 Visual
Format profile                           : Simple@L3
Format settings, BVOP                    : No
Format settings, QPel                    : No
Format settings, GMC                     : No warppoints
Format settings, Matrix                  : Default (H.263)
Codec ID                                 : XVID
Codec ID/Hint                            : XviD
Duration                                 : 13 s 240 ms
Bit rate                                 : 288 kb/s
Width                                    : 640 pixels
Height                                   : 360 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 25.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Compression mode                         : Lossy
Bits/(Pixel*Frame)                       : 0.050
Stream size                              : 466 KiB (50%)
Writing library                          : XviD 1.0.1 (UTC 2004-06-05)

Audio
ID                                       : 1
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 3
Format settings                          : Joint stereo / MS Stereo
Codec ID                                 : 55
Codec ID/Hint                            : MP3
Duration                                 : 13 s 212 ms
Bit rate mode                            : Constant
Bit rate                                 : 96.0 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 32.0 kHz
Compression mode                         : Lossy
Stream size                              : 155 KiB (17%)
Alignment                                : Aligned on interleaves
Interleave, duration                     : 36  ms (0.90 video frame)
Writing library                          : LAME3.98

MPEG4 Modifier v1.4.6 -> Video Info:
Packed bitstream:  No
QPel:              No
GMC:               No
Interlaced:        No
Aspect ratio:      Square pixels
Quant type:        H.263
FourCC:            XVID
User data:         XviD0035

I-VOPs: 2 (0,60%)
P-VOPs: 238 (71,90%)
B-VOPs: 0 (0,00%)
S-VOPs: 0 (0,00%)
N-VOPs: 91 (27,49%)

GSpot v2.70a: also indicates presence of N-VOP=DF (Duplicate Frames)

Sample: Xvid (Simple Profile L3), mp3.avi https://www120.zippyshare.com/v/Wl6LtORS/file.html
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on July 22, 2019, 02:42:37 PM
Quote from: mederi on July 22, 2019, 01:58:44 PM
If I load the video and use right-arrow key to move 1 frame to the right, it jumps to a next key frame at 12 seconds.

Avidemux skips to the next keyframe only if decoding the next frame has hopelessly failed.

QuoteSample: Xvid (Simple Profile L3), mp3.avi https://www120.zippyshare.com/v/Wl6LtORS/file.html

I get a 403 (Forbidden) trying to load this link.
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: mederi on July 22, 2019, 03:18:14 PM
Try another one (this time "[ ] Set as Private Upload" checkbox disabled): https://www36.zippyshare.com/v/2eETTviM/file.html
The size of the sample is 0.9 MB so I cannot attach the file directly here (max 128KB). I can access the zippyshare file without any problem. Where should I upload it (some simple site, no registration)?
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on July 22, 2019, 03:53:01 PM
Please try WeTransfer (no registration, no email address required, even if it looks like they would want one). I get the same "Forbidden" response for the new link.
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on July 22, 2019, 04:03:57 PM
Interestingly, I get 403 from a nginx just trying to load the zippyshare.com homepage. Provider-side censorship?
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: mederi on July 22, 2019, 04:04:52 PM
O.K. WeTransfer: https://we.tl/t-nDJlcToLWg
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on July 22, 2019, 04:07:02 PM
Thanks, this one worked, examining the file.
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on July 22, 2019, 04:21:46 PM
Indeed, the decoding of the first GOP fails, starting with the second one everything is fine. This is probably related to the fact that frames 1-92 are empty (starting frame count at zero).
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: mederi on July 22, 2019, 04:30:34 PM
GSpot tools explains (tooltips) the N-VOPs frames
QuoteN-VOP's are VOP's with no actual content, 2 variations exist: PB and DF.
PB indicates presence of "zero-time" N-VOP's, "filler" VOP's used to create a so-called "packed-bitstream" ("PB").
DF = Duplicate Frame indicates actual (non-zero-time) N-VOP's: designates a picture is unchanged from the previous one.
According to this explanation the Xvid file contains DF frames that should prolong duration of previous frame (copy the first black frame to create several seconds of black screen). Then there are some more DF frames (not many) in a full file.
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on July 22, 2019, 04:39:39 PM
I believe to have identified the bug (which I have caused).

It is not that simple, restoring the handling of empty frames for mpeg4, cut off by [videoCodec][editor] Drain decoder to retrieve last decoded video frames + move to the new lavc API (https://github.com/mean00/avidemux2/commit/ade4b2647f5af3886b091feadbd2eecd625250f4), alone doesn't allow Avidemux to handle the sample. Will look into it later.
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: mederi on July 22, 2019, 06:46:32 PM
So you are also Avideumx developer. Now I can see your nickname in Avideumux 2.7.2 Help > About: Mean / Eumagga0x2a, while there is only Mean in 2.7.1. There is also AUTHORS file in the folder, where you should also put your own stamp. Thank you very much for working on this project. People could donate it if there was any possibility (paypal).
So if I decide to contact Panasonic, I should ask them to fix their media player in TV to handle mpeg4 empty frames (N-VOP Duplicate Frames) properly. Is that right definition of the problem?
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on July 22, 2019, 07:21:49 PM
It seems that your description is correct. I'll look into the issue in Avidemux, why libavcodec returns EAGAIN when the empty frame is passed to it and how it is supposed to work in general. It may happen that the fix will arrive too late for 2.7.4. You are the first one who noticed and reported the regression which persisted for one and a half years in code and for a year in official binaries.

I'm not really a developer, just a user-turned-into-developer for the sole purpose of fixing stuff in a tool I need (and giving back a little bit to the community). Money is absolutely not involved as well as there is no interest in putting my nick somewhere, but thank you for your friendly words.
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: mederi on July 31, 2019, 02:00:27 PM
I have just tried the latest nightly build of Avidemux (avidemux_2.7.4 r190729_win32.exe). Thank you very much for prompt fix. Now I can access the first scene frame-by-frame, the MGM lion in the provided video sample. Please check the horizontal slider behaviour. If I drag it around the 0 position, the 12 seconds mushrooms of the next key frame are appearing there. Thank you for supporting Windows XP/Vista.
--- edit ---
So there is not a bug in a common codec library (libavcodec), is there?
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on July 31, 2019, 04:40:00 PM
Quote from: mederi on July 31, 2019, 02:00:27 PM
Please check the horizontal slider behaviour. If I drag it around the 0 position, the 12 seconds mushrooms of the next key frame are appearing there.

A good catch, thanks. Should be fixed by [editor] Do not reject the first frame of reference video which starts at zero as candidate in getFrameNumFromPtsOrBefore() (https://github.com/mean00/avidemux2/commit/f5d6dc9342e73f23d5b27f2e03a2c2a5d64abb68).

QuoteSo there is not a bug in a common codec library (libavcodec), is there?

Unsure about it. But if the behavior is consistent and we can handle it, why not?
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on August 04, 2019, 10:02:28 AM
Please try a 190804 build.

VC++ (https://avidemux.org/nightly/vsWin64/) Windows 64 bits
MinGW (https://avidemux.org/nightly/win64/) Windows 64 bits
MinGW Legacy Compatibility (https://avidemux.org/nightly/win32/) Windows 32 bits
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: mederi on August 08, 2019, 09:46:08 AM
avidemux_2.7.4 r190804_win32 or latest avidemux_2.7.4 r190808_win32 seem to be O.K. Thank you very much.
Just one more catch: Avidemux reports wrong frame type of several frames in P-FRM (04) row of frames in provided video sample. There are 5 more I-FRM (04) frames recognised (each 16 frames) when moved manually frame-by-frame.
--- EDIT ---
Please enable "[v] Use XVID fcc (else DIVX)" checkbox by default in "Mpeg4 ASP (xvid4)" encoder configuration. I tried to play some DIVX or DX50 videos in several TVs (including smart ones), but they were unsupported there. A fcc change to XVID helped. It seems that XVID is a preferred fcc to DIVX and widely supported nowadays.
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on August 08, 2019, 10:11:00 AM
Quote from: mederi on August 08, 2019, 09:46:08 AM
Just one more catch: Avidemux reports wrong frame type of several frames in P-FRM (04) row of frames in provided video sample. There are 5 more I-FRM (04) frames recognised (each 16 frames) when moved manually frame-by-frame.

It would be each 12th frame if you reduce the cache size to 12 decoded pictures, right?
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: mederi on August 08, 2019, 10:32:15 AM
You are right: 12 frame cache => each 12th frame
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on August 08, 2019, 11:24:52 AM
Should be fixed now (https://github.com/mean00/avidemux2/commit/ecb27949d3adb82381ec202f05d8bc21b354afc3), thank you for your timely report.
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on August 08, 2019, 01:54:30 PM
Regarding the default xvid fourcc, I would suggest saving the desired encoder and muxer configuration as default settings in Avidemux if you happen to use xvid often. Different from bugs like broken handling of empty frames, corrective actions on user part are readily available here

All smart TVs out there support H.264, this is a codec one would want to use nowadays. Xvid and other obsolete mpeg4 brands are destined to go extinct.
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: mederi on August 10, 2019, 11:51:17 AM
O.K., no problem. I have just suggested that Xvid encoder deserves its own XVID FourCC by default and a fix would be probably simple (false -> true) somewhere in Avidemux code. I really prefer the H.264 as a modern and better compression, but recenly I just used the Xvid to encode some videos to match the rest ones archived long time ago.

I can nonfirm that avidemux_2.7.4 r190810_win32 has solved the reported issues.

Next I would like to ask you, whether it is a correct behaviour (not a bug) that Avidemux is always "Unpacking bitstream" when loading an mpeg4 video even if it does not contain a packed bitstream. Avidemux 2.5 contains a detection of PB.
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on August 10, 2019, 04:47:09 PM
Quote from: mederi on August 10, 2019, 11:51:17 AM
I have just suggested that Xvid encoder deserves its own XVID FourCC by default and a fix would be probably simple (false -> true) somewhere in Avidemux code.

This change would be even utterly trivial (https://github.com/mean00/avidemux2/blob/master/avidemux_plugins/ADM_videoEncoder/xvid4/ADM_xvid4.h#L49), but I'm just not yet convinced that it were necessary. While we're at it, it would rather make sense IMHO to add an option to the xvid plugin to enable frame dropping.

QuoteI can nonfirm that avidemux_2.7.4 r190810_win32 has solved the reported issues.

Thanks. A few corrections are still to be done.

QuoteNext I would like to ask you, whether it is a correct behaviour (not a bug) that Avidemux is always "Unpacking bitstream" when loading an mpeg4 video even if it does not contain a packed bitstream. Avidemux 2.5 contains a detection of PB.

It is just a debug message which is always printed when the function which checks for PB is not about to return an error.
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: mederi on August 14, 2019, 03:17:33 PM
Quote from: eumagga0x2a on August 10, 2019, 04:47:09 PM
While we're at it, it would rather make sense IMHO to add an option to the xvid plugin to enable frame dropping.
Well, the more options the better and it would be more universal program. Currently Avidemux offers simplified version of Xvid encoder. There are missing some ASP options (PB, QPel, GMC), but I also would not recommend to use them as there are shitty hardware players, that would refuse to play such files, but it can be useful for somebody.
I checked some more recent AVI files. GSpot program indicates a presence of AVID and a tooltip says:
QuoteIndicates presence of AVI"[D]" (dup or dropped) frames: zero length AVI chunks that instruct the player to "repeat previous frame".
So it seem that there are 2 types of frame dropping: Xvid stream (NVOP DF) and AVI container related (AVID). Probably AVID is better supported nowadays.

Then I found out, that Avidemux produces a bit strange AVI files. If I check such a file in AVI FourCC Code Changer program, then there is a weird entry in the second row FourCC Used Codec, a character followed by a long space. One sample even causes an error in MediaInfo program. Please check what is different from the standard produced by Virtual Dub (Virtaul Dub Mod widely used as AVI muxer at its best times).
https://we.tl/t-k9kbWrJGKa

Quote
It is just a debug message which is always printed when the function which checks for PB is not about to return an error.
I mean that Avidemux always unpacks bitstreams also with files without PB. If an mpeg4 stream does not contain the Packed Bitstream, then what is Avidemux unpacking? The progress bar [Unpacking bitstream: 0% ---> 100%] appears on the screen and the process can take quite a long time. Does it really do something useful for Avidemux or waste it just time with non-PB mpeg4 streams? Avidemux should detect a presence of PB, then only if it is present, it should unpack it, otherwise skip the unpacking as the stream is already unpacked. Or do I misunderstand it?
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on August 14, 2019, 04:22:36 PM
MediaInfo tool which is available for my Linux distribution doesn't have any difficulties handling any of the two samples you provided. The first apparent difference is audio fcc handler which Avidemux sets to 0 while VirtualDub sets it to 1. I don't think I have time to inspect the source of avic to find out what makes it fail.

Avidemux checks the entire video for packed bitstream, it doesn't unpack it if there is nothing to unpack. I don't know yet whether there is a way to rule out presence of PB without actually checking it.
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: mederi on August 15, 2019, 10:43:27 AM
Thank you for your reply. A MediaInfo guy replied:
QuoteI confirm the issue, it is a bad one (crash on Win10 too), trying to fix
that ASAP.
There is just another version of AVI that other softwares have to cope with.
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on August 15, 2019, 11:41:06 AM
Quote from: mederi on August 15, 2019, 10:43:27 AM
There is just another version of AVI that other softwares have to cope with.

It would be very helpful if you could point to exact aspects of the output of the AVI muxer in Avidemux which break the specs (if there are any).

Calling or enumerating things differently is not unusual even in perfectly documented formats like MP4. Recently, a user reported that HEVC videos in MP4 as produced by Avidemux could not be played with QuickTime (yeah, on macOS this still matters). It turned out to be that there are two possible valid fourcc sequences to mark HEVC in MP4 and practically every piece of software supports both. Except QuickTime. We were on the wrong side, now changed to QuickTime's liking.

In general, AVI is a legacy format which should go sooner rather than later. Unfortunately, there are some codecs around which libavformat can't mux into MKV, so we still need the ability to output AVI.

Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: eumagga0x2a on August 15, 2019, 11:55:21 AM
QuoteSo it seem that there are 2 types of frame dropping: Xvid stream (NVOP DF) and AVI container related (AVID). Probably AVID is better supported nowadays.

Avidemux drops true zero-length frames in the AVI (OpenDML) demuxer, which may be not always the right way to handle them (this may result in a highly irregular frame rate, which may pose a bigger problem). I played with disabling this function, identified and fixed some issues I encountered shortly before the 2.7.4 release.

Adding N-VOP support to Xvid encoder is possible and easy, but I don't think this feature really matters nowadays, while a VP9 encoder is sorely missed.
Title: Re: Issue with Xvid video in Avideumx 2.7.x
Post by: mederi on August 17, 2019, 04:55:08 PM
Hi! AVI is still aroud, still alive :). I will try to find some information about its structure and implementations. I think, that Xvid/DivX+MP3 in AVI container is a legendary classic that ruled the media world for over a decade at least here in Europe. I think that all techniques and hacks (MP3 with variable bitrate, multiple audio and subtitle streams, packed bistream, B frames, OpenDML AVIv2) of Microsoft AVI are already well known (VirtualDub Mod, AVI-Mux GUI (http://www.alexander-noe.com/video/amg/)). I wish I could fix it myself, but I am not C/C++ programmer, so I cannot help much here. Nowadays I really prefer free MKV that can store almost any combination of multimedia streams.
I agree with you, that now the fixed Avidemux 2.7.4 can properly decode empty frames and it is not necessary to add them to Xvid encoder since its current implementation is all simplified and sufficient for most users. Thanks.

Avidemux could be one of the first to support a VP9 successor, free codec AV1 (https://en.wikipedia.org/wiki/AV1) and thus help to spread it. VLC already supports AV1 decoding since v3.0.5 (they have developed fast dav1d decoder) and VLC 4 nightlies should already support its decoding and encoding.