Avidemux 2.6.4 (32bit) - Undersizing issue with Mpeg4 ASP.

[AQUAR]

I mentioned this issue in another thread "Avidemux 2.6.4 - H.264 encode by video size is massively broken".
That thread is about an oversizing issue on Mpeg4 AVC rather than an undersizing issue with Mpeg4 ASP.
Hence this new thread to discuss these observations seperately.
 
Normally I encode to achieve a certain bits per pixel value (video qualiy).
In turn this defines a certain video stream size.
With Avidemux 2.5.6. everything works out as expected but not so with Avidemux 2.6.4.

Sample file:
http://www.datafilehost.com/d/0bc8d5ab

Setting the video size on recoding this sample with MPEG4 ASP (xvid4) produces an unpredicatable result.
Example:
2 pass - video size at 10 MB gives video bit rate of 766 Kbps
2 pass - video size at 30 MB gives video bit rate of 1800 Kbps - about 18MB file size
2 pass - video size at  50 MB also gives video bit rate of 1800 Kbps - identical 18MB file size as for "30 MB" case.

Recoding with single pass constant bit rate also produces unpredictable end results.

Not sure if this issue is isolated to my particular setup (just a normal windows 7 - 64 bit - clean install).

[AQUAR]

One thought I had about why this might be happening is that:

The xvid profile in Avidemux 2.5.6 is preset to advanced simple level 5 whereas in Avidemux 2.6.4 it is selectable up to advanced level 4 only.
As these profile levels define upper bitrates maybe the encoding process was being limited as a result of the profile setting.

Unfortunately, testing this theory by choosing the lowest profile "simple level 0"  it proved not to be the case.
That is, no matter the profile setting, the result was the same at the end of the encode.
If the bitrate was limited by the profile selection then in theory the "simple level 0" would create an even smaller video stream size.

[mean]

There are 2 problems :
1- The file is incorrectly demuxed. you have ~ one frame dropped out of two. That more or less halves the bitrate
2- I added the dialog entry to set min/max Q. If you set min Q=1, you'll raise the size

I think 1/ is the main culprit

[mean]

1/ File is not correct. Did you copy it from AVI ?

[AQUAR]

Thanks mean.

I noticed the r8887 change log for xvid to allow setting min & max Q.

Interesting that as much as half the frames are being dropped.
I guess these dropped frames are replaced with duplicates since the frame rate / video duration remains the same.
Must admit I didn't see that from looking at the video.
Are the frames being dropped at the decoding stage or recoding stage?

This is what I did:

The original video was AVC in AVI. 
Avidemux could play it fine but not edit it.

I created 2 remuxed MKV versions:  An avidemux copy into MKV and a remux with MKVMerge.
After that both MKV versions are "editable"
From those MKV's I cut edit a sample and both undersized with XVID.

Then I took another video with XVID in AVI.
All okay in Avidemux and cut a sample.
Recoded to XVID in AVI and it also undersized a lot:

Its possible both video's are problematic for "XviD64" causing frames to be dropped.
I'll try another sample (xvid without a packed bitstream this time) but think the result will be the same.
Maybe the sample is just too small?

With XviD63 (avidemux 2.5.6) the video stream size is only slightly undersized but more than double that with XviD64.
So it seems that most frames are being recoded with the older library.

Maybe this is one of the reasons why they dropped support for xvid in handbrake?

Just my thought:
Avidemux 2.6.4 would be a standout if it could transcode VFR AVC/MKV into XVID/AVI.
XVID/AVI is definitely not the latest and greatest, but it is the most accepted combination across media players.


Media info for the original AVC/AVI Video:
ID : 0
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Main@L3.0
Format settings, CABAC : Yes
Format settings, ReFrames : 2 frames
Codec ID : H264
Duration : 1h 44mn
Bit rate : 1 733 Kbps
Width : 720 pixels
Height : 380 pixels
Display aspect ratio : 1.895
Frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.253
Stream size : 1.26 GiB (89%)
Writing library : x264 core
Color primaries : BT.601-6 525, BT.1358 525, BT.1700 NTSC, SMPTE 170M
Transfer characteristics : BT.709-5, BT.1361
Matrix coefficients : BT.601-6 525, BT.1358 525, BT.1700 NTSC, SMPTE 170M

Media info for the second video (XVID/AVI):
ID : 0
Format : MPEG-4 Visual
Format profile : Advanced Simple@L5
Format settings, BVOP : 2
Format settings, QPel : No
Format settings, GMC : No warppoints
Format settings, Matrix : Default (H.263)
Muxing mode : Packed bitstream
Codec ID : XVID
Codec ID/Hint : XviD
Duration : 1h 45mn
Bit rate : 1 393 Kbps
Width : 720 pixels
Height : 400 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.202
Stream size : 1.03 GiB (75%)
Writing library : XviD 64

[mean]

A small explanation:
2.6 uses time to edit video.
Either it can trust the container to give accurate timestamps or it cannot.

For h264, avi cannot be trusted. Hence you cannot seek in the file.
Mkv can be trusted to provide reliable timestamps.

Now you copied a chunk of the video from avi (i.e. without valid timestamps) to mkv
So you ended up with mkv with invalid timestamps

As it is mkv, avidemux trusts them, but upon decoding finds  inconsistent timestamp and drop them.

Bottom line :
* Either use avi and edit after encoding to mkv. the resulting vfr/h264 in mkv is editable
* Use a true mkv with valid timestamps

[AQUAR]

Thanks,

The quick explanation clarifies the editing issue but I wasn't really concerned with that.
Are you also saying that any inconsistent/lack of timestamp information is responsible for the dropped frames/undersizing?

I ask because I also got the same undersizing result with, xvid in avi, recoded to xvid in avi.
Would not the time info in this case be derived from the fact that the video is CFR.




   

[mean]

The one created by mkvmerge might be valid

[styrol]

Maybe this is one of the reasons why they dropped support for xvid in handbrake?

No, it's not. MPEG-4 Part 2 producing encoders (like Xvid) are indeed rather old. Why focusing development on old technics?

"0.9.4 will also include a drive towards refocusing on the project's core strengths and pruning dead weight. [..] Obsolete, low-quality, and unmaintained container and codec interfaces will be dropped, including AVI, OGM, and Xvid." source

XVID/AVI is definitely not the latest and greatest, but it is the most accepted combination across media players.

Your are talking about (old) standalone players. Your predicate was correct a few years ago, when (some) people burned their video to CDs.
I was used to encode videos using 3ivX / XviD eight years ago. 2004/2005 H.264 entered (so I switched to Sorenson Video 3, Apple H.264 and finally x.264). Today many people stream their H.264 encoded videos to their Smart TV or are feeding their TV using attached storage devices. Digital broadcasting and Blu-ray, both use h.264.

I would not put my resources on converting videos from MPEG-4 Part 10 (H.264) to MPEG-4 Part 2 potentially loosing quality and achieving greater file size.

[AQUAR]

@ Styrol

I have read the public version of the reasons why handbrake has dropped support for MPEG-4 part2.
There may be more to it than that, but they are perfectly entitled to develop their program in whatever fashion they chose.

Dropping support for mpeg-4 part2 (often = xvid/avi) is telling people to get with the latest or to get lost.
Not very empathetic to those least able to make the move to AVC.

Transcoding vfr avc/mkv to xvid/avi is what is needed to bridge the old with the new.
Many (like me) believe that dropping that kind of support should be in the future.
Others (like you) think it should be now (I bet you have an AVC capable multimedia device).

We agree completely except for timing.

I hope Avidemux remains kind to all video recoders (unlike handbrake).

(PS I keep all my AVC coded movies as well)

[AQUAR]

I still tried more variations in feeding 2.6.4 just to see what comes out.

Result  with various samples of MKV, plain old AVI and even bypassing the avidemux decoder (by frame serving raw video via avsproxy) is:
Encode to MPEG-4 ASP causes undersizing no matter what container it is put in.
Encode the same input to MPEG-4 AVC doesn't cause undersizing, even if putting it in an AVI.

Writing library Xvid64 is not the problem either (tried 2.5.6 64 bit as it used Xvid64).

I think the only item left is how the time based avidemux goes about feeding the MPEG-4 ASP encoder.
Timestamp issue or forcing profile constraints?

Even tried that bad piece of sample video again (the one that demuxes badly - causing frame drops) by recoding it to AVC/MKV.
It still undersized greatly from the video size setting but at least the bits per pixel shot up to a high 0.55 bpp.
(Pity the sample video caused a double whammy on the undersizing!).

   

[AQUAR]

Thanks mean for putting back the option to set the quantisation scaling.
Allowing the Q value to drop to 1 instead of 2 does (of course) gives a better match to a generous file size setting.

For those reading this xvid thread, a Q minimum = 1 is generally considered to waste file size for the little quality gain.
But I can see the difference and, considering todays low data storage costs, think it is worth the overhead.

Also interesting is that the basic differential analysis above (usage testing) got close to pin pointing the component responsible for the undersizing.
I just mention that to point out that some user self help is a good thing.

Now, if mean could compile back the option to set ASL5 and pixel aspect ratio for Xvid, I could ditch third party fiddles.

Lastly: Xvid encoding still crashes with 2.6.4 64 bit - faulting module is xvidcore.dll

[mean]

xvid encoding with 64bits avidemux should be ok now
It was miscompiled

[AQUAR]

Great, I will try it out.
Was r8892 recompiled or do I wait for r8893?

Edit: New r8892 hashes differently so it seems updated.
Edit 2: Verified on win7 (64 bit).

[AQUAR]

Noticed r8895 changelog for setting xvid basic aspect ratio.
Much appreciated.