r8469: x264 encodes under ridiculously large bitrate--resolved

[theat934]

I can attest to previous postings about a likely issue with x264 encoding bitrate.

I am trying out nightly version r8469 (> v2.6.1, received through git, built on MacOS 10.7.5 using Homebrew). When I request to encode in an average bitrate two-pass mode, no matter what bitrate I specify, I am always getting a ridiculously large bitrate that can go up to 30x the rate I specify

The pic I am attaching is from an encoding uisng x264 in an average bitrate two-pass mode at 1000 kbps. The average bitrate tolerance under the Quantization tab of the codec configuration is 100%, yet you can clearly see the encoding advancing at about 39000 kbps.

[Spellbinder]

I can also confirm this type of behavior.

[Jan Gruuthuse]

Perhaps this is os X related? Done several tests. I can not confirm this behavior on Ubuntu 12.04 lts 64-bit. I'm just wondering about the second pass? This looks completely different to what is showing here? I used a 720p mpeg-ts as source and tried withmp4 muxer 1500 Kbps and 3000 Kbps and mp4v2 muxer with 15000 Kbps. MediaInfo shows the same.




Can't confirm if the system x264 or avidemux x264 is used. x264 0.129.2230: git://git.videolan.org/x264.git

[theat934]

Spellbinder, Jan,
Thank you both for your feedback. I think Spellbinder in his case works from WindowsOS, so maybe it might not be strictly a MacOS-related issue.

Jan gave me a good hint about checking the x264 end. I uninstalled Avidemux and the x264 codec it was using. Then I installed a different x264 release and rebuilt Avidemux, to make the latter dependent on a different x264 release. I made these steps sequentially with x264 releases r2197.4, r2189, r2184, and r2245. The bevavior of Avidemux remains the same erratic one.

I'm probably thinking there is rather some issue in passing the bitrate value from Avidemux to x264.
Other possible culprits in the MacOS case could be:
* A different post-2.6.1 problem that might surface as this issue when compiling on MacOS.
* A possibly more inconspicuous issue due to the updated XCode 4.6 command line tools needed by Homebrew to build Avidemux.

Eventually I settled for the current x264 r2197.4, and then came the icing on the cake. The first time I tried to test encoding again, I used the same example file with the same x264 settings as I have been doing throughout this posting. When the 2nd pass arrived, the encoding went smoothly at the bitrate I had specified. I repeated the same test subsequently but I was always getting the too-high bitrate issue.

Beats me, folks 

[Jan Gruuthuse]

On linux and windows avidemux, 2.6.# creates .idx2, 2.5.6 creates .idx files for each video loaded manually. Delete these?
Sometimes it helps to delete config2 and jobs.sql between avidemux versions, included nightly, could be in a hidden folder.

When testing same with same video, perhaps deleting test.stats and test.stats.mbtree? Normally these are overwritten by the newly created test.stats.tmp and test.stats.mbtree.tmp.
Only other thing I can think of is some form of caching is taking place and playing havoc between the different softwares? If you alternate with 2 different video clips in testing, does it still show the same behavior?
That's all my user input.

[Spellbinder]

OS: Win7 64bit

On a clean install of r8467 I made a few tests with vbr (2pass). Now I have the issue that the vbr always is to low. When I set vbr (2 pass) to 1500 the result was a video with 658. When set to 400 it was 174. (Deleted all test files inbetween).

There definitly seems to be a problem with the bitrate...

[Spellbinder]

One additional thing: I noticed that the fps in all of the test files is also to low (around 10 instead of 25 like the original).

Maybe this is a related issue?

[Jan Gruuthuse]

Have no clue for the fps issue?
As you can see in the Media info screen captures. A 2 pass vbr (Variable bitrate) with its default settings and a given value (yellow ones) 1500 Kbps gives 761 Kbps. The given value is not the required value, it is the maximum allowed for the vbr. The behavior is set in the Quantiser settings of x264 setting:

H.264 encoding guide: Quantizer

[Spellbinder]

Hi Jan,

Thanks for the tip.

But shoudnt the "Average Bitrate" not be the, well... Average bitrate and not the maximum bitrate?

Another thing is that the "Quantizer Control" settings are greyed out and I cant change them. I can only change the "Adaptive Quantisation".

Concerning the framerate I posted this mediainfo results in another thread:

Complete name                            : INFO\test_ORIGINAL.mp4
Frame rate mode                          : Variable
Frame rate                               : 29.970 fps
Minimum frame rate                       : 29.004 fps
Maximum frame rate                       : 30.000 fps

Complete name                            : INFO\test_h264_changeFPS.mp4
Frame rate mode                          : Variable
Frame rate                               : 14.270 fps
Minimum frame rate                       : 9.986 fps
Maximum frame rate                       : 100.000 fps

Complete name                            : INFO\test_h264_resampleFPS.mp4
Frame rate mode                          : Variable
Frame rate                               : 29.970 fps
Minimum frame rate                       : 9.991 fps
Maximum frame rate                       : 100.000 fps

Do you know how I get a video with the originals constant bitrate?

Many Thanks,

Spellbinder

[Jan Gruuthuse]

No sorry, looks like CBR in x264 gives similar results as VBR option. Wondering why those settings are not availble to you? Guessing now: Double Kbps to wanted target. 1500 Kbps use 3000 Kbps. Should be close. Test with small 1 mintue video clips. Make sure you not using 2.6.1 but 2.6.1  => r8458. Only thing I noticed:
8394   17 Jan 2013   gruntster   

[x264] Qt dialog fixes for Motion Estimation settings (submitted by KoolAidMan): * Maximum Motion Vector Search Range now saves to me_range, not mv_range * Maximum Motion Vector Length option is now available, and saves to mv_range. In the case where the box is unchecked, it sets it to -1 for auto. * Minimum buffer between threads option is now available, and saves to mv_range_thread. In the case where the box is unchecked, it sets it to -1 for auto. * Added json serialization for any of the above options which were not available * Added logging to the x264 settings dump for any above settings which were missing

8458   03 Feb 2013   mean   

[x264] more settings, patch by steginger

http://avidemux.org/nightly/win32/
http://avidemux.org/nightly/

[Spellbinder]

Update:

Fresh Install and first Encode had ridiculously high bitrate again...

Avidemux 2.6.1 SVN r8473 (32-bit)
---------------------------------
Built using GCC 4.7.2 and MSVC++ 10.0 SP1.
Packaged with NSIS 2.46.

Included libraries:
Aften           git-89aee3d496bb2a89f046025402626ee12a12969f
Expat           2.1.0
FAAC            1.28
FAAD2           2.7
Fontconfig      2.10.2
Freetype        2.4.11
FriBidi         0.19.5
LAME            3.99.5
libogg          1.3.0
libvorbis       1.3.3
libvpx          1.1.0
NSPR            4.9.4
opencore-amr    0.1.3
Qt              4.8.4
SpiderMonkey    1.7.0
SQLite          3.7.15.2
TwoLAME         0.3.13
x264            r2245
Xvid            1.3.2
zlib            1.2.7

Included applications:
Avisynth Proxy GUI 2.20

[Jan Gruuthuse]

use windows search (option include search in system folders and hidden) for config2. Somewhere ..\AppData\Roaming\avidemux\config2
when you ave that folder delete config2 and jobs.sql if in a avidemux folder. The try again.
If that fails save a project file  append this to your posting and upload a sample video source file (5 seconds in duration or approx. 20 MB in size, you have this issue with, to rapidshare or similar service and provide a download link to it in the same posting.
Still wondering what is going on.

[Spellbinder]

deleted config2+jobs.sql
same result

Project:

Editor.closeVideo();
Editor.openVideo("I:/TEST/test_ORIGINAL.mp4");
Editor.segments.clear()
Editor.segments.add(0, 7941388, 0);
Editor.setMarkers(0, 7941388);

X264VideoEncoder.configuration.general_params = "2PASSBITRATE=1000";
X264VideoEncoder.configuration.i_scenecut_threshold = "30";
X264VideoEncoder.configuration.MaxBFrame = "16";
X264VideoEncoder.configuration.i_bframe_adaptive = "2";
X264VideoEncoder.configuration.analyze_b_p8x8 = "False";
X264VideoEncoder.configuration.analyze_b_p16x16 = "True";
X264VideoEncoder.configuration.analyze_b_b16x16 = "True";
X264VideoEncoder.configuration.analyze_direct_mv_pred = "3";
X264VideoEncoder.configuration.analyze_me_method = "2";
X264VideoEncoder.configuration.analyze_mv_range = "32";
X264VideoEncoder.configuration.analyze_subpel_refine = "8";
X264VideoEncoder.configuration.analyze_trellis = "2";
X264VideoEncoder.configuration.analyze_psy_rd = "0.000000";
X264VideoEncoder.configuration.analyze_fast_pskip = "False";
X264VideoEncoder.configuration.analyze_intra_luma = "21";
X264VideoEncoder.configuration.ratecontrol_qp_max = "0";
X264VideoEncoder.configuration.ratecontrol_qp_step = "0";
X264VideoEncoder.configuration.ratecontrol_ip_factor = "1.000000";
X264VideoEncoder.configuration.ratecontrol_pb_factor = "1.000000";
X264VideoEncoder.configuration.ratecontrol_lookahead = "50";
Editor.currentVideoEncoder = X264VideoEncoder;
Editor.audioOutputs.clear();

audioOutput = new CopyAudioEncoder();
Editor.audioOutputs.add(0, audioOutput);
Editor.audioOutputs[0].drcEnabled = false;
Editor.audioOutputs[0].timeShift = 0;

Editor.currentMuxer = Mp4v2Muxer;

[Jan Gruuthuse]

Defaults found in avidemux 2.6.1 r8474 non matching in bold
This reference is not present: X264VideoEncoder.configuration.analyze_b_b16x16 = "True";
as you can see in code below others are not present in your project?

X264VideoEncoder.configuration.general_params = "2PASSBITRATE=1000"; "general.params=2PASSBITRATE=1500"
X264VideoEncoder.configuration.i_scenecut_threshold = "30"; "i_scenecut_threshold=40"
X264VideoEncoder.configuration.MaxBFrame = "16"; "MaxBFrame=3"
X264VideoEncoder.configuration.i_bframe_adaptive = "2"; "i_bframe_adaptive=1"
X264VideoEncoder.configuration.analyze_b_p8x8 = "False"; "analyze.b_p8x8=True"
X264VideoEncoder.configuration.analyze_b_p16x16 = "True"; "analyze.b_p16x16=False"
X264VideoEncoder.configuration.analyze_b_b16x16 = "True";
X264VideoEncoder.configuration.analyze_direct_mv_pred = "3"; "analyze.direct_mv_pred=1"
X264VideoEncoder.configuration.analyze_me_method = "2"; "analyze.me_method=1"
X264VideoEncoder.configuration.analyze_mv_range = "32"; "analyze.mv_range=-1"
X264VideoEncoder.configuration.analyze_subpel_refine = "8"; "analyze.subpel_refine=7"
X264VideoEncoder.configuration.analyze_trellis = "2"; "analyze.trellis=1"
X264VideoEncoder.configuration.analyze_psy_rd = "0.000000"; "analyze.psy_rd=1.000000"
X264VideoEncoder.configuration.analyze_fast_pskip = "False"; "analyze.fast_pskip=True"
X264VideoEncoder.configuration.analyze_intra_luma = "21"; "analyze.inter_luma=21"
X264VideoEncoder.configuration.ratecontrol_qp_max = "0"; "ratecontrol.qp_max=69"
X264VideoEncoder.configuration.ratecontrol_qp_step = "0"; "ratecontrol.qp_step=4"
X264VideoEncoder.configuration.ratecontrol_ip_factor = "1.000000"; "ratecontrol.ip_factor=1.400000"
X264VideoEncoder.configuration.ratecontrol_pb_factor = "1.000000"; "ratecontrol.pb_factor=1.300000"
X264VideoEncoder.configuration.ratecontrol_lookahead = "50"; "ratecontrol.lookahead=40"
Editor.currentVideoEncoder = X264VideoEncoder;
Editor.audioOutputs.clear();

audioOutput = new CopyAudioEncoder();
Editor.audioOutputs.add(0, audioOutput);
Editor.audioOutputs[0].drcEnabled = false;
Editor.audioOutputs[0].timeShift = 0;

Editor.currentMuxer = Mp4v2Muxer;

Tinypy project saved in 2.6.1:

Code Select Expand

#PY  <- Needed to identify #
#--automatically built--

adm = Avidemux()
adm.loadVideo("/720ptest.ts")
adm.clearSegments()
adm.addSegment(0, 0, 90920000)
adm.markerA = 0
adm.markerB = 90920000
adm.videoCodec("x264", "general.params=2PASSBITRATE=1500", "general.threads=99", "general.fast_first_pass=True", "level=4294967295", "vui.sar_height=1", "vui.sar_width=1", "MaxRefFrames=3", "MinIdr=25", "MaxIdr=250", "i_scenecut_threshold=40"
, "intra_refresh=False", "MaxBFrame=3", "i_bframe_adaptive=1", "i_bframe_bias=0", "i_bframe_pyramid=2", "b_deblocking_filter=True", "i_deblocking_filter_alphac0=0", "i_deblocking_filter_beta=0", "cabac=True"
, "interlaced=False", "constrained_intra=False", "tff=True", "fake_interlaced=False", "analyze.b_8x8=True", "analyze.b_i4x4=True", "analyze.b_i8x8=True", "analyze.b_p8x8=True", "analyze.b_p16x16=False"
, "analyze.b_b16x16=False", "analyze.weighted_pred=2", "analyze.weighted_bipred=True", "analyze.direct_mv_pred=1", "analyze.chroma_offset=0", "analyze.me_method=1", "analyze.me_range=16", "analyze.mv_range=-1"
, "analyze.mv_range_thread=-1", "analyze.subpel_refine=7", "analyze.chroma_me=True", "analyze.mixed_references=True", "analyze.trellis=1", "analyze.psy_rd=1.000000", "analyze.psy_trellis=0.000000", "analyze.fast_pskip=True"
, "analyze.dct_decimate=True", "analyze.noise_reduction=0", "analyze.psy=True", "analyze.intra_luma=11", "analyze.inter_luma=21", "ratecontrol.rc_method=0", "ratecontrol.qp_constant=0", "ratecontrol.qp_min=0"
, "ratecontrol.qp_max=69", "ratecontrol.qp_step=4", "ratecontrol.bitrate=0", "ratecontrol.rate_tolerance=1.000000", "ratecontrol.vbv_max_bitrate=0", "ratecontrol.vbv_buffer_size=0", "ratecontrol.vbv_buffer_init=0"
, "ratecontrol.ip_factor=1.400000", "ratecontrol.pb_factor=1.300000", "ratecontrol.aq_mode=1", "ratecontrol.aq_strength=1.000000", "ratecontrol.mb_tree=True", "ratecontrol.lookahead=40")
adm.audioClearTracks()
adm.audioAddTrack(0)
adm.audioCodec(0, "copy");
adm.audioSetDrc(0, 0)
adm.audioSetShift(0, 0,0)
adm.audioAddTrack(1)
adm.audioCodec(1, "copy");
adm.audioSetDrc(1, 0)
adm.audioSetShift(1, 0,0)
adm.audioAddTrack(2)
adm.audioCodec(2, "copy");
adm.audioSetDrc(2, 0)
adm.audioSetShift(2, 0,0)
adm.setContainer("MP4V2", "optimize=0", "add_itunes_metadata=0")

[Spellbinder]

The Project was a Qtscript Project. This is the Tinypy Project with the Avidemux Default x264 settings (just changed to 1500 2 pass VBR):

#PY  <- Needed to identify #
#--automatically built--

adm = Avidemux()
adm.loadVideo("I:/TEST/original_short.mp4")
adm.clearSegments()
adm.addSegment(0, 0, 7942000)
adm.markerA = 0
adm.markerB = 7942000
adm.videoCodec("x264", "general.params=2PASSBITRATE=1500", "general.threads=99", "general.fast_first_pass=True", "level=4294967295", "vui.sar_height=1", "vui.sar_width=1", "MaxRefFrames=3", "MinIdr=25", "MaxIdr=250", "i_scenecut_threshold=40"
, "intra_refresh=False", "MaxBFrame=3", "i_bframe_adaptive=1", "i_bframe_bias=0", "i_bframe_pyramid=2", "b_deblocking_filter=True", "i_deblocking_filter_alphac0=0", "i_deblocking_filter_beta=0", "cabac=True"
, "interlaced=False", "constrained_intra=False", "tff=True", "fake_interlaced=False", "analyze.b_8x8=True", "analyze.b_i4x4=True", "analyze.b_i8x8=True", "analyze.b_p8x8=True", "analyze.b_p16x16=False"
, "analyze.b_b16x16=False", "analyze.weighted_pred=2", "analyze.weighted_bipred=True", "analyze.direct_mv_pred=1", "analyze.chroma_offset=0", "analyze.me_method=1", "analyze.me_range=16", "analyze.mv_range=-1"
, "analyze.mv_range_thread=-1", "analyze.subpel_refine=7", "analyze.chroma_me=True", "analyze.mixed_references=True", "analyze.trellis=1", "analyze.psy_rd=1.000000", "analyze.psy_trellis=0.000000", "analyze.fast_pskip=True"
, "analyze.dct_decimate=True", "analyze.noise_reduction=0", "analyze.psy=True", "analyze.intra_luma=11", "analyze.inter_luma=21", "ratecontrol.rc_method=0", "ratecontrol.qp_constant=0", "ratecontrol.qp_min=0"
, "ratecontrol.qp_max=69", "ratecontrol.qp_step=4", "ratecontrol.bitrate=0", "ratecontrol.rate_tolerance=1.000000", "ratecontrol.vbv_max_bitrate=0", "ratecontrol.vbv_buffer_size=0", "ratecontrol.vbv_buffer_init=0"
, "ratecontrol.ip_factor=1.400000", "ratecontrol.pb_factor=1.300000", "ratecontrol.aq_mode=1", "ratecontrol.aq_strength=1.000000", "ratecontrol.mb_tree=True", "ratecontrol.lookahead=40")
adm.audioClearTracks()
adm.audioAddTrack(0)
adm.audioCodec(0, "copy");
adm.audioSetDrc(0, 0)
adm.audioSetShift(0, 0,0)
adm.setContainer("MP4V2", "optimize=0", "add_itunes_metadata=0")

This encode had a bitrate of about 850k again, so roughly halfed...

Framerate to low as well:

Frame rate mode                          : Variable
Frame rate                               : 13.220 fps
Original frame rate                      : 29.970 fps
Minimum frame rate                       : 9.986 fps
Maximum frame rate                       : 29.990 fps

I will try to upload original and encode during the day.