Choosing the right video encoder

[vlotty]

Hi all-- new to Avidemux and I'm impressed with how powerful this software is.  It's take a bit of research for me to start to figure out how to get things to work, especially with the queue.  Now, I'm a bit overwhelmed by the sheer number of settings available depending on the encoder. 

I've settled on H.264 in an MP4 container, because I don't have subtitles or need for multiple audio tracks.  I'm just straight joining, cutting and re-encoding avi files that were originally encoded as 1080p MJPEG / PCM to hopefully save some disk space and take advantage of hardware acceleration on my Mac Mini 2012 (Core i5 dual-core). 

For encoding H.264 I see that there's the software encoder "MPEG4 AVC (x264)" and there's a hardware accelerated encoder "VideoToolbox".  Another post indicated that VideoToolbox can output higher quality at the expense of disk space compared to the software encoder.  From what I understand, the software encoder quality setting under "rate control" dictates the level of compression/quality, and that 16-25 is a good range, with 16 being the upper limit for quality that we can perceive? 

My questions-- VideoToolbox encodes WAY WAY faster than the software encoder, as expected, so I would prefer to use this if possible.  What is considered a "high" bitrate?  The default is 2000kbps (max 4000).  I find that when I set that to 3000kbps, it seems to match the output bitrate of the pure software encoded video of the same input file.  Are there any good rules of thumb to preserve quality without generating huge files? 

If I do go with the software encoder, is the default quality setting of 20 sufficient, or what is the rule of thumb there? 

I know my source video is already compressed and basically I want to preserve roughly the same level of quality as much as I can, and hopefully save some disk space as well if possible.  The tutorial has on H.264 has a lot of info but not sure what to focus on for my purposes since there's so much.  VLC reports the source video has an input and stream bitrate of around 15000kb/s.  Thanks for any help! 

[eumagga0x2a]

I've settled on H.264 in an MP4 container, because I don't have subtitles or need for multiple audio tracks.  I'm just straight joining, cutting and re-encoding avi files that were originally encoded as 1080p MJPEG / PCM to hopefully save some disk space and take advantage of hardware acceleration on my Mac Mini 2012 (Core i5 dual-core).

There haven't been any fresh macOS nightlies for quite a long time (the latest was generated on Nov 24). I am inclined to recommend to build current git master of Avidemux from source, which is pretty easy on macOS (albeit not completely trivial as on Linux) if you run Sierra or later.

For encoding H.264 I see that there's the software encoder "MPEG4 AVC (x264)" and there's a hardware accelerated encoder "VideoToolbox".  Another post indicated that VideoToolbox can output higher quality at the expense of disk space compared to the software encoder.

As a rule, hardware accelerated encoders achieve lower compression (i.e. they require higher bitrates) to achieve the same visual quality as x264. Obviously, any encoder will produce awful quality if you configure it to do so.

From what I understand, the software encoder quality setting under "rate control" dictates the level of compression/quality, and that 16-25 is a good range, with 16 being the upper limit for quality that we can perceive?

FFmpeg wiki has a very good description of x264 CRF values: https://trac.ffmpeg.org/wiki/Encode/H.264

My questions-- VideoToolbox encodes WAY WAY faster than the software encoder, as expected, so I would prefer to use this if possible.  What is considered a "high" bitrate?  The default is 2000kbps (max 4000).

I chose 2000kbps as it allowed to achieve acceptable quality for 720p 25fps source video. For 1080p at 25fps, multiply this with (1080/720)^2, which gives 4500kbps. This should be sufficient in most cases.

If I do go with the software encoder, is the default quality setting of 20 sufficient, or what is the rule of thumb there?

More than sufficient for my expectations, but I would not recommend going above 25. The range 20-23 provides a very good tradeoff between compression and quality.

I know my source video is already compressed and basically I want to preserve roughly the same level of quality as much as I can, and hopefully save some disk space as well if possible.

Any not purely intra-based codec will be vastly superior to MJPEG compression-wise.

[vlotty]

There haven't been any fresh macOS nightlies for quite a long time (the latest was generated on Nov 24). I am inclined to recommend to build current git master of Avidemux from source, which is pretty easy on macOS (albeit not completely trivial as on Linux) if you run Sierra or later.

I'm using the latest compiled, 2.7.4.  Will the nightlies have more optimized encoders? 

Any not purely intra-based codec will be vastly superior to MJPEG compression-wise.

Thank you for your replies!  I will test out the VideoToolbox encoder at 4500kbps and see how that looks and compare file size.  Why does the hardware encoder achieve lower compression?  Isn't VideoToolbox also encoding x264? 

One thing I noticed is that the contrast/color space seems to change slightly when I have encoded x264 from MJPEG for both the software and videotoolbox encoders.  It appears to have slightly more contrast and maybe a bit darker, if that makes sense, compared to the source.  Is there something in the settings that is causing this?  (I am not adding filters)

[eumagga0x2a]

There haven't been any fresh macOS nightlies for quite a long time (the latest was generated on Nov 24). I am inclined to recommend to build current git master of Avidemux from source, which is pretty easy on macOS (albeit not completely trivial as on Linux) if you run Sierra or later.

I'm using the latest compiled, 2.7.4.

The latest compile was from Nov. 24, but now a fresh official macOS nightly has been just uploaded to https://avidemux.org/nightly/osx_catalina/. The last release (2.7.4 for macOS) dates back to August.

Will the nightlies have more optimized encoders?

Probably some library version will be updated, but generally no, there are other reasons including but not limited to better handling both of constant and variable frame rate.

Why does the hardware encoder achieve lower compression?

I can only speculate that implementing all the potential of the codec in hardware would increase the costs disproportionately or hit the limit of available space on the chip.

Isn't VideoToolbox also encoding x264?

No, both encode to H.264. x264 is an encoder implementation.

One thing I noticed is that the contrast/color space seems to change slightly when I have encoded x264 from MJPEG for both the software and videotoolbox encoders.  It appears to have slightly more contrast and maybe a bit darker, if that makes sense, compared to the source.  Is there something in the settings that is causing this?  (I am not adding filters)

Probably full color range (0-255) vs limited color range (16-235). Most videos use the latter. AFAIK we don't support full color range for x264 yet.

[vlotty]

There haven't been any fresh macOS nightlies for quite a long time (the latest was generated on Nov 24). I am inclined to recommend to build current git master of Avidemux from source, which is pretty easy on macOS (albeit not completely trivial as on Linux) if you run Sierra or later.

The latest compile was from Nov. 24, but now a fresh official macOS nightly has been just uploaded to https://avidemux.org/nightly/osx_catalina/. The last release (2.7.4 for macOS) dates back to August.

Will there be versions released for Mojave in the future (2.7.5 etc), or only for Catalina?  Not sure I want to upgrade my machine to Catalina yet... I suppose I can look into building my own binaries. 

Thanks again for your help!

[eumagga0x2a]

It is quite possible that builds generated on Catalina run just fine on Mojave.