Why does 2.7.1 produce significantly smaller files than 2.7.4?

Started by Who, March 03, 2020, 09:59:03 PM

Previous topic - Next topic

Who

I'm not sure if this is the correct place to post this, if not please feel free to move it.  So here is the situation. Starting with a .ts file of 2.69 GB, if I load that directly into Avidemux 2.7.4, set the video output as Mpeg4 AVC (X264) and in configure change only the CRF setting to 22 and change nothing else, and the set output format as MP4 Muxer (those settings are the same for all these tests), and do some editing to take out parts I don't want (again making the exact same edits in all tests), I get a .mp4 file with a resulting size of 1.38 GB.

However, if I first use ffmpeg to convert that file to a .mp4 file using

ffmpeg -i input.ts -c copy output.mp4

And load that file into into Avidemux 2.7.4, using the exact same settings and making the exact same edits, I get a file size of 952.4 MB!  Note this is after doing a simple copy in ffmpeg with no compression.

But wait, there's more - if I take that same .mp4 file and load it into Avidemux 2.7.1, and use the same settings and make the exact same edits, the resulting file size is only 776.4 MB!!!

So you may wonder why I don't just use the older version.  Well, I do for some files.  But I have run into three issues.  First, I have never been able to get 2.7.1 to load a .ts file, which means that with that version I have to do the intermediate conversion to a .mp4 file first.  And the second issue is that when you do a conversion to .mp4, the video and audio almost always go out of sync.  There are ways to fix that using the "Shift" setting but then you have to run mediainfo on the .ts file to find out what the audio offset is.  But the third issue is that depending on which software produced the .ts file in the first place, if you use ffmpeg to convert it to a .mp4 file sometimes Avidemux (both versions) gets confused and cannot playback or encode the video; the result is really "jumpy" and pretty much unviewable.  That does not happen if you start with the .ts file and use Avidemux 2.7.4, and also in that case Avidemux gets the audio and video sync correct, but then it produces the huge files (and again, I'm using CRF 22 in all cases, that's the only setting I am changing, other than that I am using the defaults).

I did note that a few settings had changed between 2.7.1 and 2.7.4 so I tested saving the default 2.7.1 settings and loading them into 2.7.4, but that apparently had absolutely no effect, the output file size was exactly the same either way.  I have tested this several times starting with different .ts files.  I would love it if there were some simple trick to get 2.7.4 to produce the smaller files like 2.7.1 does, but if there is one I can't seem to figure it out.  Any suggestions?

EDIT:  I'm running Avidemux under MacOS High Sierra version 10.13.6, if that makes any difference.

eumagga0x2a

QuoteHowever, if I first use ffmpeg to convert that file to a .mp4 file using

ffmpeg -i input.ts -c copy output.ts

And load that file into into Avidemux 2.7.4, using the exact same settings and making the exact same edits, I get a file size of 952.4 MB!

The above command doesn't convert to mp4, but it would drop all tracks beyond one video and one audio track, if I'm not mistaken. This should not have any effect on the video bitrate after re-encoding, though.

Please provide more info about the source TS file, i.e. the output of ffprobe, mediainfo or better both. Generally, it is difficult to tell anything without a sample and clear steps to reproduce. The ancient 2.7.1 used an older version of x264, which matters.

2.7.4 is quite outdated too. Unfortunately, there are no recent nightlies which would run on High Sierra. Do you have Homebrew installed? If the disk space is sufficient for Xcode, you could give compiling Avidemux from source a try then.

Who

For some reason the forum software is fighting me on posting this...

output.ts was a typo, should have been output.mp4, sorry.  I edited my original post to fix it.

I don't have Homebrew or Xcode installed and don't have the knowledge to compile from source.  I have heard of those programs but I recall hearing that Xcode is huge and I don't have a lot of spare HD space.

ffprobe:

ffprobe version 4.2.1-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2007-2019 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libfribidi --enable-libass --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mpegts, from 'input.ts':
  Duration: 00:32:00.01, start: 1.405689, bitrate: 11192 kb/s
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuvj420p(pc, bt709, progressive), 1920x1080, 59.94 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 192 kb/s


Mediainfo:

General
ID                                       : 1 (0x1)
Complete name                            : input.ts
Format                                   : MPEG-TS
File size                                : 2.50 GiB
Duration                                 : 31mn 59s
Overall bit rate mode                    : Variable
Overall bit rate                         : 11.2 Mbps

Video
ID                                       : 256 (0x100)
Menu ID                                  : 1 (0x1)
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.2
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 1 frame
Format settings, GOP                     : M=1, N=60
Codec ID                                 : 27
Duration                                 : 31mn 59s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Color range                              : Full
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio
ID                                       : 257 (0x101)
Menu ID                                  : 1 (0x1)
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format version                           : Version 4
Format profile                           : LC
Muxing mode                              : ADTS
Codec ID                                 : 15
Duration                                 : 31mn 59s
Bit rate mode                            : Variable
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 48.0 KHz
Frame rate                               : 46.875 fps (1024 spf)
Compression mode                         : Lossy
Delay relative to video                  : -27ms

Menu
ID                                       : 4096 (0x1000)
Menu ID                                  : 1 (0x1)
Duration                                 : 31mn 59s
List                                     : 256 (0x100) (AVC) / 257 (0x101) (AAC)
Service name                             : Service01
Service provider                         : FFmpeg
Service type                             : digital television


eumagga0x2a

Thanks, the output of ffprobe and mediainfo shows nothing extraordinary regarding the source TS file. The compression rate of the source might be quite low due to

Format settings, ReFrames                : 1 frame
Format settings, GOP                     : M=1, N=60


Would you please post the output of mediainfo for the files produced by Avidemux 2.7.1 and by Avidemux 2.7.4 as result of re-encoding this source stream using x264? This would allow to compare encoder configuration.

Quote from: Who on March 04, 2020, 12:07:28 PM
For some reason the forum software is fighting me on posting this...

I'm sorry, CleanTalk causes a huge number of false positives, it looks like users responding to requests for information are especially badly affected. I have zero influence on its working. Please don't hesitate to send info via PM if CleanTalk intervenes again.

QuoteI don't have Homebrew or Xcode installed and don't have the knowledge to compile from source.

Being familiar with the basics of Terminal usage is the only knowledge which were really useful.

QuoteI have heard of those programs but I recall hearing that Xcode is huge

Yes, it is, unfortunately. According to Finder it occupies 17.08 GB (15.91 GiB).

Quoteand I don't have a lot of spare HD space.

That is a pity as there has been a number of important fixes (including macOS-specific ones) since the 2.7.4 release.


Who

Mediainfo outputs, going from the largest file to the smallest:

This is using version 2.7.4 when encoding directly from a .ts file (output file size 1.38 GB) using CRF 22:

General
Complete name                            : output.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 1.29 GiB
Duration                                 : 22mn 7s
Overall bit rate                         : 8 347 Kbps
Encoded date                             : UTC 1904-01-01 00:00:00
Tagged date                              : UTC 1904-01-01 00:00:00
Writing application                      : Lavf58.20.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 22mn 6s
Bit rate                                 : 8 153 Kbps
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Frame rate                               : 59.940 (59940/1000) fps
Minimum frame rate                       : 19.968 fps
Maximum frame rate                       : 97.276 fps
Original frame rate                      : 25.000 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.066
Stream size                              : 1.26 GiB (98%)
Writing library                          : x264 core 155 r2917 0a84d98
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x133 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=22.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Encoded date                             : UTC 1904-01-01 00:00:00
Tagged date                              : UTC 1904-01-01 00:00:00

Audio
ID                                       : 2
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : 40
Duration                                 : 22mn 7s
Bit rate mode                            : Constant
Bit rate                                 : 192 Kbps
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 48.0 KHz
Frame rate                               : 46.875 fps (1024 spf)
Compression mode                         : Lossy
Stream size                              : 30.0 MiB (2%)
Default                                  : Yes
Alternate group                          : 1
Encoded date                             : UTC 1904-01-01 00:00:00
Tagged date                              : UTC 1904-01-01 00:00:00


This is version 2.7.4 when encoding from a .mp4 file produced by ffmpeg (952.4 MB), using the default settings except for CRF 22:

General
Complete name                            : output.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 908 MiB
Duration                                 : 22mn 7s
Overall bit rate                         : 5 741 Kbps
Encoded date                             : UTC 1904-01-01 00:00:00
Tagged date                              : UTC 1904-01-01 00:00:00
Writing application                      : Lavf58.20.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.2
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 22mn 7s
Bit rate                                 : 5 539 Kbps
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Frame rate                               : 59.940 (59940/1000) fps
Minimum frame rate                       : 19.968 fps
Maximum frame rate                       : 97.248 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.045
Stream size                              : 876 MiB (96%)
Writing library                          : x264 core 155 r2917 0a84d98
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x133 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=22.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Encoded date                             : UTC 1904-01-01 00:00:00
Tagged date                              : UTC 1904-01-01 00:00:00

Audio
ID                                       : 2
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : 40
Duration                                 : 22mn 7s
Bit rate mode                            : Constant
Bit rate                                 : 192 Kbps
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 48.0 KHz
Frame rate                               : 46.875 fps (1024 spf)
Compression mode                         : Lossy
Stream size                              : 30.0 MiB (3%)
Default                                  : Yes
Alternate group                          : 1
Encoded date                             : UTC 1904-01-01 00:00:00
Tagged date                              : UTC 1904-01-01 00:00:00


This is version 2.7.4 when encoding from a .mp4 file produced by ffmpeg (952.4 MB), using settings tweaked to more closely copy the 2.7.1 defaults but still using CRF 22.  Note that changing those settings made absolutely no difference in the size of the file from the previous run:

General
Complete name                            : output.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 908 MiB
Duration                                 : 22mn 7s
Overall bit rate                         : 5 741 Kbps
Encoded date                             : UTC 1904-01-01 00:00:00
Tagged date                              : UTC 1904-01-01 00:00:00
Writing application                      : Lavf58.20.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.2
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 22mn 7s
Bit rate                                 : 5 539 Kbps
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Frame rate                               : 59.940 (59940/1000) fps
Minimum frame rate                       : 19.968 fps
Maximum frame rate                       : 97.248 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.045
Stream size                              : 876 MiB (96%)
Writing library                          : x264 core 155 r2917 0a84d98
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x133 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=22.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Encoded date                             : UTC 1904-01-01 00:00:00
Tagged date                              : UTC 1904-01-01 00:00:00

Audio
ID                                       : 2
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : 40
Duration                                 : 22mn 7s
Bit rate mode                            : Constant
Bit rate                                 : 192 Kbps
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 48.0 KHz
Frame rate                               : 46.875 fps (1024 spf)
Compression mode                         : Lossy
Stream size                              : 30.0 MiB (3%)
Default                                  : Yes
Alternate group                          : 1
Encoded date                             : UTC 1904-01-01 00:00:00
Tagged date                              : UTC 1904-01-01 00:00:00


And finally this is using version 2.7.1, encoding from a .mp4 file produced by ffmpeg (776.4 MB), still using CRF 22:

General
Complete name                            : output.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 740 MiB
Duration                                 : 22mn 7s
Overall bit rate                         : 4 680 Kbps
Encoded date                             : UTC 1904-01-01 00:00:00
Tagged date                              : UTC 1904-01-01 00:00:00
Writing application                      : Lavf57.71.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5.1
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 22mn 7s
Bit rate                                 : 4 476 Kbps
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Frame rate                               : 59.940 (59940/1000) fps
Minimum frame rate                       : 19.970 fps
Maximum frame rate                       : 97.295 fps
Original frame rate                      : 97.295 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.036
Stream size                              : 708 MiB (96%)
Writing library                          : x264 core 148 r2795 aaa9aa8
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x133 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=22.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Encoded date                             : UTC 1904-01-01 00:00:00
Tagged date                              : UTC 1904-01-01 00:00:00

Audio
ID                                       : 2
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : 40
Duration                                 : 22mn 7s
Bit rate mode                            : Constant
Bit rate                                 : 192 Kbps
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 48.0 KHz
Frame rate                               : 46.875 fps (1024 spf)
Compression mode                         : Lossy
Stream size                              : 30.0 MiB (4%)
Default                                  : Yes
Alternate group                          : 1
Encoded date                             : UTC 1904-01-01 00:00:00
Tagged date                              : UTC 1904-01-01 00:00:00


QuoteThat is a pity as there has been a number of important fixes (including macOS-specific ones) since the 2.7.4 release.

I hope there will be a newer release for MacOS then, that will run under High Sierra.  I don't want to upgrade to Catalina because Apple dropped support for 32-bit programs with that version, and I still have a couple of those that I rely on.  Just out of curiosity, are you sure your newer versions won't run under High Sierra?  I ask because I discovered that 2.7.1 is supposed to be the latest version that High Sierra will run according to the web site, yet it seems to be running 2.7.4 just fine, except for this problem.

Thanks for looking at these, and I hope you can spot the problem.

eumagga0x2a

Thank you, I see huge differences in bitrate between different output files with the bitrate from re-encoding the mp4 source in 2.7.4 closely matching the expectations for 1080p@60fps content with medium complexity, but don't have any explanation so far. When I take a TS file (a DVB-T2 sample, which means video is HEVC) and remux it with ffmpeg to mp4, the results of re-encoding these files with x264 using a current Avidemux build are almost identical in size.

If the content is innocuous enough, could you please provide a fragment of the source TS file as a sample via WeTrasfer (no email address required!), Mega, Dropbox or Google Drive?

dd if=/path/to/source.ts of=/path/to/fragment.ts bs=1m count=100

2.7.1 was built with a much older version of x264 (148 r2795) than 2.7.4 (155 r2917, my private builds off the git master use the same x264 version), there is no reasonable way back. If different versions of x264 handle different CRF values differently, I don't think that anything can be done about it, especially as the differences may be attributed to x264 build configuration as provided by Homebrew. The only mystery to solve is the effect of remuxing TS to mp4 in case of your source file.

If you need a particular bitrate / output video stream size, you should use the 2pass mode then.

Quote from: Who on March 05, 2020, 01:02:18 PM
Just out of curiosity, are you sure your newer versions won't run under High Sierra?  I ask because I discovered that 2.7.1 is supposed to be the latest version that High Sierra will run according to the web site, yet it seems to be running 2.7.4 just fine, except for this problem.

From https://avidemux.org/smif/index.php/topic,19030.msg88428.html#msg88428 I know that current nightlies don't run on High Sierra. The reason has been identified, but not resolved yet, waiting for some other work being completed first.