Scripted 50% video file size reduction of multiple files in a folder possible?

Started by Dennis, September 21, 2020, 06:18:04 pm

Previous topic - Next topic

Dennis

Hi all, I would like to reduce the file size of a couple of mp4 videos to 50% in a folder by script.
I already found a .py script to convert files in a directory in a loop.
I searched the web how to use varibales in the .py file but haven't found any hint.
Thought I could use get_fileszize() in the loop, calculate 50% of every file in Megabyte and set it as variable in adm.videoCodec( ... "general.params=2PASS=VARIABLE" ...) or something like this.

Any Ideas?

Dennis

Reconsidered it. I use a fixed bitrate instead of file size now. Videos have the same resolution mostly. So this will work the way I need it.

But it would be interessting to know if variables are possible nonetheless.

eumagga0x2a

Quote from: Dennis on September 21, 2020, 06:18:04 pmI searched the web how to use varibales in the .py file but haven't found any hint.

It depends on the type of variables, e.g. if the variable is an integer and you need it to be used as string, you do

adm = Avidemux()
...
quality = 25
adm.videoCodec("x264", "useAdvancedConfiguration=True", "general.params=AQ=" + str(quality), ...)

If the variable is already a string, you skip str():

adm = Avidemux()
...
quality = str(25)
adm.videoCodec("x264", "useAdvancedConfiguration=True", "general.params=AQ=" + quality, ...)

The difficutly is rather that there is no method to query file size in tinyPy, if I am not mistaken, so that the script needs to be automatically rewritten by other means.

Quote from: Dennis on September 22, 2020, 09:39:36 amI use a fixed bitrate instead of file size now.

This is the worst option by far, its sole purpose is to deal with situations when the output stream needs to be sent over a fixed-bandwidth connection.

Dennis

Thanks a lot for the info! So it is basically possible to work with variables.

I found the get_file_size() command here: https://www.avidemux.org/admWiki/doku.php?id=tinypy:gui
But it seems to have to do with the gui and not the tinypy script? I'm not familiar with it yet.
Somewhere I read about like adm.bitrate to get the bitrate of a file. I thought I could read it out in the loop and set the variable before the next compile start. I havent't tried yet if this works.
But the fixed bitrate does it currently.

You wrote a fixed bitrate is worst case. What would be better to use?
I just want to reduze product videos which I get from time to time before uploading to a webshop.
Mostly the half bitrate has no visible quality loss in those videos (for example with 1920x1080 12.000kbit -> 4.000kbit or 17000kbit -> 8000kbit I haven't seen a notable impact).

eumagga0x2a

Quote from: Dennis on September 23, 2020, 05:04:40 pmI found the get_file_size() command here: https://www.avidemux.org/admWiki/doku.php?id=tinypy:gui
But it seems to have to do with the gui and not the tinypy script?

It means you can do

sz = get_file_size("/full/path/to/file")
to store the size up to the max value of uint32_t (i.e. up to 4 GiB) in variable "sz".

I'd suggest using COMPRESS_2PASS_BITRATE (general.params.mode) average bitrate mode, but only if the source video bitrate is really too high.

Dennis

Oh nice, that's what I thought.
So my additional lines in the code below (# comments near the bottom in the loop) could work basically?
This would be the idea with the file size. Didn't tested it yet.
(Got the code somwehere in the internet btw. It works well basically.)

Your suggestion is to use COMPRESS_2PASS_BITRATE. This needs to set a bitrate. In my dynamic idea with variable it would be necessary to read the video bitrate of the current file. Is there a command for this? I only found adm.getVideoCodec() or adm.audioBitrate(int track) (but no bitrate for video. Don't know if these adm. commands can be used for this idea.

I will try a bit with the video bitrate. Maybe it turns out there is a bitrate which works well in general.


ext="mp4"

def convert(filein):
    fileout=filein+".mp4"
    print(filein+"=>"+fileout)
    if(0 == adm.loadVideo(filein)):
        ui.displayError("oops","cannot load "+filein)
        raise
       
    adm.save(fileout)
    print("Done")

# Main

ui=Gui()
adm=Avidemux()
adm.videoCodec("x264", "useAdvancedConfiguration=True", "general.params=2PASS=" + filesize, "general.threads=0", "general.preset=ultrafast", "general.tuning=none", "general.profile=baseline", "general.fast_decode=False", "general.zero_latency=False"
, "general.fast_first_pass=True", "general.blueray_compatibility=False", "general.fake_interlaced=False", "level=-1", "vui.sar_height=1", "vui.sar_width=1", "MaxRefFrames=3", "MinIdr=25", "MaxIdr=250"
, "i_scenecut_threshold=40", "intra_refresh=False", "MaxBFrame=3", "i_bframe_adaptive=1", "i_bframe_bias=0", "i_bframe_pyramid=2", "b_deblocking_filter=True", "i_deblocking_filter_alphac0=0", "i_deblocking_filter_beta=0"
, "cabac=True", "interlaced=False", "constrained_intra=False", "tff=True", "fake_interlaced=False", "analyze.b_8x8=True", "analyze.b_i4x4=True", "analyze.b_i8x8=True", "analyze.b_p8x8=True", "analyze.b_p16x16=False"
, "analyze.b_b16x16=False", "analyze.weighted_pred=2", "analyze.weighted_bipred=True", "analyze.direct_mv_pred=1", "analyze.chroma_offset=0", "analyze.me_method=1", "analyze.me_range=16", "analyze.mv_range=-1"
, "analyze.mv_range_thread=-1", "analyze.subpel_refine=7", "analyze.chroma_me=True", "analyze.mixed_references=True", "analyze.trellis=1", "analyze.psy_rd=1.000000", "analyze.psy_trellis=0.000000", "analyze.fast_pskip=True"
, "analyze.dct_decimate=True", "analyze.noise_reduction=0", "analyze.psy=True", "analyze.intra_luma=11", "analyze.inter_luma=21", "ratecontrol.rc_method=0", "ratecontrol.qp_constant=0", "ratecontrol.qp_min=10"
, "ratecontrol.qp_max=51", "ratecontrol.qp_step=4", "ratecontrol.bitrate=0", "ratecontrol.rate_tolerance=1.000000", "ratecontrol.vbv_max_bitrate=0", "ratecontrol.vbv_buffer_size=0", "ratecontrol.vbv_buffer_init=1"
, "ratecontrol.ip_factor=1.400000", "ratecontrol.pb_factor=1.300000", "ratecontrol.aq_mode=1", "ratecontrol.aq_strength=1.000000", "ratecontrol.mb_tree=True", "ratecontrol.lookahead=40")
adm.audioClearTracks()
adm.setContainer("MP4", "muxerType=0", "optimize=1", "forceAspectRatio=False", "aspectRatio=1", "rotation=0")
#

folder=ui.dirSelect("Select mp4 source folder")

if(folder is None):
    ui.displayError("oops","no folder selected")
    raise

list=get_folder_content(folder,ext)

for i in list:
      # filesize=get_file_size(i) (with additional "/(1024*1024)/2" it should be the half size in MB ?)
      # in MAIN part: "general.params=2PASS=15" -> "general.params=2PASS=" + filesize or "general.params=2PASS=" + str(filesize)
        convert(i)
print("Done")

eumagga0x2a

Okay, let's go for file size (it is translated to average bitrate internally anyway). You might try the following partially untested script:

ui = Gui()
adm = Avidemux()
ext = "mp4"
sep = "/" # for *nix; replace with "\\" for Windows


def convert(filein, folder, filesize):
    filename = basename(filein)
    fileout = folder + sep + "converted_" + filename
    print(filein + " => " + fileout + "\n")
    if 0 == adm.loadVideo(filein):
        ui.displayError("oops", "cannot load " + filein)
        return 0
    adm.setContainer("MP4", "muxerType=0", "optimize=1", "forceAspectRatio=False", "aspectRatio=1", "rotation=0", "clockfreq=0")
    adm.videoCodec("x264", "useAdvancedConfiguration=True", "general.params=2PASS=" + str(filesize), "general.threads=0", "general.preset=ultrafast", "general.tuning=none", "general.profile=high", "general.fast_decode=False", "general.zero_latency=False"
, "general.fast_first_pass=True", "general.blueray_compatibility=False", "general.fake_interlaced=False", "level=-1", "vui.sar_height=1", "vui.sar_width=1", "MaxRefFrames=3", "MinIdr=25", "MaxIdr=250"
, "i_scenecut_threshold=40", "intra_refresh=False", "MaxBFrame=3", "i_bframe_adaptive=1", "i_bframe_bias=0", "i_bframe_pyramid=2", "b_deblocking_filter=True", "i_deblocking_filter_alphac0=0", "i_deblocking_filter_beta=0"
, "cabac=True", "interlaced=False", "constrained_intra=False", "tff=True", "fake_interlaced=False", "analyze.b_8x8=True", "analyze.b_i4x4=True", "analyze.b_i8x8=True", "analyze.b_p8x8=True", "analyze.b_p16x16=False"
, "analyze.b_b16x16=False", "analyze.weighted_pred=2", "analyze.weighted_bipred=True", "analyze.direct_mv_pred=1", "analyze.chroma_offset=0", "analyze.me_method=1", "analyze.me_range=16", "analyze.mv_range=-1"
, "analyze.mv_range_thread=-1", "analyze.subpel_refine=7", "analyze.chroma_me=True", "analyze.mixed_references=True", "analyze.trellis=1", "analyze.psy_rd=1.000000", "analyze.psy_trellis=0.000000", "analyze.fast_pskip=True"
, "analyze.dct_decimate=True", "analyze.noise_reduction=0", "analyze.psy=True", "analyze.intra_luma=11", "analyze.inter_luma=21", "ratecontrol.rc_method=0", "ratecontrol.qp_constant=0", "ratecontrol.qp_min=10"
, "ratecontrol.qp_max=51", "ratecontrol.qp_step=4", "ratecontrol.bitrate=0", "ratecontrol.rate_tolerance=1.000000", "ratecontrol.vbv_max_bitrate=0", "ratecontrol.vbv_buffer_size=0", "ratecontrol.vbv_buffer_init=1"
, "ratecontrol.ip_factor=1.400000", "ratecontrol.pb_factor=1.300000", "ratecontrol.aq_mode=1", "ratecontrol.aq_strength=1.000000", "ratecontrol.mb_tree=True", "ratecontrol.lookahead=40")
    return adm.save(fileout)


indir = ui.dirSelect("Select mp4 source folder")
if indir is None:
    ui.displayError("Oops", "No source folder selected")
    return

outdir = ui.dirSelect("Select output folder")
if outdir is None:
    ui.displayError("Oops", "No output folder selected")
    return

list = get_folder_content(indir, ext)
if list is None:
    ui.displayError("Oops", "No files to convert")
    return

total = 0
counter = 0

for i in list:
    total += 1
    outsize = get_file_size(i)
    outsize /= 1024*1024*2
    counter += convert(i, outdir, outsize)

if not counter:
    ui.displayInfo("Warning", "No files converted")
    return

if counter == 1:
    ui.displayInfo("Finished", "One file out of " + str(total) + " converted")
    return

ui.displayInfo("Finished", str(counter) + " files out of " + str(total) + " converted")

I used different input and output folders, it looked safer for me. I also replaced "general.profile=baseline" with "general.profile=high" (we should replace the default, almost all devices nowadays support High) as the Baseline profile is detrimental for quality and compression rate.

Dennis

Found some time to play with your code. I got some errors first and hence I put your code step by step in the working one.
After testing with the videos I found all videos I get can be converted to 1000kbit without visable difference. So i discarded the file size variable idea.

Haven't seen a difference between general.profile=baseline and general.profile=high with 1000kbit (general.params=2PASSBITRATE=1000) so far with these videos.

The in- and out dir thing works well, thanks a lot for that!
Only the directory selection window will not show the title text defined in "ui.dirSelect()" in any way I tried.

I changed the file counter-idea a bit and fill total with the array length of the filelist.
And I tried to show a permament counter infobox while converting. But haven't found a solution.
The print() box only pops up shortly in between the converting processes and the ui.displayinfo() function blocks the flow and must be clicked away with "OK" every time.
Googled a bit and learned about modal and modeless dialog boxes (https://wxpython.org/Phoenix/docs/html/wx.Dialog.html), but this doesn't work in tinypy I worry. Any ideas by chance? But it isn't important.

ext="mp4"

def convert(filein, folder):
    filename = basename(filein)
    fileout=folder + "\\" + filename + "-1000kbit.mp4"
    print(filein+"=>"+fileout)
    if(0 == adm.loadVideo(filein)):
        ui.displayError("oops","cannot load "+filein)
        raise
    adm.save(fileout)   

#Main

ui=Gui()
adm=Avidemux()
adm.videoCodec("x264", "useAdvancedConfiguration=True", "general.params=2PASSBITRATE=1000", "general.threads=0", "general.preset=ultrafast", "general.tuning=none", "general.profile=high", "general.fast_decode=False", "general.zero_latency=False"
, "general.fast_first_pass=True", "general.blueray_compatibility=False", "general.fake_interlaced=False", "level=-1", "vui.sar_height=1", "vui.sar_width=1", "MaxRefFrames=3", "MinIdr=25", "MaxIdr=250"
, "i_scenecut_threshold=40", "intra_refresh=False", "MaxBFrame=3", "i_bframe_adaptive=1", "i_bframe_bias=0", "i_bframe_pyramid=2", "b_deblocking_filter=True", "i_deblocking_filter_alphac0=0", "i_deblocking_filter_beta=0"
, "cabac=True", "interlaced=False", "constrained_intra=False", "tff=True", "fake_interlaced=False", "analyze.b_8x8=True", "analyze.b_i4x4=True", "analyze.b_i8x8=True", "analyze.b_p8x8=True", "analyze.b_p16x16=False"
, "analyze.b_b16x16=False", "analyze.weighted_pred=2", "analyze.weighted_bipred=True", "analyze.direct_mv_pred=1", "analyze.chroma_offset=0", "analyze.me_method=1", "analyze.me_range=16", "analyze.mv_range=-1"
, "analyze.mv_range_thread=-1", "analyze.subpel_refine=7", "analyze.chroma_me=True", "analyze.mixed_references=True", "analyze.trellis=1", "analyze.psy_rd=1.000000", "analyze.psy_trellis=0.000000", "analyze.fast_pskip=True"
, "analyze.dct_decimate=True", "analyze.noise_reduction=0", "analyze.psy=True", "analyze.intra_luma=11", "analyze.inter_luma=21", "ratecontrol.rc_method=0", "ratecontrol.qp_constant=0", "ratecontrol.qp_min=10"
, "ratecontrol.qp_max=51", "ratecontrol.qp_step=4", "ratecontrol.bitrate=0", "ratecontrol.rate_tolerance=1.000000", "ratecontrol.vbv_max_bitrate=0", "ratecontrol.vbv_buffer_size=0", "ratecontrol.vbv_buffer_init=1"
, "ratecontrol.ip_factor=1.400000", "ratecontrol.pb_factor=1.300000", "ratecontrol.aq_mode=1", "ratecontrol.aq_strength=1.000000", "ratecontrol.mb_tree=True", "ratecontrol.lookahead=40")
adm.audioClearTracks()
adm.setContainer("MP4", "muxerType=0", "optimize=1", "forceAspectRatio=False", "aspectRatio=1", "rotation=0")
#

indir=ui.dirSelect("Select mp4 source folder","Select mp4 source folder")
if(indir is None):
    ui.displayError("oops","no folder selected")
    raise

outdir = ui.dirSelect("Select output folder")
if outdir is None:
    ui.displayError("Oops", "No output folder selected")
    raise

list=get_folder_content(indir,ext)
if list is None:
    ui.displayError("Oops", "No files to convert")
    raise

total = len(list)
counter = 0

for i in list:
  convert(i, outdir)
  counter += 1
  #ui.displayInfo("Videokonvertierung", str(counter) + " file(s) of " + str(total) + " converted.")
  #print("Videokonvertierung", str(counter) + " file(s) of " + str(total) + " converted.")
ui.displayInfo("Videokonvertierung abgeschlossen.", str(counter) + " file(s) of " + str(total) + " converted.")

eumagga0x2a

Quote from: Dennis on October 03, 2020, 08:22:08 amI got some errors first

Presumably because your Avidemux build is outdated, but it would be necessary to know the errors to tell for sure.

Quote from: Dennis on October 03, 2020, 08:22:08 amHaven't seen a difference between general.profile=baseline and general.profile=high with 1000kbit (general.params=2PASSBITRATE=1000) so far with these

Baseline = no B-frames and all codec features which require more built-in memory available in the target decoder device to work disabled = worse quality per output size unit. If this really low average bitrate works for you, then it is fine.

Quote from: Dennis on October 03, 2020, 08:22:08 amOnly the directory selection window will not show the title text defined in "ui.dirSelect()" in any way I tried.

Presumably because your Avidemux build is really old, this was fixed by [tinyPy] Pass argument value in dirSelect() to FileSel_SelectDir() almost a year ago.

Regarding dialogs, we can use only methods provided by Avidemux python interface.