A few Queue problems/bugs

[Bearbear]

I'm currently running Avidemux 2.7.5 on Windows 10 build 1803, but for a while now I've come across a few queue bugs that I don't believe were present in a fairly older version of Avidemux.

The first is when I queue up a few jobs and then run them, it now spawns extra progress windows that seemingly don't do anything - as in the percentage bar or text never changes on 2 of them (see attachment screenshot). Uninstalling/reinstalling doesn't alleviate this problem. The extra progress windows don't appear to do any harm and close when all jobs are finished, but it's strange that they're even there when they didn't used to be.

The second problem is that when I open an mkv file, do some simple cutting and queue the job to output as another mkv, it always fails. What's more bizarre is that if I load the same .py job into the main program and simply save it from there (not using the Queue), it saves fine.

Here's an example .py job that fails with the Queue:

Code Select Expand

#PY  <- Needed to identify #
#--automatically built--

adm = Avidemux()
adm.loadVideo("D:/foo.mkv")
adm.clearSegments()
adm.addSegment(0, 0, 1092483332)
adm.markerA = 0
adm.markerB = 1092483332
adm.setPostProc(3, 3, 0)
adm.videoCodec("x265", "useAdvancedConfiguration=True", "general.params=AQ=25", "general.poolThreads=99", "general.frameThreads=0", "general.preset=ultrafast", "general.tuning=none", "general.profile=main", "level=-1", "vui.sar_height=1"
, "vui.sar_width=1", "MaxRefFrames=3", "MinIdr=25", "MaxIdr=250", "i_scenecut_threshold=40", "MaxBFrame=3", "i_bframe_adaptive=1", "i_bframe_bias=0", "i_bframe_pyramid=2", "b_deblocking_filter=True", "interlaced_mode=0"
, "constrained_intra=False", "lookahead=40", "weighted_pred=2", "weighted_bipred=True", "cb_chroma_offset=0", "cr_chroma_offset=0", "me_method=3", "me_range=16", "subpel_refine=5", "trellis=1", "psy_rd=1.000000"
, "fast_pskip=True", "dct_decimate=True", "noise_reduction=0", "noise_reduction_intra=0", "noise_reduction_inter=0", "strong_intra_smoothing=True", "ratecontrol.rc_method=0", "ratecontrol.qp_constant=0"
, "ratecontrol.qp_step=4", "ratecontrol.bitrate=0", "ratecontrol.rate_tolerance=1.000000", "ratecontrol.vbv_max_bitrate=0", "ratecontrol.vbv_buffer_size=0", "ratecontrol.vbv_buffer_init=1", "ratecontrol.ip_factor=1.400000"
, "ratecontrol.pb_factor=1.300000", "ratecontrol.aq_mode=2", "ratecontrol.aq_strength=1.000000", "ratecontrol.cu_tree=True", "ratecontrol.strict_cbr=False")
adm.addVideoFilter("swscale", "width=1280", "height=720", "algo=2", "sourceAR=0", "targetAR=0", "lockAR=True", "roundup=False")
adm.audioClearTracks()
adm.setSourceTrackLanguage(0,"und")
adm.audioAddTrack(0)
adm.audioCodec(0, "LavAAC");
adm.audioSetDrc(0, 0)
adm.audioSetShift(0, 0, 0)
adm.setContainer("MKV", "forceDisplayWidth=False", "displayWidth=1280", "displayAspectRatio=0")

The problem seems to be specifically with mkv files, mp4 files are fine. Is it a problem with adm.loadVideo and mkv files? Unfortunately the command line window closes too fast for me to see what kind of errors it shows.

Whilst I'm here, is there a way to disable only the error message "Cannot go to next keyframe"? I'm aware you can change the overall message level via a dropdown in the options, but it's only the next keyframe error message that I'd like to disable since the other messages are useful.

[eumagga0x2a]

Do the problems disappear when you let the Jobs GUI launch the Qt version of Avidemux instead of the CLI one?

The example is not simple cutting. It is re-encoding with x265 to HEVC. Does it work when you use x264 instead?

Whilst I'm here, is there a way to disable only the error message "Cannot go to next keyframe"?

No, there is just a hardcoded timer which prevents a new popup if the previous one was shown less than 5 seconds ago.

[eumagga0x2a]

Could you please also attach jobslog.txt from %localappdata%\avidemux from a run showing the issues?

I can't reproduce the problems including opening multiple progress dialogs on macOS, where the CLI version seems to work fine.

[Bearbear]

I can't seem to post the reply I originally wrote since the jobslog contains emails, which I therefore edited out, but the forum thinks I'm still spamming since I'm trying to post the same reply. Amazing.

I'll try editing this post:

Do the problems disappear when you let the Jobs GUI launch the Qt version of Avidemux instead of the CLI one?

The 3 progress windows still spawn when I tick the "use Qt version" checkbox and run (see attachment). However, the mkv job does work, I presume this is because it's loading the job through the main program. For now I guess I'll use the Qt version option since it was annoying coming back to my PC many hours later after I started my queue and seeing several jobs had failed, all of which were where a mkv was used as the input file.

Does it work when you use x264 instead?

Both problems still occur if I change to x264.

Here's my jobslog.txt:

Code Select Expand

*************************
  Avidemux Jobs v2.7.5
*************************
http://www.avidemux.org
Code      : Mean, JSC, Gruntster
GFX       : Nestor Di ,
Design    : Jakub Misak
FreeBSD   : Anish Mistry,
Audio     : Mihail Zenkov
MacOsX    : Kuisathaverat
Win32     : Gruntster

Build Target: Microsoft Windows (x86-64)
Operating System: Microsoft Windows (6.2.9200; 64-bit)
Initializing WinSock
WinSock ok
[ADMJob::jobInit] 09:01:32-237 Initializing database (C:\Users\me\AppData\Roaming\avidemux\jobs.sql)
[ADM_jobCheckVersion] 09:01:32-238 Db version 3, our version 3
[ADM_jobCheckVersion] 09:01:32-238 Same version, continuing..
[ADMJob::jobInit] 09:01:32-238 Successfully connected to jobs database..
[loadTranslator] 09:01:32-260 Using system language
[loadTranslator] 09:01:32-260 Initializing language en_GB
[loadTranslator] 09:01:32-260 Translation folder is <C:\Program Files\Avidemux\qt5\i18n\>
[loadTranslation] 09:01:32-260 [Locale] Loading language file C:\Program Files\Avidemux\qt5\i18n\qtbase_en_GB [loadTranslation] 09:01:32-260 FAILED
[loadTranslation] 09:01:32-260 [Locale] Loading language file C:\Program Files\Avidemux\qt5\i18n\avidemux_en_GB [loadTranslation] 09:01:32-261 succeeded
[loadTranslator] 09:01:32-261 Updating translations...
[loadTranslator] 09:01:32-261 [Locale] Test: &Edit -> &Edit

[ADM_socket::createBindAndAccept] 09:01:32-471 Binding on 127.0.0.1:0
[ADM_socket::createBindAndAccept] 09:01:32-471 Socket bound to port 9246
[jobWindow::jobWindow] 09:01:32-471 Socket bound to 9246
*
*
[jobWindow::refreshList] 09:01:32-471 Found 2 jobs
[jobProgress::jobProgress] 09:01:43-197 Starting progress dialog
[jobProgress::jobProgress] 09:01:43-232 Starting progress dialog
[jobProgress::jobProgress] 09:01:43-263 Starting progress dialog
*
*
[jobWindow::refreshList] 09:01:43-317 Found 2 jobs
[spawnProcess] 09:01:43-378 Starting <avidemux_cli.exe>
[spawnProcess] 09:01:43-378 0 : --nogui
[spawnProcess] 09:01:43-378 1 : --slave 9246
[spawnProcess] 09:01:43-378 2 : --run "C:\Users\me\AppData\Roaming\avidemux\jobs\\Job 2019-10-04 100024.py"
[spawnProcess] 09:01:43-378 3 : --save "D:/test1.mkv"
[spawnProcess] 09:01:43-378 4 : --quit
[spawnProcess] 09:01:43-378 =>avidemux_cli.exe --nogui  --slave 9246 --run "C:\Users\me\AppData\Roaming\avidemux\jobs\\Job 2019-10-04 100024.py"  --save "D:/test1.mkv"  --quit
[spawnProcess] 09:01:43-378 ==================== Start of spawner process job ================
[spawnProcess] 09:01:43-384 ==================== End of spawner process job ================
[jobWindow::spawnChild] 09:01:44-383 Spawning successfull
[ADM_socket::waitForConnect] 09:01:44-383 Accepting...
[ADM_commandSocket::handshake] 09:01:44-383 Waiting for hello message...
[ADM_commandSocket::handshake] 09:01:44-383 Got hello message, continuing...
[ADM_commandSocket::pollMessage] 09:01:45-383 Timeout on socket
[jobWindow::runOneJob] 09:01:46-385 Got a new message 3
Progress 3 %
[jobWindow::runOneJob] 09:01:47-009 Got a new message 3
Progress 8 %
[ADM_commandSocket::pollMessage] 09:01:48-012 Timeout on socket
[jobWindow::runOneJob] 09:01:49-012 Got a new message 3
Progress 13 %
[jobWindow::runOneJob] 09:01:49-040 Got a new message 3
Progress 18 %
[ADM_commandSocket::pollMessage] 09:01:50-041 Timeout on socket
[jobWindow::runOneJob] 09:01:52-517 Got a new message 3
Progress 23 %
[jobWindow::runOneJob] 09:01:52-518 Got a new message 3
Progress 27 %
[jobWindow::runOneJob] 09:01:52-519 Got a new message 3
Progress 32 %
[jobWindow::runOneJob] 09:01:53-058 Got a new message 3
Progress 36 %
[jobWindow::runOneJob] 09:01:54-058 Got a new message 3
Progress 40 %
[jobWindow::runOneJob] 09:01:55-070 Got a new message 3
Progress 45 %
[jobWindow::runOneJob] 09:01:56-071 Got a new message 3
Progress 50 %
[jobWindow::runOneJob] 09:01:57-072 Got a new message 3
Progress 55 %
[ADM_commandSocket::pollMessage] 09:01:58-075 Timeout on socket
[jobWindow::runOneJob] 09:01:59-075 Got a new message 3
Progress 60 %
[jobWindow::runOneJob] 09:01:59-097 Got a new message 3
Progress 65 %
[jobWindow::runOneJob] 09:02:00-099 Got a new message 3
Progress 70 %
[ADM_commandSocket::pollMessage] 09:02:01-099 Timeout on socket
[jobWindow::runOneJob] 09:02:02-100 Got a new message 3
Progress 75 %
[jobWindow::runOneJob] 09:02:02-105 Got a new message 3
Progress 80 %
[ADM_commandSocket::pollMessage] 09:02:03-106 Timeout on socket
[jobWindow::runOneJob] 09:02:04-107 Got a new message 3
Progress 85 %
[jobWindow::runOneJob] 09:02:04-126 Got a new message 3
Progress 89 %
[ADM_commandSocket::pollMessage] 09:02:05-127 Timeout on socket
[jobWindow::runOneJob] 09:02:06-128 Got a new message 3
Progress 94 %
[jobWindow::runOneJob] 09:02:06-128 Got a new message 2
[jobWindow::runOneJob] 09:02:06-128 Result is 1
*
*
[jobWindow::refreshList] 09:02:06-149 Found 2 jobs
[jobWindow::runOneJob] 09:02:06-150 Running job id = 1
*
*
[jobWindow::refreshList] 09:02:06-163 Found 2 jobs
[spawnProcess] 09:02:06-163 Starting <avidemux_cli.exe>
[spawnProcess] 09:02:06-163 0 : --nogui
[spawnProcess] 09:02:06-163 1 : --slave 9246
[spawnProcess] 09:02:06-163 2 : --run "C:\Users\me\AppData\Roaming\avidemux\jobs\\Job 2019-10-04 100109.py"
[spawnProcess] 09:02:06-163 3 : --save "D:/mkvtest.mkv"
[spawnProcess] 09:02:06-163 4 : --quit
[spawnProcess] 09:02:06-163 =>avidemux_cli.exe --nogui  --slave 9246 --run "C:\Users\me\AppData\Roaming\avidemux\jobs\\Job 2019-10-04 100109.py"  --save "D:/mkvtest.mkv"  --quit
[spawnProcess] 09:02:06-163 ==================== Start of spawner process job ================
[spawnProcess] 09:02:06-164 ==================== End of spawner process job ================
[jobWindow::spawnChild] 09:02:07-167 Spawning successfull
[ADM_socket::waitForConnect] 09:02:07-167 Accepting...
[ADM_commandSocket::handshake] 09:02:07-167 Waiting for hello message...
[ADM_commandSocket::handshake] 09:02:07-167 Got hello message, continuing...
[ADM_commandSocket::getMessage] 09:02:07-277 command error rxing data
[jobWindow::runOneJob] 09:02:08-286 Exiting loop
[jobWindow::runOneJob] 09:02:08-286 ** End of slave process **
[jobWindow::runOneJob] 09:02:08-286 ** End of slave process **
[jobWindow::runOneJob] 09:02:08-286 ** End of slave process **
*
*
[jobWindow::refreshList] 09:02:08-315 Found 2 jobs
[jobWindow::runOneJob] 09:02:08-316 Running job id = 2
[jobProgress::~jobProgress] 09:02:08-316 Deleting progress..
Cleaning up
[ADMJob::jobShutDown] 09:02:08-320 Shutting down jobs database
[onexit] 09:02:08-320
Goodbye...

From what I can see, the error it gives for the mkv file is command error rxing data

[eumagga0x2a]

Could you please run the command the Jobs GUI tries to run

Code Select Expand

avidemux_cli.exe --nogui --run "C:\Users\me\AppData\Roaming\avidemux\jobs\\Job 2019-10-04 100109.py" --save "D:/mkvtest2.mkv" --quit

in cmd.exe window, just without the "--slave" option and with a new file name to avoid trying to overwrite an existing output file? Don't forget to cd /d "Path to folder with avidemux_cli.exe" before executing the command (you've edited out this part of jobslog.txt). Alternatively, call avidemux_cli.exe with full path.

The real error would remain readable in the cmd.exe window.

The only difference between the Mp4 and Matroska demuxers I can think of which might be relevant here is that the Matroska demuxer opens a progress dialog  while indexing the file, which can be dismissed, so that loading the file fails. Maybe there is a bug in the silent mode (--nogui) implementation automatically dismissing this dialog, I need to check it.

[Bearbear]

Don't forget to cd /d "Path to folder with avidemux_cli.exe" before executing the command (you've edited out this part of jobslog.txt).

The only thing I edited out from that jobslog.txt were the emails which this forum disallows.

I've had to attach the output of cmd.exe (see cmdoutput.txt) since it exceeds 20000 characters, the limit of a post on this forum.

Small side request, would it be possible for the Queue save file name dialog box to mimic the standard Save dialog box?

Currently if you select an Output Format, i.e. Mkv Muxer, when using standard Save, the "Save as type" dropdown correctly uses whatever you selected as the Output Format (see attachment save_extension.png). However, when using the Queue, the "Save as type" dropdown always uses "All files (*.*)" which means the automatically generated file name ends with an asterisk for the file extension (see attachment queue_extension.png).

[eumagga0x2a]

I think I can reproduce the behavior you describe with multiple progress dialogs and failures by pressing the "Run jobs" button a second time before  all jobs have been completed.

[Bearbear]

I think I can reproduce the behavior you describe with multiple progress dialogs and failures by pressing the "Run jobs" button a second time before  all jobs have been completed.

I only ever press the button once though. Is it a case of the "Run jobs" button running the operation twice, once on mouse click down and again on mouse click up?

Edit: I just tested my theory by only pressing down on the run jobs button and not letting go of the mouse and it still spawns multiple progress dialogs

Edit2: For me when I keep on pressing the Run jobs button, it forces Avidemux CLI to encode the next job in the queue, so I had multiple CLI windows up of all the jobs Avidemux was concurrently encoding. More progress dialogs do appear too (beyond the "normal" 3), but if I press the Run job button once more above the total number of queued jobs (i.e. 3 queued jobs, I press the run jobs button 4 times), all the progress dialogs close, but the files do still encode.

Here's a short video of it happening: https://streamable.com/qgjgs

3 Progress dialogs always appear after pressing the Run jobs button once, subsequent presses produces 1 Progress dialog and 1 CLI window for every remaining job in the queue. However, pressing the Run job button when there are no jobs remaining closes all the Progress dialogs and also the main Avidemux jobs window as well, leaving only the CLI windows.

[eumagga0x2a]

On macOS, just clicking on "Run jobs" once results in a normal operation, i.e. no multiple progress dialogs, no failures.

Looking into enabling extension filtering for "Add to queue".

[eumagga0x2a]

The latter should be fixed by [gui] Enable default extension filtering for file dialog when adding job to queue.

[Bearbear]

The latter should be fixed by [gui] Enable default extension filtering for file dialog when adding job to queue.

Awesome, thank you! I'm assuming or at least hoping your fix will be added to the master branch of Avidemux since I have no means of compiling the code myself

[eumagga0x2a]

It is in the master branch, the functionality will be available in the next 64 bits nightly (the patch doesn't apply to legacy-compat without additional work, so it won't be in 32 bits builds for now).

[Bearbear]

I've done a few more tests since as I said in a previous post "I don't believe [these bugs] were present in a fairly older version of Avidemux".

I installed http://www.avidemux.org/nightly/win32_support/avidemux_2.6.20_r170428_win32.exe dated Apr 28 2017 and both problems of 3 progress dialogs and loading a mkv via the queue are not present, in other words, everything works fine and as intended.

I then installed the current latest 32bits (avidemux_2.7.5 r191003_win32.exe dated Oct 3 2019) and only 1 progress dialog appears when pressing "run jobs". I therefore believe the 3 progress dialog bug is 64 bits only.

I cannot determine what causes the mkv bug that I'm experiencing. Sometimes the job will run, but most of the time (99%) it will not with avidemux_2.7.5 r191003_win32.exe. I cannot get the mkv job to run at all with either of the latest 64bit nightlies (avidemux_2.7.5 r191003_win64.exe & avidemux_r191003_win64Qt5_65.zip).

[eumagga0x2a]

I got to test it briefly with the latest MinGW nightly on Windows 10.

As suspected, auto-dismissing the MKV indexing dialog in the CLI version is the direct culprit of the problem. As a surprise, the dialog gets dismissed independently from the --nogui option, and as a big surprise, the timestamp in the job name, provided by ADM_getTimeDateAsString() is returned as UNKNOWN which means that something is wrong either with ADM_getSecondsSinceEpoch(), with localtime() or with strftime().

Finally, with all spawned avidemux_cli.exe processes exiting immediately, Jobs GUI crashes with access violation while cleaning up the progress dialog (I did not get to see multiple progress dialogs, but can't rule out that they were obscured by big cmd.exe windows as the whole process took only a couple of seconds, too short to react and to switch tasks when unprepared).

This makes a pretty full menu for tonight

[eumagga0x2a]

Should be fixed now by commits [cli/DIA_working] Fix use of uninitialized memory which caused loading Matroska files to fail randomly and [Qt/jobs] Safeguard against concurrent execution of jobWindow::runAllJob() and subsequent crash from NULL pointer dereference, please try a future nightly.

Earlier versions were not affected by the first bug because it was impossible to cancel indexing of MKV files.

No idea ATM why clicking on "Run jobs" triggers multiple click events and thus multiple calls to runAllJob() with a more up-to-date Qt version on your system (the legacy-compat branch used to generate 32 bits builds uses a very old Qt). With the patch, it should not have detrimental effects anymore.

Thank you for your report.