2.7.6 crashes reproducibly on its own py script: Cannot set up encoder

[gamez]

Hi

I am transcoding and cutting into pieces a TV recording from MPEG2 TS to MP4 format:

Code Select Expand

#PY  <- Needed to identify #
#--automatically built--

adm = Avidemux()
if not adm.loadVideo("Z:/TVRip/20180113 1911 - ProSieben - Storm Hunters.ts"):
    raise("Cannot load Z:/TVRip/20180113 1911 - ProSieben - Storm Hunters.ts")
adm.clearSegments()
adm.addSegment(0, 173016240, 918760016)
adm.addSegment(0, 1681096192, 783000064)
adm.addSegment(0, 3069656320, 1342759680)
adm.addSegment(0, 4939136512, 1209920000)
adm.addSegment(0, 6826176512, 574719552)
adm.markerA = 0
adm.markerB = 4829159312
adm.setPostProc(3, 3, 0)
adm.videoCodec("x264", "useAdvancedConfiguration=True", "general.params=2PASSBITRATE=3376", "general.threads=0", "general.preset=ultrafast", "general.tuning=none", "general.profile=baseline", "general.fast_decode=False", "general.zero_latency=False"
, "general.fast_first_pass=True", "general.blueray_compatibility=False", "general.fake_interlaced=False", "level=-1", "vui.sar_height=1", "vui.sar_width=1", "MaxRefFrames=3", "MinIdr=25", "MaxIdr=250"
, "i_scenecut_threshold=40", "intra_refresh=False", "MaxBFrame=3", "i_bframe_adaptive=1", "i_bframe_bias=0", "i_bframe_pyramid=2", "b_deblocking_filter=True", "i_deblocking_filter_alphac0=0", "i_deblocking_filter_beta=0"
, "cabac=True", "interlaced=False", "constrained_intra=False", "tff=True", "fake_interlaced=False", "analyze.b_8x8=True", "analyze.b_i4x4=True", "analyze.b_i8x8=True", "analyze.b_p8x8=True", "analyze.b_p16x16=False"
, "analyze.b_b16x16=False", "analyze.weighted_pred=2", "analyze.weighted_bipred=True", "analyze.direct_mv_pred=1", "analyze.chroma_offset=0", "analyze.me_method=1", "analyze.me_range=16", "analyze.mv_range=-1"
, "analyze.mv_range_thread=-1", "analyze.subpel_refine=7", "analyze.chroma_me=True", "analyze.mixed_references=True", "analyze.trellis=1", "analyze.psy_rd=1.000000", "analyze.psy_trellis=0.000000", "analyze.fast_pskip=True"
, "analyze.dct_decimate=True", "analyze.noise_reduction=0", "analyze.psy=True", "analyze.intra_luma=11", "analyze.inter_luma=21", "ratecontrol.rc_method=0", "ratecontrol.qp_constant=0", "ratecontrol.qp_min=10"
, "ratecontrol.qp_max=51", "ratecontrol.qp_step=4", "ratecontrol.bitrate=0", "ratecontrol.rate_tolerance=1.000000", "ratecontrol.vbv_max_bitrate=0", "ratecontrol.vbv_buffer_size=0", "ratecontrol.vbv_buffer_init=1"
, "ratecontrol.ip_factor=1.400000", "ratecontrol.pb_factor=1.300000", "ratecontrol.aq_mode=1", "ratecontrol.aq_strength=1.000000", "ratecontrol.mb_tree=True", "ratecontrol.lookahead=40")
adm.addVideoFilter("swscale", "width=720", "height=404", "algo=2", "sourceAR=0", "targetAR=0", "lockAR=False", "roundup=False")
adm.audioClearTracks()
adm.setSourceTrackLanguage(0,"deu")
adm.setSourceTrackLanguage(1,"deu")
adm.audioAddTrack(0)
adm.audioCodec(0, "copy", "bitrate=192", "mode=0");
adm.audioSetDrc(0, 0)
adm.audioSetShift(0, 0, 0)
adm.setContainer("MP4V2", "optimize=0", "add_itunes_metadata=0")

This py script was saved before pushing the "save" button and was processed OK.

However when I close avidemux, reopen the script (which opens and displays he cut TS OK) and try to save it again, avidemux crashes with a "Cannot set up encoder. The configuration supplied to the encoder may be incompatible or the encoder may depend on features unavailable on this system." popup message. The _cli pretty much behaves the same way.


Shouldn't avidemux be able to re-execute its own scripts?

g.

[eumagga0x2a]

A few notes to the setup:

Code Select Expand

adm.setPostProc(3, 3, 0)

Down with post-processing. At best (video is decoded in hardware), it does nothing.

Code Select Expand

adm.addVideoFilter("swscale", "width=720", "height=404", "algo=2", "sourceAR=0", "targetAR=0", "lockAR=False", "roundup=False")

The source is most likely interlaced. If it is interlaced, you should insert a deiterlacing filter as the very first transformation filter in the chain.

Code Select Expand

adm.audioCodec(0, "copy", "bitrate=192", "mode=0");

I wonder how do you manage to get this mixup (the dummy codec "copy" doesn't allow any parameters).

Code Select Expand

adm.setContainer("MP4V2", "optimize=0", "add_itunes_metadata=0")

The MP4v2 muxer is deprecated, please use MP4.

However when I close avidemux, reopen the script (which opens and displays he cut TS OK) and try to save it again, avidemux crashes with a "Cannot set up encoder. The configuration supplied to the encoder may be incompatible or the encoder may depend on features unavailable on this system." popup message.

You cite a pretty clear formulated error message, do you really experience a subsequent crash? If you do, please provide info about the crash, there might be some in %localappdata%\avidemux\admlog.txt (the log file will be overwritten when Avidemux is restarted).

Shouldn't avidemux be able to re-execute its own scripts?

It has successfully executed the script. It doesn't verify that the configuration passed to the encoder it to encoder liking, however. This is beyond the scope of the internal script writer.

In any case, please use a more recent build, The last release (2.7.6) has got long in the tooth.

[gamez]

Hi

I am very confused by your reply. The script was as a whole created and generated by avidemux itself without any manual changes from my side. This should clarify your questions related to setPostProc and audioCodec.

Regarding setContainer: I had also tried all other available containers with the exact same result. avidemux crashes already much earlier, when trying to start and setup the encoder. The conatiner is not the issue in my belief. I also tried many other codecs, again with the exact same result.

The MP4v2 muxer is deprecated, please use MP4.

I thought it was the other way around... Never mind, the "old" MP4 produces the exact same error.

avidemux doesn't crash. It gives an error and says that outputting the file has failed.

BTW, what is very strange: When I load the script, the very first frame from the .ts is shown in the display, although the first segment starts with a PTS of 2 min 53 sec into the file. Only when I start navigating, the segment's first frame is shown at position 0 (actually not quite zero, but at 256 ms). Also, the "Cannot set up encoder" is followed by "Error seeking to 256 ms".

Actually I think there is a problem in avidemux when he tries to find the start of the first PTS in the segment, and using that as the new starting point of its processing.

g.

[eumagga0x2a]

The script was as a whole created and generated by avidemux itself without any manual changes from my side. This should clarify your questions related to setPostProc and audioCodec.

I pointed out some things in the script which raised questions. They are not necessarily knife-sharp focussed on addressing the failure to setup encoder.

I recommended you to disable post-processing in Avidemux preferences. The scripted audioCodec configuration is invalid, I need to check how it is possible that Avidemux creats it this way.

Regarding setContainer: I had also tried all other available containers with the exact same result.

I didn't mean that. Just please don't use MP4v2 if you want to save to mp4, use MP4. Of course, this is not related to the failure.

I thought it was the other way around...

Exactly. libmp4v2-based "MP4v2" is old and stale (it is not present in VC++ builds anyway), the other one is very much current.

Actually I think there is a problem in avidemux when he tries to find the start of the first PTS in the segment, and using that as the new starting point of its processing.

This is another, unrelated problem, I think.

[gamez]

Hi

The hickup in the audioCodec config string seems to be related to switching from a compressor setting (like AC3 Aften) to the copy setting, and/or in-/excluding source audio tracks into the target (actually my .ts recording has one mp2 and one ac3 track).

I strongly believe that there is a problem with carrying over the selected range of frames of the segments from the interactive editing to the py script, and/or using those directly from a reloaded py script without loading the base file (.ts) in the first place. There is some de-synchronization going on that I will try to isolate further in my tests. Could also be related to the type of frame (I, B or P) that is present at the segment borders since in case of an MPEG-2 transport stream the full frame(s) needed to construct an image inside the selected segment(s) might lie outside of it.

I will let you know what I find.

g.

[eumagga0x2a]

Please just leave out all potentially problematic parts for now and add the deinterlacer (and use the latest nightly, of course):

Code Select Expand

#PY  <- Needed to identify #
#--automatically built--

adm = Avidemux()
if not adm.loadVideo("Z:/TVRip/20180113 1911 - ProSieben - Storm Hunters.ts"):
    raise("Cannot load Z:/TVRip/20180113 1911 - ProSieben - Storm Hunters.ts")
adm.videoCodec("x264", "useAdvancedConfiguration=True", "general.params=2PASSBITRATE=3376", "general.threads=0", "general.preset=ultrafast", "general.tuning=none", "general.profile=baseline", "general.fast_decode=False", "general.zero_latency=False"
, "general.fast_first_pass=True", "general.blueray_compatibility=False", "general.fake_interlaced=False", "level=-1", "vui.sar_height=1", "vui.sar_width=1", "MaxRefFrames=3", "MinIdr=25", "MaxIdr=250"
, "i_scenecut_threshold=40", "intra_refresh=False", "MaxBFrame=3", "i_bframe_adaptive=1", "i_bframe_bias=0", "i_bframe_pyramid=2", "b_deblocking_filter=True", "i_deblocking_filter_alphac0=0", "i_deblocking_filter_beta=0"
, "cabac=True", "interlaced=False", "constrained_intra=False", "tff=True", "fake_interlaced=False", "analyze.b_8x8=True", "analyze.b_i4x4=True", "analyze.b_i8x8=True", "analyze.b_p8x8=True", "analyze.b_p16x16=False"
, "analyze.b_b16x16=False", "analyze.weighted_pred=2", "analyze.weighted_bipred=True", "analyze.direct_mv_pred=1", "analyze.chroma_offset=0", "analyze.me_method=1", "analyze.me_range=16", "analyze.mv_range=-1"
, "analyze.mv_range_thread=-1", "analyze.subpel_refine=7", "analyze.chroma_me=True", "analyze.mixed_references=True", "analyze.trellis=1", "analyze.psy_rd=1.000000", "analyze.psy_trellis=0.000000", "analyze.fast_pskip=True"
, "analyze.dct_decimate=True", "analyze.noise_reduction=0", "analyze.psy=True", "analyze.intra_luma=11", "analyze.inter_luma=21", "ratecontrol.rc_method=0", "ratecontrol.qp_constant=0", "ratecontrol.qp_min=10"
, "ratecontrol.qp_max=51", "ratecontrol.qp_step=4", "ratecontrol.bitrate=0", "ratecontrol.rate_tolerance=1.000000", "ratecontrol.vbv_max_bitrate=0", "ratecontrol.vbv_buffer_size=0", "ratecontrol.vbv_buffer_init=1"
, "ratecontrol.ip_factor=1.400000", "ratecontrol.pb_factor=1.300000", "ratecontrol.aq_mode=1", "ratecontrol.aq_strength=1.000000", "ratecontrol.mb_tree=True", "ratecontrol.lookahead=40")
adm.addVideoFilter("yadif", "mode=1", "parity=0", "deint=0")
adm.setContainer("MP4", "muxerType=0", "optimize=1", "forceAspectRatio=True", "aspectRatio=1", "rotation=0", "clockfreq=0")
If x264 still doesn't like the configuration, we should check deeper (libx264 used by my Avidemux build accepts it).

[gamez]

The problem is related to segments and/or markers.
Running the encoder with the whole .ts file never did give any problems, neither in interactive nor in batch mode.

[eumagga0x2a]

Please provide the log.

[gamez]

So it turns out that, occasionally, the number saved to the py script to indicate the start of the segment is wrong.

For example, when I cut the .ts at the I-frame at 04:54.496 this is saved as 294496244. This can be explained by 4 mins and 54 secs and 496 ms being equal to 294496 ms, followed by the number 244 I cannot explain. An encoder run (in batch mode with --load py and with this starting position succeeds.

The next frame at position 04:54.536 (in a 25 fps .ts video, every frame "occupies" 40 ms) is a B-frame and is saved as 294536244. An encoder run also works OK.

But then, the next frame at 04:54.576 is again a B-frame and for some reason saved as 294576256, and the encoder crashes immediately. When I change it to 294576244, it succeeds.

Again, the next frame at 04:54.616 is a P-frame and is wrongly saved as 294616256, and the encoder crashes. After correction to 294616244, it works.

I don't know what the trailing 244 (or 256) stands for, but as least I have found the reason.

g.

[eumagga0x2a]

For example, when I cut the .ts at the I-frame at 04:54.496 this is saved as 294496244. This can be explained by 4 mins and 54 secs and 496 ms being equal to 294496 ms, followed by the number 244 I cannot explain.

Internally, Avidemux keeps track of time in microseconds (µs).

But then, the next frame at 04:54.576 is again a B-frame and for some reason saved as 294576256, and the encoder crashes immediately.

Might be some rounding error. Please provide the log from the latest nightly after deleting *.idx2 so that I can assess the situation. A failure to setup encoder is not a crash.

[gamez]

Please provide the log

I would love to, if only this forum would allow me to embed it into my post, attach to it as zip, or link to it on pastebin. But all of that is detected as spam and forbidden. I guess I'm out of luck and options.

BTW, besides the bugs with the audioCodec conf and the glitch in recording correct microseconds, I'm convinced there's a bug in having addSegment and markerA/B play nicely together with each other whenever a segment doesn't start at file position 0. In such cases the script runs differently depending on whether you specify adm.markerA = 0 and adm.markerB = <length_of_segment> or leave them out. Normally the result should be the same but it is not. And as soon as you stack several segments together, the results seem to become fully unpredictable. I am suspecting that the markers sometimes refers back to the initial .ts file from where the segments stem from.

I will continue my investigations and post what I find (if the forum allows me to).

g.

[eumagga0x2a]

If CleanTalk misbehaves again and blocks totally legitimate posts as spam, it may help trying to post from a private browser window, else please don't hesitate to provide the info per private message.

It is definitely possible to attach files with filename extension .txt like happened here.

addSegment() expects time in reference (in source video) as the second and the third argument, their values are specific to the particular video. The markers A and B are a completely different story.

[gamez]

Hi

tx for the suggestion, so I sent you a PM.

Besides, in your view, should putting adm.markerA = 0 and adm.markerB = <sum_of_length_of_segment(s)> yield the same result as leaving them both out (what is what I preferably do)? In all my logic the answer should be yes, but in fact, it is no.

In my understanding, markers delimit the portion of the file that you want to encode. The problem here is, what means "file": the base (.ts) file or the concatenated segments?

In my tests, for a single segment, it's still quite OK, but for more than one concatenated segment with gaps in between, the encoding goes a little berserk and tends to continue after the end of the first segment instead of "jumping" to the second one, sometimes even until the very end of the base .ts file. So yes, there is interdependance between markers and segments.

g.

[eumagga0x2a]

Code Select Expand

[setup] 22:22:00-665 Creating x264 encoder
[setup] 22:22:00-665 x264, setup ok
Encoder setup is not an issue, the reason for failure is

Code Select Expand

[TimeToFrame] 22:22:00-650 We reached frame 7362 with a PTS of 294616244 when looking for PTS 294576256
[getFrameNumFromPtsOrBefore] 22:22:00-650 Best candidate for time 00:04:54,576 in reference is frame 7361
[seektoTime] 22:22:00-650 Seeking to frame 7356 at 294496 ms
[DecodePictureUpToIntra] 22:22:00-650 Decoding up to intra frame 7356, ref: 0
[edCache] Flush
[seektoTime] 22:22:00-665 Image not found, looked for 294576256 us, got pts=294616244 us
the specified start of the segment in reference which doesn't match a frame.

Code Select Expand

# adm.setPostProc(3, 3, 0)
This is not what I meant, I suggested to disable post-processing in Avidemux preferences alltogether, equivalent to

Code Select Expand

adm.setPostProc(0, 0, 0)

Code Select Expand

*************************
  Avidemux v2.7.7 (200714_c5a3dd2fb4b) .
*************************
 http://www.avidemux.org
 Code      : Mean, JSC, Grant Pedersen,eumagga0x2a
 GFX       : Nestor Di, nestordi@augcyl.org
 Design    : Jakub Misak
 FreeBSD   : Anish Mistry, amistry@am-productions.biz
 Audio     : Mihail Zenkov
 Mac OS X  : Kuisathaverat, Harry van der Wolf
 Win32     : Grant Pedersen
 
Compiler: GCC 5.4.0
Build Target: Microsoft Windows (x86)
User Interface: CLI
Operating System: Microsoft Windows (6.2.9200; 32-bit)
Time: Tue Nov 03 23:22:00 2020
 
0: C:\Program Files\Avidemux 2.7 - 32 bits\avidemux_cli.exe
I really should have asked first. The legacy compatibility branch, used to generate 32-bit builds, has not been updated since release, I don't know whether I'll be able to backport the truckload of commits from git master for a final 32-bit 2.7.7 (which will be the anticipated end of support for Windows XP and 32-bit in general). I can only recommend to switch to a 64-bit operating system.

[eumagga0x2a]

Besides, in your view, should putting adm.markerA = 0 and adm.markerB = <sum_of_length_of_segment(s)> yield the same result as leaving them both out (what is what I preferably do)? In all my logic the answer should be yes, but in fact, it is no.

In my logic, the answer is "yes" too.

In my understanding, markers delimit the portion of the file that you want to encode. The problem here is, what means "file": the base (.ts) file or the concatenated segments?

Markers are in linear time (sum of segment durations), the number of files is irrelevant. Segments are in reference time.

In my tests, for a single segment, it's still quite OK, but for more than one concatenated segment with gaps in between, the encoding goes a little berserk and tends to continue after the end of the first segment instead of "jumping" to the second one, sometimes even until the very end of the base .ts file.

Specifying arbitrary values for segment start / end may cause unexpected behaviour, right. Doing this may break segment switching. Values, created by Avidemux GUI, should work if the video is free of defects (except for bugs – in this matter it is fatal that you cannot try a current build as you run a 32-bit Windows).