Nvidia HEVC problems

[eumagga0x2a]

With lookahead at 0 there were no I-frames placed, then I tried lookahead of 31 and 10, and they were placed exactly where needed.

Thanks for confirmation.

I used to think that scene cuts were always on by default, or at least on "better" presets. With voukoder plugin (ffmpeg-based) and my ffmpeg scripts I just always used "slow" preset so I thought that makes the difference, because I had the auto I-frames. I guess this could've been just because I always specified lookahead.

The "slow" preset is "hq" + what NVIDIA calls "2-pass" (which is not 2-pass one would think of). The "slow" preset is definitely slower on my hardware than just "hq" despite no scenecut and no lookahead support, so the encoder is putting more effort in... something.

If you happen to find some spare time, it would be indeed interesting to determine experimentally whether "slow" on your hardware implies scenecut detection even with lookahead option not specified on the ffmpeg command line.

Just to clarify, when you say libavcodec, it's the same thing that's in FFMPEG, right?

Yes, the same library. FFmpeg = libavutil + libavcodec + libavformat + libavfilter + libswscale + libpostproc + libavdevice + libavresample + libswresample + a set of command-line frontends to use all of them.

Avidemux bundles libavutil, libavcodec, libavformat, libswscale and libpostproc. Some parts are patched to expose internal, highly sophisticated functions to external usage by Avidemux (very few patches actually modify behaviour).

One more thing, in Avidemux, for both encoders, in variable bitrate mode, Quality value entry field is now unlocked, which confused me again. Is it supposed to be this way? It has an effect even in VBR mode?

This is typical corner cutting. The sematics of the parameter is silently switched from constant quantisation parameter (qp) in "Constant Quantizer" (constqp) mode to constant quality (cq) in "Variable Bitrate" mode. It should have a major effect in VBR mode, with "0" meaning "automatic quality".

[Urik]

The "slow" preset is "hq" + what NVIDIA calls "2-pass" (which is not 2-pass one would think of)

Yes, apart from the description, nothing about it resembles 2-pass (in the regular sense) at all. It just goes in one pass in ffmpeg, perhaps just slower.

If you happen to find some spare time, it would be indeed interesting to determine experimentally whether "slow" on your hardware implies scenecut detection even with lookahead option not specified on the ffmpeg command line.

I think I tested yesterday, but at this point I start to forget what I tested because of multiple settings etc.

I re-tested, and it seems that in Avidemux there's no scenecut detection regardless of preset.
The rate control mode doesn't seem to affect it, as well as additional b-frame related settings and weighted prediction.

I also tested same with my ffmpeg scripts, where for the test I removed lookahead and did not define gop, and there, in

• h264_nvenc > I DO get scenecut detection in Slow and hq preset, but don't get it with default p4 preset.
• hevc_nvenc > I DON'T get scenecut detection even with slow preset (without lookahead)
  

Settings were the same between h264/hevc apart from omission of b-frames on hevc and main profile instead of high.

My ffmpeg is fairly new though, it's a December "git" build (not "release") and it even requires a fairly new driver (because of API version or something). It has these extra "p" presets

-preset            <int>        E..V...... Set the encoding preset (from 0 to 18) (default p4)
     default         0            E..V......
     slow            1            E..V...... hq 2 passes
     medium          2            E..V...... hq 1 pass
     fast            3            E..V...... hp 1 pass
     hp              4            E..V......
     hq              5            E..V......
     bd              6            E..V......
     ll              7            E..V...... low latency
     llhq            8            E..V...... low latency hq
     llhp            9            E..V...... low latency hp
     lossless        10           E..V......
     losslesshp      11           E..V......
     p1              12           E..V...... fastest (lowest quality)
     p2              13           E..V...... faster (lower quality)
     p3              14           E..V...... fast (low quality)
     p4              15           E..V...... medium (default)
     p5              16           E..V...... slow (good quality)
     p6              17           E..V...... slower (better quality)
     p7              18           E..V...... slowest (best quality)

So I guess it's all kind of tied to driver support, and GPU family / nvenc generation.

I've noticed there's a new VC++ build (of avidemux) now, since personally (normally) I always use VC, I'll be using the updated one now, cool.

[eumagga0x2a]

I re-tested, and it seems that in Avidemux there's no scenecut detection regardless of preset.

Was it with lookahead set to 0?

I also tested same with my ffmpeg scripts, where for the test I removed lookahead and did not define gop, and there, in

• h264_nvenc > I DO get scenecut detection in Slow and hq preset, but don't get it with default p4 preset.
• hevc_nvenc > I DON'T get scenecut detection even with slow preset (without lookahead)

Very interesting, thank you for sharing your findings.

Support for the new P1 - P7 presets has been added in FFmpeg only on the trunk so far, Avidemux uses the 4.2.x branch.

[Urik]

Was it with lookahead set to 0?

Yes. Just tested again. And then with lookahead - even value of 1 is enough to trigger scenecut. I used GOP of 250 in most tests so it's more apparent.

[Jiehfeng]

Hey, sorry for the inactivity. Tried the latest nightly build and the Constant Bitrate mode, works perfectly now!

[eumagga0x2a]

Tried the latest nightly build and the Constant Bitrate mode, works perfectly now!

Great, glad to hear that. I hope that other, more useful RC modes work fine too (CBR is obviously the worst one quality- and efficiency-wise, but good for troubleshooting).

[Jiehfeng]

Tried the latest nightly build and the Constant Bitrate mode, works perfectly now!

Great, glad to hear that. I hope that other, more useful RC modes work fine too (CBR is obviously the worst one quality- and efficiency-wise, but good for troubleshooting).

Just tried the others, all work fine!

[macnuts]

Thank you for fixing NVidia HEVC settings, it encodes great in Avidemux GUI application now.

Yet there are problems due to some kind of Nvidia H264 and Nvidia HEVC encoders clash.
I've checked with the latest avidemux_2.7.8 r210306_win64.exe and the problem is still there.

• NVidia HEVC video output jobs are wrongly encoded as Nvidia H264.
• NVidia HEVC saved defaults (Ctrl+Alt+D) are loaded as Nvidia H264.

Looks like the problem is that both encoders share the same name ffNvEnc, see log entries below.

Code Select Expand

[videoEncoder6]Name :ffNvEnc ApiVersion :7 Description :Nvidia hw encoder
[nvCheckDll] 09:39:52-136 checking nvcuda.dll-> 1
[nvCheckDll] 09:39:52-136 checking nvEncodeAPI64.dll-> 1
[VideoEncoder6] Registered filter C:\Program Files\Avidemux 2.7 Qt 64 bits\plugins\videoEncoders\libADM_ve_ffNvencH264.dll as  Nvidia hw encoder
[videoEncoder6]Name :ffNvEnc ApiVersion :7 Description :Nvidia hw encoder
[nvCheckDll] 09:39:52-137 checking nvcuda.dll-> 1
[nvCheckDll] 09:39:52-138 checking nvEncodeAPI64.dll-> 1
[VideoEncoder6] Registered filter C:\Program Files\Avidemux 2.7 Qt 64 bits\plugins\videoEncoders\libADM_ve_ffNvencHEVC.dll as  Nvidia hw encoder
My current workaround is to change the extension of the libADM_ve_ffNvencH264.dll so it is ignored by avidemux but it would be nice to have this sorted out.

[eumagga0x2a]

Indeed, a pity that no one reported this bug during the last release cycle (my NVIDIA hardware doesn't support encoding to HEVC so that this went unnoticed). If you can handle a hex editor, please replace the byte sequence

Code Select Expand

66 66 4E 76 45 6E 63 00
with

Code Select Expand

66 66 4E 76 45 6E 34 00
(ffNvEnc --> ffNvEn4) in libADM_ve_fNvencH264.dll and with

Code Select Expand

66 66 4E 76 45 6E 35 00
(ffNvEnc --> ffNvEn5) in libADM_ve_ffNvencHEVC.dll respectively as a workaround to resolve the conflict for now.

[eumagga0x2a]

Should be fixed by this commit. The NVENC encoder name in old project files needs to be edited to make them compatible with future Avidemux builds.

[macnuts]

Indeed, a pity that no one reported this bug during the last release cycle (my NVIDIA hardware doesn't support encoding to HEVC so that this went unnoticed). If you can handle a hex editor, please replace the byte sequence

Code Select Expand

66 66 4E 76 45 6E 63 00
with

Code Select Expand

66 66 4E 76 45 6E 34 00
(ffNvEnc --> ffNvEn4) in libADM_ve_fNvencH264.dll and with

Code Select Expand

66 66 4E 76 45 6E 35 00
(ffNvEnc --> ffNvEn5) in libADM_ve_ffNvencHEVC.dll respectively as a workaround to resolve the conflict for now.

Renaming the first occuranace of ffNvEnc\0 in both files as per instructions above did the trick, thank you.
Encoders are now recognized with their new names and there are no more problems with default settings or jobs (checked using both encoders).