Author Topic: nVidia HEVC encoder not working very well  (Read 3750 times)

Shira

  • Newbie
  • *
  • Posts: 5
nVidia HEVC encoder not working very well
« on: December 17, 2016, 05:05:58 AM »
Hi, been using Avidemux for a long time, mostly encoding game recordings using H.264 nvidia (GTX 660) encoder in avidemux. Today I got a 1070 GTX so I was excited to try the h.265 encoder, but there's something very wrong, I'm testing with a short video file, the encoder is very fast but when I play the video it's very choppy when I try to skip back/forward, the image freezes for 3-4 seconds or more then continues. When using the software based encoder instead, there are no issues with the encoded file.
I tried also with another app: MediaCoder, using also NVENC H.265, no issues there.

Some data:

Avidemux H.265+NVIDIA:

Quote
Format                      : Matroska
Format version              : Version 4 / Version 2
File size                   : 34.4 MiB
Duration                    : 1 min 50 s
Overall bit rate            : 2 622 kb/s
Writing application         : Lavf57.25.100
Writing library             : Lavf57.25.100 / Lavf57.25.100

Video
ID                          : 1
Format                      : HEVC
Format/Info                 : High Efficiency Video Coding
Format profile              : Main@L4@High
Codec ID                    : V_MPEGH/ISO/HEVC
Duration                    : 1 min 50 s
Bit rate                    : 2 378 kb/s
Width                       : 640 pixels
Height                      : 360 pixels
Display aspect ratio        : 16:9
Frame rate mode             : Constant
Frame rate                  : 23.976 (24000/1001) FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Bits/(Pixel*Frame)          : 0.430
Stream size                 : 31.2 MiB (91%)
Default                     : Yes
Forced                      : No
DURATION                    : 00:01:50.193000000


MediaCoder H.265+NVIDIA:

Quote
Format                      : Matroska
Format version              : Version 4 / Version 2
File size                   : 14.2 MiB
Duration                    : 1 min 50 s
Overall bit rate            : 1 080 kb/s
Encoded date                : UTC 2016-12-17 04:49:48
Writing application         : mkvmerge v9.1.0 ('Little Earthquakes') 64bit
Writing library             : libebml v1.3.3 + libmatroska v1.4.4

Video
ID                          : 1
Format                      : HEVC
Format/Info                 : High Efficiency Video Coding
Format profile              : Main@L2.1@Main
Codec ID                    : V_MPEGH/ISO/HEVC
Duration                    : 1 min 49 s
Bit rate                    : 1 015 kb/s
Width                       : 640 pixels
Height                      : 360 pixels
Display aspect ratio        : 16:9
Frame rate mode             : Constant
Frame rate                  : 23.976 (24000/1001) FPS
Original frame rate         : 23.000 FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Bits/(Pixel*Frame)          : 0.184
Stream size                 : 13.3 MiB (94%)
Default                     : Yes
Forced                      : No

Avidemux H.265 cpu:

Quote
Format                      : Matroska
Format version              : Version 4 / Version 2
File size                   : 12.2 MiB
Duration                    : 1 min 50 s
Overall bit rate            : 926 kb/s
Writing application         : Lavf57.25.100
Writing library             : Lavf57.25.100 / Lavf57.25.100

Video
ID                          : 1
Format                      : HEVC
Format/Info                 : High Efficiency Video Coding
Format profile              : Main@L2.1@Main
Codec ID                    : V_MPEGH/ISO/HEVC
Duration                    : 1 min 50 s
Bit rate                    : 716 kb/s
Width                       : 640 pixels
Height                      : 360 pixels
Display aspect ratio        : 16:9
Frame rate mode             : Constant
Frame rate                  : 23.976 (24000/1001) FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Bits/(Pixel*Frame)          : 0.130
Stream size                 : 9.42 MiB (77%)
Writing library             : x265 1.9:[Windows][GCC 4.9.1][64 bit] 8bit
Encoding settings           : wpp / ctu=64 / min-cu-size=8 / max-tu-size=32 / tu-intra-depth=1 / tu-inter-depth=1 / me=3 / subme=5 / merange=16 / no-rect / no-amp / max-merge=2 / temporal-mvp / early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=250 / min-keyint=25 / scenecut=40 / rc-lookahead=40 / lookahead-slices=0 / bframes=3 / bframe-bias=0 / b-adapt=1 / ref=3 / limit-refs=3 / no-limit-modes / weightp / weightb / aq-mode=2 / qg-size=32 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=1.00 / rdoq-level=0 / psy-rdoq=0.00 / signhide / deblock / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=20.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30
Default                     : Yes
Forced                      : No
DURATION                    : 00:01:50.360000000

Jan Gruuthuse

  • Hero Member
  • *****
  • Posts: 6060
Re: nVidia HEVC encoder not working very well
« Reply #1 on: December 17, 2016, 09:13:51 AM »
Can you please state your used OS, version and if it is 32- or 64-bit operating?
Upload the sample source video to test with?

ps.: upload:
Use a free dropbox account, https://www.wetransfer.com/, mega or similar web service (free public access, without registration to download your uploaded video) thank you.

Shira

  • Newbie
  • *
  • Posts: 5
Re: nVidia HEVC encoder not working very well
« Reply #2 on: December 17, 2016, 10:20:28 AM »
Hi, using latest versions

Avidemux 64-bit 2.6.15
Windows 10 x64 Pro 1607

Avidemux H265
https://drive.google.com/open?id=0B0Kd3pQSz9EkaXQ2MHh4ellIMVk

Mediacoder H265
https://drive.google.com/open?id=0B0Kd3pQSz9EkRGlZSjJlUXZDM0U

eumagga0x2a

  • Moderator
  • Hero Member
  • *****
  • Posts: 2738
Re: nVidia HEVC encoder not working very well
« Reply #3 on: December 17, 2016, 11:01:04 AM »
The first video triggers the same error as mentioned in https://github.com/mpv-player/mpv/issues/3440, which implies a muxing issue. The seeking to keyframes is totally broken with this video in Avidemux, it navigates to P-frames instead of intra (but displays correct frame type).

Shira

  • Newbie
  • *
  • Posts: 5
Re: nVidia HEVC encoder not working very well
« Reply #4 on: December 17, 2016, 11:27:49 AM »
Someone mentioned VLC there, so I tried, it kinda does the same:



and thumbnail preview:



Need me to keep the test files up for a while?

EDIT: same issue with H264 (nvidia enc)
« Last Edit: December 17, 2016, 11:32:37 AM by Shira »

eumagga0x2a

  • Moderator
  • Hero Member
  • *****
  • Posts: 2738
Re: nVidia HEVC encoder not working very well
« Reply #5 on: December 17, 2016, 11:35:27 AM »
Once Mean, the only one whe can understand and fix the issue, has downloaded the samples (maybe this has already happend?), you can take them offline, sure.

mean

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10967
Re: nVidia HEVC encoder not working very well
« Reply #6 on: December 17, 2016, 07:27:13 PM »
For some reasons, all frames are tagged as intra

eumagga0x2a

  • Moderator
  • Hero Member
  • *****
  • Posts: 2738
Re: nVidia HEVC encoder not working very well
« Reply #7 on: December 17, 2016, 07:37:04 PM »
For some reasons, all frames are tagged as intra

But UI_setFrameType gets it right, right?

edit: I wonder if it is possible to workaround such cases rechecking the frame type at the target PTS and iterating until there is a true intra.
« Last Edit: December 17, 2016, 07:40:11 PM by eumagga0x2a »

mean

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10967
Re: nVidia HEVC encoder not working very well
« Reply #8 on: December 17, 2016, 07:53:23 PM »
Seems fixed in the win64 nightly

eumagga0x2a

  • Moderator
  • Hero Member
  • *****
  • Posts: 2738
Re: nVidia HEVC encoder not working very well
« Reply #9 on: December 17, 2016, 08:09:29 PM »
What is required to cross-build with Nvenc support in MXE?

mean

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10967
Re: nVidia HEVC encoder not working very well
« Reply #10 on: December 17, 2016, 09:10:34 PM »
Just the nvenc header
nvEncodeAPI.h

eumagga0x2a

  • Moderator
  • Hero Member
  • *****
  • Posts: 2738
Re: nVidia HEVC encoder not working very well
« Reply #11 on: December 17, 2016, 09:21:56 PM »
Thank you, just realized how easy it is (the header turns out to be available via rpmfusion.org, just needed the latest nvidia binary drivers to use the feature). Trying to figure out how to build on Linux first (the Debian-specific value for PATHS  – /usr/include/x86_64-linux-gnu – must be replaced by /usr/include/nvenc on Fedora, but then the ffmpeg configure fails to find the header).
« Last Edit: December 17, 2016, 09:31:03 PM by eumagga0x2a »

eumagga0x2a

  • Moderator
  • Hero Member
  • *****
  • Posts: 2738
Re: nVidia HEVC encoder not working very well
« Reply #12 on: December 17, 2016, 09:26:07 PM »
Quick and very dirty workaround works:

Code: [Select]
# ln -s /usr/include/nvenc/nvEncodeAPI.h /usr/include/
Building now.

eumagga0x2a

  • Moderator
  • Hero Member
  • *****
  • Posts: 2738
Re: nVidia HEVC encoder not working very well
« Reply #13 on: December 17, 2016, 10:43:33 PM »
I've opened a pull request for a (hopefully) proper fix for nvenc detection and ffmpeg configuration.

I'm very impressed by the astonishing speed of encoding in Avidemux with NVENC on Linux (h264 only, HEVC is not available with my hardware). Confirming the issue reported by Shira fixed.

Shira

  • Newbie
  • *
  • Posts: 5
Re: nVidia HEVC encoder not working very well
« Reply #14 on: December 18, 2016, 05:42:51 PM »
Seems fixed in the win64 nightly

where do you get the nightlies?