Avidemux 2.7.1 can't load 30Gb MPEG-TS HEVC file

[douche]

reproduce
1) import this file
2) avidemux stucks at indexing on 17-18%
3) file

Code Select Expand

General
ID                                       : 1 (0x1)
Complete name                            : E:\FIFA.World.Cup.2018.Round.of.16.Spain.Vs.Russia.2160p.WEB-DL.AAC2.0.HDR.HEVC.ts
Format                                   : MPEG-TS
File size                                : 29.5 GiB
Duration                                 : 3 h 35 min
Overall bit rate mode                    : Variable
Overall bit rate                         : 19.6 Mb/s

Video
ID                                       : 256 (0x100)
Menu ID                                  : 1 (0x1)
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Commercial name                          : HDR10
Format profile                           : Main 10@L5.1@Main
Codec ID                                 : 36
Duration                                 : 3 h 35 min
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 50.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : BT.709
Mastering display luminance              : min: 0.0000 cd/m2, max: 0 cd/m2

Audio
ID                                       : 257 (0x101)
Menu ID                                  : 1 (0x1)
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format version                           : Version 2
Format profile                           : LC
Muxing mode                              : ADTS
Codec ID                                 : 15
Duration                                 : 3 h 35 min
Bit rate mode                            : Variable
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy

Menu
ID                                       : 4096 (0x1000)
Menu ID                                  : 1 (0x1)
Duration                                 : 3 h 35 min
List                                     : 256 (0x100) (HEVC) / 257 (0x101) (AAC)
Service name                             : Service01
Service provider                         : FFmpeg
Service type                             : digital television


[eumagga0x2a]

I don't think I'll try this at home, but would you please provide admlog.txt (compress it with 7z, gzip or zip if necessary) resulting from the attempt to index the video?

[douche]

idx2 file filled until 00:39:00.564 of video.
attached log.

[eumagga0x2a]

I'm sorry, but could you please downgrade to the latest release, delete the idx2 file and retry? I have silenced the debug printout from the H265 indexer post-release as it produced incredible amount of messages causing other debug information being lost.

[douche]

Can't attach file it's too big (-> https://www.dropbox.com/s/0uqjgn7zqh75r75/admlog.7z?dl=0)
cut of log

Code Select Expand

SliceType==> 3 xxx
Startcode =35:NAL_H265_AUD, decoding image=1,00:03:30,900
Startcode =39:NAL_H265_SEI_PREFIX, decoding image=0,00:03:30,900
Startcode =0:NAL_H265_TRAIL_N, decoding image=0,00:03:30,900
SliceType==> 3 xxx
Startcode =35:NAL_H265_AUD, decoding image=1,00:03:30,902
Startcode =39:NAL_H265_SEI_PREFIX, decoding image=0,00:03:30,902
Startcode =0:NAL_H265_TRAIL_N, decoding image=0,00:03:30,902
SliceType==> 3 xxx
Startcode =35:NAL_H265_AUD, decoding image=1,00:03:30,904
Startcode =39:NAL_H265_SEI_PREFIX, decoding image=0,00:03:30,904
Startcode =0:NAL_H265_TRAIL_N, decoding image=0,00:03:30,904
SliceType==> 3 xxx
[Ts Demuxer] Pes Packet too big
[Ts Demuxer] Pes for Pid =0x256 does not contain payload start
[Ts Demuxer] Pes for Pid =0x256 does not contain payload start
[Ts Demuxer] Pes for Pid =0x256 does not contain payload start
[Ts Demuxer] Pes for Pid =0x256 does not contain payload start
[Ts Demuxer] Pes for Pid =0x256 does not contain payload start
[Ts Demuxer] Pes for Pid =0x256 does not contain payload start
[Ts Demuxer] Pes for Pid =0x256 does not contain payload start
[Ts Demuxer] Pes for Pid =0x256 does not contain payload start
[Ts Demuxer] Pes for Pid =0x256 does not contain payload start
[Ts Demuxer] Pes for Pid =0x256 does not contain payload start
[Ts Demuxer] Pes for Pid =0x256 does not contain payload start
[Ts Demuxer] Pes for Pid =0x256 does not contain payload start
[Ts Demuxer] Pes for Pid =0x256 does not contain payload start


[eumagga0x2a]

Thank you, actually you have already pinpointed the issue:

Code Select Expand

[Ts Demuxer] Pes Packet too big

The PES packet payload size limit TS_PES_MAX_LIMIT, currently at https://github.com/mean00/avidemux2/blob/master/avidemux_plugins/ADM_demuxers/MpegTS/dmxTSPacket.cpp#L26, set to 2 MiB 9 years ago, is not sufficient for this stream. It would be great if you could provide a small (~300 MiB) sample, cut in binary way (like with the dd tool on Linux), from somewhere near the middle of the original TS file.

[douche]

I split file in 101 pieces by 300MiB with total commander.
So, near 20%
https://www.dropbox.com/s/of9g8xqxme7iyc2/FIFA.World.Cup.2018.Round.of.16.Spain.Vs.Russia.2160p.WEB-DL.AAC2.0.HDR.HEVC.020?dl=0
near 50%
https://www.dropbox.com/s/hgz0l80v5arsuk6/FIFA.World.Cup.2018.Round.of.16.Spain.Vs.Russia.2160p.WEB-DL.AAC2.0.HDR.HEVC.050?dl=0
near 80%
https://www.dropbox.com/s/iwpaqd0lgd6wc7b/FIFA.World.Cup.2018.Round.of.16.Spain.Vs.Russia.2160p.WEB-DL.AAC2.0.HDR.HEVC.080?dl=0
near 17%
https://www.dropbox.com/s/ihxrdji2oodq4m0/FIFA.World.Cup.2018.Round.of.16.Spain.Vs.Russia.2160p.WEB-DL.AAC2.0.HDR.HEVC.017?dl=0

Avidemux stucks at .017 and .080 files.

[eumagga0x2a]

Thank you for the samples. TS_PES_MAX_LIMIT should be at least 2637792 to allow Avidemux to index the stream. I'm trying to find out whether it is safe to bump the limit.

[eumagga0x2a]

Please try the latest nightly (r180807 or later) and report back whether it is able to index all your 4k TS files.

[douche]

All my huge .ts files processed.
Thanks.