Seeking in TS file not working in some Sky recordings

[busch]

Hello there 

I have an annoying problem with recordings from Sky Germany. The recordings are stored as Transport Stream (using an enimga2 based receiver).

Indexing towards the end does not work properly, so seeking is not possible. There's a huge gap (several minutes) starting from around before the end credits roll right after the end of the movie when the Sky logo is showing again. This happens in A LOT movies of Sky I've recorded (dating back a year or so). Other stations are fine. I guess there's some sh*** happening at that point, because they sometimes insert ads right when the end credits start rolling.    

Seeking via mouse in this area shows nothing. When using the cursor UP+DOWN keys Avidemux jumps to the first/last I-frame forth and back, skipping that part where basically the end credits roll. However, playback into this area from the beginning of the "last good I-frame" works quite fine. And then even cursor LEFT+RIGHT work there to step to the next/previous B-, P- and even I-frames. But pressing UP+DOWN then jumps right to the start/end of that area.

After remuxing that file into a MKV container (my usual workflow) the result seems to be corrupted towards the end: jumping around by randomly clicking the seekbar in VLC seems a bit slow and after jumping into that end credits area VLC starts hanging. In contrast, the original TS file works quite fine in VLC.

I'm using the latest Avidemux Win64 (190102_8fa92ee0d24-fflibs 4.1) and even the old release version 2.7.1 and they both behave the same.  I quickyl tried the Win32 version from today, and there's no difference.

Fortunately I worked around this issue by using plain ffmpeg to convert the TS file to a MP4 container first. With that MP4 file seeking in Avidemux works fine and even the remuxed MKV from Avidemux works flawless in VLC.

I chopped off the last 200 MB of a movie that shows that behaviour. I'm currently looking where to upload it.

best regards,
busch

[eumagga0x2a]

Please use WeTransfer, Mega, Dropbox or Google Drive to provide a sample.

If keyframes are not marked as such in a portion of the stream, the result will be as you describe.

[busch]

Here is the snippet: https://we.tl/t-ISaSlZZCce

I've already tried some analyzing tools (ffprobe, media info etc), but couldn't see any problem. Probably due to lack of knowledge. 

[eumagga0x2a]

Thank you for the sample, Avidemux detects a lot of keyframes while indexing, but the internal lavcodec disagrees (edit: it is exactly the other way round) (it thinks also that the video is interlaced). I'll look into it.

[eumagga0x2a]

The thing is that there are two kinds of complete frames: intra (I-frame) and IDR. The latter means that no future frame may need any data from any frame before the IDR in order to be decoded. Only IDR guarantee that a video, cut at these frames, will be free of artifacts because of missing reference pictures. Keyframe based navigation seeks to IDR frames.

When no frames in a MPEG-TS file are explicitely marked as IDR, Avidemux optimistically assumes that all intras may be treated like IDR.

In your sample, some frames (very few) are really marked by the stream generator as IDR and Avidemux obeys, marking only those frames as keyframes (the frame type in Avidemux GUI is set post-factum by the decoder). This is the origin of the terribly long gap where no keyframe based navigation is possible.

In this particular case, it turns out to be that the stream generator was at least mostly or even all the time wrong and treating intra as IDR like ffmpeg does works fine. But what works for this stream may cause broken videos after cut in another one.

[eumagga0x2a]

I would suggest the following workaround: open the corresponding .idx2 file with a text editor like Notepad++ which can handle Unix-type newline characters (LF alone vs CR LF on Windows) and replace all occurrences of

Code Select Expand

IF:

with leading space (which might get lost when posted here) with

Code Select Expand

DF:

(also with leading space) so that a line of the index representing a GOP like

Code Select Expand

Video at:0005dc9c:001c Pts:4176147336:4176129336  IF:01939d:0:0 BF:0049b8:-7200:3600 BF:001d2b:-10800:-1 BF:001dc8:-3600:10800 PF:00f1eb:14400:14400 BF:004b45:7200:18000 BF:002269:3600:-1 BF:002130:10800:25200 PF:00b083:28800:28800 BF:004cb9:21600:32400 BF:001947:18000:-1 BF:001a36:25200:39600 PF:00fae4:43200:43200 BF:004f2b:36000:46800 BF:002265:32400:-1 BF:00227c:39600:54000 PF:00cab6:57600:57600 BF:0040f4:50400:61200 BF:001bac:46800:-1 BF:001b20:54000:68400 PF:00d8dd:72000:72000 BF:004262:64800:75600 BF:001e07:61200:-1 BF:001c5e:68400:82800

turns into

Code Select Expand

Video at:0005dc9c:001c Pts:4176147336:4176129336  DF:01939d:0:0 BF:0049b8:-7200:3600 BF:001d2b:-10800:-1 BF:001dc8:-3600:10800 PF:00f1eb:14400:14400 BF:004b45:7200:18000 BF:002269:3600:-1 BF:002130:10800:25200 PF:00b083:28800:28800 BF:004cb9:21600:32400 BF:001947:18000:-1 BF:001a36:25200:39600 PF:00fae4:43200:43200 BF:004f2b:36000:46800 BF:002265:32400:-1 BF:00227c:39600:54000 PF:00cab6:57600:57600 BF:0040f4:50400:61200 BF:001bac:46800:-1 BF:001b20:54000:68400 PF:00d8dd:72000:72000 BF:004262:64800:75600 BF:001e07:61200:-1 BF:001c5e:68400:82800

Save the index file, reload the video in Avidemux. Now all intras will be treated as IDR = keyframes.

[busch]

haha this works    Neat solution! The resulting mkv works like a charme in VLC. I'll try to automate this in my workflow.

Is there any chance to detect such flaky source files in Avidemux and treat them differently? I bet there are a few more people with such problems out there...

[fish]

This thread is interesting in relation to DVB-T HD streams. There is definitely a problem which randomly causes Avidemux to mess up the GOP around certain I-Frame edits causing a freeze or artifacts between I-Frames. One strange thing I have noticed is that a stream of the same show from a '+1' channel, edits in exactly the same way ie, edits at the same points cause problems/don't cause problems. I think I will have a try at this, even I MIGHT be able to manage something like this.

[eumagga0x2a]

This is a known issue, seems to have become less grave with ffmpeg >= 4.1. It looks like Avidemux is not at fault, it is the stream being too optimistic about the IDR quality of I-frames (the exact opposite of the situation identified in this topic).

The alternative would be to decode (at least frame headers) the entire video and eventually disallow copy mode. Not viable.

[fish]

Ah ok, disabling copy mode would not make any sense, especially when moving the edit as little as one I-frame back or forwards often sorts out the problem.