Broken frames due to H.264 Open-GOP (DVB MPEG-TS) ?

[odin]

Hi,
I'm using avidemux to cut mpeg-ts streams in copy-mode at I-frames.
It's working quite good, and I never had problems with SD content (mpeg2).
But HD content (h.264) is often causing broken frames :-( (e.g. the frames after a deleted section).

I tried to verify, if the broken frames were caused by Open-GOP, but according ffprobe all i-frames are IDR frames (i.e. it's Closed-GOP, as according to a forum post, key_frame=1 means "IDR"):

Code Select Expand

# ffprobe -select_streams v -show_frames -of csv -show_entries frame=key_frame,pict_type video.mts
frame,1,I
But I don't know, how reliable ffprobe and that forum post really is ...  :-(

And avidemux Adm-Log also says "Closed-GOP" when opening the file:

Code Select Expand

Simple loading:
file: input.mts, size: 10485760
found 1 files
Done
[TsPacket] Detecting TS/TS2...
[TsPacket] Score : 188:20, 192:1 out of 20
[TsPacket] Probably TS1 (188)...
[ADM_tsAccess] 23:04:32-899 Creating audio track, pid=1528, muxing =0
[TsDemuxerer] Reading index
[TsDemuxer] *****Found 0 I*****, 230 B, 36 P
[TsH264] *****Found 11 IDR*****
[TsH264] Remapping IDR to I and I TO P
[psDemux] Fps not handled for DTS increment
[tsDemux] Found 277 video frames


But when saving a selection, it writes something about non-closed-gop:

Code Select Expand

[HandleAction] 23:13:57-789 ************ SAVE_VIDEO **************
...
[getNonClosedGopDelay] 23:15:34-613 frame 30 is early
[getNonClosedGopDelay] 23:15:34-614 frame 31 is early
[getNonClosedGopDelay] 23:15:34-616 Found maximum non closed gop delay = 140000


Question 1: am I right, that cutting or merging an open-gop video with avidemux will cause broken frames?

Question 2: How can I verify, if my video is open-gop?

Question 3: Is it possible, that the TV station which encoded the h.264 stream wrote wrong frametype values, i.e. they flagged it as IDR, but instead it's only an open-gop I-frame?

Thanks for your support,
Martin

PS: Sample source video (10 MB mpeg-ts): Download link

My system:
- Win. 10 64 bit,
- avidemux latest beta 2.7.0 (180222)  (same with v2.6)

[eumagga0x2a]

1) No, it should not. Avidemux takes precautions not to drop early B-frames.

2) If I am not mistaken, this doesn't matter at least in Avidemux because Avidemux marks only real IDR frames as I-frames. Only if there are really no IDR frames after the first one in the stream or they are very rare, you get problems.

3) Shit happens, but not very often.

[odin]

>> Question 1: am I right, that cutting or merging an open-gop video with avidemux will cause broken frames?
> 1) No, it should not. Avidemux takes precautions not to drop early B-frames.

I guess, avidemux has implemented Open-GOP support only for 30%:

• Select range + Save: OK
  (early B-frames are maybe hidden with negative PTS)
• Select range + Delete: BROKEN
  (see below)
• Merge Open-GOP files: BROKEN

According to my analysis, avidemux outputs a file like #2 below for a delete operation (for more details, see 2 posts below).
Probably the only "correct" solution is to output some frames with display-duration=0 in order to keep references valid, like #3 below (when playing with normal speed):


Gfx/Spreadsheet link

And if the video contains also forward references, it's getting one level more complicated:



PS: In case it's possible to implement this feature with "duration=0" frames, then this would have the additional very big advantage, that arbitrary cutting even on non-I-frames would be possible in copy mode (as all the necessary ref. frames (incl. the I frame) could be saved into the output with duration=0).

[eumagga0x2a]

Have you verified this with a sample? If yes, could you please share the sample?

[odin]

Have you verified this with a sample? If yes, could you please share the sample?

I figured out what's happening. It's a little bit similar to the previous "broken output 2":

a) Instead of deleting frames #5+6 from the output file, avidemux deleted #4+5 (during the "Save" operation) (probably to keep back reference 8-->6 intact).
b) But frame #6 is broken now due to broken back reference.
c) Timestamp from #6 is shifted to #4 (i.e. frame #4 is replaced by #6).
d) Additionally, B frame #2 is "missing" (seems to be a decoding error, maybe due to a broken forward reference to the next missing I-frame #4? ffprobe outputs the error: co located POCs unavailable)

In real life, it looks like this (in total 6 damaged/missing frames before the deleted section):
(click for full size)

Several rows/columns were hidden in the img. For all details see here.

Sample input file:   https://drive.google.com/uc?export=download&id=1fNi3g2oA6lNilyDt689Kpoc_vxToNNKYFg
Sample output file: https://drive.google.com/uc?export=download&id=1r2-uUuSwxs3dsW1Wp1s-Xs8X7wEmA4Ymzw
Avidemux adm log: see attachment below

[eumagga0x2a]

Thank you for your extensive research, I'll try to find time to look at the sample tonight. Avidemux does not trace references, it just delays PTS enough to ensure PTS-DTS offset is sufficient for early B-frames to be preserved. This led to ever growing PTS delay with each generation of saving a video in copy mode and was the reason why I modified the logic here, adding extra delay only if the already existing offset is smaller than one required to accommodate the longest sequence of early B-frames in the video. I must check first whether this change is related to the problem you see (if the 2.7.0 release behaves exactly the same, my patch is not at fault).

[odin]

I don't know if the output from 2.7 is really exactly like 2.6, but the output was always somehow broken after a delete / merge in copy mode.

I made some additional tests to work around the delete problem, by selecting + saving the beginning of the video (first part), then selecting + saving the end of the video (2nd part), and finally merging 1+2 together (everything via mpeg-ts files). From a logical point of view, the result should be identical to the "delete" example from above.

But in this case, the output is even more damaged than before, as also frames in the 2nd part are broken (in total 11 broken/missing frames):


When looking at the intermediate files (save_1.ts , save_2.ts), they seem to be (more or less) ok.
But the end of "save_1.ts" is somehow  damaged: avidemux does not show the final I frame, and ffprobe shows some error + DTS timestamp=N/A for last 2 frames (but the I frame is appearing  there):

Code Select Expand

#ffprobe -select_streams v -show_frames -of csv -show_entries frame=key_frame,pkt_pts_time,pkt_dts_time,best_effort_timestamp_time,pkt_pos,pkt_size,pict_type,coded_picture_number save_1.ts
Input #0, mpegts, from 'save_1.ts':
  Duration: 00:00:02.03, start: 0.004000, bitrate: 8513 kb/s
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k tbn, 100 tbc
    Stream #0:1[0x101](deu): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s
    Stream #0:2[0x102](mis): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s
    Stream #0:3[0x103](deu): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 448 kb/s
frame,1,0.777833,0.757833,0.777833,564,74839,I,4
frame,0,0.857833,0.777833,0.857833,214696,709,B,11
frame,0,0.797833,0.797833,0.797833,274104,768,B,12
frame,0,0.817833,0.817833,0.817833,278804,876,B,13
...
frame,0,1.797833,1.917833,1.917833,1619432,35943,P,61
frame,0,1.997833,1.937833,1.937833,1953132,2070,B,68
frame,0,1.957833,1.957833,1.957833,2001636,2741,B,69
[NULL @ 000000000462f3a0] missing picture in access unit with size 6
[h264 @ 0000000004b92020] no frame!
frame,0,1.917833,N/A,1.917833,1818900,41617,P,67
frame,1,1.977833,N/A,1.977833,2005020,138796,I,70

[eumagga0x2a]

If you conducted the merge test with the release, this is expected as the release is unable to decode last frames of a video (fixed in the nightlies).

[odin]

If you conducted the merge test with the release, this is expected as the release is unable to decode last frames of a video (fixed in the nightlies).

I used version 2.7.0  180222, but also the version from today behaves the same:
http://www.avidemux.org/nightly/win64/avidemux_2.7.0 r180304_win64.exe

[fish]

Wow, thank you for looking into this in detail. I have noticed this behaviour when editing specifically DVB-T HD files for some time. If it is the same problem, it results in frequent but inconsistent GOP freezes and extra frames being removed, after removing sections of video in copy mode. One weird thing is that I-Frames in these type of files appear to be consistently on the third frame after a scene change, rather than the first frame after a scene change. Is there any reason why a video encoder would do this or are they being decoded wrongly. Additionally, if the cut really is made at the third frame after a scene change (the indicated I-Frame), if all video after that I-Frame is removed, we should expect 2/3 frames after the final scene change to be present in the saved file but they never are. Thanks again for trying to tackle this issue, I don't have the knowledge or the tools you have, but I know something is not right, specifically when editing this type of file. 

[kwsk]

Here is same problem
After cut h264 ts file, ffmpeg encode with error

Code Select Expand


[NULL @ 0x3c1aee0] missing picture in access unit with size 14
[h264 @ 0x3c9a2e0] no frame!

My ipod nano 7th gen failed to play file which cut with avidemux 2.7.1 (It skipped to the end)
(Original file was no problem)

[eumagga0x2a]

Please perform re-encoding in Avidemux (and use the latest nightly whenever possible), not in ffmpeg. Avidemux has a workaround not to get stuck at the cut boundary.

The best possible solution in reach would be to disallow copy mode in such video streams.

[kwsk]

It was unable to use Avidemux muxer becuase I need to use audio track not encoded
I avoided problem with using container convert

ts (original file) -> mkv( Avidemux cut ) -> mp4 (ffmpeg encode with resize)
then, I can play file with ipod nano

Thank you

[eumagga0x2a]

It was unable to use Avidemux muxer becuase I need to use audio track not encoded

Which audio codec was used in the original stream then?

I avoided problem with using container convert

ts (original file) -> mkv( Avidemux cut ) -> mp4 (ffmpeg encode with resize)

You stated above that ffmpeg failed after cutting the stream with Avidemux. Have I misunderstood you?