Missing frames and corruption at cut/splice points in copy mode.

[radorn]

I'm editting DVB recordings with video in MPEG2 and AVC formats.
I'm careful to leave an I-frame or P-frame at the tail of the GOP in front of the cut and an I-frame at the beginning of the GOP behind it, But the the resulting file frequently has momentary video corruption at the cut point in the case of AVC streams and lost frames when it is MPEG2.
It is my understanding this shouldn't happen and something is going wrong.

I just started noticing this recently and I'm not sure how long it has been going on.

Also, there seems to be some trouble with the language tag for audio tracks whe opening files. With .ts stream files it seems to fail sometimes, but with matroska files, even those exported by Avidemux itself, it seems to always fail and just gives me "unknown" in the language field.

[eumagga0x2a]

I'm careful to leave an I-frame or P-frame at the tail of the GOP in front of the cut

Currently, this is not necessary and leaving a non-IDR keyframe as the last picture before cut (i.e. placing the A marker of a deletion at a frame directly or closely following a non-IDR keyframe) is outright harmful as in case of open GOP H.264 streams, such a keyframe may be not decodable standalone.

and an I-frame at the beginning of the GOP behind it

Yes, this is mandatory. Just placing A and B markers of a deletion on keyframes does the best possible job.

But the the resulting file frequently has momentary video corruption at the cut point in the case of AVC streams

Depending on stream structure, this is unavoidable in open GOP type H.264 and HEVC streams. Those early B-frames we cannot drop because later frames use them for reference won't be fully decodable as a rule. By dropping early B-frames not used for reference we reduce the number of damaged frames to the minimum possible.

and lost frames when it is MPEG2

In case of MPEG-2, all early B-frames at cuts are dropped as they are never used for reference, yes. One cannot expect the picture right before the A marker to be kept when in copy mode and the stream contains B-frames.

Also, there seems to be some trouble with the language tag for audio tracks whe opening files. With .ts stream files it seems to fail sometimes, but with matroska files, even those exported by Avidemux itself, it seems to always fail and just gives me "unknown" in the language field.

Just tested with a MKV video exported by Avidemux with audio track language set in the "Select audio track" dialog and then the result opened in Avidemux – works fine for me. Please provide more details. There are plenty of MKV and other video files with audio track language left unset, so doesn't expect this metadata to be present everywhere. Of course, if you set audio language in Avidemux prior to exporting a video, the resulting MKV should retain this information.

As always, everything applies to the latest nightly available. If you are not using it, please update.

[radorn]

Currently, this is not necessary and leaving a non-IDR keyframe as the last picture before cut (i.e. placing the A marker of a deletion at a frame directly or closely following a non-IDR keyframe) is outright harmful as in case of open GOP H.264 streams, such a keyframe may be not decodable standalone.

But if I leave a B frame right before [A] won't it be lost in the resulting stream?
I do that so I can have control over what I'm losing and what I'm keeping, choosing the most advantageous cutpoint possible.

Just placing A and B markers of a deletion on keyframes does the best possible job.

I used to do that, but then I noticed I was losing frames before the cut, and I attributed this to the trailing B-frames being left without their second reference, so I began looking after P's and I's. Made sense to me, and I thought it was working as I intended, but, maybe it wasn't(?)

I did a new test with an MPEG2 stream right before posting this and I finally understand what's going on:

-B-B-P-B-B-P[A-B]I-B-B-P-B-B-P-  I have this sequence and make that cut noted by A-B
-B-B-P-_-_-P-I-B-B-P-B-B-P-  In the output file, the B frames before the last P before the cut are gone.

I would understand the B's being dropped if I left them hanging by themselves, but shouldn't a P close the GOP and preserve those B's?

Just tested with a MKV video exported by Avidemux with audio track language set in the "Select audio track" dialog and then the result opened in Avidemux – works fine for me. Please provide more details. There are plenty of MKV and other video files with audio track language left unset, so doesn't expect this metadata to be present everywhere. Of course, if you set audio language in Avidemux prior to exporting a video, the resulting MKV should retain this information.

No, all the mkvs I'm talking about here are generated by me, either with avidemux or mkvtoolnix, and I know they have language metadata as it shows in SMPlayer and MediaInfo, and also MKVtoolkinx.
But Avidemux fails to detect it very often. Sometimes it will detect it, with the same file with which it failed just the previous attempt. I don't know... it's very strange. I'm trying to find a 100% reproducible scenario but I feel like I'm being trolled by these programs now...

----------

I don't understand some of what you are saying about AVC and HEVC. Particularly I just don't get how these non-IDR keyframes work at all. All the time I'm editting streams where I get these non-IDR keyframe warnings when trying to cut out the advertisements. Previously, I found out that if I went a few GOPs back, until it stops complaining about the cut and, then, leave that single I-frame and delete everything until the I-frame I need (where the actua program comes back), very often it seems to work fine and I just have to suffer a random frame from the ad-interlude in the middle of that cut in the output file. Sometimes there are multiple non-IDRs I need to preserve (2, 3... 4 once), in which case the likelyhood that the output file will work acceptably gets increasingly small. I suppose that some later frames somehow depend on this non-IDR keyframe thing to properly decode, even if they are I-frames... BUT if, instead of cutting out the ads, I just export the desired content sections to individual files, these decode just fine, like they don't need the non-IDR keyframe at all. But then If I join these individual files into a single stream (be it with Avidemux or MKVtoolnix), then the result is the same as if was editing the original TS stream in Avidemux and ignoring the non-IDR warnings.
It seems that those non-IDRs can be safely ignored if they aren't present, but if they are and they don't like the situation, they screw everyone else. Or is it when they are missing? What's the deal? I'm scratching my head here! Those non-IDR party poopers!

[eumagga0x2a]

I do that so I can have control over what I'm losing and what I'm keeping

The truth is that you can't when in copy mode.

I'll try to answer all the questions to open GOP, non-IDR keyframes and dropped frames a few days later. Meanwhile, please ensure you use the latest nightly.

all the mkvs I'm talking about here are generated by me, either with avidemux or mkvtoolnix, and I know they have language metadata as it shows in SMPlayer and MediaInfo, and also MKVtoolkinx.
But Avidemux fails to detect it very often. Sometimes it will detect it, with the same file with which it failed just the previous attempt.

If you can reproduce this with the latest nightly (please ensure that no other Avidemux builds are installed for the duration of testing), please provide such an mkv file as a sample.

[radorn]

Thanks, I would really like to understand how to predict what the output will be, because what I'm seeing sure is counter-intuitive. Particularly the MPEG2 frame-dropping.

[eumagga0x2a]

Independently from codec, the picture marked as the start of deletion is not included into the output.

-B-B-P-B-B-P[A-B]I-B-B-P-B-B-P-  I have this sequence and make that cut noted by A-B

If the above means that you put the A marker to a P-frame, this P-frame and all following frames in the stream order (i.e. the B-frames preceding the P-frame in display order) up to the keyframe at the B marker are dropped.

[eumagga0x2a]

I don't understand some of what you are saying about AVC and HEVC. Particularly I just don't get how these non-IDR keyframes work at all.

"Non-IDR keyframe" means that the decoder can start decoding from that (regular, non-IDR) frame and there won't be incomplete pictures when the decoder is ready to output them to display. It absolutely doesn't imply that you can concatenate non-contiguous parts of a stream at such frames and get anything usable.

In general, open GOP streams as used by modern digital broadcasts cannot be edited in copy mode by deletions, cuts or by pasting without becoming fundamentally invalid (broken). It would be necessary to rewrite at least all slice headers after a cut up to the next IDR (which may never come) to fix the stream. Avidemux cannot do this, implementing such a logic would be at a similar level of complexity as writing a H.264 encoder and it would also greatly impact the performance in copy mode.

The only copy mode operation which preserves open GOP stream integrity is truncation.

However, a lot of video decoders cope well with such invalid streams, dealing gracefully with discontinuities of frame numbers and picture order count (POC) in slice headers. Just libavcodec in FFmpeg trusts obviously invalid POCs and stubbornly waits for them to grow again, forcing Avidemux to warn users about unsuitable cut points as libavcodec is tremendously popular (we use it at the heart of the application too!).

All the time I'm editting streams where I get these non-IDR keyframe warnings when trying to cut out the advertisements. Previously, I found out that if I went a few GOPs back, until it stops complaining about the cut and, then, leave that single I-frame and delete everything until the I-frame I need (where the actua program comes back), very often it seems to work fine and I just have to suffer a random frame from the ad-interlude in the middle of that cut in the output file. Sometimes there are multiple non-IDRs I need to preserve (2, 3... 4 once), in which case the likelyhood that the output file will work acceptably gets increasingly small. I suppose that some later frames somehow depend on this non-IDR keyframe thing to properly decode, even if they are I-frames...

The order of pictures, derived from POC, is messed up by the cut. Libavcodec insists on POC of later pictures to be > POC of earlier ones even if the discontinuity is big enough to justify resetting the decoder. Your options are either to strictly follow Avidemux warnings or to ignore them and refrain from using libavcodec-based players like VLC to play the resulting video.

The term "I-frames" doesn't describe the complex reality well, unfortunately. In open GOP streams a keyframe is often spread over multiple frames, with portions of the full picture in each. The first one in such a "series" is marked as keyframe, but it cannot be decoded standalone. This solution allows to deliver much better visual quality than in the case of a purely intra-coded frame because the bandwidth is strictly limited.

Proper IDRs (purely intra-coded frames where no future frame depend on anything preceding the IDR in stream order) are inserted very rarely in digital broadcasts, and only IDRs allow truly clean cuts.

BUT if, instead of cutting out the ads, I just export the desired content sections to individual files, these decode just fine, like they don't need the non-IDR keyframe at all. But then If I join these individual files into a single stream (be it with Avidemux or MKVtoolnix), then the result is the same as if was editing the original TS stream in Avidemux and ignoring the non-IDR warnings.
It seems that those non-IDRs can be safely ignored if they aren't present, but if they are and they don't like the situation, they screw everyone else. Or is it when they are missing? What's the deal? I'm scratching my head here!

I hope I have now answered this question.

Those non-IDR party poopers!

Yes, indeed :-D

[eumagga0x2a]

By the way, if you really see some bug with audio track language in MKV files, this bug will make it happily into the next release if you don't provide samples NOW.

[radorn]

Regarding the MPEG2 cutting problem.

If the above means that you put the A marker to a P-frame

No, that's not what I meant: I conceptualize the markers as "cut points set between the current frame and the one before it", or in simpler terms "cut to the left of this". Therefore I spoke about placing the A and B "cuts" in the "space between frames" instead of "markers to a frame".

Let me try again, using your "setting markers to a frame" terminology instead:

Code Select Expand

So we have this senquence where the '[...]' represents an indeterminate ammount of frames between those seen here.
I set the A-marker to a B-frame right after a P-frame and the B-maker to an I-frame. Like this:

ORIGINAL    ...-I-B-B-P-B-B-P-B-B-I-[...]-B-B-I-B-B-P-B-B-P-B-B-I-...
MARKERS                      [A---------------B]
IN EDITOR   ...-I-B-B-P-B-B-P-----------------I-B-B-P-B-B-P-B-B-I-... (as expected)
SAVED FILE  ...-I-B-B-P-----P-----------------I-B-B-P-B-B-P-B-B-I-...
                       [WHY] are these missing?

Why cull those B-frames between the two last P-frames before the A-marker? Is that really how it is supposed to be?

--------------

Thanks for the explanation about non-IDRs. I don't understand everything but I get the idea. I had to look up POC and find lots of information about the complex internals of the streams... which will take a lot of reading to understand and won't help me with copy-mode editting anyway.

I almost never use VLC. Never liked that player very much. When I tried it for the first time many years ago, I was quite heavily into MPC (before HC, even) and installing single filters and so on. I was pretty satisfied with that situation. It bugged me to have a monolithic player that contained *everything* and you couldn't have anything that wasn't included. And I didn't like the interface either.
I have it installed, for when I find some rare file or format that doesn't work in other players, but other than that it goes mostly ignored.
My main player now is SMplayer. Not that I love it, but given the current situation, and my diminished expectations, it works for me, I guess. Can't find much fault with it.

As for the language problem I've experienced, I still can't seem to reproduce it reliably again. It's really strange. But I'm finding that maybe Avidemux doesn't like MKVToolNix's files.

[eumagga0x2a]

Why cull those B-frames between the two last P-frames before the A-marker? Is that really how it is supposed to be?

Yes, this is how it is supposed to be due to presumably open GOP type of this MPEG-2 stream. Open the scripting console, run

Code Select Expand

ed = Editor()
for frame in range(50): ed.printFrameInfo(frame)
to show flags, timing and size of the first 50 frames and examine admlog.txt. I would expect that you'll find I-frames starting with the second one followed by B-frames directly whose presentation timestamps precede the PTS of the I-frame (early B-frames). These B-frames reference pictures from the previous GOP and are dropped by Avidemux after a cut, but the delta between DTS of the I-frame and its PTS mandates that the last frames before the cut "free the stage" for the I-frame if their decode timestamps otherwise collide with the I-frame. The solitary P-frame in your example precedes the B-frames in the stream order, so it is kept, building a gap to the previous frame (in display order).

If you examine admlog.txt, you would notice debug messages from getCompressedPicture (the most important function for the copy mode) "Frame xxxx, have to switch segment, DTS limit reached" (replace "xxxx" with matching frame number starting with 1). The best strategy to avoid this is to place the A marker for the start of deletion to a P-frame. This will result in this P-frame and preceding (in display order!) B-frames being dropped, but will avoid creating a gap.

My main player now is SMplayer.

This is just another FFmpeg (libavcodec + libavformat + libavutil + libavfilter + libswscale + ...) client like VLC and us.

As for the language problem I've experienced, I still can't seem to reproduce it reliably again. It's really strange. But I'm finding that maybe Avidemux doesn't like MKVToolNix's files.

Can you please confirm that you see this with the latest nightly? If yes, this would mean that we use uninitialized memory somewhere. I would really love to avoid the hassle of running a debug Avidemux build in Valgrind just to discover that you suffer from some old long fixed bug.

[radorn]

Can you please confirm that you see this with the latest nightly? If yes, this would mean that we use uninitialized memory somewhere. I would really love to avoid the hassle of running a debug Avidemux build in Valgrind just to discover that you suffer from some old long fixed bug.

I'm currently using "Avidemux 2.8.0 (211120_fflibs 4.4.1-VC++)" and it keeps happening.
Usually I'm dealing with two languages, Spanish and English. Interestingly when a file fails, English seems more prone to be ignored and labeled as "Unknown" in Avidemux than in the case of Spanish. I tried reversing track order to see if that was the problem, but that didn't seem to make English appear and Spanish disappear. Then I made a file in Avidemux with Spanish and Aragonese (pretty much a random pick). The Avidemux export imports back into Avidemux with proper language tags, and so does the MKVtoolnix re-mux. After that, I tried again, the same video, but tagging both audios as English: This time, the Avidemux export imported with proper tags back into Avidemux, but the MKVtoolnix remux gave me two "Unknowns", even though MediaInfo, MKVtoolnix itself and SMPlayer identify them properly as English.

Also I just found that it doesn't like TS files from some channels either. I'm uploading a sample I cut out in a hex editor. easyupload(.)io/3dnmbd

This is just another FFmpeg (libavcodec + libavformat + libavutil + libavfilter + libswscale + ...) client like VLC and us.

Oh. I thought mplayer was an independent engine. Nevermind that. Since we are talking about it, do you know any alternative then?

----

I'm having to get familiarized with many of the things you mention in your explanation about the disappearing b-frames. But, if I'm getting something right, you are saying that if I want to preserve up to a certain P-frame, the best thing I can try is deleting the next one, but leave the intervening b-frames intact?

Code Select Expand

So, if I have this
...-I-B-B-P-B-B-P-B-B-I-[...]-B-B-I-B-B-P-B-B-P-B-B-I-...
and I want to get this
...-I-B-B-P-B-B-P-----------------I-B-B-P-B-B-P-B-B-I-...
then I must cut it like this?
...-I-B-B-P-B-B-P-B-B-I-[...]-B-B-I-B-B-P-B-B-P-B-B-I-...
                     [A-----------B] (fixed)
and count on the unwanted B-frames being dropped during save/export? because if I remove them, then the previous B-frames get the axe too?


[eumagga0x2a]

This time, the Avidemux export imported with proper tags back into Avidemux, but the MKVtoolnix remux gave me two "Unknowns"

This would constitute an issue in mkvtoolnix, not in Avidemux.

But, if I'm getting something right, you are saying that if I want to preserve up to a certain P-frame, the best thing I can try is deleting the next one, but leave the intervening b-frames intact?

These B-frames are NOT intervening, they follow the P-frame in the stream, please don't forget that the order of frames which primarily matters for copy mode is how they are ordered in the stream, not how they are displayed. With B-frames present, display order != stream order. And yes, you should set the start of selection for deletion to the next P-frame after the one you want to keep.

So, if I have this

Code Select Expand

...-I-B-B-P-B-B-P-B-B-I-[...]-B-B-I-B-B-P-B-B-P-B-B-I-...and I want to get this

Code Select Expand

...-I-B-B-P-B-B-P-----------------I-B-B-P-B-B-P-B-B-I-...then I must cut it like this?

Code Select Expand

...-I-B-B-P-B-B-P-B-B-I-[...]-B-B-I-B-B-P-B-B-P-B-B-I-...
                    [A---------B]and count on the unwanted B-frames being dropped during save/export?

No, in this case you should set the A marker to the keyframe (I-frame) like

Code Select Expand

...-I-B-B-P-B-B-P-B-B-I-[...]-B-B-I-B-B-P-B-B-P-B-B-I-...
                     [A-----------B]

The two B-frames directly preceding the I-frame in display order belong already to the next GOP. They are sent after the I-frame in the stream.

(The selection end points A and B match the PTS of the frames they are set to exactly, no "cutting in-between". The latter is simply an incorrect concept, but the result is basically the same as in case of delete operation the start of a selection is deleted, the end kept = the start belongs to the selected range, the end does not.)

[radorn]

This would constitute an issue in mkvtoolnix, not in Avidemux.

Perhaps, and I wouldn't have mentioned it at all if it wasn't for the fact that the files MKVToolNix puts out show the correct language tags in MediaInfo, SMPlayer, and MKVToolNix. Only Avidemux finds fault with them... Which leads me to think that's where the problem lies.
And then there's the problem with the broadcast TS sample file I uploaded too, where, again, MKVToolNix, MediaInfo and SMPlayer get the languages correctly and Avidemux doesn't. Here it is again   easyupload . io/3dnmbd

No, in this case you should set the A marker to the keyframe (I-frame) like

Yeah, that's what I meant to say, somehow I goofed with the spaces after editing and re-editing a few times.
My goof even put the marker on an non-existent inter-frame space .

These B-frames are NOT intervening, they follow the P-frame in the stream, please don't forget that the order of frames which primarily matters for copy mode is how they are ordered in the stream, not how they are displayed. With B-frames present, display order != stream order. And yes, you should set the start of selection for deletion to the next P-frame after the one you want to keep.

Well, since, in the editor, I'm working on a timeline that represents the display order and not the internal stream order, it stands that these Bs are indeed intervening between the P I want to keep and the P I want to remove. That's what I was talking about, not the underlying logic of the stream. Unless there's some secret "internal mode" in Avidemux that allows me to work directly on that.

[eumagga0x2a]

I wouldn't have mentioned it at all if it wasn't for the fact that the files MKVToolNix puts out show the correct language tags in MediaInfo, SMPlayer, and MKVToolNix. Only Avidemux finds fault with them...

Please provide such a file as a sample, thanks.

And then there's the problem with the broadcast TS sample file I uploaded too, where, again, MKVToolNix, MediaInfo and SMPlayer get the languages correctly and Avidemux doesn't.

Oops, I simply forgot about the MPEG-TS sample, my bad. Avidemux labels in this case all audio tracks as "Unknown" because it fails to extract packets of the program specific information (PSI) stream, necessary to decode PMT (program map table) which contains stream descriptors which ultimaltely provide language information.

The problems start with Avidemux thinking (not sure whether justified or not) that the section we would like to decode doesn't fit into a single packet – a situation we don't support. Avidemux discards the packet, retrieves the next one and founds its syntax to be declared as private. The same continues until the end of the file is reached. Avidemux resorts to brute force reading then, being able to find the tracks, but of course without any metadata as it is stored in PMT.

I'll try to understand why libavformat succeeds in reading out the PMT in this sample while Avidemux fails.

I'm working on a timeline that represents the display order and not the internal stream order

Sure, if you put cut points to keyframes and let Avidemux figure out which compromises are necessary to deal with the particular situation, you don't need to care. But if you have some special wishes while still wanting to use the copy mode, you better understand the intricacies behind the stage.

[radorn]

Please provide such a file as a sample, thanks.

https://ufile.io/4afqiyqw