DVB TS h264 1080i streams incorrectly decoded as r601 when it should be r709

[radorn]

Not sure I'm posting this on the correct section of the forum, but it seems a good place to start, since I'm using Avidemux on Windows 7. Please move it if necessary.

So, I'm using avidemux to cut and reencode some DVB originated streams in TS files captured by my USB-enabled tuner, and I noticed that when I reencode (as opposed to copying) HD streams, which come encoded in h264, the colors are different than when I play the TS files directly.
After fiddling around with a bunch of other programs, I've come to the conclusion that somehow AviDemux incorrectly decodes those video streams as if they were rec601, while it should treat them as rec709.

The decoded video doesn't look wrong at first glance, but when you make the comparison, it becomes undeniable.

I don't seem to find any way to coherce Avidemux to force 709 decoding, so I'm guessing there isn't one readily available.

Any ideas?

[eumagga0x2a]

First of all, please verify that the issue you experience persists with the latest available nightly build (currently from Aug 23) and specify what filter chain and what encoder you use and its configuration (please provide a project script).

[radorn]

https:// mega.nz /folder/Dh00QQoD#qTBi98Q-A5keYyDLVJnMEQ
Here's a 2 GOP sample of the original video stream, and a zip with one of the frames as decoded by several programs. Two of them generated by both my current and your suggested latest version of avidemux, saved with the "save as image" function just right after loading the original video file, in copy mode, without applying any filters or anything else, in order to reflect just what avidemux decodes the video as.
The others are generated by MPC-HC, SMPlayer, VLC, and, last, VirtualDub2, which offers the possibility of manually telling the decoder to interpret the YUV values as either BT 601 or 709 (defaults to 709 for this file).

As you'll be able to see, tools like MediaInfo will also identify the h264 stream as BT.709. Since VirtualDub allows selecting the decoded pixel format, when select 709, the colors match every other program I tested. But if I select 601, then it matches Avidemux.
Therefore I conclude that Avidemux is decoding the colors as rec 601 when it seems it should be decoding as rec709.
It may so be that it's Avidemux that's right and every other program I tried is wrong. I don't know enough to definitelly say that's not the case, but chances seem to be on the side of everyone else. Just my honest assessment.

I don't see the point of providing any filter chain or encoder settings since it seems pretty clear that the problem here is with the decoding, before any filtering or encoding happens.

[eumagga0x2a]

I don't see the point of providing any filter chain or encoder settings since it seems pretty clear that the problem here is with the decoding

No, this is by no means clear. The filter chain and encoder is essential.

[eumagga0x2a]

By the way, the link, when manually assembled, redirects me to the Mega homepage (/start). It would be still helpful to get a sample (100 MiB should be enough) of the original, unaltered, unedited MPEG-TS stream.

[eumagga0x2a]

"Save as image" = encode the current picture using the MJPEG encoder in the bundled libavformat libavcodec, applying colorspace and color range conversions when necessary.

[radorn]

Sorry about the url. This one should work https:// mega.nz /folder/Dh00QQoD#qTBi98Q-A5keYyDLVJnMEQ

The file found there is a copy-mode excrept cut with Avidemux. It's just two GOPs. I can't afford to provide 100MB because I currently am on a very limited data plan (hence why I'm going through the trouble of recording stuff off the air instead of just downloading it or watching online). The results with that MKV and the original TS are the same in all the tested programs.

When I load that video on avidemux and the other programs, the view in the avidemux window already has different colors than all the others. The file generated by the "save as image" function matches the colors in the main avidemux window.
All this is done with the file freshly loaded into Avidemux. Copy mode. No encoder selected. No Filters selected. Nothing. I really don't understand what exactly you expect me to provide in terms of encoder and filter settings.

I'll give you a project script file, if you insist. I'll put it in that same folder along with the other files.

[eumagga0x2a]

I see, we don't have infrastructure to convey colorspace information from the decoder, just the color range. When saving the current picture as JPEG, we have to leave the colorspace field in AVFrame at its default value (AVCOL_SPC_UNSPECIFIED = 2). We also don't have any means to tell some encoders like x264 what color properties it should use.

[radorn]

What if the decoder itself did the conversion from 709 to 601.
Ideally you would be able to work with all those colorspaces and even let the user do the conversion through filters if necessary, but, if avidemux is fixed on working with bt601, can't you have the decoder do the conversion before delivering the frames to avidemux? That way at least the colors would be correct.

[eumagga0x2a]

I can't afford to provide 100MB because I currently am on a very limited data plan

Would it be possible for you to provide just ~20 MiB of the original stream? Two GOPs are too short for my purposes and it must be MPEG-TS or MP4, but not MKV.

Your sample has revealed an unrelated, very serious issue which needs to be understood and fixed on the spot. Many thanks in advance.

What if the decoder itself did the conversion from 709 to 601.

At the present time, Avidemux knows nothing about bt709 vs 601, in doesn't do 601, it simply doesn't care (it cares about the range and pixel format, that's it). There is nothing wrong with the decoder, in the future Avidemux might become aware of colorspace details.

[radorn]

How do I do that, just chop a chunk off with a hex editor?
What's the smallest size that would be usable to you?

If if helps, this is the full MediaInfo report on that .TS stream

General
ID                                       : 18 (0x12)
Complete name                            : D:\DVB\_dump\rm-2021-07-18 21-30-40.ts
Format                                   : MPEG-TS
File size                                : 4.00 GiB
Duration                                 : 2 h 1 min
Overall bit rate mode                    : Variable
Overall bit rate                         : 4 716 kb/s

Video
ID                                       : 301 (0x12D)
Menu ID                                  : 103 (0x67)
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.1
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Format settings, GOP                     : M=4, N=32
Codec ID                                 : 27
Duration                                 : 2 h 1 min
Bit rate                                 : 4 159 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 25.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Interlaced
Scan type, store method                  : Separated fields
Scan order                               : Top Field First
Bits/(Pixel*Frame)                       : 0.080
Stream size                              : 3.53 GiB (88%)
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio #1
ID                                       : 303 (0x12F)
Menu ID                                  : 103 (0x67)
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 2
Codec ID                                 : 3
Duration                                 : 2 h 1 min
Bit rate mode                            : Constant
Bit rate                                 : 192 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 kHz
Frame rate                               : 41.667 FPS (1152 SPF)
Compression mode                         : Lossy
Delay relative to video                  : -1 s 382 ms
Stream size                              : 167 MiB (4%)
Language                                 : Spanish

Audio #2
ID                                       : 304 (0x130)
Menu ID                                  : 103 (0x67)
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 2
Codec ID                                 : 3
Duration                                 : 2 h 1 min
Bit rate mode                            : Constant
Bit rate                                 : 128 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 kHz
Frame rate                               : 41.667 FPS (1152 SPF)
Compression mode                         : Lossy
Delay relative to video                  : -1 s 278 ms
Stream size                              : 111 MiB (3%)
Language                                 : v.o

Menu
ID                                       : 300 (0x12C)
Menu ID                                  : 103 (0x67)
Duration                                 : 2 h 1 min
List                                     : 301 (0x12D) (AVC) / 303 (0x12F) (MPEG Audio, Spanish) / 304 (0x130) (MPEG Audio, v.o) / 305 (0x131) (MPEG Audio) / 311 (0x137) (DVB Subtitle) / 314 (0x13A) ()
Language                                 :  / Spanish / v.o
Service name                             : Realmadrid TV HD
Service provider                         : Realmadrid
Service type                             : advanced codec HD digital television
Maximum bit rate                         : 216000000

[eumagga0x2a]

How do I do that, just chop a chunk off with a hex editor?

What is the equivalent of

Code Select Expand

head -c 20971520 /path/to/source.ts > /path/to/sample.ts
on Windows?

[eumagga0x2a]

By the way, the issue is that you can't save the entire MKV you provided as MP4 due to broken DTS for the first few frames.

[radorn]

Does it have to be 20 megs, can't it be less?

[eumagga0x2a]

Let's try with 10. If Avidemux on your system is capable to detect video in the sample and the duration includes at least 3 GOPs, it is enough.