PTS of first frame

[CryGuy]

Hello,

I have a question only:
I've got an MP4 file with video=H.264 and audio=mp2. Opened in ADM, it shows the timestamp of the first frame as 0.082 s. I don't understand why. As I've already utilized the libav library myself before, I've dumped the timestamps of the file:

Min. video PTS = 0.160 s (4000/25000), min. audio PTS = 0.181 (8688/48000)
Min. video DTS = 0.080 s, min. audio DTS = 0.181

The start time of both streams (as defined in AVStream.start_time) is AV_NOPTS_VALUE. I'm curious how that 0.082 s is computed. I'm unable to derive it from the numbers I have. I'd be glad about some insight how the time of the first frame is computed. (acutally I've got a different issue, but first, I'd like to understand this)

Thank you!

[eumagga0x2a]

If a video contains B-frames, the display of the first keyframe must be delayed as Avidemux uses uint64_t to store timestamps. A B-frame depends on other frames both earlier and later in the presentation order, so everything must be delayed until these reference frames have been decoded.

FFmpeg uses int64_t for timestamps, so it can let a video with B-frames start at zero by making decode timestamps negative.

As a rule, the first frame delay is (nbRefFrames-1)*timeIncrement.

[CryGuy]

....Ok, my fault. It seems that the start_time value isn't set before a frame has been read from the stream (by calling av_read_frame).
Though, I'm afraid, I don't understand at all what you're saying. The first frame is a key frame (I frame). It has the smallest PTS (and DTS), so why do the following frames matter at all? Why not just display 0.16 s as the time for the first frame? Sorry for asking dumbly, but I don't get it.

[eumagga0x2a]

The first frame is a key frame (I frame). It has the smallest PTS (and DTS), so why do the following frames matter at all? Why not just display 0.16 s as the time for the first frame?

Why should the delay be greater than necessary? It is pointless to have a non-zero first frame DTS. If you eliminate this offset, ffmpeg and Avidemux are pretty close. Following frames matter, because the alternative to delaying the first frame would be a hole after the first frame.

Imagine, we have a stream I(0)-P(1)-B(2)-B(3)-B(4)-P(5)-B(6)-B(7)-(I...). The display order is 0-2-3-4-1-6-7-5. We can't display 2 before we have decoded 1, so we delay everything by one time increment (max ref frames = 2).

[CryGuy]

First, thanks for your reply!

Why should the delay be greater than necessary?

It's not how I want it to be because the value is already there.

For me, only two things would make sense as to which timestamp is displayed:
- As stored in the source video. This means the first frame is at 0.16 s.
- All timestamps of all frames are shifted by -0.16 so that the first frame is at zero.

In no case it makes sense for me as a user that timestamps are shifted so that the first frame is at 0.082.

It is pointless to have a non-zero first frame DTS.

For me, the DTS doesn't matter regarding this question and what is shown to the user. I've only mentioned DTS FYI if it might be relevant for explaining me how the 0.082 s have been computed.

Following frames matter, because the alternative to delaying the first frame would be a hole after the first frame.

Isn't this a question of the later encoding process? When I open a video, I expect to see what's in the file (of course, frames ordered by PTS). For frame 1, I expect to see the PTS of frame 1 in the status bar, for frame 2 the PTS of frame 2, and so on, optionally shifted to zero, or to the corresponding alignment position if multiple inputs have been appended.

We can't display 2 before we have decoded 1, so we delay everything by one time increment (max ref frames = 2).

The decoding order is an internal issue and not relevant for the question. I'd just like to see the timestamp of the displayed frame. For each frame this is indpendent from any other frame because it can simply be read from the file and doesn't need to be manipulated at the stage of displaying it.

[CryGuy]

In short, the displayed timestamp should have a meaning to the user. For example, it could be the PTS from the video file, or the distance from the start of the video. These would make sense. The value currently displayed makes no sense to me.
I hope I was clearer now. 
Thanks anyway.

[Jan Gruuthuse]

>8 In short, the displayed timestamp should have a meaning to the user. >8

What is displayed is the technically information from the video, no more no less

Explanation by a layman:
- avidemux is not a frame based editor but a time based editor, due to the newer codecs.
- There is partially video information (past) before the 1st viewable reconstructed frame, it would also need partially video information after the the 1st viewable reconstructed frame (future).
- Why in heaven would you need to complicate matters more to visibly please a user by showing another time reference instead of the used time reference within the video.
After all the strength of avidemux is editing without the need of re encoding: ([Copy] Mode.
Please remember:

Avidemux is a simple tool for simple video processing tasks. The keyword here is simple: it does not offer tools like a timeline, multitrack editing, you cannot freely move or splice audio and video clips from various sources. However, Avidemux allows you to do elementary things in a very straightforward way.

source: Avidemux Wiki

[CryGuy]

Hi Jan, thanks for replying.

What is displayed is the technically information from the video, no more no less

Sorry, but just that is not the case. If it was from the video, it would display 0.16. Instead, it shows an artifical computed value (0.082) which is not from the video, and it's still not clear which meaning it should have.

avidemux is not a frame based editor but a time based editor, due to the newer codecs.

But it displays frames. And these frames have a timestamp. I only expect it to display the corresponding timestamp.
EDIT: Am I not jumping from frame to frame in the source video when pressing the left/right arrow keys? I think I do. I'm not jumping a fixed timespan forth/back.

There is partially video information (past) before the 1st viewable reconstructed frame, it would also need partially video information after the the 1st viewable reconstructed frame (future).

Still every frame has one timestamp. It should just be displayed.

Why in heaven would you need to complicate matters more to visibly please a user by showing another time reference instead of the used time reference within the video.

What is so complicated in just displaying the timestamp of the frame? The value is 0.16, so display 0.16. That's most straightforward. Thoughts about decoding order, delays etc posted before make it unnecessarily complicated. But these were not my thoughts.

Avidemux is a simple tool...

That's absolutely ok.


Maybe there is a reason for not simply showing the information from the opened video, but yet I don't see an explanation for it.

[CryGuy]

Difference. Why?

[Jan Gruuthuse]

But it displays frames. And these frames have a timestamp. I only expect it to display the corresponding timestamp.
EDIT: Am I not jumping from frame to frame in the source video when pressing the left/right arrow keys? I think I do. I'm not jumping a fixed timespan forth/back.

The only fully decodable picture would be the intraframe. Left Right keyboard moves you trough pseudo frames between the intraframe. The concept of Frames Per Second is virtual in the newer codecs, as the frames are not really there due to video compression.

DTS: 0.08 or 0.082, perhaps that is what is showing in 00:00:00.082 ?

Keep in mind I'm (basic) user, not a coder.
is this what you're looking for:
Stack Overflow: Understanding PTS and DTS in video frames

[eumagga0x2a]

If it was from the video, it would display 0.16. Instead, it shows an artifical computed value (0.082) which is not from the video, and it's still not clear which meaning it should have.

The PTS values stored in the container are respected, but the offset is adjusted so that the first DTS is zero.

For me, only two things would make sense as to which timestamp is displayed:
- As stored in the source video. This means the first frame is at 0.16 s.
- All timestamps of all frames are shifted by -0.16 so that the first frame is at zero.

The first one would be detrimental to video append functionality (an excessive gap between the last frame of the preceding video and the start of the next one), the second outright incompatible with Avidemux design which doesn't allow negative timestamps (DTS would need to become negative this way).

I tried to play with making every video start at zero (grep the source for ADM_ZERO_OFFSET), the attempt turned out to be not viable as it broke the video append functionality.

[CryGuy]

The only fully decodable picture would be the intraframe. Left Right keyboard moves you trough pseudo frames between the intraframe. The concept of Frames Per Second is virtual in the newer codecs, as the frames are not really there due to video compression.

Sorry, but I don't think there are pseudo frames. I know there is not (necessarily) a fixed frame rate anymore. That's why there are timestamps per frame. If you navigate through an opened video,you can only navigate to positions where there is a frame in the source video:



You can navigate only to the arrow positions, not anywhere between. If that's not true, then I misunderstood everything and I have to rethink from start.

DTS: 0.08 or 0.082, perhaps that is what is showing in 00:00:00.082 ?

No, the value is exactly 0.08. I limitted display to two decimal places as the third place is always 0. If you look in the "DTS (raw)" column, the value is 2000. As the timescale for the stream is 25,000, the value is computed by 2000/25000 = 0.08 (exact).

(sorry for not appending the pictures but attachments didn't appear in the past)

[CryGuy]

The PTS values stored in the container are respected, but the offset is adjusted so that the first DTS is zero.

I didn't say they are not respected. Jan said "What is displayed is the technically information from the video, no more no less ". For me "no more no less" means that the unmodified value from the video is shown. I only said that this is not true.

For me, only two things would make sense as to which timestamp is displayed:
- As stored in the source video. This means the first frame is at 0.16 s.
- All timestamps of all frames are shifted by -0.16 so that the first frame is at zero.

The first one would be detrimental to video append functionality (an excessive gap between the last frame of the preceding video and the start of the next one), the second outright incompatible with Avidemux design which doesn't allow negative timestamps (DTS would need to become negative this way).

Regarding the first option, I've already taken this into account as I said that the ts of appended videos would have to be shifted so that there is no gap. However,
The second wouldn't be a problem as you do not have to shift to negative valus. Again, I'm only referring to what is displayed to the user!!


Anway... I don't understand what is displayed now. The 0.082 s doesn't make sense at all for me as a user. I don't say it's wrong or anything is not working, but the value is meaningless. However, I take it as it is now.

Sorry if I wasn't able to make myself clearer. Thanks guys for the discussion! I'm out.

[eumagga0x2a]

You could provide admlog.txt resulting from loading the mp4 in question in Avidemux to make the choice of first frame PTS clearer.

[CryGuy]

You could provide admlog.txt resulting from loading the mp4 in question in Avidemux to make the choice of first frame PTS clearer.

I have no doubt you can explain it to me. And I see it would answer the question I'vee initially asked.

Which value I expect to see instead is a different thing, but I don't expect ADM to be changed. (even if it would make sense IMO)

Thanks again.