AviDeMux only handles first 3h of a 16h TS file

[eumagga0x2a]

The funny thing is that timestamps wrapping around when an unsigned 32 bits integer is full like in this case is well handled in the demuxer code as long as it happens only once (i.e. after at most 0xFFFFFFFF MPEG ticks / 90000 Hz ≈ 47721,859 s ≈ 13 h 15 min). In theory, recodings up to 26 h 30 min duration should be fine. The problem is definitely related to wrap handling, but not that easy to spot.

By chance, do you have a real (not live) Linux system with write access to the location where the original huge TS file is stored to allow easy compiling of patched Avidemux versions?

By all means, please keep the original huge file to allow troubleshooting and verification in the future.

[beta-tester]

By all means, please keep the original huge file to allow troubleshooting and verification in the future.

sorry i saw your post too late. i deleted the huge file already to free diskspace - i can't restore tha file anymore.
i can recorde a new one from the same tv channel with the similar duration and try to reproduce the issue...

i will tell you tomorrow if the new recorded 16:05 hours TS file will show the same issue.
i am optimistic that i can reproduce that issue...
i think i remember that it is not the first time that i could not to handle that huge TS files in the past.

By chance, do you have a real (not live) Linux system with write access to the location where the original huge TS file is stored to allow easy compiling of patched Avidemux versions?

i have the possibility to run real linux (ubuntu 20.04 LTS amd64) on the PC.

[eumagga0x2a]

By all means, please keep the original huge file to allow troubleshooting and verification in the future.

sorry i saw your post too late. i deleted the huge file already to free diskspace - i can't restore tha file anymore.

That's a pity.

i can recorde a new one from the same tv channel with the similar duration and try to reproduce the issue...

i will tell you tomorrow if the new recorded 16:05 hours TS file will show the same issue.

If you have already started to record (DVB-C, I assume?), then please go on. Else I'll try it later on my own computer, with the benefit of direct access to a testcase if I manage to reproduce the problem.

i have the possibility to run real linux (ubuntu 20.04 LTS amd64) on the PC.

Great, if you happen to get hold of a sample suitable to reproduce the problem by tomorrow, I'll post patches here to add debug messages which I hope would help to analyse the issue and guide you through the trivial process of building Avidemux on Linux.

[beta-tester]

ok, recording 16:05 TS file finished, same issue with ADM and that TS file on windows (nightly 2.7.5 20200504) and on linux (2.7.4).

i copied the TS file to a virtual box machine where ubuntu 20.04 LTS is installed.
there i compiled avidemux 2.7.4 from source - so the build environment is prepared.
i also installed TeamViewer in case you need remote access to that machine.

if you want, i will send you the TeamViewer ID and the password to remote access the machine.
but i can't keep that PC with the virtual machine turned on permanently...

if i should do testing, please tell me from where i have to download the prepared nighly source code of ADM.

[eumagga0x2a]

Great, thanks! Here we go. I assume git is already installed. To build Avidemux from the current git, do

Code Select Expand

git clone https://github.com/mean00/avidemux2.git
cd avidemux2
git submodule update --init --recursive

If you have already all build deps, skip

Code Select Expand

bash createDebFromSourceUbuntu.bash --deps-only

We are not going to use any packaging, therefore the next step is

Code Select Expand

bash bootStrap.bash

If everything went well, copy "run_avidemux_template.sh" as "avidemux" (or any other name you like to use to launch Avidemux) to a directory in your $PATH and make it executable. You should be now able to run Avidemux by executing

Code Select Expand

avidemux

in the terminal. Once all that works, I'll attach patches here to produce diagnostic messages hopefully helpful during troubleshooting. I'd like to avoid pushing them to the repository.

Would you please provide the new *.idx2 file?

[eumagga0x2a]

Wrap handling is probably just the red herring. This was from the old admlog.txt:

Code Select Expand

[addReferenceVideo] 08:32:43-962 Original frame increment 00:00:00,020 = 20000 us
[addReferenceVideo] 08:32:44-036 min increment 00:00:00,020 = 20000 us for frame 1
[addReferenceVideo] 08:32:44-036 max increment 339:53:08,480 = 18446744025987712772 us for frame 2386055
[addReferenceVideo] 08:32:44-036 About 20000 microseconds per frame, 2894693 frames
[getVideoDuration] 08:32:44-036 Found maxPts =02:49:32,811, 3 frames from the end
[getVideoDuration] 08:32:44-037 Found maxDts =02:49:32,711, 2 frames from the end
[getVideoDuration] 08:32:44-037 Using PTS..
[getVideoDuration] 08:32:44-037 Using duration of 02:49:32,871
[addReferenceVideo] 08:32:44-037 The first frame DTS = 770 ms
[addReferenceVideo] 08:32:44-037 The first frame has a PTS > 0, adjusting to 850 ms
[updateStartTime] 08:32:44-037 Using pts2=00:00:00,850 to get dts, as this video does not start at zero
[ADM_verifyDts] 08:32:44-038 Verifying DTS....
[ADM_verifyDts] 08:32:44-038 Checking from 1 to 2894693

And now:

Code Select Expand

[addReferenceVideo] 10:24:26-490 Original frame increment 00:00:00,020 = 20000 us
[addReferenceVideo] 10:24:26-551 min increment 00:00:00,020 = 20000 us for frame 1
[addReferenceVideo] 10:24:26-551 max increment 339:53:08,480 = 18446744025987712772 us for frame 2386048
[addReferenceVideo] 10:24:26-551 About 20000 microseconds per frame, 2894766 frames
[getVideoDuration] 10:24:26-552 Found maxPts =02:49:34,401, 1 frames from the end
[getVideoDuration] 10:24:26-552 Found maxDts =02:49:34,341, 0 frames from the end
[getVideoDuration] 10:24:26-552 Using PTS..
[getVideoDuration] 10:24:26-552 Using duration of 02:49:34,421
[addReferenceVideo] 10:24:26-552 The first frame DTS = 900 ms
[addReferenceVideo] 10:24:26-552 The first frame has a PTS > 0, adjusting to 980 ms
[updateStartTime] 10:24:26-553 Using pts2=00:00:00,980 to get dts, as this video does not start at zero
[ADM_verifyDts] 10:24:26-553 Verifying DTS....
[ADM_verifyDts] 10:24:26-553 Checking from 1 to 2894766

Exactly the same bogus max increment value at almost identical frame number – this time no timestamp wrap even close to that spot.

[eumagga0x2a]

Could be a rounding error in timeConvert(), however, I'd expect that 2.7.4. behaves differently here.

[beta-tester]

do i understand it right, the new recorded TS file have fewer wrapping issues than the old one?
does it mean the stream in the TS file is kind of broken?

i sent you the index file and the log from the console from the linux version now.

[eumagga0x2a]

Thank you, I've got the files from a Linux build.

No, the stream is not broken. It is just that my first idea about the location in code which might be responsible was wrong.

edit: Avidemux can't handle more than one timestamp wrap in a stream, but that is a different problem.

[eumagga0x2a]

Could you please open the script console in Avidemux on Linux while the new huge TS file is loaded, and run the following commands?

Code Select Expand

ed = Editor()
for frame in range(2386000,2386100): ed.printTiming(frame)

The output is printed to the terminal (there should not be any errors), please post it.

[beta-tester]

here the output...

[eumagga0x2a]

Thank you, so timing is wrapping around at UINT_MAX in MPEG-TS ticks offset from the start (not absolute value):

Code Select Expand

Frame 2386039 Flags 0000 (P/F) DTS 13:15:21,680 PTS 13:15:21,780
Frame 2386040 Flags 4000 (B/F) DTS 13:15:21,700 PTS 13:15:21,740
Frame 2386041 Flags 4000 (B/F) DTS xx:xx:xx,xxx PTS 13:15:21,720
Frame 2386042 Flags 4000 (B/F) DTS 13:15:21,740 PTS 13:15:21,760
Frame 2386043 Flags 0000 (P/F) DTS 13:15:21,760 PTS 00:00:00,001
Frame 2386044 Flags 4000 (B/F) DTS 13:15:21,780 PTS 13:15:21,820
Frame 2386045 Flags 4000 (B/F) DTS xx:xx:xx,xxx PTS 13:15:21,800
Frame 2386046 Flags 4000 (B/F) DTS 13:15:21,820 PTS 13:15:21,840
Frame 2386047 Flags 0000 (P/F) DTS 13:15:21,840 PTS 00:00:00,081
Frame 2386048 Flags 4000 (B/F) DTS 00:00:00,001 PTS 00:00:00,041
Frame 2386049 Flags 4000 (B/F) DTS xx:xx:xx,xxx PTS 00:00:00,021
Frame 2386050 Flags 4000 (B/F) DTS 00:00:00,041 PTS 00:00:00,061
Frame 2386051 Flags 0000 (P/F) DTS 00:00:00,061 PTS 00:00:00,161
Frame 2386052 Flags 4000 (B/F) DTS 00:00:00,081 PTS 00:00:00,121

[eumagga0x2a]

Could you please test whether the attached patch fixes the problem? You don't need to re-index the TS file.

In the avidemux2 directory (i.e. in the top source directory)

Code Select Expand

patch -p1 < /path/to/demuxer-mpegts-try-to-fix-wrap.patch

(it should apply cleanly), then (1)

Code Select Expand

bash bootStrap.bash --rebuild --without-core --without-qt --without-cli

To reset changes made by the patch, you can use

Code Select Expand

git checkout avidemux_plugins/ADM_demuxers/MpegTS/ADM_tsComputeTimeStamp.cpp

in the top source directory followed by (1).

[beta-tester]

looks loke a deadlock...
one CPU core is at 100%
nothing happens.

i killed the process once. (p1.log)

after that i tried with re indexing the TS file; the indexing process finished successfully but then again ADM hangs at the same situation.

EDIT: ubuntu is now complaining, that avmdemux3_pt5 is not responding [force quit][wait]

[eumagga0x2a]

Oops, thank you, next try (you need to adjust the patch command accordingly).