Hello!
I have to resize a lot of *.ts by repacking them to *.mkv. While everything works fine with avidemux with GUI, when trying to use ffmpeg on the command line because of performance issue, I got errors.
Is it possible to get the settings for ffmpeg that are used in avidemux.
Input *.ts:
Input #0, mpegts, from 'Test.ts':
Duration: 02:05:36.08, start: 0.028789, bitrate: 8515 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k tbn, 100 tbc
Stream #0:1[0x101](deu): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 448 kb/s
Stream #0:2[0x102](mis): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 448 kb/s
Output:
Should be *.mkv
This is what I am using with avidemux: Works fine :-)
#PY <- Needed to identify #
#--automatically built--
adm = Avidemux()
adm.loadVideo("D:/video/raw/Test.ts")
adm.clearSegments()
adm.addSegment(0, 0, 7536160000)
adm.markerA = 0
adm.markerB = 7536160000
adm.videoCodec("Copy")
adm.audioClearTracks()
adm.setSourceTrackLanguage(0,"deu")
adm.setSourceTrackLanguage(1,"mis")
adm.audioAddTrack(0)
adm.audioCodec(0, "copy");
adm.audioSetDrc(0, 0)
adm.audioSetShift(0, 0,0)
adm.audioAddTrack(1)
adm.audioCodec(1, "copy");
adm.audioSetDrc(1, 0)
adm.audioSetShift(1, 0,0)
adm.setContainer("MKV", "forceDisplayWidth=False", "displayWidth=1280", "displayAspectRatio=0")
I no, that I am not 100% in the right forum, but avidemux do the right things while ffmpeg direct don't.
Best regards
Manfred
Quote from: manbau00 on September 21, 2018, 08:20:47 AM
I have to resize a lot of *.ts by repacking them to *.mkv. While everything works fine with avidemux with GUI, when trying to use ffmpeg on the command line because of performance issue, I got errors.
Please specify what is the performance issue you experience with Avidemux and what are the error messages from ffmpeg verbatim.
Hi!
Ad performance (network data transfer rate) issue, I want to do the conversation on the file server on which the files are stored. But on this machine I will do as less package installation as possible. So hopefully I have not to use a remote machine with avidemux installed to do the conversation because that needs a lot of time for transferring data.
Ad ffmpeg error,
# ffmpeg -i Test.mpg -map 0 -c copy Test.mkv
ffmpeg version 3.4.4-0ubuntu0.18.04.1 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 7 (Ubuntu 7.3.0-16ubuntu3)
configuration: --prefix=/usr --extra-version=0ubuntu0.18.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
libavutil 55. 78.100 / 55. 78.100
libavcodec 57.107.100 / 57.107.100
libavformat 57. 83.100 / 57. 83.100
libavdevice 57. 10.100 / 57. 10.100
libavfilter 6.107.100 / 6.107.100
libavresample 3. 7. 0 / 3. 7. 0
libswscale 4. 8.100 / 4. 8.100
libswresample 2. 9.100 / 2. 9.100
libpostproc 54. 7.100 / 54. 7.100
Input #0, mpegts, from 'Test.mpg':
Duration: 02:05:36.08, start: 0.028789, bitrate: 8515 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k tbn, 100 tbc
Stream #0:1[0x101](deu): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 448 kb/s
Stream #0:2[0x102](mis): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 448 kb/s
Output #0, matroska, to 'Test.mkv':
Metadata:
encoder : Lavf57.83.100
Stream #0:0: Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 50 fps, 50 tbr, 1k tbn, 90k tbc
Stream #0:1(deu): Audio: ac3 ([0]
- / 0x2000), 48000 Hz, 5.1(side), fltp, 448 kb/s
Stream #0:2(mis): Audio: ac3 ([0] - / 0x2000), 48000 Hz, 5.1(side), fltp, 448 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Stream #0:2 -> #0:2 (copy)
Press [q] to stop, [?] for help
[matroska @ 0x5634dbae5cc0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[matroska @ 0x5634dbae5cc0] Can't write packet with unknown timestamp
av_interleaved_write_frame(): Invalid argument
frame= 2 fps=0.0 q=-1.0 Lsize= 165kB time=00:00:00.00 bitrate=450074.7kbits/s speed=0.539x
video:171kB audio:4kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!
The same happens with (-fflags +genpts)
# ffmpeg -fflags +genpts -i Test.mpg -map 0 -c copy Test.mkv
ffmpeg version 3.4.4-0ubuntu0.18.04.1 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 7 (Ubuntu 7.3.0-16ubuntu3)
configuration: --prefix=/usr --extra-version=0ubuntu0.18.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
libavutil 55. 78.100 / 55. 78.100
libavcodec 57.107.100 / 57.107.100
libavformat 57. 83.100 / 57. 83.100
libavdevice 57. 10.100 / 57. 10.100
libavfilter 6.107.100 / 6.107.100
libavresample 3. 7. 0 / 3. 7. 0
libswscale 4. 8.100 / 4. 8.100
libswresample 2. 9.100 / 2. 9.100
libpostproc 54. 7.100 / 54. 7.100
Input #0, mpegts, from 'Test.mpg':
Duration: 02:05:36.08, start: 0.028789, bitrate: 8515 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k tbn, 100 tbc
Stream #0:1[0x101](deu): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 448 kb/s
Stream #0:2[0x102](mis): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 448 kb/s
File 'Test.mkv' already exists. Overwrite ? [y/N] y
Output #0, matroska, to 'Test.mkv':
Metadata:
encoder : Lavf57.83.100
Stream #0:0: Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 50 fps, 50 tbr, 1k tbn, 90k tbc
Stream #0:1(deu): Audio: ac3 ([0] - / 0x2000), 48000 Hz, 5.1(side), fltp, 448 kb/s
Stream #0:2(mis): Audio: ac3 ([0] - / 0x2000), 48000 Hz, 5.1(side), fltp, 448 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Stream #0:2 -> #0:2 (copy)
Press [q] to stop, [?] for help
[matroska @ 0x5623633b4c60] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[matroska @ 0x5623633b4c60] Can't write packet with unknown timestamp
av_interleaved_write_frame(): Invalid argument
frame= 2 fps=0.0 q=-1.0 Lsize= 165kB time=00:00:00.00 bitrate=450074.7kbits/s speed=0.956x
video:171kB audio:4kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!
Regards
Manfred
Which application is responsible for generating a transport stream with a H.264 video without valid PTS? For this specific case, you can't do anything useful with ffmpeg (Avidemux offers a time-consuming workaround which might be able to restore presentation timestamps).
This is true that handling large video files with Avidemux over a slow network connection is no fun.
Mostly often the files are from DVB-S2 streams stored with mythtv.
Is it possible to do the restoration of the presentation timestamps with some bash scripting? Where can I find the code fragment which do the trick in the source code in avidemux?
Could you please provide a sample?
The PTS check and reconstruction is done by ADM_Composer::checkForValidPts (https://github.com/mean00/avidemux2/blob/master/avidemux/common/ADM_editor/src/utils/ADM_edCheckForInvalidPts.cpp#L36), called from ADM_Composer::addFile (https://github.com/mean00/avidemux2/blob/master/avidemux/common/ADM_editor/src/ADM_edit.cpp#L167). I doubt that this matches your experience when loading these streams in Avidemux.
No, bash scripting can't accomplish this task :-)
Here is the sample. It is cut and stream copied into MpegTS container with avidemux.
https://secure.edolos.com/owncloud/public.php?service=files&t=2e80c8a8130b3a97c2b84bffa32155f8
Regards
Manfred
I'm sorry, please provide a sample of the original source transport stream, not a remuxed one.
dd if=/path/to/source of=/path/to/sample bs=1M count=300
Hi,
I am not allowed to post!
An Error Has Occurred!
CleanTalk: *** Forbidden. Please enable JavaScript. Message seems to be spam. ***
Can you help me?
Regards
Manfred
Hi!
I put the Message into a text document. Hopefully this works.
Regards
Manfred
Thank you, I hope to be able to look at it later today.
I can reproduce the issue (also with my own captures from DVB-C). The reason is not that PTS were unset but a few frames pretty close to the start of the cut stream get invalid (not monotonically increasing) DTS. Avidemux doesn't see DTS going back (it regards DTS of some frames as unset, which may be correct and harmless).
Interestingly, skipping the start of the stream with say ffmpeg -ss 0.2 -i Sample02_cut_with_avidemux.ts allows to remux it to MKV without errors, but PTS in this MKV are totally broken.
From the practical POV, if you have to use Avidemux anyway to delete portions of the video, why not saving the result to a MKV right away?
Hi,
for the first, many thanks for your analysis of the issue.
Ad practical POV, yes I am willing to do so for the future. But I have already many damaged files.
Do you think skipping the first frames and trying -ss 0.2 -fflags +genpts is a solution? Could this restore the PTS?
Could this also be fixed in Avidemux that it does not happen again?
Regards
Manfred
I'll tried the following:
ffmpeg -ss 0.2 -fflags +genpts -i Test.mpg -map 0 -c copy -f matroska -ignore_unknown -sn -y Test.mkv
But now it happens later. After the 300MB.
Input #0, mpegts, from 'Test.mpg':
Duration: 02:05:36.08, start: 0.028789, bitrate: 8515 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k tbn, 100 tbc
Stream #0:1[0x101](deu): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 448 kb/s
Stream #0:2[0x102](mis): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 448 kb/s
Output #0, matroska, to 'Test.mkv':
Metadata:
encoder : Lavf57.83.100
Stream #0:0: Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 50 fps, 50 tbr, 1k tbn, 90k tbc
Stream #0:1(deu): Audio: ac3 ([0]
- / 0x2000), 48000 Hz, 5.1(side), fltp, 448 kb/s
Stream #0:2(mis): Audio: ac3 ([0] - / 0x2000), 48000 Hz, 5.1(side), fltp, 448 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Stream #0:2 -> #0:2 (copy)
Press [q] to stop, [?] for help
[matroska @ 0x5577f9e44f60] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[matroska @ 0x5577f9e44f60] Can't write packet with unknown timestamp
av_interleaved_write_frame(): Invalid argument
frame=19272 fps=1223 q=-1.0 Lsize= 411301kB time=00:06:25.85 bitrate=8732.3kbits/s speed=24.5x
video:368744kB audio:42220kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.081901%
Conversion failed!
Quote from: manbau00 on September 22, 2018, 06:40:56 PM
Do you think skipping the first frames and trying -ss 0.2 -fflags +genpts is a solution? Could this restore the PTS?
No. BTW, -ss belongs before -i. PTS are broken by ffmpeg when saving after seek.
QuoteCould this also be fixed in Avidemux that it does not happen again?
I'll try to look into it, don't have time ATM though.
The search for the actual bug in Avidemux is ongoing, but the nature of the problem becomes apparent: when indexing TS files, Avidemux gets the size of each keyframe slightly wrong (in the samples I tried +6 bytes), which means that following PES packets are shifted 6 bytes relative to respective frame boundaries. This confuses ffmpeg which sees the beginning of a second frame in the first packet but doesn't have DTS/PTS for it yet. When the second packet arrives from the demuxer, it pushes it with timestamps unset to the muxer so that saving fails.
If you edit the .idx2 file to subtract 6 from the first field following each IF or DF marker, reload the TS file in Avidemux and save a portion of it in copy mode using the MpegTS muxer, ffmpeg is able to remux resulting video to mkv without problems.
Fine, hopefully you find it.
At the moment I use ffmpeg to cut the ts files.
Quote from: eumagga0x2a on October 09, 2018, 07:00:44 PM
the nature of the problem becomes apparent: when indexing TS files, Avidemux gets the size of each keyframe slightly wrong (in the samples I tried +6 bytes)
It turned out to be that it is the internal FFmpeg that adds these 6 bytes to each packet with H.264 video, my first analysis was plain wrong.
Please try the latest nightly (https://avidemux.org/nightly/), it contains a fix by Mean for mpegts with H.264 video (not yet for HEVC).
The problem is that during indexing of H.264 streams Avidemux sets the frame boundary to the first NAL unit after the access unit delimiter (AUD), it should be at AUD. The internal libavformat notices that the packet doesn't start with AUD and generates an AUD itself (this increases the length of each packet by 6 bytes for H.264 and by 7 bytes for HEVC). Now we get two consecutive delimiters – the one which came with the original stream and actually belongs to the following frame, but now is at the end of the current frame, and another one that libavformat added for us.
When ffmpeg parses such a stream, it interprets the second AUD as an empty packet with unknown timestamp – at this point remuxing fails. The fix removes trailing AUDs before the frame is passed to the muxer.
Things are a little bit more complex for HEVC (Avidemux sets frame boundary to a different type of NALU) , but the general outlines of the problem are the same.
Test with H264 looks good.
First I cut the file with avidemux and remuxed it to MPEG-TS. Afterwards I remuxed it to mkv with ffmpeg.
ffmpeg -hide_banner -i Test.ts -map 0 -c copy -ignore_unknown -sn -f matroska -y Test.mkv
Input #0, mpegts, from 'Test.ts':
Duration: 01:30:21.34, start: 0.025122, bitrate: 11977 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k tbn, 100 tbc
Stream #0:1[0x101](deu): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s
Stream #0:2[0x102](mis): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s
Stream #0:3[0x103](deu): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 448 kb/s
Output #0, matroska, to 'Test.mkv':
Metadata:
encoder : Lavf57.83.100
Stream #0:0: Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 50 fps, 50 tbr, 1k tbn, 90k tbc
Stream #0:1(deu): Audio: mp2 (P[0][0][0] / 0x0050), 48000 Hz, stereo, s16p, 192 kb/s
Stream #0:2(mis): Audio: mp2 (P[0][0][0] / 0x0050), 48000 Hz, stereo, s16p, 192 kb/s
Stream #0:3(deu): Audio: ac3 ([0]
- / 0x2000), 48000 Hz, stereo, fltp, 448 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Stream #0:2 -> #0:2 (copy)
Stream #0:3 -> #0:3 (copy)
Press [q] to stop, [?] for help
frame=271066 fps=8174 q=-1.0 Lsize= 7250751kB time=01:30:21.29 bitrate=10956.4kbits/s speed= 163x
video:6693272kB audio:550600kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.094967%
Is there a way to reconstruct the damaged MPEG-TS Files? Or is the information lost forever?
Quote from: manbau00 on October 26, 2018, 08:16:22 AM
Is there a way to reconstruct the damaged MPEG-TS Files? Or is the information lost forever?
In case of H.264, no information is lost at all. In case of HEVC, it looks like dropping SEI_PREFIX NALUs mostly doesn't cause any trouble. As of now, the new AUD remover doesn't handle HEVC correctly as it removes AUD but not the following SEI_PREFIX NALUs which actually belong to the following frame. In any case, no picture data is lost.
To fix mpegts files containing a H.264 video track with multiple consecutive AUDs, save them in copy mode using Avidemux (using a nightly build containing the fix mentioned above).