Please add ability to disable admlog.txt

Started by wokoxe2060, June 06, 2021, 02:22:48 AM

Previous topic - Next topic

wokoxe2060

I very need this feature. It does unnecessary operations. I don't need any logs. This file has grown to 2 gb before I noticed.
Please add option in settings or in config file to disable this file entirely.

eumagga0x2a

The log file is truncated (overwritten) each time Avidemux is started, so it must have grown that big during a single Avidemux session. There is a single known reason for it to grow indefinitely while Avidemux seems to do nothing (turn mouse wheel atop of the thumb slider to move it out of the neutral position and move away the mouse). I'm fixing this right now.

If you have deleted the huge log without creating a copy first, it is impossible to tell what exactly caused the issue.

Quote from: wokoxe2060 on June 06, 2021, 02:22:48 AMI don't need any logs.

Avidemux development profits from the log's availability.


wokoxe2060

Quote from: eumagga0x2a on June 06, 2021, 08:47:06 PMIf you have deleted the huge log without creating a copy first, it is impossible to tell what exactly caused the issue.
It was caused because video was broken and gave huge amount of DTS errors.
Quote from: eumagga0x2a on June 06, 2021, 08:47:06 PMAvidemux development profits from the log's availability.
Enable it by default, and let expert users disable unwanted functionality

eumagga0x2a

Quote from: wokoxe2060 on June 08, 2021, 02:19:28 AMIt was caused because video was broken and gave huge amount of DTS errors.

Could you please provide more details? If the content of the video is innocuous enough, could you please provide the source (a small portion of the source, if the video is a MPEG-TS file) as a sample via WeTransfer (doesn't require an email address, but has a 2 GiB size limit), Mega, Dropbox or Google Drive?

Ideally, we should catch broken DTS in the demuxer or in the editor at the latest. The collision avoidance in the muxer code, probably responsible for a good deal of noise in the log, is the last ditch effort. However, at 60fps, it should take over two days worth of video footage to inflate the log to 2 GiB. I would like to investigate how this could happen.

(Of course, I assume you tried the latest nightly.)

Quote from: wokoxe2060 on June 08, 2021, 02:19:28 AMEnable it by default, and let expert users disable unwanted functionality

You would not have reported the issue if this feature would have been implemented the way you want.

burf

I've noticed this too... my log will grow enormous, when I've loaded an incomplete file downloaded via bittorrent. The initial loading process, when it's scanning the file takes much longer, & invariably the log file grows over 2GBs. And that's prior to editing any scenes I need. I've never paid much mind, once I noticed it. Just opening & closing Avidemux clears it.

wokoxe2060

Quote from: eumagga0x2a on June 08, 2021, 10:47:14 PMIdeally, we should catch broken DTS in the demuxer or in the editor at the latest. The collision avoidance in the muxer code, probably responsible for a good deal of noise in the log, is the last ditch effort. However, at 60fps, it should take over two days worth of video footage to inflate the log to 2 GiB. I would like to investigate how this could happen.
Yes, it's ffmpeg spamming a ton of Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2519 >= 2519 errors.
This video was made after recording Youtube DASH stream after 2 hours. Seems like when it hit the 2 hours mark, it starts recording the whole stream from the start, for over and over until the stream is ended. And instead of 4 GB output takes 25 GB. If you decode this video to raw bitstream, Like this:
ffmpeg -loglevel panic -i omg.mkv -c copy omg.h264
ffmpeg -i omg.h264 -c copy omg2.mkv
you can see exactly how it loops. Look at right corner of my video to see the time. Youtube even in browser has problems after 2 hours, before video is processed. But you can rewind it fine when stream is running.


I especially need to record DASH stream, because over HLS you can't get higher resolution than 1080p and VP9 codec, also it doesn't start from the beginning. I even made this method for myself, there is no description in web, all other tools like youtube-dl or streamlink are using HLS recording with these limitations.

I cut a small portion of video using ffmpeg -ss 2:00:00 -t 50 -i out.mkv -c copy omg.mkv and uploaded it to google drive https://drive.google.com/file/d/1Y1DZcpYUUThZMsiRGpU1fqbm7CBEsMK_/. With duration of 42 seconds it takes 1.75 gb! But it's working fine in player.  Also, when I tried to make a video less than 50 seconds, it outputted fine, without hidden video. After making that trick with bitstream you can see real duration is 1 hour 45 minutes. Maybe you could fix this bug in ffmpeg instead? Anyway you can't easily fix that broken video, you need to cut it from begginning to the end, with audio sync and other, also mentioning before 2 hours would be fine. I used avidemux to join a few parts into a normal video, and later discovered that log file takes so much space. For ffmpeg i just use -loglevel panic.

Quote from: eumagga0x2a on June 08, 2021, 10:47:14 PMYou would not have reported the issue if this feature would have been implemented the way you want.
I know that my video is broken with ffmpeg and huge size so I don't need 2 gb+ logs

eumagga0x2a

So the problem is the inability of the tool used to capture YouTube streams to recognize wrap-around, isn't it? Avidemux and FFmpeg are just collateral damage. (I don't mean that handling of PTS collisions in the Matroska demuxer in Avidemux doesn't need a fix.)

What tool do you use for this task? Does it output a fragmented MP4 which is then passed to Avidemux?

wokoxe2060

Quote from: eumagga0x2a on June 17, 2021, 05:09:54 PMSo the problem is the inability of the tool used to capture YouTube streams to recognize wrap-around, isn't it? Avidemux and FFmpeg are just collateral damage. (I don't mean that handling of PTS collisions in the Matroska demuxer in Avidemux doesn't need a fix.)

What tool do you use for this task? Does it output a fragmented MP4 which is then passed to Avidemux?
Uhm, I'm using ffmpeg for capturing these streams, after getting video link from youtube-dl...

eumagga0x2a

Could you please provide a YouTube ID with a short (if possible) stream to test the wrap-around? Capturing the London bus ride (obviously not live, ytid: s2tssDyJLU0) seems to work just fine, but I didn't try recording for long, so no wrap-around.

Quote from: wokoxe2060 on June 18, 2021, 09:46:22 AMafter getting video link from youtube-dl...

Are you doing this with a custom script in a loop? Or how else it is supposed to work? Why not simply

youtube-dl --hls-use-mpegts https://www.youtube.com/watch?v=<your YouTube-ID goes here>and renaming the generated fake "mp4" to "ts"? Keeping MPEG-TS format has the benefit of being able to cut the file at any offset and still getting a working video.

wokoxe2060

#10
Quote from: eumagga0x2a on June 18, 2021, 07:07:07 PMAre you doing this with a custom script in a loop? Or how else it is supposed to work? Why not simply
youtube-dl -F --print-traffic linkAfter youtube-dl prints [youtube]: Downloading MPD manifest, find there url with DASH in it: send: b'GET /api/manifest/dash/expire/1624576477/ei/fb3UYLjNKcTR7gSPy5C...
ffmpeg -i "https://manifest.googlevideo.com/<url>"Append it to domain and check avaliable streams
...
  Stream #0:16: Video: vp9 (Profile 0), yuv420p(tv, bt709), 1920x1080, 1k tbr, 1k tbn, 1k tbc (default)
    Metadata:
      variant_bitrate : 3016000
      id              : 303
  Stream #0:17: Video: vp9 (Profile 0), yuv420p(tv, bt709), 2560x1440, 1k tbr, 1k tbn, 1k tbc (default)
    Metadata:
      variant_bitrate : 9016000
      id              : 308
  Stream #0:18: Video: vp9 (Profile 0), yuv420p(tv, bt709), 3840x2160, 1k tbr, 1k tbn, 1k tbc (default)
    Metadata:
      variant_bitrate : 12016000
      id              : 313
  Stream #0:19: Video: vp9 (Profile 0), yuv420p(tv, bt709), 3840x2160, 1k tbr, 1k tbn, 1k tbc (default)
    Metadata:
      variant_bitrate : 18016000
      id              : 315
  Stream #0:20: Audio: aac (HE-AAC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 0 kb/s (default)
    Metadata:
      variant_bitrate : 64000
      id              : 139
  Stream #0:21: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 0 kb/s (default)
    Metadata:
      variant_bitrate : 144000
      id              : 140
ffmpeg -y -i "https://manifest.googlevideo.com/<url>" -map 19 -map 0:a:1-c copy 1.mkvSelect highest avaliable quality with map, the number will change depending of avaliable resolutions. for audio they're the same
Quoteyoutube-dl --hls-use-mpegts
This downloads HLS stream which is limited to 1080 as said earlier unlike DASH
QuoteCould you please provide a YouTube ID with a short (if possible) stream to test the wrap-around?
That wrap-around only happens on long streams that longer than 2 hours. So, I started recording stream using dash link, and after 2 hours of recording it starting wrap-around. You can select lowest resolution for less space

Sorry for late reply, I will answer faster 😬

eumagga0x2a

Quote from: wokoxe2060 on June 24, 2021, 05:21:35 PMyoutube-dl -F --print-traffic linkAfter youtube-dl prints [youtube]: Downloading MPD manifest,

The version of youtube-dl which is included in Fedora (currently 2021.05.16) doesn't print this for "link" substituted by URLs with YT-IDs I tried, so please provide an ID which exhibits the behaviour you describe. If I missed something, please help me to identify the missing bits.

Quote from: wokoxe2060 on June 24, 2021, 05:21:35 PMThis downloads HLS stream which is limited to 1080

Honestly, is 1080p not HD enough?

wokoxe2060

I can't post my message because it's too big and CleanTalk swears about contacts
You cannot view this attachment.

eumagga0x2a

Thank you and sorry for the delay, I was on metered internet for a few days. The ID in the attached instructions is no more, so I started experimenting with _UTMDjWRHBk instead.

Generally, it works as advertized, but I would strongly recommend to output to robust format like MPEG-TS instead of a fragile Matroska. If timestamps wrap around at some point, it is possible to cut a .ts file at any location and it will still work. After that, it is trivial to remux the recording to MKV or whatever.

wokoxe2060

#14
Quote from: eumagga0x2a on June 28, 2021, 04:40:41 PMGenerally, it works as advertized, but I would strongly recommend to output to robust format like MPEG-TS instead of a fragile Matroska. If timestamps wrap around at some point, it is possible to cut a .ts file at any location and it will still work. After that, it is trivial to remux the recording to MKV or whatever.
That's interesting idea, but VP9 itsn't supported in M2TS officialy. ffmpeg says:

ffmpeg -i 1.webm" -c copy 1.m2ts

[mpegts @ 00000268ac5200c0] Stream 0, codec vp9, is muxed as a private data stream and may not be recognized upon reading.
[mpegts @ 00000268ac5200c0] Stream 1, codec opus, is muxed as a private data stream and may not be recognized upon reading.
frame= 3522 fps=0.0 q=-1.0 Lsize=   21960kB time=00:01:57.44 bitrate=1531.8kbits/s speed=2.25e+03x
video:18142kB audio:1923kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 9.445446%

So video and audio exist in m2ts file, size are about the same.
ffmpeg -i 1.m2ts

  Stream #0:0[0x1011]: Data: bin_data ([6][0][0][0] / 0x0006)
  Stream #0:1[0x1100](eng): Audio: opus (Opus / 0x7375704F), 48000 Hz, stereo, fltp
it can detect Opus audio, but it doesn't know what is first stream.
Maybe you know how to specify a codec in input file? I tried different ways, but they're not working
ffmpeg -vcodec vp9 -i 1.m2ts -map 0:0 -c copy 1.mkv

[matroska @ 000001fa8ae4d600] Tag [6][0][0][0] incompatible with output codec id '100359' ([0][0][0][0])
Could not write header for output file #0 (incorrect codec parameters ?): Invalid data found when processing input
Error initializing output stream 0:0 --
Mkvtoolnix can't detect video stream too, but also incorrectly detect opus as ac-3