Avidemux Forum

I got a list of 350 files. Am I supposed to append all of them one by one? Are you *******ing me?
You said I could whine here, so I'm effing whining. Christ.

Sort input files in the file manager window in the order they should be appended, select them all (if you can't sort them automatically in the file manager, you have to select them one by one in the right order, obviously) and drag'n'drop the whole selection onto Avidemux window.

Quote from: eumagga0x2a on July 15, 2020, 02:34:06 PM
Sort input files in the file manager window in the order they should be appended, select them all (if you can't sort them automatically in the file manager, you have to select them one by one in the right order, obviously) and drag'n'drop the whole selection onto Avidemux window.

By file manager you mean like Windows, right? In case people use Linux or something, I understand. I use Windows 7.
I tried that, it crashed at number 169.

By file manager I mean an application used to manage files, e.g. Windows Explorer or Total Commander on Windows, Finder on macOS, Dolphin, Nautilus etc. on Linux.

Quote from: john2453754 on July 15, 2020, 04:39:44 PM
I tried that, it crashed at number 169.

Please reproduce the crash, then compress (zip, 7z) and attach admlog.txt from %localappdata% to your reply. If you can indentify the particular video which crashes Avidemux, it would be easier when you provide the log just from starting Avidemux afresh and loading this video.

As appending a number of videos means creating demuxer, decoder, cache and encoder instance for each of them – all operations requiring considerable amounts of memory – it may happen that the resources get exhausted. If the crash is not related to a particular video but just to the number of them, reducing cache size from the default size of 16 images to the minimum of 8 might help somewhat.

Quote from: eumagga0x2a on July 15, 2020, 05:14:27 PM
Please reproduce the crash, then compress (zip, 7z) and attach admlog.txt from %localappdata% to your reply.

Oops, %localappdata%\avidemux\ of course.

Looks like the whole site died for a moment there?
So yeah, tried it again, tried reducing the cache size like you said.

It always crashes on the same file, #169. However it plays fine as a standalone video (MPC-HC 1.9.6). I tried to convert it to webm using WebmConverter (formerly for Webm for R***), both version 2.26 and 1.0 which I keep around, because it's very simple, and has worked for years and occasionally even works better than the newer one. Anyway it converts fine. It seems to be fine as a file.

So I finally thought of the obvious, and tried starting from (and including) #169. With 350 files total, I noted that was 182, so more, in case it ended up working.
What happened is it crashed after the same number of files, 170, and so crashed at #338.
It was tracking the memory usage every time (with the basic Task Manager), and it seemed to crash when reaching about 5.2G of memory. But I'm supposed to have 16G.

So here are the admlogs, I kept an "8 cache" one and a 16 one, and also included the .idx2 files for #168 and 169 in each case, in case they have any information or if it means anything.

It doesn't look like a memory exhaustion issue. As the failure happens always after a fixed number of files appended, I guess this is the operating system refusing to allocate any more file descriptors for Avidemux (we need one per video and one per each audio track in a file). It doesn't look like anything can be done about it.

Maybe the number could be increased using https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/setmaxstdio?view=vs-2019 (https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/setmaxstdio?view=vs-2019), but the use case of that many appended videos is pretty exotic.

Only problem though : I've been using an old, shitty little program called SolveigMM Video Splitter for a while, because it's simple, easy, and it just works. It can split and join videos, provided in the latter case they share the exact same characteristics. And it can do it.

edit : disregard my previous version of this post if you saw it, it was fairly dumb. But the above point remains.

Quote from: john2453754 on July 17, 2020, 04:46:50 PM
I've been using an old, shitty little program called SolveigMM Video Splitter for a while, because it's simple, easy, and it just works. It can split and join videos, provided in the latter case they share the exact same characteristics.

Well, this is exactly what we do.

QuoteAnd it can do it.

Great, so they either use a different C library, bump the limit, don't create a separate file descriptor for audio (which would cause a terrible lot of seeks) or temporarily close files which are loaded, but not currently read or written. I'll try not to forget to look into it later. This is not something which is very easy to test.

In case of success, this will benefit maybe 0.001% of user base.

Quote from: john2453754 on July 15, 2020, 01:27:33 PM
I got a list of 350 files. Am I supposed to append all of them one by one? Are you *******ing me?
You said I could whine here, so I'm effing whining. Christ.

You can easily contentate TS files in FFMPEG and then load the combined file in ADM, if needed.   I didn't have 350 TS files here, so I used a single TS file to make 350 sequentially-numbered copies.

Original.ts is 20.537s in duration, 1080p59.94 and 59.4MB

Create_350_copies.bat
@echo off
SETLOCAL EnableDelayedExpansion
for /L %%i IN (1,1,350) do call :docopy %%i
goto :EOF

:docopy
set FN=000%1
set FN=%FN:~-4%
copy original.ts Sequence_%FN%.ts

The resulting copies will be named Sequence_0001.ts to Sequence_0350.ts

Now create a list of them in the filelist format FFMPEG uses, in which each line looks something like:

file 'Sequence_0215.ts'

In Windows File Explorer, when you have navigated to the the directory where you created your file sequence, in the address bar at the top type cmd to get to the command prompt in this directory. Then type:

for %i in (sequence_*.ts) do @echo file '%i' >> filelist.txt

If no non-sequence TS files are present here (e.g. you've already deleted original.ts) you could simply it to just:

for %i in (*.ts) do @echo file '%i' >> filelist.txt

Note ">>" will append to an existing list or create a new one if it doesn't already exist. So, if you want to rerun this command, delete the old filelist.txt first by typing del filelist.txt

If you want to check the contents of the created list:

Type type, leave a space and then type either f or fi and press Tab to auto-match the correct filename (press Tab again if an incorrect first filename match is returned):

type filelist.txt


Once you have the list created, and assuming that FFMPEG.exe is in either a directory mentioned in your environmental variable PATH list or you have copied it to the current directory you're working in, type:

ffmpeg -f concat -safe 0 -i filelist.txt -c copy big_file.ts

This is what I saw:
ffmpeg version N-98478-g1ec2b3de5a Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 10.1.0 (Rev3, Built by MSYS2 project)
  configuration:  --cc='ccache gcc' --cxx='ccache g++' --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-schannel --enable-zlib --enable-sdl2 --enable-ffnvcodec --enable-nvdec --enable-cuda-llvm --enable-gmp --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libdav1d --enable-libaom --disable-debug --extra-libs=-liconv --enable-gpl --enable-version3
  libavutil      56. 55.100 / 56. 55.100
  libavcodec     58. 96.100 / 58. 96.100
  libavformat    58. 48.100 / 58. 48.100
  libavdevice    58. 11.101 / 58. 11.101
  libavfilter     7. 87.100 /  7. 87.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
Input #0, concat, from 'filelist.txt':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc
    Stream #0:1(und): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 119 kb/s
Output #0, mpegts, to 'big_file.ts':
  Metadata:
    encoder         : Lavf58.48.100
    Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 59.94 fps, 59.94 tbr, 90k tbn, 90k tbc
    Stream #0:1(und): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 119 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=430850 fps=2736 q=-1.0 Lsize=21314973kB time=01:59:55.46 bitrate=24267.0kbits/s speed=45.7x
video:20666735kB audio:115094kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.565431%


FFMPEG's build and library info can be hidden in the screen display by including the -hide_banner switch.
If you which to repeat the concat command a 2nd time, FFMPEG will warn you that big_file.txt already exists and do you want to overwrite it? You can override this with the -y switch.

ffmpeg -f concat -safe 0 -i filelist.txt -c copy -y -hide_banner big_file.ts

On my SSD, the concatenation process proceeded at a speed of 45.7x so, for example, a group of files totalling 45mins in duration would take about 1min to concatenate together.

big_file.ts was 1:59:55 when played in VLC and the concat switch-over points played back cleanly.  Size was 20.3GB.

Dan.

Thank you for all of that. I'm just not familiar with cmd, .bat, or ffmpeg, or command lines in general because f*** writing everything by hand in that shitty, tiny window. But I imagine there exists alternatives, editors and whatnot. Just never looked into it.
And I actually did so just last night, understood how to update ffmpeg, and managed to use it (for another purpose, convert gifs into webms), and I'm pretty happy to have managed to make it work (just on cmd, no .bat). I still don't quite understand how the code works in details, or simply why so many things are just not simplified more, but whatever. I don't know if anybody's interested, especially not you guys since you know your shit, but here's the code :
for /f "tokens=1 delims=." %a in ('dir /b *.gif') do ffmpeg -i "%a.gif" -an -c:v libvpx -pix_fmt yuv420p -threads 15 -slices 8 -lag-in-frames 16  -auto-alt-ref 1 -qmin 13 -crf 15 -qmax 17 -qcomp 1 -b:v 0 "%a.webm"
Plus right-click > "Open command window here". I tried to do that part automatically with Autohotkey, but... whatever, I couldn't manage to do it, this isn't a blog.

Anyway thank you, I'll try to read and understand your post, and look into those .bat things.

A batch file in Windows/DOS has ".bat"  file extension. In Linux, it's called a script file. It's just a list of commands that you would type at the command prompt. You create it in a text editor e.g. Window's Notepad and give it a ".bat" file extension.  To run it, you just type its name. So, if I type Hello and press Enter, the command processor looks for hello.com, hello.exe, hello.cmd, hello.bat, and I think hello.vbs (Visual BASIC program). It looks for these programs first in the current directory, then in your PATH list of directories. (Type path to see what's currently in this list and set to see all the EVs that Windows is using.)

Note: when you type cmd in the File Explorer address bar to open a command-prompt window, you are actually running cmd.exe (Windows command processor) which is situated in C:\Windows\System32

The command prompt, ending in ">", shows you your current directory. So "C:\Temp>" would mean you're currently situated on C: drive in the temp directory, (Windows also calls directories "folders"), which is located directly beneath the root directory, which on C: is C:\. To see what's in a directory you use the DIR command. So from C:\Temp> typing either DIR or dir, (the Windows command-processor isn't case-sensitive), would show you what's there. From any other directory on C: to see what's in c:\temp you'd use dir \temp or from anywhere on D: drive you would use dir c:\temp. To see just the .TS files in the current directory, you use the "*" wildcard: dir *.ts. To see all the TS files whose filenames start with "test" you'd use dir test*.ts. The DOS commands usually have a help screen. To see this, use the "/?" switch i.e. dir /?

You called it a small window, but you can increase it and change the font size to see things better. You can scroll back, and use the Up Arrow (goes back through your command history so you can reuse commands), Tab key (auto-completes matching filenames) and some of the function keys (e.g. F8) to make working at the command prompt easier. (You can also cut-and-paste filenames and text between a Windows program and the command-prompt window.) If you are using FFMPEG (it's a very powerful free AV processing program) you'll be going to a command prompt often.

Once you've worked out the commands to put in a FFMPEG batchfile, you can place a shortcut to it on your Windows desktop and then drag-and-drop media file(s) onto this icon. I do this for an easy way to convert and process media files.

Dan.

Yes I know some bits of that, thank you a lot, I'm not asking for a tutorial right here and there ;D  I'll have to do my own research.

Quote from: eumagga0x2a on July 17, 2020, 04:40:31 PMMaybe the number could be increased using https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/setmaxstdio?view=vs-2019 (https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/setmaxstdio?view=vs-2019), but the use case of that many appended videos is pretty exotic.

I am having an issue where I am trying to append many files also (from security camera footage).
My problem is, when I try to drag my files into a new session (28 x 4MB files); they don't get into Avidemux in order.
:(

Quote from: timaavid on January 02, 2021, 07:07:14 AMwhen I try to drag my files into a new session (28 x 4MB files); they don't get into Avidemux in order

If Windows, try dragging from Everything (https://www.voidtools.com/) (into Avidemux) & see if that helps.

(I'd think a real file manager should also work.
Windows Explorer, on the rare time I used that, did not work [for correctly ordered drag&drop into mpv.net].
https://forum.doom9.org/showthread.php?p=1972286#post1972286)

Any possibility to implement "sequence" file open option from Virtualdub? The one that can open a sequence of files like "file00, file01" by selecting only first file from the sequence? This will solve all problems with "ordered drag&drop", because user could name them in correct order in advance.

Elstar,

• Create a text or bat file that renames the files so that they are recognized in the order you want. ---  You can make a long list of command lines using Excel.  I've attached a txt command line file.  Just search and replace the text 'file' with whatever filename you need.  Then you can either copy all the file content into a command window that is in the directory with just the media files you want to rename.
• Then run a copy command to append them together into one .mp4 file.

This is how much work it takes to fix your file name order problem and create a media file.

- Open a command window [Ctrl] r - type in cmd [Enter]
- In the command window, move to the folder with your list of media files. cd \mymedia\mediafiles01
- Open the number conversion text file from explorer with a double click.
- Select all lines in the text file. [Ctrl] a
- Copy it to your cache [Ctrl] c
- Click in the command window that is in the directory.
- [Ctrl] v to paste - The command window will rename all your media files in about 1 minute.
- Type in the command window: copy /b *.ts newmedia1.mp4

You will probably then need to remux the newmedia1.mp4 using copy with AVIdemux and the result .mp4 should play correctly for browser playback.

I know that seems tricky.  It takes me all of about 30 seconds of command clicks and waiting a few minutes for everything to process.  It is unlikely that you will find a mouse click GUI app that can do this faster.

Hope that helps.

I know how to concatenate, I wonder, if it will work correctly when doing it with non-TS source, especially, with B-frames...

Quote from: Elstar` on January 07, 2023, 10:33:41 PMI wonder, if it will work correctly when doing it with non-TS source, especially, with B-frames...

Even with MPEG-TS, this is not going to work unless fragments belonged to a single contiguous file.

Quote from: timaavid on January 02, 2021, 07:07:14 AMI am having an issue where I am trying to append many files also (from security camera footage).
My problem is, when I try to drag my files into a new session (28 x 4MB files); they don't get into Avidemux in order.
:(

Never had an issue in Win 7 (AVD 2.8.2). Just two rules to abide by.

1.. Files must be in the correct order in the directory. How they're named, or selected is not important.
2.. Most important. You must click and drag from the first file. If you have 8 files and click and drag from file 5, the order they appear in AVD would be 5. 6. 7. 8. 1. 2. 3. 4.

Sark