I use AVIdemux to join hundreds or thousands of time-lapse images together into a video. On my Windows system, I typically multi-select a set of sequential JPG images from a folder, all having identical resolutions and format, and drag&drop them onto the AVIdemux GUI window. Then after AVIdemux has automatically appended all the images together, I save the resulting video as an MKV file. That's a great feature!
BUT HERE'S THE PROBLEM - if I drop more than several hundred JPG images onto AVIdemux, the program fails badly, and crashes hard. It can even lockup my entire computer, requiring a reboot (didn't now that was possible on Win10).
Typically, my JPG images are 1024x768 and average 100-200KB each. Or sometimes I use 2592x1944 which average about 1.5MB each. In either case, if the set of images I wish to append totals more than several hundred MB, it will always crash AVIdemux.
I have tried many different versions of AVIdemux, going way back to versions from years ago, both 32bit and 64bit versions, on both WinXP and Win10 systems, and in all cases this same bug exists. I'm currently using AVIdemux v2.7 VC++ 64 bits on Win10.
I have read others suggest only appending a few hundred images at a time, and then later use AVIdemux to append all the resulting MKV files. That does work, but is terribly tedious if I'm trying to append 15K images together, which I'd like to do. After all 15K images (frames) only represents 10 minutes of video at 25 frames/sec.
This bug wouldn't be as bad if it simply refused to append more frames than AVIdemux could handle, and reported how many frames were successfully appended. As is, I don't know exactly what the limits are. I drop a set of images onto AVIdemux and cross my fingers. Sometimes it works, other times it crashes.
It seems this should be a simple matter of using proper memory management. Are some arrays of fixed length being over filled (use linked lists instead)? Is the internal storage area needing to be contiguous in RAM, and can't work with virtual memory? In any case, assessing illegal "out of bounds" memory and crashing my system is not acceptable. This is a simple bug to reproduce, so can't it be fixed?
I apologize if this same bug has been report many other times. I did a quick search and didn't find a similar report. But I'm new to this site, so I may have missed something.
If you know a better work around then what was mentioned above, please comment.
And if the software developers would like me to beta test a fix, please contact me.
Quote from: JulianDude on November 05, 2019, 11:32:03 AM
I use AVIdemux to join hundreds or thousands of time-lapse images together into a video. On my Windows system, I typically multi-select a set of sequential JPG images from a folder, all having identical resolutions and format, and drag&drop them onto the AVIdemux GUI window.
Don't do that. Make sure the images are sequntially named (pic-00000.jpg, pic-00001.jpg etc.) and open just the first one. All following images will be appended automatically in the demuxer, not requiring creation of thousands of decoder instances, image buffers etc.
QuoteBUT HERE'S THE PROBLEM - if I drop more than several hundred JPG images onto AVIdemux, the program fails badly, and crashes hard. It can even lockup my entire computer, requiring a reboot (didn't now that was possible on Win10).
We almost everywhere assume that memory allocation always succeeds, this is definitely worth a fix, but if the OS locks up on memory exhaustion by a user space application, this doesn't look good for the OS (it should just kill the app).
QuoteI'm currently using AVIdemux v2.7 VC++ 64 bits on Win10.
I assume it is 2.7.5 VC++. Only the latest nightly (nightlies provide most of the time the best experience) or the most recent release (2.7.5 for Windows) are supported here.
I created a 10-JPG test set and batchfile to test this:
https://dl.dropbox.com/s/kakno6i22x9r12a/adm_filename_sequence_test.zip
The 10 JPGs have had their canvas size standardised to 1500x1000. Since these images were already cropped without regard to AR, I specified the same canvas size (in XnView). This means the images will have a variable black-area around them (looks funny when playing). The same frame size for all JPGs is important for this test.
Originally, I used a single test JPG. Then I used my batchfile to make up to 10,000 sequentially-numbered versions of it. But then I decided it would be better to use a set of different images so that, as you single-stepped though the output frames, you could see that ADM was indeed using different images. So I switched to a 10-image sequence, test0.jpg-test9.jpg. The average filesize of each of these images is approx. 370KB. You can adjust the batchfile to make a 1-10,000 sequential filename sequence in the test_directory sub-directory. So, if you were to set the number of images to 15, and have this batchfile and the 10 JPGs in a C:\Test directory, it will create an output sequence of:
C:\Test\test_directory\Sequence_0000.jpg
C:\Test\test_directory\Sequence_0001.jpg
C:\Test\test_directory\Sequence_0002.jpg
C:\Test\test_directory\Sequence_0003.jpg
C:\Test\test_directory\Sequence_0004.jpg
C:\Test\test_directory\Sequence_0005.jpg
C:\Test\test_directory\Sequence_0006.jpg
C:\Test\test_directory\Sequence_0007.jpg
C:\Test\test_directory\Sequence_0008.jpg
C:\Test\test_directory\Sequence_0009.jpg
C:\Test\test_directory\Sequence_0010.jpg
C:\Test\test_directory\Sequence_0011.jpg
C:\Test\test_directory\Sequence_0012.jpg
C:\Test\test_directory\Sequence_0013.jpg
C:\Test\test_directory\Sequence_0014.jpg
With the 5 images in Sequence_0010-14.jpg being the same as in Sequence_0000-4.jpg
Create_numbered_copies_of_jpg.bat@ECHO OFF
REM From a 10-image input sequence of sequential filenames,
REM create an output sequence of 1-10000 sequential filenames.
SETLOCAL EnableDelayedExpansion
SET num_of_files=100
IF NOT EXIST "test_directory" MD "test_directory"
FOR /L %%i IN (0,10,%num_of_files%) DO CALL :docopy %%i
FOR /L %%i IN (1,10,%num_of_files%) DO CALL :docopy %%i
FOR /L %%i IN (2,10,%num_of_files%) DO CALL :docopy %%i
FOR /L %%i IN (3,10,%num_of_files%) DO CALL :docopy %%i
FOR /L %%i IN (4,10,%num_of_files%) DO CALL :docopy %%i
FOR /L %%i IN (5,10,%num_of_files%) DO CALL :docopy %%i
FOR /L %%i IN (6,10,%num_of_files%) DO CALL :docopy %%i
FOR /L %%i IN (7,10,%num_of_files%) DO CALL :docopy %%i
FOR /L %%i IN (8,10,%num_of_files%) DO CALL :docopy %%i
FOR /L %%i IN (9,10,%num_of_files%) DO CALL :docopy %%i
GOTO :eof
:docopy
REM Since files are being numbered from zero, skip creating the %num_of_files% filename.
IF %~1==%num_of_files% EXIT /B
SET FNum=000%~1
REM Use the mechanism below to control the number of digits used in the input and output filenames.
REM So a %FNum% of 00100 becomes: %inFN% of 0 and %outFN%" of 0100.
SET InFN=%FNum:~-1%
SET OutFN=%FNum:~-4%
REM For example, with a %num_of_files% of 15, "4" is used twice in filenames:
REM COPY test4.jpg test_directory\Sequence_0004.jpg
REM COPY test4.jpg test_directory\Sequence_0014.jpg
COPY test%InFN%.jpg test_directory\Sequence_%OutFN%.jpgI've tested ADM VC++ 2.7.5 191104 & 2.7.5 191020 (Win64), both with up to a 10,000-image sequence, and they created a 3GB MP4 without issue.
Here's my procedure in ADM:
- Open ADM. Video Output: Mpeg AVC (x264). Configure | Preset "ultrafast" (I wanted to speed up the processing). Audio Output: Copy. Output Format: MP4 Muxer.
- File | Open first image in sequence. (Here: C:\Test\test_directory\Sequence_0000.jpg). It takes a little longer for this first image to appear in ADM with a 10,000-file sequence.
- File | Save.
Note: I didn't test this using drag&drop input selection.
According to MediaInfo, the new 10,000-frame MP4 was 3GB, 280sec duration, 25fps.
How do you specify a different fps, if needed?
Note: by changing
SET OutFN=%FNum:~-4% to
=%FNum:~-5% you could produce up to a 100,000 filename sequence.
Dan.
If ADM can handle 10000 sequentually numbered jpg images, then ADM must be able to look at a folder and determine that list of JPGs filenames. So why then can't ADM also build the same list of filenames via Drag&Drop? Why in the sequentially named case it works flawlessly (as others indicate), but in the D&D case it can easily crash? Seems most of the same file loading and memory management code should be used in both cases. The only difference is the method of indicating the set of images.
eumagga0x2a's suggestion that sequentially numbered images be used instead of Drag&Drop is a pain for me since my jpgs are time lapse frames that are numbered with a time stamp. I would need to go thru another step to rename all my images sequentially. Since Drag&Drop is a standard method to supply a set of filesnames to a program, shouldn't that method be debugged, since it already works to a limited extent. The only issue with d&d is figuring out the file order. ADM could either simply use the order as provided by d&d, or internally build the list of filenames and then sort them by name. But doing the sort is problematic, since ADM wouldn't know what the user really desires (ascending/decending, by name or date, etc). An ascending filename sort is probably best since it'd match the sequentially named file method. Though just getting a raw list of names from d&d may be more powerful (assuming users know how to multiple select files correctly for d&d to do as they desire).
dosdan wondered how you set the video frame rate when appending a set of images together. I don't see a way in the existing GUI (am I missing something?). Seems it's hard coded to be 25 fps. I typically save MKV's from ADM (with no futher loss in quality, since it basically saves as a Motion JPEG file, without recoding). And then I use MKVmerge to set my desired frame rate, add soundtrack, etc. Ultimately, I may edit or recode the video file to save space, but I do that outside of AVIdemux.
Now back to the topic - can drag&drop in ADM be fixed?
Quote from: JulianDude on November 06, 2019, 08:21:43 AM
The only issue with d&d is figuring out the file order.
I think it's more than that. There's a limit in the length of the string of pathnames you can drop onto a program. I don't think drag-and-drop is designed for thousands of filenames.
Here's a batchfile to test this limit.
Drag_and_Drop_test.bat@ECHO OFF
SETLOCAL EnableDelayedExpansion
IF EXIST logfile.txt DEL logfile.txt
ECHO %* >> logfile.txt
ECHO. >> logfile.txt
:files_loop
FOR %%F IN ("%~1") DO (
ECHO %%F >> logfile.txt
SHIFT
)
IF "%~1" NEQ "" GOTO files_loop
IF EXIST logfile.txt START /I logfile.txt
Create a shortcut on your desktop to this batchfile and then drag a Ctrl-A selected group of JPG files (not a directory name containing these files) onto the shortcut icon. After running, it will open Notepad, if it's associated with .TXT files, and show at the top a big contiguous line of what was dropped on to it using the %* (all parameters) variable.
Then it will show each filename, one per line, without its path and with just one set of inverted commas ("%~1") around it since Windows only includes "" in the drag_and_drop list if the file has a space in its pathname. So by getting the filenames stripped of their "", if present, and then putting "" back on all filenames, it makes batch operations, with a mixture of filenames, more reliable. This list of filenames will show you if the drag_and_drop operation is in
the correct sorted order.
(https://dl.dropbox.com/s/vva81sw9g7euf8a/logfile.txt.jpg)
When you drag_and_drop, the full pathname is included for each file. I can drop 160 files of pathname format "C:\Filename_test\test_directory\Sequence_nnnn.jpg". Allowing a space between each pathname, that's just over 8,000 characters. But If I try and drop 170 files, it fails. If you are operating in say a first-level directory, rather than the 2nd-level one used in this example, and the directory name and filesnames are short, you will be able to increase the number of pathnames you could drag-and-drop before hitting a 8000-character limit.
While this is a batchfile, I think programs would also be faced with this length limit.
Dan.
Quote from: JulianDude on November 06, 2019, 08:00:10 AM
If ADM can handle 10000 sequentually numbered jpg images, then ADM must be able to look at a folder and determine that list of JPGs filenames. So why then can't ADM also build the same list of filenames via Drag&Drop? Why in the sequentially named case it works flawlessly (as others indicate), but in the D&D case it can easily crash?
Opening the first image in a sequence so that the demuxer creates a "video" with frames actually being individual files (the editor sees just a single video) and appending N individual files in the editor (the editor sees N very short videos, each requiring its own decoder instance and image buffer, and arranges them sequentially in the timeline) invoke very different paths in the application and serve different purposes.
QuoteSeems most of the same file loading and memory management code should be used in both cases.
No, not at all.
QuoteThe only difference is the method of indicating the set of images.
This statement is not correct.
QuoteNow back to the topic - can drag&drop in ADM be fixed?
The issue is not drag&drop, the issue is usage of new instead of malloc() to allocate buffers. The latter would allow to catch memory exhaustion and either cause an assert (an orderly crash which allows Avidemux to save state of editing) or at best a failure to load more files. This is definitely worth a fix (or rather a bunch of fixes all over the code tree).
QuoteThe only issue with d&d is figuring out the file order. ADM could either simply use the order as provided by d&d
It does exactly that.
Quoteor internally build the list of filenames and then sort them by name. But doing the sort is problematic, since ADM wouldn't know what the user really desires (ascending/decending, by name or date, etc).
Or just the order the files were selected. For this reason d&d is not going to change.
Quoteeumagga0x2a's suggestion that sequentially numbered images be used instead of Drag&Drop is a pain for me since my jpgs are time lapse frames that are numbered with a time stamp. I would need to go thru another step to rename all my images sequentially.
I acknowledge the inconvenience, but this is unavoidable. It should not be too difficult to write a script automating the step when you exactly know which pattern the original filenames follow. I would not want to trust demuxer with this task.
Quote from: JulianDude on November 06, 2019, 09:06:33 AM
dosdan wondered how you set the video frame rate when appending a set of images together. I don't see a way in the existing GUI (am I missing something?). Seems it's hard coded to be 25 fps.
Yes, this is how the demuxer (https://github.com/mean00/avidemux2/tree/master/avidemux_plugins/ADM_demuxers/Pictures) handles sequentially named pictures.
QuoteI typically save MKV's from ADM (with no futher loss in quality, since it basically saves as a Motion JPEG file, without recoding). And then I use MKVmerge to set my desired frame rate, add soundtrack, etc. Ultimately, I may edit or recode the video file to save space, but I do that outside of AVIdemux.
If you re-encode in Avidemux, you can set the desired frame rate with the changeFps video filter. You can't do it in copy mode, however (this might be a valid RFE).
Quote from: JulianDude on November 06, 2019, 08:21:43 AM
eumagga0x2a's suggestion that sequentially numbered images be used instead of Drag&Drop is a pain for me since my jpgs are time lapse frames that are numbered with a time stamp.
I'd expect time-stamped filenames to sort OK. Can you provide a few samples of the filenames from a long-duration sequence.
Or you could provide a few versions of the full filelist of all the JPEG files in a sequence. In the directory they are situated in, run:
DIR *.JPG /B > filelist_b.txt DIR *.JPG /B /ON > filelist_on.txt "/ON", order by filename, produces a slightly different order compared to a straight
DIR /B. For example, filenames beginning with an underscore ("_") are sorted differently. Not sure yet which could suit your situation better.
DIR *.JPG /OD /TC > filelist_tc.txtDIR *.JPG /OD /TW > filelist_tw.txtZip these 4 text files up and either attach the zipfile here, or provide a link to it on a file-sharing site e.g. Dropbox.
Rather than expecting ADM to be able to handle every variant of sequential naming around, I think it's reasonable seeing whether these names can be completely replaced with a standard sequentially-numbered version. The way this would operate is say you had 10,000 files ready to combine into 1 clip. You would move or copy them all into a work directory. Then click on a batchfile to process everything (or, alternatively, every JPEG) within that directory. It would have the smarts to sort the list correctly and replace all the filenames with a standard sequential numbering. Then the batchfile would run ADM, where you would use File | Open to select the first renamed file in this directory (if always using the same work directory each time, set it as the default using Ctrl-Alt-D) and let ADM work its magic. And, if sequential loading works from the CLI, (not sure about this), you could have this first-file-in-a-sequence loading, (after renaming, it will always have the same name), done automatically.
Bulk Rename Utility (GUI) and Bulk Rename Command Line are fairly powerful tools. Once set up, this should be fairly straightforward to perform. https://www.bulkrenameutility.co.uk/
Dan.
BTW, once you're dealing with a standard numerical-naming sequence, you can try other programs, such as FFMPEG. Here it handled an up-to-10,000-JPG sequence, (I tested using the full 10,000), looking for Sequence_nnnn.jpg in the test_directory sub-directory. With FFMPEG you can specify both the input-reading and output-writing framerates. If you just specify the input framerate ("-framerate nn"), it uses it for both. Optional switches below are "-hide-banner" to reduce the screen clutter, (you could also add "-loglevel error" which will hide standard messages and show just the errors), and "-y" to overwrite an existing output file without prompting. I've told it to use the x264 library for the video codec, and to use the "ultrafast" preset from that library. And, by using the .MKV file extension in the output filename, I've told it to use this muxer for the container format.
ffmpeg -framerate 24 -i test_directory\Sequence_%04d.jpg -hide_banner -y -c:v libx264 -preset ultrafast output.mkv
Since you are using MJPEGs and don't want to re-compress them you could use just:
ffmpeg -framerate 24 -i test_directory\Sequence_%04d.jpg -hide_banner -y -c:v copy output.mkv
Once, you place this in a batchfile it's quite easy to reuse. Either have FFMPEG.EXE in the current directory, have the directory where it's located specified in your Window PATH environmental variable list e.g. c:\utils, or fully specify it's location in the batchfile e.g. c:\utils\ffmpeg
https://trac.ffmpeg.org/wiki/Slideshow
https://stackoverflow.com/questions/24961127/how-to-create-a-video-from-images-with-ffmpeg
https://trac.ffmpeg.org/wiki/Encode/H.264
Dan.
A real albeit cursory check for usage of new instead of malloc to allocate buffers revealed no significant findspots. Even in the creation of EditorCache in ADM_EditorSegment::addReferenceVideo, which might be a major consumer of memory besides creation of decoder instances incl. their huge AVCodecContext structures, the cache constructor ultimately calls our malloc wrapper.
While it look as if it would allow Avidemux to crash in an orderly way by assert, in reality, it might be too late to invoke the crash handler when a malloc call fails.
Quote from: dosdan on November 07, 2019, 12:10:45 AM
BTW, once you're dealing with a standard numerical-naming sequence, you can try other programs, such as FFMPEG. Here it handled an up-to-10,000-JPG sequence, (I tested using the full 10,000),
FYI, the "Pictures" demuxer in Avidemux sets 99999 as the upper limit.
For dosdan - here's the file naming formats I've seen used with several different time-lapse capture programs:
I frequently capture time-lapse digital images using a microscope camera. The software I prefer only supports one file naming convention, which is based on the current timestamp. It has the following format:
Example "20191107-154534-214.jpg"
The left 8 digits represents the date = YYYYMMDD
The middle 6 digits represent a time of day = hhmmss, using a 24 hour clock.
The right 3 digits are additional milliseconds.
All fields are padded with zeros, so the full timestamp is always 19 characters long, not including the filename extension (.jpg, .png, .tif, or .bmp).
So the above example represents 2019 Nov 7, at 15h45m34.214s
Some other time-lapse capture software I've tried supports a choice of file naming formats (without any millisecond field included).
YYMMDDhhmmss
YYYYMMDDhhmmss
YY-MM-DD-hh-mm-ss
YYYY-MM-DD-hh-mm-ss
Or some programs allow captures with the frames sequentially numbered, starting at some giving number, and increasing by 1 each frame. These are typically padded with zeros to 4 digits, eg. "0123.jpg" This choice would be ideal for ADM and FFmpeg, but unfortunately this naming choice isn't available in my favorite time-lapse software, and it does not provide important time-stamp information in the filename.
dosdan, your test of drag&drop onto a short-cut link may not be a good test for how drag&drop should work when dropping a set of images onto the ADM application window. In your test, all the file paths are being passed as a single command line argument to Notepad (correct?). But in a windows program there are various API's that can be used instead. You can get a message when something is dropped, and then there are functions that allow you to step thru the list of filenames. It would make no sense that that would be limited to 8000 characters total. Certainly Windows has no such limit when dragging and dropping large sets of files between directories, and I've never heard of such a limitation. Have you?
Not to suggest a complete re-write of ADM, but wouldn't it be possible to write some wrapper code around a drag&drop event? That code could create a list of arbitrarily named images (or videos?) to append together, and then handle that set in the same way as the current code creates a video based on sequentially numbering images? But if the current code that processes a sequence of numbered files does not analyze the directory first, and build such a list, then that could be added. In that way, whether images are being appended via drag&drop onto the ADM window or via sequentially named files, nearly all the processing could be handled in exactly the same way, with the exception of a small amount of front-end wrapper code that makes the two cases look identical. Creating these wrappers may be easier than debugging the existing drag&drop code (since it was mentioned it uses different memory management - malloc vs new)? On the other hand, maybe you can find the reason that drag&drop can currently crash ADM, and continue using the existing code?
BTW, I am amazed at the activity on this site. Keep up the good work. It is appreciated!
Drag&drop is identical with performing the append action in the "File" menu, this is so by design and is not going to change without a very valid reason. When a file is passed to the code which probes its content, it is too late to decide that we actually wanted to treat it as a frame of a single Motion JPEG video, auto-appended in the "Pictures" demuxer.
Limiting auto-append to sequentially named image files allows to pass a single bounds-checked file name to the demuxer which then can deduce all file names by incrementing a counter. From the point of view of Avidemux, it has loaded just a single file, all the complexity of handling individual images encapsulated and entirely concealed by the demuxer.
However, it doesn't mean that we should waste memory in scenarios with images appended in the editor, [editor] Reduce editor cache size for ultra-short ref videos like images to save memory (https://github.com/mean00/avidemux2/commit/8a144fb4586c4b5dcf33a65652ea3bd5dda3856e) should substantially reduce memory usage here (the cosmetical part of the changeset unfortunately forgotten in the commit message).
Additionally, a bug in editor (seek to the end broken in a video consisting of individial images appended in the editor) was found and fixed (https://github.com/mean00/avidemux2/commit/b29132f09f695d7299d12b267d1a890475dba59c), thank you for your input.
Quote from: JulianDude on November 07, 2019, 12:24:23 PM
Example "20191107-154534-214.jpg"
The left 8 digits represents the date = YYYYMMDD
The middle 6 digits represent a time of day = hhmmss, using a 24 hour clock.
The right 3 digits are additional milliseconds.
All fields are padded with zeros, so the full timestamp is always 19 characters long, not including the filename extension (.jpg, .png, .tif, or .bmp).
So the above example represents 2019 Nov 7, at 15h45m34.214s
There's no problem with this sorting correctly under Windows, by name, on an NTFS-formatted drive. Quoting from https://ss64.com/nt/dir.html
Sorting
The default Sort Order, if you don't specify anything with /O, on an NTFS drive will be in sort-of-alphabetical order or on a FAT USB thumb drive, then the order will be based on the order in which files were created and deleted and the lengths of their names.So, now we come to the problem of the image names not being sequential. Using FFMPEG, there is no need to copy or sequentially rename/renumber these files, as FFMPEG can accept a list of filenames. So the first step in this exercise is to produce a set of files that have their frames showing sequential numbers. This will be used to make a set of test JPGS so you can confirm that the correct sort order is used and that no frames are missing: each image shows one frame-number.
(https://dl.dropbox.com/s/dceg5q9poimz6rc/frame_number.jpg)
Note: I reduced the dimensions of this screen capture to save display space in the forum, so it looks more jagged here than it really is.
If distributed as an un-compressed AVI, this clip would be enormous. But as an MP4 it's only 2.5MB. Here is a 1000-frame MP4, with its frames numbered from 0000-0999, available for download:
https://dl.dropbox.com/s/q6s9v1ea4cgcr6g/1000_sequence.mp4
While you could use this file to create 1,000 separate image JPGs, which is what I did for much of setting this up, I think 250 frames, which is 10s at 25fps, will be a better choice for when we later change the duration of the sequence, as it makes the mental arithmetic easier. I'd suggest creating a
C:\Test directory, copying FFPMEG.EXE to this directory if it's not in your PATH list and, from there, using the
md extracted command to make a
C:\Test\extracted sub-directory for all the test images. All our actions will performed from the directory a level up from
extracted.
To extract the first 250 images from 1000_sequence.mp4 use:
ffmpeg -i 1000_sequence.mp4 -qscale:v 2 -frames:v 250 extracted\output_%03d.jpgThis will produce this sequence of image files:
20191107-154001-214.jpg to
20191107-154250-214.jpg in the extracted sub-directory. Each is 1680x1050. It doesn't matter here that this is not correct HHMMSS naming. The important thing for this test is that the filename ending ("-214") is not varying sequentially. This will confuse programs that look at the end of the filename to determine the suitability for sequential loading.
-qscale:v 2 controls the quality of the the JPGs. "2" is fine.
-frames:v 250 limits the extraction to the first 250 frames. Leave it out if you want the full 1000 frames.
Now, FFMPEG is flexible enough to auto-load this sequence:
ffmpeg -i extracted\20191107-154%03d-214.jpg -c:v copy test.mkvBut this wouldn't work if the naming format was actually HHMMSS. For that we need to read from a list of name-ordered filenames. This list can also be used to alter the duration each image will be displayed.
Create_list.bat@echo off
SETLOCAL EnableDelayedExpansion
if exist mylist.txt del mylist.txt
for %%I in ("extracted\*.jpg") do (echo file '%%~fI' >> mylist.txt)
start /i mylist.txt
The start of mylist.txt looks like:
file 'C:\Test\extracted\20191107-154001-214.jpg'
file 'C:\Test\extracted\20191107-154002-214.jpg'
file 'C:\Test\extracted\20191107-154003-214.jpg' The concat demuxer in FFMPEG wants each entry in this list to start with "file".
'%%~fI' produces the full pathname. If I had used just
"%%I" it would have produced the relative path of
file 'extracted\20191107-154001-214.jpg' While that works, it's probably safer to use the full pathname.
To assemble the MKV (which will be in C:\Test) use:
ffmpeg -f concat -safe 0 -i mylist.txt -hide_banner -y -c:v copy output.mkv-f concat forces FFMPEG to use Concat demuxer, which can read from a filelist, for input.
-safe 0, before specifying the name of the input list, stops the demuxer getting upset over the presence of hyphens in the filename (valid under Windows).
Output.mkv is 10s, 25fps and 17.8MB. If your media player has a blank screen, (probably won't be an issue with MJPEG), include the -
pix_fmt yuv420p option.
In actual use, you'd create the list and then use it to concatenate the file and wrap it up a MKV, using just one batchfile, rather than the two, separate operations I've showing here. If needed, you could put a
pause command between these two sections of the batchfile, allowing you to delete any untanted lines in mylist.txt in Notepad, before saving the changes, closing Notepad and pressing Enter to continue on to the concatenation stage.
Say the time-lapse action is too fast or too slow in the output file. The file list can include 2-lines per image, with the 2nd line specifying the image duration in secs. To add this feature, change the main line in Create_list.bat to:
for %%I in ("extracted\*.jpg") do (echo file '%%~fI' >> mylist.txt & echo duration 0.04 >> mylist.txt)The start of the list then looks like
file 'C:\Test\extracted\20191107-154001-214.jpg'
duration 0.04
file 'C:\Test\extracted\20191107-154002-214.jpg'
duration 0.04 An image duration of 0.04s is the duration of one frame at 25fps (40ms). Increasing this will start replicating frames, slowing down playback, while reducing it will drop some image frames, speeding up playback. You can alter the image duration value in the batch line above to suit yourself.
An example: setting the duration to 0.2 changes the overall duration of the output clip from 10s -> almost 50s. And MediaInfo shows a change in framerate from 25fps -> 5fps. It also changes the number of frames in the clip from 250 -> 1245. Due to a minor bug in the Concat demuxer, it drops the last image (here, 5 frames) if you specify duration. The work-around for this is mentioned in https://trac.ffmpeg.org/wiki/Slideshow
(Due to a quirk, the last image has to be specified twice - the 2nd time without any duration directive)But the small difference in overall duration is usually not worth worrying about.
Despite the reported increase in frames with this playback slow-down, (I checked this out in VirtualDub), the output file is still 17.8MB.
Dan.
I want to thank everyone who has helped find a workable solution to my time-lapse image appending issue. I particularly appreciate all the help dosdan and eumagga0x2a provided. Since my image filenames are based on a timestamp, to simplify my workflow, I have chosen not to rename them as a sequence of numbered frames. Based on using ffmpeg.exe, I created a slightly modified batch files to what dosdan suggested. I've attached a copy. I'm also accessing ffmpeg via another small batch file so I didn't need to add ffmpeg's path to an environment path variable.
I did notice that the image filenames can't contain any single quote marks. I tried changing dosdan's suggested code to use double quote marks around the paths in the created list file, so the names could then contain single quotes. But apparently ffmpeg didn't like that. Maybe there is a solution, but I gave up.
Also, my batch file needs to be placed in same directory as the image files. I experimented with dropping a folder of images onto my batch file icon, which I'd prefer. And I got that to work using "FOR /R", but had to eliminate the directory file sort feature (which I feel is necessary), and I didn't know how to limit that /R solution to avoid potential sub-folders. I'm not a batch file guru like some of you guys.
BTW, I am happy with this solution, but will look forward to any image drag&drop improvements that may be made to ADM. Thx again.
JulianDude, here's part of your batchfile:
call ffmpeg -f concat -safe 0 -i _appendList.txt -hide_banner -y -c:v copy _Output.mkv
del _appendList.txt
REM -------------------------------------------------------------------------------
REM NOTE: This batch runs ffmpeg via another small batch file, called ffmpeg.bat. The ffmpeg.bat file contains a REM path to where you actually store ffmpeg.exe. ffmpeg.bat should be placed in a folder that is already on Windows REM search path. That way, ffmpeg.exe can be found without needing to include a full path to it in a cmdline or batch REM file, nor needing to add another path to Window's environment variables.
REM The good place to put ffmpeg.bat is at:
REM "%USERPROFILE%\AppData\Local\Microsoft\WindowsApps"
REM ------- Here is an example ffmpeg.bat (don't include the "REM's"). Edit for your ffmpeg path ------
REM @echo off
REM "C:\PortableEXEs\_Portable EXEs\ffmpeg-20191108-e700038-win64-static\bin\ffmpeg.exe" %*
I think this is a messy solution: calling a batchfile named FFMPEG.BAT just because FFMPEG.EXE is not in the current directory. You now need to have FFMPEG.BAT in either your current directory or in a pathed directory.
I also don't like the idea of including the version name of a commonly-used (for the video fraternity, at least) utility. It's an OK practice if you have a program that uninstalls itself when upgrading, as it will remove the old program directory. Otherwise you end up with multiple directories for multiple upgrades of a program. Utility programs usually are stand-alone programs without installation programs, so it's a straight copy-to-install.
Instead, I'd suggest having a pathed directory e.g. C:\utils or C:\utilities into which you put all these types of program. You can also put all your batchfiles here. (If you text editor remembers your last used directory or keeps reloading one or more text files you have recently been working on in previous sessions and haven't yet closed, e.g. Notepad++, every time you go to create or edit a batchfile it will be looking in the correct place. (Alternatively, have another pathed directory, C:\batchfiles.)
In my case, since I regularly build Win64 versions of FFMPEG (https://github.com/m-ab-s/media-autobuild_suite), I have both C:\utils (I don't keep FFMPEG.EXE & FFPROBE.EXE here) and C:\Mingw\media-autobuild_suite-master\local64\bin-video (which is where the compiled video programs end up) included in my PATH list.
I'll look into single inverted commas and FFMPEG. Perhaps it's just a Concat demux input restriction?
Now, consider portable operation. You could carry both ffmpeg.exe & _Append all Images.bat around on a USB stick. Let's call this USB's drive "X:". We're currently in a sub-directory on C:, say "C:\test directory\extracted". From here we issue:
"X: _Append all Images" produced by typing x:_ or x:_a and then pressing the Tab key to find a filename match on X:, pressing Tab again if it was an incorrect match.
This would have an problem if FFMPEG.EXE isn't in the PATH list. However, assuming that FFMPEG.EXE & _Append all Images.bat will be found in the same location you can use a variant of the %0 parameter. This will only operate inside a batchfile.
Trying this test batchfile while we are on C: and the batchfile is on X:
Find starting loc.bat
@echo off
cls
echo Batchfile name and loc used on command line: %0
echo Batchfile drive letter: %~d0
echo Batchfile directory: "%~p0"
echo Batchfile full path: "%~dp0"
echo Full pathname to be used to run FFMPEG: "%~dp0ffmpeg.exe"
Running this from C: with the batchfile on X:
"x:Find starting loc.bat"
Result:
Batchfile name and loc used on command line: "x:Find starting loc.bat"
Batchfile drive letter: X:
Batchfile directory: "\"
Batchfile full path: "X:\"
Full pathname to be used to run FFMPEG: "X:\ffmpeg.exe"
And assuming we were in the 2nd level extract sub-directory (c:\test directory\extracted), with the program and batch files situated in the directory above this (c:\test directory), we can use "..\" (directory above this one) to run the batchfile from where we are.
"..\find starting loc.bat"
Result:
Batchfile name and loc used on command line: "..\Find starting loc.bat"
Batchfile drive letter: C:
Batchfile directory: "\test directory\"
Batchfile full path: "C:\test directory\"
Full pathname to be used to run FFMPEG: "C:\test directory\ffmpeg.exe"
So, if you are prepared to keep ffmpeg.exe and _Append all Images.bat together, and they're not in the PATH list, I suggest changing the start of your FFMPEG line inside your batchfile from
call ffmpeg
to
"%~dp0ffmpeg.exe"
Dan.