Request: Video frame-by-frame replacement filter

[Who]

I have a suggestion for a filter:  Let's say you have your main project video which is loaded normally, but it has a short spot where you want to replace just a few video frames with the same number of frames from a completely different video (or the same amount of time).  So what I would like to see is a filter very similar to the Add Logo filter, but instead of specifying a png or jpg static image you could actually specify a completely different video clip.  For whatever number of frames or amount of time is specified, the filter would do a 1:1 replacement, taking a frame from the second file and using the video image from that instead of the image in the original file, then moving on to the next frame (in both files) until the time or frame count runs out.  That way, if you have a video that is mostly good but has one bad spot, but you have another way to obtain just the bad spot (with or without audio) or if you have some other type of replacement video, it would do the frame-by-frame replacement.  Obviously the video in the two files would need to have the same dimensions, but the audio format (or lack of audio) in the second file should not matter.

If it's not too much trouble I hope you'll consider this; it would be very useful to me and possibly to many other users.  Please note I am not wanting something that only works keyframe to keyframe - I would like something that can replace any number of random frames in any position.  I have not given any consideration as to what should happen if the end of file 2 is reached before the time or frame count runs out (maybe loop back to the beginning of file 2) but that's not something I'd ever do except by accident.  Also, as with Add Logo it would be nice if there was a way to do fade in and fade out but that's not essential (speaking of which, in Add Logo would you maybe consider making the fade in and fade out separate settings, so you can do one but not the other?).  Thank you for your consideration.

[eumagga0x2a]

You can achieve this without any modifications to Avidemux by exporting the audio track, appending the video which should be used as overlay, deleting the part of the first video which should be replaced, copying the exact duration from the appended video to clipboard, navigating to the last frame before the deleted part, pasting the copied part and adding the exported audio as an external audio track.

[Who]

You can achieve this without any modifications to Avidemux by exporting the audio track, appending the video which should be used as overlay, deleting the part of the first video which should be replaced, copying the exact duration from the appended video to clipboard, navigating to the last frame before the deleted part, pasting the copied part and adding the exported audio as an external audio track.

I'm sorry, maybe that made sense to you when you wrote it but it did not make a bit of sense to me - I just can't follow what you are doing here.  But in any case, it sure seems like having a built-in filter to do this would make it a lot easier.

[eumagga0x2a]

I'll try to explain better as I doubt that creating a video overlay can be done within a video filter, at least it would require duplicating a big part of Avidemux functionality in such a video filter, especially all timing and frame order sanity checks normally done by the editor (even if you don't mind that the overlay plays at a wrong speed, you would for sure expect that the frame order is correct) while directly using stuff from other parts like demuxers and video decoders – I don't think this fits into Avidemux architecture.

Now back to the way to accomplish the task with existing tools.

Let's assume you are going to replace a part of video ("video 1") starting at 00:10:00.200 and ending at 00:10:20.320 (duration: 20 seconds and 120 milliseconds) with a piece of another video ("video 2"). Let us also assume that the other video has the same dimensions and the same frame rate (else it needs to be filtered and re-encoded to match these requirements first).

First of all, load video 1 and select the desired video codec.

If the other video has a matching audio track which you can use, the operation degrades to a straightforward copy and paste task, else the next step is to export ("Audio" --> "Save Audio") the audio track to a file.

The third step is to append video 2 to video 1 ("Open" --> "Append" or drop video 2 onto Avidmeux window).

Then seek to time 00:10:00.200 (Ctrl+T), set the marker A at this location (with alternative keyboard shortcuts enabled, this amounts to pressing the key "I" like in "in").

Then seek to 10:20.320 and set the marker B (with alternative keyboard shortcuts enabled, press "O" like in "out").

Press "Delete" ("Backspace" on macOS) to delete this part of video.

Now navigate to the frame in appended video 2 which should be the first frame of the patch / overlay. Set marker A here.

Navigate to a frame 20s 120ms later and set marker B, check that the selection duration is shown really as 00:00:20.120, else set marker B to a different frame.

Copy the selection to clipboard (Ctrl+C).

Go to time 00:10:00.200 and paste (Ctrl+V) the content of the clipboard. Verify that the beginning and the end are at the right spot, else undo the paste operation (Ctrl+Z), try the previous (or the next) frame, paste again.

Add an external audio track ("Audio" --> "Select Track" --> ".... Add audio track" for Track 1) if you can't use audio from video 2 for the duration of the patch.

Reset markers ("R" with alternative keyboard shortcuts enabled) and set marker B to the last frame of the desired output range (e.g. to the last frame still belonging to video 1).

Save the video.

The result should have the stretch of the video 1 patched over by the corresponding part of video 2 while keeping audio from video 1.

For fade effect at the boundaries, you need to add two instances of the "fade" video filter, one for the beginning and one for the end of the patch.

[Who]

Thank you for your very detailed response, I get it now!  One thing I actually did not realize was that copy and paste works in Avidemux, so just knowing that will come in real handy.  Also I did not know that you could replace the existing audio track with an external one in Avidemux.

I do have to ask, though - if you do an audio replacement like that, will it make a difference if the audio format is different, for example let's say the original video had just stereo audio and your replacement external audio is AAC encoded 5.1 audio - is that sort of replacement possible?  That has nothing to do with my original question, by the way, it's just something that occurred to me while reading your response.

Thanks again for taking the time to do that detailed reply!

[signy13]

BTW I did quite similar thing some time ago (I had two different recordings of the same movie with some TV station announcement at different time periods of both movies so I mixed one movie from two recordings) and I made cuts in GUI, then I saved it as a TinyPy project and then appended and mixed bits of movies together by editing the TinyPy script:

• loading and appending files
  Code Select Expand
  adm.loadVideo("/home/.../file1.mkv")
adm.appendVideo("/home/.../file2.mkv")
• and then combining segments of file 0 (file1.mkv) and file 1 (file2.mkv) by
  Code Select Expand
  adm.clearSegments()
adm.addSegment(0, 127357000, 960000000)
adm.addSegment(1, 1011080000, 300010000)
adm.addSegment(0, 1387357000, 5763420000)
etc.

For me it was easier to edit the script file and run it in Avidemux than doing the whole thing in GUI.

[Who]

BTW I did quite similar thing some time ago (I had two different recordings of the same movie with some TV station announcement at different time periods of both movies so I mixed one movie from two recordings) and I made cuts in GUI, then I saved it as a TinyPy project and then appended and mixed bits of movies together by editing the TinyPy script:

• loading and appending files
  Code Select Expand
  adm.loadVideo("/home/.../file1.mkv")
adm.appendVideo("/home/.../file2.mkv")
• and then combining segments of file 0 (file1.mkv) and file 1 (file2.mkv) by
  Code Select Expand
  adm.clearSegments()
adm.addSegment(0, 127357000, 960000000)
adm.addSegment(1, 1011080000, 300010000)
adm.addSegment(0, 1387357000, 5763420000)
etc.

For me it was easier to edit the script file and run it in Avidemux than doing the whole thing in GUI.

How did you determine the times (second and third numbers)?

[signy13]

How did you determine the times (second and third numbers)?

I am sorry I missed your question... I am afraid it is too late for my answer, so just in case other people needed the same info:

• I open both movies in two instances of Avidemux.
• I cut beginning of both movies so that both of them start at the same time (for example I find a specific frame (picture) before the movie and cut off everything befort shis frame). I save scripts (basically "not cut scripts") of this movies for later use.
• I find exact place I want to cut-off from the first movie and cut it off (making a "hole"). Then I find the same place and cut off everything else. Then I save scripts of this movies - these are "cut scripts" for the first "hole".
• If I need more cuts I re-load movies by using "not cut scripts" and repeat the previous step.

The times you asked for I find in the cut scripts.