Avidemux Forum

Avidemux => Main version 2.6 => Topic started by: Spellbinder on December 25, 2013, 07:14:59 PM

Title: Smart-Copy
Post by: Spellbinder on December 25, 2013, 07:14:59 PM
First of all let me say thank you for your great work! I really like avidemux and use it very often. Mainly to cut commercials out of tv captures.

But there is one feature that I am missing very much: Smart-Copy

I am no developer so I canÃ,´t code myself, but I thought I give it a try from a business analyst point of view what it would take to implement this featureââ,¬Â¦

First of all letÃ,´s define what ââ,¬Å"smart-copyââ,¬Â means:

Copy mode is currently limited to cut only at I-Frames. Only way to cut at non I-Frames at the moment is to completely re-encode the complete video. Since re-encoding always reduces quality this is not good and takes quite a while, too. So to bridge the gap there is ââ,¬Å"smart-copyââ,¬Â. It will encode only the necessary parts at the cut-points and otherwise copy the video. So only a very small part of the video is encoded and the rest stays the same.

So what would it take to implement this?

1.   Identify the parts that need to be encoded. Avidemux already seems to check if cutpoints are I-Frames when in copy mode. So identifying the non I-Frames should be easy?
2.   Specify the exact from-to of the part that needs to be encoded. This would be the non-Iframe cutpoint until the next I-Frame (excluded).
3.   After all parts that need to be encoded have been specified they would need to be encoded. Avidemux can also do this right now as well 
4.   After all the necessary parts have been encoded avidemux would have to append all the parts together with the copy parts of the video in between. Appending is also already possible in avidemux.

Only questions left is what encoder setting to use. In my opinion they donââ,¬â,,¢t have to be perfect. We are talking about a very small fraction of the final video. So I would determine the video codec and the average bitrate and encode it with 2-pass using the same average bitrate. If you really would like to make it ââ,¬Å"fancyââ,¬Â determine the profile and levels (for h264) and limit the encode to that of the video you want to cut, so that they are not changed for the new video. Rest of the settings use ââ,¬Å"best-practicesââ,¬Â. Alternatively you can let the user do all the encoding settings himself when selecting ââ,¬Å"smart-copyââ,¬Â option.

Sounds easy? Or not? Let me know what you think about this concept and if it possible to implement.

Many Thanks!
Title: Re: Smart-Copy
Post by: Papou on December 25, 2013, 08:34:35 PM
User reply:
What you describe is what Avidemux (including 2.5) can already do with MP4 ASP (DivX etc).

What is wished is that it did it for MP4 AVC (H264) as well.

My present wish is that Avidemux warned that a cut is GOING TO be made at a non I frame, "are you sure?" so that the user can correct what he's doing, rather than warn BEFORE ENCODING that at cut is HAS BEEN made at a non I frame, at a time when it's practically impossible to correct what was done.

Regarding MP4 AVC smart copy, it's a pity that
1) instead of re-encoding, it does not seem to be possible to synthesize an I frame to replace the first kept P frame so that the following P frames would refer to them the same way.
2) it is not possible to mark frames as NON DISPLAY (skip) so that (on high speed CPUs anyway), decoding follows its normal course but small GOPs are skipped and only what is wanted is displayed.
Title: Re: Smart-Copy
Post by: AQUAR on December 26, 2013, 12:12:58 PM
Probably the same reason why you can't do partial.
That is - avidemux 2.6 is not in control of pulling out the required frames, as  needed to create a new IDR frame for orphaned frames at the cut points.
Title: Re: Smart-Copy
Post by: Papou on December 26, 2013, 01:58:37 PM
How then does Avidemux get "in control" to pull out the frames needed to reencode an orphaned frame set and to insert the reencoding at the cut point?
Title: Re: Smart-Copy
Post by: AQUAR on December 27, 2013, 05:39:32 AM
@ Papou
Avidemux 2.6 is aimed at being able to deal with AVC video.
The wiki explaines some of the changed approach taken to do that.

The question you pose is way to deep for an enduser to tackle.
Developers might repond here if it is considered value adding to us endusers (but I think not!).
Or update the wiki in the future for the curious geeks amongst us.

Version 2.5.6 handles PTS and DTS itself for frame re-ordering, so it can pull out any frame it needs to create a new IDR frame as reference for the recoded orphaned frames (ie the smartcopy process).
With version 2.6 the frame re-ordering is done by the decoder, because AVC is much more complex wrt these virtual frames.

Anyway, this is just my limited understanding of how these thing work, and I am often wrong.
Title: Re: Smart-Copy
Post by: lenildosb on December 29, 2013, 10:45:19 PM
hi guys, smarty copy only working with version Avidemux 2.4 GTK+!!!! use is the best version of avidemux!!!!
Title: Re: Smart-Copy
Post by: styrol on December 30, 2013, 02:21:49 PM
Quote from: lenildosb on December 29, 2013, 10:45:19 PM
hi guys, smarty copy only working with version Avidemux 2.4 GTK+!!!! use is the best version of avidemux!!!!
Of courseââ,¬Â¦ it seems you've forgotten to delete (http://www.avidemux.org/smf/index.php/topic,11740.msg61285.html#msg61285) version 2.4ââ,¬Â¦
Title: Re: Smart-Copy
Post by: lenildosb on December 30, 2013, 09:17:39 PM
instead of improving the program, they get worse!!so I use the version 2.4
Title: Re: Smart-Copy
Post by: AQUAR on December 31, 2013, 03:00:52 AM
I pity people who express opinions without ever including even a smidgen of substance.
Seems they tend to be, easily peeved, ignorant of merit and self centred/entitled.

So lenildosb, are you one of them?
If not can you be more specific as to why 2.4 GTK+ is the BEST ever version?
Title: Re: Smart-Copy
Post by: zakk on December 31, 2013, 08:12:03 AM
Quote from: AQUAR on December 31, 2013, 03:00:52 AM
So lenildosb, are you one of them?
Look at his posts, he is just trolling. Just stop answering.
Title: Re: Smart-Copy
Post by: AQUAR on December 31, 2013, 09:53:27 AM
So True - will do that from now on (ie add to my 'ignore list'!).
Title: Re: Smart-Copy
Post by: Papou on December 31, 2013, 10:47:13 PM
Grow up.
Title: Re: Smart-Copy
Post by: Spellbinder on January 02, 2014, 01:49:02 PM
So coming back to my initial post:

What does a daveloper think of my take of smart-copy? COuld it be implemented this way?
Title: Re: Smart-Copy
Post by: mean on January 02, 2014, 04:08:55 PM
For it to work, the encoder setting should be the same between the re-encoded part and the original part
If not, when using format such as .mp4, it will not work
Title: Re: Smart-Copy
Post by: UKenGB on January 26, 2014, 04:57:24 PM
I'm completely confused about SmartCopy. I'm using 2.6.7 on OSX Mavericks and I can find no option to set SmartCopy as suggested in the Wiki docs that as far as I know is the only available documentation. If it is no longer possible in 2.6, how is one supposed to e.g. cut from the beginning a section which doesn't end on an I Frame?

I have some very convenient black frames where I want to make the cut, but they're all B Frames and the next I Frame is too far. I want the resulting video to start with the black frame, but having made the cut, when trying to save I am warned that this is not good, but if I Save anyway, the video file won't play, e.g. Quicktime thinks it's over 4 hours long, when it's actually only 59 mins.

If instead I set the B marker to an I Frame, the resulting file is perfect. So it's obviously crucial to make cuts ending at I Frames which may not be where it is wanted - unless there is some smartness in the cutting process which now seems to be absent.

Is there some method of achieving cuts not on I Frames of which I am unaware?

Will we see the re-introduction of such 'SmartCutting' in the future?

If not, then at least the docs should be updated.
Title: Re: Smart-Copy
Post by: styrol on January 26, 2014, 05:11:15 PM
QuoteIf it is no longer possible in 2.6, how is one supposed to e.g. cut from the beginning a section which doesn't end on an I Frame?
You have to reencode. If you encode your source videos by yourself, you can determine the keyframe rate (e.g. every 12 Frames) in your encoder software.

Smart Copy works (worked only) on MPEG-4-Part-2-encoded files.
Title: Re: Smart-Copy
Post by: UKenGB on January 26, 2014, 05:35:05 PM
Quote from: styrol on January 26, 2014, 05:11:15 PM
You have to reencode. If you encode your source videos by yourself, you can determine the keyframe rate (e.g. every 12 Frames) in your encoder software.

Smart Copy works (worked only) on MPEG-4-Part-2-encoded files.

Re-encoding is simply not an option. The whole point of using something like Avidemux is an ability to simply 'Copy' the existing audio and video, i.e. pass-thru what already has been encoded, degrading it further by re-encoding the whole thing. If you have to convert to a different encoder, then you have no choice, but it seems to me that many people (me included) want to keep the same encoded streams, but simply cut and trim them.

I can't see how setting a different keyframe rate can change anything. There will still be B and P Frames between the I Frames and there'll still be a problem when you want a cut in between I Frames.

Is there a universal SmartCopy actually in development?
Title: Re: Smart-Copy
Post by: styrol on January 26, 2014, 06:40:11 PM
Quotebut it seems to me that many people (me included) want to keep the same encoded streams, but simply cut and trim them.
So do I, but most questions in the forum are about converting (h.264 options) and using filters. So many users do other tasks using Avidemux. And I know a lot of users recording videos in HD and then compacting their stuff (re-encoding without visible quality lost).

QuoteIs there a universal SmartCopy actually in development?
I think you don't have much knowledge how frames are decoded and encoded in H.264. Smart-Copy is great, but I think it's hard to implement it. There are a few commercial programs doing Smart-Copy, if I remember correctly.

QuoteI can't see how setting a different keyframe rate can change anything.
Hypothetically, if you use "25 keyframes per second" at a frame rate of 25fps, then you get only I-Frames. If there are 3 keyframes per second, yes, you can only cut at I-Frames but you could cut indeed quite precisely.



Title: Re: Smart-Copy
Post by: AQUAR on January 27, 2014, 12:06:14 PM
Just a comment about the docs.
There are 2 versions of Avidemux with most of the docs being relevant to the 2.5 branch.

AVC provides a fantastic ratio of compression to quality but you pay the price with increased complexity.
Some of that cost is - difficult to cut/edit at any nilly willy virtual frame.
Avidemux 2.6 is working very nicely with AVC but it has to cut at real frames (as mentioned earlier).



Title: Re: Smart-Copy
Post by: UKenGB on January 27, 2014, 01:32:52 PM
Quote from: styrol on January 26, 2014, 06:40:11 PM
I think you don't have much knowledge how frames are decoded and encoded in H.264.
And I think you're obviously not qualified to judge that. The fact I ask for an answer to a specific question doesn't mean I'm completely ignorant of the entire subject.

QuoteSmart-Copy is great, but I think it's hard to implement it.
I'm quite sure it is.

QuoteThere are a few commercial programs doing Smart-Copy, if I remember correctly.
So not impossible then and hence not a stupid question to ask.

Also, as I'm not the OP, it's also clear that I'm not the only one who wants this feature.  :)

Quote
QuoteI can't see how setting a different keyframe rate can change anything.
Hypothetically, if you use "25 keyframes per second" at a frame rate of 25fps, then you get only I-Frames. If there are 3 keyframes per second, yes, you can only cut at I-Frames but you could cut indeed quite precisely.
Yes I had figured that out for myself, but presumably it's at the expense of file size so not the ideal solution, or not a solution at all depending on storage limitations.
Title: Re: Smart-Copy
Post by: AQUAR on January 28, 2014, 07:42:07 AM
@ UKenGB - People here were just trying to help you from a position based on the information You provided.
I am sure no offense was implied, but just the same, no-one here has a crystal ball that tells them what posters know or do not know.

I wish endusers would stop this audacity to demand changes without ever showing a smidgen of appreciation for this free and featurefull video editor. It certainly makes responding to issues less attractive and probably why so few bother at all.

Since you are not completely ignorant of the subject matter, why not read some related posts and improve your understanding as to how Avidemux 2.6 operates to get along with AVC.

AFAIK, from my limited enduser understanding,  Avidemux 2.6 relies on the AVC decoder for the complex frame reordering process needed to pop out the next frame in the video stream. Hence Avidemux cannot dictate the decoder to pop out a particular virtual frame as defined by DTS/PTS only the next frame (reference frames are okay!).

Only the developer can give you a more accurate appraisal of the why and how of current and future features and limitations.


Title: Re: Smart-Copy
Post by: UKenGB on January 28, 2014, 01:30:26 PM
Styrol has made it abundantly clear he has no desire to help and only makes disparaging (and incorrect) remarks about other's posts.

Look, I'm getting a bit fed up with this forum to be honest. I have already stated (here or in another thread) that I think Avidemux is great, but that doesn't mean it's perfect and when I see a thread someone else has started asking for a feature I think would be useful, it would be remiss of me if I didn't say, yes, I'd like that too. It's one thing to defend Avidemux by saying the feature would be hard to implement, but to proclaim I'm ignorant of the entire subject, just for saying I'd also like that feature is unacceptably rude, arrogant (and wrong).

Regarding the actual feature under discussion, I never demanded it, nor suggested it was easy to implement. Some other software seems to have managed to incorporate a Smartcopy and yes, I would really like it if Avidemux could too. While we're about it, I'd also like it to be able to add additional audio tracks based on an original track already in the file. Am I demanding either, no. Would I like to see them in Avidemux, yes. Does that make me a terrible person, apparently yes. Do I care, in truth, no.

If some forum members cannot accept someone else simply suggesting another feature they'd like to see included, then I would suggest they have no place on a forum such as this.

Me, I've lost interest in this forum and am unsubscribing from this thread. So go and beat up on someone else.
Title: Re: Smart-Copy
Post by: zakk on January 28, 2014, 04:58:46 PM
Quote from: UKenGB on January 28, 2014, 01:30:26 PMhe has no desire to help and only makes disparaging (and incorrect) remarks about other's posts.

UkenGB, you have my full support and total agreement on this. I've used Avidemux and it's forum for years and some years ago, a few guys subscribed and start being rude to others, often with useless posts. You've already found one of them. I've already made remarks at two of them and got very violent reactions and little change of behaviour. The best thing is to try to ignore them, or delete your topic and restart another. But from now on I will start to make remarks again, since I don't want to loose new users just for 3 Avidemux worshipers who get mad if their favorite software gets a criticism.
Title: Re: Smart-Copy
Post by: AQUAR on January 29, 2014, 01:50:29 AM
@ Zakk are you including me in those 3?
If so I warn you,  stop defending endusers that inadvertently or otherwise incite negative reactions, especially when I am involved.
This kind of thing has NOTHING to do with Avidemux worship nonsense.

@ UKenGB
. Have a look at zakk's input before you fall for this false support.
. There is a broad spectrum in the way people interpret conversation. 
  Sometimes that crosstalk leads to unintended anger instead of goodwill from both parties.
. Styrol did try to help you and its not his fault you took that effort as useless or arrogant.
  Maybe try nurturing the conversation you initiated.
  Maybe avoid taking exception to some input provided by the only person that gave you time and effort.
  You could have kept the conversation pleasant by being the hero here.
. Of course You're perfectly fine in mentioning or supporting a video need that avidemux is lacking. That is very welcome here.
. If you leave in a huff and a puff then that is an unfortunate conclusion. Hopefully you do find what you need elsewhere.

Finally if everytime an enduser has to leave because of an incorrect remark this forum would only have one member.
Title: Re: Smart-Copy
Post by: zakk on January 29, 2014, 07:48:09 AM
Here' is another one.
(Now I'm unsubscribing from this topic, since I have nothing more to say and won't learn anything new.)
Title: Re: Smart-Copy
Post by: Spellbinder on January 29, 2014, 07:58:29 AM
Sorry guys, but I would like to go back to the topic:

So, what I understand is that my proposed solution could work, if it is possible to determine the videos encoder settings, BUT it would not be a quite "dirty" solution. So the question is if a developer would give it a try or if a developer is willing to try a "clean" solution. That would be GREATLY appriciated!
Title: Re: Smart-Copy
Post by: AQUAR on January 29, 2014, 10:05:18 AM
@ Spellbinder
Its an interesting topic for sure.
I wonder if there are various smartcopy concepts for AVC video that simply are not a practical fit for avidemux.
On AVC it would definitely be a great edit/cut feature.

Since avidemux is open source, anyone with the knowhow can generate smartcopy code for consideration.
In reality, Mean, the author of avidemux, is the only active developer wrt video processing code.
If there is going to be a smartcopy mode along the way you propose, Mean is the man to convince.
Title: Re: Smart-Copy
Post by: Papou on January 30, 2014, 02:53:44 PM
 
Quote from: styrol on January 26, 2014, 06:40:11 PM
Quotebut it seems to me that many people (me included) want to keep the same encoded streams, but simply cut and trim them.
So do I, but most questions in the forum are about converting (h.264 options) and using filters. So many users do other tasks using Avidemux.
And I know a lot of users recording videos in HD and then compacting their stuff (re-encoding without visible quality lost).

The lack of discussions about AVC smart-copy is certainly not the reason why it's not implemented but a consequence.  The surprise is that the smart-copy that once was much thanked and applauded as a best pride of Avidemux has suddenly almost been called a sin just to fancy about.
No visible quality loss is no wonder if, said shortly, HD quality is better than the screen quality!
The problem comes with editing very different Youtube videos whose uploaders don't seem to realize that repeated reencoding loses much quality (MPEG -> FLV -> MP4 etc.) and seemingly also that they can be edited.

A typical case is Youtube Howto videos that, in principle, are understood by only 15% of the world (because of English) but in fact often 5% (because of the accent), of which you can't read the program menus because it's camera shot instead of desktop captured and during which the explanation can hold on for 20 seconds because the speaker has gone to fetch a pencil or something.  Did you speak of the dumb users of this forum?
If it were only me, I'd make the Howtos with screen captures if possible, I'd edit them with smart-copy and I'd put the speech in subtitles automatically translated in every languages by Google Translate to raise the audience from 5% to 85%.

And, if the big culprit is AVC, then I would simply use good old ASP because smart-copy is weighing more on the scale than the compression performance of AVC.
Title: Re: Smart-Copy
Post by: AQUAR on January 31, 2014, 02:28:56 AM
@ Papou
Your first sentence has multiple negation and can easily be mis-interpreted by some visitors here.
I point that out because that too is the kind of sin that you wish others would not commit.

Back to topic:

Avidemux 2.6 branch is a beta program that takes a different approach to its 2.5 branch cousin in getting at video frames.
The suprise/confusion is that most people continue to believe that 2.6 is an upgrade to 2.5.
These people automatically conclude that 2.6 has dropped much loved features, filters and configurability.
Smartcopy is one such feature where the 2.5 approach is not compatible with 2.6.
I don't agree with you at all in that talking about AVC smartcopy is frowned upon in this forum.

Slightly of topic:

The reality is that some people are easily transgressed when their perception is challenged in some way.
This kind of trend is scaring away people with the goodwill to respond, especially when rewarded with frustration and arrogance.
(And I betterer not mention those that delight in being a complicit factor.)
The heat when this topic arises has nothing to do with talking about smartcopy, its about people forgetting to be civil to each other.

Your perception of dumb users is also bit one sided IMHO. 
Its not so dumb when the Youtube videos are about presenting subject information, as opposed to best practise spatial resolution.
Of course it would be nicer to look at a good quality Youtube video (some are horrible!).
Title: Re: Smart-Copy
Post by: Papou on February 14, 2014, 09:21:33 PM
Suppose I split a video on I-frames into files 1, 2 and 3.
Then I edit file 2, split it on any frame and re-encode the end part into file 2a.
Then I append files 1, 2a and 3.
Haven't I done a smart copy?  Why would Avidemux 2.6 be unable to do that?
In principle, the only possible gotcha is selecting a compatible encoding.
But Avidemux should know.
Aren't all operations used here supposed to work?

Unfortunately, I tried that with a Youtube H.264 / AVC video.
I re-encoded part 2 with Codec Mpeg4 AVC (x264) standard options.
1, 2a and 3 play nicely on any renderer when standing alone, all 30 fps.
But after appending, part 2 shows a non moving picture or gray frame.

What's the trickery there?
NB: this (last) question is not a sinning suggestion to implement smart copy.
Just why a re-encoded part cannot be appended to its home video.
Title: Re: Smart-Copy
Post by: AQUAR on February 15, 2014, 04:23:51 AM
@ Papou
QuoteAren't all operations used here supposed to work?
That would be the aim of a work in progress, and should be the case of a final.

QuoteIn principle, the only possible gotcha is selecting a compatible encoding.
In practice there is more trickery involved, as you found out.
To get at that trickery, you need to know the ins and outs of all the dependancies involved.
Way beyond me, so I bow to the expert opinion of the person here in the know - Mean the developer.
And Mr Mean says its too difficult.

That still leaves You with a desire for AVC smartcopy in Avidemux, just like the rest of us (me included).
Some options now are:
1) Accept that some implementation of a smart copy is not likely to be forthcoming from mean.
2) Dig into the principles of operation and back up that new knowledge with some smartcopy code for avidemux.
    There are many such type of contributions from members, as avidemux is open source.
3) Find a work around fiddle, like you tried with cut - recode - append. Personally, I applaud and appreciate such shared initiative.
4) Find an editor that has smartcopy on AVC. etc

Here is a variation on your intiative.
And one you must take with a pinch of salt because I also am just a Noob.
Say that section 2 is what you want to cut out of your video, except for some frames at the start and some frames at the end.
These frames of course would be inside a GOP and are therefore not Idr frames.
Now save a few GOPS from the start and end of the cut and compatibly recode them ALL as Idr frames
(ie create a segment 2A and a segment 2B all with Idr frames)
Now edit/cut those segment for the wanted frames leaving 2a and 2b (still all Idr frames).
Then append 1 plus 2a plus 2b plus 3 and see if that avoids strange frames at the intended cut points.
Title: Re: Smart-Copy
Post by: Papou on February 15, 2014, 01:23:12 PM
Quote from: AQUAR on February 15, 2014, 04:23:51 AM
@ Papou
QuoteAren't all operations used here supposed to work?
That would be the aim of a work in progress, and should be the case of a final.

QuoteIn principle, the only possible gotcha is selecting a compatible encoding.
In practice there is more trickery involved, as you found out.
To get at that trickery, you need to know the ins and outs of all the dependancies involved.
Way beyond me, so I bow to the expert opinion of the person here in the know - Mean the developer.
And Mr Mean says its too difficult.

Your explanation why the very simple, common appending of videos with like characteristics (Mpeg4 AVC (x264)  30 fps same size) does not work is very very, very, unclear.
Title: Re: Smart-Copy
Post by: AQUAR on February 16, 2014, 05:32:33 AM
QuoteYour explanation why the very simple, common appending of videos with like characteristics (Mpeg4 AVC (x264)  30 fps same size) does not work is very very, very, unclear.
Of course as I didn't explain it AT ALL.  And neither did I say appending doesn't work.
I can't help it if people mistake a beta program as a final/proven program. 

Smartcopy is the topic here not the append function.
So what I did do was reference your own initiative for a "smart copy" like process.
I added some info for you to try out if you wished too (don't expect others to do it for you).
And I qualified that info with "take it with a pinch of salt".

I have some comprehension of why the smartcopy process is difficult to implement.
Not going to elaborate on that, as that comprehension is at Noob level.

If you reject the responses already given, try turning the pages of the book yourself.
Title: Re: Smart-Copy
Post by: lenildosb on February 16, 2014, 09:09:30 PM
hi friend @Papou, use the version Avidemux 2.4 GTK+, it has the function Smart-Copy!!!
Title: Re: Smart-Copy
Post by: AQUAR on February 17, 2014, 09:28:10 AM
Everyone - don't take any notice of this response from lenildosb.

Avidemux 2.4 is designed to work with older codecs and so is not for AVC.