Remove duplicates after telecine crashes Avidemux

[yami]

After using the decomb telecide (set on no strategy/bottom) and actually other filters to tackle progressive frames (why does it always leave some way ghosting every 5th frame? It looks like it does not detect where in the combed frame to look for to get rid of the combing... though it looks like it is always the frame made up from the frame before and the one after... I mean, couldn't it just drop this frame?) - well after using decomb telecide ^^- I try to get rid of duplicate frames (I need to, as it is jerky after the telecide) with decomb decimate, Avidemux usually crashes. I attached a crash report to my post.
So, this post is actually about 2 issues first, the crashing with using decomb decimate and the more general problem of dividing the frames apart.
Here is the avi snippet from the dv: https://mega.co.nz/#!O1YFmAbR!CGEetQfMDUKkk_Xn114YzEThzn1y6xmEvJZaMsu87Rk (Test_vidi_capture.avi 22.5 MB)

Edit: I was doing a bit of research with the telecine... and came across VapourSynth. Well, I am a typical Mac-User, I am afraid. I managed to sort of brew it, but failed at the editor there XD - what sort of looked very interesting (which is, why I tried that in the first place) was this filter: http://www.vapoursynth.com/doc/plugins/vivtc.html Would there be a way for a sage person, to make a port of this for Avidemux?

[AQUAR]

Maybe you are using these filters in an incorrect manner for the media?

If the original source is film (24 progressive frames per sec) then telecining that source will give 5 interlaced video frames for every 4 film frames. 3 are still progressive and 2 are created by mixing the interlaced fields of the third frame with one field from the previous frame and one from the next frame in an odd even manner.  So you create 5 video frames from a set of 4 film frames.

If you decomb telecide with no strategy you will get transitional content (fuzzyness) whenever the 2nd or 4th frame of the original film has movement in it relative to the 3rd frame (frames 3 and 4 after telecining!).
You need an inverse telecine strategy and then decimate the extra frame (ie back to 24 Fps from 30 Fps).

If the original source is video (as opposed to film!) then a deinterlacing filter like Yadif might work out better for you (presuming its interlaced).

Of course I don't know what sort of media you are applying these filters to (If I have time I'll check out the sample)?
Not on a Mac either so can't verify the crash.

[yami]

Thanks for the reply
The source is the capture of an NTSC tape (orig. Japan, with a japanese VCR XD) via Canopus, via Vidi, transformed to avi via MpegStreamclip (video track 'untouched' , audio encoded to MPEG (why did I do that?)). I heeded the advice given here: http://www.avidemux.org/admWiki/doku.php?id=using:video_filter_decomb_telecide&s[]=telecine, but found, that you still see residues of the combed pattern... I tried other combinations, but it all looked... even less good.
The residues I am talking about look like this:

and this:

And it appears, as movement in the scenes allow and to be expected, every fifth frame.

With some combinations I can make the stripes turn into discoloured blobs, sort of, but neither is how it should look like... The suggested NTSC->PAL strategy results in Avidemux crashing.
Any help to solve this is gladly accepted

[AQUAR]

"deinterlacing" artifacts become embedded with improper recoding of source material.

I have several media files that were incorrectly recoded (mostly into avi).
Often these are recodes where the interlaced material is resized before proper deinterlacing.
These all have frames with terrible combing artifacts that you cannot get rid off.

I use avisynth filters that selectively blend only areas with combing.
These stripes are typically in motion high contrast edges.
Blending will blur those areas with the stripes (so just pretend they are motion blur!).



 

[yami]

Hmmmm... So, this may actually be a problem occuring because I have to somehow change the container of the dv to avi?

Hmmmm... Maybe, Vidi is sub-optimal for capturing too. So, back to square one, capturing.

[AQUAR]

No its not really a container issue.

Was just saying that some video material is messed up and nothing much you can do about it.

In your case its quite possible you are applying the decomb telecide plugin incorrectly.
Try - change the field order, apply an inverse telecine and then decimate the unwanted frame.

Note: Decimating is desirable when you apply the inverse telecine strategy.

As I read your post, you are selecting the no strategy option and probably not selecting any option to deal with the combing artifacts (ie blending).
Result is one "dirty" frame in 5 with combing artifacts.
Suggests to me some form of pulldown has been applied to progressive material to create 60 fields/s (30 Frames/s) video.
Probably 3:3:2:2 as that creates one "dirty" frame in five (with nice judder effect too!)

You have to decimate that dirty frame and if ADM crashes doing that, then try another MAC video processing program that will do it.

Else just try Yadif and see if you get a nicer result.

[Jan Gruuthuse]

There could be an additional problem to: whilst capturing, the digitiser (capturing device) is probably not capturing all the visual Lines.
NTSC =  525 scan lines. 483 scan lines make up the visible raster.
There is also a difference in black levels: black level and blanking level of the signal are identical (at 0 IRE), as they are in PAL, while in American NTSC, black level is slightly higher (7.5 IRE) than blanking level.
Plenty of discussions going on in other specialised forums regarding digitizing/capturing NTSC-J manga. Use S-VHS connection if possible.

[yami]

The basic set-up is a Sony SLV-R7 tape player, it has VHS-S and TBC (It was only released in Japan, so I had my troubles with the Kanji first ^^ - S-VHS is switched on) and only does the Japanese NTSC as output attached to it comes a Canopus ADVC-100. From there directly into the firewire port. The dips on the canopus are set for NTSC Japan and analog input. What I find curious, as soon as I press start to play a tape, the S-VHS light turns off though... I don't know, if it is something with the tapes then, though they should support this.... There exists some very old hardsubs from these tapes, where there is no combing and no discolorations. Sadly, the guy who made them has disappeared... Well.
At least, with this setup, the conflicts of lines and NTSC and PAL/PAL 'interpretations' should be ruled out to the point the data reaches the firewire port. Then the first difficulty comes, I guess, which is saving the incoming data somehow on the mac. So far, Vidi was the simplest program to do this, but it might well have problems with the NTSC, though the tapes displayed OK with it...
Would you have any idea (command line too) how to fetch the dv/avi stream better from firewire?

Edit: we are trying to set up something on linux, to have better options for capturing....

[Jan Gruuthuse]

>8 >8
Edit: we are trying to set up something on linux, to have better options for capturing....

That could become very tricky: in ubuntu 12.04 perhaps even since 11.04 they changed something in the ieee1394 (firewire) drivers and only firewire security cams are supported.
I could no longer use Sony Digital 8 cam (ieee1394). I'll check tomorrow, If I can still boot that old system.

Has something to do with this:

if your camera is compliant with the digital camera specs 1394ta.org. Notice: all consumer grade cameras in which you can insert a video tape are not compliant with the above mentioned specifications and therefore cannot be controlled with the AV/C protocol, please refer to 1394.org

1st update: The last time I could use DV capturing (ieee1394) with kino was with How to fix FireWire capture in Ubuntu 10.04 (Lucid)
2nd update: Kino is still up and running on ubuntu 14.04.4, even if kino project is dead   

[yami]

I found a little app for mac, that grabs the dv directly from the firewire input. It's part of the Firewire SDK in Apple's developer kit and called AVCVideoCap.app. The filesize is appropriate, I think.
Also, I double-checked the dips on the Canopus, they are set on NTSC/Japan, so the output should theoretically be something to work on, without ruined lines and so on... I hope so, at last.
I really don't have an idea, why it should not work... but it doesn't.
One weak point really is, that I have to convert the dv into avi and Programs always seem to want to do more with the file than just changing the container and leave the inside untouched.

[Jan Gruuthuse]

For now avidemux 2.5.6 or 2.6.8 does not handle  this dv.
demo DV video: pond.dv 110 MB download

Don't go to .avi stay in .DV and test Handbrake:

Handbrake does: pond.mkv 26 MB download AVC codec

use download button on top right hand on each of the player page

[AQUAR]

Its true that capture devices typically crop the NTSC signal by virtue of sampling only the visible components of the scan lines.
And then make up the pixels for each horizontal field line from the sample space. (higher sampling rate = better capture!)

Presumeably the capture process itself will not add extra combing issues (has already plently of other issues!).
I think at this stage it still is all about how you process the frames that are presented as two interlaced fields weaved together.

Who knows what was done between original source and the yami recording, and then we add to that the post capture processing/compression.
If yami is getting 4 progressive frames and one blended frame (pretty good really) I would look at the cadence of the captured stuff and persue an inverse telecine / decimate route (still thinking advanced pulldown!).

Note: IMHO the decomb telecide and decimate filters need to be configured as a complementary set.
Hence why I think you are using these filters incorrectly (just my opinion!).   

But I've tried to apply my logic on various medias before and don't always get a predictable result.

Also, as this is anime stuff, maybe look at shifted fields.
Come to think of it - shifted fileds can also be caused by the capture device!.
Also (lots of also's!) the cadence of DV camcorders might be 2:3:3:2.
Someone might know if these avidemux filters can deal with that.



   

[Jan Gruuthuse]

Correction:
IEEE-1934 is up and running on 14.04.1 LTS
Kino gives you these posible capture files
- DV AVI Type 1 (not working in avidemux)
- DV AVI Type 2 (working directly in avidemux)
- RAW DV (not working in avidemux)
- Quictime DV (video DVDS) (working directly in avidemux)
attached required installs for 14.04.1 /10.04 LTS

@yami: Big thank you for bringing this up, now I have firewire consumer digi8 back on ubuntu

latest: with Sony digi8 i.link
14.04.4 only camera capture, no tape access
10.04 both camera and tape access

[yami]

^^@ Jan ...
I did a bit of fine-tuning with the telecine filters.

adm.addVideoFilter("telecide", "order=0", "back=2", "back_saved=2", "guide=1", "gthresh=10.000000", "post=2", "chroma=True", "vthresh=50.000000", "vthresh_saved=50.000000", "bthresh=50.000000", "dthresh=1.000000", "blend=True", "nt=10", "y0=0", "y1=0", "hints=1", "show=False", "debug=False")
adm.addVideoFilter("decimate", "cycle=5", "mode=0", "quality=0", "show=False", "debug=False", "threshold=0.000000", "threshold2=1.000000")

To the effect, that Avidemux doesn't crash and most of the telecine actually is either removed or else processed and blended into a frame, only failing with rather small areas like this:


The captured file out of the Canopus seems to be a - RAW DV (not working in avidemux), also, the telecine pattern seems to change with some scenes. With making the avi from the dv with MpegStreamclip, I tried to keep the 'original' track, like, not touching the interlace, chosing progressive in the options and everything NTSC. The resulting Avi had approximatly the same size. But comparing the specs, it sort of didn't leave things untouched (only the audio can be left untouched), maybe this causes the troubles. Actually, it would be ideal, if Avidemux would be able to open the dv-raws - (/me points wildly at a future project to have this added to Avidemux).
Another thing, that was rather prominent is the rainbowing. I don't think, it can be adjusted with the normal colour enhancement filters?

As an alternative, I try my luck with VapourSynth, if ever I get to know, how to use the python shell and how to cut, when you can't really see which frame you are in... Which brings met to one of the things I really like about Avidemux - you see what you do.

[Jan Gruuthuse]

Create a one minute clip with the difficult scene included ant test with that to get an more favourable end result.
Check if you can change the captured output file with AVCVideoCap from RAW DV to DV AVI Type 2 or Quictime DV

ps: iMovie can capture DV and export as Quicktime DV. If no filters are used, there should be no re-encode, just a wrapper change.

iMovie doesn't convert DV to AIC anyway, doing a sidetrip through an SDK is really unnecessary. Just opening up the iMovie cache folder allows you to find and use the raw .DV files for other apps. QT architecture happily accepts raw .DV files as easily as DV.MOV files, so most apps based on QT (FCP/FCS/FCE, MpegStreamclip, etc) will also. They MIGHT want to rewrap upon saving/storage/export, but that's a fairly trivial and quick matter, being a non-reencoding process.