Audio loses sync after separating fields

[modoversus]

Hello guys, I am using Avidemux to deinterlace 240p video content from old video game consoles.  I capture them using an El gato game capture. The resulting .TS file is very sharp, colorful and interlaced 720x480 at 29.970 fps. In order to use these files in Adobe Premiere I must deinterlace them in Avidemux.

What I do is, in filters, I select in this order:

1.- Separate fields. To obtain 2 fields of 240p.
2.- Resample FPS, from 119.89 to NTSC 59.93.
3.- Resize to 720x480, Lanzcos3, since it does not offer the Nearest Neighbour option, which is the best for pixel art.

The result is a great looking progressive 480p video at 59.941 FPS that lasts the exact same amount of time, and it's way better than what the Elgato software and Adobe can do to an interlaced file. Unfortunatley, the audio loses sync. I have attempted to do this process to two different video clips, and both lost sync.

What would be the best way to avoid losing the audio sync?

Thank you in advance.

[mean]

Please provide a sample file
(unedited)

[modoversus]

I have uploaded the TS file I am using for this test. (48 seconds long).
You can download it from here: http://www.modoversus.com/sample_interlaced_240p_TS.rar

Please let me know if I should upload any other file.

Thanks.

[mean]

Is that one ok for you ?
http://fixounet.free.fr/deinterlaced.mkv

[modoversus]

Thanks  for the video and quick response!

The thing about 240p videogames is that they frequently used flickering graphics to create the illusion of transparency.  This video I am using for test is using this effect in the rainbow in the back, and in the shadows of the characters whenever they are under the sun, on the left of the level.

In the video you have kindly provided me, the shadows and the ranibow spend various seconds either completly solid or completly gone. Most deinterlacing methods tend to either blend the frames to create the transparency effect, which leads to a less sharp image and darker colors, or to loose the flickering effects for many frames at a time by either making the graphics completly solid or removing them. The rainbow is solid for most of the video, and the shadows on the ground for the characters are missing.  (I even bought a scaler device that takes the RGB 240i video and escalates it to 720p, and had the exact same result for the flickering effects, which is why I am trying with software to preserve them.)

Using separate fields, the result of the test is this: http://modoversus.com/separated_fields.rar
It does not look as sharp compared to the video you uploaded, but the flickering effects look very close to the interlaced video and how it looks on the game console.

[mean]

ok, but in term of sync ?

[mean]

I tried to mimick what you do :
* separate field
* Resample FPS to 59.94 (not 59.84)
* Encode audio to mp3
* Save as mkv

It looks fine
I suspect the mp4 container to introduce a delay. Could you try with mkv ?

BTW, just dropping one field will have the same result with better speed

[AQUAR]

Interesting issue.

Each frame in the .ts seems to be, 2 progressive frames from the 240P with each P frame converted to an up-sampled field, and weaved together.
Is the capture done that way on purpose, so as to preserve the mechanics behind the transparent rainbow (viz, on in one frame then off in the next frame)?

Can't help with the out of sync issue.
But just as a thought - maybe the capture has dropped frames and any explicit AV-sync timing is "lost" in the ADM separate field filter process.
Mean would know if that is probable (since his approach has slightly adjusted FPS!).

[modoversus]

I tried to mimick what you do :
* separate field
* Resample FPS to 59.94 (not 59.84)
* Encode audio to mp3
* Save as mkv

It looks fine
I suspect the mp4 container to introduce a delay. Could you try with mkv ?

BTW, just dropping one field will have the same result with better speed

Mean, I tried it as you recommended, (without dropping one field). I'm happy to report that this obtained the desaired result! Kept the effects and the sound in sync. Thank you so much for the advice! 

[modoversus]

Interesting issue.

Each frame in the .ts seems to be, 2 progressive frames from the 240P with each P frame converted to an up-sampled field, and weaved together.
Is the capture done that way on purpose, so as to preserve the mechanics behind the transparent rainbow (viz, on in one frame then off in the next frame)?

Can't help with the out of sync issue.
But just as a thought - maybe the capture has dropped frames and any explicit AV-sync timing is "lost" in the ADM separate field filter process.
Mean would know if that is probable (since his approach has slightly adjusted FPS!).

Probably yes. Old videogame consoles had limited ability to provide semi-transparent colors, so to achieve this they would flicker the graphic elements to fake the transparency, or would use a mesh of pixels. Old video game consoles (and computers that connect to TVs) from the late 70s up to maybe the year 1999 actually display 240p video, which is then somehow displayed as 480i on standard definition sets and arcade monitors. According to Wikipedia (https://en.wikipedia.org/wiki/Low-definition_television#Resolutions):

Older video game consoles and home computers generated a nonstandard NTSC or PAL signal which sent a single field type which prevented fields from interlacing. This is equivalent to 240p and 288p respectively, and was used due to requiring less resources and producing a progressive and stable signal.

Some video capture devices and HD TVs do not recognize 240p or treat it as 480i, which creates issues. I used the El Gato Game Capture HD device to capture it, as it can recognize 240p /288p  trough Composite and S-Video cables (the sample video used S-Video), tough sadly not trough component. Other capture devices are not able to recognize the 240p signal and do not capture the video.

The whole thing about de-interlacing, upscaling old video games is a bit complicated and expensive proposition. The best solution I had found was a device called the Framemeister RGB, a 350 USD device specially designed to upscale 240p video to HD standards with a lag of about 1 frame or less. Since I could not afford one, and was only interested in recording video, I was looking for a solution on software that could create a similar look, which Demux has provided.

[AQUAR]

Thanks for the info, it has satisfied my curiosity.
I'll add this practical use for the separate field filter to my bag of tricks.

[AQUAR]

Just for interest have you tried using dgbob mode=1 order=1 threshold=0
I think that will extract each field in the .ts and fills in the "black scan lines" by basic nearest neighbour interpolation.

You end up with 720 * 480 frames that are upsampled versions of their corresponding progressive frames from the 240P source.
This condenses the multi filter workflow into a single filter work flow.
That will also preserve the transparency (flicker trick!) seen in the source video (rainbow and shadows on the animated figures!).

Also the captured data is for non square pixels and so you need to adjust for the square pixel PAR of computer monitors.
In other words --- the .TS specifies a DAR of 4:3 but the ADM recode specifies a DAR 0f 3:2 (and so needs a PAR adjustment!).

Curious question - what is the frame rate of the 240P source material.

[Jan Gruuthuse]

>8 >8 Curious question - what is the frame rate of the 240P source material. >8 >8

>8 >8 I capture them using an El gato game capture. The resulting .TS file is very sharp, colorful and interlaced 720x480 at 29.970 fps. >8 >8

1.- Separate fields. To obtain 2 fields of 240p.
2.- Resample FPS, from 119.89 to NTSC 59.93.

[AQUAR]

@ Jan
Something maths doesn't quite seem to stack up to me - thats why I asked the question.

The .ts frame rate is 30 fps (60 fields per second as its interlaced!).
Each .ts frame is in fact 2 frames from the 240P source.
So the frame rate of the 240P source must be 60 fps (twice the .ts frame rate!).
At the end of these processes we are getting progressive frames equal to the "up-sampled" 240P frames and displayed at a frame rate of 60 fps.
All good so far (I think!) and none of the frames are duplicated.

But, are those games consoles really outputting 240P @ 60 fps?
I thought it was more like 30 fps (frames per second!).

PS:
Its so hot here (38 celcius at the moment) that I am probably missing something fundamental.

[Jan Gruuthuse]

The game is outputting probably interlaced 30 fps.
The recording is 240P (progressive: fullframe = 2 half interlaced together)
splitting 240p back to 2 half interlaced frames
frame rate is staying the same: one gives full frame the other 2 half frames
Add to that going from analogue to digital.
...
just guessing, never used 60Hz devices, lucky me.