Repair B-Frames

[Felix Leiter]

I use a 'Xoro HRS 8590 LAN'-receiver to record 720p german television.
Unfortunately the receiver splits the recordings into 4GB-parts (which means every 35 to 40 minutes) (It makes no difference if the attached USB-harddrive is formatted FAT32 or NTFS).

I use Avidemux v2.6.10* to:
- join the parts
- exclude multiple audiostreams
- crop
- delete the logo
- convert it to mp4 x264

It bothers me that I often get some short pixel errors (ca 1sec, see attachment) due to the 4 GB-split not exactly at an I-frame (that's my theory).
There is no difference between joining the parts with the terminal-command 'cat' and dragging the first part onto the Avidemux-window, followed by the question "... sequential file names. Should they be all loaded?"
I read about the function 'tools' -> "Rebuild I & B frames" (Also I read something about meta-filters to reencode only parts of the whole video).
I can't find any of them in my OSX-version of Avidemux - are they windows-only? Or are they plugins I have to install separately?

Any help -or other solutions to my problem- is highly appreciated !
Thank you very much,

   Felix


* in newer versions (v2.6.911 and v2.6.11) the position-slider does not work properly / at all (OSX 10.10.5).

[mean]

Normally if you name them sequentially, just load the 1st one and let it append the others, the problem should not happen
Are you sure that's the case ?

[AQUAR]

Functions like --  'tools' -> "Rebuild I & B frames" are not present in ADM 2.6^.
The reading material probably refers to ADM 2.5 (its a different program under the skin).

ADM 2.6 also does not have a smart copy function (on account of complexity with current codecs).
So there is no part recoding where cuts aren't on Intra frames.

Hard to see why a split file when joined back together would not have a contiguous GOP at that joint.

[Jan Gruuthuse]

Xoro HRS 8590 LAN Digital Satellite Receiver records in .ts, Standard: MPEG 2/MPEG4 AVC h.264

Transfer recording from USB hard disc to computer Hard Disc
- Load 1st part (main title) when asked to append/load other parts sequential answer no
- Append 2nd part (presume 001), when asked to append/load other parts sequential answer no
- Append 3rd part (presume 002), when asked to append/load other parts sequential answer no
- Append 4th part (presume 002), when asked to append/load other parts sequential answer no
...
Save this to new video. Close loaded video. Load the newly saved video. Then on this new video:
- exclude multiple audiostreams
- crop
- delete the logo
- convert it to mp4 x264


check if your firmware is latest?

Firmware für XORO HRS 8590 LAN
---------------------------------------
Version: V1.09.13908
Firmware-Datum: 28.07.2015

can you post the MediaInfo for this recording

[Felix Leiter]

Thanks a lot everyone for your help so far!
Your answers made a couple of things clearer for me.

@ AUAR:
I downloaded Avidemux v2.5.2 but it won't start under El Capitan.

I believe the the joined file has no contiguous GOP because the receiver makes its split when the 4 GB sizelimit is reached, regardless if there is an I-frame or not. So parts of the pictureinformation from start of segment 2 until the next I-frame is missing (still my interpretation).

@ Jan Gruuthuse:
On my first movies I used the terminal-command 'cat' before I realized that ADM does the same job. As far as I can remember the results at he joining points were the same.
When I do it like you suggested, I get a slightly different result:
A) "Should all sequential files be loaded?" -> yes (previous workflow)
=> the motion runs flawlessly but there are heavy pixel errors for about one second
B) "Should all sequential files be loaded?" -> no (your suggestion)
=> the motion stucks for a moment but there are only very few pixel errors

You say "Save this to new video". Which videoformat should I use as for ADM has 'AVI Muxer' as the preset output format?

For information on videofiles I use 'VideoSpec v.0.9.8', but I used MediaInfo as well and added both results (left: first segment; right: next segment).
Interesting that only for the first segment the information "GOP: M=6, N=32" is supplied.

I will check the firmware of the receiver this evening.

Thanks again very much for all your help.
So I have to decide whether I prefer pixel errors or stuck motion ?   

[Jan Gruuthuse]

I would use Mpeg-ts for output for joining, at this stage you could drop unwanted audio tracks to.
Menu -> Audio -> Select Track -> Unselect unwanted tracks here.
Then save.
Check if your used media player(s) play the .MKV without problems
If you do need .avi, do check avidemux preferences: Menu -> Edit -> Preferences:
[Output]
[v] Create OpenDML files <= mark this one, see if this improves your avi issue.

more avi info (German) https://de.wikipedia.org/wiki/Audio_Video_Interleave

[mean]

Again, if you let it autoappend, the files should be considered as a single file split across several
It should not matter where the split occurred

If it does not work, you should try to merge all the files ( like "cat a b c d > merged.ts") and see if the problem still happens on the merged file
If yes => not a splitting problem
If no => The split management code is bugged, and i'll look into it

[Jan Gruuthuse]

As mean says autoappend should work. If it doesn't.

I just noticed: should 0000.ts and 0001.ts be sequential? If yes: You have a problem!

Audio track's between file1 an file2 don't match:

File 1: audio track 1: Mpeg-1 layer 2 (mp2)
File 2: audio track 1: AC3-A52

File 1: audio track 2: Mpeg-1 layer 2 (mp2)
File 2: audio track 2: AC3-A52

File 1: audio track 3: AC3-A52 5.1
File 2: audio track 3: AC3-A52 5.1

File 1: audio track 4: AC3-A52 2
File 2: audio track 4: AC3-A52 2

[Jan Gruuthuse]

Again, if you let it autoappend, the files should be considered as a single file split across several
It should not matter where the split occurred

If it does not work, you should try to merge all the files ( like "cat a b c d > merged.ts") and see if the problem still happens on the merged file
If yes => not a splitting problem
If no => The split management code is bugged, and i'll look into it

Take note autoappending files should be from the same continuous recording!
The idx2 file should be considerately bigger then if you would only have loaded part1 and stopped appending/autoappending.

[AQUAR]

@ Felix Leiter
As Mean said - it doesn't matter where the split happens, because when you join the next segment the GOP at that point will become whole again.
This of course is assuming that no "virtual frames" are lost in the process of joining (I join split files all the time - never had a problem!).

[Felix Leiter]

Thanks again for your patience and your detailed answers !
As you suggested, I went back to the terminal-way to join the segments and it also led to pixel errors. But I found out something interesting: while QuickTime-player (and Quickview) displayed only a few pixel errors, the picture was massively distorted when watching the joined TS-file with VLC !
So the player plays a role, how severe it looks (unfortunately I watch the final results with my raspberry and there it appears rather VLC-like than QT-like).

Maybe a solution:
I then installed the 'TS-Doctor'-demo on my Win7-bootcamp-partition. With this program I joined the segments and verified them. It did not find any error, but when I exported them to one single TS-file, there were no errors to be seen at the affected position (ADM, VLC, QT-player).
I will use this workflow on some more recordings. When it works this way, -and there is no similar OSX-tool- I will purchase the full version of TS-Doctor  ... and report my results here.

Thanks a lot and happy Easter,
  Felix

btw: I use the next-to-last firmware. It has the very same versionnumber (sic!) but with the date 'Apr 29 2015'. I didn't update yet because the new fw comes with advantages ... and disadvantages.

[Jan Gruuthuse]

You could also test avidemux 2.6 on your Win7-bootcamp-partition, win32(-bit) or win(64-bit): http://www.avidemux.org/nightly/

[Felix Leiter]

After some further tests, this is my new workflow:
I use TS-Doctor to join and verify the ts-stream. Then I deselect unwanted audio and cut the file to keep only the main movie. Advantage: When cutting with TS-Doctor, it exports the srt-file with the correct timestamps to keep movie and subtitles in sync.
After that I use Avidemux as before to crop, remove the logo and convert the movie to a mp4-file.

I noticed that Avidemux produces larger files with a higher bitrate when using the workflow mentioned above (join and cut with TS-Doctor) compared to the former workflow (join per terminal, cut with Avidemux)
Examples:
duration: 2h 53min - TS-Doctor+Avidemux => 7,4GB, 5697 Kbps / Terminal+Avidemux => 4GB, 3082 Kbps
duration: 1h 35min - TS-Doctor+Avidemux => 3,5GB, 4840 Kbps / Terminal+Avidemux => 2GB, 2855 Kbps
Maybe someone can explain this behavior (... or provide a link)?

Two times my ts-files were damaged. Avidemux converted them and came up with the message "saved the file successfully" but that was not true: the conversion was aborted and the file was incomplete. (There was no difference between a ts-file joined with TS-Doctor or Terminal).
May workaround: I used Handbrake to convert these defect ts-files. This way I get pixelerrors at the defect position but the playback moves on and the file is complete. Only drawback: no way to remove the logo.

[Thomas029]

repair B-