I think these two bugs are related, but in either case, I hope they can be fixed. These are both SD captures, not HD.
This first file (contained in a zip file) is 15 seconds. It's very odd that AviDemux shows that there's only 1 keyframe, the first frame. Yet it behaves like there are 29 key-frames, as evidenced that you can jump to all 29 key-frames. But only the first one shows that it's an I-frame. All the others show as a P-frame. Obviously just a mistake in how the program SHOWS what kind of frame. We really need it to correctly show that it's an I-frame, not a P-frame.
http://www.tomsgoodfiles.com/2013-2-20-22-38-17.sample-SD3-clip-from-Tom-Cook.zip (http://www.tomsgoodfiles.com/2013-2-20-22-38-17.sample-SD3-clip-from-Tom-Cook.zip)
I think the 2nd file has the same problem (I believe there are multiple key-frames) but behaves differently. AviDemux crashes. The capture begins while viewing my DVR menu of shows to play. I start recording, then pick the show and play it. Total is also about 15 seconds. The Colossus successfully records situations like this. Maybe there's something funky about what happens when the DVR switches from showing its menu to showing the show. It's probably briefly changing what format of audio is recorded. At the very least, we need to be able to have AviDemux chop off the menu stuff at the beginning, by being able to skip to later keyframes. But in this case, it blows up. In other cases, it has not blown up, but it refuses to see any more key-frames (labelled as "P") and jumps to the very end of the file. (I can't share those, as they are way too big to upload anywhere. I will try to create a small one.) The menu is very useful to record in the raw captured file, as it contains info about the clip, so I hope we won't be told "don't capture that". Here's the 2nd file.
http://www.tomsgoodfiles.com/2013-2-20-22-43-10.sample-SD4-clip-from-Tom-Cook.zip (http://www.tomsgoodfiles.com/2013-2-20-22-43-10.sample-SD4-clip-from-Tom-Cook.zip)
I was able to create another file with the DVR "shows menu" captured at the start and AviDemux did NOT blow up.
http://www.tomsgoodfiles.com/2013-2-20-23-42-4.sample-SD5-clip-from-Tom-Cook.zip (http://www.tomsgoodfiles.com/2013-2-20-23-42-4.sample-SD5-clip-from-Tom-Cook.zip)
It's a minute long, has 2 I-frames (at least that AviDemux can see and jump to), and plays through the whole file nicely, just as the other 2 do. But there's only those 2 I-frames it can jump to. Yet, I have to believe there are many key-frames throughout the entire 60 seconds. Need to be able to get to them in a file such as this.
Again, if AviDemux could get to those keyframes, I think it would show erroneously as P-frames (the first bug I reported).
(On a separate but connected matter, does anyone know of a free prog that can allow me to see the frame type (I, P, or B) of every frame in a video? It would be good to be able to confirm when AviDemux begins labelling the frames correctly by an outside source.)
2013-2-20-23-42-4.sample-SD5-clip-from-Tom-Cook.M2TS:
[mpegts @ 0x12569c0] Continuity check failed for pid 0 expected 15 got 0
[mpegts @ 0x12569c0] Continuity check failed for pid 31 expected 12 got 0
[mpegts @ 0x12569c0] Continuity check failed for pid 4113 expected 9 got 0
[mpegts @ 0x12569c0] Continuity check failed for pid 4352 expected 3 got 0
Is this capturing from settop box with haupage capture board? My guess is you need to record the program stream separately from the DVR desktop capturing. Different streams = different info, is not as much a frame issue. If that is not possible you need to use lead in / lead out time for recording program so you can make the cut after switching ...
Quote from: Jan Gruuthuse on February 21, 2013, 10:21:48 AM
2013-2-20-23-42-4.sample-SD5-clip-from-Tom-Cook.M2TS:
[mpegts @ 0x12569c0] Continuity check failed for pid 0 expected 15 got 0
[mpegts @ 0x12569c0] Continuity check failed for pid 31 expected 12 got 0
[mpegts @ 0x12569c0] Continuity check failed for pid 4113 expected 9 got 0
[mpegts @ 0x12569c0] Continuity check failed for pid 4352 expected 3 got 0
How did you get this info? What tool? Tell please?
QuoteIs this capturing from settop box with haupage capture board? My guess is you need to record the program stream separately from the DVR desktop capturing.
Can't do. The DVR has the tuner.
QuoteDifferent streams = different info, is not as much a frame issue. If that is not possible you need to use lead in / lead out time for recording program so you can make the cut after switching ...
Sounds like you're tell me to do what I had said I hoped you wouldn't, namely, "don't record that". I really think a program like this needs to be smart enough to skip over garbage in possibly-damaged files, so that things can be reclaimed.
And I'll still wait for someone to look into the P-frame/I-frame identity issue (the first issue I brought up). The frame type needs to report accurately.
Never said don't record that. Problem is switching to and from dvr desktop and channel watched on dvr. DVR desktop is stream output on itself and completly different to the stream from the program. (if I all understand this correctly)
1st solution don't start recording on DVR desktop: start recording when DVR switched to full channel output.
other solution and depending where you get your signal from. For the record I'm only speaking FTA reception cable/dvb-t/dvb-s. No idea if this is possible with iptv (internet TV).
I make my recordings currently on Linux HD receiver with an image on it (+- 165 euro). The streams itself are recorded and not the desktop/video output.
The output was generated trying to rescue the mpeg-ts stream, but failed in this case (was even worse). some times this trick does works:
rename file you wan't to save to input.M2TS.
ffmpeg -i input.M2TS -acodec copy -vcodec copy rescue.tsif that fails:
ffmpeg -i input.M2TS -acodec copy -vcodec copy rescue.mkv
QuoteNever said don't record that. Problem is switching to and from dvr desktop and channel watched on dvr. DVR desktop is stream output on itself and completely different to the stream from the program. (if I all understand this correctly)
1st solution don't start recording on DVR desktop: start recording when DVR switched to full channel output.
Well, you ARE saying "don't record that". And I'll concede (even though the capture program can capture all, and WMP can play back okay). I'll start capturing AFTER the DVR-menu has fully switched to playing the real show (skipping the first seconds of the real show). But even there I'm having problems.
Let me back up and report that before starting this thread, I was in contact with Hauppauge customer service. Upon my allegation of lack of I-frames scattered throughout an SD captured file, he asked me to send him a clip and he'd pass it on to the developer. I capped a short "live" segment. I got back a report from them that there WERE I-frames in it. He showed an image from the *expensive* ($1295) StreamEye software. I'm attaching the image here. It shows I-frames present. I loaded the file into AviD and THAT'S when I discovered that AviD was showing some I-frames with the "P" label. It's hard to go forward in getting the capture problem solved until the program I'm using to detect I-frames (and subsequent processing thereby--I-frame editing) can correctly report the frame-type.
Am having real trouble creating a SHORT clip that shows the problem. Yet, every time I record a 4-hour DVR-prerecorded show (leaving it overnight with a time-limit set on the capture so it stops shortly after the show is over), it appears there's no I-frames (beyond the first) and AviD either crashes or a seek to the 2nd keyframe seeks to the end of the file, implying there are no I-frames scattered throughout.
Is there a maximum duration over which AviD cannot correctly process? I have numerous shows over 4-hours that I must capture. And I must have I-frames throughout, not just the 1st frame.
Could you try recording in AVCHD .TS format instead of .M2TS? And see if the problem persists? Hauppauge Colossus Recording formats: AVCHD (.TS and .M2TS) and .MP4.
The problem starts here with last clip when you restart a clip on the DVR while you're are recording. This causes some kind of issue for avidemux (signal break, ....)
Still I was able to pass this point, but this is not workable:
Start of clip: press keyboard arrow up and again: you can't go any further at this point Now press left arrow key, now you can move forward and backward only with left/right keyboard arrows. Using mark A and mark B you can now mark section you want to copy/save.
Hope you have better success with .TS format?
This is all from user perspective! I'm just a avidemux user, that is all.
from 2013-2-20-23-42-4.sample-SD5-clip-from-Tom-Cook
part1: http://rapidshare.com/files/124895466/md5part1test.ts
part2: http://rapidshare.com/files/1356645340/md5part2test.ts
"Restart a clip" on the DVR fits in to the catagorey of "recording the DVR menu and some content in the same captured clip". Not simply "restart", but ANY case of capping both DVR menu and show content. So I'm heeding your advice and NOT starting a capture til after the recorded program is playing.
Right now I'm doing a 2-hr cap following the above new rule, to see if AviD will like it. I'll have to try out your .TS results later when not capping.
But as far as .mp4 vs .ts vs .m2ts, I have wrestled with that TWICE. At first, I followed the advice of one Hauppauge guy who suggested .mp4. I followed that advice for days of capping experimentation. Then learning from here and other places that .mp4 isn't the best for editing, I looked into .ts and .m2ts. Either should be good according to sources which I can't quote. I elected .m2ts as the only difference is extra timecode info, which I intuitively suspect could be helpful to some players. (I figure it NEVER hurts to have timecode info.) So that's why my 2nd evaluation led me to settle on .m2ts for all files. I don't understand why .ts would have any advantage to anything.
Have you looked at my SD3 clip?? I allege that AviD is falsely reporting I-frames to be P-frames. Until that is corrected, we are operating blindly. We need to get that corrected before plodding onward.
IS THERE a time limit on file size with AviD? 1 hr? 2 hr? A half-hour SD capture (skipping the DVR-menu of course) worked. I'm doing a 2-hr cap now.
Mpeg-ts is what is used in broadcasting: dvb-t/dvb-s/s2. M2TS is more geared to blue-ray and more strict (my impression).
4 hour should not be a problem. Don't think I ever personally worked with over 4 hour recordings myself. Think I had once a multiple part 34 GB recording.
Quote from: Support: Colossus: faqThere are three formats which you can choose when recording a video:
.TS which is a generic 'transport stream' compatible with many digital media players
.M2TS which is compatible with the Sony Playstation3
.MP4 which is compatible with the XBox360
Quote from: TCmullet on February 21, 2013, 03:23:16 PM
Have you looked at my SD3 clip?? I allege that AviD is falsely reporting I-frames to be P-frames. Until that is corrected, we are operating blindly. We need to get that corrected before plodding onward.
Avidemux 2.6.1 (avchd and h264) are not using frames as such. Is a time based editor and not a frame based editor.
That may be partially true, Jan. (I realize that a change from frame-based to time-based occurred between 2.5 and 2.6.) But it's still true that "frames" show up in the display window, and the "frame type" shows up to the right of the timecode window. It show I, P, or B depending on whether the *frame* is an I, P, or B. It's important that it show the correct letter, and I am alleging (with proof provided in my SD3 clip) that it is not.
When you say it is a "time based editor" not a "frame based editor", I say it is a GOP based editor. It copies GOPs, which I trust you know is a "group of pictures" that starts with an I-frame. Knowing that I have to edit only at I-frame points (because that's all AviD will allow) makes me frustrated that these SD captures "appear" to not have I-frames scattered throughout. But then Hauppauge tells me that they ARE there and that I'm blind. I say, yes, I am blind b/c AviDemux is incorrectly labelling the I-frames as P-frames. This REALLY REALLY REALLY needs to be corrected as the starting point for further problem solving. I beg the developer to please correct the mislabelled I-frames.
Jan, when you asked if I could try recording in .ts vs .m2ts, I failed to see that you said "AVCHD". I'm not recording HD, but SD. The HDs have worked fine. An I-frame every half second, and the rest are P-frames. It's SD recordings that either don't have more than 1 I-frame or AviDemux is mislabelling the numerous I-frames as P-frames. I plead that the mislabellings be corrected, no matter what other issues are being discussed.
sd3 handles well in 2.6.1 r8483 keyframe every 0.500 seconds. Cuts at start or where need to join: can only be made on the ones selected with up/down keyboard arrow or (https://avidemux.org/smif/proxy.php?request=http%3A%2F%2Fwww.avidemux.org%2FadmWiki%2Flib%2Fexe%2Ffetch.php%3Fmedia%3Dusing%3Aprevious_keyframe-qt.png&hash=da1f5bd5d5a24f06391bc49cc0ed216457f30094) / (https://avidemux.org/smif/proxy.php?request=http%3A%2F%2Fwww.avidemux.org%2FadmWiki%2Flib%2Fexe%2Ffetch.php%3Fmedia%3Dusing%3Anext_keyframe-qt.png&hash=0a172dd91ef16d1d1f12fb5e5ee95cbd6395cb8f)
See also for partly explanation: http://www.avidemux.org/smf/index.php/topic,10608.msg57272.html#msg57272.
Jan, I looked there and found this: "Pseudo IDR=I with recovery=0. As far as navigating is concerned, avidemux uses pseudo IDR as IDR, hence you can use the scrollbar. To display the frame type, avidemux uses the decoder output and the decoder says it is an I (not IDR) which is considered as a P as far as avidemux is concerned".
Apparently I have to learn what "IDR" and "pseudo IDR" are. (Sheesh.) Please help. So what does he mean by "decoder says it's an I which is considered as a P"??
Can you or someone recommend a tool to determine what kind of frames each of the frames is? (If AviDemux won't be repaired to show the true state, like I had greatly hoped.)
Is beyond me to. Has to do with: not all frames are really there, most of frames only contain partial information and use previous/upcoming frames to rebuild picture information over certain time period. .500 seconds for your video, seen already where 1 to 1.5 second interval is appearing.
Don't have windows, so can't check if this will work and show frame info for your SD recording? MPEG-2 Transport Stream packet analyser (http://www.pjdaniel.org.uk/mpeg/)
Have a look to in this part of the forum: Documentation & Tips (http://www.avidemux.org/smf/index.php/board,10.0.html) see there some scripts for batch processing. Don't know if these use same source material as you use.
I still process my video recordings manually with avidemux, so fast no need for re-encoding: remove lead in: mark A at begin of clip, fast forward to 1st block end to cut: up/down arrow for exact cut point, . Ctrl X that block is cut out. Move to next block to remove up/down arrow select begin for cut, mark A, move to end of block to cut, up/down arrow end point, mark B, ctrl X, that block is gone. When finished cutting goto end of video mark B. Save video with copy for both audio/video and 2 minutes later your video is saved. On average 4/5 one hour recordings with 15 minute lead in/out.
Quote from: Jan Gruuthuse on February 21, 2013, 03:03:58 PM
The problem starts here with last clip when you restart a clip on the DVR while you're are recording. This causes some kind of issue for avidemux (signal break, ....)
Still I was able to pass this point, but this is not workable:
Start of clip: press keyboard arrow up and again: you can't go any further at this point Now press left arrow key, now you can move forward and backward only with left/right keyboard arrows. Using mark A and mark B you can now mark section you want to copy/save.
Hope you have better success with .TS format?
This is all from user perspective! I'm just a avidemux user, that is all.
from 2013-2-20-23-42-4.sample-SD5-clip-from-Tom-Cook
part1: http://rapidshare.com/files/124895466/md5part1test.ts
part2: http://rapidshare.com/files/1356645340/md5part2test.ts
I failed to thank you for creating those, Jan. THANK YOU! I had downloaded them and played them. But now I attempted to do your editing myself using the same SD5 clip. (But I used the Mpeg TS muxer, and kept extensions ".M2TS".) Yes, I could "up arrow" (or "next intra frame") from the first intra-frame (which was the first frame in the clip) to the 2nd one (over 11 secs. in). Then I could delete all after that and save the 1st portion. Then appropriate Ctrl-Zs to restore to the beginning, do "next intra frame" again (to the same one as before), delete all BEFORE that and save the 2nd (and last) portion. Then I loaded both saved files into 2 AviDemuxes. They play fine, and more importantly, each part now has MANY "key frames" (labelled P) that AviD WILL jump to!!
So apparently the problems at that 11 second point totally prevented AviD from seeing (or at least getting to) all those "hidden" keyframes, whether they were in the 1st portion (the portion BEFORE the bad transition), or in the 2nd portion.
But I'm agreeing now that as long as I'm capturing digital audio via the optical cable, that I must not record the DVR menu navigation screens (what I think you were referring to as the DVR "desktop").
The Bright House DVR, which is a Cisco Explorer 8640HDC (a very good one I think), generates a seamless transition from menu screen to recorded show, BUT apparently only under the following circumstances:
1. Composite or S-video out and analog audio. It provides a rock solid vertical sync, which has made it possible to always include the menu at the start of the capture with no problem for what I've been using for years; the Dazzle DVC-II Mpeg2 SD capture card. If there was a problem, DVC would have barfed, but they must have a sync generator or TBC right before the outputs. Never been a problem for me, til now getting the Colossus AND optical audio cable. (The Dazzle was strickly analog audio.) I always wanted to cap a few seconds of the DVR menu b/c it gave show ID info. I did this b/c the capture software generated it's own cryptic file names, I'd capture a bunch of files, and I'd want to easily tie the show name, epis.title, and date/time to the file without having to write it down on paper. Later I could load it into editor (Mpeg2VCR), mentally note the title, etc., then quickly exit and RENAME the file. (Editing would be done later.)
2. SOMETIMES it will provide a seamless (and therefore problem-free) transition from DVR menu to show, if the show playing previously was the same SD/HD format as the show about to be captured and audio is wired to capture from analog.
3. SOMETIMES it wil be seamless if same as #2 but the audio is digital (via optical cable) AND it's the same digital format as what was previously playing! (This can be a playback of the very show itself, invoked before going back to the menu again, befoe the capture starts.) But even then, it may not be seamless; it's a gamble.
One problem appears to be that while the Colossus can accept changing audio formats and store both safely in the file with each audio segment's respective video, AviDemux at present does not like that. Therefore I must go to the effort to not mix those audio formats in the same file. Simple solution: always start capture after the DVR menu has disappeared and the desired show is playing. Worse solution, but doable... Give up optical audio and go back to analog. (But I've tasted the formats that optical provides, and "I ain't going back!")
Also, I've done a bit of hunting and learned that it's not just I, P, and B frames any more. There's IDR and possibly more. I think I see that the author wasn't really intending to report frame type, but only give it out as it relates to the programs internal perspective. (To the program, a certain kind of I-frame is like a P-frame, so it outputs the "P".)
The important thing is that maybe a messed up file like this one CAN be salvaged by cutting the two parts apart. THEN the keyframes apear just fine and further editing can be done.
And I willl report to Hauppauge that this mystery seems to be solved.
Avidemux handles multiple audio tracks, don't know if Hauppauge can provide a solution for that? Avidemux 2.6.1: Main Menu: Audio: Select Track.
Understand why you keep .M2TS, (overkill, if you ask me: digital->analog->digital) Still do small recording in .ts format: DVR menu, switching to channel. And see how avidemux 2.6.1 reacts to this recording? Glad things moved forward for you.