"[tsPacketLinear] Refill failed for pid :", then hangs with 100% cpu

Started by mm0359, September 15, 2016, 11:18:46 pm

Previous topic - Next topic

mm0359

Windows: XP, Vista. (win32)
avidemux.exe: since at least v2.6.8, up to v2.6.14.
avidemux_cli.exe: tried 2.6.13 (160915_a067a661879) only.

Loading the following .ts fails : it hangs (cpu=100%) and (after being killed) leaves a +/- 1nn B .ts.idx2.

Issue 1: do not hang.
Issue 2: index the file.

NB: VLC v1.1/v2.2 manage to read it (fine).

mm0359

Sample (0-16 MiB): topic-17154_GD160915_RefillFailed-Hang.ts
Auth: AnyUsername + "Refill failed"


MediaInfo v0.7.88

General
ID                                       : 4040 (0xFC8)
Complete name                            : ...\topic-17154_GD160915_RefillFailed-Hang.ts
Format                                   : MPEG-TS
File size                                : 16.0 MiB
Duration                                 : 1 min 0 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 2 180 kb/s

Video
ID                                       : 2060 (0x80C)
Menu ID                                  : 4040 (0xFC8)
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : Main@L3
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Format settings, GOP                     : M=4, N=24
Format settings, picture structure       : Frame
Codec ID                                 : 27
Duration                                 : 58 s 920 ms
Bit rate                                 : 1 620 kb/s
Width                                    : 720 pixels
Height                                   : 576 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 25.000 FPS
Standard                                 : Component
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Interlaced
Scan type, store method                  : Interleaved fields
Scan order                               : Top Field First
Bits/(Pixel*Frame)                       : 0.156
Stream size                              : 11.4 MiB (71%)
Color range                              : Limited
Color primaries                          : BT.601 PAL
Transfer characteristics                 : BT.470 System B, BT.470 System G
Matrix coefficients                      : BT.601

Audio #1
ID                                       : 2061 (0x80D)
Menu ID                                  : 4040 (0xFC8)
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 2
Mode                                     : Joint stereo
Mode extension                           : MS Stereo
Codec ID                                 : 4
Duration                                 : 59 s 976 ms
Bit rate mode                            : Constant
Bit rate                                 : 192 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 kHz
Compression mode                         : Lossy
Delay relative to video                  : -554 ms
Stream size                              : 1.37 MiB (9%)
Language                                 : French

Audio #2
ID                                       : 2062 (0x80E)
Menu ID                                  : 4040 (0xFC8)
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 2
Mode                                     : Joint stereo
Codec ID                                 : 4
Duration                                 : 1 min 0 s
Bit rate mode                            : Constant
Bit rate                                 : 128 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 kHz
Compression mode                         : Lossy
Delay relative to video                  : -563 ms
Stream size                              : 938 KiB (6%)
Language                                 : English

Audio #3
ID                                       : 2067 (0x813)
Menu ID                                  : 4040 (0xFC8)
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 2
Mode                                     : Joint stereo
Codec ID                                 : 4
Duration                                 : 1 min 0 s
Bit rate mode                            : Constant
Bit rate                                 : 128 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 kHz
Compression mode                         : Lossy
Delay relative to video                  : -595 ms
Stream size                              : 938 KiB (6%)
Language                                 : qad
Language, more info                      : Visual impaired commentary

Text #1
ID                                       : 2068 (0x814)-888
Menu ID                                  : 4040 (0xFC8)
Format                                   : Teletext Subtitle
Language                                 : French
Language, more info                      : For hearing impaired people

Text #2
ID                                       : 2068 (0x814)-889
Menu ID                                  : 4040 (0xFC8)
Format                                   : Teletext Subtitle
Language                                 : French

Menu
ID                                       : 160 (0xA0)
Menu ID                                  : 4040 (0xFC8)
Duration                                 : 1 min 0 s
List                                     : 2060 (0x80C) (AVC) / 2061 (0x80D) (MPEG Audio, French) / 2062 (0x80E) (MPEG Audio, English) / 2067 (0x813) (MPEG Audio, qad) / 2068 (0x814)-888 (Teletext Subtitle, fr) / 2068 (0x814)-889 (Teletext Subtitle, fr)
Language                                 :  / French / English / qad / French / French
Service name                             : C8
Service provider                         : Harmonic
Service type                             : digital television


admlog.txt:

  Avidemux v2.6.14 (0) .
...
[PMT]--Decoding ES Descriptor--
[PMT]          Type=0x1b pid=26c size=0
[PMT]  Adding pid 0x26c (620) , type H264 Video, language=unknown
...
Starting H264 indexer
...
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[tsPacketLinear] Refill failed for pid :26c


mm0359

Whole/tiny file (44 kiB): topic-17154_PpHoo160916_SyncLost-RefillFailed-Hang.ts
Auth: AnyUsername + "SyncLostRefillFailed"


MediaInfo v0.7.88

General
Complete name                            : ...\topic-17154_PpHoo160916_SyncLost-RefillFailed-Hang.ts
Format                                   : MPEG-TS
Format profile                           : No PAT/PMT
File size                                : 44.0 KiB
Duration                                 : 72 ms
Overall bit rate mode                    : Constant
Overall bit rate                         : 5 006 kb/s

Audio
ID                                       : 2011 (0x7DB)
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 2
Mode                                     : Joint stereo
Mode extension                           : MS Stereo
Duration                                 : 72 ms
Bit rate mode                            : Constant
Bit rate                                 : 192 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 kHz
Compression mode                         : Lossy
Stream size                              : 1.69 KiB (4%)


Don't care about indexing, but don't hang : report an error.

admlog.txt:

  Avidemux v2.6.14 (0) .
...
List of found PID:
Pid=2010
Pid=2011
Pid=2017
...
Starting H264 indexer
...
[Ts Demuxer] Pes for Pid =0x2010 does not contain payload start
[tsPacket] Sync lost (0x2)
[tsPacketLinear] Refill failed for pid :7da


mean


mm0359

Quote from: mm0359 on September 15, 2016, 11:18:46 pm
Issue 1: do not hang.


[index/h264] Abort if we reach end of file while searching for a valid startcode
fixed the hang issue :)


  Avidemux v2.6.14 (160922_fcdc702bf47) .
...
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[tsPacketLinear] Refill failed for pid :26c

[TSDemuxer] Failed..
[avsProxy] Cannot open that
...


(Tested the 2 .ts files, with cli and gui, on WinXP.)

mm0359

Quote from: mm0359 on September 15, 2016, 11:18:46 pm
Issue 2: index the file.

NB: VLC v1.1/v2.2 manage to read it (fine).


IDs listed by:
Avidemux v2.6.14_r160927: 620 and 630. (Broken ones!?)
MediaInfo v0.7.88: 2060, 2061, 2062, 2067 and 2068. (Good ones!?)
VLC v2.2.4: 620 and 630, plus 2060, 2061, 2062, 2067 and 2068.


  Avidemux v2.6.14 (160927_ef745605b41) .
...
[TsPacket] Detecting TS/TS2...
[TsPacket] Score : 188:20, 192:1 out of 20
[TsPacket] Probably TS1 (188)...
[getNextPSI] Section Syntax is set to private
[TsDemuxer] Pat : Prg:3681 Pid: 0x0042
<<< PMT : 0/1>>>
[TsDemuxer] Looking for PMT : 0x42
[getNextPSI] Section Syntax is set to private
[TsDemuxer] PCR 0xe26c, len=20
[PMT]--Decoding Program info--
[PMT]            Left : 16 bytes
[PMT]--Decoding ES Descriptor--
[PMT]          Type=0x1b pid=26c size=0
[PMT]  Adding pid 0x26c (620) , type H264 Video, language=unknown
[PMT]          Type=0xf pid=276 size=6
[PMT]     Tag 0xa , len 4,  66 72 61 00
[PMT]  Adding pid 0x276 (630) , type Mpeg AAC ADTS, language=fra
[PMT] Left :0 bytes
[TS Demuxer] Found 2 interesting tracks
[T Demuxer] Kept 2 tracks
[TS Demuxer] Probed...
...


Could Avidemux find the other tracks (too)? And use them (instead)?

Jan Gruuthuse

Avidemux keeps only 4 audio tracks
in avidemux menu Audio: Select Track: Track 1 [v] Enabled -> [Track 0 from video (.....) ] -< can you select more then 2 audio tracks?

pid 2060, 2061, 2062, 2067 and 2068, are these definitely audio tracks or could these be something else like subtitle tracks / teletext, .... streams?

mm0359

Quote from: Jan Gruuthuse on September 30, 2016, 05:22:54 am
Avidemux keeps only 4 audio tracks
in avidemux menu Audio: Select Track: Track 1 [v] Enabled -> [Track 0 from video (.....) ] -< can you select more then 2 audio tracks?


I can't get that far, as this file cannot be opened ("no demuxer").

Quote from: Jan Gruuthuse on September 30, 2016, 05:22:54 am
pid 2060, 2061, 2062, 2067 and 2068, are these definitely audio tracks or could these be something else like subtitle tracks / teletext, .... streams?


1 video + 3 audio + 1 (double) subtitles.
See MediaInfo data.

Jan Gruuthuse

can you record a 10 second original of such mpeg-ts and upload?
ps:- Use a free www.wetransfer.com, dropbox account, mega or similar webservice (free public access, without registration to download your uploaded video) thank you.


Jan Gruuthuse

could not get to the file, until now:
QuotePaint event
Paint event
[TS Demuxer] Probing...
[checkMarker]  [Ts Demuxer] Sync ok :5577 Sync ko :0
  [detectTs]  [TS Demuxer] 188 bytes packet detected
[TSDemuxer] Analyzing file..
Checking if there are several files...
No.

Simple loading:
file: ~/Downloads/topic-17154_GD160915_RefillFailed-Hang.ts, size: 16777216
found 1 files
Done
[TsPacket] Detecting TS/TS2...
[TsPacket] Score : 188:20, 192:1 out of 20
[TsPacket] Probably TS1 (188)...
[getNextPSI]  Section Syntax is set to private
[TsDemuxer] Pat : Prg:3681 Pid: 0x0042
<<< PMT : 0/1>>>
[TsDemuxer] Looking for PMT : 0x42
[getNextPSI]  Section Syntax is set to private
[TsDemuxer] PCR 0xe26c, len=20
[PMT]--Decoding Program info--
[PMT]            Left : 16 bytes
[PMT]--Decoding ES Descriptor--
[PMT]          Type=0x1b pid=26c size=0
[PMT]  Adding pid 0x26c (620) , type H264 Video, language=unknown
[PMT]          Type=0xf pid=276 size=6
[PMT]     Tag 0xa , len 4,  66 72 61 00
[PMT]  Adding pid 0x276 (630) , type Mpeg AAC ADTS, language=fra
[PMT] Left :0 bytes
[TS Demuxer] Found 2 interesting tracks
[T Demuxer] Kept 2 tracks
[TS Demuxer] Probed...

Simple loading:
file: ~/Downloads/topic-17154_GD160915_RefillFailed-Hang.ts, size: 16777216
found 1 files
Done
[TsPacket] Detecting TS/TS2...
[TsPacket] Score : 188:20, 192:1 out of 20
[TsPacket] Probably TS1 (188)...
[Ts Demuxer] Pes for Pid =0x630 does not contain payload start
[Ts Demuxer] Pes for Pid =0x630 does not contain payload start
[Ts Demuxer] Pes for Pid =0x630 does not contain payload start
[tsGetAudioInfo]  Cannot get pes packet for AAC track
[TsIndexer] Audio probed, 0 found, doing video
Starting H264 indexer

Simple loading:
file: ~/Downloads/topic-17154_GD160915_RefillFailed-Hang.ts, size: 16777216
found 1 files
Done
[TsPacket] Detecting TS/TS2...
[TsPacket] Score : 188:20, 192:1 out of 20
[TsPacket] Probably TS1 (188)...
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[Ts Demuxer] Pes for Pid =0x620 does not contain payload start
[tsPacketLinear] Refill failed for pid :26c

[TSDemuxer] Failed..
[flvHeader] Cannot open that
[asfHeader] Cannot open that
[avsProxy] Cannot open that
[probe]   [picHeader] Cannot open that
  [MP4] Cannot open that...
[mxfHeader] Cannot open that (136c0247)
[openDML] Cannot open that
[mkvHeader] Cannot open that
match :76 / 455 (probeSize:1048576)
[PS Demuxer] Not a ps file

Jan Gruuthuse

probably more going on:
2 video streams and a programme stream:

mm0359

@ Jan,

"Same" Avidemux log and VLC data as I posted/referred_to.
Thanks for the confirmation.

Quote from: Jan Gruuthuse on September 30, 2016, 09:44:23 am
2 video streams and a programme stream:


I assume this (very) file was kind of broken when it was recorded (on my tv box).
The only question is: could Avidemux "recover" it, as MediaInfo and VLC do?
(I don't have FFmpeg/others to check how they deal with this file...)

Note that, in VLC, while 620 and 630 are ("partially") listed as "Codec/Streams", they do not appear as "Audio track" and "Video track".
I wish Avidemux would do the same...

NB:
I get a "programme stream" on some of the tv recordings I do.
I assume it is unrelated to this/any issue.

PS:
VirtualDub 1.10.4: "unknown or unsupported file type".

Jan Gruuthuse