Support stream headers

[phunkyfish]

v21.3.0

• Add stream_headers property for the addon: to be appended to the stream URL after '|'

[matthuisman]

with this, kodi dies even earlier when it tries to log the property

2023-10-11 19:22:17.986 T:22168   debug <general>: SECTION:LoadDLL(C:\Kodi\Kodi21\portable_data\addons\inputstream.ffmpegdirect\inputstream.ffmpegdirect.dll)
2023-10-11 19:22:17.988 T:25984   debug <general>: Loading settings for plugin://plugin.video.nz.freeview/?_=play&_play=1&slug=mjh-tvnz-1&_is_live=1&_noresume=.pvr
2023-10-11 19:22:17.988 T:22168   debug <general>: AddOnLog: inputstream.ffmpegdirect: inputstream.ffmpegdirect: SetVideoResolution()
2023-10-11 19:22:17.988 T:22168    info <general>: AddOnLog: inputstream.ffmpegdirect: inputstream.ffmpegdirect: OpenStream() - Num Props: 5
2023-10-11 19:22:17.988 T:22168    info <general>: AddOnLog: inputstream.ffmpegdirect: inputstream.ffmpegdirect property: inputstream.ffmpegdirect.is_realtime_stream = true
2023-10-11 19:22:17.988 T:22168    info <general>: AddOnLog: inputstream.ffmpegdirect: inputstream.ffmpegdirect property: inputstream.ffmpegdirect.manifest_type = hls
2023-10-11 19:22:17.988 T:22168    info <general>: AddOnLog: inputstream.ffmpegdirect: inputstream.ffmpegdirect property: inputstream.ffmpegdirect.open_mode = curl

stream_headers is the next property i set.

Once again, if i change the user-agent - its OK

2023-10-11 19:24:38.639 T:3616     info <general>: AddOnLog: inputstream.ffmpegdirect: inputstream.ffmpegdirect property: inputstream.ffmpegdirect.is_realtime_stream = true
2023-10-11 19:24:38.639 T:3616     info <general>: AddOnLog: inputstream.ffmpegdirect: inputstream.ffmpegdirect property: inputstream.ffmpegdirect.manifest_type = hls
2023-10-11 19:24:38.639 T:3616     info <general>: AddOnLog: inputstream.ffmpegdirect: inputstream.ffmpegdirect property: inputstream.ffmpegdirect.open_mode = curl
2023-10-11 19:24:38.639 T:3616     info <general>: AddOnLog: inputstream.ffmpegdirect: inputstream.ffmpegdirect property: inputstream.ffmpegdirect.stream_headers = referer=&seekable=0&user-agent=ABC
2023-10-11 19:24:38.639 T:3616     info <general>: AddOnLog: inputstream.ffmpegdirect: inputstream.ffmpegdirect property: inputstream.ffmpegdirect.stream_mode = timeshift

[phunkyfish]

That means the bug is nothing to do with GetRedacted(), it’s just to do with simple logging. Very strange.

[rayep]

@phunkyfish I'm trying to use this feature as the movie CDN is blocking the default user-agent sent by FFMPEG (Lavf/60.3.100) - Using Kodi v21-RC1 Windows, FFMPEGDirect addon version - 21.3.4. From the debug logs, I can see the ffmpeg library is skipping the #KODIPROP values which could be the reason.

Please advise.

Debug logs excerpt:
2024-03-11 05:01:46.779 T:24088 info <general>: Creating InputStream 2024-03-11 05:01:46.779 T:31632 debug <general>: Thread BackgroundLoader start, auto delete: false 2024-03-11 05:01:46.779 T:24088 info <general>: Creating Demuxer 2024-03-11 05:01:46.779 T:24088 debug <general>: ffmpeg[0x18818d92530]: Probing hls score:100 size:591 2024-03-11 05:01:46.779 T:24088 debug <general>: ffmpeg[0x18818d92530]: Format hls probed with size=2048 and score=100 2024-03-11 05:01:46.779 T:24088 debug <general>: CDVDDemuxFFmpeg::Open - probing detected format [hls] 2024-03-11 05:01:46.780 T:24088 info <general>: ffmpeg[0x18818d92530]: [hls] Skip ('#EXT-X-VERSION:3') 2024-03-11 05:01:46.780 T:24088 info <general>: ffmpeg[0x18818d92530]: [hls] Skip ('#KODIPROP:inputstream=inputstream.ffmpegdirect') 2024-03-11 05:01:46.780 T:24088 info <general>: ffmpeg[0x18818d92530]: [hls] Skip ('#KODIPROP:inputstream.ffmpegdirect.open_mode=ffmpeg')

HTTP Failure messages:
`
2024-03-11 05:01:46.832 T:24088 debug : ffmpeg[0x18818d92530]: [tcp] Successfully connected to 172.67.178.120 port 443
2024-03-11 05:01:46.890 T:24088 debug : ffmpeg[0x18818d92530]: [https] request: GET /plhq/str/brhtmlv1/e3536425408cf44be0b613f7e9f265fdb47079ab07e01d9812a056e0a262747e60d49cbaee026c9436e31bbc376a6fee%7CUser-Agent=VLC/3.0.20 HTTP/1.1

2024-03-11 05:01:46.890 T:24088 debug : ffmpeg[0x18818d92530]: [https] User-Agent: Lavf/60.3.100

2024-03-11 05:01:46.890 T:24088 debug : ffmpeg[0x18818d92530]: [https] Accept: /

2024-03-11 05:01:46.890 T:24088 debug : ffmpeg[0x18818d92530]: [https] Range: bytes=0-

2024-03-11 05:01:46.890 T:24088 debug : ffmpeg[0x18818d92530]: [https] Connection: keep-alive

2024-03-11 05:01:46.890 T:24088 debug : ffmpeg[0x18818d92530]: [https] Host: str01.strplayallu01.site

2024-03-11 05:01:46.890 T:24088 debug : ffmpeg[0x18818d92530]: [https] Icy-MetaData: 1

2024-03-11 05:01:46.890 T:24088 debug : ffmpeg[0x18818d92530]: [https]

2024-03-11 05:01:46.895 T:25080 debug : ------ Window Init (DialogBusy.xml) ------
2024-03-11 05:01:47.032 T:24088 debug : ffmpeg[0x18818d92530]: [https] header='HTTP/1.1 403 Forbidden'
2024-03-11 05:01:47.032 T:24088 debug : ffmpeg[0x18818d92530]: [https] http_code=403
2024-03-11 05:01:47.032 T:24088 debug : ffmpeg[0x18818d92530]: [https] HTTP error 403 Forbidden
2024-03-11 05:01:47.032 T:24088 debug : ffmpeg[0x18818d92530]: [hls] Failed to open segment 0 of playlist 0
2024-03-11 05:01:47.032 T:24088 debug : ffmpeg[0x18818d92530]: [hls] Segment 0 of playlist 0 failed too many times, skipping
`

M3u8 file content:
#EXTM3U #EXT-X-VERSION:3 #EXT-X-TARGETDURATION:6 #EXT-X-PLAYLIST-TYPE:VOD #KODIPROP:inputstream=inputstream.ffmpegdirect #KODIPROP:inputstream.ffmpegdirect.open_mode=ffmpeg #EXTINF:6, https://str01.strplayallu01.site/plhq/str/brhtmlv1/e3536425408cf44be0b613f7e9f265fdb47079ab07e01d9812a056e0a262747e60d49cbaee026c9436e31bbc376a6fee|User-Agent=VLC/3.0.20 #EXTINF:6, https://str01.strplayallu01.site/plhq/str/brhtmlv1/e3536425408cf44be0b613f7e9f265fdb47079ab07e01d9812a056e0a262747eb36abfb3c1530f58bd6e3d653e248d75ee88692d0affd02763ecad018e5dc47b|User-Agent=VLC/3.0.20 #EXT-X-ENDLIST

CDN is successfully accepting if request has user-agent set to "VLC/3.0.20" - valided the working behavior using ffplay --user_agent VLC/3.0.20 command.

Thank you!

[phunkyfish]

And our you saying you can’t set the user agent directly in the addon settings?

[rayep]

And our you saying you can’t set the user agent directly in the addon settings?

I've set the user agent under addon settings and still got 403 response from CDN for the request made by Kodi and then upon checking the logs I found that It's skipping ffmpegdirect plugin which explains why the user agent change is not working.

The content I'm trying to watch is general VOD (not DRM protected one) so not sure if ffmpegdirect can only be triggered only DRM enabled contents which would be parsed and fed through my python addon..

Logs clearly state that Kodi identified the stream as HLS and chose to skip ffmpegdirect.

I can retest if you confirm the user-agent overwrite is supposed to work even with HLS VOD.

For now, I've added a threaded HTTP relay server capability to my addon which would rewrite the request with custom user agent and that works as expected. 😊

[phunkyfish]

If you want to use ffmpegdirect you need to have KODIPROPs for each M3U entry. There should be no reason for it not to work. Make sure you set a mimetype KODIPROP so it knows it's a HLS stream. Also, setting a user agent should work playing with Kodi or ffmoegdirect, so you shouldn't really need it.

Just checking but you definitely have ffmpegdirect installed?

Note, that this should work fine with unprotected streams, most streams are.

[rayep]

Thank you, @phunkyfish. I've added two KODIPROPs - #KODIPROP:inputstream=inputstream.ffmpegdirect & #KODIPROP:inputstream.ffmpegdirect.open_mode=ffmpeg just looked at the commit ( I'm not very good with C/C++ programs) and saw a reference for "inputstream.ffmpegdirect.stream_headers".

Do I need to add stream_headers as well? Also, KODIPROP need to be added for all M3U entries? 😮 i.e., if M3U8 has 100+ entries then all those entries should be appended with the props?

Attached the M3U8 file
test.zip
I tried where I've added the User-Agent=VALUE by appending it to the stream url with pipe '|' symbol, maybe that's incorrect way of doing things?

Regarding the ffmpegdirect addon installation, yes, I've installed the Omega-RC1 version + installed ffmpegdirect v21.3.4 to test it. Also, can I install just the ffmpegdirect addon 21.x version with 20.x Kodi GA release? If yes, how can I do it? I only see build instructions for Linux & macOS and not for Windows. :(

[phunkyfish]

On windows you just need to install it, it comes from binary add-on repos, Windows, OSX and Android all work this way.

What you have won't work, KODIPROPS only work in the M3U file that iptvsimple loads. M3U8s or HLS streams are like channels and each of those can have KODIPROPS.

Here is an example. I don't think you need stream headers at all. All you need is a correctly formed M3U file

#EXTM3U
#EXTINF:0 ,Channel 1
#KODIPROP:inputstream=inputstream.ffmpegdirect
#KODIPROP:inputstream.ffmpegdirect.open_mode=ffmpeg
#KODIPROP:inputstream.ffmpegdirect.manifest_type=hls
#KODIPROP:mimetype=application/x-mpegURL
https://streamserver.streams.com/channel1.m3u8|User-Agent=VLC/3.0.20
#EXTINF:0 ,Channel 2
#KODIPROP:inputstream=inputstream.ffmpegdirect
#KODIPROP:inputstream.ffmpegdirect.open_mode=ffmpeg
#KODIPROP:inputstream.ffmpegdirect.manifest_type=hls
#KODIPROP:mimetype=application/x-mpegURL
https://streamserver.streams.com/channel2.m3u8|User-Agent=VLC/3.0.20

[rayep]

Thank you, @phunkyfish. Yep that's what I thought too that mine won't work as I'm not using IPTV stream or IPTV client. Mine is a movie stream that I parse from internet service.

So no way to add custom stream headers for movie streams that we feed to Kodi through python addon?

[phunkyfish]

So you call the parent steam or the child segments individually?

you should still be able to set the right properties

[rayep]

So you call the parent steam or the child segments individually?

you should still be able to set the right properties

I pass the master m3u8 link and let Kodi to handle the rest.

Movie site I parse made some changes to their CDN which then started to block stream request made by Kodi (ffmpeg & curl user-agent) so thought of downloading m3u8 and then modified it have the KODIPROP values which doesnt work.

So you're saying that If I add the right properties to all child streams as youve called out in the earlier comment. It should work?

[rayep]

@phunkyfish I'm confused when you said "I don't think you need stream headers at all".. so adding user-agent to the URL with | is not the stream headers? (I thought thats what we call as stream headers)

Maybe my understanding about stream headers are wrong.. 😁 my apologies if I mixed things up.

[phunkyfish]

Sorry, what I meant was the stream headers property in this PR.

using the | after the URL to the master m3u8 file that you pass to kodi should work.

there is no reason to do anything to the child URLs.

[rayep]

@phunkyfish URL that'll get the master m3u8 is not having user agent filtering - it's the child urls that's inside the m3u8, hence I've downloaded the m3u8 file stored it in filesystem and used a regex substitute to add the |User-Agent=VLC/3.0.20 to the end of all child stream URLs and then fed that file path to Kodi for playing - this doesn't work. 😕

[phunkyfish]

Yes, but ffmpeg processes the user-agent on the master m3u8 and will automatically add to any child segments. The child segments will be added in realtime as the stream progresses so adding them to the initial ones is of no use. Does this make sense?

inputstream.adaptive also supports HLS. So if you have no luck with ffmpeg you can always try that. That addon supports the stream headers properly natively so worth trying if ffmpeg is a dead end.