[PR #225] [CLOSED] Add WHIP/WHEP support for ultra-low-latency streaming, ingest, and playback #228

New issue

Closed

opened 2026-03-02 04:00:49 +03:00 by kerem · 0 comments

kerem commented

2026-03-02 04:00:49 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/eerimoq/moblin/pull/225
Author: @anishmenon
Created: 2/9/2026
Status: ❌ Closed

Base: main ← Head: main

📝 Commits (1)

efbe6d7 feat: add WHIP/WHEP support for streaming, ingest, and playbackAdd WHIP publishing as a stream protocol for ultra-low-latencystreaming to WHIP-compatible endpoints. Add a built-in WHIP serverfor receiving WebRTC streams from OBS/FFmpeg as camera sources.Add a WHEP client for subscribing to remote WebRTC streams.- WHIP publishing alongside RTMP, SRT, RIST in stream settings- WHIP server with ICE negotiation, trickle ICE, session teardown- WHEP client with configurable URL and latency per stream- RTMP-style video decode pipeline for smooth ingest playback- H.264 STAP-A decoding and recvonly track fixes in RTCHaishinKit- WHIP/WHEP streams as selectable camera sources in scenes

📊 Changes

474 files changed (+37253 additions, -1865 deletions)

View changed files

📝 Common/Localizable.xcstrings (+75 -300)
📝 Common/Various/CommonUtils.swift (+9 -0)
📝 Common/Various/Validate.swift (+8 -0)
📝 Moblin Screen Recording/Moblin Screen Recording.entitlements (+1 -1)
📝 Moblin.xcodeproj/project.pbxproj (+44 -10)
📝 Moblin.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved (+10 -1)
📝 Moblin/Info.plist (+27 -31)
📝 Moblin/Media/HaishinKit/Codec/Video/VideoDecoder.swift (+20 -0)
➕ Moblin/Media/HaishinKit/WhipStream.swift (+172 -0)
➕ Moblin/Media/WhepClient/WhepClient.swift (+198 -0)
➕ Moblin/Media/WhipServer/WhipServer.swift (+593 -0)
📝 Moblin/Moblin.entitlements (+21 -21)
📝 Moblin/MoblinApp.swift (+8 -1)
📝 Moblin/RemoteControl/RemoteControl.swift (+3 -3)
📝 Moblin/RemoteControl/Web/js/index.mjs (+2 -2)
📝 Moblin/StreamingPlatforms/YouTube/YouTubeApi.swift (+1 -12)
📝 Moblin/Various/Media.swift (+33 -0)
📝 Moblin/Various/MoblinSettingsUrl.swift (+0 -1)
📝 Moblin/Various/Model/Model.swift (+5 -6)
📝 Moblin/Various/Model/ModelAppleWatch.swift (+290 -17)

...and 80 more files

📄 Description

Summary

Adds full WHIP and WHEP support to Moblin, enabling ultra-low-latency WebRTC streaming in all three directions:

WHIP Publishing (Stream out): Stream from Moblin to any WHIP-compatible endpoint (e.g., Cloudflare, Dolby.io, self-hosted servers) — just like RTMP/SRT but with sub-second latency. Available as a stream protocol option under Settings → Streams.
WHIP Server (Ingest in): Run a local WHIP server so external apps like OBS or FFmpeg can publish streams into Moblin over the local network. Ingested streams appear as selectable camera sources in scenes. Available under Settings → Ingests.
WHEP Client (Playback): Subscribe to remote WebRTC streams via WHEP and use them as camera sources within Moblin. Available under Settings → Ingests.

Motivation

Addresses community feature requests for ultra-low-latency streaming using WebRTC. WHIP/WHEP (RFC 9725 / draft) are the standard HTTP-based signaling protocols for WebRTC, supported by OBS, FFmpeg 8+, Cloudflare, and many other platforms. Compared to RTMP, WHIP delivers significantly lower latency (sub-second vs. 2+ seconds).

What's included

Publishing (stream out):

WHIP as a stream protocol alongside RTMP, SRT, and RIST
Accepts https://, http://, whip://, and whips:// endpoint URLs
H.264 video + Opus audio encoding via WebRTC
Configurable ICE servers and retry settings

Ingest (stream in):

Built-in WHIP server with HTTP endpoint for receiving streams
Full ICE negotiation, trickle ICE (PATCH), and session teardown (DELETE)
RTMP-style video decode pipeline (PTS retiming before decode) for smooth real-time playback
Configurable per-stream latency buffer

Playback (subscribe):

WHEP client for subscribing to external WebRTC sources
Configurable URL, latency, and enable/disable per stream

Shared:

WHIP/WHEP streams appear as selectable camera sources in scene and widget configuration
Vendored HaishinKit.swift with patches for H.264 STAP-A (NAL type 24) decoding and recvonly track lifecycle management in RTCHaishinKit
Decode session auto-recovery on persistent errors

Test plan

Publish from Moblin via WHIP to a WHIP-compatible server and verify playback on the receiving end
Stream from OBS via WHIP to Moblin's WHIP server and verify real-time video/audio in a scene
Subscribe to a remote WebRTC source via WHEP and verify playback as a camera source
Verify trickle ICE works with clients that send candidates via PATCH
Verify session teardown (DELETE) and reconnection for all three modes
Verify WHIP/WHEP camera sources are selectable in scene and widget configuration

The key change is making the three use cases (publish, ingest, playback) explicit and distinct so reviewers and users immediately understand what each one does.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/eerimoq/moblin/pull/225 **Author:** [@anishmenon](https://github.com/anishmenon) **Created:** 2/9/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (1) - [`efbe6d7`](https://github.com/eerimoq/moblin/commit/efbe6d7cc84fac29423b2384727010638e62ffcd) feat: add WHIP/WHEP support for streaming, ingest, and playbackAdd WHIP publishing as a stream protocol for ultra-low-latencystreaming to WHIP-compatible endpoints. Add a built-in WHIP serverfor receiving WebRTC streams from OBS/FFmpeg as camera sources.Add a WHEP client for subscribing to remote WebRTC streams.- WHIP publishing alongside RTMP, SRT, RIST in stream settings- WHIP server with ICE negotiation, trickle ICE, session teardown- WHEP client with configurable URL and latency per stream- RTMP-style video decode pipeline for smooth ingest playback- H.264 STAP-A decoding and recvonly track fixes in RTCHaishinKit- WHIP/WHEP streams as selectable camera sources in scenes ### 📊 Changes **474 files changed** (+37253 additions, -1865 deletions) <details> <summary>View changed files</summary> 📝 `Common/Localizable.xcstrings` (+75 -300) 📝 `Common/Various/CommonUtils.swift` (+9 -0) 📝 `Common/Various/Validate.swift` (+8 -0) 📝 `Moblin Screen Recording/Moblin Screen Recording.entitlements` (+1 -1) 📝 `Moblin.xcodeproj/project.pbxproj` (+44 -10) 📝 `Moblin.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved` (+10 -1) 📝 `Moblin/Info.plist` (+27 -31) 📝 `Moblin/Media/HaishinKit/Codec/Video/VideoDecoder.swift` (+20 -0) ➕ `Moblin/Media/HaishinKit/WhipStream.swift` (+172 -0) ➕ `Moblin/Media/WhepClient/WhepClient.swift` (+198 -0) ➕ `Moblin/Media/WhipServer/WhipServer.swift` (+593 -0) 📝 `Moblin/Moblin.entitlements` (+21 -21) 📝 `Moblin/MoblinApp.swift` (+8 -1) 📝 `Moblin/RemoteControl/RemoteControl.swift` (+3 -3) 📝 `Moblin/RemoteControl/Web/js/index.mjs` (+2 -2) 📝 `Moblin/StreamingPlatforms/YouTube/YouTubeApi.swift` (+1 -12) 📝 `Moblin/Various/Media.swift` (+33 -0) 📝 `Moblin/Various/MoblinSettingsUrl.swift` (+0 -1) 📝 `Moblin/Various/Model/Model.swift` (+5 -6) 📝 `Moblin/Various/Model/ModelAppleWatch.swift` (+290 -17) _...and 80 more files_ </details> ### 📄 Description ## Summary Adds full WHIP and WHEP support to Moblin, enabling ultra-low-latency WebRTC streaming in all three directions: - **WHIP Publishing (Stream out):** Stream from Moblin to any WHIP-compatible endpoint (e.g., Cloudflare, Dolby.io, self-hosted servers) — just like RTMP/SRT but with sub-second latency. Available as a stream protocol option under **Settings → Streams**. - **WHIP Server (Ingest in):** Run a local WHIP server so external apps like OBS or FFmpeg can publish streams into Moblin over the local network. Ingested streams appear as selectable camera sources in scenes. Available under **Settings → Ingests**. - **WHEP Client (Playback):** Subscribe to remote WebRTC streams via WHEP and use them as camera sources within Moblin. Available under **Settings → Ingests**. ## Motivation Addresses community feature requests for ultra-low-latency streaming using WebRTC. WHIP/WHEP (RFC 9725 / draft) are the standard HTTP-based signaling protocols for WebRTC, supported by OBS, FFmpeg 8+, Cloudflare, and many other platforms. Compared to RTMP, WHIP delivers significantly lower latency (sub-second vs. 2+ seconds). ## What's included **Publishing (stream out):** - WHIP as a stream protocol alongside RTMP, SRT, and RIST - Accepts `https://`, `http://`, `whip://`, and `whips://` endpoint URLs - H.264 video + Opus audio encoding via WebRTC - Configurable ICE servers and retry settings **Ingest (stream in):** - Built-in WHIP server with HTTP endpoint for receiving streams - Full ICE negotiation, trickle ICE (PATCH), and session teardown (DELETE) - RTMP-style video decode pipeline (PTS retiming before decode) for smooth real-time playback - Configurable per-stream latency buffer **Playback (subscribe):** - WHEP client for subscribing to external WebRTC sources - Configurable URL, latency, and enable/disable per stream **Shared:** - WHIP/WHEP streams appear as selectable camera sources in scene and widget configuration - Vendored HaishinKit.swift with patches for H.264 STAP-A (NAL type 24) decoding and recvonly track lifecycle management in RTCHaishinKit - Decode session auto-recovery on persistent errors ## Test plan - [ ] Publish from Moblin via WHIP to a WHIP-compatible server and verify playback on the receiving end - [ ] Stream from OBS via WHIP to Moblin's WHIP server and verify real-time video/audio in a scene - [ ] Subscribe to a remote WebRTC source via WHEP and verify playback as a camera source - [ ] Verify trickle ICE works with clients that send candidates via PATCH - [ ] Verify session teardown (DELETE) and reconnection for all three modes - [ ] Verify WHIP/WHEP camera sources are selectable in scene and widget configuration --- The key change is making the three use cases (publish, ingest, playback) explicit and distinct so reviewers and users immediately understand what each one does. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>