Fast-forwarding media support on the Web

A brief history of media on the Web

Media on the Web has gone through three main stages until now. Before the release of HTML5, audio and video were essentially not on the Web, and only available through plug-ins. Plug-ins were widely available on desktop browsers, but smartphones were around the corner, and the Web slowly transitioned to a plug-in free platform.

Then HTML5, first published as a W3C Recommendation in 2014, came along. HTML5 brought media to the Web through the <audio> and <video> tags. Plugins were no longer needed, in theory at least. In practice though, the media industry was willing to stream content on the Web, which was not immediately possible with HTML5 in an interoperable way. HTTP-based adaptive streaming solutions, such as Dynamic Adaptive Streaming over HTTP (MPEG DASH) or HTTP live Streaming (HLS), were developed to improve the user experience when streaming content on the Web, but most HTML5 browsers did not support these mechanisms out of the box. All in all, even though HTML5 brought support for media content, the Web platform remained impractical for professional media usage.

Media Source Extensions (MSE) and Encrypted Media Extensions (EME), published as W3C Recommendations in 2016 and 2017, added the missing blocks to address video on demand (VoD) requirements on the Web, making it possible to implement adaptive streaming mechanisms and to control playback of encrypted content across browsers.

Media on the Web in 2018

A number of use cases, which were arguably of lower priority as long as the above core technologies had not been developed and implemented across browsers, are now moving to the forefront of standardization discussions around media.

Sticking to building blocks, given the heterogeneity of media content supported across platforms, one core technology that is still missing today is the ability for Web applications to tell the media capabilities of the underlying platform (decoding, encoding, and output capabilities for a given format) with enough details to make an optimal decision when picking media content for the user. A solution is being incubated in the Web Platform Incubator Community Group (WICG) with the Media Capabilities specification. That specification should make further progress in 2018.

TV displays, projectors and regular computer displays now support High Dynamic Range (HDR) and Wide Color Gamut content. How can these wider color spaces be supported on the Web platform? How to map luminance levels when mixing HDR and non-HDR content? The Color on the Web Community Group is exploring these questions. The CSS Working Group already started to address extensions to CSS to support other color spaces (CSS Colors Level 4 and CSS Media Queries Level 4).

All embedded media devices (TV sets, set-top boxes, HDMI dongles, smartphones, etc.) have now embraced Web technologies one way or the other, although with varying levels of support for key technologies. making it difficult for content providers to develop media Web applications that run smoothly across media devices. To reduce this fragmentation, the Web Media API Community Group maintains a baseline of Web APIs that are well supported across main Web browsers and that embedded devices should support as well. This effort, done in collaboration with the CTA Wave Project, led to publication of the Web Media API Snapshot 2017 in December 2017. This group may transition to a W3C Working Group this year to produce yearly snapshots of the Web platform for media applications.

Talking about media devices and displays, second screen scenarios are a space to watch in 2018. Users typically own and switch between multiple devices nowadays — for instance, they may discover media content on their smartphones but will want to play back that content on their large TV sets. The Second Screen Community Group incubates an Open Screen Protocol on top of which the Presentation API and the Remote Playback API could be implemented, with a view to providing full interoperability across devices in the longer term.

Last but not least, streaming live linear content on the Web remains a challenge. The first version of MSE and EME addressed Video on Demand needs, but provided only superficial support for live linear content scenarios. New requirements for these specs, such as the need to give Web applications some control over the internal buffering to prioritize low latency over continuity of media playback, are being discussed. They may trigger renewed work on these specifications down the line.

Tracking progress

The Roadmap of Media Technologies for the Web highlights the topics mentioned above, other identified gaps, as well as on-going developments, not mentioned here for brevity — for example, the thorough work on captioning with Timed Text Markup Language (TTML) and the Web Video Text Tracks Format (WebVTT), or extensions needed to support 360° videos, discussed in the Immersive Web Community Group.

The Media & Entertainment Interest Group tracks progress of on-going activities at W3C and elsewhere, and assign priorities of possible standardization efforts at W3C. The group holds monthly calls, focused on a different technology each time. It serves as steering committee for the standardization of media-related features at W3C. Interested parties are welcome to join and help improve media support on the Web!