With this week’s headline of “Microsoft ‘worked with Apple’ for Silverlight on iPhoneâ€, we’re once more reminded of the grim reality that is video on the internet. In the article, Microsoft proudly boasts how they use standardized H264-encoded video and a HTML5 video element to serve up video to the iPhone, which would otherwise refuse to play its proprietary content due to lack of a browser plugin.
Since the ages of the first images in web browsers, we’ve been eager to see full-motion video on the web. While it was first normal to let people download video files, often heavily compressed and only playable on specific systems, in the late 1990s, the increase of broadband adoption and standardization of internet protocols allowed streaming video to become commonplace. Flash, Quicktime, Windows Media and Realplayer emerged as proprietary solutions for internet video streaming.
Travel back in time to today, and we’re still seeing a struggle. Flash has become largely dominant thanks to its usage on websites like Youtube, and is almost adequately entrenched to be called a standard. However, when the iPhone was introduced, Google accommodated it by serving up a more open type of video: H264-encoded MP4 files. The interesting thing about this video type is that it’s not just iPhone that supports it; it plays on almost any modern piece of electronics capable of decoding video. The market of desktop video encoding had struggled to find a means of delivering regular (non-streamed) video that could be played on any system, and found MP4 to be an excellent standard.
Back to the article I mentioned earlier. Microsoft is trying to get a piece of the big content delivery cake too, and with Silverlight they are attempting to make something that’s cheaper and more efficient than Flash. Seeing how Flash isn’t even capable of playing video without making my Mac heat up to a comfortable baking oven temperature, I can’t blame them for trying, but what’s so striking about their iPhone-compatibility plugin is that they could easily serve up all their so-called Silverlight-backed videos with the HTML5 video tag and H264-encoded MP4 video files. They won’t, however – they’ll only allow the iPhone to get to the standardized video format. Why?
What we’re seeing is a group of aggressive businesses that want nothing else but becoming the standard for the moving picture on the web. Once settled, they own the way we watch videos online, and thanks to Flash video’s currently overwhelming popularity, you can see how well this works out for us. Linux and Mac users that watch Flash videos online are always confronted with the terribly dysfunctional state of the plugin Adobe offers. In the mean time, Adobe is hard at work to do entirely useless stuff like adding multi-touch to Flash. There is no improvement on the utterly broken state of Flash, because there is no incentive. Until Flash is dethroned as the way most videos are viewed online, we won’t see Adobe even trying to fix the plugins of the less popular platforms or introduce competitive features.
Imagine if this had happened to HTML and the internet standards we take for granted today. Perhaps there would be eight different standards for separating form and content, or stylesheets. Perhaps we’d still require browsers to take a second to determine what type of document was being served up. Quite possibly, 90% of the pages would use a substandard, but ‘good enough’ markup system that was understood by the market-leading browser. Which browser that would’ve been is quite easy to figure out: Microsoft’s Internet Explorer. Let’s not even begin to think how it would’ve been for Linux and Mac-bound users.
If I were you, I’d start using Clicktoflash on your Mac, as I had recommended earlier, and adopt the HTML5 video tag as soon as you can. The future, and thus the fall of the Video Empire, really can’t come soon enough.
Agree wholeheartedly with your post. I cannot wait until HTML5 video is becoming commonplace.
Completely agree, I block Flash in all of my browsers anyway :)
The downside with HTML5 video as it stands is that there is no set standard for which video codec you can depend on the browser supporting without requiring a plug-in.
Firefox and Opera only support Ogg Theora.
Chrome supports Ogg Theora and H.264.
Safari only supports H.264.
Internet Explorer doesn’t even know what HTML5 is yet. (Sounds like it will with version 9 however)
Because of that, the spec has removed stating which codec should be supported and has made the video tag similar to the img tag.
Playing a 720p movie in flash takes 80% (average) of my CPU cycles, playing a 1080p movie in VLC takes only 40% (average).
It is the worst, least efficient and most crashy piece of software I have ever used. It pisses me off. I HATE FLASH, I HATE IT! Thank god Snow Leopard made Flash a separate process.
I use Opera as my main browser and I applaud them for pushing web standards. While HTML5 is gaining acceptance at a pretty good rate, I am really looking forward to the video tag becoming widely accepted as well, FLV conversion could be a task of the past.
Agree! I am using ClicktoFlash since 3 weeks and it is awesome.
This picture is awesome too.
Well written! Thumbs up!
“Travel back in time to today…”
Uh…how exactly does that work? ;)
Silverlight = DRM. Yes, MS wants to own the space, but they want to own it with the content providers happy about DRM as well. (see: Netflix)
Nice to see some people think alike! Yeah, join the movement against senseless Flash use! Actually don’t use it all ;) To bad this website uses Adobe Flash to show video!
First: yes, Flash for video is very inefficient, driving any gearhead nuts. It’s just a video!
Here’s my thing: without the multimedia features (beyond simple video playback) the state of video on the web would be a pale shadow of its current form. Think of all the custom interactivity (really Flashes’ true self) that’s enabled by it with video. Vimeo, viddler (cool annotation UI), Youtube’s post-video menu, etc. wouldn’t exist. I’ll remind you that video online only exploded when Youtube used Flash (read:ubiquitous) which everyone could view without QT, Real, or WMP.
Without flash, we’d be stuck with a scrollbar and some VCR buttons. I’m not saying the tradeoffs aren’t a pain. I hope Adobe will just fix Flash (they could IMHO), and I don’t think Silverlight is all that bad (the Olympics video was incredible).
I do think it’s telling that Fox and ABC et. al. have made custom player plugins for their shows’ playback, which seem to do HD pretty well. Anyone know how those are done (Java, other)? They seem to have it right as to performance.
My main point is that video online is now more than just a box for the stream to play in. Advanced interactivity is expected, demanded now. At the moment, only Flash and Silverlight seem to have that capability (and the tools and scripting languages to go with it).
Great points, Allan. Thanks for bringing something fresh to the discussion.
I agree with Allan White. Without Flash, the special menus on youtube like the new 3D abilities wouldn’t exist, or they would require some heavy (and possibly incompatible among browsers) Javascript. Flash might be slow, and not well-coded internally, but it is what it is and it has helped spearheading the usage of video online.
H.264 is a very kewl codec which Flash uses. The same interactivity can be used with JavaScript you don’t need freakin’ Flash to play it. Thank God HTML5 is moving away from these proprietary plugins, it just declares there is a video to play and Ogg Theora isn’t common good.
Really? Got an example of advanced interactivity with video (incl. things like fullscreen, live scaling, vector graphics, typography, etc.)? I’d love to see it. With the slow rendering of a lot of browser of JS, that might trading one evil for another. Of course, you can do amazing things with JS these days (280slides.com, etc.).
I’d be the first to agree that when a user just wants to play a video – no other interactivity required – a basic “video object” would be nice. But, that’s not the only use case – am I right?
> The interesting thing about [MP4] is that it’s not just iPhone that supports it;
> it plays on almost any modern piece of electronics capable of decoding video.
That is because it is the consumer electronics video standard. It’s essentially an online CD/DVD. The whole point is to be universal or it doesn’t work.
FlashPlayer 9 and FlashPlayer 10 video is also encoded in ISO MPEG-4 H.264/AAC, same as everybody else, so the video codec is not an issue. Anyone who tells you the codec is an issue either doesn’t know about audio video or has a proprietary codec to sell you.
Again: the video codec is not an issue. If you are publishing audio video and you want people to be able to see and hear it, use MPEG-4 H.264/AAC. Other encodings are not consumer video, they cannot be played outside of authoring environments on a Mac or PC. If you use another encoding you are essentially making video that only plays in video editors, not video players. You’re cutting 95% or more of your audience out of the loop.
The only outstanding issue in online video is who makes the video player that plays your inline video in the browser, and what language is the player controlled with? W3C will tell you the browser maker makes the player and it is controlled with HTML5 (which includes JavaScript and CSS, you don’t have to mention them separately) and there are specific API’s for controlling the video playback in HTML5, which runs on everything. Adobe will tell you that they make the player, and it is controlled with ActionScript3, which also has API’s for video playback, but which can only be made with their tools, and which only runs in one app from one vendor and then only on Mac and PC. Adobe is offering very little aside from confusion.
> The downside with HTML5 video as it stands is that there is no set standard
> for which video codec
No, that is not a downside. HTML5 is a Web app standard, it is not an audio video standard. Audio video standardization is outside the scope of HTML5, which describes how to markup video, not how to encode it. It is not now and has never been the job of W3C to choose the chips that go inside an iPod or Blu-Ray player, it is not their job to tell a movie or music maker how to encode their final product. W3C decides that the video tag should be called “video” and what attributes it can have, and how it can be nested to create valid markup, and how a browser should react to that video tag in various contexts. W3C does not have audio video experts, they are not a replacement for Motion Picture Experts Group, aka MPEG. MPEG is to audio video what W3C is to Web apps. MPEG decides what codec will be burned into the video decoder chip in consumer electronics devices, which do not have big PC-style CPU’s and cannot run arbitrary codecs like they are software apps. Just as your DVD player has an MPEG-2 decoder chip in it, your Blu-Ray player has an MPEG-4 decoder chip in it. (There is no MPEG-3 because the name would be confused with MP3, which is either MPEG-1 audio layer 3 if it is under 128 kbits or MPEG-2 audio layer 3 if it is 128 kbits and up.)
And just as there are now passionate debates about how the markup in HTML5 should look by Web developers and browser makers, 10 years ago there were equally passionate debates about MPEG-4 by audio video developers and consumer electronics manufacturers. MPEG-4 was delayed for 6 months at one point because a “content tax” was added, obligating music and audio publishers to pay a fee per item they publish. That is an old scam that technology people keep trying to pull, and Apple refused to implement MP4 until the content tax was removed.
The idea that a bunch of Web developers come in 10 years after MPEG-4 and say no, we’re not participating, we’re not going to play all the existing video from YouTube and iTunes, we’re going to do something else is ridiculous because then you’re saying YouTube is not part of the Web. Google did a study and found that if YouTube were in Ogg codec, the Internet would not have enough bandwidth for them even if they were the only website. Never mind the CPU time required to do that conversion from MPEG-4 to Ogg at this point.
Also, Web developers and W3C have to have humility when it comes to standardization because HTML standardization has always failed, while MPEG standardization has always succeeded. We did not make HTML4 apps during HTML4, we made IE apps. During HTML3 we did not make HTML3 apps, we made Netscape apps. Here we are in MPEG-4 and it’s in both QuickTime and FlashPlayer, it’s in both iPod and Zune, it’s in both YouTube and iTunes, it’s in iPhone and Android and other smartphones, it’s in Blu-Ray and the DVD-ROM’s that have HD video on them. It’s frickin’ everywhere. HTML5 has the potential to be the first truly successful markup standardization. Let’s see W3C and Web developers and browser makers focus on that and if you are successful, you’ll certainly have serious input into MPEG-5 and get it done in such a way that it is even more Web-friendly than MPEG-4.
But the idea that browser makers have something to teach consumer electronics about standardization is a case of STFU if ever there was one.
> FLV conversion could be a task of the past.
It has been a task of the past since 2008 and FlashPlayer 9, which uses MPEG-4 H.264/AAC video. We are on FlashPlayer 10 now and 11 coming soon. You can create one MP4 and serve it to Mac/PC in FlashPlayer and to every other kind of device as a plain hyperlink under the FlashPlayer movie. The one MP4 video file already plays everywhere. The codec is not an issue. The issue is just how the inline video is presented. So you have no excuse for the fact that your video is not playing on iPod and iPhone and other smartphones and media players. They are easy to support along with FlashPlayer with only one source MP4 media file. The time you are waiting for is already here.
> My main point is that video online is now more than just a box for the stream
> to play in.
You’ve hit a key point, but have it backwards by promoting FlashPlayer as the solution.
If you cut a hole in the browser and show FlashPlayer or other plug-in, you always have a box for the video to play in and that is all. The rest of the Web page is another application, a separate environment. In HTML5, you do not cut a hole in the browser to show video, you create a video tag with DOM Scripting just as you would create an img tag or other Web page element, and control it with the same techniques you would use to control a slideshow or rollover button or any other interactivity in the browser. So the video is actually part of the entire Web page. The controls can be anywhere on the page, they can look and feel just like the websites navigation bar and other interactive elements. The video can respond to or trigger any event within the Web app.
So if you want more than just a box playing video, you have to put the video natively into the Web app with HTML5. The first time a Web developer experiments with this and realizes that DOM Scripting now does interactive audio video their eyes light up “wait a second!” there are thousands of untapped possibilities.
> Advanced interactivity is expected, demanded now. At the moment,
> only Flash and Silverlight seem to have that capability (and the tools and scripting
> languages to go with it).
No. HTML5 very specifically has an API for controlling audio and video. That is how you create the play button, the time slider, the progress bar, and other controls that we typically see in a FlashPlayer video player. As I said, because these elements are native within the Web app, they not only interact with the video, but they can interact with any other element of the Web page. You can have the website’s navigation and logo go translucent when “Play” is pushed, you can have all of the elements of the Web page respond in some way to what is going on with the video. It’s all in one environment.
Apple has been criticized for a few years now because the interactivity for QuickTime Player has been getting extremely crufty and there was no attempt to improve it. The reason is that Apple internally decided long ago to support HTML5. Essentially, Apple’s interactive video focus moved from QuickTime Player to Safari. Now, interactive audio video is done just like interactive slideshows and rollover buttons. There are millions of people who can already author it and already have the standard Web development tools.
> Really? Got an example of advanced interactivity with video
All of the video on apple.com is done in HTML5, which contains interactive video API’s. HTML5 also has vector graphics, 3D graphics, and in Safari that is all GPU accelerated so that it currently runs much faster than the equivalent presentation in FlashPlayer while taking less CPU power and less battery life. This stuff is still fresh but the sky is the limit because it’s Web native. All the Web tools and techniques now apply automatically to audio video.
Another example is iTunes LP and iTunes extras. They are all HTML5.
Keep in mind that within FlashPlayer all you have is ActionScript and an API. When you click on a play/pause button in FlashPlayer, an ActionScript event fires and a function is called. When you click on a play/pause button in an HTML5 Web app, a JavaScript event fires and a function is called. With HTML5, this extends to include every element in the page, not just the elements within the FlashPlayer window. In practice it is easy to do many things in HTML5 that are either hard or impossible with FlashPlayer.
> With the slow rendering of a lot of browser of JS
JavaScript in Safari or Chrome is faster than ActionScript in FlashPlayer, and native browser content is GPU-accelerated in many cases and FlashPlayer is not.