H.264 hardware decoding in Mac OS X

ArsTechnica recently asked Adobe if we would use the recently added video acceleration API in Mac OS X 10.6.3. The answer was yes and today we are making it available as a beta version (it was also a feature included in the release candidates builds we have been making available on labs.adobe.com since April 7th 2010, though not turned on; more details at the end).

Background

The primary reason this API exists is because we have been working with Apple to come up with a way to reduce power consumption on Macs. As we see more and more HD content on the web it is critical that we take advantage of hardware resources when available. In that context it is important to understand that this API targets HD content, not SD or smaller sized video. In fact SD sized content will not be accelerated in most cases. The decision of what content is accelerated and on which machine it is supported is up to Apple.

Machine and OS support

The new video acceleration API is only available in Mac OS X 10.6.3 or later and is limited to GPUs models such as NVIDIA GeForce 9400M, GeForce 320M or GeForce GT 330M. For more details you can look at Apple’s technote. Here is a list of the Mac models currently supported:

  • MacBooks shipped after January 21st, 2009
  • Mac Minis shipped after March 3rd, 2009
  • MacBook Pros shipped after October 14th, 2008
  • iMacs which shipped after the first quarter of 2009

(Mac Pros are not supported as of today)

How do I know that it is working?

After you install the new beta version of Adobe Flash Player (code named “Gala”) and play a video you will sometimes notice a white rectangle overlaying the video. This is the sign that hardware decoding is currently active.

If the white rectangle is missing, Adobe Flash Player has reverted to software decoding. We will of course remove the white rectangle for the final release.

What are the limitations right now?

  • Some resolutions are not supported. Specifically YouTube does sometimes provide a resolution of 864 * 480 pixels for their 480p content which forces a software fallback.
  • Resolutions smaller than 480 * 320 pixels are not accelerated on NVIDIA GeForce 9400M based Macs. On NVidia GeForce 320M and GeForce GT 330M the threshold can be a bit higher. These choices are picked by Apple and balance power usage of the CPU vs. GPU for their particular hardware. Remember that using the GPU for video decoding does not always result in overall power savings. This is something you can only decide on based on the exact type of hardware combination and the content you are trying to play. Playing video has a fixed baseline cost in the GPU for instance which is not the case when you decode on a CPU.
  • The software decoder in Adobe Flash Player is more forgiving when it comes to improperly encoded video files, it works around many issues. The hardware decoder can not handle some of these cases. You might notice that some videos will have ‘jumpy’ frames, i.e. frames are out of order (we have seen that with some files uploaded to YouTube). This is usually because Composition Time Offsets are not properly set up.
  • The hardware decoder is limited to 2 instances at a time. This limit is total to the system. If you have more than 2 videos open at the time the 3rd one will fall back to software decoding. This is even the case when a video is on a hidden tab (This is another reason that hardware decoding is reserved for high resolutions).
  • In the current release of Mac OS X 10.6.3 hardware accelerated decoding will sometimes stop to function until you restart Safari. We are in process to resolve this issue with Apple. But if you can reproduce this consistently with a specific URL please let us know.

Safari and Performance

Compared to QuickTime based video playback support in Safari 4.0.x on Mac OS X 10.6.3 (or your standalone VLC/QuickTime player that is) there is still room for improvement in Flash Player. We have a good plan on how to proceed, which will allow us to leverage all the hardware resources available to us.

Video playback is generally hardware accelerated on two levels: 1. Decoding H.264 bit streams itself and 2. Displaying & scaling the decoded YUV12 formatted video frames. The new API provided by Apple only covers H.264 decoding and we are well aware that we need to accelerate the display and scaling of video. CAOpenGLLayer is the vehicle for that. We are looking at how we can get this implemented soon, but it’s simply too late to include this into Flash Player 10.1.

Previous release candidates

As some have noticed, previous release candidates we have made available on labs.adobe.com referenced this hardware decoding API provided by Apple. We are not in a position yet to enable this by default (hence the extra beta version we are making available) as this has only seen very limited testing by the engineers. Because of some of the issues I mentioned above, we want to put the hardware acceleration functionality through a full public beta cycle before including it in a final shipping version of Flash Player.

If you decide to install this beta version please let us know if you encounter any issues and file bugs here.

Leave a comment

112 Comments.

  1. Hi Tinic, are there any browser dependencies? You mention only Safari above, and the Labs page doesn’t seem to have any info.

    Other dependencies are this March’s Mac OS X 10.6.3, atop (basically) any MacBook shipped during 2009 or later. Correct?

    Is there a test page that we know should invoke the acceleration, for quick diagnostics?

    tx, jd

    [TU: Should work in all browsers since this does not depend on hardware blitting]

  2. Brilliant, finally Apple have decided to not be so anti-flash and actually help you – help us end users.

  3. I had Chrome running but the installer didn’t prompt me to quit it. After manually restarting Chrome, it still had my previous version installed.

    It does work in Safari and Opera, though.

  4. What about the nvidia GT-130 on the 24in 3.0Ghz iMaC ????

  5. Any idea why this isn’t working on BBC iPlayer’s HD content? The rectangle shows on their ident’s before HD programmes, but not on the actual content. The streams are both 3.2MBit h264. Does this not work on DRM wrapped video or something?

    • BBC iPlayer content is not visible to us here in the US. In a test/guest site they set up for us a few months back, it appears that content encoded in the last month or so is interlaced. We divert to SW decode mode for interlaced content. Interlaced encoding options are complex and we feel not well supported on a wide range of drivers. Sites really shouldn’t be using interlaced formats anyway, it just creates more unecessary cpu overhead.

  6. good job, but notice that theres a macbook of 2008 with a 9400M until it has been renamed in macbook pro in 2009!

  7. wow this movie is well written and intelligent . Spike Jonze brings the tale to life in the movie … http://tinyurl.com/WherethewildthingsareYT

  8. Nice work! Now the 1080p content on YouTube is actually usable :D

  9. Any hope that GeForce 8600M GT would be supported ?

  10. It would be interesting that Adobe publishes some benchmark results regarding power consumption with or without GPU hardware acceleration depending on video size, MAC model, graphic card. Even a simple multimeter between the computer and the power outlet is relevant for an overall computer power consumption overview.

  11. Playing eg. Scrabble on Facebook is still a CPU hog on the Mac, and this isn’t solved by accessing the hardware decode API. It sets my fans going as often as when playing World of Warcraft – no exaggeration. Sorry, but this change is a drop in the ocean, and makes little difference in the grand scheme of Flash implementation.

  12. This is cool stuff. I wish Apple was able to use ATI HD2600 hardware or higher for hardware acceleration… But it seems Apple is not able to support their own hardware. HD2600 also supports hardware accelerated H.264 decoding.

  13. Interesting post. I’ve just implemented the video acceleration API in the Plex Media Center, and will use some of your technical insights to tidy up the implementation a bit. Thanks very much.

  14. Good stuff! focus on Flash, not on iPhone or whatever is the current hype.

  15. This is indeed good. Agree with Sirhc though that I can see no reason why this would be supported on the short-lived “Aluminium MacBook” which was available between 10/08 and 06/09 (MB466LL/A and MB467LL/A). These had the 9400M as well.

  16. Machines supported….. iMacs which shipped after the first quarter of 2009

    So, my 27″ Core i7 iMac with a Radeon 4850 in it (not mentioned in the list of supported GPU’s) mysteriously is supported?

  17. IntrepidSilence

    This also works on the nVidia 9600M as well (tested). Adobe should make sure to list every card that is supported, not just a few or they are going to get bombarded with questions.

    • The 9600M isn’t supported, but the 9400M of your MacBook Pro can still be used for decoding when the graphic settings are set to performance mode.

      • Then…. how come the little square box is shown as I’m watching flash videos using 9600M?

  18. not all imacs after 2009 support this feature. 27″ versions come with ati chips.

  19. Great news! Thanks for getting it out so quickly.

  20. repetitive audio feedback when player video from dailyshow.com

  21. Thanks for the insight.

    The higher end 15″ and 17″ MBP offer both the 9400M and the 9600M, with the 9600M available if you want to use discrete graphics and want more video performance.

    If I am reading your post (and Apple’s technote) correctly, I will have to step down to the 9400M to get hardware decoding? If so, that seems a little backward, even though it appears it’s out of your control :sad:

  22. Mac Pros aren’t supported? I don’t get it. Mac Pros typically have the best video cards of any Mac. They are also the machines most Mac Flash developers use. Why would you not support this line?

    • The API isn’t even a month old, I’m sure Apple will add support for your GPU eventually.

    • Mac Pros have enough CPU cores that you can just dedicate one for Flash. No need for GPU decoding. (Just kidding… :smile: )

  23. Has anybody really got a free iPad? I just saw this on Twitter: Want a FREE iPad? I just got mine! Enter Your Zip Code Now @ http://x.vu/27624

  24. This is getting a bit more subjective, but I much prefer the Zune Marketplace. The interface is colorful, has more flair, and some cool features like ‘Mixview’ that let you quickly see related albums, songs, or other users related to what you’re listening to. Clicking on one of those will center on that item, and another set of “neighbors” will come into view, allowing you to navigate around exploring by similar artists, songs, or users. Speaking of users, the Zune “Social” is also great fun, letting you find others with shared tastes and becoming friends with them. You then can listen to a playlist created based on an amalgamation of what all your friends are listening to, which is also enjoyable. Those concerned with privacy will be relieved to know you can prevent the public from seeing your personal listening habits if you so choose.

  25. Thank you for your help!

  26. Hi there I have troubles adding this blog page to my RSS reader. My browser is K-Melon, could that be causing the trouble?

  27. Great news. To my understanding, this will then be the state of hardware accelleration as of the launch of 10.1. Is that correct?

    *) Windows: hardware decoding / hardware scaling in fullscreen.
    *) MAC: hardware decoding / hardware scaling in fullscreen.
    *) Linux: no hardware decoding / no hardware scaling.

    Also, do you have a ballpark in terms of performance improvements? We’re looking at 50% less CPU usage for a 480p / 720p video, or 10%?

  28. Well I thought there might be some performance hit by not getting RGB directly (accelerated) thus needing to do color conversion in software / OGL. But thankfully it’s the bitstream decoding that’s the hardest for H.264 resource wise.

    Thanks for sharing, good to see that you worked with Apple on this and got it out as soon as Apple supported it. Looks fairly limited to VP3 at the present though.

  29. Please make the Flash Player Open Source and integrate it to the webkit engine … ! Then all Flash wars will stop and you will benefit from the power of the community… Flash SWF as fixed mime type for all browsers, integrated into their code.

  30. Interesting comments. I’ve recently implemented the VDADecoder API for XBMC for Mac and so far it’s been very stable and quite the performance boost for 9400M based OSX boxes. I have one of the 2008 Al MacBooks with the 9400M chipset and it’s very nice. About time Apple released an API for accelerated video decode :) Now let’s expand it to mpeg and VC1. Nvidia’s VP3 internal API can handle both of those under Windows and Linux (vdpau).

  31. It does seem that everybody is into this kind of stuff lately. Don’t really understand it though, but thanks for trying to explain it. Appreciate you shedding light into this matter. Keep it up

  32. Great post from an expert and it will be a great knowledge to us and thank you very much for sharing this valuable information with us.

  33. ajG8VS tfnmhixtpvrn, [url=http://fdvrocpfidti.com/]fdvrocpfidti[/url], [link=http://enqswcwbcqhz.com/]enqswcwbcqhz[/link], http://pvayqcerpzzc.com/

  34. Siggi Schwarz

    Ahhh, I see! That’s also why Youtube videos run in fullscreen smoothly on an ancient Pentium 3/GForce MX400 in Windows 98 but stutter on an Athlon X2/7900GS in Linux ???

    Maybe the API issue is a part of the problem, but only a VERY small one.

  35. Replies are closed for the previous entry, so here goes:

    On Windows, Flash calls timeBeginPeriod(1). This drains battery like crazy. When there are no active instances (i.e. only background running at 8fps tops), it’d be super-nice for it to call timeEndPeriod(1).

    Thanks for your work!

  36. javascript:grin(‘:!:’)Note BBC have stated HD output is progressive, and re-confirmed this.
    This information is from the official BBC iPlayer messageboard. (Which also contains a post from Chris nuuja of Adobe FlashPlayer):

    Our engineer who signs off on the iPlayer profiles also confirmed that its progressive scan @25fps. That is, the 1280x720p profile has been confirmed in both the encoder and the encoded files. http://www.bbc.co.uk/dna/mbiplayer/NF13735683?thread=7461203&skip=0#p95356179
    Readers may also be interested in “HD Streaming has something changed?”thread on bbc messageboard

  37. The recent Gala preview player crashes my Mac after a while. Seems it has a lot of memory leaks… After a while the flash webpages, especially with video are not usable anymore / the browser hangs. I also cannot reuse the current stable version of the Flash Player, because all flash websites that use wmode “direct” will not work anymore. It seems that the 10.1 rc2 and gala change the flash player config or are not deinstalled properly with Adobe flash player uninstaller.

    Please fix it. Thanks a lot. Keep up the great work. Make Flash the no 1.

  38. Hello

    the new recent 10.1 rc4 is unbelievable slow on Mac Firefox. Scrolling is amost impossible now if a flash movie is opened. Plase check it out with flash movies playing at 60 fps. Thanks.

  39. Hello,

    The 10.1 rc4 flash player is out right now, is there any release note for new things on this version published yet? It will be better if there’s a new blog post for this rc :grin:

    Oh, and i tested it on windows7 64bit machine with chrome as default browser, nothing unusual so far.

    But from comments before me, looks like there are problems on Mac machine.

  40. The new 10.1 rc5 for OSX 10.6.3 makes Webkit stutter while scrolling a webpage. Also the cpu usage has increased. I read at Adobe the 10.1 version will improve performance, not degrade it. Did I misunderstand it?

Trackbacks and Pingbacks: