Notes on Qt5, Multimedia, OpenMax Acceleration, 5.1 Audio
setting up build envt
Beginner’s guide to cross-compile Qt5 on RaspberryPi
Multimedia and Proprietary Codecs needed for HW Acceleration
- Complete QtMediaHub. QtMediaHub is our primary vehicle to showcase Qt and QML
- Empower externals to contribute to Qt project
- Have Qt run fantastically on 2 boards (Raspberry Pi and BCM 7425 are the candidates)
- Contribute to missing components in Qt that make it interesting for non-Nokia
*Quote from yakovlev on September 21, 2011, 23:19*
Just to add some information to my own question, from what I've been able to determine.
Raspberry Pi uses an embedded acceleration API known as OpenMAX. This API basically requires a separate codec be supplied in the "binary blob" for each media type.
OpenMAX seems to have limited linux support, mainly with gstreamer and possibly also with VLC. Unfortunately, it sounds like most of the linux media applications use ffmpeg, which lacks OpenMAX backend support. Neither VA-API nor VDPAU are supported.
To make a long story short (too late) the ARM core will hopefully be sufficient for SD video, as that is all that most media applications will have access to. Selected media applications will be able to use the accelerated codecs, but only if each individual codec is licensed by the foundation (included in the binary blob) which will be a significant licensing cost per codec. There is no "generic" back end that can be used to accelerate all video codecs using OpenGL 2.0 (or something similar) primitives.
Also in the thread about 5.1 audio here:
*Quote from liz on October 27, 2011, 22:30*
I have a *lot* to say about the price Dolby demands for licences, which will be going into a blog post later on when we've finalised all the codec licences for the board. In short, the licence we have to buy for Dolby 5.1 takes up 4% of the $25 cost of the board, which seems absolutely obscene when you consider everything else that makes up the other 96%. (Hardware prices have been really manageable, and h.264, which does a hell of a lot more, costs a hell of a lot less - the licence for 5.1 was a real stand-out in terms of price.) It's a racket, and it's particularly infuriating because we know that a lot of our audience won't find the device usable without it.
I can't confirm yet whether or not it'll be on all the devices - we may go so far as to have two separate options you can buy, one with 5.1 and one without, calling out the cost as we go. Whatever we end up doing, I do want to draw attention to just how much of the cost of a unit is sunk in licensing fees. Pretty much all of the hardware manufacturers we've dealt with have been extremely cool, have given us discounts because they want to support what the charity's doing, and have been very easy to work with. Not so the codec licensing guys, who have been absolutely immovable.
I've not seen any updates as to codec licensing but my search-fu on these forums (great content, awful forum interface) is not too strong. Liz is probably the person to ask if you're interested in the current state of play, I'm content to wait until it's all announced. I trust the RasPi team to get us the best deal they can. If that means I've got to pay an extra quid or three seperately to get more GPU accelerated codec support but not to burden others with unnecessary cost, that's cool. If they can include most of it at the already given price-point, even better.
Bellagio OpenMax IL
Various "Bellagio" packages are available in Raspbian:
libomxil-bellagio-dev - implementation of OpenMAX IL, development files
libomxil-bellagio-doc - Documentation of the Bellagio OpenMAX IL
libomxil-bellagio0 - implementation of OpenMAX IL, run-time library
libomxil-bellagio0-components-alsa - ALSA source/sink components for Bellagio OpenMAX IL
libomxil-bellagio0-components-base - components for Bellagio OpenMAX IL
libomxil-bellagio0-components-base-dbg - components for Bellagio OpenMAX IL, debugging symbols
libomxil-bellagio0-components-camera - Motorola Camera components for Bellagio OpenMAX IL
libomxil-bellagio0-components-fbdevsink - Frame Buffer Video Sink components for Bellagio OpenMAX IL
libomxil-bellagio0-components-mad - MAD MP3 decoder components for Bellagio OpenMAX IL
libomxil-bellagio0-components-videosrc - V4L2 video source components for Bellagio OpenMAX IL
libomxil-bellagio0-components-vorbis - vorbis decoder components for Bellagio OpenMAX IL
libomxil-bellagio0-components-xvideo - XVideo components for Bellagio OpenMAX IL
libomxil-bellagio0-dbg - implementation of OpenMAX IL, debugging symbols
first install Bellagio's omxil from, or . Version 0.9.1 is recommended.
Install as usual:
./configure --prefix="$HOME/omx" make make install
Register the components:
You'll get an output like this:
Scanning directory /home/felipec/omx/lib/bellagio/ Scanning openMAX libary /home/felipec/omx/lib/bellagio/libomxvorbis.so Component OMX.st.audio_decoder.ogg.single registered Specific role OMX.st.audio_decoder.ogg.single registered 1 OpenMAX IL ST static components with 1 roles successfully scanned
Make sure you have:
- "OMX.st.audio_decoder.mp3.mad" for MP3 decoding
- "OMX.st.audio_decoder.ogg.single" for Vorbis decoding
- "OMX.st.video_decoder.mpeg4" for MPEG4 video decoding
It's always better to get the latest code:
git clone git://anongit.freedesktop.org/gstreamer/gst-openmax ./autogen.sh --noconfigure ./configure --prefix="$HOME/omx" make install
To actually use the plug-in:
export GST_PLUGIN_PATH=$HOME/omx/lib/gstreamer-0.10 export LD_LIBRARY_PATH=$HOME/omx/lib
You'll be able to play audio and video with MP3, Vorbis, MPEG4, H.264, or H.263 content using Bellagio's components.
For MP3 you'll also need the mp3parse element, otherwise you'll get:
GStreamer-CRITICAL **: gst_segment_clip: assertion `segment->format == format' failed
If you want to see what's happening: