This chapter gives a brief overview of OpenMAX IL on the Raspberry Pi. It looks at the component model and the processing model. Further details are given in later chapters.
OpenMAX is designed to play audio and video. It consists of
The preferred programming layer is OpenMAX AL, the application layer. Unfortunately the RPi does not support OpenMAX AL. What it does support is OpenMAX IL, the integration layer. Now this isn't nice to program to, as the following quotes attest:
My own experience backs up these quotes, sadly. I've spent hours staring blankly at a screen, with the specification on one side, the debugger open on the other. I've found that a bottle of red wine helps - not with the programming, but in dealing with the frustration! It's a hard and rocky road ahead...
The basic concept is of a Component, which is an audio/video (or other) processing unit of some type, such as a volume control, a mixer, an output device. Each Component has zero or more input and output ports. Each port has a type, such as an audio port, a video port or a timer port. The type is important: you manipulate different types with different structures. Each port can have one or more buffers that carry data either into or out of a component.
OpenMAX IL components use buffers to carry data. A component will usually process data from an input buffer and place it on an output buffer. This processing is not visible to the API, and so allows vendors to implement components in hardware or software, built on top of other A/V components, etc. OpenMAX IL gives mechanisms for setting and getting parameters of components, for calling standard functions on the components, or for getting data in and out of components.
While some of the OpenMAX IL calls are synchronous, those that require possibly substantial amounts of processing are asynchronous, communicating the results through callback functions. This leads naturally to a multi-threaded processing model, although OpenMAX IL does not visibly use any thread libraries and should be agnostic to how an IL client uses threads. The Broadcom examples for the Raspberry Pi use Broadcom's VideoCore O/S (vcos) threads.
There are two mechanisms for getting data into and out of components. The first is where the IL client makes calls on the component. All components are required to support this mechanism. The second is where a tunnel is set up between two components for data to flow along a shared buffer. A component is not required to support this mechanism, but the Broadcom ones do.
Once you have got everything set up, the typical programming model for OpenMAX is:
There is however and immense amount of detail to each of these steps. It becomes more complex if a "pipeline" of components is involved, such as a decoder feeding into a format converter feeding into a rendering component. However, the principles are basically unchanged.
This chapter has given a brief overview of the architecture of OpenMAX IL. Later chapters will flesh this out in full detail.
Copyright © Jan Newmarch, email@example.com
" Programming AudioVideo on the Raspberry Pi GPU " by Jan Newmarch is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License .
Based on a work at https://jan.newmarch.name/RPi/ .
If you like this book, please contribute using PayPal
Or Flattr me: