Upto: Table of Contents of full book "Programming and Using Linux Sound"

Using the Raspberry Pi's GPU with OpenMAX

The Raspberry Pi (RPi) has a very underpowered CPU but a very powerful GPU. Using the GPU is not well documented. This chapter looks at drawing, rendering images, etc, using the GPU.

Resources

Acknowledgements

In many ways this chapter is a walk-through of some of the examples in the RPi's /opt/vc/src/hello_pi directory. These are written by a variety of authors, too many to acknowledge individually. Thanks to all of you!

OpenMAX

We discussed OpenMAX in general in the chapter OpenMAX, applying it to sound on the RPi and other systems. OpenMAX is an extremely difficult API to work with - if you have any sense you will walk away right now. By dom: "I have written a fair bit of openmax client code and find it very hard. You have to get an awful lot right before you get anything useful out. Just lots of OMX_ErrorInvalidState, and OMX_ErrorBadParameter messages if you are lucky. Nothing happening at all if you are not..." dom is quite correct - I estimate that my productivity in dealing with this API is done from the proverbial 10 lines per day to less than 10 lines per week. And lots of that is staring at the screen in complete bewilderment as to why it isn't working, with no error messages or anything to tell me what is going on (or isn't).

Compilation

Just to add a wrinkle to OpenMAX's complexity, you have to be careful to get all the flags correct in compiling programs - or, guess what? - things don't work then either. I suspect it is things like word-alignment that are critical, but I've haven't checked OpenMAX in detail. Here are what seems to work:

	
INCLUDES = -DSTANDALONE -D__STDC_CONSTANT_MACROS -D__STDC_LIMIT_MACROS -DTARGET_POSIX -D_LINUX -fPIC \
    -DPIC -D_REENTRANT -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -U_FORTIFY_SOURCE -Wall -g \
    -DHAVE_LIBOPENMAX=2 -DOMX -DOMX_SKIP64BIT -ftree-vectorize -pipe -DUSE_EXTERNAL_OMX -DHAVE_LIBBCM_HOST \
    -DUSE_EXTERNAL_LIBBCM_HOST -DUSE_VCHIQ_ARM -Wno-psabi -I/opt/vc/include/ -I/opt/vc/include/interface/vcos/pthreads \
    -I/opt/vc/include/interface/vmcs_host/linux -I./ -I/opt/vc/src/hello_pi/libs/ilclient -I/opt/vc/src/hello_pi/libs/vgfont

LDFLAGS = -Wl,--whole-archive -lilclient -L/opt/vc/lib/ -lopenmaxil -lbcm_host -lvcos -lvchiq_arm \
    -lpthread -lrt -L/opt/vc/src/hello_pi/libs/ilclient -L/opt/vc/src/hello_pi/libs/vgfont \
    -Wl,--no-whole-archive -rdynamic
	
      

You are welcome to try reducing the number of flags - good luck!

State changes

Execution of an OpenMAX component is driven by a state-transition model. A component will start in the Loaded state, transition to the Idle state and then to the Executing state. At least, that is the idea. In practice, the state transitions will only occur for a component when conditions are exactly right for that component.

The hardest transition to get right is from Loaded to Idle. Any enabled component has ports which have to be set up and if that is not done exactly right then nothing will happen. Being exactly right means that all of the resources such as buffers have to be allocated and configured correctly, which is often not so easy. So most programmers seem to default to disabling all of a component's ports at the beginning in order that this transition will occur, and then try enabling them afterwards. [The dramatic over-emphasis in this section is the result of bitter experience, both by myself and - I am sure - all of the other OpenMAX programmers!]

The ilclient library

The RPI has a library ilclient intended to make things easier. This library is not directly portable - it relies on VideoCore VC threads. This could presumably be replaced. The primary assistance that it appears to give to me is:

That sounds awful, essentially turning concurrent programming into sequential programming, but in fact it doesn't matter much: much of the execution flow is just waiting in one thread or another, and this just reduces the number of threads you have to think of... It's also a protective style of programming: by call and wait at least you have some chance of pinning down where your program has silently ground to a halt.

For example, a port for a component may be in disabled state. You can't allocate buffers for it until a call has been made to enable it. Then buffers can be allocated. After that (assuming no other issues) the port should be able to transition to enabled state. So wait for that to occur, to ensure that it does. Typical code is

	
    // enable output port of decoder
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandPortEnable,
		    decoder->imageDecoder->outPort, NULL);

    // and allocate a buffer
    int             ret = OMX_AllocateBuffer(decoder->imageDecoder->handle,
					     &decoder->pOutputBufferHeader,
					     decoder->imageDecoder->
					     outPort,
					     NULL,
					     portdef.nBufferSize);
    printf("Output port buffer allocated\n");

    if (ret != OMX_ErrorNone) {
	perror("Eror allocating buffer");
	return OMXJPEG_ERROR_MEMORY;
    }

    // wait for enable to complete
    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete,
			    OMX_CommandPortEnable, 1,
			    decoder->imageDecoder->outPort, 1, 0,
			    TIMEOUT_MS);
    printf("Decoder output port enabled\n");
	
      

ilclient errors

Calls to the ilclient library will sometimes throw error messages such as this

	
assertion failure:ilclient.c:747:ilclient_change_component_state():error == OMX_ErrorNone

Program received signal SIGABRT, Aborted.
0xb6e41bfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
	
      

This doesn't tell you where in your code the call was made, and gives a totally useless error message.

Okay (you may think), run it inside a debugger such as gdb and when it stops, ask for a backtrace:

	
(gdb) where
#0  0xb6e41bfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
#1  0xb6e4597c in abort () from /lib/arm-linux-gnueabihf/libc.so.6
#2  0xb6f825a4 in ?? () from /opt/vc/lib/libvcos.so
#3  0xb6f825a4 in ?? () from /opt/vc/lib/libvcos.so
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) 
	
      

Ho hum, no luck there! You need to run the program inside the debugger to figure out which ilclient call broke, stepping past each ilclient call until one of them breaks. In this case, it was a call to ilclient_change_component_state (about the fifth such call I had made). Then run the program again, this time stepping into the offending call. Then you can see the details of each call made: in my case I was trying to change state with insufficient resources set:

	
(gdb) 
365	    ilclient_change_component_state(decoder->imageRender->component,
(gdb) step
ilclient_change_component_state (comp=0x4b150, state=OMX_StateExecuting)
    at ilclient.c:746
746	   error = OMX_SendCommand(comp->comp, OMX_CommandStateSet, state, NULL);
(gdb) next
747	   vc_assert(error == OMX_ErrorNone);
(gdb) print error
$1 = OMX_ErrorInsufficientResources
	
      

That's very tedious. Welcome to OpenMAX programming.

Creating a component

The ilclient library has a convenience function to create a component, ilclient_create_component. This takes four parameters. The first is an ILCLIENT_T which an application gets by a call to ilclient_init(). The second is the address to store the component.

The third is a 'simple' form of the component's name. Now this isn't the name used in the OpenMAX specification. And it isn't the actual name of the component. For an image decoder object these are image_decoder and OMX.broadcom.image_decode respectively. The name used is formed by dropping OMX.broadcom. from the component's name, to give image_decode. The full list of Broadcom components is at VMCS-X OpenMAX IL Components and this lists the names used.

The fourth parameter is a bit-wise OR of several flags. A major one is ILCLIENT_DISABLE_ALL_PORTS which - as might be expected - disables all ports of the component, to make it easier to transition to idle state later. Two other significant flags are ILCLIENT_ENABLE_INPUT_BUFFERS and ILCLIENT_ENABLE_OUTPUT_BUFFERS. Now these don't actually enable any buffers for the ports, create buffers, or apparently do anything really useful. However, if you don't enable the appropriate buffers here and later try to actually enable them using an OMX_CommandPortEnable command, then the library will throw a cryptic error message complaining about an illegal operation. I suppose that is trying to ensure that you only make valid calls for that component, but I wish the error was not so un-helpful.

A typical component creation will look like

	
 ilclient_create_component(decoder->client,
			&decoder->imageDecoder->component,
			"image_decode",
			ILCLIENT_DISABLE_ALL_PORTS
			|
			ILCLIENT_ENABLE_INPUT_BUFFERS
			|
			ILCLIENT_ENABLE_OUTPUT_BUFFERS)
	
      

Decoding a JPEG image

OpenMAX on the RPi has a standard component OMX.broadcom.image_decode. The RPi documentation for it is here . It has two ports. The input port has (3) buffers to take a JPEG image, and one output port for the decoded image. The input buffers have a default size, and if the image is large you just cycle through them, filling and emptying each buffer in turn.

The program starts off fairly easily in main by reading the JPEG file into a byte-buffer of the right size. The call to bcm_host_init is required to initialise the Broadcom libraries. The JPEG image decoder is then created and asked to decode the image.

	
int
main(int argc, char *argv[])
{
    OPENMAX_JPEG_DECODER *pDecoder;
    char           *sourceImage;
    size_t          imageSize;
    int             s;
    if (argc < 2) {
	printf("Usage: %s <filename>\n", argv[0]);
	return -1;
    }
    FILE           *fp = fopen(argv[1], "rb");
    if (!fp) {
	printf("File %s not found.\n", argv[1]);
    }
    fseek(fp, 0L, SEEK_END);
    imageSize = ftell(fp);
    fseek(fp, 0L, SEEK_SET);
    sourceImage = malloc(imageSize);
    assert(sourceImage != NULL);
    s = fread(sourceImage, 1, imageSize, fp);
    assert(s == imageSize);
    fclose(fp);
    bcm_host_init();
    s = setupOpenMaxJpegDecoder(&pDecoder);
    assert(s == 0);
    s = decodeImage(pDecoder, sourceImage, imageSize);
    assert(s == 0);
    cleanup(pDecoder);
    free(sourceImage);
    return 0;
}
	
      

The call to setupOpenMaxJpegDecoder builds some data structures and calls to prepareImageDecoder to initialise the decoder and startupImageDecoder to move it into executing state, so that it can then decode the image.

	
int
setupOpenMaxJpegDecoder(OPENMAX_JPEG_DECODER ** pDecoder)
{
    *pDecoder = malloc(sizeof(OPENMAX_JPEG_DECODER));
    if (pDecoder[0] == NULL) {
	perror("malloc decoder");
	return OMXJPEG_ERROR_MEMORY;
    }
    memset(*pDecoder, 0, sizeof(OPENMAX_JPEG_DECODER));

    if ((pDecoder[0]->client = ilclient_init()) == NULL) {
	perror("ilclient_init");
	return OMXJPEG_ERROR_ILCLIENT_INIT;
    }

    if (OMX_Init() != OMX_ErrorNone) {
	ilclient_destroy(pDecoder[0]->client);
	perror("OMX_Init");
	return OMXJPEG_ERROR_OMX_INIT;
    }
    // prepare the image decoder
    int             ret = prepareImageDecoder(pDecoder[0]);
    if (ret != OMXJPEG_OK)
	return ret;

    ret = startupImageDecoder(pDecoder[0]);
    if (ret != OMXJPEG_OK)
	return ret;

    return OMXJPEG_OK;
}
	
      

The call to prepareImageDecoder creates the component and establishes its input and output port numbers (which should be 320 and 321 respectively by the OpenMAX specification). These two ports are disabled, but they are enabled to have buffers.

The function startupImageDecoder is a heavy-duty function. It has to establish the format that it will accept from the input file by

	
    // set input image format
    OMX_IMAGE_PARAM_PORTFORMATTYPE imagePortFormat;
    memset(&imagePortFormat, 0, sizeof(OMX_IMAGE_PARAM_PORTFORMATTYPE));
    imagePortFormat.nSize = sizeof(OMX_IMAGE_PARAM_PORTFORMATTYPE);
    imagePortFormat.nVersion.nVersion = OMX_VERSION;
    imagePortFormat.nPortIndex = decoder->imageDecoder->inPort;
    imagePortFormat.eCompressionFormat = OMX_IMAGE_CodingJPEG;
    OMX_SetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamImagePortFormat, &imagePortFormat);
	
      

Then it queries for the buffer requirements, building an OMX_PARAM_PORTDEFINITIONTYPE portdef structure and populating it with a get parameter call.

	
    // get buffer requirements
    OMX_PARAM_PORTDEFINITIONTYPE portdef;
    portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
    portdef.nVersion.nVersion = OMX_VERSION;
    portdef.nPortIndex = decoder->imageDecoder->inPort;
    OMX_GetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamPortDefinition, &portdef);
	
      

Then we can make a call to enable the input port, allocate the input buffers and wait for the port to become enabled

	
    // enable the port and setup the buffers
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandPortEnable,
		    decoder->imageDecoder->inPort, NULL);
    decoder->inputBufferHeaderCount = portdef.nBufferCountActual;
    // allocate pointer array
    decoder->ppInputBufferHeader =
	(OMX_BUFFERHEADERTYPE **) malloc(sizeof(void) *
					 decoder->inputBufferHeaderCount);
    // allocate each buffer
    int             i;
    for (i = 0; i < decoder->inputBufferHeaderCount; i++) {
	if (OMX_AllocateBuffer(decoder->imageDecoder->handle,
			       &decoder->ppInputBufferHeader[i],
			       decoder->imageDecoder->inPort,
			       (void *) i,
			       portdef.nBufferSize) != OMX_ErrorNone) {
	    perror("Allocate decode buffer");
	    return OMXJPEG_ERROR_MEMORY;
	}
    }
    // wait for port enable to complete - which it should once buffers are 
    // assigned
    int             ret =
	ilclient_wait_for_event(decoder->imageDecoder->component,
				OMX_EventCmdComplete,
				OMX_CommandPortEnable, 0,
				decoder->imageDecoder->inPort, 0,
				0, TIMEOUT_MS);
    if (ret != 0) {
	fprintf(stderr, "Did not get port enable %d\n", ret);
	return OMXJPEG_ERROR_EXECUTING;
    }
	
      

Finally, we can move the component into executing state. But being paranoic about the behaviour of this API, we wait to ensure that it actually does make the state transition requested:

	
    // start executing the decoder 
    ret = OMX_SendCommand(decoder->imageDecoder-gt;handle,
			  OMX_CommandStateSet, OMX_StateExecuting, NULL);
    if (ret != 0) {
	fprintf(stderr, "Error starting image decoder %x\n", ret);
	return OMXJPEG_ERROR_EXECUTING;
    }
    ret = ilclient_wait_for_event(decoder-gt;imageDecoder-gt;component,
				  OMX_EventCmdComplete,
				  OMX_StateExecuting, 0, 0, 1, 0,
				  TIMEOUT_MS);
    if (ret != 0) {
	fprintf(stderr, "Did not receive executing stat %d\n", ret);
	// return OMXJPEG_ERROR_EXECUTING;
    }
	
      

The function decodeImage starts up normally enough. It loads the input buffers in a circular fashion from the JPEG image loaded into the sourceImage array. Each buffer is emptied by a call to OMX_EmptyThisBuffer. When the entire image has been loaded, the final buffer has the OMX_BUFFERFLAG_EOS flag set to indicate that the image is complete.

The snag with this component is that you don't know how large the decoded image will be until the decoder has done at least some work on it. So while the input buffers can be assigned statically up front, filled and emptied, we don't know what the size of the output buffer should be. Fortunately OpneMAX manages this by raising a PortSettingsChanged event when enough information is gained from the JPEG image to know the size of the decoded image.

If we were into concurrent programming, we would catch a PortSettingsChanged event in an event handling thread and work from there. The ilclient library tries to force a sequential mode of operation. So whenever a buffer is emptied, the application will go into a loop either waiting for a PortSettingsChanged event to occur and timing out after 5 milliseconds (I think) if it doesn't or exiting if the input buffer is empty. This code is a bit messy!

	
	// wait for buffer to empty or port changed event
	int             done = 0;
	while ((done == 0) || (decoder->pOutputBufferHeader == NULL)) {
	    if (decoder->pOutputBufferHeader == NULL) {
		ret =
		    ilclient_wait_for_event
		    (decoder->imageDecoder->component,
		     OMX_EventPortSettingsChanged,
		     decoder->imageDecoder->outPort, 0, 0, 1, 0, 5);

		if (ret == 0) {
		    portSettingsChanged(decoder);
		}
	    } else {
		ret =
		    ilclient_remove_event(decoder->imageDecoder->component,
					  OMX_EventPortSettingsChanged,
					  decoder->imageDecoder->outPort,
					  0, 0, 1);
		if (ret == 0)
		    portSettingsChangedAgain(decoder);

	    }

	    // check to see if buffer is now empty
	    if (pBufHeader->nFilledLen == 0)
		done = 1;

	    if ((done == 0)
		|| (decoder->pOutputBufferHeader == NULL)) {
		printf("Buffer is now size %d\n", pBufHeader->nFilledLen);
		sleep(1);
	    }
	}
	
      

If a PortSettingsChanged event occurs, the application calls portSettingsChanged in the main thread. This queries the (single) output port for the width and height values now set, switches the component to enabled state, allocates a buffer and waits for it to move to enabled state. We can also collect information about the format of the decoded image as well as other features such as size.

Once the output buffer has been created, we can make a call to fill the buffer. We should only have to do this once, so a flag bFilled is used to control this.

	
	// fill the buffer if we have created the buffer
	if (bFilled == 0) {
	    if ((decoder->pOutputBufferHeader == NULL)) {
		portSettingsChanged(decoder);
	    }
	    OMX_PARAM_U32TYPE param;
	    param.nSize = sizeof(OMX_PARAM_U32TYPE);
	    param.nVersion.nVersion = OMX_VERSION;
	    param.nPortIndex = decoder->imageDecoder->outPort;
	   
	    OMX_GetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamActiveStream, &param);
	    printf("Active stream %d\n", param.nU32);

	    printf("Trying to fill output buffer\n");
	    printState(decoder->imageDecoder->handle);
	    ret = OMX_FillThisBuffer(decoder->imageDecoder->handle,
				     decoder->pOutputBufferHeader);
	    
	    if (ret != OMX_ErrorNone) {
		perror("Filling output buffer");
		fprintf(stderr, "Error code %x\n", ret);
		return OMXJPEG_ERROR_MEMORY;
	    }

	    bFilled = 1;
	}
	
      

Once the input buffers have been filled and emptied the output buffer has been sized, allocated and a call made to fill it, the application has nothing to do but wait until the output buffer is filled. OpenMAX should generate a OMX_BUFFERFLAG_EOS when this happens, so we just wait

	
    // wait for end of stream events
    int             ret =
	ilclient_wait_for_event(decoder->imageDecoder->component,
				OMX_EventBufferFlag,
				decoder->imageDecoder->outPort, 1,
				OMX_BUFFERFLAG_EOS, 1,
				0, 2);
    if (ret != 0) {
	fprintf(stderr, "No EOS event on image decoder %d\n", ret);
    } else  {
	fprintf(stderr, "EOS event on image decoder %d\n", ret);
    }
	
      

Ooops! the EOS event doesn't seem to get generated in practice (contrary to the specification) but the program seems to work anyway :-(.

At this point we can do something like save the decoded image to a file, or do further processing.

The final code is jpeg-decoder.c

	/*
Copyright (c) 2012, Matt Ownby
                    Anthong Sale
Copyright (c) 2014: Jan Newmarch

All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
    * Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.
    * Neither the name of the copyright holder nor the
      names of its contributors may be used to endorse or promote products
      derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

#include <stdio.h>
#include <assert.h>
#include "jpeg.h"

#define TIMEOUT_MS 2000

typedef struct _COMPONENT_DETAILS {
    COMPONENT_T    *component;
    OMX_HANDLETYPE  handle;
    int             inPort;
    int             outPort;
} COMPONENT_DETAILS;

struct _OPENMAX_JPEG_DECODER {
    ILCLIENT_T     *client;
    COMPONENT_DETAILS *imageDecoder;
#if IR
    COMPONENT_DETAILS *imageResizer;
#endif
    OMX_BUFFERHEADERTYPE **ppInputBufferHeader;
    int             inputBufferHeaderCount;
    OMX_BUFFERHEADERTYPE *pOutputBufferHeader;
};

int             bufferIndex = 0;	// index to buffer array

void printState(OMX_HANDLETYPE handle) {
    OMX_STATETYPE state;
    OMX_ERRORTYPE err;

    err = OMX_GetState(handle, &state);
    if (err != OMX_ErrorNone) {
        fprintf(stderr, "Error on getting state\n");
        exit(1);
    }
    switch (state) {
    case OMX_StateLoaded: printf("StateLoaded\n"); break;
    case OMX_StateIdle: printf("StateIdle\n"); break;
    case OMX_StateExecuting: printf("StateExecuting\n"); break;
    case OMX_StatePause: printf("StatePause\n"); break;
    case OMX_StateWaitForResources: printf("StateWiat\n"); break;
    default:  printf("State unknown\n"); break;
    }
}


int
portSettingsChanged(OPENMAX_JPEG_DECODER * decoder)
{
    OMX_PARAM_PORTDEFINITIONTYPE portdef;

    // CLEANUP

    printf("Pport settings changed\n");
    // need to setup the input for the resizer with the output of the
    // decoder
    portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
    portdef.nVersion.nVersion = OMX_VERSION;
    portdef.nPortIndex = decoder->imageDecoder->outPort;
    OMX_GetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    unsigned int    uWidth =
	(unsigned int) portdef.format.image.nFrameWidth;
    unsigned int    uHeight =
	(unsigned int) portdef.format.image.nFrameHeight;

    // enable ports
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandPortEnable,
		    decoder->imageDecoder->outPort, NULL);

    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete,
			    OMX_CommandPortEnable, 1,
			    decoder->imageDecoder->outPort, 1, 0,
			    TIMEOUT_MS);
    printf("Decoder output port enabled\n");

    OMX_IMAGE_PARAM_PORTFORMATTYPE iportdef;

    printf("Port settings changed\n");

    // show some logging so user knows it's working
    printf
	("Width: %u Height: %u Output Color Format: 0x%x Buffer Size: %u\n",
	 (unsigned int) portdef.format.image.nFrameWidth,
	 (unsigned int) portdef.format.image.nFrameHeight,
	 (unsigned int) portdef.format.image.eColorFormat,
	 (unsigned int) portdef.nBufferSize);
    fflush(stdout);


    // enable output port of decoder
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandPortEnable,
		    decoder->imageDecoder->outPort, NULL);


    int             ret = OMX_AllocateBuffer(decoder->imageDecoder->handle,
					     &decoder->pOutputBufferHeader,
					     decoder->imageDecoder->
					     outPort,
					     NULL,
					     portdef.nBufferSize);
    printf("Output port buffer allocated\n");

    if (ret != OMX_ErrorNone) {
	perror("Eror allocating buffer");
	return OMXJPEG_ERROR_MEMORY;
    }


    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete,
			    OMX_CommandPortEnable, 1,
			    decoder->imageDecoder->outPort, 1, 0,
			    TIMEOUT_MS);
    printf("Decoder output port rnabled\n");

    return OMXJPEG_OK;
}

int
portSettingsChangedAgain(OPENMAX_JPEG_DECODER * decoder)
{
    printf("Port settings changed again\n");
    ilclient_disable_port(decoder->imageDecoder->component,
			  decoder->imageDecoder->outPort);

    OMX_PARAM_PORTDEFINITIONTYPE portdef;

    // need to setup the input for the resizer with the output of the
    // decoder
    portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
    portdef.nVersion.nVersion = OMX_VERSION;
    portdef.nPortIndex = decoder->imageDecoder->outPort;
    OMX_GetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    // enable output of decoder
    ilclient_enable_port(decoder->imageDecoder->component,
			 decoder->imageDecoder->outPort);

    // need to wait for this event
    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventPortSettingsChanged,
			    decoder->imageDecoder->outPort, 1,
			    0, 0, 0, TIMEOUT_MS);
    return OMXJPEG_OK;
}

int
prepareImageDecoder(OPENMAX_JPEG_DECODER * decoder)
{
    decoder->imageDecoder = malloc(sizeof(COMPONENT_DETAILS));
    if (decoder->imageDecoder == NULL) {
	perror("malloc image decoder");
	return OMXJPEG_ERROR_MEMORY;
    }

    int ret = ilclient_create_component(decoder->client,
					&decoder->imageDecoder->component,
					"image_decode",
					ILCLIENT_DISABLE_ALL_PORTS
					|
					ILCLIENT_ENABLE_INPUT_BUFFERS
					|
					ILCLIENT_ENABLE_OUTPUT_BUFFERS);
    
    if (ret != 0) {
	perror("image decode");
	return OMXJPEG_ERROR_CREATING_COMP;
    }
    // grab the handle for later use in OMX calls directly
    decoder->imageDecoder->handle =
	ILC_GET_HANDLE(decoder->imageDecoder->component);

    // get and store the ports
    OMX_PORT_PARAM_TYPE port;
    port.nSize = sizeof(OMX_PORT_PARAM_TYPE);
    port.nVersion.nVersion = OMX_VERSION;

    OMX_GetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamImageInit, &port);
    if (port.nPorts != 2) {
	return OMXJPEG_ERROR_WRONG_NO_PORTS;
    }
    decoder->imageDecoder->inPort = port.nStartPortNumber;
    decoder->imageDecoder->outPort = port.nStartPortNumber + 1;

   decoder->pOutputBufferHeader = NULL;

    return OMXJPEG_OK;
}

int
startupImageDecoder(OPENMAX_JPEG_DECODER * decoder)
{
    // move to idle
    ilclient_change_component_state(decoder->imageDecoder->component,
				    OMX_StateIdle);

    // set input image format
    OMX_IMAGE_PARAM_PORTFORMATTYPE imagePortFormat;
    memset(&imagePortFormat, 0, sizeof(OMX_IMAGE_PARAM_PORTFORMATTYPE));
    imagePortFormat.nSize = sizeof(OMX_IMAGE_PARAM_PORTFORMATTYPE);
    imagePortFormat.nVersion.nVersion = OMX_VERSION;
    imagePortFormat.nPortIndex = decoder->imageDecoder->inPort;
    imagePortFormat.eCompressionFormat = OMX_IMAGE_CodingJPEG;
    OMX_SetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamImagePortFormat, &imagePortFormat);

    // get buffer requirements
    OMX_PARAM_PORTDEFINITIONTYPE portdef;
    portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
    portdef.nVersion.nVersion = OMX_VERSION;
    portdef.nPortIndex = decoder->imageDecoder->inPort;
    OMX_GetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    // enable the port and setup the buffers
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandPortEnable,
		    decoder->imageDecoder->inPort, NULL);
    decoder->inputBufferHeaderCount = portdef.nBufferCountActual;
    // allocate pointer array
    decoder->ppInputBufferHeader =
	(OMX_BUFFERHEADERTYPE **) malloc(sizeof(void) *
					 decoder->inputBufferHeaderCount);
    // allocate each buffer
    int             i;
    for (i = 0; i < decoder->inputBufferHeaderCount; i++) {
	if (OMX_AllocateBuffer(decoder->imageDecoder->handle,
			       &decoder->ppInputBufferHeader[i],
			       decoder->imageDecoder->inPort,
			       (void *) i,
			       portdef.nBufferSize) != OMX_ErrorNone) {
	    perror("Allocate decode buffer");
	    return OMXJPEG_ERROR_MEMORY;
	}
    }
    // wait for port enable to complete - which it should once buffers are 
    // assigned
    int             ret =
	ilclient_wait_for_event(decoder->imageDecoder->component,
				OMX_EventCmdComplete,
				OMX_CommandPortEnable, 0,
				decoder->imageDecoder->inPort, 0,
				0, TIMEOUT_MS);
    if (ret != 0) {
	fprintf(stderr, "Did not get port enable %d\n", ret);
	return OMXJPEG_ERROR_EXECUTING;
    }
    // start executing the decoder 
    ret = OMX_SendCommand(decoder->imageDecoder->handle,
			  OMX_CommandStateSet, OMX_StateExecuting, NULL);
    if (ret != 0) {
	fprintf(stderr, "Error starting image decoder %x\n", ret);
	return OMXJPEG_ERROR_EXECUTING;
    }
    ret = ilclient_wait_for_event(decoder->imageDecoder->component,
				  OMX_EventCmdComplete,
				  OMX_StateExecuting, 0, 0, 1, 0,
				  TIMEOUT_MS);
    if (ret != 0) {
	fprintf(stderr, "Did not receive executing stat %d\n", ret);
	// return OMXJPEG_ERROR_EXECUTING;
    }

    return OMXJPEG_OK;
}

// this function run the boilerplate to setup the openmax components;
int
setupOpenMaxJpegDecoder(OPENMAX_JPEG_DECODER ** pDecoder)
{
    *pDecoder = malloc(sizeof(OPENMAX_JPEG_DECODER));
    if (pDecoder[0] == NULL) {
	perror("malloc decoder");
	return OMXJPEG_ERROR_MEMORY;
    }
    memset(*pDecoder, 0, sizeof(OPENMAX_JPEG_DECODER));

    if ((pDecoder[0]->client = ilclient_init()) == NULL) {
	perror("ilclient_init");
	return OMXJPEG_ERROR_ILCLIENT_INIT;
    }

    if (OMX_Init() != OMX_ErrorNone) {
	ilclient_destroy(pDecoder[0]->client);
	perror("OMX_Init");
	return OMXJPEG_ERROR_OMX_INIT;
    }
    // prepare the image decoder
    int             ret = prepareImageDecoder(pDecoder[0]);
    if (ret != OMXJPEG_OK)
	return ret;

    ret = startupImageDecoder(pDecoder[0]);
    if (ret != OMXJPEG_OK)
	return ret;

    return OMXJPEG_OK;
}

// this function passed the jpeg image buffer in, and returns the decoded
// image
int
decodeImage(OPENMAX_JPEG_DECODER * decoder, char *sourceImage,
	    size_t imageSize)
{
    char           *sourceOffset = sourceImage;	// we store a seperate
						// buffer ot image so we
						// can offset it
    size_t          toread = 0;	// bytes left to read from buffer
    toread += imageSize;
    int             bFilled = 0;	// have we filled our output
					// buffer
    bufferIndex = 0;

    while (toread > 0) {
	// get next buffer from array
	OMX_BUFFERHEADERTYPE *pBufHeader =
	    decoder->ppInputBufferHeader[bufferIndex];

	// step index and reset to 0 if required
	bufferIndex++;
	if (bufferIndex >= decoder->inputBufferHeaderCount)
	    bufferIndex = 0;

	// work out the next chunk to load into the decoder
	if (toread > pBufHeader->nAllocLen)
	    pBufHeader->nFilledLen = pBufHeader->nAllocLen;
	else
	    pBufHeader->nFilledLen = toread;

	toread = toread - pBufHeader->nFilledLen;

	// pass the bytes to the buffer
	memcpy(pBufHeader->pBuffer, sourceOffset, pBufHeader->nFilledLen);
	printf("Read into buffer %d\n", pBufHeader->nFilledLen);

	// update the buffer pointer and set the input flags

	sourceOffset = sourceOffset + pBufHeader->nFilledLen;
	pBufHeader->nOffset = 0;
	pBufHeader->nFlags = 0;
	if (toread <= 0) {
	    pBufHeader->nFlags = OMX_BUFFERFLAG_EOS;
	}
	// empty the current buffer
	printf("Emptying buffer\n");
	int             ret =
	    OMX_EmptyThisBuffer(decoder->imageDecoder->handle,
				pBufHeader);

	if (ret != OMX_ErrorNone) {
	    perror("Empty input buffer");
	    fprintf(stderr, "return code %x\n", ret);
	    return OMXJPEG_ERROR_MEMORY;
	}
	// wait for buffer to empty or port changed event
	int             done = 0;
	while ((done == 0) || (decoder->pOutputBufferHeader == NULL)) {
	    if (decoder->pOutputBufferHeader == NULL) {
		ret =
		    ilclient_wait_for_event
		    (decoder->imageDecoder->component,
		     OMX_EventPortSettingsChanged,
		     decoder->imageDecoder->outPort, 0, 0, 1, 0, 5);

		if (ret == 0) {
		    portSettingsChanged(decoder);
		}
	    } else {
		ret =
		    ilclient_remove_event(decoder->imageDecoder->component,
					  OMX_EventPortSettingsChanged,
					  decoder->imageDecoder->outPort,
					  0, 0, 1);
		if (ret == 0)
		    portSettingsChangedAgain(decoder);

	    }

	    // check to see if buffer is now empty
	    if (pBufHeader->nFilledLen == 0)
		done = 1;

	    if ((done == 0)
		|| (decoder->pOutputBufferHeader == NULL)) {
		printf("Buffer is now size %d\n", pBufHeader->nFilledLen);
		sleep(1);
	    }
	}

	// fill the buffer if we have created the buffer
	if (bFilled == 0) {
	    if ((decoder->pOutputBufferHeader == NULL)) {
		portSettingsChanged(decoder);
	    }
	    OMX_PARAM_U32TYPE param;
	    param.nSize = sizeof(OMX_PARAM_U32TYPE);
	    param.nVersion.nVersion = OMX_VERSION;
	    param.nPortIndex = decoder->imageDecoder->outPort;
	   
	    OMX_GetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamActiveStream, &param);
	    printf("Active stream %d\n", param.nU32);

	    printf("Trying to fill output buffer\n");
	    printState(decoder->imageDecoder->handle);
	    ret = OMX_FillThisBuffer(decoder->imageDecoder->handle,
				     decoder->pOutputBufferHeader);
	    
	    if (ret != OMX_ErrorNone) {
		perror("Filling output buffer");
		fprintf(stderr, "Error code %x\n", ret);
		return OMXJPEG_ERROR_MEMORY;
	    }

	    bFilled = 1;
	}
    }

    // wait for buffer to fill
    /*
     * while(pBufHeader->nFilledLen == 0) { sleep(5); } 
     */

    // wait for end of stream events
    int             ret =
	ilclient_wait_for_event(decoder->imageDecoder->component,
				OMX_EventBufferFlag,
				decoder->imageDecoder->outPort, 1,
				OMX_BUFFERFLAG_EOS, 1,
				0, 2);
    if (ret != 0) {
	fprintf(stderr, "No EOS event on image decoder %d\n", ret);
    } else  {
	fprintf(stderr, "EOS event on image decoder %d\n", ret);
    }

    printf("Resized %d\n", decoder->pOutputBufferHeader->nFilledLen);
    FILE *fp = fopen("out", "w");
    int n;
    for (n = 0; n < decoder->pOutputBufferHeader->nFilledLen; n++) {
	fputc(decoder->pOutputBufferHeader->pBuffer[n], fp);
    }
    fclose(fp);
    printf("File written\n");

    return OMXJPEG_OK;
}

// this function cleans up the decoder.
void
cleanup(OPENMAX_JPEG_DECODER * decoder)
{
    // flush everything through
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandFlush, decoder->imageDecoder->outPort,
		    NULL);
    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandFlush, 0,
			    decoder->imageDecoder->outPort, 0, 0,
			    TIMEOUT_MS);

    OMX_SendCommand(decoder->imageDecoder->handle, OMX_CommandPortDisable,
		    decoder->imageDecoder->inPort, NULL);

    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandPortDisable,
			    0, decoder->imageDecoder->outPort, 0, 0,
			    TIMEOUT_MS);

    int             i = 0;
    for (i = 0; i < decoder->inputBufferHeaderCount; i++) {
	OMX_BUFFERHEADERTYPE *vpBufHeader =
	    decoder->ppInputBufferHeader[i];

	OMX_FreeBuffer(decoder->imageDecoder->handle,
		       decoder->imageDecoder->inPort, vpBufHeader);
    }

    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandPortDisable,
			    0, decoder->imageDecoder->inPort, 0, 0,
			    TIMEOUT_MS);

    OMX_FreeBuffer(decoder->imageDecoder->handle,
		   decoder->imageDecoder->outPort,
		   decoder->pOutputBufferHeader);

    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandPortDisable,
			    0, decoder->imageDecoder->outPort, 0, 0,
			    TIMEOUT_MS);

    ilclient_change_component_state(decoder->imageDecoder->component,
				    OMX_StateIdle);


    ilclient_change_component_state(decoder->imageDecoder->component,
				    OMX_StateLoaded);


    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandStateSet, 0,
			    OMX_StateLoaded, 0, 0, TIMEOUT_MS);

    OMX_Deinit();

    if (decoder->client != NULL) {
	ilclient_destroy(decoder->client);
    }
}

int
main(int argc, char *argv[])
{
    OPENMAX_JPEG_DECODER *pDecoder;
    char           *sourceImage;
    size_t          imageSize;
    int             s;
    if (argc < 2) {
	printf("Usage: %s <filename>\n", argv[0]);
	return -1;
    }
    FILE           *fp = fopen(argv[1], "rb");
    if (!fp) {
	printf("File %s not found.\n", argv[1]);
    }
    fseek(fp, 0L, SEEK_END);
    imageSize = ftell(fp);
    fseek(fp, 0L, SEEK_SET);
    sourceImage = malloc(imageSize);
    assert(sourceImage != NULL);
    s = fread(sourceImage, 1, imageSize, fp);
    assert(s == imageSize);
    fclose(fp);
    bcm_host_init();
    s = setupOpenMaxJpegDecoder(&pDecoder);
    assert(s == 0);
    s = decodeImage(pDecoder, sourceImage, imageSize);
    assert(s == 0);
    cleanup(pDecoder);
    free(sourceImage);
    printf("Success\n");
    return 0;
}

      

Format of decoded image

There are two principal colour formats used for images: YUV and RGB. It is simple to translate from one to the other. But the Broadcom image_decode component won't do any translation: it will only give you YUV images. This is according to the OpenMAX specifications for a JPEG decoder. If you want RGB, you have to do it yourself or use another Broadcom component, the resize component. This is not a standard OpenMAX component (see below).

Tunnelling

If you want to connect two or more OpenMAX components then these two components have to be able to pass information between them. This means that the port formats must be the same between one component's output port and the next component's input port, and there must be a means of transferring from output buffers to input buffers.

OpenMAX supplies two methods to do this:

Doing it yourself is always supported. In this case the client has to work out compatable port formats, and co-ordinate buffers. For example, when one component fills a buffer the client could copy it to the next component's input buffer. This is inefficient. Or the client could use the first component's output buffer as the next component's input buffer. But that would mean that the shared buffer would not be free for the first component to refill until the second component had received a buffer-empty event. Messy.

The alternative is tunnelling where the client asks the components to do all this for them. The OpenMAX components support tunnelling, which makes life a lot easier.

Tunnelling is used to connect two ports. It can only be done when the two components are in the Loaded state or when the ports are disabled. The call OMX_SetupTunnel will then connect the two ports as in

	
    // establish tunnel between decoder output and resizer input
    OMX_SetupTunnel(decoder->imageDecoder->handle,
		    decoder->imageDecoder->outPort,
		    decoder->imageResizer->handle,
		    decoder->imageResizer->inPort);
	
      

A typical use of this is to disable the relevant ports while they are in Idle state, and then to fill and empty the input buffers of the first component. This will trigger a PortSettingsChanged event on the first component's output port. At this point, with formats and things like image sizes determined, the two ports can be connected by a tunnel. There is no need to allocate buffers: the tunnel does that.

Decoding a JPEG image into RGB format

The OpenMAX component image_resize will decode a JPEG image, but only to YUV format. Broadcom has an additional non-standard component resize which can not only resize an uncompresssed image but also convert it to RGB format.

Matt Ownby and Anthong Sale did a marvellous job of unravelling the sequence of steps required to set up tunnelling between a decoder and resizer. Their logic is captured in the portSettingsChanged function. This adds tunnelling to the decoder program given earlier.

	
int
portSettingsChanged(OPENMAX_JPEG_DECODER * decoder)
{
    OMX_PARAM_PORTDEFINITIONTYPE portdef;

    // need to setup the input for the resizer with the output of the
    // decoder
    portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
    portdef.nVersion.nVersion = OMX_VERSION;
    portdef.nPortIndex = decoder->imageDecoder->outPort;
    OMX_GetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    unsigned int    uWidth =
	(unsigned int) portdef.format.image.nFrameWidth;
    unsigned int    uHeight =
	(unsigned int) portdef.format.image.nFrameHeight;

    // tell resizer input what the decoder output will be providing
    portdef.nPortIndex = decoder->imageResizer->inPort;
    OMX_SetParameter(decoder->imageResizer->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    // establish tunnel between decoder output and resizer input
    OMX_SetupTunnel(decoder->imageDecoder->handle,
		    decoder->imageDecoder->outPort,
		    decoder->imageResizer->handle,
		    decoder->imageResizer->inPort);

    // enable ports
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandPortEnable,
		    decoder->imageDecoder->outPort, NULL);
    OMX_SendCommand(decoder->imageResizer->handle,
		    OMX_CommandPortEnable,
		    decoder->imageResizer->inPort, NULL);

    // put resizer in idle state (this allows the outport of the decoder
    // to become enabled)
    OMX_SendCommand(decoder->imageResizer->handle,
		    OMX_CommandStateSet, OMX_StateIdle, NULL);

    // wait for state change complete
    ilclient_wait_for_event(decoder->imageResizer->component,
			    OMX_EventCmdComplete,
			    OMX_CommandStateSet, 1,
			    OMX_StateIdle, 1, 0, TIMEOUT_MS); 

    // once the state changes, both ports should become enabled and the
    // resizer
    // output should generate a settings changed event
    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete,
			    OMX_CommandPortEnable, 1,
			    decoder->imageDecoder->outPort, 1, 0,
			    TIMEOUT_MS);
    ilclient_wait_for_event(decoder->imageResizer->component,
			    OMX_EventCmdComplete, OMX_CommandPortEnable, 1,
			    decoder->imageResizer->inPort, 1, 0,
			    TIMEOUT_MS);
    ilclient_wait_for_event(decoder->imageResizer->component,
			    OMX_EventPortSettingsChanged,
			    decoder->imageResizer->outPort, 1, 0, 1, 0,
			    TIMEOUT_MS);

    ilclient_disable_port(decoder->imageResizer->component,
			  decoder->imageResizer->outPort);

    // query output buffer requirements for resizer
    portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
    portdef.nVersion.nVersion = OMX_VERSION;
    portdef.nPortIndex = decoder->imageResizer->outPort;
    OMX_GetParameter(decoder->imageResizer->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    // change output color format and dimensions to match input
    portdef.format.image.eCompressionFormat = OMX_IMAGE_CodingUnused;
    portdef.format.image.eColorFormat = OMX_COLOR_Format32bitABGR8888;
    portdef.format.image.nFrameWidth = uWidth;
    portdef.format.image.nFrameHeight = uHeight;
    portdef.format.image.nStride = 0;
    portdef.format.image.nSliceHeight = 0;
    portdef.format.image.bFlagErrorConcealment = OMX_FALSE;

    OMX_SetParameter(decoder->imageResizer->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    // grab output requirements again to get actual buffer size
    // requirement (and buffer count requirement!)
    OMX_GetParameter(decoder->imageResizer->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    // move resizer into executing state
    ilclient_change_component_state(decoder->imageResizer->component,
				    OMX_StateExecuting);
    // show some logging so user knows it's working
    printf
	("Width: %u Height: %u Output Color Format: 0x%x Buffer Size: %u\n",
	 (unsigned int) portdef.format.image.nFrameWidth,
	 (unsigned int) portdef.format.image.nFrameHeight,
	 (unsigned int) portdef.format.image.eColorFormat,
	 (unsigned int) portdef.nBufferSize);
    fflush(stdout);

    // enable output port of resizer
    OMX_SendCommand(decoder->imageResizer->handle,
		    OMX_CommandPortEnable,
		    decoder->imageResizer->outPort, NULL);

    int             ret = OMX_AllocateBuffer(decoder->imageResizer->handle,
					     &decoder->pOutputBufferHeader,
					     decoder->imageResizer->
					     outPort,
					     NULL,
					     portdef.nBufferSize);
    if (ret != OMX_ErrorNone) {
	perror("Eror allocating buffer");
	return OMXJPEG_ERROR_MEMORY;
    }

    ilclient_wait_for_event(decoder->imageResizer->component,
			    OMX_EventCmdComplete,
			    OMX_CommandPortEnable, 1,
			    decoder->imageResizer->outPort, 1, 0,
			    TIMEOUT_MS);

    return OMXJPEG_OK;
}
	
      

The only change we make to their code is to add a function to dump the decoded and converted RGB image as a TGA file. The function save_image_as_TGA is called when the output buffer of the resizer is non-empty. Just a few wrinkles in this. The image has a height and width. The resizer will have aligned those up to a 16-byte multiple, the Stride and the SliceHeight. We have asked for a 32-bit format ( OMX_COLOR_Format32bitABGR8888) and so we have to specify this. The TGA format is specified here.

The co-ordinate system between TGA and the Broadcom is inverted. We are lazy here because it simplifies the code, and save the image upside down. The source is almost completely from the RPi /opt/vc/src/hello_pi/hello_jpeg/jpeg.c program, here as decodejpeg2rgb.c:

	/*
Copyright (c) 2012, Matt Ownby
                    Anthong Sale
Copyright (c) 2014, Jan Newmarch

All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
    * Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.
    * Neither the name of the copyright holder nor the
      names of its contributors may be used to endorse or promote products
      derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

#include <stdio.h>
#include <assert.h>
#include "jpeg.h"

#define TIMEOUT_MS 2000

typedef struct _COMPONENT_DETAILS {
    COMPONENT_T    *component;
    OMX_HANDLETYPE  handle;
    int             inPort;
    int             outPort;
} COMPONENT_DETAILS;

struct _OPENMAX_JPEG_DECODER {
    ILCLIENT_T     *client;
    COMPONENT_DETAILS *imageDecoder;
    COMPONENT_DETAILS *imageResizer;
    OMX_BUFFERHEADERTYPE **ppInputBufferHeader;
    int             inputBufferHeaderCount;
    OMX_BUFFERHEADERTYPE *pOutputBufferHeader;
};

int             bufferIndex = 0;	// index to buffer array

int
portSettingsChanged(OPENMAX_JPEG_DECODER * decoder)
{
    OMX_PARAM_PORTDEFINITIONTYPE portdef;

    // need to setup the input for the resizer with the output of the
    // decoder
    portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
    portdef.nVersion.nVersion = OMX_VERSION;
    portdef.nPortIndex = decoder->imageDecoder->outPort;
    OMX_GetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    unsigned int    uWidth =
	(unsigned int) portdef.format.image.nFrameWidth;
    unsigned int    uHeight =
	(unsigned int) portdef.format.image.nFrameHeight;

    // tell resizer input what the decoder output will be providing
    portdef.nPortIndex = decoder->imageResizer->inPort;
    OMX_SetParameter(decoder->imageResizer->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    // establish tunnel between decoder output and resizer input
    OMX_SetupTunnel(decoder->imageDecoder->handle,
		    decoder->imageDecoder->outPort,
		    decoder->imageResizer->handle,
		    decoder->imageResizer->inPort);

    // enable ports
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandPortEnable,
		    decoder->imageDecoder->outPort, NULL);
    OMX_SendCommand(decoder->imageResizer->handle,
		    OMX_CommandPortEnable,
		    decoder->imageResizer->inPort, NULL);

    // put resizer in idle state (this allows the outport of the decoder
    // to become enabled)
    OMX_SendCommand(decoder->imageResizer->handle,
		    OMX_CommandStateSet, OMX_StateIdle, NULL);

    // wait for state change complete
    ilclient_wait_for_event(decoder->imageResizer->component,
			    OMX_EventCmdComplete,
			    OMX_CommandStateSet, 1,
			    OMX_StateIdle, 1, 0, TIMEOUT_MS);

    // once the state changes, both ports should become enabled and the
    // resizer
    // output should generate a settings changed event
    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete,
			    OMX_CommandPortEnable, 1,
			    decoder->imageDecoder->outPort, 1, 0,
			    TIMEOUT_MS);
    ilclient_wait_for_event(decoder->imageResizer->component,
			    OMX_EventCmdComplete, OMX_CommandPortEnable, 1,
			    decoder->imageResizer->inPort, 1, 0,
			    TIMEOUT_MS);
    ilclient_wait_for_event(decoder->imageResizer->component,
			    OMX_EventPortSettingsChanged,
			    decoder->imageResizer->outPort, 1, 0, 1, 0,
			    TIMEOUT_MS);

    ilclient_disable_port(decoder->imageResizer->component,
			  decoder->imageResizer->outPort);

    // query output buffer requirements for resizer
    portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
    portdef.nVersion.nVersion = OMX_VERSION;
    portdef.nPortIndex = decoder->imageResizer->outPort;
    OMX_GetParameter(decoder->imageResizer->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    // change output color format and dimensions to match input
    portdef.format.image.eCompressionFormat = OMX_IMAGE_CodingUnused;
    portdef.format.image.eColorFormat = OMX_COLOR_Format32bitABGR8888;
    portdef.format.image.nFrameWidth = uWidth;
    portdef.format.image.nFrameHeight = uHeight;
    portdef.format.image.nStride = 0;
    portdef.format.image.nSliceHeight = 0;
    portdef.format.image.bFlagErrorConcealment = OMX_FALSE;

    OMX_SetParameter(decoder->imageResizer->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    // grab output requirements again to get actual buffer size
    // requirement (and buffer count requirement!)
    OMX_GetParameter(decoder->imageResizer->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    // move resizer into executing state
    ilclient_change_component_state(decoder->imageResizer->component,
				    OMX_StateExecuting);

    // show some logging so user knows it's working
    printf
	("Width: %u Height: %u Output Color Format: 0x%x Buffer Size: %u\n",
	 (unsigned int) portdef.format.image.nFrameWidth,
	 (unsigned int) portdef.format.image.nFrameHeight,
	 (unsigned int) portdef.format.image.eColorFormat,
	 (unsigned int) portdef.nBufferSize);
    fflush(stdout);

    // enable output port of resizer
    OMX_SendCommand(decoder->imageResizer->handle,
		    OMX_CommandPortEnable,
		    decoder->imageResizer->outPort, NULL);

    int             ret = OMX_AllocateBuffer(decoder->imageResizer->handle,
					     &decoder->pOutputBufferHeader,
					     decoder->imageResizer->
					     outPort,
					     NULL,
					     portdef.nBufferSize);
    if (ret != OMX_ErrorNone) {
	perror("Eror allocating buffer");
	return OMXJPEG_ERROR_MEMORY;
    }

    ilclient_wait_for_event(decoder->imageResizer->component,
			    OMX_EventCmdComplete,
			    OMX_CommandPortEnable, 1,
			    decoder->imageResizer->outPort, 1, 0,
			    TIMEOUT_MS);

    return OMXJPEG_OK;
}

int
portSettingsChangedAgain(OPENMAX_JPEG_DECODER * decoder)
{
    ilclient_disable_port(decoder->imageDecoder->component,
			  decoder->imageDecoder->outPort);
    ilclient_disable_port(decoder->imageResizer->component,
			  decoder->imageResizer->inPort);

    OMX_PARAM_PORTDEFINITIONTYPE portdef;

    // need to setup the input for the resizer with the output of the
    // decoder
    portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
    portdef.nVersion.nVersion = OMX_VERSION;
    portdef.nPortIndex = decoder->imageDecoder->outPort;
    OMX_GetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    // tell resizer input what the decoder output will be providing
    portdef.nPortIndex = decoder->imageResizer->inPort;
    OMX_SetParameter(decoder->imageResizer->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    // enable output of decoder and input of resizer (ie enable tunnel)
    ilclient_enable_port(decoder->imageDecoder->component,
			 decoder->imageDecoder->outPort);
    ilclient_enable_port(decoder->imageResizer->component,
			 decoder->imageResizer->inPort);

    // need to wait for this event
    ilclient_wait_for_event(decoder->imageResizer->component,
			    OMX_EventPortSettingsChanged,
			    decoder->imageResizer->outPort, 1,
			    0, 0, 0, TIMEOUT_MS);

    return OMXJPEG_OK;
}

int
prepareResizer(OPENMAX_JPEG_DECODER * decoder)
{
    decoder->imageResizer = malloc(sizeof(COMPONENT_DETAILS));
    if (decoder->imageResizer == NULL) {
	perror("malloc image resizer");
	return OMXJPEG_ERROR_MEMORY;
    }

    int             ret = ilclient_create_component(decoder->client,
						    &decoder->
						    imageResizer->
						    component,
						    "resize",
						    ILCLIENT_DISABLE_ALL_PORTS
						    |
						    ILCLIENT_ENABLE_INPUT_BUFFERS
						    |
						    ILCLIENT_ENABLE_OUTPUT_BUFFERS);
    if (ret != 0) {
	perror("image resizer");
	return OMXJPEG_ERROR_CREATING_COMP;
    }
    // grab the handle for later use
    decoder->imageResizer->handle =
	ILC_GET_HANDLE(decoder->imageResizer->component);

    // get and store the ports
    OMX_PORT_PARAM_TYPE port;
    port.nSize = sizeof(OMX_PORT_PARAM_TYPE);
    port.nVersion.nVersion = OMX_VERSION;

    OMX_GetParameter(ILC_GET_HANDLE(decoder->imageResizer->component),
		     OMX_IndexParamImageInit, &port);
    if (port.nPorts != 2) {
	return OMXJPEG_ERROR_WRONG_NO_PORTS;
    }
    decoder->imageResizer->inPort = port.nStartPortNumber;
    decoder->imageResizer->outPort = port.nStartPortNumber + 1;

    decoder->pOutputBufferHeader = NULL;

    return OMXJPEG_OK;
}

int
prepareImageDecoder(OPENMAX_JPEG_DECODER * decoder)
{
    decoder->imageDecoder = malloc(sizeof(COMPONENT_DETAILS));
    if (decoder->imageDecoder == NULL) {
	perror("malloc image decoder");
	return OMXJPEG_ERROR_MEMORY;
    }

    int             ret = ilclient_create_component(decoder->client,
						    &decoder->
						    imageDecoder->
						    component,
						    "image_decode",
						    ILCLIENT_DISABLE_ALL_PORTS
						    |
						    ILCLIENT_ENABLE_INPUT_BUFFERS);

    if (ret != 0) {
	perror("image decode");
	return OMXJPEG_ERROR_CREATING_COMP;
    }
    // grab the handle for later use in OMX calls directly
    decoder->imageDecoder->handle =
	ILC_GET_HANDLE(decoder->imageDecoder->component);

    // get and store the ports
    OMX_PORT_PARAM_TYPE port;
    port.nSize = sizeof(OMX_PORT_PARAM_TYPE);
    port.nVersion.nVersion = OMX_VERSION;

    OMX_GetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamImageInit, &port);
    if (port.nPorts != 2) {
	return OMXJPEG_ERROR_WRONG_NO_PORTS;
    }
    decoder->imageDecoder->inPort = port.nStartPortNumber;
    decoder->imageDecoder->outPort = port.nStartPortNumber + 1;

    return OMXJPEG_OK;
}

int
startupImageDecoder(OPENMAX_JPEG_DECODER * decoder)
{
    // move to idle
    ilclient_change_component_state(decoder->imageDecoder->component,
				    OMX_StateIdle);

    // set input image format
    OMX_IMAGE_PARAM_PORTFORMATTYPE imagePortFormat;
    memset(&imagePortFormat, 0, sizeof(OMX_IMAGE_PARAM_PORTFORMATTYPE));
    imagePortFormat.nSize = sizeof(OMX_IMAGE_PARAM_PORTFORMATTYPE);
    imagePortFormat.nVersion.nVersion = OMX_VERSION;
    imagePortFormat.nPortIndex = decoder->imageDecoder->inPort;
    imagePortFormat.eCompressionFormat = OMX_IMAGE_CodingJPEG;
    OMX_SetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamImagePortFormat, &imagePortFormat);

    // get buffer requirements
    OMX_PARAM_PORTDEFINITIONTYPE portdef;
    portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
    portdef.nVersion.nVersion = OMX_VERSION;
    portdef.nPortIndex = decoder->imageDecoder->inPort;
    OMX_GetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    // enable the port and setup the buffers
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandPortEnable,
		    decoder->imageDecoder->inPort, NULL);
    decoder->inputBufferHeaderCount = portdef.nBufferCountActual;
    // allocate pointer array
    decoder->ppInputBufferHeader =
	(OMX_BUFFERHEADERTYPE **) malloc(sizeof(void) *
					 decoder->inputBufferHeaderCount);
    // allocate each buffer
    int             i;
    for (i = 0; i < decoder->inputBufferHeaderCount; i++) {
	if (OMX_AllocateBuffer(decoder->imageDecoder->handle,
			       &decoder->ppInputBufferHeader[i],
			       decoder->imageDecoder->inPort,
			       (void *) i,
			       portdef.nBufferSize) != OMX_ErrorNone) {
	    perror("Allocate decode buffer");
	    return OMXJPEG_ERROR_MEMORY;
	}
    }
    // wait for port enable to complete - which it should once buffers are 
    // assigned
    int             ret =
	ilclient_wait_for_event(decoder->imageDecoder->component,
				OMX_EventCmdComplete,
				OMX_CommandPortEnable, 0,
				decoder->imageDecoder->inPort, 0,
				0, TIMEOUT_MS);
    if (ret != 0) {
	fprintf(stderr, "Did not get port enable %d\n", ret);
	return OMXJPEG_ERROR_EXECUTING;
    }
    // start executing the decoder 
    ret = OMX_SendCommand(decoder->imageDecoder->handle,
			  OMX_CommandStateSet, OMX_StateExecuting, NULL);
    if (ret != 0) {
	fprintf(stderr, "Error starting image decoder %x\n", ret);
	return OMXJPEG_ERROR_EXECUTING;
    }
    ret = ilclient_wait_for_event(decoder->imageDecoder->component,
				  OMX_EventCmdComplete,
				  OMX_StateExecuting, 0, 0, 1, 0,
				  TIMEOUT_MS);
    if (ret != 0) {
	fprintf(stderr, "Did not receive executing stat %d\n", ret);
	// return OMXJPEG_ERROR_EXECUTING;
    }

    return OMXJPEG_OK;
}

// this function run the boilerplate to setup the openmax components;
int
setupOpenMaxJpegDecoder(OPENMAX_JPEG_DECODER ** pDecoder)
{
    *pDecoder = malloc(sizeof(OPENMAX_JPEG_DECODER));
    if (pDecoder[0] == NULL) {
	perror("malloc decoder");
	return OMXJPEG_ERROR_MEMORY;
    }
    memset(*pDecoder, 0, sizeof(OPENMAX_JPEG_DECODER));

    if ((pDecoder[0]->client = ilclient_init()) == NULL) {
	perror("ilclient_init");
	return OMXJPEG_ERROR_ILCLIENT_INIT;
    }

    if (OMX_Init() != OMX_ErrorNone) {
	ilclient_destroy(pDecoder[0]->client);
	perror("OMX_Init");
	return OMXJPEG_ERROR_OMX_INIT;
    }
    // prepare the image decoder
    int             ret = prepareImageDecoder(pDecoder[0]);
    if (ret != OMXJPEG_OK)
	return ret;

    ret = prepareResizer(pDecoder[0]);
    if (ret != OMXJPEG_OK)
	return ret;

    ret = startupImageDecoder(pDecoder[0]);
    if (ret != OMXJPEG_OK)
	return ret;

    return OMXJPEG_OK;
}

void save_image_as_TGA(OPENMAX_JPEG_DECODER * decoder) {
    OMX_PARAM_PORTDEFINITIONTYPE portdef;
    unsigned int uWidth, uHeight, nSize, nStride, nSliceHeight;
#define TGA_HEADER_SIZE 18
    char header[TGA_HEADER_SIZE];

    // query output buffer requirements for resizer
    portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
    portdef.nVersion.nVersion = OMX_VERSION;
    portdef.nPortIndex = decoder->imageResizer->outPort;
    OMX_GetParameter(decoder->imageResizer->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    uWidth =     portdef.format.image.nFrameWidth;
    uHeight = portdef.format.image.nFrameHeight;
    nSliceHeight = portdef.format.image.nSliceHeight;
    nStride = portdef.format.image.nStride;
    nSize = portdef.nBufferSize;
    //nSize = portdef.nFilledLen;
    printf("Width is %d stride %d height is %d slice height %d, size %d\n", 
	   uWidth, nStride, uHeight, nSliceHeight, nSize);
    // image in decoder->pOutputBufferHeader
    

    memset(header, 0, TGA_HEADER_SIZE);
    // header[0] = nSize;
    header[0x2] = 2;
    header[0xD] = (nStride/4) >> 8;
    header[0xC] = (nStride/4) & 0xFF;
    //header[0xF] = uHeight >> 8;
    //header[0xE] = uHeight & 0xFF;
    header[0xF] = nSliceHeight >> 8;
    header[0xE] = nSliceHeight & 0xFF;
    header[0x10] = 32;

    FILE *fout = fopen("out.tga", "w");
    fwrite(header, sizeof(char), TGA_HEADER_SIZE, fout);

    // This writes the image upside down but is simple
    // Should do it by rows, from end to beginning
    int nwrote;
    nwrote = fwrite(decoder->pOutputBufferHeader->pBuffer, sizeof(char),
	   nSize, fout);
    printf("Written %d\n", nwrote);
    fclose(fout);
}

// this function passed the jpeg image buffer in, and returns the decoded
// image
int
decodeImage(OPENMAX_JPEG_DECODER * decoder, char *sourceImage,
	    size_t imageSize)
{
    char           *sourceOffset = sourceImage;	// we store a seperate
						// buffer ot image so we
						// can offset it
    size_t          toread = 0;	// bytes left to read from buffer
    toread += imageSize;
    int             bFilled = 0;	// have we filled our output
					// buffer
    bufferIndex = 0;

    OMX_BUFFERHEADERTYPE *pBufHeader;
    while (toread > 0) {
	// get next buffer from array
	pBufHeader =
	    decoder->ppInputBufferHeader[bufferIndex];

	// step index and reset to 0 if required
	bufferIndex++;
	if (bufferIndex >= decoder->inputBufferHeaderCount)
	    bufferIndex = 0;

	// work out the next chunk to load into the decoder
	if (toread > pBufHeader->nAllocLen)
	    pBufHeader->nFilledLen = pBufHeader->nAllocLen;
	else
	    pBufHeader->nFilledLen = toread;

	toread = toread - pBufHeader->nFilledLen;

	// pass the bytes to the buffer
	memcpy(pBufHeader->pBuffer, sourceOffset, pBufHeader->nFilledLen);

	// update the buffer pointer and set the input flags

	sourceOffset = sourceOffset + pBufHeader->nFilledLen;
	pBufHeader->nOffset = 0;
	pBufHeader->nFlags = 0;
	if (toread <= 0) {
	    pBufHeader->nFlags = OMX_BUFFERFLAG_EOS;
	}
	// empty the current buffer
	int             ret =
	    OMX_EmptyThisBuffer(decoder->imageDecoder->handle,
				pBufHeader);

	if (ret != OMX_ErrorNone) {
	    perror("Empty input buffer");
	    fprintf(stderr, "return code %x\n", ret);
	    return OMXJPEG_ERROR_MEMORY;
	}
	// wait for buffer to empty or port changed event
	int             done = 0;
	while ((done == 0) || (decoder->pOutputBufferHeader == NULL)) {
	    if (decoder->pOutputBufferHeader == NULL) {
		ret =
		    ilclient_wait_for_event
		    (decoder->imageDecoder->component,
		     OMX_EventPortSettingsChanged,
		     decoder->imageDecoder->outPort, 0, 0, 1, 0, 5);

		if (ret == 0) {
		    portSettingsChanged(decoder);
		}
	    } else {
		ret =
		    ilclient_remove_event(decoder->imageDecoder->component,
					  OMX_EventPortSettingsChanged,
					  decoder->imageDecoder->outPort,
					  0, 0, 1);
		if (ret == 0)
		    portSettingsChangedAgain(decoder);

	    }

	    // check to see if buffer is now empty
	    if (pBufHeader->nFilledLen == 0)
		done = 1;

	    if ((done == 0)
		|| (decoder->pOutputBufferHeader == NULL))
		sleep(1);
	}

	// fill the buffer if we have created the buffer
	if (bFilled == 0) {
	    if ((decoder->pOutputBufferHeader == NULL)) {
		portSettingsChanged(decoder);
	    }
	    ret = OMX_FillThisBuffer(decoder->imageResizer->handle,
				     decoder->pOutputBufferHeader);
	    if (ret != OMX_ErrorNone) {
		perror("Filling output buffer");
		fprintf(stderr, "Error code %x\n", ret);
		return OMXJPEG_ERROR_MEMORY;
	    }

	    bFilled = 1;
	}
    }

    // wait for buffer to fill
    /*
    while(pBufHeader->nFilledLen == 0) { printf("Sleeping\n"); sleep(5); } 
    */

    // wait for end of stream events
    int             ret =
	ilclient_wait_for_event(decoder->imageDecoder->component,
				OMX_EventBufferFlag,
				decoder->imageDecoder->outPort, 1,
				OMX_BUFFERFLAG_EOS, 1,
				0, 2);
    if (ret != 0) {
	fprintf(stderr, "No EOS event on image decoder %d\n", ret);
    }
    ret = ilclient_wait_for_event(decoder->imageResizer->component,
				  OMX_EventBufferFlag,
				  decoder->imageResizer->outPort, 1,
				  OMX_BUFFERFLAG_EOS, 1, 0, 2);
    if (ret != 0) {
	fprintf(stderr, "No EOS event on image resizer %d\n", ret);
    }

    printf("Filled buffer len is %d\n", pBufHeader->nFilledLen);
    save_image_as_TGA(decoder);

    return OMXJPEG_OK;
}

// this function cleans up the decoder.
void
cleanup(OPENMAX_JPEG_DECODER * decoder)
{
    // flush everything through
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandFlush, decoder->imageDecoder->outPort,
		    NULL);
    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandFlush, 0,
			    decoder->imageDecoder->outPort, 0, 0,
			    TIMEOUT_MS);
    OMX_SendCommand(decoder->imageResizer->handle, OMX_CommandFlush,
		    decoder->imageResizer->inPort, NULL);
    ilclient_wait_for_event(decoder->imageResizer->component,
			    OMX_EventCmdComplete, OMX_CommandFlush, 0,
			    decoder->imageResizer->inPort, 1, 0,
			    TIMEOUT_MS);

    OMX_SendCommand(decoder->imageDecoder->handle, OMX_CommandPortDisable,
		    decoder->imageDecoder->inPort, NULL);

    int             i = 0;
    for (i = 0; i < decoder->inputBufferHeaderCount; i++) {
	OMX_BUFFERHEADERTYPE *vpBufHeader =
	    decoder->ppInputBufferHeader[i];

	OMX_FreeBuffer(decoder->imageDecoder->handle,
		       decoder->imageDecoder->inPort, vpBufHeader);
    }

    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandPortDisable,
			    0, decoder->imageDecoder->inPort, 0, 0,
			    TIMEOUT_MS);

    OMX_SendCommand(decoder->imageResizer->handle, OMX_CommandPortDisable,
		    decoder->imageResizer->outPort, NULL);

    OMX_FreeBuffer(decoder->imageResizer->handle,
		   decoder->imageResizer->outPort,
		   decoder->pOutputBufferHeader);

    ilclient_wait_for_event(decoder->imageResizer->component,
			    OMX_EventCmdComplete, OMX_CommandPortDisable,
			    0, decoder->imageResizer->outPort, 0, 0,
			    TIMEOUT_MS);

    OMX_SendCommand(decoder->imageDecoder->handle, OMX_CommandPortDisable,
		    decoder->imageDecoder->outPort, NULL);
    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandPortDisable,
			    0, decoder->imageDecoder->outPort, 0, 0,
			    TIMEOUT_MS);

    OMX_SendCommand(decoder->imageResizer->handle, OMX_CommandPortDisable,
		    decoder->imageResizer->inPort, NULL);
    ilclient_wait_for_event(decoder->imageResizer->component,
			    OMX_EventCmdComplete, OMX_CommandPortDisable,
			    0, decoder->imageResizer->inPort, 0, 0,
			    TIMEOUT_MS);

    OMX_SetupTunnel(decoder->imageDecoder->handle,
		    decoder->imageDecoder->outPort, NULL, 0);
    OMX_SetupTunnel(decoder->imageResizer->handle,
		    decoder->imageResizer->inPort, NULL, 0);

    ilclient_change_component_state(decoder->imageDecoder->component,
				    OMX_StateIdle);
    ilclient_change_component_state(decoder->imageResizer->component,
				    OMX_StateIdle);

    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandStateSet, 0,
			    OMX_StateIdle, 0, 0, TIMEOUT_MS);
    ilclient_wait_for_event(decoder->imageResizer->component,
			    OMX_EventCmdComplete, OMX_CommandStateSet, 0,
			    OMX_StateIdle, 0, 0, TIMEOUT_MS);

    ilclient_change_component_state(decoder->imageDecoder->component,
				    OMX_StateLoaded);
    ilclient_change_component_state(decoder->imageResizer->component,
				    OMX_StateLoaded);

    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandStateSet, 0,
			    OMX_StateLoaded, 0, 0, TIMEOUT_MS);
    ilclient_wait_for_event(decoder->imageResizer->component,
			    OMX_EventCmdComplete, OMX_CommandStateSet, 0,
			    OMX_StateLoaded, 0, 0, TIMEOUT_MS);

    OMX_Deinit();

    if (decoder->client != NULL) {
	ilclient_destroy(decoder->client);
    }
}

int
main(int argc, char *argv[])
{
    OPENMAX_JPEG_DECODER *pDecoder;
    char           *sourceImage;
    size_t          imageSize;
    int             s;
    if (argc < 2) {
	printf("Usage: %s <filename>\n", argv[0]);
	return -1;
    }
    FILE           *fp = fopen(argv[1], "rb");
    if (!fp) {
	printf("File %s not found.\n", argv[1]);
    }
    fseek(fp, 0L, SEEK_END);
    imageSize = ftell(fp);
    fseek(fp, 0L, SEEK_SET);
    sourceImage = malloc(imageSize);
    assert(sourceImage != NULL);
    s = fread(sourceImage, 1, imageSize, fp);
    assert(s == imageSize);
    fclose(fp);
    bcm_host_init();
    s = setupOpenMaxJpegDecoder(&pDecoder);
    assert(s == 0);
    s = decodeImage(pDecoder, sourceImage, imageSize);
    assert(s == 0);
    cleanup(pDecoder);
    free(sourceImage);
    return 0;
}

      

Rendering a JPEG image

In order to display am image we need to render it on some output device. There is no specific component to render images. Broadcom has a video_render which will accept both a video stream or a single image. Now this component only has one input port, and that port is a video port. If we want to take an image from an output image port, we would have to play some format conversion games. Or, we can connect an image output port to a video input port by using a tunnel, and let the components sort it out themselves. This is much easier!

To render a JPEG image, we only need to use an image_decode and a video_render component. We need to set up the input port for the decoder as in the last example. The video render component has no output port so nothing needs to be done for that. The two ports which are connected by a tunnel do all necessary setups and conversions themselves. So this is actually simpler than the last program: load and empty input buffers with the JPEG image, setup the tunnel when port settings change, and ... that's it! Note that we still defer setting up the tunnel until the port settings have changed - otherwise the buffer sizes would be wrong.

The program is renderjpeg.c

	/*
Copyright (c) 2012, Matt Ownby
                    Anthong Sale
Copyright (c) 2014, Jan Newmarch

All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
    * Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.
    * Neither the name of the copyright holder nor the
      names of its contributors may be used to endorse or promote products
      derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

#include <stdio.h>
#include <assert.h>
#include "jpeg.h"

#define TIMEOUT_MS 2000

typedef struct _COMPONENT_DETAILS {
    COMPONENT_T    *component;
    OMX_HANDLETYPE  handle;
    int             inPort;
    int             outPort;
} COMPONENT_DETAILS;

struct _OPENMAX_JPEG_DECODER {
    ILCLIENT_T     *client;
    COMPONENT_DETAILS *imageDecoder;
    COMPONENT_DETAILS *imageRender;
    OMX_BUFFERHEADERTYPE **ppInputBufferHeader;
    int             inputBufferHeaderCount;
    OMX_BUFFERHEADERTYPE *pOutputBufferHeader;
};

int             bufferIndex = 0;	// index to buffer array

int
portSettingsChanged(OPENMAX_JPEG_DECODER * decoder)
{
    OMX_PARAM_PORTDEFINITIONTYPE portdef;
    int ret;

    printf("Port settings changed\n");

    // disable ports
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandPortDisable,
		    decoder->imageDecoder->outPort, NULL);
    OMX_SendCommand(decoder->imageRender->handle,
		    OMX_CommandPortDisable,
		    decoder->imageRender->inPort, NULL);

    // state to idle
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandStateSet, OMX_StateIdle, NULL);
     OMX_SendCommand(decoder->imageRender->handle,
		    OMX_CommandStateSet, OMX_StateIdle, NULL);

    // establish tunnel between decoder output and render input
    ret = OMX_SetupTunnel(decoder->imageDecoder->handle,
		    decoder->imageDecoder->outPort,
		    decoder->imageRender->handle,
		    decoder->imageRender->inPort);
    if (ret != OMX_ErrorNone) {
	fprintf(stderr, "Error setting up tunnel %X\n", ret);
	return OMXJPEG_ERROR_MEMORY;
    } else {
	printf("Tunnel set up ok\n");
    }

    // wait for ports to become disabled
    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete,
			    OMX_CommandPortDisable, 1,
			    decoder->imageDecoder->outPort, 1, 0,
			    TIMEOUT_MS);
    ilclient_wait_for_event(decoder->imageRender->component,
			    OMX_EventCmdComplete, OMX_CommandPortDisable, 1,
			    decoder->imageRender->inPort, 1, 0,
			    TIMEOUT_MS);
    printf("Port disable complete\n");

    // enable ports
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandPortEnable,
		    decoder->imageDecoder->outPort, NULL);
    OMX_SendCommand(decoder->imageRender->handle,
		    OMX_CommandPortEnable,
		    decoder->imageRender->inPort, NULL);

    // wait for state change complete
    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete,
			    OMX_CommandStateSet, 1,
			    OMX_StateIdle, 1, 0, TIMEOUT_MS);
    printf("Decoder in idle state\n");

    ilclient_wait_for_event(decoder->imageRender->component,
			    OMX_EventCmdComplete,
			    OMX_CommandStateSet, 1,
			    OMX_StateIdle, 1, 0, TIMEOUT_MS);
    printf("Render in idle state\n");

    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete,
			    OMX_CommandPortEnable, 1,
			    decoder->imageDecoder->outPort, 1, 0,
			    TIMEOUT_MS);
    ilclient_wait_for_event(decoder->imageRender->component,
			    OMX_EventCmdComplete, OMX_CommandPortEnable, 1,
			    decoder->imageRender->inPort, 1, 0,
			    TIMEOUT_MS);
    printf("Ports re-enabled\n");

    return OMXJPEG_OK;
}

int
prepareRender(OPENMAX_JPEG_DECODER * decoder)
{
    decoder->imageRender = malloc(sizeof(COMPONENT_DETAILS));
    if (decoder->imageRender == NULL) {
	perror("malloc image render");
	return OMXJPEG_ERROR_MEMORY;
    }

    int             ret = ilclient_create_component(decoder->client,
						    &decoder->
						    imageRender->
						    component,
						    "video_render",
						    ILCLIENT_DISABLE_ALL_PORTS
						    |
						    ILCLIENT_ENABLE_INPUT_BUFFERS
						    );
    if (ret != 0) {
	perror("image render");
	return OMXJPEG_ERROR_CREATING_COMP;
    }
    // grab the handle for later use
    decoder->imageRender->handle =
	ILC_GET_HANDLE(decoder->imageRender->component);

    // get and store the ports
    OMX_PORT_PARAM_TYPE port;
    port.nSize = sizeof(OMX_PORT_PARAM_TYPE);
    port.nVersion.nVersion = OMX_VERSION;

    OMX_GetParameter(ILC_GET_HANDLE(decoder->imageRender->component),
		     OMX_IndexParamVideoInit, &port);
    if (port.nPorts != 1) {
	return OMXJPEG_ERROR_WRONG_NO_PORTS;
    }
    decoder->imageRender->inPort = port.nStartPortNumber;
    /*
    decoder->imageRender->outPort = port.nStartPortNumber + 1;
    */

    decoder->pOutputBufferHeader = NULL;

    return OMXJPEG_OK;
}

int
prepareImageDecoder(OPENMAX_JPEG_DECODER * decoder)
{
    decoder->imageDecoder = malloc(sizeof(COMPONENT_DETAILS));
    if (decoder->imageDecoder == NULL) {
	perror("malloc image decoder");
	return OMXJPEG_ERROR_MEMORY;
    }

    int             ret = ilclient_create_component(decoder->client,
						    &decoder->
						    imageDecoder->
						    component,
						    "image_decode",
						    ILCLIENT_DISABLE_ALL_PORTS
						    |
						    ILCLIENT_ENABLE_INPUT_BUFFERS);

    if (ret != 0) {
	perror("image decode");
	return OMXJPEG_ERROR_CREATING_COMP;
    }
    // grab the handle for later use in OMX calls directly
    decoder->imageDecoder->handle =
	ILC_GET_HANDLE(decoder->imageDecoder->component);

    // get and store the ports
    OMX_PORT_PARAM_TYPE port;
    port.nSize = sizeof(OMX_PORT_PARAM_TYPE);
    port.nVersion.nVersion = OMX_VERSION;

    OMX_GetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamImageInit, &port);
    if (port.nPorts != 2) {
	return OMXJPEG_ERROR_WRONG_NO_PORTS;
    }
    decoder->imageDecoder->inPort = port.nStartPortNumber;
    decoder->imageDecoder->outPort = port.nStartPortNumber + 1;

    return OMXJPEG_OK;
}

int
startupImageDecoder(OPENMAX_JPEG_DECODER * decoder)
{
    // move to idle
    ilclient_change_component_state(decoder->imageDecoder->component,
				    OMX_StateIdle);

    // set input image format
    OMX_IMAGE_PARAM_PORTFORMATTYPE imagePortFormat;
    memset(&imagePortFormat, 0, sizeof(OMX_IMAGE_PARAM_PORTFORMATTYPE));
    imagePortFormat.nSize = sizeof(OMX_IMAGE_PARAM_PORTFORMATTYPE);
    imagePortFormat.nVersion.nVersion = OMX_VERSION;
    imagePortFormat.nPortIndex = decoder->imageDecoder->inPort;
    imagePortFormat.eCompressionFormat = OMX_IMAGE_CodingJPEG;
    OMX_SetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamImagePortFormat, &imagePortFormat);

    // get buffer requirements
    OMX_PARAM_PORTDEFINITIONTYPE portdef;
    portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
    portdef.nVersion.nVersion = OMX_VERSION;
    portdef.nPortIndex = decoder->imageDecoder->inPort;
    OMX_GetParameter(decoder->imageDecoder->handle,
		     OMX_IndexParamPortDefinition, &portdef);

    // enable the port and setup the buffers
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandPortEnable,
		    decoder->imageDecoder->inPort, NULL);
    decoder->inputBufferHeaderCount = portdef.nBufferCountActual;
    // allocate pointer array
    decoder->ppInputBufferHeader =
	(OMX_BUFFERHEADERTYPE **) malloc(sizeof(void) *
					 decoder->inputBufferHeaderCount);
    // allocate each buffer
    int             i;
    for (i = 0; i < decoder->inputBufferHeaderCount; i++) {
	if (OMX_AllocateBuffer(decoder->imageDecoder->handle,
			       &decoder->ppInputBufferHeader[i],
			       decoder->imageDecoder->inPort,
			       (void *) i,
			       portdef.nBufferSize) != OMX_ErrorNone) {
	    perror("Allocate decode buffer");
	    return OMXJPEG_ERROR_MEMORY;
	}
    }
    // wait for port enable to complete - which it should once buffers are 
    // assigned
    int             ret =
	ilclient_wait_for_event(decoder->imageDecoder->component,
				OMX_EventCmdComplete,
				OMX_CommandPortEnable, 0,
				decoder->imageDecoder->inPort, 0,
				0, TIMEOUT_MS);
    if (ret != 0) {
	fprintf(stderr, "Did not get port enable %d\n", ret);
	return OMXJPEG_ERROR_EXECUTING;
    }
    // start executing the decoder 
    ret = OMX_SendCommand(decoder->imageDecoder->handle,
			  OMX_CommandStateSet, OMX_StateExecuting, NULL);
    if (ret != 0) {
	fprintf(stderr, "Error starting image decoder %x\n", ret);
	return OMXJPEG_ERROR_EXECUTING;
    }
    ret = ilclient_wait_for_event(decoder->imageDecoder->component,
				  OMX_EventCmdComplete,
				  OMX_StateExecuting, 0, 0, 1, 0,
				  TIMEOUT_MS);
    if (ret != 0) {
	fprintf(stderr, "Did not receive executing stat %d\n", ret);
	// return OMXJPEG_ERROR_EXECUTING;
    }

    return OMXJPEG_OK;
}

// this function run the boilerplate to setup the openmax components;
int
setupOpenMaxJpegDecoder(OPENMAX_JPEG_DECODER ** pDecoder)
{
    *pDecoder = malloc(sizeof(OPENMAX_JPEG_DECODER));
    if (pDecoder[0] == NULL) {
	perror("malloc decoder");
	return OMXJPEG_ERROR_MEMORY;
    }
    memset(*pDecoder, 0, sizeof(OPENMAX_JPEG_DECODER));

    if ((pDecoder[0]->client = ilclient_init()) == NULL) {
	perror("ilclient_init");
	return OMXJPEG_ERROR_ILCLIENT_INIT;
    }

    if (OMX_Init() != OMX_ErrorNone) {
	ilclient_destroy(pDecoder[0]->client);
	perror("OMX_Init");
	return OMXJPEG_ERROR_OMX_INIT;
    }
    // prepare the image decoder
    int             ret = prepareImageDecoder(pDecoder[0]);
    if (ret != OMXJPEG_OK)
	return ret;

    ret = prepareRender(pDecoder[0]);
    if (ret != OMXJPEG_OK)
	return ret;

    ret = startupImageDecoder(pDecoder[0]);
    if (ret != OMXJPEG_OK)
	return ret;

    return OMXJPEG_OK;
}

// this function passed the jpeg image buffer in, and returns the decoded
// image
int
decodeImage(OPENMAX_JPEG_DECODER * decoder, char *sourceImage,
	    size_t imageSize)
{
    char           *sourceOffset = sourceImage;	// we store a seperate
						// buffer ot image so we
						// can offset it
    size_t          toread = 0;	// bytes left to read from buffer
    toread += imageSize;
    int             bFilled = 0;	// have we filled our output
    int bTunnelCreated = 0;			// buffer

    bufferIndex = 0;

    OMX_BUFFERHEADERTYPE *pBufHeader;
    while (toread > 0) {
	// get next buffer from array
	pBufHeader =
	    decoder->ppInputBufferHeader[bufferIndex];

	// step index and reset to 0 if required
	bufferIndex++;
	if (bufferIndex >= decoder->inputBufferHeaderCount)
	    bufferIndex = 0;

	// work out the next chunk to load into the decoder
	if (toread > pBufHeader->nAllocLen)
	    pBufHeader->nFilledLen = pBufHeader->nAllocLen;
	else
	    pBufHeader->nFilledLen = toread;

	toread = toread - pBufHeader->nFilledLen;

	// pass the bytes to the buffer
	memcpy(pBufHeader->pBuffer, sourceOffset, pBufHeader->nFilledLen);

	// update the buffer pointer and set the input flags

	sourceOffset = sourceOffset + pBufHeader->nFilledLen;
	pBufHeader->nOffset = 0;
	pBufHeader->nFlags = 0;
	if (toread <= 0) {
	    pBufHeader->nFlags = OMX_BUFFERFLAG_EOS;
	    printf("EOS added to stream\n");
	}
	printf("Input buffer called to empty %d\n",
	       pBufHeader->nFilledLen);

	// empty the current buffer
	int             ret =
	    OMX_EmptyThisBuffer(decoder->imageDecoder->handle,
				pBufHeader);

	if (ret != OMX_ErrorNone) {
	    fprintf(stderr, "Empty input buffer return code %x\n", ret);
	    return OMXJPEG_ERROR_MEMORY;
	}

	// wait for buffer to empty or port changed event
	int             done = 0;
	//while ((done == 0) || (decoder->pOutputBufferHeader == NULL)) {
	while (done == 0) {
	    // if (decoder->pOutputBufferHeader == NULL) {
	    if (! bTunnelCreated) {
		printf("Tunnel not yet created\n");
		ret =
		    ilclient_wait_for_event
		    (decoder->imageDecoder->component,
		     OMX_EventPortSettingsChanged,
		     decoder->imageDecoder->outPort, 0, 0, 1, 0, 5);

		if ((ret == 0) && ! bTunnelCreated) {
		    portSettingsChanged(decoder);
		    bTunnelCreated = 1;
		}
	    }

	    // check to see if buffer is now empty
	    if (pBufHeader->nFilledLen == 0) {
		done = 1;
		printf("Input buffer finished emptying\n");
	    } else {
		printf("Still %d in input buffer\n", 
		       pBufHeader->nFilledLen);
		// is there an event we can wait on instead of this?
		sleep(1);
	    } 
	}
    }

    // wait for end of stream events
    int             ret =
	ilclient_wait_for_event(decoder->imageDecoder->component,
				OMX_EventBufferFlag,
				decoder->imageDecoder->outPort, 1,
				OMX_BUFFERFLAG_EOS, 1,
				0, 2);
    if (ret != 0) {
	fprintf(stderr, "No EOS event on image decoder %d\n", ret);
    } else {
	printf("EOS event on decoder\n");
    }
    return OMXJPEG_OK;
}

// this function cleans up the decoder.
void
cleanup(OPENMAX_JPEG_DECODER * decoder)
{
    // flush everything through
    OMX_SendCommand(decoder->imageDecoder->handle,
		    OMX_CommandFlush, decoder->imageDecoder->outPort,
		    NULL);
    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandFlush, 0,
			    decoder->imageDecoder->outPort, 0, 0,
			    TIMEOUT_MS);
    OMX_SendCommand(decoder->imageRender->handle, OMX_CommandFlush,
		    decoder->imageRender->inPort, NULL);
    ilclient_wait_for_event(decoder->imageRender->component,
			    OMX_EventCmdComplete, OMX_CommandFlush, 0,
			    decoder->imageRender->inPort, 1, 0,
			    TIMEOUT_MS);

    OMX_SendCommand(decoder->imageDecoder->handle, OMX_CommandPortDisable,
		    decoder->imageDecoder->inPort, NULL);

    int             i = 0;
    for (i = 0; i < decoder->inputBufferHeaderCount; i++) {
	OMX_BUFFERHEADERTYPE *vpBufHeader =
	    decoder->ppInputBufferHeader[i];

	OMX_FreeBuffer(decoder->imageDecoder->handle,
		       decoder->imageDecoder->inPort, vpBufHeader);
    }

    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandPortDisable,
			    0, decoder->imageDecoder->inPort, 0, 0,
			    TIMEOUT_MS);

    OMX_SendCommand(decoder->imageDecoder->handle, OMX_CommandPortDisable,
		    decoder->imageDecoder->outPort, NULL);
    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandPortDisable,
			    0, decoder->imageDecoder->outPort, 0, 0,
			    TIMEOUT_MS);

    OMX_SendCommand(decoder->imageRender->handle, OMX_CommandPortDisable,
		    decoder->imageRender->inPort, NULL);
    ilclient_wait_for_event(decoder->imageRender->component,
			    OMX_EventCmdComplete, OMX_CommandPortDisable,
			    0, decoder->imageRender->inPort, 0, 0,
			    TIMEOUT_MS);

    OMX_SetupTunnel(decoder->imageDecoder->handle,
		    decoder->imageDecoder->outPort, NULL, 0);
    OMX_SetupTunnel(decoder->imageRender->handle,
		    decoder->imageRender->inPort, NULL, 0);

    ilclient_change_component_state(decoder->imageDecoder->component,
				    OMX_StateIdle);
    ilclient_change_component_state(decoder->imageRender->component,
				    OMX_StateIdle);

    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandStateSet, 0,
			    OMX_StateIdle, 0, 0, TIMEOUT_MS);
    ilclient_wait_for_event(decoder->imageRender->component,
			    OMX_EventCmdComplete, OMX_CommandStateSet, 0,
			    OMX_StateIdle, 0, 0, TIMEOUT_MS);

    ilclient_change_component_state(decoder->imageDecoder->component,
				    OMX_StateLoaded);
    ilclient_change_component_state(decoder->imageRender->component,
				    OMX_StateLoaded);

    ilclient_wait_for_event(decoder->imageDecoder->component,
			    OMX_EventCmdComplete, OMX_CommandStateSet, 0,
			    OMX_StateLoaded, 0, 0, TIMEOUT_MS);
    ilclient_wait_for_event(decoder->imageRender->component,
			    OMX_EventCmdComplete, OMX_CommandStateSet, 0,
			    OMX_StateLoaded, 0, 0, TIMEOUT_MS);

    OMX_Deinit();

    if (decoder->client != NULL) {
	ilclient_destroy(decoder->client);
    }
}

int
main(int argc, char *argv[])
{
    OPENMAX_JPEG_DECODER *pDecoder;
    char           *sourceImage;
    size_t          imageSize;
    int             s;
    if (argc < 2) {
	printf("Usage: %s <filename>\n", argv[0]);
	return -1;
    }
    FILE           *fp = fopen(argv[1], "rb");
    if (!fp) {
	printf("File %s not found.\n", argv[1]);
    }
    fseek(fp, 0L, SEEK_END);
    imageSize = ftell(fp);
    fseek(fp, 0L, SEEK_SET);
    sourceImage = malloc(imageSize);
    assert(sourceImage != NULL);
    s = fread(sourceImage, 1, imageSize, fp);
    assert(s == imageSize);
    fclose(fp);
    printf("File size is %d\n", imageSize);
    bcm_host_init();
    s = setupOpenMaxJpegDecoder(&pDecoder);
    assert(s == 0);
    s = decodeImage(pDecoder, sourceImage, imageSize);
    assert(s == 0);

    // the image will be displayed until the program exits 
    sleep(100);
    cleanup(pDecoder);
    free(sourceImage);
    return 0;
}

      

Rendering a JPEG image without tunnelling

Tunnelling gives an end-to-end means of decoding and rendering an image. If we wanted to do things in the middle - say, overlay some text - we would need to have access to the intermediate stages. In this section we repeat the previous section, but without using tunnelling.

We have to explicitly manage the communication between the decode and render components. For the decode component, we have to allocate a buffer. We don't want to copy data between decode and render buffers so the render component uses the decoder's output buffers. Then as the decoder fills its output buffer, the render component empties that same buffer.

There are multiple hiccups along the way

Apart from that, it is the usual games of playing with state, enabling and disabling ports until it works

Conclusion

The RPi has a useful GPU. Programming it using OpenMAX is hard work, though. This chapter has covered some of the issues. The topic is explored more in my book Programming the Raspberry Pi's GPU


Copyright © Jan Newmarch, jan@newmarch.name
Creative Commons License
"Programming and Using Linux Sound - in depth" by Jan Newmarch is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License .
Based on a work at https://jan.newmarch.name/LinuxSound/ .

If you like this book, please contribute using PayPal

Or Flattr me:
Flattr this book