]> Contents Introduction Distributed audio Parameters for A/V Source/sink interfaces Content interfaces Transport interfaces Linkages An HTTP source An HTTP OggVorbis source An HTTP sink Content sinks Sink implementation Servers Clients A file sink A file sink with GUI Conclusion This chapter considers an extended example, using Jini in a home audio situation. It uses many of the concepts of earlier chapters and shows how Jini can be used to build non-trivial systems Traditional information systems concentrated on modelling information flows, and quite explicitly avoided physical systems. The advent of object-oriented systems changed this emphasis, with increased emphasis on the behaviour of real objects and how this could form the basis of an information system. In the meantime, there has been a huge amount of work in control systems, and increasing the computational power of everyday things such as dishwashers, washing machines and so on. One area in which the convergence of computer systems and devices has become a major commercial area is that of audio/visual systems. The change from analogue to digital sytems has opened up a large area, which goes far beyond copying MP3 files from one computer to another The home A/V area is a battleground for ideologies and commercial interests. On the one hand are the set-top vendors, owning the cable systems that pump entertainment into many homes. Their vision is to widen their pipe, while still maintaining control. The professional audio and hifi community on the other hand, see the hifi system as control centre. And of course, the computer community sees the PC as centre of any home A/V system, due to its processing power, well-developed software systems, and the ability to handle digital signals without difficulty. I belong to the PC-centric community to some extent - for even there, there are divergences of opinion. Most current A/V systems such as the Java Media Framework and the Microsoft Media Platform treat the A/V sources and sinks as though they are on the same machine, so that all processing is done locally. Agreed, JMF allows network access using HTTP or RTP, but it tries to hide the network layer and make all components appear to be local. The mantra from Sun for many years is "the network is the computer." This could be applied to the A/V world: "the network is the A/V system." What makes it interesting for the A/V system is what a network can do: a wireless network can support friends visiting with their own A/V systems, joining in with yours to share music; it can support music following you around the house, switching from one set of speakers to another. This chapter is my attempt at building a network wireless audio system using Jini. There have been many efforts to distribute A/V. Much of this is concerned with large servers, and these efforts have paid off with streaming media systems such as RealAudio. I want to look at a more local situation: my house is a medium size house, and now that I have a wireless network I can work in the lounge room, the family room, my study or even in one of the bedrooms. I like "music while you work" - either CDs, or the various community radio stations that I subscribe to, and possibly streaming audio from other stations in the world later. I don't have any children or partner there at the moment, but if I did, then they would have their own music sources and sinks and would share the house network. Friends might come and visit, with their own A/V sources and sinks and just join the house network. In a little while, guitars and microphones will have Bluetooth cards,so we will be able to have a local network band. The wireless network density in my neighbourhood is low, but eventually I should be able to join a local community network, which should give me metropolitan access. I live in a city rich in music (Melbourne, Australia) and sometimes feel that I hardly need to go out because the local radio stations (RRR, PBS-FM) are so good, but soon I would also hope to tune into the folk concert on the other side of town through the community wireless network. Okay: so how do we build middleware for an A/V network that is network-centric, rather than proprieter-centric? There has been one attempt that I know of to build a network-based A/V system, by Marco Lohse ("An open.middleware architecture for network-integrated multimedia"). This is CORBA-based, which gives it network objects. But a lot of their system has to be built on top of CORBA because it doesn't quite support what they want. Much of this extra structure seems to fall out quite easily under Jini. I am approaching the rest of this chapter from a software-engineering viewpoint, trying to make a system as simple as possible for consumers (clients). If you have any comments on this, please let me know - after all, this is the system I using in my house right now, so if it can be made better, then I at least will be grateful! There are many variables that affect how A/V is sourced, moved around a network and delivered

Transport

The transport layer may be reliable (slow) TCP, unreliable (faster) UDP, HTTP (even slower), with some QOS such as RTP or some other network technology protocol such as Bluetooth or FireWire

Format

There are an enormous number of formats, from encumbered formats such as MP3 (for which you are supposed to pay license fees for encoders and decoders), unencumbered equivalents such as Ogg-Vorbis, compressed (MP3 and Ogg-Vorbis) or uncompressed (Sun AU or M/S WAV), lossy or lossless. In addition, there are many wrinkles in each format: little- or big-endian; 8, 16 or 32 bit; mono, stereo, 5-1,...; sample rate such as 44.1khz, 8khz, etc

Content description

Audio comes from many different sources: tracks off a CD, streaming audio from an FM station, speech off a telephone line. The MPEG-7 concentrates on technical aspects of an audio signal in attempts to classify it, while the CD databases (CDDB) such as freedb classify CDs by Artist/Title - which breaks down with compilation CDs and most classical CDs (who is the artist - the composer, the conductor or the orchestra?)

Push/pull

An audio stream may be "pushed", such as an FM radio stream that is always playing. Or it may be "pulled" by a client from a server, such as in fetching an MP3 file from an HTTP server

Interfaces should contain all the information about how to access services. With audio, all the information about a service can be quite complex: for example, a service might offer a CD track encoded in 16-bit stereo, big-endian, 44.1khz sampling in WAV format from an HTTP server. This information may be needed by a consumer that wants to play the file. But in the type of A/V system I want to build there are three players: Sources of A/V data Sinks for A/V data Controller clients to link sources and sinks From the controller viewpoint, most of this information is irrelevant: it will just want to link sources to sinks, and leave it to them to decide how and if they can communicate. For simplicity we define two interfaces: Source and Sink. To avoid making implementation decisions about pull versus push, we have methods to tell a source about a sink, a sink about a source, to tell the source to play and the sink to record. Again, how they decide how to do this is upto the source and sink. Sometimes this won't work: an HTTP source may not be able to deliver to an RTP sink, or a WAV file may not be managed by an MP3 player. If they don't succeed in negotiating tranport and content, then an exception should be thrown. This violates the principle that a service should be usable based on its interface alone, but considerably simplifies matters for controller clients. Notice that neither a source nor a sink have names or other descriptive information. we choose to consider all this information as "additional service information" that can be given by Entry objects. A controller that wants to play a sequence of audio tracks to a sink will need to know when one track is finished in order to start the next. The play() and record() methods could block till finished, or return immediately and post an event on completion. The second method allows more flexibility, and so needs add/remove listener methods for the events. Finally, there are the exceptions that can be thrown by the methods. Attempting to add a source that a sink cannot handle should throw an exception such as IncompatableSourceException. A sink that can handle only a small number of sources (for example, only one) could throw an exception if too many sources are added. A source that is already playing may not be able to satisfy a new request to play. These considerations lead to a pair of high-level interfaces which seem to be suitable for controllers to manage sources and sinks: and The Java Media Framework (JMF) has methods such as getSupportedContentTypes() which returns an array of strings. Other media toolkits have similar mechanisms. This isn't type-safe: it relies on all parties having the same strings and attaching the same meaning to each. In addition to this, if a new type comes along, there isn't a reliable means of specifying this information to others. A type-safe system can at least specify this by class files. I have chosen to use interfaces instead of strings: a WAV interface, an Ogg interface, etc. This doesn't easily allow extension to the multiplicity of content type variations (bit size, sampling rate, etc), but the current content handlers seem to be able to handle most of these variations anyway, so it seems feasible to ignore them at an application level. The content interfaces are just place-holders: package presentation; public interface Ogg extends java.rmi.Remote { } A source that could make an audio stream available in OggVorbis format would signal this by implementing the Ogg interface. A sink that can manage OggVorbis streams would also implement this interface. In a similar way, I have chosen to represent the transport mechanisms by interfaces. A transport sink will get the information from a source using some unspecified network transport mechanism. The audio stream can be made available to any other object by exposing an InputStream. This is a standard Java stream, not the special one used by JMF. Similarly, a transport source would make an output stream available for source-side objects to write data into. and By separating the transport and content layers, we have a model that follows a part of the ISO 7-layer model: transport and presentation layers. The communication paths for a "pull" sink are The classes involved in a "pull" sink could look like where the choice of transport and content implementation is based on the interfaces supported by the source and the preferences of the sink. An HTTP source makes an audio stream available as a document from an HTTP server. All that it needs to tell a sink about is the URL for the document. There is a little hiccup in this: a Java URL can be an http URL, or a file URL, an ftp URL, etc. So I have defined a class HttpURL to enforce that it is a URL accessible by the HTTP protocol. The Java URL class is final, so we can't extend it and have to wrap around it. The HttpSource interface just exposes an HttpURL: The interface allows a sink to determine that the transport protocol it should use is the HTTP protocol, and what URL it should use to fetch the document. An HTTP source is a "pull" source: that is, a sink will fetch the data from it. A source of this type doesn't need to worry about listeners or playing the source data. All it needs to do is store the URL. An implementation of this could be If the document is an OggVorbis document, then the service signals this by implementing the Ogg interface In an identical manner, a document can be a WAV or MP3C document by implementing the WAV or MP3C interface respectively. An HTTP sink needs to find the URL from an HttpSource, open a connection to it and get an InputStream. The Java URL class makes this quite straightforward: A ContentSink will get an InputStream from a TransportSink. Then it can read bytes from this stream and interpret it based on the content type it understands. There are some content handlers in the Java JMF. But many are missing: MP3 files are encumbered by patent rights, and encoders and decoders should cost (someone!) money to use, so currently there is no MP3 player in JMF (well, there is an MPEG movie player which can be called with no video stream...). There is very little activity from Sun on new codecs, and there is no current OggVorbis player. There are attempts to fill the gaps: for example, there is a pure Java Ogg Vorbis decoder, JOrbis, and an MP3 decoder for JMF, JFFMPEG. But the situation has not settled down to any clarity yet. Im the meantime, it is easier to make O/S system calls into players such as mpg123 or sox (under Unix). I have generically labelled these as playmp3, etc, and these read from a pipeline. I am being lazy here: I should have a MP3ContentSink, etc, and createSink() should act like a proper factory and return the right kind of content sink. The playogg script for my Linux system is #!/bin/sh if [ $# -eq 0 ] then infile="-" else infile="$1" fi play -t ogg -c 2 $infile wait # ensure /dev/dsp is free sleep 3 # and give it extra time to be really free :-( while playmp3 is #!/bin/sh if [ $# -eq 0 ] then infile="-" else infile="$1" fi mpg123 -s $infile | sox -t raw -r 44100 -s -w -c 2 - -t ossdsp -w -s /dev/dsp wait # ensure that /dev/dsp is given up sleep 2 # and then give it more time, since "wait" isn't enough (Some of these programs won't work properly if the artsd daemon is running in the KDE enviroment. I typically kill it off, although there must be a "better" way.) A sink must create the appropriate transport and content handlers, and link the two together. It needs to look after listeners, and post events to them when they occur. This sink will handle a TCP and HTTP connections, and will manage WAV, Ogg and MP3 content. Each source will need a server to create, advertise it and keep it alive. So will each sink. A sink server is audio.pull.SinkServer This gets information from a configuration file, such as /http_sink_server.config It exports a proxy for a SinkImpl which will handle TCP and HTTP connections, and will manage WAV, Ogg and MP3 content. This sink server depends on the following classes audio.pull.SinkServer audio.pull.SinkImpl audio.pull.SinkImpl$CopyIO (an inner class) audio.transport.HttpSinkImpl audio.transport.TcpSinkImpl audio.pull.ContentSink audio.transport.TransportSink All the classes in the audio.common package These can be collected into a jar file such as audio.pull.SinkServer.jar and run with a configuration such as the above, as in java -classpath audio.pull.SinkServer.jar audio.pull.SinkServer http_sink_server.config An individual piece of music may be a song, a movement from a classical symphony, an instrumental piece, and so on. Individual pieces of music may be collected together in many ways: a symphony is formed of movements; a pop CD is made up of individual songs; a CD may be made up from a collection of CDs; a boxed set of CDs will be made up of CDs themselves; the complete oeuvres of a composer is another classification. How do we want to represent all of these possibilities as services? Databases of CDs such as CDDB have a simplistic solution: a CD is classified by an Artist/Title. So CD is a collection of pieces, with no other structure. This breaks down with "best of" collections and almost all classical music - who is the artist? The composer? The conductor? The orchestra? The soloist? MPEG-7 is vastly overkill from our point of view, and only a tiny part ("The Collection Structure DS") has anything to say about organising music from our perspective. MPEG7-Lite as used by the UPnP Audio/Visual framework (ContentDirectory1.0) has a simplified structure, but is quite good for representing the different possibilities just described. But UPnP is a device-oriented system, where a device (such as a PVR) is responsible for all the individual items stored on it. Although the device may contain a complex directory structure, the individual components of this are not "first class" objects, directly visible and addressable. This would make it hard to, say, set up a "playlist" across a set of devices such as a PVR, an iPod and a home server storing copies of LPs. The REST community criticises Web services (using SOAP) on the grounds that services have no "addressable endpoint" and that data returned from a service is an XML document that is not addressable at all. UPnP A/V directories are not addressable. In both cases this leads to a loss of flexibility in that clients and services can only work within the bounds of the supplied services and are hence restricted in what they can do - in the case of UPnP it is hard to build up cross-device playlists. So we adopt the extreme viewpoint: every piece of music is advertised as its own service. That allows any other service to build and structure service hierarchies in any way that it wants to. For example, a new service could link photos from an external web site to pieces of music, which would be hard to do with web services or UPnP. We shall present three servers: one that will just advertise a single piece of music available from an HTTP server, one that will advertise a group of pieces (such as a CD of many pieces) and one that will advertise a collection of groups (such as all the CDs on a disk). The file server will advertise one file as a service. Details of the file will need to be stored in a configuration file such as sting.cfg. The service itself will just be an HttpSource of some kind. Descriptive information (such as artist, name of song, etc) is not given as part of the service, but as service information by Entry objects: The server is quite straightforward: it gets information about exporter and service entries from a configuration file and advertises the source as a service using a JoinManager. The file source server requires the following classes audio.httpsource.FileServer audio.httpsource.FileServer audio.http.HttpSourceImpl audio.http.HttpOggSourceImpl audio.http.HttpMP3SourceImpl audio.http.HttpWAVSourceImpl audio.presentation.MP3 audio.presentation.WAV audio.presentation.Ogg audio.transport.HttpURL audio.transport.HttpSource All the classes in the audio.common package These can be collected into a jar file such as audio.httpsource.FileServer.jar and run with a configuration such as the above, as in java -classpath audio.httpsource.FileServer.jar audio.httpsource.FileServer sting.cfg Much music comes on CDs, on LPs, tapes or cassettes, or in some similarly structured format (even a radio show has a structure). This structure often mirrors that of a "directory". So a CD might contains a directory of tracks, and so on. A directory can be a service in its own right, so there is an interface to define it. The directory defines the minimum about each of its services: their ServiceID's. This allows a directory to contain any type of service: individual songs, other directories, even image services or other services. For a directory like this to work, each service must have a persistent service ID, but this is expected of a Jini service anyway. While a directory can hold any type of service, we will only look at a directory that contains sound file services available from an HTTP server. The minimal description for each service is then its URL (so that a service can be constructed) and a service ID so that it can be part of the directory. The set of URLs is given in an array of trackURLs in a configuration, and the service IDs are stored as objects in a file referenced by the same configuration. Additional information such as the name of the directory can be given as an Entry property in the configuration. Similarly extra information about each piece such as its track name can also be given there. Miscellaneous information relevant to a directory derived from a CD is how to look it up on a CDDB (CD data-base) server. A typical configuration might look like clapton.cfg This server is a bit of a departure from previous ones in this book: it creates and advertises a number of services, not just one. Each service will need to respond to its own requests and not to those of others. So each one will need to have a separate dispatcher for its own requests. This is done quite simply by exporting each service by a separate exporter - in fact this is a requirement, as an exporter can only one service. However, the servers we have looked at have typically had a JoinManager, a LookupDiscoveryManager and a LeaseManager. A single LeaseManager can manage any number of classes, so it is enough to have a single copy and then use it to manage all leases. A single LookupDiscoveryManager is required too but there is no need for more than one. The case of JoinManager is different: it can only manage a single esrvice. For a small number of services this does not matter. But if you consider that an iPod can hold 10,000 services, then we might expect in our situation to have to deal with 10,000 services. Individual JoinManager's do not scale well here, and the memory requirements are quite large. So it is better to avoid this class and handle lookup service discovery directly ourselves. The final source server that we shall build generalises the idea of a "set of CDs", by looking at a set of directories. If a set of configurations are each stored in files in a common directory, then by running through the directory and advertising each directory we will have advertised the entire set. This server takes a directory name as command line argument and creates a new DirectoryServer for each configuration file ending in .cfg. All of the directory servers can share the same LeaseManager. A client will locate sources and sinks and allow a user to make selections from them. Each sink will be told about the selected sources, and each source will be told about the selected sinks. The client may register itself as a listener for events (such as STOP) from the services. Then the client will ask the sources to play() and the sinks to record(). A really basic client will just find a sink and a source (any source, any sink), tell each other about the other and then play/record to audio stream. This can be done by The basic client requires the following classes audio.client.BasicClient All the classes in the audio.common package These can be collected into a jar file such as audio.client.BasicClient.jar and run with a configuration such as the above, as in java -classpath audio.client.BasicClient.jar audio.client.BasicClient A more complex client will monitor the sinks and sources (and directories) and display them in suitable panels. These panels will allow selections of sources and sink. The GUIClient looks after service management. The ClientFrame looks after display and selection of sources/directories/sinks. It shows each of these in Swing JTabbedPane. The directory information is displayed using a JTree (this is good for playing a CD in track order). The individual sources are shown in a JList with the services alphabetically sorted. The individual sinks are shown in a JList. The ClientFrame is When source(s) and a sink are selected and a "play" button is pressed, a PlayFrame is displayed. This shows the currently chosen set of sources. This frame listens for events from the sink so that it can tell when a particular track has finished. Then it will start playing the next one. It gets the information about the "next one" (and all of its successors) from the "handback" object, which has a list of the remaining tracks to play A variety of different sources and sinks can be handled by this framework. This is not just different tracks from a CD and different players around the home, but can include different implementations. For example, the equivalent of a cassette recorder can be built by making a sink save its input into a file instead of sending it to the soundcard. This section looks at what is involved in doing this. If a service is going to save audio to a file, then it will need to allow a client to specify this file. At the very least, it will need to expose a setFile method. But it could be more complex than this: the client may need to browse the service file system before choosing a name: for example, to record the "Bitches brew" jazz program off PBS-FM it may want to change to an appropriate directory and check that it isn't going to overwrite an earlier recording. File browsing is usually handled in a GUI environment by JFileChooser. In a non-GUI environment there are methods such as File.list() to list a directory. However, these are local methods in local classes: the JFileChooser has no implicit support for browsing the filesystem of a remote Jini system. But it does have a hook to allow such remote browsing, through the FileSystemView class, which is intended to isolate O/S dependent code into a single class. Based on the FileSystemView class, we can define an interface that captures remote file system browsing and choosing a file While the client sees the interface, the server needs to implement it. This can be done by keeping a FileSystemView object on the server side and passing most calls through to it This sink implementation follows the structure of the earlier ones, although it could be condensed by directly writing to the output file. In this implementation, this is done by ContentSink writing to the file instead of to a "play" program: The classes listed above are sufficient for a non-GUI client to interact with a file sink by making remote method calls on it. This could be suitable for an automated recording system ("record this show each week") where a client starts to record to the service at a given time each week and stops the record once the time is up. If the client is a GUI client, then it may want to allow the user to browse the remote file system using a JFileChooser. This can be done using the UI packages recently approved for Jini which are discussed in their own chapter. Here we just use them. The JFileChooser has a FileSystemViewer object. This encapsulates many operating specific methods, such as File[] getFiles() and File createNewFolder(). The JFileChooser will run in a client, but we want it to be able to access a remote file system where it will save files. The FileSink discussed in the last section can almost be used for this, except it is a remote object and all of its methods throw RemoteException. So we have to wrap it in an object that will catch all these exceptions and return suitable values. For want of any better idea, we return null, although possibly a warning dialog should be popped up first. The exported service will be a proxy for a SinkImpl which implements the FileSink interface. The client will want to run a dialog which pops up a JFileChooser with a FileSystemViewer set to a RemoteFileSystemViewerWrapper which wraps around the FileSink service. When a selection is made, it can set the file in the service. As discussed in the chapter on user interfaces, it is not possible to just send a FileSinkJDialog to a client, because it may not have the classes or memory to handle such an object. The first stage in hiding this is to use a factory object to produce it. This factory follows exactly the pattern for this factory, just changing the returned object to a FileSinkJDialog Finally from the server side, the factory has to marshalled (to hide its classes) and embedded into a UIDescriptor entry object as part of the service advertisement. This is accomplished by the getEntry() method, which is called in preparing the Entry[] array Finally, the client has to be modified to look for UI objects and to use them as part of the user interaction. The client is the same as the GUI client discussed earlier except that in the PlayFrame object it checks for a UIDescriptor This chapter has discussed a framework for distributed audio. Jini makes it fairly straightforward to handle service advertisement and discovery, telling services about each other and generatikng and handling remote events. The architecture is extensible just be adding in more interfaces and implementations. For example, I am currently adding a "file sink" so that I can record audio to file. If this is linked to an FM tuner card then I will be able to record my favourite shows from a timer program. ©right;