x
Testing Java applets introduces a new level of variation in test environments, in that the applet must run within a variety of browsers. This paper looks at some of the issues arising from this, and reports on a test framework under development to answer these issues.
Java [Java] is a relatively new language that can be used to build standalone applications. In addition, it can be used to build applets. Java applets are distributed programs that execute on a variety of platforms, usually embedded within a Web browser. This environment gives rise to a number of unique problems. This paper details some of these issues, and describes work in progress to build a tool to aid in testing applets.
./Configure make make test make install
The Java language and core libraries are designed for a wide set of environments. Strictly, they are designed for one environment: the Java Virtual Machine [JVM], but this has been implemented on many platforms, in many different ways. The slogan is ``write once, run anywhere''. While true, there is often an addendum: ``write once, test everywhere''. There are sufficient differences between the Virtual Machine implementations and in the libraries to make this essential.
The so-called Java Pure initiative [PureJava] attempts to eliminate many programming practises that can result in variant behaviour (probably incorrect) on different platforms. However, it can only guarantee ``purity'', and this is not as strong as portability, let alone correctness.
Java applets run within browsers. In addition to the wide set of platforms and different Java Virtual Machines, the browser itself will add further variety to the environment. Some of the differences are
init()
code than once.
Applet
widget, and this will be contained
by a Frame
within the browser. However, this
Frame
is not part of the Java specifications,
and is implemented in different ways for
different browsers.
This widens the set of situations for which tests must be performed. In addition to ``test everywhere'' is ``test on every browser, in various user set configurations''.
Applications have fewer entry points to test against. A
common test method is to simulate user interaction in some
way. For example, for applications with a graphical user
interface, events can be simulated and placed on the event
queue. Sophisticated techniques may be available for this:
for example, the replayXt
system of the author
[replayXt]
uses action
's available in any Xt widget to
drive simulated user events. Non-windows applications may
be driven by systems such as expect
,
which will supply a pseudo-terminal and feed user actions
into this, for consumption by the application [expect].
To test an applet, a suitable harness must be present. This must be able to load an applet and run a test script against it. The tool JavaStar runs a Java program around the applet, which is generated for that applet [JavaStar]. This Java program can be run across all platforms supporting the Java VM, but does not run within a browser. Therefore, it fails to test under different browser conditions.
The test harness needs to run within each browser.Therefore it needs to be an applet itself. This applet needs to load the applet to be tested and run it within the same environment.
expect
and
replayXt
scripts are written in tcl
[tcl],
whereas JavaStar scripts are written in Java. For standalone
applications the choice of language is under the control
of the tester since they tend to be more in charge of the
environment. For applets, one can envision a more open
testing situation, where any arbitrary user can run the test
suite.
This places limitations on the scripting language.
For example, the tcl
test suite is written in
tcl
since it is known that it will be available.
JavaScript uses Java since the Java VM must be available
to test Java applications or applets. However, this is not
the only possibility: any scripting language written in
Java can be downloaded to run within any applet. Jacl [Jacl]
is a tcl
interpreter written in Java, and a
tcl
interpreter can be run from within any
applet. In particular, it can be run from an applet
implementing a test harness for other applets.
tcl
applications include a test suite.
Applets are loaded across the network into browsers,
where they run. Test scripts could potentially exist on
the browser side, but this would involve replication on
every test site.
Explicit replication of test code through mechanisms such
as tar
files would be fairly inflexible.
Test code should exist on the same Web site as the applet.
This can then either be loaded up to a browser either as
a parameter to the test harness (which is an applet),
or the test harness can upload a script file itself.
Locating tests on the server with the applet they are testing is a logical choice.
The exclusions are in fact more severe than this. If an applet
harness needs to load an applet, then the tested applet
can only come from the same site as the harness. The harness
will need to load the code using a ClassLoader
,
and access to these is restricted. This is to prevent an
applet using a class loader that breaks the sandbox model.
In addition, a harness that generates user events and adds
then to the tested applet's event queue will need access to
that queue, and this is currently restricted. This is because
there is presently only a single event queue for any
application/applet, and if a browser such as HotJava allowed
access to its event queue then buttons such as the
``File Save As..'' could be pressed - leading to access to
the local file system.
Working within the security restrictions is an unavoidable issue. While many of these restrictions are fairly easy to manage, the issue of a single versus multiple event queues is not. In JDK 1.2 multiple event queues are promised, but have not appeared in the beta versions available so far. The capabilities security model introduced in Netscape 4 and refined in HotJava may provide an alternative mechanism, but at a security risk.
replayXt
to call semantic-level actions for
Xt widgets.
The original set of GUI elements for Java was the set of
AWT objects. These are built above native code GUI objects.
For example, the Java Button
is built on top
of the Motif XmPushButton
on Unix platforms.
These objects are virtually impossible to drive in any
sensible way.
Events are divided into two classes: input events (such as
mouse click and keypress) and semantic events. Input events
may be generated by an application,
but are discarded before they reach
the native objects [Newmarch].
Semantic events may be generated, and they may affect Java
GUI objects such as Lists, but they never reach the
underlying object either.
There is no
standardised mechanism for interacting with Java objects
at the semantic level.
The AWT is being replaced with the Swing objects [Swing], written entirely in Java. Events generated and placed into the event queue do what one would hope - they affect internal state and external appearance of the GUI objects. This is all good stuff. It is not too hard to do this - provided you have access to the event queue.
Generating events for Swing objects has the
same problems that it does in all window systems: (x,y)
co-ordinates often have to be specified, and this locks
you down to a particular geometry. In some situations this
is not a problem: for example, a Button can be queried for
its width and height and then co-ordinates can be calculated
that place the event in the centre of the Button.
However, some widgets, such as FileChooser
have internal structure which is not made visible.
FileChooser
often has an
OK
button. The location of this button is
dependant on a geometry manager that will place it in
varying positions depending on the space it has available.
There is currently no means of querying the widget for
the location of any of its components, so this is unreliable.
There is another problem with event generation in Java. The Java event is not translated into a native event, and to some extent these still underly part of the GUI libraries. So it is possible to generate a Java event that corresponds to mouse moved, but the actual mouse will not be moved. This will be an issue for demonstration systems that rely on an observer tracking the motion of the mouse throughout the demonstration.
Some Swing objects have semantic methods such as
doClick()
on the Swing JButton
.
This redraws the object (several times) and calls all
the action listeners. However, not all objects have a
complete set of methods. For example, FileChooser
has no method to click on the OK button. This particular object
is not yet finalised in Swing 1.0, and we may hope for a
complete set of semantic methods for all objects.
replayJava
framework already addresses
some of these issues [replayJava].
It is designed as a general framework
to run Java applications and applets through a scripting
language. The scripting language is tcl
,
and this is accessible to browsers because there is an
interpreter written in Java. The framework can drive
applications through the event queue or through semantic
methods.
Driving applets by the event queue
is dependant on a proper implementation
of getEventQueue()
for applets, that doesn't
just attempt to return the the system event queue (which will
often fail for security reasons).
Using events on structured objects such as FileChooser
requires knowledge of the location of internal components and methods
to do this are not yet available.
Driving applets by semantic methods requires a complete set
of methods for all GUI objects.
A tool is under development that can be used to test applications and applets. Deficiences in the Java libraries that are exposed by this are being reported to JavaSoft, hopefully to be fixed in later releases.