Web services Tutorials for Distributed Systems CMT607, CM0356 and CM0456

UPDATE: 10 March 2011. The distsys.zip file has been updated to include code for the Appinions API. Download the new zip to use this, and see the notes below about Appinions.

There are further code examples available from the course text book website. (See the link on the left). Instructions for these tutorials are available in the relevant chapters of the book. They include examples of deploying JXTA, Jini, Web Services, and an Atom Publishing Protocol Server.

-->

Once you have downloaded the zip file, extract it to somewhere on your local machine where you have read and write access to the files (i.e. your home directory or wherever you put your code).

Once you have extracted the file you should see a directory structure like this:

				distsys/
				   |
				   |
				   |
				   |____________
				   |           |
				   |           |
				simple/       web/	
							

There are two main directories because they rely on different libraries. The simple directory is not really more simple, but rather has many fewer dependencies. The web directory contains code that has numerous dependencies.

NOTE: If you have any problems while following these examples, please consult the person running the lab. They should be able to solve most problems. If they cannot, with what the problem is, and the output from the terminal, if any.

The Simple Examples

The simple examples are in the simple directory. These examples only require starting a server and then viewing the pages in a browser. The simple directory looks like this:

				simple/
				   |
				   |
				   |
				   |_____________________
				   |      |      |      |	
				   |      |      |      |
				  bin/  files/ server/  classes/
				                 |
				                 |_______
				                 |       |
				                 |       |
				                src/     lib/
				                 
				
							

The bin directory contains shell and batch files to build and run the examples. The files directory contains the static html files that are served by the server examples. The server directory contains the server Java codes. The classes directory is where the Java class files are compiled to.

To build the examples, cd to the bin directory from the terminal. On Mac or Linux, type:

./build.sh

On windows type:

build.bat

hit the return key, and you should see the classes being compiled. Now run the simple file server: On Mac or Linux, type:

./run-file-server.sh

On windows type:

run-file-server.bat

On initialization of the server you should something like following in the terminal window:

2009-01-30 16:49:36.167::INFO: Logging to STDERR via org.mortbay.log.StdErrLog
2009-01-30 16:49:37.333::INFO: jetty-6.1.5
2009-01-30 16:49:37.477::INFO: Started SocketConnector@0.0.0.0:8080
							

If there are no errors showing in the terminal, then you should have a server running. Open up your browser and type into the address bar:

http://localhost:8080/

The server is running on your local machine ('localhost' or 127.0.0.1) and listening on port number 8080. By default the browser looks for a page called index.html or index.htm if only a directory path is given. So index.html is the page you should now be seeing:

NOTE: The servers run indefinitely. Make sure you shut them down when you are finished with them. Hold down the Ctrl key and hit C to kill them off.

The home page contains a few links to pages that are in the files directory. These include

  • A simple Ajax example that downloads an xml file from the server and inserts the contents into an html table.
  • An Ajax example that uses the appinions enabled server (see below) to get some data from the appinions server and display it in the page
  • An xhtml example of a book.
  • An xhtml example using the hCalendar microformat.
Ajax Example

This populates a table with data from the local server. The file that the JavaScript gets hold of is the data.xml in the files directory. If you look at the code you'll see that the JavaScript has to already know what the structure of the xml document it is getting will be. No WSDL here!

Ajax with Appinions Example

The Appinions example requires some further explanation. In order to access data from the Appinions server you require an API key. API keys are common for these kinds of services. They allow the server to track your requests and make sure you are not abusing the system. The Appinions API key gives you something like 1000 requests a day and two per second. The key is easy to get hold of. To get your key, go to http://developer.appinions.com/member/register and fill in the form. Then log in as a member. You can then register an Application. Go to http://developer.appinions.com/apps/register and fill in the form. You can give your Cardiff user URL for the application, or any other URL you have write access to. Where is asks what the application will do, you can just say it's testing the API. Once you have filled in the form, you will be emailed two API keys. You can also view them online. For the example here you will need to use the opinions search key. Appinions also has a group API. You can experiment with this API as well.

Copy and paste the API key into the Javascript of the appinions.html file in the files directory. Replace the value of the 'myApiKey' ('1234') with your API key. Then you need to shut down the file server if it is running (Ctrl+C) and then cd to the bin directory and type on Mac or Linux:

./run-appinions-server.sh

or on Windows:

run-appinions-server.bat

This will start a local server that can talk to appinions.

If you now go to http://localhost:8080/appinions.html, data will load from Appinions. The page makes two requests - one non-Ajax request which returns the raw JSON in an iframe, and the Ajax request that the Javascript does some simple parsing on. The local JavaScript creates a URL using the local server's address concatenated with the search API path and a query string that includes your API key, a request for json data and search value of 'libya'. When the local server sees a request path corresponding to an appinions request it prepends the request path with the appinions server's address and then makes a request to appinions. The returned data is sent back to the browser. Because the data arrives via the local server, the browser allows the JavaScript to access the data and display it.

NOTE: If you get a java.net.BindException: Address already in use in the terminal, it means there is already a server running on Port 8080. Check that you have shut down all the servers.

the Appinoins API has a number of query parameters available that you can experiment with. For a full description of the API, go to http://developer.appinions.com/docs

XHTML Book Example

This is a simple example of a book type, serialized as XHTML. One of the benefits of XHTML is that it can be used to display data in a non-lossy way. The machine readable data values are inside tags with specific class names, allowing a machine to parse the data structure. Data that helps human readable rendition are outside these tags, but can still be styled and viewed.

XHTML hCalendar Example

hCalendar is a microformat based on the iCalendar specification. An hCalendar can contain a number of VEvents - virtual Events. VEvents can also be described stand-alone, in which case the hCalendar is implicit. Note the use of the abbr element in the XHTML. This is used to display human readable forms of certain data types, e.g. dates, while still retaining the machine readable version. The title attribute is used to hold the machine readable version, while the actual text content of the abbr tag contains the human readable version.

The Google API

The files directory also contains an html page containing code to access the Google Map API (google.html). You can get an API key from here. This page explains how to get started and what the API is made up of. The Google Maps API is the most used in mashups and therefore has a lot of examples, support and a blog. The API key is good for a certain domain name, which means in theory you need to have your page hosted somewhere. However, you can use the key on your localhost if you type in http://localhost in the text box asking for your web site URL.

You are encouraged to play around with the Java source code and the HTML code in the open-source spirit of "learn by hacking". If you don't understand the code, change small bits and look at the results. Keep a track of what you have changed, so you can undo your changes if things go pear-shaped.

The Web Examples

the web directory looks like this:

				  web/
				   |
				   |
				   |
				   |______________
				   |      |      |
				   |      |      |
				samples/  lib/   etc/
				   |
				   |
				   |______________
				   |      |      |
				   |      |      |
				jaxws/  rest/  common/      
							

The jaxws and rest directories contain a src, classes and bin directory. The common directoy contains just src and classes directories. As with the simple directory, the src directory contains Java source files, the classes directory are where the Java source files are compiled to and the bin directory contains shell and batch scripts.

The JAX-WS Example

lets look at the jaxws examples first. JAX-WS is a Java specification for developing Web Services. http://java.sun.com/webservices/technologies/ gives an overview of the Web Services related Java specifications, and https://jax-ws.dev.java.net/ gives an overview of JAX-WS in particular. JAX-WS makes use of the JAXB specification. JAXB defines rules for mapping XML types to Java objects and vise-versa. JAX-WS builds on top of this to define rules for mapping Java objects to WSDL and back again. Hence, together, they provide a comprehensive set of specifications for exposing and consuming Web Services using Java.

The src/distsys/ws/server directory contains the server side Java code. There are three classes:

  • ContactService - the service interface
  • ContactServiceImpl - the service implementation
  • Server - the server that exposes the service implementation

The service simply echoes back the object that the client sends to it. This object is a Java Contact object that is defined in the common directory - common/src/distsys/common/Contact.

The server-side uses JAX-WS defined Java Annotations, to allow the JAX-WS libraries to to work out how to define the Java object in WSDL. In particular it uses the @WebService annotation. The service interface class uses it to flag to the JAX-WS runtime that this should be exposed as a Web Service. The service implementation uses it to define the interface the implementation class adheres to, and the service name. JAX-WS, combined with JAXB does the rest! The Contact class that is accepted as input and returned as output to the service follows the Java Bean conventions for method and variable naming. This means we do not have to do anything special to get JAXB to serialize it to XML. A Bean compliant class is a Java class that adheres to certain rules:

  • The methods have a naming convention in which instance variable names can be extracted from the method and vise versa. The method for retrieving a value is the name of the value with the first letter of the variable name capitalized, prepended by "get" to get the value, or "set" to set the value ("is" can be used for getting the value of a boolean). As an example, if a class has an instance variable called name, then the method for setting the value of the variable will be called setName, and the method for getting the value will be called getName.
  • It has a default (no args) constructor -

These properties allow a runtime to use the Java reflection API to build a Java object from XML, and serialize one to XML without any other special knowledge. Because the Contact class adheres to this pattern, we do not need to add any other processing instructions via annotations or anything else. To build and run the example, do the following:

cd to the bin directory in the jaxws directory. On Mac and Linux type:

./build.sh

On windows type:

build.bat

Hit the return key. If you see no errors, then the Java files have compiled.

Now run the server. On Mac or Linux, type:

./run-server.sh

On windows type:

run-server.bat

The server should now be up and running. Now open another terminal window and cd to the same bin directory. On Mac or Linux type:

./run-client.sh

On windows type:

run-client.bat

You should see the following output in the terminal window:

DynamicClient.main echoed contact = distsys.ws.server.Contact@8bf360
First Name:Miles
Last Name:Davis
Email:m.davis@bluenote.com	
							

Have a look at the code in the src/distsys/ws/client directory. This contains the client Java class DynamicClient. The client takes the location of the WSDL file exposed by the server. You can look at this file while the server is running in your browser. Go to http://localhost:8080/ContactService?wsdl. (Click on the image to see it full size.)

This WSDL is parsed by the client to generate Java classes representing the types in the WSDL. The client code does not use any Java standards to do this. Instead it uses the built-in capabilities of CXF, the Web Services framework which is incubating at the Apache Software Foundation. The web examples here are based on the CXF framework. CXF generates Java classes on the fly which are based on the WSDL file. This is apparent if you look at the class name of the class that DynamicClient is asking the Class Loader to get hold of:

Class cls = Thread.currentThread().getContextClassLoader().loadClass("distsys.ws.server.Contact");	
							

The distsys.ws.server.Contact does not exist at compile time. The class the server is using is the distsys.common.Contact class. The package name of the generated class is based on the WSDL file, specifically the target namespace of the schema in which the contact type is defined. Have look at the WSDL in the browser again. The first few lines should look like this:

<wsdl:definitions name="ContactService" targetNamespace="http://server.ws.distsys/">
  <wsdl:types>
    <xs:schema attributeFormDefault="unqualified" elementFormDefault="unqualified"
        targetNamespace="http://server.ws.distsys/">
							

The target namespace of the schema in which the Contact type is defined forms the basis of the package name of the generated class (the http:// scheme is removed and the path is reversed).

The client uses reflection to create an object of the class type, and then invoke methods on it. These methods follow the Java bean naming conventions. Have a look in the WSDL at the complex type defining the Contact type. The method names use the field values (e.g. "firstName") of the schema type, capitalize the first letter (e.g. "FirstName"), and then prepend this with "get" and "set". The client uses the setters before sending the object, and then uses the getters when the object comes back from the server. The output in the terminal is the result of those getters.

The REST Example

The REST example is located in the rest directory. The structure of this directory is the same as the web directory - it contains bin, src and classes directories. This example comes pretty much straight from the CXF examples. You can see their notes on how this works at http://cwiki.apache.org/CXF20DOC/http-binding.html. Here's a brief overview.

The example uses JAX-WS annotations to tell the runtime what the service is. It also uses annotations defined by the Java REST Annotations project which is (also) incubating, this time at Codehaus (you can't say you're not using cutting edge technology!). The REST annotations map a Java method to an HTTP method (e.g. GET, PUT, DELETE) and to a URI template. A URI template is a means of inserting variables into a URI which are replaced at runtime with an actual value. The variables are surrounded with curly braces. For example:

http://example.org/users/{userid}

is a template in which the value {userid} is replaced with user's actual id at runtime. Templates allow you to build up patterns for URIs. The templates in the example allow Java methods to be mapped to a request to GET a particular Customer using the customer id in the URI, e.g. a GET on the URI

http://localhost:8080/xml/customers/123

will return a customer representation to the client.

the client directory in rest/src/distsys/ contains a class that invokes the server at different URIs using different HTTP methods to list customers, get a particular customer, create a custmer, update a customer and delete a customer. The GET methods can also be run from your browser.

To build and run the example do the following:

cd to the bin directory in the rest directory. On Mac and Linux type:

./build.sh

On windows type:

build.bat

Hit the return key. If you see no errors, then the Java files have compiled.

Now run the server. On Mac or Linux, type:

./run-server.sh

On windows type:

run-server.bat

The server should now be up and running. Now open another terminal window and cd to the same bin directory. On Mac or Linux type:

./run-client.sh

On windows type:

run-client.bat

You should see the results of the various requests in the terminal. In the server side terminal, you should see some output for each method that was called on the class. Notice in the ClientMain class, that the XML is generated 'by hand'. In the absence of WSDL, there is no obvious way to generate Java classes that will represent the serialized xml objects.

Tasks

There are some classes in the common directory that I have not discussed so far:

  • ICalendar - this class is a (very) simple implementation of the ICalendar specification.
  • VEvent - this is used by the ICalendar class to encapsulate events.
  • XhtmlUtils - this class contains a number of utility methods for parsing XHTML and generating documents and outputting xhtml.
  • Pet - a class representing a pet
Task 1

Using the Pet class, create a Web Service that can store pets. When a client asks the server to store a Pet, it sends a Pet representation. The server responds with an id for the pet. The client can then assign this id to its local version of its pet, and use the id for later requests. A client can get a pet from the server using the id as a parameter. The server returns a pet representation. A client can also delete a pet, if the unthinkable should happen, using the id as a parameter. A client can also request the server to perform a compatibility check on two pets to check if they could produce valid offspring. The server accepts two pet ids to perform this operation and returns a boolean indicating whether they are compatible or not. The algorithm for checking compatibility can be as complex as you like. A minimum requirement is that both pets are stored on the server, they are of opposite sex, and of the same species. The Web Service uses SOAP for messaging and WSDL to describe itself.

NOTE:the java.util.Date value of a Pet's birthday will get deserialized as a javax.xml.datatype.XMLGregorianCalendar object by JAXB.

Task 2

Create an XHTML representation of a Pet object. Include a picture of the pet. Don't forget to validate the XHTML at: http://validator.w3.org/

Task 3

Write a Java program that can scrape an iCalendar from an XHTML page at a given URL (use the vcalendar.html page in the files directory), and generate a calendar event file which can be imported into your calendar software. Mac's iCal, Linux Evolution and Microsoft Outlook support iCalendar objects. The program should write out a calendar file with a .ics extension. You may presume that the date formatting of the page you are scraping is the same as that defined in the VEvent class, i.e. "yyyyMMdd'T'HHmm". Handling multiple date formats is quite complex in Java. Also, certain characters need to be escaped in text content of a VEvent: "\", ";", "," and "\n". You can presume that the calendar events you are parsing do not contain these characters.

The Jini Examples

the Jini examples are available as a self-contained zip file from here

Exercise:

After running the examples, build a remote file storage service using Jini. Details are available from the zip file. Click on index page once you have unzipped it.

The JXTA Examples

the JXTA examples are available as a self-contained zip file from here

For more information on programming with JXTA, take a look at the programmer's guide available here.

Exercise:

After digesting the programmer's guide, try building a command line chat application using JXTA. Peers should advertise an input pipe, for receiving messages. They discover pipes using the user name of their buddies. You start the application by typing in your user name followed by the name of the buddy you want to connect to, e.g.,

	java -classpath .:classes JxtaChat alice dave
							

would launch a JxtaChat application with a user name of 'alice', and try to find the user 'dave' on the network. If discovered, it would prompt for your input, and print out any responses from dave.

Hints:

JXTA uses the java.util.logging package for logging. To turn this off, and clean up terminal output, you need to reference a logging config file with logging turned off, from the command line using the system property java.util.logging.config.file, i.e.,

java -Djava.util.logging.config.file=logging.properties JxtaChat alice dave

An example logging file is available here

When running the code on the same machine, run the users from different directories. Before running them, run the startJxta application from their directories to configure the peers. In particular, make sure the TCP port used is different for each of the peers (Advanced tab, TCP Enabled, next to the manual checkbox). You may also need to set the Network Interface (IP address) from the drop down menu as well. Otherwise you can get clashes with both peers trying to use the same port. To reconfigure a peer, go the directory from which you ran startJxta and delete the .jxta directory.