Friday, November 25, 2011

x!, a very simple xml library

A couple of weeks ago, having gotten annoyed about the amount of boilerplate you need in Java in order to do some very simple XML actions I decided to put something in place that would get me close to the fluency of the Groovy and JScript XML extensions with a very small interface. Indeed the original proposal can be contained in a tweet and the only methods I have added since are parent() and prefix() which doesn't quite fit in the standard 140 characters.

So there is basically only one class to worry about X, this allows you to read and write values, select children and perform the normal input and output operations. Everything throws unchecked exceptions, and where possible Attr and Elements are treated the same so for example set(val) and get() will work for both in a consistent manner.

You can find simpler examples in the unit tests in the github project, but the most interesting use cases for me are when used with JAX-RS to access and program REST resources that accept XML. XPath makes Duck Typing easy and if you are careful your code can ignore minor schema changes, this is in stark contrast to static JAX-B binding for example.

Take this example, it looks at a remote resource and sets it as being offline using the Jersey client. In the simplest case you need to explicitly tell Jersey about the X message body readers and writers; but other than that the code is quite straight forward.

   ClientConfig clientConfig = new DefaultClientConfig(
      XMessageBodyWriter.class,XMessageBodyReader.class);    
   Client client = Client.create(clientConfig);
   client.addFilter(new HTTPBasicAuthFilter(...));
        
   WebResource resource = client.resource("http://hudson...");

   resource.put(
      resource.get(X.class)
         .set("//n:offline", "true"));
        
   System.out.println(resource.get(X.class).get("//n:offline"));

Note in the last line we use the pre-defined namespace prefix n which is always populated by the first node in the tree. You can also perform the select, or selectList for a list, explicitly to get hold of the value:

   resource.put(
      resource.get(X.class)
         .select("//n:offline").set("true"));
        
   System.out.println(resource.get(X.class)
     .select("//n:offline").get());

You can also use x! in JAX-RS resources, here is a very simple hello world example using the same message body readers and writers as before. Since they are marked as @Provider your container should be able to pick them up for you.

@Path("/hello")
public class HelloMessage {
    @POST
    @Produces("text/xml")
    @Consumes("text/xml")
    public X hello(X input) {

        String name = input.select("//name").get();

        X response = X.in("http://www.example.com", "message");
        response
            .children().create("name")
                .set(name).parent()  // Think of .parent() like a CR
            .children().create("message")
                .set("Hello " + name)
                .set("@lang", "en");

        return response;
    }
}

I am not entirely sure about the flow to create new objects, feedback is always appreciated of course. The use of @ to set attributes is quick as internally this doesn't result in a XPath query. Currently you can only work on direct children when creating new attributes, for more complex paths you need to select the direct parent to create new attributes. You can still set values on attributes that do exist with complex xpath expressions though.

    // @attr exists
    x.select("....@attr").set("value");
    or 
    s.set("...@attr", "value");

Here is a simple request and response from this service just for comparison you can get the code to reflect the xml quite closely.

// Example input message
<message>
   <name>Bob></name>
</message>

// Example response

<message xmlns="http://www.example.com">
   <name>Bob></name>
   <message lang="en">Hello Bob></message>
</message>

This really was a thought experiment that got out of control; but I would welcome feedback, suggestions, and of course as it is on GitHub patches.

One interesting direction is replacing the current DOM implementation with a streaming version which might be possible for certain read and write options depending on the ordering of the data.

Wednesday, October 19, 2011

A little script to update job descriptions on Matrix jobs

We have started to use Matrix jobs in Hudson to try and reduce the amount of clutter on the front page; but we have run into problems in that the job description plugin we used to add the LABEL name to the job id only works for the subordinate tasks. This leaves the top level job without a description.

There are of course other ways you can do this; but since I was playing with the new REST api yesterday, I am going to use this. You could of course use the old RESTy API or indeed the command line tool to perform similar feats. This week I mostly just have a hammer.

Before you use the proxy classes you need to first create a client with the right security credentials configured:

   public static Client createClient() {
       Client client = Client.create();
       client.addFilter(new HTTPBasicAuthFilter("xxxxx@oracle.com","xxxxxxx"));
       return client;
   }

The first part of the job is going to be to create proxies for both the parent project and of the child variant:

   public static void main(String[] args) {

       if (args.length!=2) {
           System.err.println("Expecting two parameters project and variant");
       }

       String project = args[0];
       String variant = project + "%2F" + args[1]; 

       System.out.printf("Will copy missing description from %s to %s\n", variant, project);

       // Create a Jersey client
       Client client = createClient();

       // Create a suitable proxy using the client we configured
       Projects hudsonProjects = projects(client);

       // Handles to the right resources
       Projects.ProjectNameBuilds projectBuilds = hudsonProjects.projectNameBuilds(project);
       Projects.ProjectNameBuilds variantBuilds = hudsonProjects.projectNameBuilds(variant);

Then we can make a quick map of the variant build number to descriptions by getting the first build object:

       // Copy the descriptions out from the variant
       Map<Integer, String> variantToDescription = new HashMap<Integer,String>();
       BuildsDTO variantBuildsDTO = variantBuilds.getAsApplicationXml(BuildsDTO.class);
       for (BuildDTO build : variantBuildsDTO.getBuild()) {
           if (build.getDescription()!=null)
           {
               variantToDescription.put(
                   build.getNumber(), 
                   build.getDescription());
           }
       }

Now it turns out that the current version of the API doesn't allow you to update build descriptions; but we can easily read them using this API and then update them using a simple form POST using the same client.


       // Update the main project descriptions
       BuildsDTO projectBuildsDTO = projectBuilds.getAsApplicationXml(BuildsDTO.class);
       for (BuildDTO build : projectBuildsDTO.getBuild()) {

           String description = build.getDescription();
           // Update description if it has not already been set
           if (description == null || description.length()==0) {

               String variantDesc = variantToDescription.get(build.getNumber());
               if (variantDesc!=null && variantDesc.length()!=0) {

                   // We need to set the description; but nothing in the REST API
                   // so we can use the url property to perform a form submit that will
                   // update the description for us.

                   MultivaluedMap map = new Form();
                   map.add("description", variantDesc);
                   map.add("Submit", "Submit");
                   WebResource buildResource = client.resource(
                       build.getUrl() + "submitDescription");
                   ClientResponse response = buildResource.type(MediaType.APPLICATION_FORM_URLENCODED_TYPE)
                       .post(ClientResponse.class, map);
                   System.out.printf("Updating build %d with description %s gave response %d\n",
                                     build.getNumber(), variantDesc, response.getStatus());

               }
           }
       }

Monday, October 17, 2011

Getting hold of the schemas for the Hudson REST API, and generating a client

I have been playing with the newish REST-like interface in Hudson 2.1.2 and I wanted to generate a client; but as it only uses Jersey 1.5 so it doesn't contain my fix to publish and connect them up directly. Luckily you can just down load these files from the github repository. You ideally want to add these to the wadl description of the service so download the root WADL using a tool such as wget from http://hudson.my.com/rest/application.wadl and then you can add the required imports.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<application xmlns="http://research.sun.com/wadl/2006/10">
    <doc xmlns:jersey="http://jersey.dev.java.net/" jersey:generatedBy="Jersey: 1.5 01/14/2011 12:36 PM"/>
    
    <grammars>
      <include href="build.xsd" />
      <include href="common.xsd" />
      <include href="fault.xsd" />
      <include href="project.xsd" />
    </grammars>
    
    <resources base="http://hudson/rest/">

    ....

Now you can either use the schemas as is and make sure you have annox on your class path when you generate the model classes, or you can find and replace jaxb:extensionBindingPrefixes="xjc annox" with jaxb:extensionBindingPrefixes="xjc".

This will give you a wadl that will generate the interface and the model classes in the same step; we will have to wait for later version of Hudson to connect the types up to the methods. For the moment though if you have a quick look at the WADL it is relatively obvious which types are for which.


/**
 *
 */
@Generated(value =
           { "wadl|file:/tmp/workspaceWorkDir5662539174912846886.abbot/Hudson/Hudson-WADL/src/wadl/application.wadl",
             "run|ca8e07f8-00d8-40ab-bde3-b15cee244091" }, comments = "wadl2java, http://wadl.java.net",
           date = "2011-10-17T14:31:17.818+01:00")
public class Hudson_Rest {


    public static void main(String[] args) {

        // Write out all projects status
        Hudson_Rest.Projects hudsonProjects = projects();
        ProjectsDTO projects = hudsonProjects.getAsApplicationXml(ProjectsDTO.class);
        for (ProjectDTO project : projects.getProject()) {
            System.out.printf("++++ %s +++++\n", project.getName());
            System.out.println(project.getHealth().getDescription());
        }

        // Query the enablement of one particular project
        Hudson_Rest.Projects.ProjectName hudsonAbbotSf = hudsonProjects.projectName("abbot_SF");
        ProjectDTO abbot_SF = hudsonAbbotSf.getAsApplicationJson(ProjectDTO.class);
        System.out.println("Abbot_SF is enabled " + abbot_SF.isEnabled());
   }

   ...
}

So this is a simple read operation, I will look at writing back through this API which will involve configuring the client for authentication in a later installment.

Tuesday, October 11, 2011

A few simple command line operations on zip files every java developer should know

Java developer spend a lot of time working with zip files it always surprises me that many reach for winzip when some really quite simple command line operation will do. These should all work fine on Unix or using Cygwin, I have consciously used unzip rather than jar as it is more readily available.

First is a quite simple question, does this file file contain a particular file?

unzip -l Example.jar | grep LookingFor.java
or the slightly prettier version with jar:
jar -tf Example.jar | grep LookingFor.java

Second is that you want to look at the content of a file in the zip.

unzip -pq Example.jar META-INF/MANIFEST.MF 
Thirdly you sometimes need to search a bunch of jar files looking for a particular class file: (Thanks a lot to Mark Warner for optimising this for me)
find . -name "*.jar" -exec sh -c 'jar -tf {} | grep "ProtocolHandlerT3"' \; -exec echo ^^^^^^^ in {} \;

Finally the last operation is that you want to be able to compare two zip files, if you have a some UI tools for this then great but this little script will give you a quick command line diff:

#!/bin/bash

FIRST_DIR=`mktemp -d`
SECOND_DIR=`mktemp -d`

echo Unzipping $1 to ${FIRST_DIR}
unzip -q $1 -d ${FIRST_DIR}
echo Unzipping $2 to ${SECOND_DIR}
unzip -q $2 -d ${SECOND_DIR}

diff -r ${FIRST_DIR} ${SECOND_DIR} 

rm -rf ${FIRST_DIR}
rm -rf ${SECOND_DIR}

Monday, September 12, 2011

Replacment for Endpoint.publish when using Jersey

Sometimes when working with JAX-WS services it is nice to be able to test the service without having to deploy to a heavyweight server. You can do this using the Endpoint.publish method and just pass in an instance of your server class.

There is something similar in Jersey, but it isn't part of the JAX-RS specification. So you can publish a single resource as follows:

package client;


import com.sun.jersey.api.container.httpserver.HttpServerFactory;
import com.sun.jersey.api.core.ClassNamesResourceConfig;
import com.sun.net.httpserver.HttpServer;

import java.io.IOException;


public class Test {
    
    public static void main(String[] args) throws IOException {

        HttpServer create = HttpServerFactory.create(
            "http://localhost:9001/",new ClassNamesResourceConfig(Hello.class));
        create.start();
    }
}

Update: 12th December 2013: The code for this has slightly changed in Jersey 2.0, the following will publish a resource using the HttpServer that comes with the JDK (you will need to include a dep on the jersey-container-jdk-http component):


import com.sun.net.httpserver.HttpServer;

import java.net.URI;

import org.glassfish.jersey.jdkhttp.JdkHttpServerFactory;
import org.glassfish.jersey.server.ResourceConfig;


public class Test {
    public static void main(String[] args) {
        
        HttpServer d = JdkHttpServerFactory.createHttpServer(
            URI.create("http://localhost:9001"), 
            new ResourceConfig().register(new Hello()));  
   }

Wednesday, September 7, 2011

Improvements in the WADL client generator

I have recently been working on bringing the useful WADL client generator as originally penned by Marc Hadley more in line with currently technology. I have updated it so that it now uses the Jersey client and also I have made a few modifications to make it easier to use. There is plenty of documentation on the WADL site on how to run the tool in various configurations, but I thought it worth while just to show a simple example of what you might expect.

You can download a snapshot of version 1.1 from here, and hopefully soon a final version of 1.1 will be avaliable here if I have guessed the URL correctly. If you pick the version after the 6th of September it should contain all the bits your need.

So as an example we have a really simple hello world service that has a path param to select the correct languauge, the WADL and the Schema generated in Jersey 1.9 are as follows:


<!-- application.wadl -->

<?xml version = '1.0' encoding = 'UTF-8' standalone = 'yes'?>
<application xmlns="http://wadl.dev.java.net/2009/02">
    <doc xmlns:jersey="http://jersey.java.net/" jersey:generatedBy="Jersey: 1.9 09/02/2011 11:17 AM"/>
    <grammars>
        <include href="application.wadl/xsd0.xsd">
            <doc title="Generated" xml:lang="en"/>
        </include>
    </grammars>
    <resources base="http://localhost:7101/cr/jersey/">
        <resource path="webservice">
            <resource path="{lang}">
                <param xmlns:xs="http://www.w3.org/2001/XMLSchema" name="lang" style="template" type="xs:string"/>
                <resource path="greeting">
                    <method id="hello" name="GET">
                        <response>
                            <representation xmlns:ns2="http://example.com/hello" element="ns2:bean" mediaType="*/*"/>
                        </response>
                    </method>
                </resource>
            </resource>
        </resource>
    </resources>
</application>

<!-- application.wadl/xsd0.xsd -->

<?xml version = '1.0' standalone = 'yes'?>
<xs:schema version="1.0" targetNamespace="http://example.com/hello" xmlns:tns="http://example.com/hello" xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:element name="bean" type="tns:bean"/>

  <xs:complexType name="bean">
    <xs:sequence>
      <xs:element name="message" type="xs:string" minOccurs="0"/>
    </xs:sequence>
  </xs:complexType>
</xs:schema>

Running this through the wadl tool you end up with a java class that look like this along with the relevant JAX-B classes for the schema type Bean.

package client;

import java.util.HashMap;

import javax.annotation.Generated;

import javax.ws.rs.core.UriBuilder;

import com.example.hello.Bean;

import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.GenericType;
import com.sun.jersey.api.client.WebResource;


/**
 *
 */
@Generated(value = { "wadl|http://localhost:7101/cr/jersey/application.wadl" },
           comments = "wadl2java, http://wadl.java.net", date = "2011-09-07T11:53:39.206+01:00")
public class Localhost_CrJersey {


    public static Localhost_CrJersey.Webservice webservice(Client client) {
        return new Localhost_CrJersey.Webservice(client);
    }

    public static Localhost_CrJersey.Webservice webservice() {
        return webservice(Client.create());
    }

    public static class Webservice {

        private Client _client;
        private UriBuilder _uriBuilder;
        private HashMap<String, Object> _templateAndMatrixParameterValues;

        /**
         * Create new instance using existing Client instance
         *
         */
        public Webservice(Client client) {
            _client = client;
            _uriBuilder = UriBuilder.fromPath("http://localhost:7101/cr/jersey/");
            _uriBuilder = _uriBuilder.path("webservice");
            _templateAndMatrixParameterValues = new HashMap<String, Object>();
        }

        /**
         * Create new instance
         *
         */
        public Webservice() {
            this(Client.create());
        }

        public Localhost_CrJersey.Webservice.Lang lang(String lang) {
            return new Localhost_CrJersey.Webservice.Lang(_client, lang);
        }

        public static class Lang {

            private Client _client;
            private UriBuilder _uriBuilder;
            private HashMap<String, Object> _templateAndMatrixParameterValues;

            /**
             * Create new instance using existing Client instance
             *
             */
            public Lang(Client client, String lang) {
                _client = client;
                _uriBuilder = UriBuilder.fromPath("http://localhost:7101/cr/jersey/");
                _uriBuilder = _uriBuilder.path("webservice");
                _uriBuilder = _uriBuilder.path("{lang}");
                _templateAndMatrixParameterValues = new HashMap<String, Object>();
                _templateAndMatrixParameterValues.put("lang", lang);
            }

            /**
             * Create new instance
             *
             */
            public Lang(String lang) {
                this(Client.create(), lang);
            }

            /**
             * Get lang
             *
             */
            public String getLang() {
                return ((String)_templateAndMatrixParameterValues.get("lang"));
            }

            /**
             * Set lang
             *
             */
            public Localhost_CrJersey.Webservice.Lang setLang(String lang) {
                _templateAndMatrixParameterValues.put("lang", lang);
                return this;
            }

            public Localhost_CrJersey.Webservice.Lang.Greeting greeting() {
                return new Localhost_CrJersey.Webservice.Lang.Greeting(_client,
                                                                       ((String)_templateAndMatrixParameterValues.get("lang")));
            }

            public static class Greeting {

                private Client _client;
                private UriBuilder _uriBuilder;
                private HashMap<String, Object> _templateAndMatrixParameterValues;

                /**
                 * Create new instance using existing Client instance
                 *
                 */
                public Greeting(Client client, String lang) {
                    _client = client;
                    _uriBuilder = UriBuilder.fromPath("http://localhost:7101/cr/jersey/");
                    _uriBuilder = _uriBuilder.path("webservice");
                    _uriBuilder = _uriBuilder.path("{lang}");
                    _uriBuilder = _uriBuilder.path("greeting");
                    _templateAndMatrixParameterValues = new HashMap<String, Object>();
                    _templateAndMatrixParameterValues.put("lang", lang);
                }

                /**
                 * Create new instance
                 *
                 */
                public Greeting(String lang) {
                    this(Client.create(), lang);
                }

                /**
                 * Get lang
                 *
                 */
                public String getLang() {
                    return ((String)_templateAndMatrixParameterValues.get("lang"));
                }

                /**
                 * Set lang
                 *
                 */
                public Localhost_CrJersey.Webservice.Lang.Greeting setLang(String lang) {
                    _templateAndMatrixParameterValues.put("lang", lang);
                    return this;
                }

                public Bean getAsBean() {
                    UriBuilder localUriBuilder = _uriBuilder.clone();
                    WebResource resource =
                        _client.resource(localUriBuilder.buildFromMap(_templateAndMatrixParameterValues));
                    WebResource.Builder resourceBuilder = resource.getRequestBuilder();
                    resourceBuilder = resourceBuilder.accept("*/*");
                    return resourceBuilder.method("GET", Bean.class);
                }

                public <T> T getAs(GenericType<T> returnType) {
                    UriBuilder localUriBuilder = _uriBuilder.clone();
                    WebResource resource =
                        _client.resource(localUriBuilder.buildFromMap(_templateAndMatrixParameterValues));
                    WebResource.Builder resourceBuilder = resource.getRequestBuilder();
                    resourceBuilder = resourceBuilder.accept("*/*");
                    return resourceBuilder.method("GET", returnType);
                }

                public <T> T getAs(Class<T> returnType) {
                    UriBuilder localUriBuilder = _uriBuilder.clone();
                    WebResource resource =
                        _client.resource(localUriBuilder.buildFromMap(_templateAndMatrixParameterValues));
                    WebResource.Builder resourceBuilder = resource.getRequestBuilder();
                    resourceBuilder = resourceBuilder.accept("*/*");
                    return resourceBuilder.method("GET", returnType);
                }

            }

        }

    }

}

This looks little bit wordy, but things become a little bit simpler if you just look at the interface that has been generated. For example you can now access each part of the path directly using the accessor methods. In this example we are asking for just the english version of the greeting:

public class Test
{
   public static void main(String[] args) {
 
      Bean bean = Localhost_CrJersey.webservice().lang("en").greeting().getAsBean();
      System.out.println(bean.getMessage());

   }
}

Finally for each method that returns a JAX-B element there are also two that take Class.class or GenericType as parameters. This allows the client to access the ClientResponse object, or any other mapping in a way that just wasn't possible in the old API:

public class Test
{
   public static void main(String[] args) {
 
      ClientResponse response = Localhost_CrJersey.webservice().lang("en").greeting().getAs(ClientResponse.class);
      System.out.println(response.getStatus());

   }
}

Now this client currently has limitations in that it doesn't know about HATEOAS and you can't control the URL with deployment descriptors; but it is a start that can hopefully evolve into a more generally useful tool. Also I have played with the constructors for Wadl2Java a little bit to make easier to integrate into IDE tooling, it is the work of a couple of afternoons to wrap this up in a wizard and integrate this into your IDE of choice.

Wednesday, July 13, 2011

Auttomatic XML Schema generation for Jersey WADLs

I have been doing a little bit of work recently on Jersey implementation and one of the precursors to this has been to try to get the WADL that Jersey will generation by default to contain just a little bit more information with regards to the data being transferred. This makes it possible to help the user when generating client code and running testing tools against resources.

To this end I have put together a WADL generator decorator that examines all the JAX-B classes used by the -RS application and generates a bunch of XML Schema files. This is now in 1.9-SNAPSHOT which you can download from the Jersey web site in the normal way. (If you want to use the JResponse part of this you will need a build after the 13th of July)

This feature is not enabled by default in 1.9; but hopefully with some good feedback and a small amount of caching I might convince the Jersey bods to make this the default. For the moment you need to create and register a WsdlGeneratorConfig class to get this to work. So your class might look like this:

package examples;

import com.sun.jersey.api.wadl.config.WadlGeneratorConfig;
import com.sun.jersey.api.wadl.config.WadlGeneratorDescription;
import com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator;

import java.util.List;

public class SchemaGenConfig extends WadlGeneratorConfig {

    @Override
    public List<WadlGeneratorDescription> configure() {
        return generator( 
                WadlGeneratorJAXBGrammarGenerator.class   ).descriptions();
    }
}

You then need to make this part of the initialization of the Jersey servlet, so your web.xml might looks like this:

<?xml version = '1.0' encoding = 'windows-1252'?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
         version="2.5">
  <servlet>
    <servlet-name>jersey</servlet-name>
    <servlet-class>com.sun.jersey.spi.container.servlet.ServletContainer</servlet-class>
    <init-param>
      <param-name>com.sun.jersey.config.property.WadlGeneratorConfig</param-name>
      <param-value>examples.SchemaGenConfig</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>jersey</servlet-name>
    <url-pattern>/jersey/*</url-pattern>
  </servlet-mapping>
</web-app>

For the purposes of this blog I am just going to show the three basic references to entities that the code supports. So at the moment if will obviously process classes that are directly referenced and either a return value or as the entity parameter on a method. The code also supports the Jersey specific class JResponse, which is a subclass of Response that can have a generic parameter. (Hopefully this oversight will be fixed in JAX-RS 2.0)

package examples;

import com.sun.jersey.api.JReponse;

import examples.types.IndirectReturn;
import examples.types.SimpleParam;
import examples.types.SimpleReturn;

import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;

@Path("/root")
public class RootResource {
    
    @GET
    @Produces("application/simple+xml")
    public SimpleReturn get() {
        return new SimpleReturn();
    }
    
    @GET
    @Produces("application/indrect+xml")
    public JResponse<IndirectReturn> getIndirect() {
        return JResponse.ok(new IndirectReturn() )
            .type( "application/indrect+xml" ).build();
    }
    
    @PUT
    @Consumes("application/simple+xml")
    public void put(SimpleParam param) {
        
    }
        
}

The type classes are all pretty trivial so I won't show them here. The only important factor is that they have the @XmlRootElement annotation on them. Although not shown here you can also use the JAX-B annotation @XmlSeeAlso on the resource classes to reference other classes that are not directly or indirectly referenced from the resource files. The most common use case for this is when you have a subtype of a class.

So enough of the java code, lets see what the WADL that is generated looks like:

<?xml version="1.0" encoding="UTF-8"?>
<application xmlns="http://wadl.dev.java.net/2009/02">
    <doc xmlns:jersey="http://jersey.java.net/" jersey:generatedBy="Jersey: 1.9-SNAPSHOT 07/13/2011 11:13 AM"/>
    <grammars>
        <include href="application.wadl/xsd0.xsd">
            <doc xml:lang="en" title="Generated"/>
        </include>
    </grammars>
    <resources base="http://localhost:7101/JerseySchemaGen-Examples-context-root/jersey/">
        <resource path="/root">
            <method name="GET" id="get">
                <response>
                    <representation xmlns:ns2="urn:example"
                        mediaType="application/simple+xml" element="ns2:simpleReturn"/>
                </response>
            </method>
            <method name="PUT" id="put">
                <request>
                    <representation xmlns:ns2="urn:example"
                        mediaType="application/simple+xml" element="ns2:simpleParam"/>
                </request>
            </method>
            <method name="GET" id="getIndirect">
                <response>
                    <representation xmlns:ns2="urn:example"
                        mediaType="application/indrect+xml" element="ns2:indirectReturn"/>
                </response>
            </method>
        </resource>
    </resources>
</application>

In this example there is only one schema in the grammar section; but the code supports multiple schemas being generated with references between them. Let look at the schema for this example, note I did say the classes were trivial!

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema version="1.0" targetNamespace="urn:example"
    xmlns:tns="urn:example" xmlns:xs="http://www.w3.org/2001/XMLSchema">
    <xs:element name="indirectReturn" type="tns:indirectReturn"/>
    <xs:element name="simpleParam" type="tns:simpleParam"/>
    <xs:element name="simpleReturn" type="tns:simpleReturn"/>
    <xs:complexType name="indirectReturn">
        <xs:sequence/>
    </xs:complexType>
    <xs:complexType name="simpleReturn">
        <xs:sequence/>
    </xs:complexType>
    <xs:complexType name="simpleParam">
        <xs:sequence/>
    </xs:complexType>
</xs:schema>

There is still some internal work to be done on caching; but the basics are in place. Feedback would be appreciated, particularly in cases where the code doesn't see the referenced classes. Finally thanks to Pavel Bucek for being patient as I learned the ropes.

Update 5th September 2011 This feature has been enabled by default so you no longer have to perform any of the WadlGeneration configuration when working with 1.9 final release of Jersey.

Friday, July 8, 2011

Slightly tidier version of super-type tokens without anonymous inner classes

If you have ever used or written a fluent API eventually you have to deal with the fact that in Java generic types are erased. For example in the current Jersey client API you find the following problems:


   // Can't do generic class literals, compile error

   List<Employee> result = resource.get(List<Employee>.class);

   // Code won't function as it doesn't know the generic type

   List<Employee> result = resource.get(List.class);

Most API's solve this using Grafter's Gadget which creates a class where a trivial subclass bakes in the generic type. For example in REST the class in called GenericType:


   // Creates a anonymous instance of GenericType with the generic parameters
   // baked in

   List<Employee> result = resource.get(new GenericType<List<Employee>>() {});

I have never liked this because well it is just a little bit untidy for my liking, creating an inner class for this one time use doesn't feel right. You can of course extract this to a constant; but I was looking for a better way to define a literal. It turns out that with a big of playing about you can write code to do the following:


   private static final GenericType<List<String>> LIST_STRING =
           GenericType.field();

The trick here is to examine the calling stack when creating the GenericType instance to final the class that the field be being attached to. You can then go back and work out generic type of the field is when the constants is used. Here is a rough sketch of the code integrating with the Jersey GenericType:



public class GenericType<T> {

    public static final <U> GenericType<U> field()
    {
       return new FieldGenericType<U>();
    }

    // Rest of GenericType implementation
    // .....
}



class FieldGenericType<T>
      extends GenericType<T> {

    // TODO if we are not allowed to create a security manager just
    // use new Exception().getStackTraceElement() and then get hold
    // of the string name of the class
    static class ModifiedSecurityManager extends SecurityManager {

        public Class[] getClassContext() {
            return super.getClassContext();
        }
    }
    static final ModifiedSecurityManager sm = new ModifiedSecurityManager();


    /**
     * Provide an adapted version of the paramaterized type based on the
     * value of the field.
     */

    static class AdaptedParameterizedType implements ParameterizedType {

            // Find the class that created us, a few steps up the stack
            private Class owningClass = sm.getClassContext()[4];
            // The owning instances, can't be set in the constructor
            private FieldGenericType instance;
            private ParameterizedType type;

            /** Lazily create the type **/
            private ParameterizedType getType() {

                // TODO make thread safe
                //

                if (type == null) {

                    // Look for a field of type GenericType which is the same
                    // references the "instance" of the FieldGenericType
                    //

                    found:
                    for (Field field : owningClass.getDeclaredFields()) {
                        final Class<?> fieldType = field.getType();
                        if (GenericType.class.isAssignableFrom(fieldType))
                        {
                            try {
                                field.setAccessible(true);
                                if (field.get(owningClass) == instance) {
                                    // We need the generic parameter of the GenericType class
                                    //
                                    type = (ParameterizedType)
                                            ((ParameterizedType)field.getGenericType()).getActualTypeArguments()[0];
                                }
                            } catch (IllegalArgumentException ex) {
                                Logger.getLogger(Example.class.getName()).log(Level.SEVERE, null, ex);
                            } catch (IllegalAccessException ex) {
                                Logger.getLogger(Example.class.getName()).log(Level.SEVERE, null, ex);
                            }
                        }
                    }

                    //

                    throw new Error("Cannot find the related field");
                }

                return type;
            }

            public Type[] getActualTypeArguments() {
                return getType().getActualTypeArguments();
            }

            public Type getRawType() {
                return getType().getRawType();
            }

            public Type getOwnerType() {
                return getType().getOwnerType();
            }

            public String toString() {
                return getType().toString();
            }
        }     


    public FieldGenericType() {
        super(new AdaptedParameterizedType() );

        // Inject the this value, now it has been created
        ((AdaptedParameterizedType)getType()).instance = this;


    }
}

Perhaps not that much tidier for day to day coding; but not using anonymous inner classes just feels like the right thing to do. I guess time will tell if I actually end up using it. I can see it being very useful when trying to generate boilerplate code in an IDE.

Monday, July 4, 2011

Off-loading your test execution using Hudson

One of the annoyances if you have a load of tests for a particular project is that running them can tie up your machine for a while. There are also sometimes consistency issues when running on different machines with different window managers and operating system versions. One thing we do is for every main hudson job we create a copy just to run the tests for a particular developer. It solves the "it runs on my machine" dilemma and frees up the developer to get on with other work. In our part of the organization running the tests on another machine is a mandatory step before merging to main.

This example uses subversion; but it most likely work equally well with any other source control system with a similar interface. It also assumes that all the developers are working on branches, you are doing this right?.

First of all you take a copy of you original job, in my case I am running test for the abbot project so I am starting with "abbot_SF", and create a copy of this job with a suitable postfix. In our case the job is called "abbot_SF_TestTrans". You need to first disable any normal build triggers so this job only gets used when there is a manual action. We keep the results separate as it makes the normal build results much easier to interpret.

So to this new job add a new Parameter "BRANCH" of type string and update the repository URL to contain this parameter, so in my case https://abbot.svn.sourceforge.net/svnroot/abbot/abbot/branches/$BRANCH. You will also notice that I have the used the description setting plugin to capture the branch name automatically. (You could also just put the $BRANCH parameter in the description field and leave the RegEx blank). After you have finished you have something like this, ignore the red ink as that appears to be a local configuration issue.

Now if you click build on the job you get a nice interface where you can ask Hudson to build a particular branch, leaving the developer to get on with the next job while the build machine does it's business.

Speaking of that, it looks like I have some bugs to fix...

P.S. You might also want to tick the "Execute Concurrent Builds if necessary" flag so that if you have more than one developer you can make proper use of your farm.

Wednesday, June 15, 2011

Working around the need for a network connection in Spring and WebLogic SCA tooling in JDeveloper

Unfortunately the Spring tooling we have in JDeveloper 11.1.2 and PS5 contains a bug where it will try to access the network for a .dtd in order to validate the xml files. If you are running the HTTP Analyzer you might see something like the following:

This is fine if you have a network connection, if annoying, but with a timeout of 5 minutes in some cases it can be really annoying if you are trying to work where you cannot see the internet as a whole.

We are going to use a feature of the HTTP Analyzer to use a recording of this response to pretend to be the service in question. For of all we need a log file that contains the response in question, so select the message and press the save button:

Then we have to create a Tape Rule which allows the analyzer to respond using messages from a log file in response to request to a particular URL:

You need to set up the rule as follows, note the reference URL is not part of the definition; but can be used to validate the rule filter settings.

And you should be good to go, next time you start up JDeveloper start the HTTP Analyzer first, and you should see that the messages are being provided by the tape rule rather than the network.

We don't have a handy way to start the analyzer when the tool does; so you have to be careful not to have any open editors. But for some cases this is going to be a useful workaround.

Wednesday, May 25, 2011

Jersey issue when running with ADF / Oracle XDK

We have discovered a late breaking bug in WLS that prevents Jersey applications from de-serializing XML and in some cases JSON documents. (So GET will work but PUT or POST will fail) This currently affects JDeveloper 11.1.1.5.0 aka PS4. You might see an exception that looks like this:

javax.xml.parsers.FactoryConfigurationError: WebLogicSAXParser
cannot be created.SAX feature
'
http://xml.org/sax/features/external-general-entities'
not
supported.
at weblogic.xml.jaxp.RegistrySAXParser.<init>(RegistrySAXParser.java:73)
at weblogic.xml.jaxp.RegistrySAXParser.<init>(RegistrySAXParser.java:46)
at
weblogic.xml.jaxp.RegistrySAXParserFactory.newSAXParser(RegistrySAXParserFacto
ry.java:91) 
...

When you create ADF components in an application you will find that at the .EAR level a weblogic-application.xml is created at deployment time which override the default WLS xml parser version to use the Oracle XDK version. You might see something like this:

<parser-factory>
<saxparser-factory>oracle.xml.jaxp.JXSAXParserFactory</saxparser-factory>
<document-builder-factory>oracle.xml.jaxp.JXDocumentBuilderFactory</document-builder-factory>
<transformer-factory>oracle.xml.jaxp.JXSAXTransformerFactory</transformer-factory>
</parser-factory>

When Jersey tries to convert a document to JAX-B classes it sensible sets a feature on the XML parser that prevents resolving external entities to prevent certain security attacks. This code in SAXParserContentProvider does understand that some parser don't support this feature as you can see:

@Override
    protected SAXParserFactory getInstance() {
        SAXParserFactory f = SAXParserFactory.newInstance();

        f.setNamespaceAware(true);
        
        if (!disableXmlSecurity) {
            try {
                f.setFeature("http://xml.org/sax/features/external-general-entities", Boolean.FALSE);
            } catch (Exception ex) {
                throw new RuntimeException("Security features for the SAX parser could not be enabled",  ex);
            }

            try {
                f.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, Boolean.TRUE);
            } catch (Exception ex) {
                LOGGER.log(Level.WARNING, 
                        "JAXP feature XMLConstants.FEATURE_SECURE_PROCESSING cannot be set on a SAXParserFactory. " +
                        "External general entity processing is disbaled but other potential securty related" +
                        " features will not be enabled.", 
                        ex);
            }
        }

        return f;
    }

This would normally be the end of it if it was not for bug 12543845, where the invalid features is incorrectly stored prevent the future creation of any parsers.

Luckily there is a workaround in Jersey, you can simply set this using a Servlet init param as follows:

<servlet>
    <servlet-name>jersey</servlet-name>
    <servlet-class>com.sun.jersey.
spi.container.servlet.ServletContainer</servlet-class>
   <init-param>

<param-name>com.sun.jersey.config.feature.DisableXmlSecurity</param-name>
      <param-value>true</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
  </servlet> 

Be careful not to apply this property when you are not using the XDK parser otherwise you might open yourself up to a range of attacks which is not going to be a good thing.

Updated 30th March 2012: You might also need a similar workaround for the Jersey client as shown here. (I have raised this Jersey issue to see if this can be improved)

DefaultClientConfig cc = new DefaultClientConfig();
  cc.getFeatures().put(
    FeaturesAndProperties.FEATURE_DISABLE_XML_SECURITY, true);
  Client c = Client.create(cc);

Updated 15th August 2014: Slightly different for JAX-RS 2.0 client under Jersey, set this property on a configurable such as your client instance:

   configurable.property(MessageProperties.XML_SECURITY_DISABLE, Boolean.TRUE);


Tuesday, May 10, 2011

JDeveloper, the Endorsed Directory, bootclasspath, and JAX-WS 2.2.x

I have been working with JAX-WS 2.2.x recently and one of problem you might fact whilst playing with a client is that it uses more recent version of the JAX-WS API than that supplied with JDK 1.6. This shouldn't be a problem normally but there are some issues that might get in your way when using JDeveloper.

The first thing to note is that I am going to ignore the standard advice to user the java.endorsed.dirs system property as it can only specify a directory and not individual jar files. This is not always a problem; but if your jars are not packaged in a convenient way it means you have to move then to a new directory. Instead I am going for the slightly less standard -Xbootclasspath/p: property which prepends the jars to the boot classpath with much the same effect and I can specify the paths to individual jars. I will note with the screen grabs where you might go differently if you want to ignore this advice.

First things first you need to configure the compiler otherwise when you try to compile a web services client, in this case, you find error messages relating to the constructors on the Service class. You can find these settings under project properties. You could use the -endorsedpath here if you wish. For my purposes I had to place both the jax-xml-bind.jar and jax-xml-ws.jar on the bootclasspath. Of course use you platform specific path separator between the two paths. (If you are doing JAX-WS you are going to need the matching version of JAX-B)

Unfortunately you might find that this point that the project will fail to compile due to bug 12538161, there is a workaround which is to ask the compilation to happen as a separate process. This is going to be slightly slower; but in most cases you are not going to notice the difference.

Depending on your deployment context you might also like to update the run options for the your project, again I have used the bootclasspath but you could use the -Djava.endorsed.dirs depending on how you dependencies are laid out.

You can compile and run against the new API which will solve 99% of problem in the area. Fortunately only a few Java API's are packaged in this way. Hopefully in JDK 8 we can get the modularization sorted out so this sort of thing will be less of a problem.

Monday, April 11, 2011

Generating an inline schema in JDeveloper

A question that comes up now and then is how to generate a WSDL with the Schema in-line rather than as a separate file. This is required for certain mobile platforms and in particular Siebel Tools. This feature wasn't added to the JAX-WS RI until version 2.2.1 which is ahead of the version use in JDeveloper. It is possible to get some integration using the external tools command to make use of a later version of this tool in JDeveloper.

To get this to work you first need to download JAX-WS RI 2.2.1 or later. The configuration is similar to what we have used previously on this blog with a few specific modifications.

The program executable is simply the location of the wsgen.sh or wsget.bat file that comes with the RI download. The second line is more complicated:

-Xendorsed -d ${project.outputdirectory} -s ${project.outputdirectory} -cp ${project.outputdirectory};${project.classpath} ${target.class} -r ${file.dir} -wsdl  -inlineSchemas

Reading left to right we are: making sure we used the endorsed property to use the updated API; specifying the project classes directory for any output; specifying the project classpath; the class name from the current selection; where the wsdl file should be written out, the same one as the java file; and finally that we want to generate a wsdl and that it should be inline.

In the rest of the wizard you are likely to want to configure the tool to appear in the navigator and only appear for java source files. All pretty vanilla stuff. Once this is configured you should have a menu item you can invoke from the JDeveloper UI. After you invoke this action if you didn't check re-load external tools in the wizard you need to press the refresh button to see any changes.

There is one remaining manual step and that is to associate the wsdl with the source file, to do this you simply need to update the wsdlLocation attribute:

package webservice;

import javax.jws.WebService;

@WebService(wsdlLocation = "/webservice/HelloService.wsdl")
public class Hello {
    public String hello(String name) {
        return "";
    }
}

This will now deploy with the new compact all in one wsdl document. JDeveloper will notice you have a file and tell you if the files get out of sync; but you will of course need to use your external tool command otherwise you will loose your in-line schema.