Thursday, June 15, 2017

Interpreting wpad.dat using jrunscript from JDK 8 to work out the right proxy.

In many environments you find that the correct proxy can only be fetched by parsing http://wpad/wpad.dat, which is a pain because it is actually a JavaScript file. Working out the right proxy to call for say a unix environment is a bit of a fiddle particularly if you company is huge and requires different proxies for different countries. JDK 8 to the rescue, as you can run a simple script in the Nashorn environment to get the suggested proxy for one host and use that for everything:

jrunscript -e "`curl -o - -s http://wpad/wpad.dat` function isPlainHostName(hostname) { return hostname.indexOf('.')==-1;}; var proxyList = FindProxyForURL('http://github.com','github.com'); var firstProxy = proxyList.split(';')[0].trim(); print('DIRECT'.equals(firstProxy) ?  '' : 'http://' + firstProxy.split(' ')[1]);"

Depending on your local environment you might have to implement one or more functions from the list that your wpad.dat expects, in our case I needed to implement isPlainHost.

The last part of the script just picks the first proxy name, as the function returns a list of PROXY is suggested order. The raw output looks like this:

PROXY london.example.com:80; PROXY sf.example.com:80; DIRECT;

So we just pick the first one with http as the prefix, as some tools such as docker are fussy about this, or returns an empty string if DIRECT is recommended.

http://london.example.com:80

Tuesday, March 14, 2017

The "debugger" reserved word in JavaScript

This is a handy little tip for cases where you just can't get your debugger to start in the right context or the framework you are using doesn't give you a clear path to know when you file might be loaded. (For example ABCS, or Application Builder Cloud Service, that I am currently working on has this problem)

In this case just add the debugger statement in your javascript and voila when run in a browser the debugger is popped up.

   someCode().then(value => {
       debugger;
   });


Always pays to read the reserved word list when learning a new language, you never know what you might find.

Monday, August 15, 2016

What is bound to generic types for method references in java 8, types and exceptions

So most of the time you are probably aware of method references that either reference instance of static methods directly, and they are useful but not necessarily that interesting.

Integer example = 1;

// We can bind the instance variables, in this case just the return type is bound but there couple be method parameters also
Supplier<String> s = example::toString;

// We can bind to static method, here the generic parameter are the method parameter and return type
Supplier<Long> f = System::currentTimeMillis;
Function<Integer,Integer> f = example::compareTo;


A less common formulation is to pass in instance methods and find the first generic parameter of of the function to be the type, this allows you to easily pass in a range of actions to operate on a common type:

Function<Integer,String> f = Integer::toString

For example you can create an equals method that works on a subset of properties using functions mapped to instance methods as in the above example:

public static <T> boolean equals(T one, T two, Function<? super T, ?>... accessors) {

    if (one == two) {
        return true;
    } else  if (one==null || two==null) {
        return false;
    }

    return Stream.of(accessors).allMatch(accessor ->
        Objects.equals(accessor.apply(one),accessor.apply(two)));
}

if (equals(one, two, Thing::getName, Thing:getOtherProperty)) ...;



Finally you can also bind the exception thrown from the method to one of the generic parameters. (Here I am using ThrowingException and ThrowingSupplier my home brew interfaces that are like there namesakes but have a generic parameter E for the exception thrown) This allows you to make you "closure" transparent to exceptions. This is more useful in a lot of cases when compared to the Stream throw nothing and "throws Exception" extremes.

ThrowingException<String,Integer,NumberFormatException> te = Integer::parseInt;


You can write funky closure methods that will throw different exceptions based on the passed in method reference does, no more catch (Exception).

public static <T, E extends Exception> T withCC(Class<?> contextClass, 
   ThrowingSupplier<T,E> action) throws E {

    Thread t = Thread.current();
    ClassLoader cl = t.getContextClassLoader();
    try {
        t.setContextClassLoader(contextClass.getClassLoader());
        return action.get();
    } finally {
        s.setContextClassLoader(cl);
    }
}
       

// Throws IOException, complier knows that this method call throws IOException

String value = withCC(Example.class, () -> {
    ...
    ... new FileOutpuStream(file); ...
    ...
});

// Throws another exception, complier knows that this method call throws RMIExeption

String value = withCC(Example.class, () -> {
        ...
        throw new RMIException();
    });


Once you understand the last two, method reference start to become far more interesting.

Thursday, June 2, 2016

Getting a name for someone to connect back to your server.

When doing test automation it is often the case you need to know the name of the current machine in order to prompt another machine to connect to it, particularly if you are running your tests in parallel. This week I was trying to get the server under test to connect back to a WireMock server running on the slave test machine.

The standard response on stack overflow is to use the following pattern to get a network address. In my version here if we can't resolve the name then we are assuming we are running on a developers laptop on VPN so all the tests are run on the same machine. (Hence localhost)

String hostName = "localhost";
try {
    InetAddress addr = InetAddress.getLocalHost();
    String suggestedName = addr.getCanonicalHostName();
    // Rough test for IP address, if IP address assume a local lookup
    // on VPN
    if (!suggestedName.matches("(\\d{1,3}\\.?){4}") && !suggestedName.contains(":")) {
        hostName = suggestedName;
    }
} catch (UnknownHostException ex) {
}

System.out.println(hostName);

The problem comes is that we have to trust the local machine settings, for example /etc/hostname, which can result in a network name that is not accessible from another machine. To counter this I wrote the following code to work over the available network interfaces to find a remotely addressable network address name that can be used to talk back to this machine. (I could use an IP address but they are harder to remember, particularly as we are moving towards IPv6)

String hostName = stream(wrap(NetworkInterface::getNetworkInterfaces).get())
        // Only alllow interfaces that are functioning
        .filter(wrap(NetworkInterface::isUp))
        // Flat map to any bound addresses
        .flatMap(n -> stream(n.getInetAddresses()))
        // Fiter out any local addresses
        .filter(ia -> !ia.isAnyLocalAddress() && !ia.isLinkLocalAddress() && !ia.isLoopbackAddress())
        // Map to a name
        .map(InetAddress::getCanonicalHostName)
        // Ignore if we just got an IP back
        .filter(suggestedName -> !suggestedName.matches("(\\d{1,3}\\.?){4}")
                                 && !suggestedName.contains(":"))
        .findFirst()
        // In my case default to localhost
        .orElse("localhost");

System.out.println(hostName);

You might notice there a are a few support methods being used in there to tidy up the code, here are the required support methods if you are interested.

@FunctionalInterface
public interface ThrowingPredicate<T, E extends Exception>{

    boolean test(T t) throws E;
}

@FunctionalInterface
public interface ThrowingSupplier<T, E extends Exception>{

    T get() throws E;
}

public static <T, E extends Exception> Predicate<T> wrap(ThrowingPredicate<T, E> th) {
    return t -> {
        try {
            return th.test(t);
        } catch (Exception ex) {
            throw new RuntimeException(ex);
        }
    };
}

public static <T, E extends Exception> Supplier<T> wrap(ThrowingSupplier<T, E> th) {
    return () -> {
        try {
            return th.get();
        } catch (Exception ex) {
            throw new RuntimeException(ex);
        }
    };
}

// http://stackoverflow.com/a/23276455
public static <T> Stream<T> stream(Enumeration<T> e) {
    return StreamSupport.stream(
            Spliterators.spliteratorUnknownSize(
                    new Iterator<T>() {
                public T next() {
                    return e.nextElement();
                }

                public boolean hasNext() {
                    return e.hasMoreElements();
                }
            },
                    Spliterator.ORDERED), false);
}

Thursday, April 7, 2016

Converting string configuration properties to other types, with a bit of Optional.

Somedays you come across some code and think that's pretty, why didn't I think of that? So my long time colleague Mark Warner has a nice twist on the standard name/value store pattern using method references to deal with converting from a String.

int size = store.getProperty("cache.limit", 500, Integer::parseInt);
    boolean enabled = store.getProperty("cache.enabled", true, Boolean::getBoolean);

I took his example and refactored it slightly to return Optional, and I ended up with the following:

public Optional<String> getProperty(
        String propertyName) {
    return Optional.ofNullable(map.get(propertyName));
}
   
public <T> Optional<T> getProperty(
        String propertyName, 
        ThrowingFunction<String,? extends T,? extends Exception> func ) {

    return getProperty(propertyName).map(val -> {
        try {
            return func.apply( val );
        } catch ( Exception e ) {
            LOGGER.severe( () -> "Invalid property transform, will default " + e.getMessage() );
            return null;
        }
    });
}

This means that the default value ends up being provided by the Optional which is a nice application of OAOO.

int size = store.getProperty("cache.limit", Integer::parseInt).orElse(500);
    boolean enabled = store.getProperty("cache.enabled", Boolean::getBoolean).orElse(true);

I think this is even tidier; but it does depend on who you feel about using Optionals.

Thursday, March 10, 2016

Lambda of Lambda, if/else from an Optional

So I got frustrated with two limitations of the Optional interface in JDK 8. The first problem is that there is no obvious way to perform an else operation in a block as there is only a isPresent method unless you are using an old school if statement. The second problem is of course the old chestnut that even if you could do that the methods would not be able to throw a checked exception. (Yes you can wrap with a RuntimeException but it is not the prettiest.)

The workaround I found was to use the map function as the success case and the orElseGet to return the failure case. In both branches the code returns an instance of ThrowingRunnable by having a lambda return a lambda. The run() is then called at the end and it can throw any exception it wants to.

@FunctionalInterface
public interface ThrowingRunnable<E extends Throwable>  {
    
    public void run() throws E;
}



Optional<Credential> credential = ....

credential.<ThrowingRunnable<IOException>>map(auth -> () -> {
                PasswordWrapper pw = auth.getToken();
                ... // something that might throw an IOException
            }).orElseGet(() -> () -> {
                        response.setStatus(401);
                        LOGGER.log(Level.INFO, "credential is not found");
                    }
            ).run();


This is possibly excessive for this particular use case; but I can see this technique being useful elsewhere and it is worth knowing what it looks like so it is not a surprise in others code.

Update 14th March 2016: Thanks to Michael Rasmussen on the DZone version of this article he noted I could use orElse rather than orElseGet to remove the second double lambda.

credential.<ThrowingRunnable<IOException>>map(auth -> () -> {
                PasswordWrapper pw = auth.getToken();
                ... // something that might throw an IOException
            }).orElse(() -> {
                        response.setStatus(401);
                        LOGGER.log(Level.INFO, "credential is not found");
                    }
            ).run();



Wednesday, March 2, 2016

Getting hold of an un-used port when writing a test

A really quick post today, I was writing a unit test that needed to put up a temporary JAX-RS resource in order to test something very specific with out the need of a large testing framework. Unfortunately the code I was using didn't know about replacing zero with a random allocated port so I had to do something different. The nice tidy code that does this looks like the following:

try(ServerSocket ss = new ServerSocket(0)) {
   freePort = ss.getLocalPort();
}

server = createServerWithPort(freePort);

But of course there is a race condition here as there could be another process on the machine running the exact same code that gets lucky with its time on the CPU - note the API I was using couldn't use the ServerSocket directly so there is a gap where the port is not tied up. So to rule out the one in a million failure chance, which always comes on a Friday at 5pm, we need to wrap this code in a loop.

while(server==null) {
    try(ServerSocket ss = new ServerSocket(0)) {
       freePort = ss.getLocalPort();
    }

    server = createServerWithPort(freePort);
}

This may seem extreme; but if you are running some kind of CI system like Hudson or Jenkins it is not unusual to be running multiple executors on the same machine. You might have two developers running preflights at the same time causing an annoying intermittent failure as described above.

Finally a version using a JDK 8 stream with a helper method to convert the IOException from ServerSocket into a RuntimeException. Some would argue this is less readable; but I find it more explicit.

server = Stream.generate(convertException(() -> {
    try(ServerSocket ss = new ServerSocket(0)) {
        return ss.getLocalPort();
    }
})).map(TestClass::createServerWithPort)
.filter(Objects::nonNull)
.findFirst().get();