Monday, November 16, 2009

ISO Language Codes

Every so often, I need to obtain a full list of ISO language codes for use in an application. I thought I'd write a reference post for future reference, and whomever else might find this kind of thing useful.

Languages are given a two-character code in ISO-639-2, which the Library of Congress is nice enough to host for us here.

I wrote a simple bit of code to generate a 'Language' table for MySQL, and posted it here:
http://www.fiestacabin.com/files/iso-lang-codes.sql.txt

Feel free to use this snippet; the table I created looks like:

CREATE TABLE IF NOT EXISTS Language (
id unsigned int not null auto_increment,
code char(2) not null,
description varchar(64),
primary key (id),
unique index iLanguage_code (code)
) ENGINE=InnoDB DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;


Just for fun, I am including my Java snippet which generated the SQL; critique at will!

package sandbox;

import java.io.File;
import java.util.List;

import org.apache.commons.io.FileUtils;
import org.apache.commons.lang.StringEscapeUtils;
import org.apache.commons.lang.StringUtils;
import org.apache.commons.lang.text.StrTokenizer;

public class ISOLangCodes {
public static void main(String[] args) throws Exception {
List<String> lines = (List<String>) FileUtils.readLines(new File("c:/temp/iso-lang-codes.txt"), "UTF-8");

for( String line : lines ){
StrTokenizer s = StrTokenizer.getCSVInstance(line).setDelimiterChar('|');
String langCode = s.getTokenArray()[2];
String langDesc = s.getTokenArray()[3];

if( StringUtils.isNotBlank(langCode) )
System.out.println(
"INSERT INTO Languages (code, description) VALUES ('" + langCode +
"', '" + StringEscapeUtils.escapeSql(langDesc) + "');");
}
}
}

Wednesday, October 14, 2009

Recursively Scanning Packages for Marked Classes with Spring

I thought I'd share a neat snippet of code which allows you to recursively scan all of the classes in a given package for those which have been marked with a particular annotation. This could be useful, for example, if you have a static factory class that wanted to know all of the classes which could be instantiated for a particular purpose, without having to add them in some sort of static block at the top of the factory.

String pkg = MyBaseClass.class.getPackage().getName();
String pkgPath = pkg.replace('.', '/');

List<Class<?>> classes = new ArrayList<Class<?>>();

PathMatchingResourcePatternResolver resourceResolver = new PathMatchingResourcePatternResolver();
Resource[] packageClasses = resourceResolver.getResources("classpath*:" + pkgPath + "/**/*.class");

for( Resource res : packageClasses ){
classes.add(Class.forName(pkg + '.' + FilenameUtils.getBaseName(res.getFilename())));
}

for( Class<?> c : classes ){
MyAnnotation myAnnotation = c.getAnnotation(MyAnnotation.class);

if( myAnnotation != null ){
// this class was marked with @MyAnnotation; handle accordingly
}
}



I could devote a whole post to Spring's resource loading capabilities; they are the nicest I've used. The above snippet utilizes Spring's PathMatchingResourcePatternResolver, which is able to list resources which match a given ant-style path description. The secret is "classpath*:some/path"; note the asterisk before the colon. This searches *all* accessible jars on the classpath, instead of stopping in the first one in which we find a match. One gotcha here is that you can't do something like "classpath*:*/service/*Service.class"; it will choke if you attempt a leading wildcard search.

Sorting internationalized lists in JSP

One topic which seems to keep coming up for me lately is sorting values for a web application which have been extracted into a resource bundle, for internationalization. Everything I've read on the web says the preferred place to do the sorting is in the servlet / controller class, with the view (in this article, JSP) merely responsible for iterating over that sorted list and displaying it in its native order.

This is all well and good for basic, non-internationalized applications, but seems to break down when dealing with resource bundles and keys. The problem, in my opinion, is that the controller should not have to worry about extracting messages from the resource bundle in order to properly sort, especially when most of the other logic for displaying resource bundle messages is already at the JSP level:

<fmt:message key="${my.label.key}">

Since the view already has a dependency on the resource bundle resolver, resolving those messages in the controller for the purpose of sorting feels wrong. The code I usually see shoe-horned into controllers is ugly, and in the worst cases, I've seen it even thrown in the service layer (*gasp*). It requires knowledge of the current locale, language, bundle, etc.

In an effort to maintain the purity of controllers everywhere, I propose some kind of "sort by resolved message" functionality in the view layer, and will present an example implementation below, as an adaptation of the JSTL c:forEach tag construct.

package com.abstractbits.web.taglib;

import java.util.Comparator;
import java.util.SortedMap;
import java.util.TreeMap;

import javax.servlet.jsp.JspTagException;
import javax.servlet.jsp.jstl.fmt.LocaleSupport;

import org.apache.commons.beanutils.BeanUtils;
import org.apache.taglibs.standard.tag.rt.core.ForEachTag;

/**
* Implementation of a loop tag construct which iterates over a given collection
* of items which have been sorted by the resolved message from a resource bundle
* for a given bundle key.
*
* http://abstractbits.blogspot.com
*
* @author Jared Stehler
*/
public class SortedForEachTag extends ForEachTag {

private enum SortOrder {asc, desc};

private String bundleKeySortProperty;

private SortOrder order = SortOrder.asc;

/**
* Property path notation to get at the bundle key in each object in the given collection
*
* @param bundleKeyProperty
*/
public void setBundleKeySortProperty(String bundleKeyProperty) {
this.bundleKeySortProperty = bundleKeyProperty;
}

/**
* The order in which to sort the bundle messages. {"asc", "desc"}
*
* @param sortOrder
*/
public void setSortOrder(String sortOrder) {
this.order = SortOrder.valueOf( sortOrder.toLowerCase() );
}

/* (non-Javadoc)
* @see javax.servlet.jsp.jstl.core.LoopTagSupport#prepare()
*/
@Override
protected void prepare() throws JspTagException {
super.prepare();

try {
SortedMap<String, Object> sortedValues = new TreeMap<String, Object>(new Comparator<String>() {
public int compare(String o1, String o2) {
int result = o1.compareTo(o2);
return SortedForEachTag.this.order == SortOrder.asc ? result : -result;
}
});

while( this.items.hasNext() ){
Object nextItem = this.items.next();

String bundleKey = (bundleKeySortProperty == null) ? (String) nextItem : BeanUtils.getProperty(nextItem, bundleKeySortProperty);
String resolvedMessage = LocaleSupport.getLocalizedMessage(this.pageContext, bundleKey);

sortedValues.put(resolvedMessage, nextItem);
}

this.items = toForEachIterator(sortedValues.values());

} catch( Exception e ){
throw new JspTagException("Error sorting items", e);
}
}
}

I extended the ForEachTag class from the JSTL standard implementation, and found that I only needed to override the prepare() method to implement my custom sorting. This custom tag uses the standard method of resolving resource messages from bundle keys, and then sorts them in ascending / descending fashion, depending on the order specified. If the bundleKeyProperty is not set, I assume the given collection is a list of Strings, each of which is a bundle key.

Here is the corresponding TLD definition snippet for this tag (I copied this from c.tld, found within the standard taglib implementation jar):

<taglib xmlns="http://java.sun.com/xml/ns/j2ee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-jsptaglibrary_2_0.xsd"
version="2.0">

<description>
JSTL extension tag library
</description>

<tlib-version>1.0</tlib-version>
<short-name>x</short-name>

<tag>
<description>
Adaptation of the classic iteration tag, supporting the notion of sorting based
on the resolved message values of the given objects' bundle keys.
</description>
<name>sortedForEach</name>
<tag-class>com.abstractbits.web.taglib.SortedForEachTag</tag-class>
<tei-class>org.apache.taglibs.standard.tei.ForEachTEI</tei-class>
<body-content>JSP</body-content>
<attribute>
<name>bundleKeySortProperty</name>
<required>false</required>
<rtexprvalue>true</rtexprvalue>
<type>boolean</type>
</attribute>
<attribute>
<name>sortOrder</name>
<required>false</required>
<rtexprvalue>true</rtexprvalue>
</attribute>
<attribute>
<description>
Collection of items to iterate over.
</description>
<name>items</name>
<required>false</required>
<rtexprvalue>true</rtexprvalue>
<type>java.lang.Object</type>
</attribute>
<attribute>
<description>
If items specified:
Iteration begins at the item located at the
specified index. First item of the collection has
index 0.
If items not specified:
Iteration begins with index set at the value
specified.
</description>
<name>begin</name>
<required>false</required>
<rtexprvalue>true</rtexprvalue>
<type>int</type>
</attribute>
<attribute>
<description>
If items specified:
Iteration ends at the item located at the
specified index (inclusive).
If items not specified:
Iteration ends when index reaches the value
specified.
</description>
<name>end</name>
<required>false</required>
<rtexprvalue>true</rtexprvalue>
<type>int</type>
</attribute>
<attribute>
<description>
Iteration will only process every step items of
the collection, starting with the first one.
</description>
<name>step</name>
<required>false</required>
<rtexprvalue>true</rtexprvalue>
<type>int</type>
</attribute>
<attribute>
<description>
Name of the exported scoped variable for the
current item of the iteration. This scoped
variable has nested visibility. Its type depends
on the object of the underlying collection.
</description>
<name>var</name>
<required>false</required>
<rtexprvalue>false</rtexprvalue>
</attribute>
<attribute>
<description>
Name of the exported scoped variable for the
status of the iteration. Object exported is of type
javax.servlet.jsp.jstl.core.LoopTagStatus. This scoped variable has nested
visibility.
</description>
<name>varStatus</name>
<required>false</required>
<rtexprvalue>false</rtexprvalue>
</attribute>
</tag>

</taglib>

That's all there is to it! I like this tag because it simplifies sorting resolved resource bundle messages, which is something I tend to need to do often, and would otherwise require an unnecessary dependency of the controller class on the resource bundle resolver. Controllers are cleaner when they only deal with bundle keys, in my opinion. I do feel like the name of the tag could be better; any suggestions? Does this already exist somewhere, and I've simply missed it?

Friday, August 14, 2009

Dynamic Classpaths

The JVM's classpath is both one of the more confusing and annoying aspects of Java. Issues always tend to come up when you are trying to make your application easily deployable on client machines, as you inevitably have to jump through some hoops with proper placement of jars, and perhaps some nifty shell scripts to tell the JVM where those jars live.

Because of a combination of the aforementioned reasons and idle curiosity, I set out to see if I could run my application entirely from a single jar, which contained its required library jars within itself. In researching this idea, I found a neat way of dynamically adding jars to your JVM's classpath at run-time.

*N.B. While I consider the code that follows to be 'cool', I don't necessarily consider this to be 'best-practice' for setting classpaths. I prefer putting all of my library jars into a single folder (e.g. './lib'), and bringing the folder onto the classpath using
'-Djava.library.dirs=./lib'.

There are two interesting features of the JVM I discovered that I want to showcase: finding out where you (the code) live in the real world (the file system), and adding jars to the classpath at runtime.

Where do I live?

Similar to Ruby's (and other scripty languages') __FILE__, you can find the actual path to your executing code in Java, as long as you are allowed to by the security / permissions manager. This info is stored in the CodeSource object, which you can get to like thus:

URI clientJarUri = DynamicClasspath.class.getProtectionDomain().getCodeSource().
getLocation().toURI();

The javadoc on the getProtectionDomain() method states: If there is a security manager installed, this method first calls the security manager's checkPermission method with a RuntimePermission("getProtectionDomain") permission to ensure it's ok to get the ProtectionDomain, so keep that in mind when attempting this on secured environments.


Where does my code live?

Once you have the location of your code jar, you can use that to find jars which are relative to it, and tell the JVM about them using the system's URLClassLoader. Classloaders in the JVM are hierarchical, with child instances delegating class requests to their parent if they can't find what they are looking for. The premise here is that we can get at that root classloader, and add to its search path. At first, it seems that the appropriate method to use (addURL()) is hidden to us, but upon further Reflection (ha ha), we have what we need to proceed:

Method addURL = URLClassLoader.class.getDeclaredMethod("addURL", new Class[] {URL.class});
addURL.setAccessible(true);

Now, we can merrily add as many URL references to jars as we desire:

ClassLoader classLoader = ClassLoader.getSystemClassLoader();
for( URL jarUrl : classpathUrlsToAdd ){
addURL.invoke(classLoader, jarUrl);
}

Putting it all together

We now have a simple mechanism for finding out where we are, and adding additional classpath references to the JVM at run-time. This could prove to be useful in certain scenarios, such as deploying apps in a self-contained jar file, as I have done. My build process added all of my library jars at the root of the 'uber-jar', and in the main-class of that jar (my 'bootstrap' class):

  1. Find my location on the file system
  2. Extract all of the jar files from within the currently executing client jar
  3. Add all of those jars to the classloader dynamically
  4. Invoke the actual main class of my app

The only annoying bit was that the bootstrap class doesn't have access to any of my library jars, and therefore I wasn't able to use my favorite library (commons-io) for extracting the jar files from the client jar :-(.



Friday, April 10, 2009

Remote Debugging JVM Applications

Recently, I just solved a tricky issue I was having with Canoo WebTest using the JVM's remote debugging capabilities, and thought it might be useful to write a brief how-to.

The JVM provides a useful feature called the Java Platform Debugger, or JPDA. When enabled via startup options, it provides a port through which debuggers (e.g. Eclipse) are able to connect and do their thing. This allows you to debug practically any application, such as Tomcat, or in my case, WebTest (running inside of an Ant process).

There are a bunch of available options for JPDA, but here is the basic incantation:

-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=8008
  • jdwp: Java Debug Wire Protocol
  • transport: indicates that debuggers should attach using sockets, as opposed to the other option, which is shared memory (dt_shmem)
  • server: when 'y', listens for a debugger application to attach (otherwise, attempt to connect to the specified debugging instance)
  • suspend: the JVM should wait for a debugger application to attach before starting up
  • address: the port to expose for debugger applications (or address to connect to in order to debug)

*Note: you may have seen a different set of arguments for starting a JVM in debug mode:
-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8008
This is required for Java versions < 5.

You can find a full list of options here.

Now that you have your application faithfully waiting for you to debug it ('Listening for transport dt_socket at address: 8008'), you just have to set up a remote debug configuration in Eclipse and run it:
  1. Run > Debug Configurations... > new Remote Java Application
  2. Connection Type: Standard (Socket Attach)
  3. Connection Properties: host and port of JDWP application
  4. Allow termination of remote VM: if selected, when you are finished with the debugger and click the terminate button in Eclipse, the remote application will exit also
That's it!

So back to my original issue. I had written a WebTest to upload images on a particular page, verify them and then click the save button, which makes use of an ajax validation mechanism to perform a server-side page validation before actually submitting the form. The problem was, my <clickButton/> wasn't doing anything. No errors, warnings, debug information of any kind in any log, simply nothing. I was already debugging the web application, so I knew that there wasn't any server-side activity holding it up, so I fired up WebTest in debug mode, downloaded the source (as well as htmlunit, which it is built on), and set some breakpoints (a few in ClickButton, as well as in htmlunit's XMLHttpRequest implementation). It turned out that XMLHttpRequest was throwing a ClassCastException, which was being silently swallowed, because my <verifyImages/> step had inadvertently changed the current response to UnexpectedPage, when HtmlPage was expected. Remove the verifyImages step, and voila, everything works!

Update: I forgot to mention that I created a JIRA issue for the verifyImages step: http://webtest-community.canoo.com/jira/browse/WT-516

Tuesday, February 24, 2009

The joys of FindBugs

Ladies and Gentlemen, I am writing to you today to speak on the joys of FindBugs. FindBugs (http://findbugs.sourceforge.net/, Eclipse update site http://findbugs.cs.umd.edu/eclipse), for the uninitiated, is a static code analysis tool, which is quite good at performing as its name might lead you to believe.

Similar to other static code analysis tools (e.g. PMD), it has a number of categories of various code issues, ranging from dodgy practices and bad form, to downright errors (null pointer deferences, anyone?). I wholeheartedly recommend running it on your code on a regular basis, and want to present below one example recently of where it helped me.

See anything wrong with the following?

new Thread( new Runnable(){
public void run() {
// long running code in a separate thread... or is it?
}
}).run();

This defect was sprinkled in various places throughout our 650+ class codebase. FindBugs quickly found the issue, providing the following helpful advice:

M M Ru] Invokes run on a thread (did you mean to start it instead?) [RU_INVOKE_RUN]

This method explicitly invokes run() on an object. In general, classes implement the Runnable interface because they are going to have their run() method invoked in a new thread, in which case Thread.start() is the right method to call.

Think of FindBugs as your fine-toothed comb; it instantly spotted a number of places in our application where we thought we were dumping long-running processes into separate threads, but in actualality were simply running them in the same.