Remote Debugging JVM Applications

Recently, I just solved a tricky issue I was having with Canoo WebTest using the JVM's remote debugging capabilities, and thought it might be useful to write a brief how-to.

The JVM provides a useful feature called the Java Platform Debugger, or JPDA. When enabled via startup options, it provides a port through which debuggers (e.g. Eclipse) are able to connect and do their thing. This allows you to debug practically any application, such as Tomcat, or in my case, WebTest (running inside of an Ant process).

There are a bunch of available options for JPDA, but here is the basic incantation:

-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=8008

• jdwp: Java Debug Wire Protocol
• transport: indicates that debuggers should attach using sockets, as opposed to the other option, which is shared memory (dt_shmem)
• server: when 'y', listens for a debugger application to attach (otherwise, attempt to connect to the specified debugging instance)
• suspend: the JVM should wait for a debugger application to attach before starting up
• address: the port to expose for debugger applications (or address to connect to in order to debug)

*Note: you may have seen a different set of arguments for starting a JVM in debug mode:

-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8008

This is required for Java versions < 5.

You can find a full list of options here.

Now that you have your application faithfully waiting for you to debug it ('Listening for transport dt_socket at address: 8008'), you just have to set up a remote debug configuration in Eclipse and run it:

1. Run > Debug Configurations... > new Remote Java Application
2. Connection Type: Standard (Socket Attach)
3. Connection Properties: host and port of JDWP application
4. Allow termination of remote VM: if selected, when you are finished with the debugger and click the terminate button in Eclipse, the remote application will exit also

That's it!

So back to my original issue. I had written a WebTest to upload images on a particular page, verify them and then click the save button, which makes use of an ajax validation mechanism to perform a server-side page validation before actually submitting the form. The problem was, my <clickButton/> wasn't doing anything. No errors, warnings, debug information of any kind in any log, simply nothing. I was already debugging the web application, so I knew that there wasn't any server-side activity holding it up, so I fired up WebTest in debug mode, downloaded the source (as well as htmlunit, which it is built on), and set some breakpoints (a few in ClickButton, as well as in htmlunit's XMLHttpRequest implementation). It turned out that XMLHttpRequest was throwing a ClassCastException, which was being silently swallowed, because my <verifyImages/> step had inadvertently changed the current response to UnexpectedPage, when HtmlPage was expected. Remove the verifyImages step, and voila, everything works!

Update: I forgot to mention that I created a JIRA issue for the verifyImages step: http://webtest-community.canoo.com/jira/browse/WT-516

The joys of FindBugs

Ladies and Gentlemen, I am writing to you today to speak on the joys of FindBugs. FindBugs (http://findbugs.sourceforge.net/, Eclipse update site http://findbugs.cs.umd.edu/eclipse), for the uninitiated, is a static code analysis tool, which is quite good at performing as its name might lead you to believe.

Similar to other static code analysis tools (e.g. PMD), it has a number of categories of various code issues, ranging from dodgy practices and bad form, to downright errors (null pointer deferences, anyone?). I wholeheartedly recommend running it on your code on a regular basis, and want to present below one example recently of where it helped me.

See anything wrong with the following?

new Thread( new Runnable(){

public void run() {

// long running code in a separate thread... or is it?

}

}).run();

This defect was sprinkled in various places throughout our 650+ class codebase. FindBugs quickly found the issue, providing the following helpful advice:

M M Ru] Invokes run on a thread (did you mean to start it instead?) [RU_INVOKE_RUN]

This method explicitly invokes run() on an object. In general, classes implement the Runnable interface because they are going to have their run() method invoked in a new thread, in which case Thread.start() is the right method to call.

Think of FindBugs as your fine-toothed comb; it instantly spotted a number of places in our application where we thought we were dumping long-running processes into separate threads, but in actualality were simply running them in the same.

Dom4j and XPath

In this, the latest installment of obscure gotchas in Java development, I'm going to discuss an interesting behavior of Dom4j, definitely something to beware of: when you use '/' or '//' to start an XPath search expression in conjunction with the instance method Node.select{Nodes | SingleNode}, the search does not start at that node! In fact, it will always start at the actual document root, contrary to what one may expect from looking at the code / API.

Allow me to illustrate with an example. Lets say you are working with this simplified XML document:

 <Account>
   <Owner>
     <ContactInfo>
       <Name>Tom Jones</Name>
       ...
     </ContactInfo>
     ...
   </Owner>
   <Cosigner>
     <ContactInfo>
       <Name>Jim Johnson</Name>
       ...
     </ContactInfo>
     ...
   </Cosigner>
 </Account>

Dom4j makes it easy to find the Cosigner node:

Node cosignerNode = document.selectSingleNode("/Account/Cosigner");

and at first glance, I thought the following code snippet would return the Cosigner's name:

cosignerNode.selectSingleNode("//ContactInfo/Name") => "Tom Jones"

Counter intuitively, this code returns 'Tom Jones'. This is because when you start an XPath query with '/' or '//', Dom4j will traverse the DOM back up to the root node to begin its search. By removing the leading slashes, it works as expected:

cosignerNode.selectSingleNode("ContactInfo/Name") => "Jim Johnson"

So in conclusion, be careful whenever you use selectSingleNode; make sure that you understand that whenever you use // relative XPath queries, the result will come from the root of the entire document, and will not be limited to children of the node on which you invoke it.

Fun with JDBC CallableStatements

I spent the better part of a day debugging an issue involving stored procedure calls over Spring JDBC, and came across some interesting gotchas which I felt might help others save some time in the future, so I thought I'd write a quick article about it.

Basically, I was given a defect where decimal values from the database were being rounded somewhere in my service method, and was able to isolate it to within my StoredProcedureImpl class. In Spring JDBC, a common approach to calling a stored procedure is to extend StoredProcedure, wherein you declare the input and output parameters and their corresponding types:

    private class MyStoredProcedure extends StoredProcedure {        
        private static final String SQL = "sysdate";

        public MyStoredProcedure(DataSource ds) {
            setDataSource(ds);
            setFunction(true);
            setSql(SQL);
            declareParameter(new SqlOutParameter("date", Types.DATE));
            compile();
        }

        public Map execute() {
            return execute(new HashMap());
        }
    }

This works well, and is indeed an elegant way of calling a stored procedure, as it eliminates all of the JDBC nastiness (try / catch / finally (try / finally / etc / etc) / etc). The problem lies in the (obscure?) fact that when you specify DECIMAL or NUMERIC types for input or output parameters in the JDBC API, you are expected to also specify a scale, or it will default to 0 (and either truncate or round, depending on your database driver).

From the Java API for CallableStatement:

void registerOutParameter(int parameterIndex,
                          int sqlType,
                          int scale)
                          throws SQLException

    Registers the parameter in ordinal position parameterIndex 
to be of JDBC type sqlType. All OUT parameters must be 
registered before a stored procedure is executed.

    The JDBC type specified by sqlType for an OUT parameter 
determines the Java type that must be used in the get method 
to read the value of that parameter.

    This version of registerOutParameter should be used 
when the parameter is of JDBC type NUMERIC or DECIMAL.

    Parameters:
        parameterIndex - the first parameter is 1, the second 
        is 2, and so on
        sqlType - the SQL type code defined by java.sql.Types.
        scale - the desired number of digits to the right 
           of the decimal point. It must be greater than or 
           equal to zero. 

As it turns out, Spring JDBC's SqlParameter class didn't support setting scale on them until 2.0.5 (and here), and SqlOutParameter didn't have this in a final release until 2.5.

So, to make a long story short (too late, I know), if you are calling stored procedures in JDBC which use DECIMAL or NUMERIC parameters in either the input or output, you must specify a scale to use, and if you are currently running a non-current version of Spring (less than 2.5.1), this is a great reason to upgrade.

P.S. When researching this issue, I noticed a new set of classes in the Spring JDBC set (SimpleJdbc) which looked cool; they may warrant an article later on...

Be wary of SimpleDateFormat

Java's SimpleDateFormat class provides a convenient method of parsing arbitrary strings into Date objects, and formatting Dates back into strings. However, like the rest of the standard Date/Time classes in the core API, there are a couple of important things you need to be aware of when working with this class. Ignore these caveats and there is a good chance you will be spending an inordinate amount of time debugging obscure issues with dates!

Can you guess the output of the following program?

import java.text.SimpleDateFormat;
import java.util.Date;

public class DateTest {

public static void main(String[] args) throws Exception {
String fmt = "yyyyMMdd";

String testDate = "20080530";

Date dt = (new SimpleDateFormat(fmt)).parse(testDate);

System.out.println(dt.toString());

testDate = "2008-05-30";

dt = (new SimpleDateFormat(fmt)).parse(testDate);
System.out.println(dt.toString());
}
}

Believe it or not, it is:

Fri May 30 00:00:00 EDT 2008
Wed Dec 05 00:00:00 EST 2007

SimpleDateFormat will not throw an error if it receives string input which does not conform to its specified format string; it will instead silently construct an incorrect date! One possible solution (although I have seen cases where this also will not work) is:

DateFormat df = new SimpleDateFormat(fmt);
df.setLenient(false);
System.out.println(df.parse(testDate));

Resulting in:

Exception in thread "main" java.text.ParseException: Unparseable date: "2008-05-30"
at java.text.DateFormat.parse(Unknown Source)
at sandbox.DateTest.main(DateTest.java:23)

This should serve as a good example as any of the importance of good unit tests!

Lastly, it is also important to know that SimpleDateFormat is not thread-safe! This means that you must not use it as a member variable of a multithreaded service class, for example (Servlet, MessageDrivenBean, etc).

Since it can be expensive to keep instantiating a new SimpleDateFormat on each service request, a good practice is to store an instance in a ThreadLocal variable:

private ThreadLocal<DateFormat> myDateFormat = new ThreadLocal<DateFormat>(){
       @Override protected DateFormat initialValue() {
           return new SimpleDateFormat("yyyyMMdd");
       }
   };