Friday, August 20, 2010

Fixing a ConcurrentModificationException

Question:
The following code throws a ConcurrentModificationException. What additional code can you add between the <FIXME>...</FIXME> tags in order to prevent this exception from being thrown?
final List<String> list = new ArrayList<String>();
list.add("HELLO");
final Iterator<String> iter = list.iterator();
System.out.println(iter.next());
list.add("WORLD");
//<FIXME>

//</FIXME>
System.out.println(iter.next());
Solution:
In this example, a ConcurrentModificationException is thrown because the Iterator detects that the list over which it is iterating has been changed. If you look into the source code for these classes you will find that when an Iterator is created, it contains an int variable called expectedModCount which is initialised to the modCount of the backing list. Whenever the backing list is structurally modified (with an add or remove operation, for example) then the modCount is incremented. As a result, the iterator's expectedModCount no longer matches the list's modCount and the iterator throws a ConcurrentModificationException.

In order to prevent this exception from being thrown, we need to bring the expectedModCount of the iterator and the modCount of the list back in line with each other. Here are a couple of ways this can be done:

1. Reflection:
Reflection is the easiest way to change the internal counters of the iterator and list. In the fix below, I have set the expectedModCount of the iterator to the same value as the modCount of the list. The code no longer throws the ConcurrentModificationException.

final List<String> list = new ArrayList<String>();
list.add("HELLO");
final Iterator<String> iter = list.iterator();
System.out.println(iter.next());
list.add("WORLD");
//<FIXME>
/* Using Reflection */
try{
  //get the modCount of the List
  Class cls = Class.forName("java.util.AbstractList");
  Field f = cls.getDeclaredField("modCount");
  f.setAccessible(true);
  int modCount = f.getInt(list);

  //change the expectedModCount of the iterator
  //to match the modCount of the list
  cls = iter.getClass();
  f = cls.getDeclaredField("expectedModCount");
  f.setAccessible(true);
  f.setInt(iter, modCount);
}
catch(ClassNotFoundException e){
  e.printStackTrace();
}
catch(NoSuchFieldException e){
  e.printStackTrace();
}
catch(IllegalAccessException e){
  e.printStackTrace();
}
//</FIXME>
System.out.println(iter.next());
2. Integer Overflow:
Another approach is to keep modifying the list until the integer modCount overflows and reaches the same value as expectedModCount. At the moment, modCount=2 and expectedModCount=1. In the fix below, I repeatedly change the list (by calling trimToSize), forcing modCount to overflow and reach expectedModCount. This code took 38s to run on my machine.
final List<String> list = new ArrayList<String>();
list.add("HELLO");
final Iterator<String> iter = list.iterator();
System.out.println(iter.next());
list.add("WORLD");
//<FIXME>
for(int i = Integer.MIN_VALUE ; i < Integer.MAX_VALUE ; i++){
  ((ArrayList)list).trimToSize();
}
//</FIXME>
System.out.println(iter.next());

Sunday, August 15, 2010

DateFormat with Multiple Threads

The DateFormat class is not thread-safe. The javadocs state that "Date formats are not synchronized. It is recommended to create separate format instances for each thread. If multiple threads access a format concurrently, it must be synchronized externally."

The following code shows how you would typically use DateFormat to convert a String to a Date in a single-threaded environment. It is more efficient to get the format as an instance variable and use it multiple times so that the system doesn't have to fetch the information about the local language and country conventions multiple times.

public class DateFormatTest {

  private final DateFormat format =
            new SimpleDateFormat("yyyyMMdd");

  public Date convert(String source)
                      throws ParseException{
    Date d = format.parse(source);
    return d;
  }
}
This code is not thread-safe. We can test it out by invoking the method using multiple threads. In the calling code below, I create a thread pool with 2 threads and submit 5 date conversion tasks to it. I then examine the results.
final DateFormatTest t = new DateFormatTest();
Callable<Date> task = new Callable<Date>(){
    public Date call() throws Exception {
        return t.convert("20100811");
    }
};

//lets try 2 threads only
ExecutorService exec = Executors.newFixedThreadPool(2);
List<Future<Date>> results =
             new ArrayList<Future<Date>>();

//perform 5 date conversions
for(int i = 0 ; i < 5 ; i++){
    results.add(exec.submit(task));
}
exec.shutdown();

//look at the results
for(Future<Date> result : results){
    System.out.println(result.get());
}
When the code is run the output is unpredictable - sometimes it prints out the correct dates, sometimes the WRONG ones (e.g. Sat Jul 31 00:00:00 BST 2012!) and at other times it throws a NumberFormatException!

How can you use DateFormat concurrently?
There are different approaches you can take to use DateFormat in a thread-safe manner:

1. Synchronization
The easiest way of making this code thread-safe is to obtain a lock on the DateFormat object before parsing the date string. This way only one thread can access the object at a time, and the other threads must wait.

public Date convert(String source)
                    throws ParseException{
  synchronized (format) {
    Date d = format.parse(source);
    return d;
  }
}

2. ThreadLocals
Another approach is to use a ThreadLocal variable to hold the DateFormat object, which means that each thread will have its own copy and doesn't need to wait for other threads to release it. This is generally more efficient than synchronising sa in the previous approach.

public class DateFormatTest {

  private static final ThreadLocal<DateFormat> df
                 = new ThreadLocal<DateFormat>(){
    @Override
    protected DateFormat initialValue() {
        return new SimpleDateFormat("yyyyMMdd");
    }
  };

  public Date convert(String source)
                     throws ParseException{
    Date d = df.get().parse(source);
    return d;
  }
}

3. Joda-Time
Joda-Time is a great, open-source alternative to the JDK's Date and Calendar API. It's DateTimeFormat is "thread-safe and immutable".

import org.joda.time.DateTime;
import org.joda.time.format.DateTimeFormat;
import org.joda.time.format.DateTimeFormatter;
import java.util.Date;

public class DateFormatTest {

  private final DateTimeFormatter fmt =
       DateTimeFormat.forPattern("yyyyMMdd");

  public Date convert(String source){
    DateTime d = fmt.parseDateTime(source);
    return d.toDate();
  }
}

Saturday, August 14, 2010

Using Compressed JMS Messages

If you are publishing large XML messages onto a JMS topic or queue, compression will give you much better performance because less data is sent over the network. Also, your JMS server can hold more messages and there is less risk of running out of memory. XML messages are great candidates for compression, due to the repetitive nature of XML.

Compressing JMS messages
The following code shows how you can create a compressed BytesMessage and publish it onto a topic:

InputStream in = null;
GZIPOutputStream out = null;
try {
  ByteArrayOutputStream bos = new
                            ByteArrayOutputStream(1024 * 64);
  out = new GZIPOutputStream(bos);

  String filename = "input.xml";
  in = new BufferedInputStream(new FileInputStream(filename));

  byte[] buf = new byte[1024 * 4];
  int len;
  while ((len = in.read(buf)) > 0) {
      out.write(buf, 0, len);
  }
  out.finish();

  //publish it
  BytesMessage msg = session.createBytesMessage();
  msg.writeBytes(bos.toByteArray());
  publisher.publish(msg);
}
catch (IOException e) {
  e.printStackTrace();
}
finally {
  if (in != null) {
    try {
        in.close();
    }
    catch (IOException ignore) {
    }
  }
  if (out != null) {
    try {
        out.close();
    }
    catch (IOException ignore) {
    }
  }
}
Decompressing JMS messages
The following code shows how you can decompress a JMS BytesMessage when your subscriber receives it and write it to file:
if (mesg instanceof BytesMessage) {
 final BytesMessage bMesg = (BytesMessage) mesg;

 byte[] sourceBytes;
 try {
    sourceBytes = new byte[(int) bMesg.getBodyLength()];
    bMesg.readBytes(sourceBytes);
    System.out.println("Read " + sourceBytes.length + " bytes");
 }
 catch (JMSException e1) {
    throw new RuntimeException(e1);
 }
 GZIPInputStream in = null;
 OutputStream out = null;
 try {
    in = new GZIPInputStream(
         new ByteArrayInputStream(sourceBytes));
    String filename = "message.xml";
    out = new FileOutputStream(filename);
    byte[] buf = new byte[1024 * 4];
    int len;
    while ((len = in.read(buf)) > 0) {
        out.write(buf, 0, len);
    }
    System.out.println("Wrote to " + filename);
 }
 catch (IOException e) {
    e.printStackTrace();
 }
 finally {
    if (in != null)
        try {
            in.close();
        }
        catch (IOException ignore) {
        }
    if (out != null)
        try {
            out.close();
        }
        catch (IOException ignore) {
        }
 }
}

Thursday, August 12, 2010

Java Monitoring Tools

JDK 6.0 comes bundled with a number of handy, but often overlooked, utilities to monitor, manage and troubleshoot Java applications. They can be found in the bin directory of your installation. Here are a few of them explained:

1) jps - report java process status
This command prints information about active java processes running on a given machine. The output contains the JVMID, the name of the main class and any arguments passed to it or the JVM.

sharfah@starship:~> jps -lmv
3936 sun.tools.jps.Jps -lmv -Xms8m
5184 test.TestClient -Xmx1024m

2) jstat - statistics monitoring tool
This command allows you to monitor memory spaces of a JVM. For example, using the "-gcutil" option will show you the utilisation of the eden (E), survivor (S0, S1), old (O) and permanent (P) generations and how long the minor and major GCs are taking. You can gather these statistics continuously by specifying a sampling interval and how many samples you wish to take.

sharfah@starship:~> jstat -gcutil 5184 1s 5
  S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT
  0.00   0.00   3.04  85.47  11.68      5    0.009     1    0.023    0.032
  0.00   0.00   3.04  85.47  11.68      5    0.009     1    0.023    0.032
  0.00   0.00   3.04  85.47  11.68      5    0.009     1    0.023    0.032
  0.00   0.00   3.04  85.47  11.68      5    0.009     1    0.023    0.032
  0.00   0.00   3.04  85.47  11.68      5    0.009     1    0.023    0.032

3) jstack - stack trace
Prints out a complete thread dump of your application (just like "kill -3"). Useful for investigating what your application is doing and identifying deadlocks.

4) jmap - memory map
Use this command to print a histogram of the heap to show you the number of instances of each java class and how much memory they are occupying. You can also dump the heap to a file in binary format and then load it into Eclipse Memory Analyser as I described here.

sharfah@starship:~> jmap -histo 5184
sharfah@starship:~> jmap -dump:format=b,file=heap.bin 5184

5) jhat - heap analysis tool
This command reads a binary heap file (produced by the jmap command, for example). It launches a local webserver so that you can browse the heap using a web browser. The cool thing is being able to execute your own queries using Object Query Language (OQL) on the heap dump.

sharfah@starship:~> jhat heap.bin

6) jinfo - configuration info
Prints out java system properties (like the classpath and library path) and JVM command line flags. Doesn't work on Windows though! Also allows you to enable/disable/change VM flags.

sharfah@starship:~> jinfo -flag PrintGCDetails  4648
-XX:-PrintGCDetails
sharfah@starship:~> jinfo -flag +PrintGCDetails 4648

Friday, July 02, 2010

Brain Teaser: Find the ages

Question:
I was visiting a friend one evening and remembered that he had three daughters. I asked him how old they were. "The product of their ages is 72," he answered. Quizzically, I asked, "Is there anything else you can tell me?" "Yes," he replied, "the sum of their ages is equal to the number of my house." I stepped outside to see what the house number was. Upon returning inside, I said to my host, "I'm sorry, but I still can't figure out their ages." He responded apologetically, "I'm sorry, I forgot to mention that my oldest daughter likes strawberry shortcake." With this information, I was able to determine all of their ages. How old is each daughter?

Answer:
The first clue is that the product of their ages is 72. Therefore, the list of all possible sets of numbers whose product is 72 is:

72 1 1
36 2 1
24 3 1
18 4 1
18 2 2
12 3 2
12 6 1
9 8 1
9 4 2
8 3 3
6 6 2
6 4 3
The second clue involves the sums of their ages which are:
72 + 1 + 1 = 74
36 + 2 + 1 = 39
24 + 3 + 1 = 28
18 + 4 + 1 = 23
18 + 2 + 2 = 22
12 + 3 + 2 = 17
12 + 6 + 1 = 19
9 + 8 + 1 = 18
9 + 4 + 2 = 15
8 + 3 + 3 = 14
6 + 6 + 2 = 14
6 + 4 + 3 = 13
Out of all these sums, two of them are equal:

8 + 3 + 3 = 14
6 + 6 + 2 = 14
The final clue is that his "oldest" daughter likes cake, which means that we can eliminate the second one of the above. Therefore, his daughter's ages are:

8 + 3 + 3 = 14
More Interview Posts