OutOfMemoryError: PermGen space

Recently, I found a problem with our system in which it ran out of PermGen space under heavy usage. The exception from the logs is:

java.lang.OutOfMemoryError: PermGen space
        at java.lang.String.intern(Native Method)

PermGen Space
The memory a jvm uses is split up into three "generations": young (eden), tenured, and permanent. PermGen refers to the permanent generation which holds meta-data describing user classes and "interned" strings. If you have a large code base and intern lots of strings, more permgen space is used.

Interning Strings
You can intern strings using String.intern() which, from the javadocs:

Returns a canonical representation for the string object.
A pool of strings, initially empty, is maintained privately by the class String.
When the intern method is invoked, if the pool already contains a string equal to this String object as determined by the equals(Object) method, then the string from the pool is returned. Otherwise, this String object is added to the pool and a reference to this String object is returned.

Interned strings are stored in permgen space, so if you interned all your strings, you would eventually run out of memory. In our application, I found that we were interning string representations of timestamps which wasn't necessary! You should only intern strings which are used frequently.

Increasing PermGen Space
The maximum size is typically 64MB. You can double it by starting your JVM with the following system properties:
-XX:MaxPermSize=128m -XX:PermSize=128m

Using JConsole to Monitor Memory
An easy way to see how much memory your JVM is using is to use jconsole. First, you need to enable monitoring on your JVM by starting it up with the following system properties:
-Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=<port>
Next start jconsole and point it at this JVM using the host and port (or pid). Select the Memory tab and then choose the Memory Pool "PS Perm Gen" chart. You should now see the usage of your PermGen space as shown in the screenshot below:

XP Uptime Record

My computer has been up for 151 days!

This is the XP machine I use every day at work and even log on from home, after work and on weekends. I'm surprised to see that it hasn't needed a reboot for so long.

Sadly, there's a powerdown in the building tonight...

Finding Uptime on XP
Use the systeminfo command and the look for "System Up Time"

C:\> systeminfo | find "System Up"
  System Up Time: 151 Days, 9 Hours, 45 Minutes, 8 Seconds

Finding Uptime on Solaris or Linux
Use the uptime command:

sharfah@starship:~> uptime
  5:46pm  up 2 day(s), 7:30, 1 user, load average: 3.79, 5.55, 5.41

You can also find out when your system booted up, by using who -b:

sharfah@starship:~> who -b
 system boot Dec 10 11:16

UNIX Fork Bomb :(){ :|:& };:

Whatever you do, please do NOT enter this code in your Unix terminal:

:(){ :|:& };:

This is one of the most elegant examples of a fork bomb, which works by creating a large number of processes very quickly in order to saturate the operating system's process table. Each process uses up CPU time and memory and so the system becomes unresponsive and is quickly brought to its knees! This is a form of denial-of-service attack.

So how does this command work? It might be easier to understand if I re-wrote it like this instead:

bomb()
{
   bomb | bomb &
}
bomb

You declare a function called bomb which calls itself recursively and pipes the output to another call of itself. The & puts the function call in the background so that the new child processes can never die. The semi-colon marks the end of the function and the final "bomb" launches the attack (by calling the function the first time).

You can only stop a fork bomb by destroying all instances of it. It is a difficult task to use another program to kill it because that would mean creating another process which the system may not have enough space for. The only guaranteed way of curing a fork bomb is to reboot.

Windows Fork bomb Example

:s
start %0
%0|%0
goto :s

Eclipse Code Templates

This post is out-of-date! For the latest version, go here: "Useful Eclipse Templates for Faster Coding".

In this post, I will share some of my Eclipse Java Code Templates.

Templates are simply "magic words" or shortcuts to standard blocks of code or text. They are very handy because once you have them setup you don't have to waste time writing boilerplate code any more! An example of a pre-defined template in Eclipse is sysout which expands to System.out.println();. All you have to do is type sysout followed by Ctrl+Space to insert the statement in your java source file.

To see what templates are defined in Eclipse:

• Open your Preferences dialog by going to Windows > Preferences
• On the navigation tree on the left, go to Java > Editor > Templates
• You will see a list of pre-defined templates
• You can add new ones by pressing the "New..." button

Here is a list of templates I have created:

1. logger

Name: logger
Context: Java statements
Description: create new Logger
Pattern:

${:import(org.apache.log4j.Logger)}
private static final Logger logger =
       Logger.getLogger(${enclosing_type}.class);

2. logd

Name: logd
Context: Java statements
Description: logger debug
Pattern:

if(logger.isDebugEnabled())
     logger.debug(${word_selection}${});${cursor}

3. logi

Name: logi
Context: Java statements
Description: logger info
Pattern:

logger.info(${word_selection}${});${cursor}

4. logerr

Name: logerr
Context: Java statements
Description: logger error
Pattern:

logger.error(${errorMessage}, ${e});

5. readfile

Name: readfile
Context: Java
Description: read text from file
Pattern:

${:import(java.io.BufferedReader,
          java.io.FileNotFoundException,
          java.io.FileReader,
          java.io.IOException)}
BufferedReader in = null;
try {
   in = new BufferedReader(new FileReader(${fileName}));
   String line;
   while ((line = in.readLine()) != null) {
      ${process}
   }
}
catch (FileNotFoundException e) {
   logger.error(e) ;
}
catch (IOException e) {
   logger.error(e) ;
} finally {
   if(in != null) in.close();
}
${cursor}

6. for (iterate over map)

Name: for
Context: Java statements
Description: iterate over map
Pattern:

for(Map.Entry<${key:argType(map,0)},${value:argType(map,1)}> entry :
                    ${map:var(java.util.Map)}.entrySet()) {
    ${key} key = entry.getKey();
    ${value} value = entry.getValue();
    ${cursor}
}

Do YOU have any useful templates? If so, share them in the comments!

Extract a tar.gz File in a Single Command [Unix]

The obvious way to extract a tar.gz archive is using the following two commands:

sharfah@starship:~> gunzip foo.tar.gz
sharfah@starship:~> tar xvf foo.tar

This is how you can achieve the same thing, but in a single command:

sharfah@starship:~> gunzip -c foo.tar.gz | tar xvf -

or alternatively:

sharfah@starship:~> gunzip < foo.tar.gz | tar xvf -

Both commands write the data produced from the gunzip command to standard out (-c in the first example and using < in the second). The output is then piped into the tar command. The "-" represents standard input.

Creating a tar.gz:
A tar.gz file is normally created using two commands as follows:

sharfah@starship:~> tar cvf foo.tar foodir
sharfah@starship:~> gzip foo.tar

Again, there is a single command to do the same thing:

sharfah@starship:~> tar cvf - foodir | gzip > foo.tar.gz

Checking CPU Utilisation on Linux

Here are a few commands which can be used to investigate CPU utilisation on Linux:

top
The top program provides a dynamic real-time view of a running system. It can display system summary information as well as a list of tasks currently being managed by the Linux kernel. The CPU usage shows the task's share of the elapsed CPU time since the last screen update, expressed as a percentage of total CPU time.

top - 18:12:28 up 40 days, 18:35,  1 user,  load average: 0.13, 0.03, 0.01
Tasks:  98 total,   1 running,  96 sleeping,   0 stopped,   1 zombie
Cpu(s):  0.2% us,  0.0% sy,  0.0% ni, 99.4% id,  0.4% wa,  0.0% hi,  0.0% si
Mem:   8002512k total,  3845332k used,  4157180k free,    64624k buffers
Swap:  9437144k total,   771048k used,  8666096k free,  1831288k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    1 root      16   0   640   80   48 S  0.0  0.0  43:04.18 init
    2 root      RT   0     0    0    0 S  0.0  0.0   0:00.70 migration/0
    3 root      34  19     0    0    0 S  0.0  0.0   0:00.02 ksoftirqd/0
    4 root      RT   0     0    0    0 S  0.0  0.0   0:00.58 migration/1
    5 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/1
    6 root      RT   0     0    0    0 S  0.0  0.0   0:00.40 migration/2
    7 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/2
    8 root      RT   0     0    0    0 S  0.0  0.0   0:00.46 migration/3

sar
The sar command can be used to display a history of CPU usage:

10:00:01   CPU     %user     %nice   %system   %iowait   %idle
10:10:01   all     34.45      2.04     30.03      0.05   33.43
10:20:01   all     34.13      1.77     29.85      0.08   34.17

mpstat
The mpstat command can be used to show the percentage of CPU usage for each processor:

> mpstat -P ALL

18:02:42     CPU   %user   %nice %system %iowait    %irq
18:02:42     all   24.77   17.51   21.56    1.19    0.01
18:02:42       0   25.00   17.92   20.27    0.94    0.00
18:02:42       1   23.81   17.03   20.87    0.88    0.00
18:02:42       2   26.28   16.44   22.23    1.54    0.01
18:02:42       3   24.00   18.65   22.86    1.39    0.01

Order ps output
The following command displays the top 10 CPU users on your system. It involves listing the processes using ps and then sorting them by CPU usage:

> ps -eo pcpu,pid,user,cmd | sort -k 1 -r | head -10

%CPU   PID USER     CMD
 0.1    13 root     [events/3]
 0.1   103 root     [kswapd0]
 0.1   102 root     [kswapd1]
 0.0     9 root     [ksoftirqd/3]
 0.0     8 root     [migration/3]
 0.0  8982 root     [lockd]
 0.0  8981 root     [rpciod]
 0.0     7 root     [ksoftirqd/2]
 0.0    75 root     [kblockd/3]

Find Your Version of glibc

This is how you can find the version of glibc installed on your Linux machine:

sharfah@starship:~> rpm -q glibc
glibc-2.3.3-98.73

Howto: Sign Automated Emails [Unix]

This summary is not available. Please click here to view the post.

How to Find Your Solaris Version

If you are developing an application which runs on Solaris, you may need to know which version of Solaris it is, so that you can use the correct JRE build, Sybase libraries etc.

You can do this using the uname command which prints out useful system information. In order to get just the version number (release level) use the -r flag:

sharfah@starship:~> uname -r
5.10

Example:

OS=`uname`
OS_VERSION=`uname -r`

if [ "$OS" = "SunOS" ]
then
 JAVA=/utils/solaris/java_${OS_VERSION}/bin
elif [ "$OS" = "Linux" ]
then
 JAVA=/utils/linux/java/bin
else
 echo "Unknown operating system $OS"
fi

If you want more details on your Solaris version, look at /etc/release.

sharfah@starship:~> cat /etc/release
                        Solaris 10 8/07 s10x_u4wos_12b X86
           Copyright 2007 Sun Microsystems, Inc.  All Rights Reserved.
                        Use is subject to license terms.
                            Assembled 16 August 2007

Eclipse Shirt Is Here!

FedEx dropped off the much anticipated Eclipse t-shirt today and I must say, it looks fab! I got it as a result of taking part in the Ganymede Around the World Contest. My coder friends are going to be so jealous!

I've posted a few pictures of the t-shirt here. Enjoy!

Related post:
Here is my review of Ganymede, for those who missed it: Eclipse Ganymede Review

Subtract Two Dates [Unix]

In this post, I will describe how you can subtract two dates (which are in the form yyyyMMdd) to get the time between them. My approach involves first parsing both dates in order to extract the year, month and day, converting them to seconds using floating point math and finally subtracting one from the other.

Bash doesn't support floating point arithmetic, but I won't let that stop me. I will use awk instead. Another possible candidate is bc.

1. Parsing the date
We need to extract the year, month and day from our date which is in yyyyMMdd e.g. 20080818. The year is composed of the first four characters, the month is the next two and the day, the final two. We can do this using awk's substr function as shown below. substr(s,m,n) returns the n-character substring of s that begins at position m.

today=20080818
echo $today | awk '{
 year=substr($1,1,4);
 month=substr($1,5,2);
 day=substr($1,7,2);
 }'

If you don't like awk, you can also get substrings using the shell's expr command:

today=20080818
year=`/usr/ucb/expr substr $today 1 4`
month=`/usr/ucb/expr substr $today 5 2`
day=`/usr/ucb/expr substr $today 7 2`

2. Converting to seconds
We will use the formula below to give us the number of days since 1/1/1970:
(year-1970)*365.25 + month*30.5 + day

Now convert the days to seconds by multiplying the days by 24 * 60 * 60:
((year-1970)*365.25 + month*30.5 + day) * 24 * 60 * 60

3. Subtracting the times
Once we have both dates in seconds, we can subtract them. Since we're still dealing with floating point we should use awk or bc for precision.

echo $seconds1 $seconds2 | awk '{print $1 - $2}'

or

echo $seconds1 - $seconds2 | bc

Putting it all together
Here is the complete shell script which takes two dates and returns the number of seconds between them:

#!/usr/bin/bash

date1=$1
date2=$2

#give the dates to awk
echo $date1 $date2 | awk '{

#parse
year1=substr($1,1,4);
month1=substr($1,5,2);
day1=substr($1,7,2);

year2=substr($2,1,4);
month2=substr($2,5,2);
day2=substr($2,7,2);

#get seconds
secs1=((year1 - 1970)*365.25+(month1*30.5)+day1)*24*60*60;
secs2=((year2 - 1970)*365.25+(month2*30.5)+day2)*24*60*60;

#subtract
print secs1 - secs2;
}'

As you can see, all of the computation is in awk! So we could put all of our awk code into a separate file called subtractDates.awk, for instance, and run it on the command line like this: echo 20080830 20080818 | awk -f subtractDates.awk

Now that we have the number of seconds between two dates, we can convert that to days, months and years using a similar approach if we have to.

fahdshariff on Twitter

I think it's time to give Twitter a try and see what all the fuss is about. You can follow me if you like (though I don't think it will be very interesting or anything): fahdshariff on Twitter.

However, if I find that it's useless or that I'm wasting too much time with it, I'm outta there! I'll report back and let you know what I think of it.

Do you love or hate Twitter? Any tips? Share them in the comments!

Check if Process is Running [Unix]

If you have a process id (pid), how can you tell if it is still running?

One obvious way to do this is to parse the output of the ps command. For example:

sharfah@starship:~> ps -ef | grep " 1234 " | grep -v grep
sharfah  1234     1  0 10:36:06 pts/65   1:20 java -server

However, a much neater way to do this is to use the kill command to send signal 0 to the process. So kill -0 will not terminate the process and the return status can be used to determine if the process is running. 0 if it is, non-zero otherwise.

sharfah@starship:~> kill -0 1234
sharfah@starship:~> echo $?
0

sharfah@starship:~> kill -0 1234x
bash: kill: 1234x: no such pid
sharfah@starship:~> echo $?
1

For those interested, the kill -l command lists signal numbers and names, but 0 is not listed.

sharfah@starship:~> kill -l
 1) SIGHUP     2) SIGINT      3) SIGQUIT   4) SIGILL
 5) SIGTRAP    6) SIGABRT     7) SIGEMT    8) SIGFPE
 9) SIGKILL   10) SIGBUS     11) SIGSEGV  12) SIGSYS
13) SIGPIPE   14) SIGALRM    15) SIGTERM  16) SIGUSR1
17) SIGUSR2   18) SIGCHLD    19) SIGPWR   20) SIGWINCH
21) SIGURG    22) SIGIO      23) SIGSTOP  24) SIGTSTP
25) SIGCONT   26) SIGTTIN    27) SIGTTOU  28) SIGVTALRM
29) SIGPROF   30) SIGXCPU    31) SIGXFSZ  32) SIGWAITING
33) SIGLWP    34) SIGFREEZE  35) SIGTHAW  36) SIGCANCEL
37) SIGLOST

Organise RSS Feeds in Google Reader

My list of RSS feeds is constantly growing. Everyday, I discover new and interesting sites and blogs which I subscribe to. They vary in content and I read them based on how important they are, their content and what mood I am in. At the time of writing this post, I have nearly 70 feeds in Google Reader, distributed haphazardly across three default folders: Fun, Geeky and News. There is also a bunch lying in the root folder.

It's time to clean up and get organised as I am spending too much time sifting through junk which I don't want to read right now. I need to get rid of obsolete feeds and create a better folder structure for those I want to keep. Using Google Reader to create folders and drag my feeds in is going to take forever and probably a thousand mouse clicks.

I am going to try an easier way. I will export my feeds from Google Reader into an OPML file and then edit that using my favourite text editor (TextPad) in order to create the structure I want.

This is how:

1. Go to your Google Reader Settings page and click on the Import/Export tab
2. Click on the Export your subscriptions as an OPML file link and save the google-reader-subscriptions.xml file to your computer
3. Open the xml file in TextPad (or any another text editor) and the format will look pretty obvious.
4. After reorganising your feeds within the OPML file, go back to your Google Reader Import page and Upload the modified OPML file

The OPML file contains nested outline elements which represent folders containing RSS feeds.

Here is an excerpt from my newly formatted OPML file, which shows different folders (Fun, Programming, Low Priority) and the feeds within:
<outline title="Fun" text="Fun"> <outline text="Geek And Poke" title="Geek And Poke" type="rss" xmlUrl="http://feeds.feedburner.com/GeekAndPoke" htmlUrl="http://geekandpoke.typepad.com/geekandpoke/" /> <outline text="xkcd.com" title="xkcd.com" type="rss" xmlUrl="http://www.xkcd.com/rss.xml" htmlUrl="http://xkcd.com/" /> </outline> <outline title="Programming" text="Programming"> <outline text="Beautiful Code" title="Beautiful Code" type="rss" xmlUrl="http://beautifulcode.oreillynet.com/index.atom" htmlUrl="http://beautifulcode.oreillynet.com/" /> <outline text="CodeBetter.Com - Stuff you need to Code Better!" title="CodeBetter.Com - Stuff you need to Code Better!" type="rss" xmlUrl="http://feeds.feedburner.com/CodeBetter" htmlUrl="http://codebetter.com/blogs/" /> <outline text="Coding Horror" title="Coding Horror" type="rss" xmlUrl="http://feeds.feedburner.com/codinghorror/" htmlUrl="http://www.codinghorror.com/blog/" /> </outline> <outline title="Low Priority" text="Low Priority"> <outline text="Digg" title="Digg" type="rss" xmlUrl="http://digg.com/rss/index.xml" htmlUrl="http://digg.com/" /> </outline> Do you have a better way to organise your RSS? Share it in the comments!

The fuser Command

fuser is one of the *nix commands that I have started seeing myself use more and more frequently. It is a very powerful command, yet often forgotten. So what is it? fuser displays the process IDs of the processes that are using the files specified.

Identifying Processes
The following example shows how you can use the fuser command to find out which process is writing a log file called log.txt. fuser tells you the process ID (24976), which you can then use in a ps command to find out what the process is. In this case, its java.

sharfah@starship:~> ls -ltr
-rw-rw-r--   1 sharfah   sharfah     2836 Aug  6 00:08 log.txt

sharfah@starship:~> fuser -u log.txt
log.txt:    24976o(sharfah)

sharfah@starship:~> ps -ef | grep 24976
  sharfah 24976 24963  0 23:49:00 ?       13:39 java -server
  sharfah  6284 17191  0 00:09:54 pts/1    0:00 grep 24976

Used in this way, fuser can help tell you which process is responsible for creating large log files on your filesystem!

Killing Processes
The -k flag, sends the SIGKILL signal to each process using the file. This is handy if you want to "kill -9" a process without hunting for its PID first. Alternatively, if you want to send another signal type, use the -s flag. e.g. to send SIGTERM use -s TERM. The -k option is equivalent to -s KILL or -s 9. Most of my shutdown scripts, simply call fuser -s TERM on the process's log file.

Because fuser works with a snapshot of the system image, it may miss processes that begin using a file while fuser is running. Also, processes reported as using a file may have stopped using it while fuser was running. These factors should discourage the use of the -k option.

Eclipse Shirt Is Coming!

I finally received the following email from Lynn Gayowski, of the Eclipse Foundation, about the shirt they promised for taking part in the Ganymede Around the World Contest:

Thank you for submitting a review of the Eclipse release to the Ganymede Around the World contest. We have an Eclipse shirt for you! Please send me your:
1) Name
2) Shirt size (men’s small, medium, large, X-large, 2X-large or women’s small, medium, large)
3) Mailing Address
4) Phone Number (Fedex requires this for deliveries)

The contest closed yesterday, so all blog entries have been passed on to a panel of judges and we hope to have the winners of the Eclipse jackets and conference pass announced in a few weeks.

Thank you for participating and best of luck in the contest.

Regards,
Lynn Gayowski
Marketing Events Manager
Eclipse Foundation, Inc.
P (613) 224-9461 ext. 234
F (613) 224-5172
lynn.gayowski@eclipse.org
www.eclipse.org

I have sent them my details and am looking forward to getting my shirt. I will be posting pictures here as soon as it arrives, so come back soon! Better still, subscribe to my feed!

Related post:
Here is my review of Ganymede, for those who missed it: Eclipse Ganymede Review

Cedric's Coding Challenge

I came across a programming challenge on Cedric Beust's blog and thought I'd have a go at it. The goal is to write a counter function that counts from 1 to max but only returns numbers whose digits don't repeat. It has been solved by many users in a number of different ways using languages such as Java, C, Perl, Erlang, Javascript, C#, Groovy, Haskell, AS3, C, Fan, LUA, J, OCaml, Factor, Forth, Lisp, Ursala and Prolog. I am going to write a Unix Shell program to solve it (simply because this hasn't been tried by anyone else yet).

My Shell Solution:

#!/usr/bin/sh
#
# Counts from 1 to max and prints numbers whose digits
# don't repeat. Displays the biggest jump and the total
# count of numbers.
# Author: sharfah

max=$1
counter=1
prev=$counter
maxdiff=0
total=0

while ( [ $counter -le $max ] )
do
    echo $counter | grep '\(.\).*\1' > /dev/null
    if [ $? -ne 0 ]
    then
        echo "$counter"
        total=`expr $total + 1`
        diff=`expr $counter - $prev`
        if [ $diff -gt $maxdiff ]
        then
            maxdiff=$diff
            from=$prev
            to=$counter
        fi
        prev=$counter
    fi
    counter=`expr $counter + 1`
done
echo "Biggest jump is $maxdiff:$from->$to"
echo "Total count of numbers is $total"

Output
Run on an Intel(R) Xeon(R) CPU 2.33GHz 8 CPU, SUSE LINUX Enterprise Server 9, it produced:

sharfah@starship:~> time ./beust.sh 10000 > beust.log
   38.68s real    12.71s user    25.80s system

sharfah@starship:~> tail beust.log
9867
9870
9871
9872
9873
9874
9875
9876
Biggest jump is 105:1098->1203
Total count of numbers is 5274

When run on Solaris 10, it took 1m26.219s.

If you think you can make this go faster, let me know how in the comments section!

SSH Without a Password

I often have to login to many different servers and hate having to type my password in everytime. What's even worse, is that I can't even cron up any nice scripts which use the SSH or SCP commands, because they would just hang asking for a password!

SSH works by the exchange and verification of information, using public and private keys, to identify hosts and users. It then provides encryption of subsequent communication, also by the use of public/private key cryptography. This post will show you how you can generate a public/private key pair, install them in the correct location and SSH without being prompted for a password.

Here are the steps involved:

1. Generate the public/private keys on the Client
First you have to generate a public/private key pair on the client, which is the machine you will be ssh'ing from. To do this, use the ssh-keygen command:

sharfah@starmobile:~> ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/sharfah/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/sharfah/.ssh/id_dsa.
Your public key has been saved in /home/sharfah/.ssh/id_dsa.pub.
The key fingerprint is:
d1:98:9d:8b:7d:f6:c1:ba:1d:cc:05:ee:0b:9d:2d:17 sharfah@starmobile

Now, if you look in your ~/.ssh directory, you will see two files:

• id_dsa (the private key)

  -----BEGIN DSA PRIVATE KEY-----
  MIIBugIBAAKBgQDBDQ/+YpRikQfo/1x1mXRy83wbLJ67hUm357Vy24ab17V1FThm
  3S14D0UoqcTN0uflDLjj3CtfGeMU85t7Kbf1DBaiQ55syPilVqzQTjpuN44A3j5K
  e1eRX6LK46lspGR/ylrVHCRxJGXZ4K1OsxPgN7RhRHlRSOs5QAGeSAsHBQIVAJU1
  lempGnCjsaPop1BiYPvRoh4NAoGAI6qObsda+DnV3qQVHmV//iThpY3Z+z81uyUF
  1qq6XRnOTIwqJuF4lm0VBb4G+8pWIn3y5Kc+051sZ+gKlHcmtLMpxh+6QVD5KoRg
  XWHW11KEQldK9TKr18Taw8AhWFvp++kOd4I2Eq287Lecr95ty8YfdXD78kS+skpa
  z7/OdhkCgYAIXN2ljv5J1XAeZDnCfOPKkxWRoJ7M4/aKqdMIHAlxp6btpCuCl2cz
  F2/e0QQHUvABWjJpFG6IUNxRxDmvOinorfXR42thOFs4pNGMUWxVS4rRTYDpRGBz
  YbuY8awyzp2rAS6uhoHbpbDjsXhAA+fOJ0Xy6mJhDsj9Hnte5OD6DAIUBD2sJqrp
  ya/kd8vZSFrepLioucY=
  -----END DSA PRIVATE KEY-----
  

• id_dsa.pub (the public key)

  ssh-dss AAAAB3NzaC1kc3MAAACBAMEND/5ilGKRB+j/XHWZdHLzfBssnruFSbfntXLbhpvXtXUVOGbdLXgPRSipxM3S5+UMuOPcK18Z4xTzm3spt/UMFqJDnmzI+KVWrNBOOm43jgDePkp7V5FfosrjqWykZH/KWtUcJHEkZdngrU6zE+A3tGFEeVFI6zlAAZ5ICwcFAAAAFQCVNZXpqRpwo7Gj6KdQYmD70aIeDQAAAIAjqo5ux1r4OdXepBUeZX/+JOGljdn7PzW7JQXWqrpdGc5MjCom4XiWbRUFvgb7ylYiffLkpz7TnWxn6AqUdya0synGH7pBUPkqhGBdYdbXUoRCV0r1MqvXxNrDwCFYW+n76Q53gjYSrbzst5yv3m3Lxh91cPvyRL6ySlrPv852GQAAAIAIXN2ljv5J1XAeZDnCfOPKkxWRoJ7M4/aKqdMIHAlxp6btpCuCl2czF2/e0QQHUvABWjJpFG6IUNxRxDmvOinorfXR42thOFs4pNGMUWxVS4rRTYDpRGBzYbuY8awyzp2rAS6uhoHbpbDjsXhAA+fOJ0Xy6mJhDsj9Hnte5OD6DA== sharfah@starmobile
  

2. Copy the public key to the Server(s)
Next, copy your public key, id_dsa.pub, to the server's authorized_keys2 file. The server is the machine you will be ssh'ing to. You can do this by editing the authorized_keys2 file on the server directly (not recommended) or with the one line command below, which appends the public key to the end of the authorized_keys2 file:

sharfah@starmobile:~/.ssh> cat id_dsa.pub | \
 ssh sharfah@starserver "cat >> ~/.ssh/authorized_keys2"

That's it! You can now SSH onto the server without being prompted for a password!

Caching Remote Stubs to Improve RMI

In a typical RMI-based distributed system, the client makes two remote calls to the server:

1. To fetch a stub from the remote RMI registry, and
2. To make a method call to the server using the stub

This is horribly inefficient. Remote method calls are slow - most tests indicate that a simple remote method call is at least 1,000 times slower than an ordinary, in-process method call (and this will only get worse as processor speed is increasing at a faster rate than network speed).

Another problem you may see, especially in a very large distributed environment, is the "too many open files" error. This occurs if your client is making lots of quick RMI calls to lots of different remote servers and there isn't enough time for the opened RMI sockets to close. (I think sockets are closed if they have been idle for at least 15 seconds). As a result, you run out of "file descriptors" and aren't able to make any more remote calls, until some have freed up. This can be a pain and you have to get your Unix SA to increase the ulimit on the client host, in order make more remote connections.

In this post, I will describe how you can use a cache to halve the number of remote calls you make. So instead of fetching a new stub each time, simply fetch the stub the first time and store it in a local stub cache (e.g. in a hash table in memory). The second time you need to make a remote call, the stub is already available locally and, hence, only one remote method invocation is necessary. BUT the stub may not be valid (e.g. if the server has been restarted). If the stub isn't valid, an instance of RemoteException will be thrown when client attempts to use it to make a remote method call. In this case, the stub should be removed from the cache and a remote lookup performed to get a fresh one.

RemoteCache.java

import java.net.MalformedURLException;
import java.rmi.NotBoundException;
import java.rmi.Remote;
import java.rmi.RemoteException;
import java.util.Hashtable;
import java.util.Map;

/**
 * A local Remote stub cache
 * Instead of fetching stubs each time over the network,
 * simply fetch them the first time and store them in a
 * local cache.
 *
 * @author Fahd Shariff
 */
public class RemoteCache
{
  private static Map<String,Remote> cache
   = new Hashtable<String, Remote>();

  /**
   * Returns a stub to the remote object.
   * This method first checks the cache
   * and if not found, looks up in the registry
   *
   * @param serverDescription the url to the server
   * @return a stub to the remote object
   * @throws MalformedURLException
   * @throws RemoteException
   * @throws NotBoundException
   */
  public static Remote get(String serverDescription)
    throws MalformedURLException,
           RemoteException,
           NotBoundException {
    Remote toReturn=cache.get(serverDescription);
    if (toReturn==null) {
      toReturn=(Remote)RMIRegistry.lookup(serverDescription);
      if (toReturn!=null) {
        cache.put(serverDescription,toReturn);
      }
    }
    return toReturn;
  }

  /**
   * Removes the specified Remote Object from the cache.
   *
   * @param serverDescription
   */
  public static void remove(String serverDescription) {
    cache.remove(serverDescription);
  }

}

Using the RemoteCache
The following code snippet shows how you use the RemoteCache to obtain a stub. There is a simple retry loop - if a RemoteException occurs, the invalid stub is removed from the cache and a fresh one is obtained.

String url = "rmi://remotehostname:2138";
while(retries<MAX_RETRIES){
  try{
    MyRemoteObject remoteObj = (MyRemoteObject)
                           RemoteCache.get(url);
    remoteObj.callMyMethod();
    break;
  }
  catch(RemoteException e){
    //stub is invalid, so remove and retry
    RemoteCache.remove(url);
    retries++;
  }
}

References:
Seamlessly Caching Stubs for Improved Performance
Expiring Data with Hashbelts

Syntax Highlighting Code in Webpages

This post is deprecated. Please read my new entry on: "Upgrading to SyntaxHighlighter 3.0"

If you're a code blogger or someone who frequently posts code snippets online, then you'll know how difficult it can be to get your code highlighted and displayed nicely on your webpage. I have tried a number of different ways, such as saving code to HTML in SciTe or using Java2HTML to produce HTML files from Java and then copying the HTML output into my webpage. These processes are time-consuming and the HTML produced is ugly so I have always been on the lookout for something that will make code posting easier.

A few months ago, I stumbled across SyntaxHighlighter and it's just what I've been looking for! All you have to do is link your webpage to some CSS and JavaScript and surround your code with a tag saying which language the code is in. It's really that simple!

This is how you can use SyntaxHighlighter to highlight your code:

1. Download SyntaxHighlighter
This is an optional step. You don't necessarily have to download it because you can just link to the free hosted version. But if you have your own server you can download the latest version of SyntaxHighlighter here and upload it.

2. Link to CSS and Javascript
Open your webpage's HTML file and add links to the SyntaxHighlighter's CSS and JavaScript files. For optimal results, place these lines at the very end of your page, just before the closing body tag.

<link type="text/css" rel="stylesheet" href="http://alexgorbatchev.com/pub/sh/current/styles/shCore.css"></link>
<link type="text/css" rel="stylesheet" href="http://alexgorbatchev.com/pub/sh/current/styles/shThemeDefault.css"></link>
<script type="text/javascript" src="http://alexgorbatchev.com/pub/sh/current/scripts/shCore.js"></script>
<script type="text/javascript" src="http://alexgorbatchev.com/pub/sh/current/scripts/shLegacy.js"></script>
<script type="text/javascript" src="http://alexgorbatchev.com/pub/sh/current/scripts/shBrushBash.js"></script>
<script type="text/javascript" src="http://alexgorbatchev.com/pub/sh/current/scripts/shBrushJava.js"></script>
<script type="text/javascript" src="http://alexgorbatchev.com/pub/sh/current/scripts/shBrushXml.js"></script>
<script type="text/javascript" src="http://alexgorbatchev.com/pub/sh/current/scripts/shBrushSql.js"></script>
<script type="text/javascript">
    SyntaxHighlighter.config.bloggerMode = true;
    SyntaxHighlighter.all();
</script>

It is not necessary to add the js files for all the languages - just for the ones you will be using.

3. Add Code
Now add the code you wish to highlight in your webpage, surrounded by the <pre> tag. Set the class attribute to one of the language aliases you wish to use, such as java, xml, sql, ruby etc.For example: brush:java

<pre title="Test SyntaxHighlighter" class="brush: java;">
public void printHello(){
    System.out.println("Hello World");
}
</pre>

4. View Page
View the webpage in your browser and you should see your syntax highlighted code snippet as:

public void printHello(){
    System.out.println("Hello World");
}

Configuration Options
Here are a couple of handy options. (Full list of here):

• If you don't want to display the line numbers column, use the gutter option e.g. gutter:false.
• If you don't want to display the top toolbar, use the toolbar:false option.

If you use an alternative to SyntaxHighlighter, share it with us in the Comments section!

Links:
SyntaxHighlighter Homepage

Get Google's new icon in Firefox's Search Box

This is how you can update your search box in Firefox to display Google's new favicon

1. Go to the searchplugins directory, where you installed Firefox. In Windows XP, this will normally be: C:\program files\Mozilla Firefox\searchplugins

2. Open google.xml in your favourite text editor.

3. Change the Image tag to the following and save:

<Image width="16" height="16">data:image/x-icon;base64,AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC1tbUFHBwcFQAAABoAAAAaAAAAFgAAAA8AAAAWBAAAGwIAABoAAAAXAAAAGQAAABkAAAAaAAAAGg0NDRiOjo4JJiYmDwAAABQAAAATAAAADQwCAR1YFRB4nCsqua0yLMSbLCyxYxoYfBgFAiwDAQAUAAAAEgAAABMAAAAUBwcHEhQUFA0AAAAOAAAACxMEARayLy%2FF7T9C%2F640K7FuJR1pZiMgYpczMZqeLSzBIAcFPQEBABAAAAANAAAADgAAAA4XFxcJAAAACgAAAABbKSNY%2F1ZO%2F3MdHY8AAAAAAAAAAAAAAAAAAAAAjzc2lHwdG7oDAAAVAAAACAAAAAoAAAAKFxcXBgAAAAcAAAAAVj43Tfx6df87BQFmAQIAAgAAAAQAAAAEAAAAAI0vJomvLCftCwEAHQAAAAQAAAAHAAAABxEREQQAAAAEAAAAAwYEAAi4bVm0uT8%2F4UwOC1wnBgMlFAQAEF4YGmnwQ0X8niQfxQEAAAgAAAADAAAABAAAAAQzMzMCHh4eAh8fHwMAAAAAKSglBJVbWGfQX16qxktJsLItKNTqP0X%2F5DxE8VklJTsAAAAAHx8fAh8fHwIfHx8C7%2B%2FvBO3t7QTt7e0E7e3tBOrr6wIAAAAAAAAAALWHgkH%2BVFD%2FxjI226J1dDYAAAAA7e7uBO3t7QTt7e0E7e3tBP%2F%2F%2Fwn%2F%2F%2F8K%2F%2F%2F%2FCv%2F%2F%2Fwv%2F%2F%2F8Eybi3HHw1LHeySkjC%2Fnd2%2F34iIZ1rbGkJ8fHxCv%2F%2F%2Fwr%2F%2F%2F8K%2F%2F%2F%2FCv%2F%2F%2Fwr%2F%2F%2F8R%2F%2F%2F%2FE%2F%2F%2F%2FxP%2F%2F%2F8P9OTiHtFGSNHPKS%2F%2F15KSeP7k4Evgb3DOXh4cjouJiCD9%2Ff4R%2F%2F%2F%2FE%2F%2F%2F%2FxP%2F%2F%2F8T%2F%2F%2F%2FGf%2F%2F%2Fxv%2F%2F%2F8b%2F%2F%2F%2FDvu3rnbrNCn%2Fo1NTk9%2Fr6wn%2F%2F%2F8C%2F9fXaqskJf9kTUxX7e%2FvGP%2F%2F%2Fxr%2F%2F%2F8b%2F%2F%2F%2FG%2F%2F%2F%2FyH%2F%2F%2F8k%2F%2F%2F%2FJP%2F%2F%2Fxb%2F0MyT00JA%2F5Byblv7%2F%2F8d%2F%2F%2F%2FFv7Ix3XUNjH%2FhFhVee%2Fz8x3%2F%2F%2F8j%2F%2F%2F%2FJP%2F%2F%2FyT%2F%2F%2F8o%2F%2F%2F%2FLf%2F%2F%2Fyz%2F%2F%2F8j%2F%2FDmZNphWv%2BOa2ll4urpJPHy8iLwcWDF0zEn%2B7edm1f%2F%2F%2F8m%2F%2F%2F%2FLP%2F%2F%2Fyz%2F%2F%2F8s%2F%2F%2F%2FMP%2F%2F%2FzX%2F%2F%2F80%2F%2F%2F%2FM%2F%2F%2F%2Fy77u7KoxlhZyrV%2Ff3XSc3Oy7z83%2F7w5NtymioZg5OHgOf%2F%2F%2FzP%2F%2F%2F80%2F%2F%2F%2FNf%2F%2F%2Fyv%2F%2F%2F89%2F%2F%2F%2FOv%2F%2F%2Fzr%2F%2F%2F84%2F%2F%2F%2FM%2F%2FT0m%2F3oaGk%2FbS0uv24ubPujouw2oqIoO3j40r%2F%2F%2F83%2F%2F%2F%2FPP%2F%2F%2FzP%2F%2F%2F8K%2F%2F%2F%2FMf%2F%2F%2Fz7%2F%2F%2F89%2F%2F%2F%2FPf%2F%2F%2Fz3%2F%2F%2F80%2F%2F%2F%2FMf%2F%2F%2FzH%2F%2F%2F8x%2F%2F%2F%2FMf%2F%2F%2FzX%2F%2F%2F88%2F%2F%2F%2FPv%2F%2F%2Fzf%2F%2F%2F8SAAAAAAAAAAAAAAAAI8AAACBAAAAAAAAAEAgAAAYQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%3D%3D</Image>

4. Restart Firefox to see the curly-g icon in your search box!

Related Posts:
Google's New Favicon
How-to write your own Search Plugin for Firefox

Big Strings and Oracle Clobs

Oracle clobs are funny things. I've been trying to store a really long string as a clob into my Oracle database via JDBC and it's been a nightmare! In this post, I will describe all of the different approaches I tried and finish with the one which finally worked!

Setup:

String Size	7631 bytes
Database version	Oracle9i Enterprise Edition Release 9.2.0.8.0
Oracle NLS_CHARACTERSET	WE8ISO8859P1
JDBC driver version	Oracle Database 10g Release 2 (10.2.0.4)
Table Column DataType	NCLOB

Attempt 1: SetBigStringTryClob
In Oracle JDBC 10g, there is a new Connection property called SetBigStringTryClob which allows the statement's setString method to process strings greater than 32765 bytes.

Properties props = new Properties();
props.put("user", "username" );
props.put("password", "password");
props.put("SetBigStringTryClob", "true");

Connection conn = DriverManager.getConnection(dbUrl,props);
PreparedStatement st = conn.prepareStatement(INSERT_SQL);
st.setString(1, bigString);
st.executeUpdate();

This attempt failed - it inserted garbage (lots of inverted question marks and other funny characters) in my clob column.

Attempt 2: setStringForClob
The Oracle specific method of setStringForClob can be used for binding data greater than 32765 bytes. Note that this method is called internally if you call setString and have SetBigStringTryClob set to true (as in Attempt 1).

OraclePreparedStatement st = (OraclePreparedStatement)
          conn.prepareStatement(INSERT_SQL);
st.setStringForClob(1, bigString) ;

This attempt failed with the same result as previous one - it inserted garbage (lots of inverted question marks and other funny characters) in my Clob column.

Attempt 3: setCLOB
Create a temporary Oracle CLOB, populate it and call setClob.

CLOB clob = CLOB.createTemporary(conn,
  true,
  oracle.sql.CLOB.DURATION_SESSION,
  Const.NCHAR);
clob.trim(0);
Writer writer = clob.getCharacterOutputStream();
writer.write(bigString.toCharArray());
writer.flush();
writer.close();
st.setClob(1, clob);

This attempt failed with: ORA-12704: character set mismatch

Attempt 4: setCharacterStream
Use setCharacterStream to get a stream to write characters to the clob.

Reader reader = new StringReader(bigString);
int readerLength = bigString.toCharArray().length;
st.setCharacterStream(1, reader, readerLength);

Failed - garbage inserted again!

Attempt 5: Insert an empty_clob() and then update it
Insert an empty_clob() into the table, retrieve the locator, and then write the data to the clob.

String sql = "INSERT INTO clob_table (clob_col) "+
             "VALUES (empty_clob())";
PreparedStatement st = conn.prepareStatement(sql);
st.executeUpdate() ;

sql = "SELECT clob_col FROM clob_table FOR UPDATE";
st = conn.prepareStatement(sql);
ResultSet rs = st.executeQuery();
rs.next();
Clob clob = rs.getClob(1);
Writer writer = clob.setCharacterStream(0);
writer.write(bigString.toCharArray());
writer.flush();
writer.close();
rs.close();

Success! However, I'm not happy with the two database calls; one to create the empty clob and the other to update it. There must be a better way!

Attempt 6: PL/SQL
Wrap the SQL insert statement in PL/SQL to work around the size limitation.

INSERT_SQL = "BEGIN INSERT INTO clob_table (clob_col) "+
             "VALUES (?); END";
st = conn.prepareStatement(sql);
st.setString(1, bigString);
st.executeUpdate();

Success! And with only one database call!
Note, that setString can only process strings of less than 32766 chararacters, so if your String is bigger than this, you should use the empty_clob() technique from Attempt 5.

Phew! After six attempts, I've finally found two which work. Why does this have to be so complicated?!

References:
Oracle JDBC FAQ
Oracle 10g JDBC API
Using Oracle 10g drivers to solve the 4000 character limitation
Handling CLOBs - Made easy with Oracle JDBC 10g

Eclipse Ganymede

Eclipse Ganymede is the annual release of Eclipse projects; this year including 23 projects. Some highlights of the release include the new p2 provisioning platform, new Equinox security features, new Ecore modeling tools, support for SOA and much more.

I have been using the new Eclipse Ganymede release for about two weeks now and think its time to write a review and share some of my experiences with the rest of the developer community. On the whole, I think the new release is as snappy as previous versions and brings some welcome improvements. Here are some of the features that I really like in this release:

Breadcrumbs:
The editor now displays a breadcrumb navigation bar showing the path to the current file. You can easily access the project and package structure as well as the individual classes, fields and methods from the bar itself. I really like this feature because I no longer have to keep my Package Explorer and Outline views open, thus saving on precious screen estate.

Enhanced Hover:
In the previous version, you had to use the awkward keyboard shortcut of Ctrl+1 on top of an Error or Warning to get the Quick Fix options. In Ganymede, all you have to do is hover over the problem and it pops up a window with links to the options. Great!

Highlight Read/Write Variables:
A small but useful addition to this release is that Mark Occurrences (Alt+Shift+O) now marks read and write accesses with different colours. Previously, Eclipse offered the ability to highlight all occurrences of a variable but now it distinguishes between read and writes for you.

Call Hierarchy for Fields and Types:
This allows you to find all the members which access the field (for read or write) and the constructors of a type. Previously, you could only find callers of a method.

Support for External Class Folders
Class folders located outside the workspace (external) can now be added to the build path. Previously, I had to build a jar file and then add it using "Add External JARs" so this is a welcome improvement! You can even add other kinds of zip archives to the build path e.g. RAR files.

Performance:
To be honest, I haven't noticed any increase or decrease in either the start-up time or interaction with the user interface. However, there has reportedly been a lot of internal improvement to the JDT compiler so that compilation can now be spread across multiple cores rather than being able to utilise only one. This should speed up build times.

There are still many more things for me to try out. In particular I would really like to try out the Test and Performance Tools Platform Project.

Got your own experiences with Ganymede? Share them in the comments.

For more details:
What's New in 3.4 (JDT)
Learn more about Ganymede

Write a Search Plugin for Firefox [Howto]

Firefox 3 currently supports plugins in two specifications. These are Apple's Sherlock format and the more recent OpenSearch syntax which is now preferred as it is supported by both Firefox (2+) and Internet Explorer (7+). In this post, I will show you how to create your own search engine plugin using the OpenSearch syntax.

1. Pick a search engine you want to create a plugin for. For this example, I have chosen Picitup which is a new image search engine (still in beta). It is not listed on Mycroft Search Engine Plugins either, so it is likely that it doesn't exist yet.

2. Find your searchplugins directory. On Windows XP, this will typically be in

%APPDATA%\Mozilla\Firefox\Profiles\xxxxxxxx.default\searchplugins

where xxxx is a random string.

3. Create a new file in the searchplugins directory called picitup.xml with the contents below:

<SearchPlugin xmlns="http://www.mozilla.org/2006/browser/search/" xmlns:os="http://a9.com/-/spec/opensearch/1.1/"> <os:ShortName>Picitup</os:ShortName> <os:Description>Search Picitup</os:Description> <os:InputEncoding>UTF-8</os:InputEncoding> <os:Image width="16" height="16"> http://www.picitup.com/picitup/images/plusico/fav.ico </os:Image> <os:Url type="text/html" method="GET" template="http://www.picitup.com/picitup/results.jsp?query={searchTerms}"> </os:Url> <SearchForm>http://www.picitup.com</SearchForm> </SearchPlugin>

The icon file is usually called favicon.ico and is found on the level of the homepage. However, in this case it is present in a different directory, which I found by viewing the HTML source of the Picitup homepage:

<link rel="SHORTCUT ICON" href="images/plusico/fav.ico">

To find the search URL, look at the HTML <form> element:

<form name="search" action="results.jsp" method="get">
  <input input type=text name=query size="38" style="border: 1px solid #BDBDBD;"/>
  <input type="hidden" name="searchType" value="image"/>
  <input type="hidden" name="addToDb" value="1"/>
  <input type="submit" value="Search" onclick="javascript:submitForm();"/>
</form>

4. Restart Firefox and thats it! Click on the Search Engine combo box and you should now be able to try out the new Picitup addon.

5. Share your plugin with the rest of the world by uploading it here: http://mycroft.mozdev.org/submitos.html

NB: After restart, Firefox will change you xml file by base64 encoding the image icon URL.

Reference:
There is detailed documentation about the OpenSearch format at OpenSearch.org and with reference to Firefox at the Mozilla Developer Center.

Firefox Sets Record 8,002,530 Downloads

Got an email from the Firefox guys today, confirming that they have indeed set a Guinness World Record for the most software downloads in 24 hours! Email quoted below:

We did it!

We set a Guinness World Record for the most software downloads in 24 hours. With your help we reached 8,002,530 downloads.

You are now part of a World Record and the proud owner of the best version of Firefox yet!

Don't forget to download your very own certificate for helping set a Guinness World Record.

View my certificate

Send Email Alerts Using Log4j [Howto]

It's really easy to add error alerting to your java application. If your application logs all errors using log4j and you want these errors emailed out to a support team or to yourself, then all you have to do is add another appender to your log4j.properties file.

Let's say you have the following class which logs an error message: import org.apache.log4j.*; public class Dummy{ private static Logger logger = Logger.getLogger(Dummy.class); public void logError(String message){ logger.error(message); } public static void main(String[] args){ new Dummy().logError("An error has occurred!"); } } And your log4j properties file is: log4j.rootLogger=INFO, R log4j.appender.R=org.apache.log4j.ConsoleAppender log4j.appender.R.layout=org.apache.log4j.PatternLayout log4j.appender.R.layout.ConversionPattern=%d [%t] %-5p %c %x - %m%n Now add a new MAIL appender, so that your properties file looks like this: log4j.rootLogger=INFO, R, MAIL log4j.appender.R=org.apache.log4j.ConsoleAppender log4j.appender.R.layout=org.apache.log4j.PatternLayout log4j.appender.R.layout.ConversionPattern=%d [%t] %-5p %c %x - %m%n log4j.appender.MAIL=org.apache.log4j.net.SMTPAppender log4j.appender.MAIL.BufferSize=10 log4j.appender.MAIL.SMTPHost=post.uk.mail.db.com log4j.appender.MAIL.From=username@abc.com log4j.appender.MAIL.to=username@abc.com log4j.appender.MAIL.Subject=[Dummy] Error Alert log4j.appender.MAIL.layout=org.apache.log4j.PatternLayout log4j.appender.MAIL.layout.ConversionPattern=%d [%t] %-5p %c %x - %m%n Don't forget to place mail.jar and activation.jar on your classpath and then run the application.

Check your inbox and voila, you have an alert without making any source code changes!

NB: By default, an email message will be sent when an ERROR or higher severity message is appended. If you want to email messages with levels less than ERROR (e.g. INFO) then you currently have to configure your own implementation of the TriggeringEventEvaluator. Setting log4j.appender.MAIL.Threshold=INFO will not work.

Further Reading:
SMTPAppender javadoc
SmtpAppender Members and Properties

Skip Tests in Maven [Tip]

If your tests are failing and you can't be bothered to fix them, this handy tip will show you how you can skip tests and build your Maven project straightaway!

Simply add the property -Dmaven.test.skip=true to your Maven command. Here is an example:

mvn -Dmaven.test.skip=true install

The compiler will not compile the test sources and will not run your JUnit tests:

[INFO] [compiler:testCompile]

[INFO] Not compiling test sources
[INFO] [surefire:test]
[INFO] Tests are skipped.

But of course, your test classes should always run, right?

Reference:

http://maven.apache.org/general.html

Read a File Without Trimming Leading Whitespace [Shell Scripting]

This is how you would normally read a file, line-by-line, in a shell script:

file=/home/fahd/dummy
while read line
do
    echo "$line"
done < $file

The problem with this is that the "read" command automatically removes leading whitespace from each line and also concatenates a line ending with a backslash with the one following. This means that you cannot properly process a file with lots of leading whitespace e.g. xml files.

The trick around this, is to redefine your IFS (Internal Field Separator) variable. By default, IFS is set to the space, tab and newline characters to delimit words for the read command. This is how you can amend your script:

file=/home/fahd/dummy
OIFS=$IFS
IFS=
while read -r line
do
    echo "$line"
done < $file
IFS=$OIFS

First save the current value of IFS into a temporary variable called OIFS. Then blank out IFS. When you have finished reading your file, set IFS back to its original value of OIFS.

We also provide the read command with the -r flag, so that it treats each backslash to be part of the input line and does not concatenate it with the next line.

Shutting Down Java Apps [Howto]

In most cases, users do not always follow the recommended procedure to exit their java applications. They may not type "quit" or "exit" and may instead hit Ctrl+C to terminate the process immediately. In these cases, the application does not get a chance to clean-up.

Fortunately, there is a way to ask the JVM to execute some clean-up code before exiting. This is done by using ShutdownHooks which are registered with System Runtime. The JVM starts all registered shutdown hooks concurrently when shutting down.

Here are the steps involved when creating shutdown hooks:

1. Create the shutdown hook class:

public class AppShutdownHook extends Thread{
    public void run(){
        logger.info("Running shutdown hook...") ;
        //cleanup e.g. close database connections
    }
}

2. Register the shutdown hook:

public class App{
    public App(){
        Runtime.getRuntime().
          addShutdownHook(new AppShutdownHook()) ;
    }
}

Note that no guarantee can be made about whether or not any shutdown hooks will be run if the JVM aborts with the SIGKILL signal (kill -9) on Unix or the TerminateProcess call on MS Windows.

However, SIGTERM (kill with no arguments) will work.

Reference:
http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Runtime.html#addShutdownHook(java.lang.Thread)

Moving House

I'm back at work today after having taken time off since Wednesday to move house. I spent Wednesday packing boxes and got the CHAPS transfer and mail redirection (£35.40 for a year) done. Keys were exchanged at 1pm on Thursday, and it took two Luton vans + three removal men (£270) to carry our stuff to the new house. We were all done by 4pm. We've unpacked our important items but are still mostly surrounded by a sea of boxes! Its good to be back at work though!

Here is a list of useful bookmarks if you're moving house: http://del.icio.us/fahdshariff/moving%2Bhouse

Firefox 3 Released!

Firefox 3 is here! Download it now and help set a Guinness World Record for the most downloads in 24 hours!

You even get a certificate!

Change Logging Levels using JMX [Howto]

This handy example shows you how you can use Java Management Extensions (JMX) to change your application's logging level.

Step 1: Create your management interface which consists of all the attributes which can be read or set and all the operations that can be invoked. In this case, we want to change the logging level of our application.

public interface MyAppMBean{
    public void setLoggingLevel(String level) ;
    public String getLoggingLevel() ;
}

Step 2: Create a class which implements the MBean interface

import java.lang.management.ManagementFactory;
import javax.management.MBeanServer;
import javax.management.ObjectName;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import com.sun.jdmk.comm.HtmlAdaptorServer;

public class MyApp implements MyAppMBean{

    private static Logger logger = Logger.getLogger(MyApp.class);

    public void go() throws Exception{
        while(true){
            logger.debug("DEBUG") ;
            logger.info("INFO") ;
            Thread.sleep(2000);
        }
    }

    public void setLoggingLevel(String level){
        logger.info("Setting logging level to: " + level);
        Level newLevel = Level.toLevel(level, Level.INFO);
        Logger.getRootLogger().setLevel(newLevel);
    }

    public String getLoggingLevel(){
        return Logger.getRootLogger().getLevel().toString() ;
    }
}

Step 3: Register the MBean with the MBeanServer. Also register an HTMLAdaptorServer which allows us to manage an MBeanServer through a web browser.

public static void main(String[] args) throws Exception{
    MyApp app = new MyApp() ;
    ObjectName objName = new ObjectName("MyApp:name=MyApp");
    MBeanServer server = ManagementFactory.getPlatformMBeanServer();
    server.registerMBean(app, objName);

    int portNumber=9393;
    ObjectName htmlName = new ObjectName(
       "MyApp:name=MyAppHtmlAdaptor,port="+portNumber) ;
    HtmlAdaptorServer html = new HtmlAdaptorServer(portNumber);
    html.setPort(portNumber);
    server.registerMBean(html, htmlName);
    html.start();
    app.go();
}

Step 4: Compile. Make sure you have log4j, jmxtools and a log4j properties file in your classpath

Step 6: Run MyApp. You need to add the following JVM properties:

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

Step 5: Open jconsole

• Click on MyApp and press Connect
• Go to the MBeans tab and choose MyApp from the tree on the left
• You can then change your Logging Level while the application is running!

Step 6: Open the web console

• Go to http://localhost:9393
• Click on name=MyApp
• Change the LoggingLevel!

Firefox 3 - Release Date Tuesday, June 17, 2008

The Mozilla Developer Center is reporting Firefox 3 will be released on Tuesday, June 17, 2008!

After more than 34 months of active development, and with the contributions of thousands, we’re proud to announce that we’re ready. It is our expectation to ship Firefox 3 this upcoming Tuesday, June 17th. Put on your party hats and get ready to download Firefox 3 - the best web browser, period.

I've been using the betas and RCs for some time now and can't wait to get my hands on this release. If you haven't already pledged to download it, and help set a world record for the most software downloaded in 24 hours, do so now: http://www.spreadfirefox.com/en-UK/worldrecord/ (Total pledges 1,109,453 so far)

Google's New Favicon

You may have noticed that Google has a new favicon, the small icon (16x16 favicon.ico image file) you see in your browser next to the URL or in your bookmarks list. It has changed from the familiar upper-case G to a lower-case one.

According to an official Google post:

The reason is that we wanted to develop a set of icons that would scale better to some new platforms like the iPhone and other mobile devices. So the new favicon is one of those, but we’ve also developed a group of logo-based icons that all hang together as a unified set.

To me, its an obvious play on the infinity sign - Google ad infinitum.

The new icon has yet to grow on me. The old one, whilst old-fashioned, was immediately recognisable as part of the Google brand.

Send Your Name to the Moon

Sign up to send your name to the moon! Names will be collected and placed onboard the Lunar Reconnaissance Orbiter (LRO) spacecraft for its historic mission bringing NASA back to the moon.

The Lunar Reconnaissance Orbiter is the first mission in NASA's Vision for Space Exploration, a plan to return to the moon and then to travel to Mars and beyond. LRO will launch no earlier than November 24, 2008, with the objectives to finding safe landing sites, locate potential resources, characterize the radiation environment, and demonstrate new technology.

You will also receive a certificate showcasing your support of the mission. Here's mine:

Increase Console Output in Eclipse [Howto]

By default, Eclipse limits the amount of text in the console output window. So if you are running a program which produces a lot of output, the Console will only display the last n lines and you lose the rest.

Here is a tip which will allow you to have unlimited output in the console:

• Go to Window > Preferences > Run/Debug > Console
• Uncheck "Limit Console Output" (Alternatively you can increase the Console buffer size.)

Hampton Court Palace - Sat 24, May 2008

We took the 1036hrs train from Waterloo and got to Hampton Court in about 30mins. We had booked our tickets on-line, so bypassed all the queues. We then spent the morning walking through the Palace, taking in the medieval architecture and spectacular tapestries. We stepped into the Tudor Kitchens to watch 16th century style cooking and then had a relaxing lunch at the Tiltyard Cafe. In the afternoon, we wandered around the beautiful Fountain and Privy Gardens and got lost in the Maze! We returned to the Tiltyard Cafe for scones and hot chocolate and then made our way back home at 1800hrs.

Here is a link to my album: http://picasaweb.google.com/fahd.shariff/HamptonCourtPalace

scribd

Today, I signed up to Scribd - the YouTube for documents. It allows you to upload and share your documents and browse other interesting documents by category. The interface has borrowed a lot of the basic design principles from YouTube. Documents can be displayed and embedded as html or the under-utilized, and faster-than-a-pdf, Flash paper format. They can be downloaded as .pdf’s, .docs, .txt, and even .mp3 files. The mp3 version is created by Scribd’s text-to-speech package that lets you listen to the text of your document in a British accent. Scribd also lets you view all the statistics generated by documents you post, such as how many votes and views your piece gets, as well as geographic location and http referrer that brought the reader there.

I've started uploading some of my interesting school and university projects which I probably don't have the need for any more.

My profile: http://www.scribd.com/people/view/934950

How Old is my Brain?

According to Dr Kawashima, my brain is a little young for its age! (By the way, I am 26 at the time of this post.)

Factorial Algorithm

The factorial of a number is the product of all positive integers less than or equal to it. e.g. the factorial of 5 is 5 x 4 x 3 x 2 x 1 = 120.

While conducting a Java interview, I like to ask my candidates to write an algorithm to calculate the factorial of a number. This is one of my favourite questions because it gives me an insight into their thinking process. It is also one of the questions that I was asked at Merrill Lynch, in my first interview out of university. (I got it right)

I am surprised at how the majority of people cannot answer this question, or get anywhere close to it. C'mon, you can do it using recursion or loop-iteration; its not that difficult! If you can't count backwards, do it forwards - its the same thing.

Here is a simple solution: //loop public int factorial(int n) { int result = 1 ; for(int i = n ; i > 0 ; i--) { result = result * i ; } return result ; } //recursion public int recursiveFactorial(int n) { if (n == 0) return 1 ; else return n * recursiveFactorial(n-1) ; }

JDK 7 - Fork-join and ParallelArrays

A few weeks ago, I had the opportunity of attending an interesting presentation by Sun's Brian Goetz on the new concurrency features in JDK 7 which will allow applications to take advantage of idle CPU cores. Here is a summary of how this works:

In the recent past we have stopped seeing increases in CPU clock rates. We have started getting more cores per chip rather than faster cores. This brings new challenges in programming. We need to learn the art of decomposing our applications into units of work that can be executed in parallel over multiple cores.

Enter the fork-join framework - a library to provide better support for fine-grained parallelism. The most promising candidates for parallelism are sorting and searching. Sorting can be parallelised with merge sort. Searching can be parallelised by searching sub-regions of the data in parallel and then merging the results. This is an example of divide-and-conquer, in which a problem is broken down into sub-problems which are solved and the results combined. In merge sort, we divide the data set into pieces, sort the pieces and merge the results. Result is still O(n log n), but sub-problems can be solved in parallel. We can apply this process recursively, until sub problems are small enough that a sequential solution would be faster.

Divide-and-conquer algorithms take this general form

// PSEUDOCODE
Result solve(Problem problem) {
    if (problem.size < SEQUENTIAL_THRESHOLD)
        return solveSequentially(problem);
    else {
        Result left, right;
        INVOKE-IN-PARALLEL {
            left = solve(extractLeftHalf(problem));
            right = solve(extractRightHalf(problem));
        }
        return combine(left, right);
    }
}

The INVOKE-IN-PARALLEL step creates two or more new tasks (fork) and suspends the current task until the new tasks complete (join). Naive implementations would create a new thread for each task and then invoke Thread.join() for the join operation. But thread creation is expensive and requires O(log n) idle threads. A non-naive implementation is available in package java.util.concurrent.forkjoin in JDK 7 or you can download jsr166y.

The fork-join framework is implemented using work stealing. You create a limited number of worker threads (best choice is Runtime.availableProcessors()). Each worker maintains a double-ended work queue (deque). When forking, worker pushes the new task at the head of its deque. When waiting or idle, worker pops a task off the head of its deque and executes it instead of sleeping. If worker's deque is empty, steals an element off the tail of the deque of another randomly chosen worker. This means that there is never any contention for head and almost never contention for tail. Stealing is infrequent - because order is LIFO, when a worker steals, it generally steals a big chunk which will keep it from having to steal again for a while.

Then pool.invoke() is called, task is placed on a random deque. That worker executes the task and usually just pushes two more tasks onto its deque. It then starts on one of the subtasks. Soon some other worker steals the other top-level subtask. Pretty soon, most of the forking is done and the tasks are distributed among the various work queues. Now the workers start on the sequential subtasks. If work is unequally distributed, corrected via stealing. The result is that there is reasonable load balancing, with no central coordination, little scheduling overhead and minimal synchronisation costs.

We can reduce the amount of boilerplate code in fork-join tasks, by using the ParallelArray classes which let you declaratively specify aggregate operations on data arrays. ParallelArrays use fork-join to efficiently execute on the available hardware.

Further reading:
http://www.ibm.com/developerworks/java/library/j-jtp11137.html
http://www.ibm.com/developerworks/java/library/j-jtp03048.html

Garbage Collection Performance Tips

The purpose of Java's Garbage Collector is to find and reclaim unreachable objects. This post will show you how you can write readable and clean code that makes the most out of the garbage collector (in terms of throughput, responsiveness etc)

• Do not be afraid to allocate small objects for intermediate results. Typically, object allocation is very cheap and reclamation of new objects is very cheap too! GCs love small, immutable objects and generational GCs love small, short-lived objects. You should use short-lived immutable objects instead of long-lived mutable ones. Use clearer, simpler code with more allocations instead of more obscure code with fewer allocations.
  
  
• Avoid large objects as they are expensive to allocate and initialise.
  
  
• Nulling references rarely helps the GC. Instead it creates clutter and in the worst case may introduce a bug. The JIT can do liveness analysis very well.
  
  
• Avoid explicit GCs. Applications do not have enough information, but GC does. Calling System.gc() at the wrong time, may start a full stop-the-world GC (in HotSpot) which can hurt performance with no benefit.
  
  
• Avoid frequent resizing of array-based data structures as this will allocate several large-ish arrays and will cause a lot of array copying. Try to size them, on initialisation, as realistically as possible.
  
  
• Only use object pools for objects that are expensive to allocate/initialise or for those that represent scarce resources e.g. database connection pools or thread pools. Use existing libraries wherever possible. Unused objects in pools are bad; they are live and so the GC must process them and they provide no benefit to the application.
  
  
• Use finalization as a last resort. Finalizable object allocation is much slower and reclamation takes at least two GC cycles; the first cycle identifies object as garbage and enqueues object on finalization queue, the second cycle reclaims space after finalize() completes. Finalizers are not like C++ destructors! There is no guarantee if or when they will be called.
  
  
• Memory leaks occur because objects are reachable but unused. This may be due to objects in the wrong scope, lapsed listeners, exceptions changing control flow, instances of inner classes, reference objects etc. You should perform reachability analysis to clear up these objects.

Programming Is Like Sex

because...

• One mistake and you have to support it for the rest of your life.
• Once you get started, you'll only stop because you're exhausted.
• It often takes another experienced person to really appreciate what you're doing.
• Conversely, there's some odd people who pride themselves on their lack of experience.
• You can do it for money or for fun.
• If you spend more time doing it than watching TV, people think you're some kind of freak.
• It's not really an appropriate topic for dinner conversation.
• There's not enough taught about it in public school.
• It doesn't make any sense at all if you try to explain it in strictly clinical terms.
• Some people are just naturally good.
• But some people will never realize how bad they are, and you're wasting your time trying to tell them.
• There are a few weirdos with bizarre practices nobody really is comfortable with.
• One little thing going wrong can ruin everything.
• It's a great way to spend a lunch break.
• Everyone acts like they're the first person to come up with a new technique.
• Everyone who's done it pokes fun at those who haven't.
• Beginners do a lot of clumsy fumbling about.
• You'll miss it if it's been a while.
• There's always someone willing to write about the only right way to do things.
• It doesn't go so well when you're drunk, but you're more likely to do it.
• Sometimes it's fun to use expensive toys.
• Other people just get in the way.
• If you try to get too fancy, you could cause big problems (unless you really know what you're doing)
• You can do it alone, but it's even better in a group
• It's great to get paid, but you'd do it for free if you had to

Sea Monsters 3D @ BFI IMAX

Last Saturday, we decided to go and watch Sea Monsters 3D at the BFI IMAX Cinema. I've never been to the IMAX before and the experience was far better than I expected! The screen is the largest in Britan and the sound system is also larger than life.

The show kicks off with a quick announcement by one of the staff welcoming us to the auditorium and telling us about the film. First, there is a light and sound show in which you even get to see the speakers behind the screen. This is followed by an animated 3D clip which is awesome and then the actual film.

A 3D film consists of two films being shown simultaneously. The 3D camera has two lenses set the same distance apart as our eyes, and films both the left and the right eye images at the same time. The two pictures are then projected through polarized glass, each eye polarized differently so that the left eye does not see the image of the right eye and vice versa. You need to put special polarising glasses on which filter the left and right images to keep the images separate. Your brain then converges the two images and it appears in 3D. Suddenly you find yourself reaching out to touch what you believe to be right before your very eyes!

Sea Monsters 3D was an amazing film which allows you to get immersed right into the depths of the ocean and follow the lives of sea creatures. You see shoals of fish swimming right past you and sharks jump out of the screen to bite your face off! The experience is so realistic that you even hear children scream in fear when something comes towards them!

You can book a film online and choose seats. We had middle aisle seats and they were perfect. I am looking forward to going there again!

http://www.bfi.org.uk/whatson/bfi_imax

---------------
Sea Monsters 3D
at BFI IMAX

Saturday 02 February 2008 at 14:10

2 seats allocated
A Seated Layout H11 - H12 at 8.50 (ADULT)

Transaction Charge: 1.00
Total Cost: 18.0