Using log4j's FallbackErrorHandler

Our applications currently use a DailyRollingFileAppender for logging, but since they run on NFS across a number of different servers, we quite often get errors due to stale NFS file handles, when log4j tries to write to the files. We sometimes also get errors if the logging mount point is missing on some of the servers.

I've been trying to find a way to switch to a different appender (such as a ConsoleAppender), if log4j fails to write to the log files. At first I thought of writing my own custom appender, to wrap up a FileAppender and a ConsoleAppender, and to switch to the ConsoleAppender if the FileAppender threw an IOException, but then I came across the FallbackErrorHandler, which allows you to configure a backup appender, which takes over if the primary appender fails for whatever reason.

This is how you can set up your log4j.xml file to use a FallbackErrorHandler:

1. Create a backup appender:
The backup appender will be used if the primary appender fails. My backup is a ConsoleAppender:

  <appender name="console" class="org.apache.log4j.ConsoleAppender">
    <param name="Target" value="System.out"/>
    <layout class="org.apache.log4j.PatternLayout">
      <param name="ConversionPattern" value="%d %-5p %30.30c - %m%n"/>
    </layout>
  </appender>

2. Add a FallbackErrorHandler to your primary appender:
My primary appender is a DailyRollingFileAppender. Add a FallbackErrorHandler to it and tell it to use the "console" (backup) appender, using the appender-ref tag. The root-ref tag refers to the logger that is currently using that appender. If you have a different logger use the logger-ref tag to refer to it instead.

  <appender name="file" class="org.apache.log4j.DailyRollingFileAppender">
  <errorHandler class="org.apache.log4j.varia.FallbackErrorHandler">
       <root-ref/>
       <appender-ref ref="console"/>
  </errorHandler>
    <param name="File" value="C:/temp/test.log"/>
    <layout class="org.apache.log4j.PatternLayout">
      <param name="ConversionPattern" value="%d %-5p %30.30c - %m%n"/>
    </layout>
  </appender>

3. Trying it out:
To test this works, make your log file read-only, or change the path of the file to one which doesn't exist. When you run your application, you will see log4j print an error to stderr, and start logging to console, instead of file. If you turn log4j debug on you will see the message: "FB: INITIATING FALLBACK PROCEDURE." before console logging begins.

The complete log4j.xml configuration:
Here is my complete config file. (I tried setting up a log4j.properties file, but ran into problems and wasn't able to.)

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
    <appender name="console" class="org.apache.log4j.ConsoleAppender">
        <param name="Target" value="System.out" />
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%d %-5p %30.30c - %m%n" />
        </layout>
    </appender>
    <appender name="file" class="org.apache.log4j.DailyRollingFileAppender">
        <errorHandler class="org.apache.log4j.varia.FallbackErrorHandler">
            <root-ref />
            <appender-ref ref="console" />
        </errorHandler>
        <param name="File" value="C:/temp/test.log" />
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%d %-5p %30.30c - %m%n" />
        </layout>
    </appender>
    <root>
        <level value="INFO" />
        <appender-ref ref="file" />
    </root>
</log4j:configuration>

Upgrading to SyntaxHighlighter 2.0

This post is deprecated. Please read my new entry on: "Upgrading to SyntaxHighlighter 3.0"

I have now upgraded this blog to use SyntaxHighlighter 2.0. It was very easy. You don't need to make changes to any of your old posts, because this release is backwards compatible.

Another thing to note is that I had to make a change to shBrushBash.js as it wasn't formatting file redirect characters (>, <) correctly.

This is how you can upgrade too:

1. Download SyntaxHighlighter v2.0
You can download it here.
If you don't have a place to upload, you can link to my free hosted version here.

2. Link to CSS and Javascript
Open your webpage's HTML file and add links to the SyntaxHighlighter's CSS and JavaScript files. For optimal results, place these lines at the very end of your page, just before the closing body tag.

<link type="text/css" rel="stylesheet" href="http://sites.google.com/site/fahdshariff/syntaxhighlighter/styles/shCore.css"></link>
<link type="text/css" rel="stylesheet" href="http://sites.google.com/site/fahdshariff/syntaxhighlighter/styles/shThemeDefault.css"></link>
<script type="text/javascript" src="http://sites.google.com/site/fahdshariff/syntaxhighlighter/scripts/shCore.js"></script>
<script type="text/javascript" src="http://sites.google.com/site/fahdshariff/syntaxhighlighter/scripts/shLegacy.js"></script>
<script type="text/javascript" src="http://sites.google.com/site/fahdshariff/syntaxhighlighter/scripts/shBrushBash.js"></script>
<script type="text/javascript" src="http://sites.google.com/site/fahdshariff/syntaxhighlighter/scripts/shBrushJava.js"></script>
<script type="text/javascript" src="http://sites.google.com/site/fahdshariff/syntaxhighlighter/scripts/shBrushXml.js"></script>
<script type="text/javascript" src="http://sites.google.com/site/fahdshariff/syntaxhighlighter/scripts/shBrushSql.js"></script>
<script type="text/javascript">
    SyntaxHighlighter.config.bloggerMode = true;
    SyntaxHighlighter.config.clipboardSwf = 'http://sites.google.com/site/fahdshariff/syntaxhighlighter/scripts/clipboard.swf';
    SyntaxHighlighter.all();
    dp.SyntaxHighlighter.HighlightAll('code');
</script>

It is not necessary to add the js files for all languages - just for the ones you will be using.

3. Add Code
Now add the code you wish to highlight in your webpage, surrounded by the <pre> tag. Set the class attribute to the language alias e.g. brush:java:

<pre class="brush: java; gutter: false;">
public void printHello(){
    System.out.println("Hello World");
}
</pre>

4. View Page
View the webpage in your browser and you should see your syntax highlighted code snippet.

Links:
SyntaxHighlighter Home Page
Syntax Highlighting Code in Webpages (with SyntaxHighlighter 1.5)

Unix Wrapper Script for Java Applications

All of my Java applications are deployed onto a Unix (Linux or Solaris) environment and so I need to have a unix wrapper shell script in order to start and stop them. I looked at Java Service Wrapper but I wanted something simpler. So I wrote my own little wrapper, based on JSW.

The wrapper allows you to invoke the following set of commands on your application:

• start: Starts up the application and writes its pid to a pid file. it also writes the hostname to the pid file. This is important, in case you try to run commands on the application on a different server. After starting, it waits a number of seconds and then checks to see if the application is still running.
• stop: Stops the application using the kill command. If it has not died within 10 seconds, it executes the harder kill -9 command.
• restart: Restarts the application by calling stop and then start.
• status:Reads the pid file and invokes kill -0 on the pid to see if the process is alive. If the process is not alive, an email alert is generated, if the $EMAIL variable has been set.
• dump: Generates a full Java thread dump by invoking kill -3 on the pid
• purge: Purges log files

Here is the wrapper script (bin/console.ksh):

#! /bin/ksh
# A generic wrapper script to stop and start components.
# Usage: console.ksh { start | stop | restart | status | purge | dump }
#

APP_NAME="APP"
APP_LONG_NAME="MyApplication"

#the location of the pid files.
PIDDIR="../pid"
PIDFILE="$PIDDIR/$APP_NAME.pid"
TIMESTAMP=$( date +%Y%m%d_%H%M%S )

#number of seconds to wait after starting up.
WAIT_AFTER_STARTUP=2

#the location of the log files
LOG_DIR="../log"
STDOUT="$LOG_DIR/${APP_NAME}.stdout.log"

PURGE_EXPRESSION="-mtime +2"

VMARGS="-Xms250M -Xmx3000M -verbose:gc"
MAINCLASS="com.blogspot.fahdshariff.MyApplication"
ARGS="arg1 arg2"

if [ -z ${JAVA_HOME} ]
then
 echo JAVA_HOME has not been set.
 exit 1
fi

START_COMMAND="${JAVA_HOME}/bin/java $VMARGS $MAINCLASS $ARGS"

#-----------------------------------------------------------------------------
# Do not modify anything beyond this point
#-----------------------------------------------------------------------------

getpid() {
  pid=""
  if [ -f "$PIDFILE" ]
  then
    if [ -r "$PIDFILE" ]
    then
      pid=`cat "$PIDFILE"|cut -d: -f1`
      host=`cat "$PIDFILE"|cut -d: -f2`
      currhost=`hostname`
      if [ "$host" != "$currhost" ]
      then
       echo "You are on the wrong host. $APP_LONG_NAME runs on $host."
       exit 1
      fi

      if [ "X$pid" != "X" ]
      then
       kill -0 $pid > /dev/null 2>&1
  if [ $? -ne 0 ]
  then
   # Process doesn't exist, so remove pidfile
   rm -f "$PIDFILE"
            echo "Removed stale pid file: $PIDFILE"
   pid=""
  fi
      fi
    else
      echo "Cannot read $PIDFILE."
      exit 1
    fi
  fi
}

#tests if the pid is alive
testpid() {
  kill -0 $pid > /dev/null 2>&1
  if [ $? -ne 0 ]
  then
   # Process is gone so remove the pid file.
 rm -f "$PIDFILE"
 pid=""
  fi
}

start() {
  echo "Starting $APP_LONG_NAME..."
  getpid
  if [ "X$pid" = "X" ]
  then
    if [ -s $STDOUT ]
    then
     #backup current file
     mv $STDOUT $STDOUT.sav.${TIMESTAMP}
    fi
    echo CLASSPATH is $CLASSPATH > $STDOUT
    echo ${START_COMMAND} >> $STDOUT
    nohup ${START_COMMAND} >> $STDOUT 2>&1 &
    echo $!:`hostname` > $PIDFILE
  else
    echo "$APP_LONG_NAME is already running."
    exit 1
  fi

  # Sleep for a few seconds to allow for intialization if required
  # then test to make sure we're still running.
  i=0
  while [ $i -lt $WAIT_AFTER_STARTUP ]
  do
    sleep 1
    i=`expr $i + 1`
  done
  if [ $WAIT_AFTER_STARTUP -gt 0 ]
  then
    getpid
    if [ "X$pid" = "X" ]
    then
      echo "WARNING: $APP_LONG_NAME may have failed to start."
      exit 1
    else
      echo "running ($pid)."
    fi
  else
    echo ""
  fi
}


stopit() {

  echo "Stopping $APP_LONG_NAME..."
  getpid
  if [ "X$pid" = "X" ]
  then
    echo "$APP_LONG_NAME was not running."
  else
    # Running so try to stop it.
    kill $pid
    if [ $? -ne 0 ]
    then
      echo "Unable to stop $APP_LONG_NAME."
      exit 1
    fi

    #  If it has not stopped in 10 tries, forcibly kill it
    savepid=$pid
    CNT=0
    TOTCNT=0
    while [ "X$pid" != "X" ]
    do
      # Show a waiting message every 5 seconds.
      if [ "$CNT" -lt "5" ]
      then
        CNT=`expr $CNT + 1`
      else
        echo "Waiting for $APP_LONG_NAME to exit..."
        CNT=0
      fi

      if [ $TOTCNT -gt 11 ]
      then
       echo "Killing by force (kill -9)"
       kill -9 $pid
      fi

      TOTCNT=`expr $TOTCNT + 1`
      sleep 1

      testpid
    done

    pid=$savepid
    testpid
    if [ "X$pid" != "X" ]
    then
      echo "Failed to stop $APP_LONG_NAME."
      exit 1
    else
      echo "Stopped $APP_LONG_NAME."
    fi
  fi
}

purge() {
  pcmd="find $LOG_DIR -follow -type f $PURGE_EXPRESSION -print -exec rm -f {} +"
  echo "Running purge: $pcmd"
  $pcmd
}

alert() {
  if [ "X$EMAIL" != "X" ]
  then
   echo "$APP_LONG_NAME is not running on `hostname`. Please check and restart." \
   | mailx -s "`hostname`: $APP_LONG_NAME is not running" "$EMAIL"
   echo "Sent alert to $EMAIL"
  fi
}

status() {
  getpid
  if [ "X$pid" = "X" ]
  then
    echo "$APP_LONG_NAME is not running."
    alert
    exit 1
  else
    echo "$APP_LONG_NAME is running (PID:$pid)."
    exit 0
  fi
}

dump() {
  echo "Dumping $APP_LONG_NAME..."
  getpid
  if [ "X$pid" = "X" ]
  then
    echo "$APP_LONG_NAME is not running."
  else
    kill -3 $pid

    if [ $? -ne 0 ]
    then
      echo "Failed to dump $APP_LONG_NAME."
      exit 1
    else
      echo "Dumped $APP_LONG_NAME."
    fi
  fi
}

####################

case "$1" in

  'start')
    start
    ;;

  'stop')
    stopit
    ;;

  'restart')
    stopit
    start
    ;;

  'status')
    status
    ;;

  'dump')
    dump
    ;;

  'purge')
    purge
    ;;

  *)
    echo "Usage: $0 { start | stop | restart | status | dump | purge }"
    exit 1
    ;;
esac

exit 0

Bash: Convert String to Array

This is how you can convert a string into an array:

s="124890"
for i in $(seq 0 $((${#s}-1)))
do
     arr[$i]=${s:$i:1}
done

${#s} refers to the length of the string s, which in this case is 6.
${s:$i:1} returns a substring of s, starting from $i, of length 1.

The seq command is a new one for me. Unfortunately, it doesn't seem to be available on Solaris, only Linux. From the man pages:

NAME
       seq - print a sequence of numbers

SYNOPSIS
       seq [OPTION]... LAST
       seq [OPTION]... FIRST LAST
       seq [OPTION]... FIRST INCREMENT LAST

To see how you can convert a string of digits to an array in other languages, read this stackoverflow question.

Double Brace Initialisation

"Double Brace Initialisation" is one of the lesser known features in Java. It can be useful for initialising collections that have to be passed to other methods.

For example, without double-brace initialisation:

Map<String,String> map = new HashMap <String,String>();
map.put("key","value");
map.put("key2","value2");
printMap(map);

With double-brace initialisation:

printMap(new HashMap <String,String>(){{
            put("key","value");
            put("key2","value2");
        }});

The first brace creates a new AnonymousInnerClass, the second declares an instance initializer block that is run when the anonymous inner class is instantiated. This only works only for non-final classes because it creates an anonymous subclass.

Don't go too overboard with this because it creates a new anonymous inner class just for making a single object which might not be very efficient! It will also create an additional class file on disk.

Another example:

//double-brace initialisation
List<String> list = new ArrayList<String>(){{
   add("apple");
   add("banana");
}};

//better alternative
List<String> list2 = Arrays.asList("apple","banana");