Friday, June 26, 2009

Fibonacci Shell Script

Here is a quick unix shell script which prints out the Fibonacci sequence:

0,1,1,2,3,5,8,13,21,34,55,89,144,...

The first two Fibonacci numbers are 0 and 1, and each remaining number is the sum of the previous two.

#!/bin/sh
prev=0
next=1

echo $prev

while(true)
do
 echo $next

 #add the two numbers
 sum=$(($prev+$next))

 #swap
 prev=$next
 next=$sum
 sleep 1
done

Monday, June 01, 2009

Using XPath in Java

Given the following xml document:
<hosts>
  <host name="starship" port="8080"/>
  <host name="firefly" port="8180"/>
</hosts>
this is how you can use the javax.xml.xpath library to run an XPath query in order to obtain a list of host names:
//create a document
DocumentBuilderFactory domFactory = 
                     DocumentBuilderFactory.newInstance();
domFactory.setNamespaceAware(true);
DocumentBuilder builder = domFactory.newDocumentBuilder();
Document doc = builder.parse("file.xml");

//create the xpath expression
XPathFactory factory = XPathFactory.newInstance();
XPath xpath = factory.newXPath();
XPathExpression expr = xpath.compile("//host/@name");

//run the xpath query
Object result = expr.evaluate(doc, XPathConstants.NODESET);

//read results
NodeList nodes = (NodeList) result;
for (int i = 0; i < nodes.getLength(); i++) {
    System.out.println(nodes.item(i).getNodeValue());
}

Thursday, May 21, 2009

find -exec vs xargs

If you want to execute a command on lots of files found by the find command, there are a few different ways this can be achieved (some more efficient than others):

-exec command {} \;
This is the traditional way. The end of the command must be punctuated by an escaped semicolon. The command argument {} is replaced by the current path name found by find. Here is a simple command which echoes file paths.

sharfah@starship:~> find . -type f -exec echo {} \;
.
./1.txt
./2.txt
This is very inefficient, because whenever find finds a file, it forks a process for your command, waits for this child process to complete and then searches for the next file. In this example, you will get the following child processes: echo .; echo ./1.txt; echo ./2.txt. So if there are 1000 files, there are 1000 child processes and find waits.

-exec command {} +
If you use a plus (+) instead of the escaped semicolon, the arguments will be grouped together before being passed to the command. The arguments must be at the end of the command.

sharfah@starship:~> find . -type f -exec echo {} +
. ./1.txt ./2.txt
In this case, only one child process is created: echo . ./1.txt ./2.txt, which is much more efficient, because it avoids a fork/exec for each single argument.

xargs
This is similar to the approach above, in that files found are bundled up (usually in batches of about 20-50 names) and sent to the command as few times as possible. find doesn't wait for your command to finish.

sharfah@starship:~> find . -type f | xargs echo
. ./1.txt ./2.txt
This approach is efficient and works well as long as you do not have funny characters (e.g. spaces) in your filenames as they won't be escaped.

Performance Testing
So which one of the above approaches is fastest? I ran a test across a directory with 10,000 files out of which 5,600 matched my find pattern. I ran the test 10 times, changing the order of the finds each time, but the results were always the same. xargs and + were very close, with \; always finishing last. Here is one result:

time find . -name "*20090430*" -exec touch {} +
real    0m31.98s
user    0m0.06s
sys     0m0.49s

time find . -name "*20090430*" | xargs touch
real    1m8.81s
user    0m0.13s
sys     0m1.07s

time find . -name "*20090430*" -exec touch {} \;
real    1m42.53s
user    0m0.17s
sys     0m2.42s
I'm going to be using the -exec command {} + method, because it is faster and can handle my funny filenames.

Saturday, May 09, 2009

Percent Sign in Crontab

From the man pages of crontab:

The sixth field of a line in a crontab file is a string that is executed by the shell at the specified times. A percent character in this field (unless escaped by \) is translated to a NEWLINE character.

Only the first line (up to a `%' or end of line) of the command field is executed by the shell. Other lines are made available to the command as standard input. Any blank line or line beginning with a `#' is a comment and is ignored.

This means that you need to escape any percent (%) characters. For example, I have a daily backup cron which writes the current crontab to a backup file every morning, and I have to escape the date command, as shown below:

01 07 * * * crontab -l > /home/user/cron.`date +\%Y\%m\%d`
Also note, that cron isn't clever enough to expand the tilde (~) character, so always use the full path to your home directory.

If you find that a cron hasn't fired, check your email in /var/mail/user.

Friday, May 08, 2009

Solaris - CPU, Memory and Version

CPU Info:
In order to find information about processors on Solaris, use the psrinfo command:
sharfah@starship:~> psrinfo -v
Status of virtual processor 0 as of: 05/08/2009 09:53:17
  on-line since 05/03/2009 00:05:06.
  The i386 processor operates at 2612 MHz,
        and has an i387 compatible floating point processor.
Status of virtual processor 1 as of: 05/08/2009 09:53:17
  on-line since 05/03/2009 00:05:12.
  The i386 processor operates at 2612 MHz,
        and has an i387 compatible floating point processor.
Status of virtual processor 2 as of: 05/08/2009 09:53:17
  on-line since 05/03/2009 00:05:14.
  The i386 processor operates at 2612 MHz,
        and has an i387 compatible floating point processor.
Status of virtual processor 3 as of: 05/08/2009 09:53:17
  on-line since 05/03/2009 00:05:16.
  The i386 processor operates at 2612 MHz,
        and has an i387 compatible floating point processor.
Status of virtual processor 4 as of: 05/08/2009 09:53:17
  on-line since 05/03/2009 00:05:18.
  The i386 processor operates at 2612 MHz,
        and has an i387 compatible floating point processor.
Status of virtual processor 5 as of: 05/08/2009 09:53:17
  on-line since 05/03/2009 00:05:20.
  The i386 processor operates at 2612 MHz,
        and has an i387 compatible floating point processor.
Status of virtual processor 6 as of: 05/08/2009 09:53:17
  on-line since 05/03/2009 00:05:22.
  The i386 processor operates at 2612 MHz,
        and has an i387 compatible floating point processor.
Status of virtual processor 7 as of: 05/08/2009 09:53:17
  on-line since 05/03/2009 00:05:24.
  The i386 processor operates at 2612 MHz,
        and has an i387 compatible floating point processor.
Memory Info:
In order to find out how much physical memory is installed, use prtconf:
sharfah@starship:~> prtconf | grep Memory
Memory size: 65536 Megabytes
Version Info:
To show machine, software revision and patch revision information use the showrev command:
sharfah@starship:~> showrev
Hostname: starship
Hostid: 80f32709
Release: 5.10
Kernel architecture: i86pc
Application architecture: i386
Hardware provider:
Kernel version: SunOS 5.10 Generic_137112-06
sharfah@starship:~> uname -a
SunOS starship 5.10 Generic_137112-06 i86pc i386 i86pc
Processes:
In order to list the processes running, use prstat (equivalent to top).
sharfah@starship:~> prstat
  PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP
 4049 sharfah   1008K  840K sleep    0    0   0:03.17 0.3% find/1
14632 sharfah      114M   68M sleep   29   10   1:19.18 0.1% java/30
Related posts:
Linux - CPU, Memory and Version