Wednesday, July 16, 2008

Big Strings and Oracle Clobs

Oracle clobs are funny things. I've been trying to store a really long string as a clob into my Oracle database via JDBC and it's been a nightmare! In this post, I will describe all of the different approaches I tried and finish with the one which finally worked!

Setup:

String Size7631 bytes
Database versionOracle9i Enterprise Edition Release 9.2.0.8.0
Oracle NLS_CHARACTERSETWE8ISO8859P1
JDBC driver versionOracle Database 10g Release 2 (10.2.0.4)
Table Column DataTypeNCLOB
Attempt 1: SetBigStringTryClob
In Oracle JDBC 10g, there is a new Connection property called SetBigStringTryClob which allows the statement's setString method to process strings greater than 32765 bytes.
Properties props = new Properties();
props.put("user", "username" );
props.put("password", "password");
props.put("SetBigStringTryClob", "true");

Connection conn = DriverManager.getConnection(dbUrl,props);
PreparedStatement st = conn.prepareStatement(INSERT_SQL);
st.setString(1, bigString);
st.executeUpdate();
This attempt failed - it inserted garbage (lots of inverted question marks and other funny characters) in my clob column.

Attempt 2: setStringForClob
The Oracle specific method of setStringForClob can be used for binding data greater than 32765 bytes. Note that this method is called internally if you call setString and have SetBigStringTryClob set to true (as in Attempt 1).

OraclePreparedStatement st = (OraclePreparedStatement)
          conn.prepareStatement(INSERT_SQL);
st.setStringForClob(1, bigString) ;
This attempt failed with the same result as previous one - it inserted garbage (lots of inverted question marks and other funny characters) in my Clob column.

Attempt 3: setCLOB
Create a temporary Oracle CLOB, populate it and call setClob.

CLOB clob = CLOB.createTemporary(conn,
  true,
  oracle.sql.CLOB.DURATION_SESSION,
  Const.NCHAR);
clob.trim(0);
Writer writer = clob.getCharacterOutputStream();
writer.write(bigString.toCharArray());
writer.flush();
writer.close();
st.setClob(1, clob);
This attempt failed with: ORA-12704: character set mismatch

Attempt 4: setCharacterStream
Use setCharacterStream to get a stream to write characters to the clob.

Reader reader = new StringReader(bigString);
int readerLength = bigString.toCharArray().length;
st.setCharacterStream(1, reader, readerLength);
Failed - garbage inserted again!

Attempt 5: Insert an empty_clob() and then update it
Insert an empty_clob() into the table, retrieve the locator, and then write the data to the clob.

String sql = "INSERT INTO clob_table (clob_col) "+
             "VALUES (empty_clob())";
PreparedStatement st = conn.prepareStatement(sql);
st.executeUpdate() ;

sql = "SELECT clob_col FROM clob_table FOR UPDATE";
st = conn.prepareStatement(sql);
ResultSet rs = st.executeQuery();
rs.next();
Clob clob = rs.getClob(1);
Writer writer = clob.setCharacterStream(0);
writer.write(bigString.toCharArray());
writer.flush();
writer.close();
rs.close();
Success! However, I'm not happy with the two database calls; one to create the empty clob and the other to update it. There must be a better way!

Attempt 6: PL/SQL
Wrap the SQL insert statement in PL/SQL to work around the size limitation.

INSERT_SQL = "BEGIN INSERT INTO clob_table (clob_col) "+
             "VALUES (?); END";
st = conn.prepareStatement(sql);
st.setString(1, bigString);
st.executeUpdate();
Success! And with only one database call!
Note, that setString can only process strings of less than 32766 chararacters, so if your String is bigger than this, you should use the empty_clob() technique from Attempt 5.

Phew! After six attempts, I've finally found two which work. Why does this have to be so complicated?!

References:
Oracle JDBC FAQ
Oracle 10g JDBC API
Using Oracle 10g drivers to solve the 4000 character limitation
Handling CLOBs - Made easy with Oracle JDBC 10g

Thursday, July 10, 2008

Eclipse Ganymede

Eclipse Ganymede is the annual release of Eclipse projects; this year including 23 projects. Some highlights of the release include the new p2 provisioning platform, new Equinox security features, new Ecore modeling tools, support for SOA and much more.

I have been using the new Eclipse Ganymede release for about two weeks now and think its time to write a review and share some of my experiences with the rest of the developer community. On the whole, I think the new release is as snappy as previous versions and brings some welcome improvements. Here are some of the features that I really like in this release:

Breadcrumbs:
The editor now displays a breadcrumb navigation bar showing the path to the current file. You can easily access the project and package structure as well as the individual classes, fields and methods from the bar itself. I really like this feature because I no longer have to keep my Package Explorer and Outline views open, thus saving on precious screen estate.

Enhanced Hover:
In the previous version, you had to use the awkward keyboard shortcut of Ctrl+1 on top of an Error or Warning to get the Quick Fix options. In Ganymede, all you have to do is hover over the problem and it pops up a window with links to the options. Great!

Highlight Read/Write Variables:
A small but useful addition to this release is that Mark Occurrences (Alt+Shift+O) now marks read and write accesses with different colours. Previously, Eclipse offered the ability to highlight all occurrences of a variable but now it distinguishes between read and writes for you.

Call Hierarchy for Fields and Types:
This allows you to find all the members which access the field (for read or write) and the constructors of a type. Previously, you could only find callers of a method.

Support for External Class Folders
Class folders located outside the workspace (external) can now be added to the build path. Previously, I had to build a jar file and then add it using "Add External JARs" so this is a welcome improvement! You can even add other kinds of zip archives to the build path e.g. RAR files.

Performance:
To be honest, I haven't noticed any increase or decrease in either the start-up time or interaction with the user interface. However, there has reportedly been a lot of internal improvement to the JDT compiler so that compilation can now be spread across multiple cores rather than being able to utilise only one. This should speed up build times.

There are still many more things for me to try out. In particular I would really like to try out the Test and Performance Tools Platform Project.

Got your own experiences with Ganymede? Share them in the comments.

For more details:
What's New in 3.4 (JDT)
Learn more about Ganymede

Friday, July 04, 2008

Write a Search Plugin for Firefox [Howto]

Firefox 3 currently supports plugins in two specifications. These are Apple's Sherlock format and the more recent OpenSearch syntax which is now preferred as it is supported by both Firefox (2+) and Internet Explorer (7+). In this post, I will show you how to create your own search engine plugin using the OpenSearch syntax.

1. Pick a search engine you want to create a plugin for. For this example, I have chosen Picitup which is a new image search engine (still in beta). It is not listed on Mycroft Search Engine Plugins either, so it is likely that it doesn't exist yet.

2. Find your searchplugins directory. On Windows XP, this will typically be in

%APPDATA%\Mozilla\Firefox\Profiles\xxxxxxxx.default\searchplugins
where xxxx is a random string.

3. Create a new file in the searchplugins directory called picitup.xml with the contents below:

The icon file is usually called favicon.ico and is found on the level of the homepage. However, in this case it is present in a different directory, which I found by viewing the HTML source of the Picitup homepage:

<link rel="SHORTCUT ICON" href="images/plusico/fav.ico">

To find the search URL, look at the HTML <form> element:

<form name="search" action="results.jsp" method="get">
  <input input type=text name=query size="38" style="border: 1px solid #BDBDBD;"/>
  <input type="hidden" name="searchType" value="image"/>
  <input type="hidden" name="addToDb" value="1"/>
  <input type="submit" value="Search" onclick="javascript:submitForm();"/>
</form>

4. Restart Firefox and thats it! Click on the Search Engine combo box and you should now be able to try out the new Picitup addon.

5. Share your plugin with the rest of the world by uploading it here: http://mycroft.mozdev.org/submitos.html

NB: After restart, Firefox will change you xml file by base64 encoding the image icon URL.

Reference:
There is detailed documentation about the OpenSearch format at OpenSearch.org and with reference to Firefox at the Mozilla Developer Center.

Thursday, July 03, 2008

Firefox Sets Record 8,002,530 Downloads

Got an email from the Firefox guys today, confirming that they have indeed set a Guinness World Record for the most software downloads in 24 hours! Email quoted below:
We did it!

We set a Guinness World Record for the most software downloads in 24 hours. With your help we reached 8,002,530 downloads.

You are now part of a World Record and the proud owner of the best version of Firefox yet!

Don't forget to download your very own certificate for helping set a Guinness World Record.

View my certificate

Wednesday, July 02, 2008

Send Email Alerts Using Log4j [Howto]

It's really easy to add error alerting to your java application. If your application logs all errors using log4j and you want these errors emailed out to a support team or to yourself, then all you have to do is add another appender to your log4j.properties file.

Let's say you have the following class which logs an error message: And your log4j properties file is: Now add a new MAIL appender, so that your properties file looks like this: Don't forget to place mail.jar and activation.jar on your classpath and then run the application.

Check your inbox and voila, you have an alert without making any source code changes!

NB: By default, an email message will be sent when an ERROR or higher severity message is appended. If you want to email messages with levels less than ERROR (e.g. INFO) then you currently have to configure your own implementation of the TriggeringEventEvaluator. Setting log4j.appender.MAIL.Threshold=INFO will not work.

Further Reading:
SMTPAppender javadoc
SmtpAppender Members and Properties