Showing posts with label XML. Show all posts
Showing posts with label XML. Show all posts

Saturday, October 07, 2017

Using XSLT 2.0 with Java and Saxon

In my previous post, I showed how you can split a string using the tokenize function in XSLT 2.0. In order to run an XSLT 2.0 stylesheet in Java, you need a transformer that supports XSLT 2.0. Unfortunately, the default JAXP transformer (com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl) does not. Instead, use Saxon, which supports both XSLT 2.0 and 3.0.

The class below shows how you can perform an XSL transformation in Java, using Saxon and the JAXP interface:

import java.io.StringReader;
import java.io.StringWriter;

import javax.xml.transform.Templates;
import javax.xml.transform.Transformer;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;

public class XSLTransformer {

  private final Templates templates;

  public XSLTransformer(final String xslFileName) throws Exception {
    templates = new net.sf.saxon.BasicTransformerFactory().newTemplates(
        new StreamSource(XSLTransformer.class.getClassLoader()
                             .getResourceAsStream(xslFileName)));
  }

  public String transform(final String xml) throws Exception {
    final Transformer transformer = templates.newTransformer();
    final StringWriter writer = new StringWriter();
    transformer.transform(new StreamSource(new StringReader(xml)),
                          new StreamResult(writer));
    return writer.toString();
  }
}

An alternative to the JAXP interface is to use Saxon's own s9api interface, which is more robust:

import java.io.StringReader;
import java.io.StringWriter;

import javax.xml.transform.stream.StreamSource;

import net.sf.saxon.s9api.Processor;
import net.sf.saxon.s9api.SaxonApiException;
import net.sf.saxon.s9api.XsltExecutable;
import net.sf.saxon.s9api.XsltTransformer;

public class SaxonTransformer {

  private final Processor processor;
  private final XsltExecutable xsltExec;

  public SaxonTransformer(final String xslFileName) throws SaxonApiException {
    processor = new Processor(false);
    xsltExec = processor.newXsltCompiler().compile(new StreamSource(
        XSLTransformer.class.getClassLoader().getResourceAsStream(xslFileName)));
  }

  public String transform(final String xml) throws Exception {
    final XsltTransformer transformer = xsltExec.load();
    transformer.setSource(new StreamSource(new StringReader(xml)));
    final StringWriter writer = new StringWriter();
    transformer.setDestination(processor.newSerializer(writer));
    transformer.transform();
    return writer.toString();
  }
}

Splitting a string using XSLT 2.0

The tokenize function available in XSL Transformations (XSLT) Version 2.0 allows you to split a string on any separator that matches a given regular expression.

The example below shows how you can split a comma-delimited string:

Input XML:

<data>
  <stringToSplit>foo,bar,baz</stringToSplit>
</data>

XSL 2.0 Stylesheet:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:template match="data">
    <items>
      <xsl:variable name="stringToSplit" select="stringToSplit" />
      <xsl:for-each select="tokenize($stringToSplit, ',')">
        <item>
          <xsl:value-of select="." />
        </item>
      </xsl:for-each>
    </items>
  </xsl:template>
</xsl:stylesheet>

Output XML:

<?xml version="1.0" encoding="UTF-8"?>
<items>
  <item>foo</item>
  <item>bar</item>
  <item>baz</item>
</items>

Saturday, July 19, 2014

Converting XML to CSV using XSLT 1.0

This post shows you how can convert a simple XML file to CSV using XSLT.

Consider the following sample XML:

<library>
  <book>
    <author>Dan Simmons</author>
    <title>Hyperion</title>
    <publishDate>1989</publishDate>
  </book>
  <book>
    <author>Douglas Adams</author>
    <title>The Hitchhiker's Guide to the Galaxy</title>
    <publishDate>1979</publishDate>
  </book>
</library>

This is the desired CSV output:

author,title,publishDate
Dan Simmons,Hyperion,1989
Douglas Adams,The Hitchhiker's Guide to the Galaxy,1979

The following XSL Style Sheet (compatible with XSLT 1.0) can be used to transform the XML into CSV. It is quite generic and can easily be configured to handle different xml elements by changing the list of fields defined ar the beginning.

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method="text" />

  <xsl:variable name="delimiter" select="','" />

  <!-- define an array containing the fields we are interested in -->
  <xsl:variable name="fieldArray">
    <field>author</field>
    <field>title</field>
    <field>publishDate</field>
  </xsl:variable>
  <xsl:param name="fields" select="document('')/*/xsl:variable[@name='fieldArray']/*" />

  <xsl:template match="/">

    <!-- output the header row -->
    <xsl:for-each select="$fields">
      <xsl:if test="position() != 1">
        <xsl:value-of select="$delimiter"/>
      </xsl:if>
      <xsl:value-of select="." />
    </xsl:for-each>

    <!-- output newline -->
    <xsl:text>&#xa;</xsl:text>

    <xsl:apply-templates select="library/book"/>
  </xsl:template>

  <xsl:template match="book">
    <xsl:variable name="currNode" select="." />

    <!-- output the data row -->
    <!-- loop over the field names and find the value of each one in the xml -->
    <xsl:for-each select="$fields">
      <xsl:if test="position() != 1">
        <xsl:value-of select="$delimiter"/>
      </xsl:if>
      <xsl:value-of select="$currNode/*[name() = current()]" />
    </xsl:for-each>

    <!-- output newline -->
    <xsl:text>&#xa;</xsl:text>
  </xsl:template>
</xsl:stylesheet>

Let's try it out:

$ xsltproc xml2csv.xsl books.xml
author,title,publishDate
Dan Simmons,Hyperion,1989
Douglas Adams,The Hitchhiker's Guide to the Galaxy,1979

Saturday, May 25, 2013

JAXB: Marshalling/Unmarshalling Example

This post shows how you can marshal a JAXB object into XML and unmarshal XML into a JAXB object.

Consider the following JAXB class:

import javax.xml.bind.annotation.*;

@XmlRootElement
public class Book {

  @XmlElement
  private String author;

  @XmlElement
  private String title;
}
Unmarshalling:
To convert an XML string into an object of class Book:
public static Book unmarshal(final String xml) throws JAXBException {
  return (Book) JAXBContext.newInstance(Book.class)
                           .createUnmarshaller()
                           .unmarshal(new StringReader(xml));
}
Marshalling:
To convert a Book object into an XML string:
public static String marshal(Book book) throws JAXBException {
  final Marshaller m = JAXBContext.newInstance(Book.class)
                                  .createMarshaller();
  m.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
  final StringWriter w = new StringWriter();
  m.marshal(book, w);
  return w.toString();
}

Saturday, December 03, 2011

Using XStream to Map a Single Element

Let's say you have the following XML which has a single element containing an attribute and some text:
<error code="99">This is an error message</error>
and you would like to convert it, using XStream, into an Error object:
public class Error {
    String message;
    int code;

    public String getMessage() {
        return message;
    }

    public int getCode() {
        return code;
    }
}
It took me a while to figure this out. It was easy getting the code attribute set in the Error object, but it just wasn't picking up the message.

Eventually, I found the ToAttributedValueConverter class which "supports the definition of one field member that will be written as value and all other field members are written as attributes."

The following code shows how the ToAttributedValueConverter is used. You specify which instance variable maps to the value of the XML element (in this case message). All other instance variables automatically map to attributes, so you don't need to explicitly annotate them with XStreamAsAttribute.

@XStreamAlias("error")
@XStreamConverter(value=ToAttributedValueConverter.class, strings={"message"})
public class Error {

  String message;

  @XStreamAlias("code")
  int code;

  public String getMessage() {
      return message;
  }

  public int getCode() {
      return code;
  }

  public static void main(String[] args) {
      XStream xStream = new XStream();
      xStream.processAnnotations(Error.class);

      String xmlResponse="<error code=\"99\">This is an error message</error>";

      Error error = (Error)xStream.fromXML(xmlResponse);
      System.out.println(error.getCode());
      System.out.println(error.getMessage());
  }
}

Sunday, October 16, 2011

Validating XML with xmllint

The following commands show you how to validate an XML file against a DTD or XSD using xmllint.

To validate an XML file against:

  • a DTD stored in the same file:
  • xmllint --valid --noout fileWithDTD.xml
    
  • a DTD stored in a separate file:
  • xmllint --dtdvalid DTD.dtd --noout fileWithoutDTD.xml
    
  • an XSD stored in a separate file:
  • xmllint --schema schema.xsd --noout file.xml
    
The --noout option suppresses the output of the xml file.

Example
To validate:

<countries>
  <country name="Afghanistan" population="22664136" area="647500">
    <language percentage="11">Turkic</language>
    <language percentage="35">Pashtu</language>
    <language percentage="50">Afghan Persian</language>
  </country>
  <country name="Albania" population="3249136" area="28750"/>
  <country name="Algeria" population="29183032" area="2381740">
    <city>
      <name>Algiers</name>
      <population>1507241</population>
    </city>
  </country>
</countries>
against:
<!ELEMENT countries (country*)>
<!ELEMENT country (language|city)*>
<!ATTLIST country name CDATA #REQUIRED>
<!ATTLIST country population CDATA #REQUIRED>
<!ATTLIST country area CDATA #REQUIRED>
<!ELEMENT language (#PCDATA)>
<!ATTLIST language percentage CDATA #REQUIRED>
<!ELEMENT city (name, population)>
<!ELEMENT name (#PCDATA)>
<!ELEMENT population (#PCDATA)>
use the command:
xmllint --dtdvalid countries.dtd --noout countries.xml

Sunday, February 20, 2011

XPaths with xmllint

xmllint is a command-line XML tool used to validate and pretty-print XML documents. More importantly, it offers an interactive shell mode which allows you to use xpaths to print out elements. For example, //body will print out the body element of an HTML document.

I wrote a useful bash function, which uses xmllint to evaluate xpaths really easily:

xpath()
{
    if [ $# -ne 2]; then
        echo "Usage: xpath xpath file"
        return 1
    fi
    xmllint --shell $2 <<< "cat $1" | sed '/^\/ >/d'
}
Example:
sharfah@starship:~> xpath "//body" index.html
<body>Hello World!</body>

Saturday, October 30, 2010

Formatting XML quickly

There are many times when I have come across badly formatted XML and need to prettify it instantly in order to aid readability or simply to paste into another document, like a blog post. There are plugins available (for example, Textpad's XMLTidy) which tidy up xml, but they involve pasting XML into a file and running a macro to clean it up, which can be slow, especially if you don't have an editor open.

So, I decided to write my own Java utility to format XML instantly. All you have to do is select some XML text, hit CTRL+C to save it to your clipboard, hit CTRL+ALT+F to invoke my formatting utility and finally hit CTRL+V to paste the nicely formatted XML somewhere else. This has made working with XML so much easier!

This is how you can set it up too:

The Java Source Code
Save the following source code to a file called XMLTidy.java and compile it using javac.

import java.awt.Toolkit;
import java.awt.datatransfer.DataFlavor;
import java.awt.datatransfer.StringSelection;
import java.awt.datatransfer.Transferable;
import java.awt.datatransfer.UnsupportedFlavorException;
import java.io.IOException;
import java.io.StringReader;
import java.io.StringWriter;
import java.io.Writer;

import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;

import org.w3c.dom.Document;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;

import com.sun.org.apache.xml.internal.serialize.OutputFormat;
import com.sun.org.apache.xml.internal.serialize.XMLSerializer;

/**
 * A useful utility for formatting xml.
 * Retrieves xml text from the system clipboard, formats it
 * and resaves it to the clipboard.
 */
public class XMLTidy {


  /**
   * Formats the specified xml string
   *
   * @param src the xml text to format
   * @return formatted xml
   * @throws ParserConfigurationException
   * @throws SAXException
   * @throws IOException
   */
  private static String tidyXml(String src)
      throws ParserConfigurationException, SAXException, IOException {
    DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
    DocumentBuilder db = dbf.newDocumentBuilder();
    InputSource is = new InputSource(new StringReader(src));
    Document document = db.parse(is);
    OutputFormat format = new OutputFormat(document);
    format.setLineWidth(65);
    format.setIndenting(true);
    format.setIndent(2);
    Writer out = new StringWriter();
    XMLSerializer serializer = new XMLSerializer(out, format);
    serializer.serialize(document);
    return out.toString();
  }

  /**
   * @return the text in the clipboard
   */
  private static String getClipboard() {
    Transferable t = Toolkit.getDefaultToolkit().getSystemClipboard()
        .getContents(null);
    try {
      if (t != null &&
          t.isDataFlavorSupported(DataFlavor.stringFlavor)) {
        String text = (String) t.getTransferData(DataFlavor.stringFlavor);
        return text;
      }
    } catch (UnsupportedFlavorException e) {
      e.printStackTrace();
    } catch (IOException e) {
      e.printStackTrace();
    }
    return "";
  }

  /**
   * @param str the text to set in the clipboard
   */
  private static void setClipboard(String str) {
    StringSelection ss = new StringSelection(str);
    Toolkit.getDefaultToolkit().getSystemClipboard().setContents(ss, null);
  }


  /**
   * Formats the xml supplied as an argument.
   * If no arguments are specified, formats the xml
   * in the clipboard.
   * @param args
   * @throws Exception
   */
  public static void main(String[] args) throws Exception {
    String in = args.length > 0 ? args[0] : getClipboard();
    if (in != null) {
      in = in.trim();
      if (in.charAt(0) == '<') {
        setClipboard(tidyXml(in));
      }
    }
  }
}
The Launcher Script
Create a bat file to launch the java program:
@echo off
%JAVA_HOME%\bin\java -cp \path\to\XMLTidy\classes XMLTidy %1
The Keyboard Shortcut
Finally create a keyboard shortcut to the launcher script as follows:
  • First, create a shortcut to the launcher script, by right-clicking the bat file and selecting "Create a shortcut".
  • Right-click the shortcut file and select "Properties".
  • Enter a "Shortcut key" on the Shortcut tab. For example, the shortcut key I use is CTRL+ALT+F
Try it out!
  • Select some badly formatted XML and copy it (using CTRL+C, for example).
  • Invoke XMLTidy by using the keyboard shortcut, CTRL+ALT+F.
  • Paste the XML (using CTRL+V, for example). The XML will be nicely formatted!

Saturday, August 21, 2010

Faster XPaths with VTD-XML

I've recently started using VTD-XML for applying XPaths on large XML documents. DOM is a memory hog and is too slow. However, VTD-XML allows you to run XPaths and provides random access to nodes, similar to DOM, but much more efficiently. You can't apply XPaths with a SAX parser nor can you access nodes randomly or traverse the document easily.

VTD-XML was 60 times faster compared to DOM when processing my XML document (20MB).

This post shows you how to use VTD-XML for fast XPath evaluation.

Sample XML:
I will use the following XML document in the examples below.

<?xml version="1.0"?>
<catalog>
 <book id="bk101">
  <author>Gambardella, Matthew</author>
  <author>Doe, John</author>
  <title>XML Developer's Guide</title>
  <genre>Computer</genre>
  <price>44.95</price>
  <publish_date>2000-10-01</publish_date>
 </book>
 <book id="bk102">
  <author>Ralls, Kim</author>
  <title>Midnight Rain</title>
  <genre>Fantasy</genre>
  <price>5.95</price>
  <publish_date>2000-12-16</publish_date>
 </book>
 <book id="bk103">
  <author>Corets, Eva</author>
  <title>Maeve Ascendant</title>
  <genre>Fantasy</genre>
  <price>5.95</price>
  <publish_date>2000-11-17</publish_date>
 </book>
</catalog>
Loading the XML document:
The following code parses the XML file and creates the navigator and autopilot objects.
final VTDGen vg = new VTDGen();
vg.parseFile("books.xml", false);
final VTDNav vn = vg.getNav();
final AutoPilot ap = new AutoPilot(vn);
Selecting all titles:
Print out all the title nodes using an XPath expression of /catalog/book/title. First call selectXPath to compile the expression and then use evalXPath to move the cursor to the selected nodes in the result.
ap.selectXPath("/catalog/book/title");
while (ap.evalXPath() != -1) {
  int val = vn.getText();
  if (val != -1) {
    String title = vn.toNormalizedString(val);
    System.out.println(title);
  }
}
Selecting all book ids and authors:
This one is a bit more involved as a book can have many authors. In the code below, I first run an XPath to select the books and then iterate over the children, selecting the author nodes.
ap.selectXPath("/catalog/book");
while (ap.evalXPath() != -1) {
  int val = vn.getAttrVal("id");
  if(val != -1){
    String id = vn.toNormalizedString(val);
    System.out.println("Book id: " + id);
  }

  if(vn.toElement(VTDNav.FIRST_CHILD,"author")){
    do{
      val = vn.getText();
      if(val != -1){
        String author = vn.toNormalizedString(val);
        System.out.println("\tAuthor:" + author);
      }
    }while(vn.toElement(VTDNav.NEXT_SIBLING,"author"));
  }
  vn.toElement(VTDNav.PARENT);
}
The output is:
Book id: bk101
 Author:Gambardella, Matthew
 Author:Doe, John
Book id: bk102
 Author:Ralls, Kim
Book id: bk103
 Author:Corets, Eva
Related Posts:
Using XPath with DOM

Wednesday, February 03, 2010

XMLBeans for Handling XML in Java

XMLBeans is a useful library which allows you to convert XML files into Java objects, call methods on the objects and then write the objects back to XML. This post shows you how you can use XMLBeans to:
  • Generate an XSD schema from an XML file
  • Compile the schema into Java classes
  • Use the classes to create an XML structure
  • Use the classes to read an XML file
Generating a Schema from an XML File
If you have an XML file, you can generate a schema using the inst2xsd command. For example, say you have the following xml:
<books>
    <book id="1">
        <title>Snow Crash</title>
        <author>Neal Stephenson</author>
    </book>
    <book id="2">
        <title>Neuromancer</title>
        <author>William Gibson</author>
    </book>
</books>
You can generate an XSD, using the following different design types: Russian Doll, Salami Slice or Venetian Blind. This is what the output looks like:

Russian Doll Design:

inst2xsd -design rd -enumerations never C:\temp\books.xml

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="books">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="book" maxOccurs="unbounded" minOccurs="0">
          <xs:complexType>
            <xs:sequence>
              <xs:element type="xs:string" name="title"/>
              <xs:element type="xs:string" name="author"/>
            </xs:sequence>
            <xs:attribute type="xs:byte" name="id" use="optional"/>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>
Salami Slice Design:
inst2xsd -design ss -enumerations never C:\temp\books.xml

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="books">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="book" maxOccurs="unbounded" minOccurs="0"/>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
  <xs:element name="author" type="xs:string"/>
  <xs:element name="title" type="xs:string"/>
  <xs:element name="book">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="title"/>
        <xs:element ref="author"/>
      </xs:sequence>
      <xs:attribute type="xs:byte" name="id" use="optional"/>
    </xs:complexType>
  </xs:element>
</xs:schema>
Venetian Blind Design:
inst2xsd -design vb -enumerations never C:\temp\books.xml

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="books" type="booksType"/>
  <xs:complexType name="booksType">
    <xs:sequence>
      <xs:element type="bookType" name="book" maxOccurs="unbounded" minOccurs="0"/>
    </xs:sequence>
  </xs:complexType>
  <xs:complexType name="bookType">
    <xs:sequence>
      <xs:element type="xs:string" name="title"/>
      <xs:element type="xs:string" name="author"/>
    </xs:sequence>
    <xs:attribute type="xs:byte" name="id" use="optional"/>
  </xs:complexType>
</xs:schema>
Compiling a Schema into Java Classes
Once you have generated a schema, you can use the Maven xmlbeans plugin to compile it into Java classes:
<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>xmlbeans-maven-plugin</artifactId>
  <version>2.3.2</version>
  <executions>
    <execution>
      <goals>
        <goal>xmlbeans</goal>
      </goals>
    </execution>
  </executions>
  <inherited>true</inherited>
  <configuration>
    <schemaDirectory>src/main/xsd</schemaDirectory>
    <sourceSchemas>
      <sourceSchema>books.xsd</sourceSchema>
    </sourceSchemas>
    <sourceGenerationDirectory>target/generated-sources/xmlbeans</sourceGenerationDirectory>
  </configuration>
</plugin>
You can also use the scomp command which will create a jar file containing the compiled classes.
C:\xmlbeans-2.4.0\bin> scomp -compiler %JAVA_HOME%\bin\javac books.xsd
Creating an XML Document Using the Compiled Classes
Add the jars to your classpath. Build an XML document as shown below (note that I am using the schema with the Russian Doll design. Different design types will produce different classes.):
BooksDocument doc = BooksDocument.Factory.newInstance();
Books books = doc.addNewBooks();
Book book = books.addNewBook();
book.setId((byte) 1);
book.setAuthor("Isaac Asimov");
book.setTitle("I, Robot");
System.out.println(doc.toString());
Produces:
<books>
  <book id="1">
    <title>I, Robot</title>
    <author>Isaac Asimov</author>
  </book>
</books>
Reading an XML Document Using the Compiled Classes
You can read an xml document and parse it into objects in the following way:
BooksDocument doc = BooksDocument.Factory.parse(
                    new File("C:\\temp\\books.xml"));
Books books = doc.getBooks();
Book[] bookArr = books.getBookArray();
for (Book book : bookArr) {
 String author = book.getAuthor();
 String title = book.getTitle();
 byte id = book.getId();
 System.out.println(author + '\t' + title);
}

Monday, June 01, 2009

Using XPath in Java

Given the following xml document:
<hosts>
  <host name="starship" port="8080"/>
  <host name="firefly" port="8180"/>
</hosts>
this is how you can use the javax.xml.xpath library to run an XPath query in order to obtain a list of host names:
//create a document
DocumentBuilderFactory domFactory = 
                     DocumentBuilderFactory.newInstance();
domFactory.setNamespaceAware(true);
DocumentBuilder builder = domFactory.newDocumentBuilder();
Document doc = builder.parse("file.xml");

//create the xpath expression
XPathFactory factory = XPathFactory.newInstance();
XPath xpath = factory.newXPath();
XPathExpression expr = xpath.compile("//host/@name");

//run the xpath query
Object result = expr.evaluate(doc, XPathConstants.NODESET);

//read results
NodeList nodes = (NodeList) result;
for (int i = 0; i < nodes.getLength(); i++) {
    System.out.println(nodes.item(i).getNodeValue());
}