Saturday, September 15, 2018

Java: Streaming a JDBC ResultSet as JSON

This post shows how you can convert a java.sql.ResultSet to JSON and stream it back to the caller. This is useful if you want to send a large dataset from a JDBC DataSource to a web application in JSON format. Streaming allows you to transfer the data, little by little, without having to load it all into the server's memory.

For example, consider the following ResultSet:

+---------+-----+
| name    | age |
+---------+-----+
| Alice   |  20 |
| Bob     |  35 |
| Charles |  50 |
+---------+-----+

The corresponding JSON is:

[
  { "name": "Alice",   "age": 20 },
  { "name": "Bob",     "age": 35 },
  { "name": "Charles", "age": 50 },
]

The following class (also available in my GitHub Repository) can be used to convert the ResultSet to JSON. Note that this class implements Spring's ResultSetExtractor, which can be used by a JdbcTemplate to extract results from a ResultSet.

/**
 * Streams a ResultSet as JSON.
 */
public class StreamingJsonResultSetExtractor implements ResultSetExtractor<Void> {

  private final OutputStream os;

  /**
   * @param os the OutputStream to stream the JSON to
   */
  public StreamingJsonResultSetExtractor(final OutputStream os) {
    this.os = os;
  }

  @Override
  public Void extractData(final ResultSet rs) {
    final var objectMapper = new ObjectMapper();
    try (var jg = objectMapper.getFactory().createGenerator(
                  os, JsonEncoding.UTF8)) {
      writeResultSetToJson(rs, jg);
      jg.flush();
    } catch (IOException | SQLException e) {
      throw new RuntimeException(e);
    }
    return null;
  }

  private static void writeResultSetToJson(final ResultSet rs,
                            final JsonGenerator jg)
                            throws SQLException, IOException {
    final var rsmd = rs.getMetaData();
    final var columnCount = rsmd.getColumnCount();
    jg.writeStartArray();
    while (rs.next()) {
      jg.writeStartObject();
      for (var i = 1; i <= columnCount; i++) {
        jg.writeObjectField(rsmd.getColumnName(i), rs.getObject(i));
      }
      jg.writeEndObject();
    }
    jg.writeEndArray();
  }
}

To use this in a web service with JAX-RS:

import javax.ws.rs.core.StreamingOutput;

@GET
@Path("runQuery")
public StreamingOutput runQuery() {
  return new StreamingOutput() {
    @Override
    public void write(final OutputStream os)
        throws IOException, WebApplicationException {
      jdbcTemplate.query("select name, age from person",
                   new StreamingJsonResultSetExtractor(os));
    }
  };
}

Saturday, September 08, 2018

Java: Mocking a ResultSet using Mockito

This post shows how you can mock a java.sql.ResultSet using Mockito. It can be used to help unit test code which performs operations on ResultSets (such as a ResultSetExtractor) without relying on an external datasource.

You can create a MockResultSet by providing a list of column names and a 2D array of data. For example:

var rs = MockResultSet.create(
           new String[] { "name", "age" }, //columns
           new Object[][] { // data
             { "Alice", 20 },
             { "Bob", 35 },
             { "Charles", 50 }
           });

The code for MockResultSet is shown below (also available in my GitHub Repository). Note that I have only mocked a few methods such as next, getString and getObject but it is quite easy to mock the rest by following the same pattern.

public class MockResultSet {

  private final Map<String, Integer> columnIndices;
  private final Object[][] data;
  private int rowIndex;

  private MockResultSet(final String[] columnNames,
                        final Object[][] data) {
    // create a map of column name to column index
    this.columnIndices = IntStream.range(0, columnNames.length)
        .boxed()
        .collect(Collectors.toMap(
            k -> columnNames[k],
            Function.identity(),
            (a, b) ->
              { throw new RuntimeException("Duplicate column " + a); },
            LinkedHashMap::new
            ));
    this.data = data;
    this.rowIndex = -1;
  }

  private ResultSet buildMock() throws SQLException {
    final var rs = mock(ResultSet.class);

    // mock rs.next()
    doAnswer(invocation -> {
      rowIndex++;
      return rowIndex < data.length;
    }).when(rs).next();

    // mock rs.getString(columnName)
    doAnswer(invocation -> {
      final var columnName = invocation.getArgumentAt(0, String.class);
      final var columnIndex = columnIndices.get(columnName);
      return (String) data[rowIndex][columnIndex];
    }).when(rs).getString(anyString());

    // mock rs.getObject(columnIndex)
    doAnswer(invocation -> {
      final var index = invocation.getArgumentAt(0, Integer.class);
      return data[rowIndex][index - 1];
    }).when(rs).getObject(anyInt());

    final var rsmd = mock(ResultSetMetaData.class);

    // mock rsmd.getColumnCount()
    doReturn(columnIndices.size()).when(rsmd).getColumnCount();

    // mock rs.getMetaData()
    doReturn(rsmd).when(rs).getMetaData();

    return rs;
  }

  /**
   * Creates the mock ResultSet.
   *
   * @param columnNames the names of the columns
   * @param data
   * @return a mocked ResultSet
   * @throws SQLException
   */
  public static ResultSet create(
                         final String[] columnNames,
                         final Object[][] data)
                         throws SQLException {
    return new MockResultSet(columnNames, data).buildMock();
  }
}

Saturday, June 02, 2018

Java 10: Collecting a Stream into an Unmodifiable Collection

Java 10 introduces several new methods to facilitate the creation of unmodifiable collections.

The List.copyOf, Set.copyOf, and Map.copyOf methods create new collection instances from existing instances. For example:

List<String> modifiable = Arrays.asList("foo", "bar");
List<String> unmodifiableCopy = List.copyOf(list);

// Note that since Java 9, you can also use "of" to create
// unmodifiable collections
List<String> unmodifiable = List.of("foo", "bar");

There are also new collector methods, toUnmodifiableList, toUnmodifiableSet, and toUnmodifiableMap, to allow the elements of a stream to be collected into an unmodifiable collection. For example:


// Java 10
Stream.of("foo", "bar").collect(toUnmodifiableList());

// before Java 10
Stream.of("foo", "bar").collect(
    collectingAndThen(toList(), Collections::unmodifiableList));

Monday, May 28, 2018

Java 10: The "var" keyword

Java 10 has introduced local variable type inference with the keyword var. This means that instead of writing:

Map<Department, List<Employee>> map = new HashMap<>();
// ...
for (Entry<Department, List<Employee>> dept : map.entrySet()) {
  List<Employee> employees = dept.getValue();
  // ...
}

you can use var to reduce boilerplate, as follows:

var map = new HashMap<Department, Employee>();
// ...
for (var dept : map.entrySet()) {
  var employees = dept.getValue();
  // ...
}

var removes the need to have explicit types written in your code. Not only does this reduce repetition but it also makes your code easier to maintain because if, for example, you decide to change the types of the objects stored in your map in the future, you would only need to alter one line of code.

Polymorphism:

Now let's take a look at how var behaves with polymorphic code. For example, if you have a class Shape with two subclasses, Square and Circle, what type will be inferred if you use var v = new Circle()? Let's try it out in JShell:

jshell> var v = new Circle();
v ==> Circle@4e9ba398

jshell> v.getClass();
$13 ==> class Circle

jshell> v = new Square();
|  Error:
|  incompatible types: Square cannot be converted to Circle
|  v = new Square();
|      ^----------^

As demonstrated above, v is of type Circle and if you try to reassign it to Square, the compiler will throw an error.

Anonymous classes:

One of the exciting things you can do with vars is create anonymous classes and refer to fields inside them! For example:

var person = new Object() {
  String name = "Joe";
  int age = 10;
};
System.out.println(person.name + ":" + person.age);

var infers the type of the anonymous class which means that you can use an anonymous class as a "holder" for intermediate values, without having to create and maintain a new class. Here is another example showing how you can create "temporary" person objects:

var persons = Stream.of("Alice", "Bob", "Charles")
    .map(s -> new Object() {
       String name = s;
       int age = 10;
    })
    .collect(toList());
persons.forEach(p -> System.out.println(p.name));
Other points to note:

You cannot use var without an explicit initialisation (assigning to null does not count) or within lambda expressions:

jshell> var v;
|  Error:
|  cannot infer type for local variable v
|    (cannot use 'var' on variable without initializer)
|  var v;
|  ^----^

jshell> var v = null;
|  Error:
|  cannot infer type for local variable v
|    (variable initializer is 'null')
|  var v = null;
|  ^-----------^

jshell> var v = () -> {}
|  Error:
|  cannot infer type for local variable v
|    (lambda expression needs an explicit target-type)
|  var v = () -> {};
|  ^---------------^

Saturday, May 26, 2018

HTML5 Date Input with jQuery Fallback

HTML5 introduced a new date input type which allows a user to enter a date using a date picker.

<input id="date" type="date" value="2018-05-26">

This is what it looks like in Chrome:

However, not all browsers support this input type. In unsupported browsers, such as Internet Explorer, you will simply see a text input field.

In this post, I will show how you can detect if a browser supports the date input type and how you can fall back to using jQuery's date picker if it doesn't.

Checking if the browser supports date input:

The following code creates an input element and sets its type to date. If the browser does not support date input, this operation will not work and the input type will degrade to text.

var input = document.createElement("input");
input.setAttribute("type", "date");
if (input.type !== "date") {
    console.log("browser does not support date input");
}
Alternatively, use the Modernizr library, which makes it easy to detect the features that a browser supports:
if (!Modernizr.inputtypes.date) {
    console.log("browser does not support date input");
}
Falling back to jQuery's date picker:

The JSFiddle below shows how you would use jQuery's date picker if the browser does not support date input.