kdb+/q - Reading and Writing a CSV File

This post shows how you can load a CSV file into kdb and write a table out from kdb to a CSV file.

Reading a CSV file:

Let's say that you'd like to load a file containing comma-separated values into an in-memory table in kdb. For example, here's my CSV file, which contains country populations (source: Worldometers):

$ head -5 worldPopulation.csv
region,country,population
Africa,Algeria,40375954
Africa,Angola,25830958
Africa,Benin,11166658
Africa,Botswana,2303820

My file has got two string columns and one integer column.

I can load it into kdb using the Zero Colon function: (types; delimiter) 0: filehandle

q) data:("SSI";enlist",") 0: `$"/path/to/worldPopulation.csv"
// "SSI" creates 2 symbol columns and one integer column

// let's look at the column types
q) meta data
c         | t f a
----------| -----
region    | s
country   | s
population| i

// check the data
q) 5#data
region country                  population
------------------------------------------
Africa Algeria                  40375954
Africa Angola                   25830958
Africa Benin                    11166658
Africa Botswana                 2303820
Africa Burkina Faso             18633725

// you can use * if you want string columns, instead of symbol ones
q) data:("**I";enlist",") 0: `$"/path/to/worldPopulation.csv"

q) meta data
c         | t f a
----------| -----
region    | C
country   | C
population| i

q) 5#data
region   country        population
----------------------------------
"Africa" "Algeria"      40375954
"Africa" "Angola"       25830958
"Africa" "Benin"        11166658
"Africa" "Botswana"     2303820
"Africa" "Burkina Faso" 18633725

Writing a CSV file:

To save a table to a CSV file, you can use the command: filehandle 0: delimiter 0: table

// first, let's try printing out the csv data to console
q) "," 0: data
"region,country,population"
"Africa,Algeria,40375954"
"Africa,Angola,25830958"
"Africa,Benin,11166658"
"Africa,Botswana,2303820"

// save it
q)(`$"/tmp/out.csv") 0: "," 0: data
`/tmp/out.csv

Java 8: Joining a Stream

In a previous post, I wrote about how you can use Guava's Joiner to easily convert an Iterable into a String, so you no longer have to iterate over it and build the String manually.

Java 8 introduces the StringJoiner, which can be used to join a Stream using Collectors.joining. For example:

Arrays.asList("apple", "banana", "cherry")
      .stream()
      .collect(Collectors.joining(", ");

To skip nulls and empty strings, you can filter the stream first:

Arrays.asList("apple", "banana", null, "", "cherry")
      .stream()
      .filter(s -> s != null && !s.isEmpty())
      .collect(Collectors.joining(", ");

I'd still use Guava's Joiner for lists though.

Ext JS - Changing the "Today" button in a date field

The datefield in Ext JS has a date picker dropdown containing a handy "Today" button, which allows you to quickly select today's date. But what if you're not interested in today's date? This post shows how you can change the text and the handler of the "Today" button in order to select a different date e.g. yesterday.

My jsfiddle for changing the "Today" button is embedded below. After the date field has been rendered, it gets a reference to the Today button via the date field's picker object and then changes its handler to select yesterday's date instead.

fahd.blog in 2015

Happy 2016, everyone!

I'd like to wish everyone a great start to an even greater new year!

In keeping with tradition, here's one last look back at fahd.blog in 2015.

During 2015, I posted 13 new entries on fahd.blog. I am also thrilled that I have more readers from all over the world! Thanks for reading and especially for giving feedback.

Top 5 posts of 2015:

• Java 8 Streams API: Grouping and Partitioning a Stream
• Java 8: Sorting a List using Lambdas and Method References
• Java 8: Creating infinite streams
• Ext JS - Reading a JSON File
• Speeding up Grep on Large Files

I'm going to be writing a lot more this year, so stay tuned for more great techie tips, tricks and hacks! :)

Related posts:

• fahd.blog in 2014
• fahd.blog in 2013
• fahd.blog in 2012
• fahd.blog in 2011
• fahd.blog in 2010

Java 8: Convert a String to a Stream of Characters

I find it a bit strange that the Java API does not provide a method to convert a String to a Stream<Character>, but this is how you can do it:

Stream<Character> stream = string.chars().mapToObj(i -> (char)i);

// or:

Stream<Character> stream = IntStream.range(0, string.length())
                                    .mapToObj(string::charAt);