Large tables can be compressed in Kdb+ by setting `.z.zd`

. Compression of data can reduce disk cost and in some cases even improve performance for applications that have fast CPUs but slow disks.

`.z.zd`

is a list of three integers consisting of logical block size, algorithm (0=none, 1=q, 2=gzip, 3=snappy, 4=lz4hc) and compression level.

Here is an example showing how to compress a table:

// Helper function that sets .z.zd and // returns the previous value of .z.zd .util.setZzd:{ origZzd:$[count key `.z.zd;.z.zd;()]; if[x~(); system"x .z.zd"; :origZzd; ]; .z.zd:x; origZzd} // create a table td:([]a:1000000?10; b:1000000?10; c:1000000?10); // save the table to disk without compression `:uncompressed set td; // save the table to disk using q IPC compression origZzd:.util.setZzd[(17;1;0)]; `:compressed set td; .util.setZzd[origZzd];

You can check compression stats by using the -21! function:

q)-21!`:compressed compressedLength | 5747890 uncompressedLength| 24000041 algorithm | 1i logicalBlockSize | 17i zipLevel | 0i

The size of the file on disk is reduced from 22.8 MB to 5.5 MB after using q IPC compression.