tag:blogger.com,1999:blog-326378282024-03-07T13:56:15.807+00:00fahd.blogLet the code do the talking...Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.comBlogger410125tag:blogger.com,1999:blog-32637828.post-7204825945955493472024-01-01T15:31:00.000+00:002024-01-01T15:31:32.144+00:00fahd.blog in 2023<p><b><font color="#000099">Happy 2024, everyone!</font></b></p><p>I'd like to wish everyone a great start to an even greater new year!</p>
<p>In keeping with tradition, here's one last look back at <a href="http://fahdshariff.blogspot.com">fahd.blog</a> in 2023.</p>
<p>During 2023, I posted 11 new entries on <a href="http://fahdshariff.blogspot.com">fahd.blog</a>. I am also thrilled that I have more readers from all over the world! Thanks for reading and especially for giving feedback.</p>
<b><font color="#000099">Top 3 posts of 2023:</font></b>
<ul>
<li><a href="https://fahdshariff.blogspot.com/2023/04/java-20-record-patterns-in-for-loops.html">Java 20: Record Patterns in For Loops</a></li>
<li><a href="https://fahdshariff.blogspot.com/2023/08/matrix-operations-in-kdbq.html">Matrix Operations in kdb+/q</a></li>
<li><a href="https://fahdshariff.blogspot.com/2023/09/linear-and-polynomial-regression-in-kdbq.html">Linear and Polynomial Regression in kdb+/q</a></li>
</ul>
<p>I'm going to be writing a lot more this year, so stay tuned for more great techie tips, tricks and hacks! :)</p>
<b><font color="#000099">Related posts:</font></b>
<ul>
<li><a href="https://fahdshariff.blogspot.com/2023/01/fahdblog-in-2022.html">fahd.blog in 2022</a></li>
<li><a href="https://fahdshariff.blogspot.com/2022/01/fahdblog-in-2021.html">fahd.blog in 2021</a></li>
<li><a href="https://fahdshariff.blogspot.com/2021/01/fahdblog-in-2020.html">fahd.blog in 2020</a></li>
<li><a href="https://fahdshariff.blogspot.com/2020/01/fahdblog-in-2019.html">fahd.blog in 2019</a></li>
<li><a href="https://fahdshariff.blogspot.com/2019/01/fahdblog-in-2018.html">fahd.blog in 2018</a></li>
<li><a href="https://fahdshariff.blogspot.com/2018/01/fahdblog-in-2017.html">fahd.blog in 2017</a></li>
<li><a href="https://fahdshariff.blogspot.com/2017/01/fahdblog-in-2016.html">fahd.blog in 2016</a></li>
<li><a href="https://fahdshariff.blogspot.com/2016/01/fahdblog-in-2015.html">fahd.blog in 2015</a></li>
<li><a href="https://fahdshariff.blogspot.com/2015/01/fahdblog-in-2014.html">fahd.blog in 2014</a></li>
<li><a href="https://fahdshariff.blogspot.com/2014/01/fahdblog-in-2013.html">fahd.blog in 2013</a></li>
<li><a href="https://fahdshariff.blogspot.com/2013/01/fahdblog-in-2012.html">fahd.blog in 2012</a></li>
<li><a href="https://fahdshariff.blogspot.com/2012/01/fahdblog-in-2011.html">fahd.blog in 2011</a></li>
<li><a href="https://fahdshariff.blogspot.com/2011/01/fahdblog-in-2010.html">fahd.blog in 2010</a></li>
</ul>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-42197101140620991452023-12-09T17:29:00.000+00:002023-12-09T17:29:09.785+00:00Python: Running Tasks in Parallel<p>The <a href="https://docs.python.org/3/library/concurrent.futures.html"><code>concurrent.futures</code></a> module can be used to run tasks in parallel in Python. Here is an example:</p>
<pre class="brush:python; gutter:false;">
import concurrent.futures
from concurrent.futures import ProcessPoolExecutor
with ProcessPoolExecutor(max_workers = 10) as executor:
futures = [executor.submit(perform_task, task) for task in tasks]
results = [future.result() for future in futures]
</pre>
<p>The <code>ProcessPoolExecutor</code> class uses a pool of processes to execute tasks asynchronously. The <code>submit</code> function immediately returns a <code>Future</code> object, and you can call <code>future.result()</code>, which will block until the task has completed.</p>
<p>Note that there is also a <code>ThreadPoolExecutor</code> class which uses a pool of threads; however, due to the <a href="https://docs.python.org/3/glossary.html#term-global-interpreter-lock">Global Interpreter Lock</a>, only one thread can execute python bytecode at any one time, which means that you will not achieve any parallelisation in most cases.</p>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-10161807573891726352023-12-03T12:00:00.001+00:002023-12-03T12:00:00.252+00:00Java 21: Sequenced Collections<p>Introduced in <a href="https://www.oracle.com/java/technologies/javase/21-relnote-issues.html">Java 21</a>, a <a href="https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/util/SequencedCollection.html"><code>SequencedCollection</code></a> is a <code>Collection</code> whose elements have a defined encounter order i.e. it has first and last elements, and the elements between them have successors and predecessors. Some examples include <code>List</code>, <code>Deque</code>, <code>SortedSet</code>, and <code>LinkedHashSet</code>. </p>
<p>The <code>SequencedCollection</code> interface provides methods to add, retrieve, and remove elements at either end of the collection. It also has a <code>reversed()</code> method which provides a reverse-ordered view of the original collection.
<p>Similarly, the new <a href="https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/util/SequencedMap.html"><code>SequencedMap</code></a> interface is a map that has a well-defined encounter order, supports operations at both ends, and is reversible. Examples include <code>LinkedHashMap</code> and <code>TreeMap</code>.</p>
<p>Example usage:</p>
<pre class="brush:java; gutter:false;">
var set = new LinkedHashSet<String>(Arrays.asList("a", "b", "c"));
set instanceof SequencedCollection
==> true
set.getFirst()
==> "a"
set.getLast()
==> "c"
set.reversed()
==> [c, b, a]
</pre>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-5812024732383244532023-12-02T17:03:00.005+00:002023-12-02T17:03:59.385+00:00Java 21: Unnamed Classes<p><a href="https://www.oracle.com/java/technologies/javase/21-relnote-issues.html">Java 21</a> introduces <a href="https://openjdk.org/jeps/445">Unnamed Classes</a> (a preview language feature) that allow you to write small programs without having an enclosing class declaration.</p>
<p>Here is the classic Hello World program that we were all taught when starting to learn Java:</p>
<pre class="brush:java; gutter:false;">
public class HelloWorld {
public static void main(String[] args) {
System.out.println("Hello, World!");
}
}
</pre>
<p>There is a lot of clutter here. Using an unnamed class, this can be simplified to:</p>
<pre class="brush:java; gutter:false;">
void main() {
System.out.println("Hello, World!");
}
</pre>
<p>Not only is the enclosing class not required, but the <code>main</code> method has also been enhanced so that it does not need to be <code>public</code>, <code>static</code> or require any arguments.</p>
<p>You can also add fields and methods to an unnamed class, as shown below:</p>
<pre class="brush:java; gutter:false;">
private static final String GREETING = "Hello, World!";
private String getGreeting() {
return GREETING;
}
void main() {
System.out.println(getGreeting());
}
</pre>
<p>Since an unnamed class cannot be instantiated or referenced by name, it is only useful as a standalone program or as an entry point to a program.</p>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-34624463202840660522023-11-25T18:23:00.001+00:002023-12-04T21:21:10.148+00:00Java 21: String Templates<p>In <a href="https://www.oracle.com/java/technologies/javase/21-relnote-issues.html">Java 21</a>, <a href="https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/StringTemplate.html">String Templates</a> have been introduced as a preview language feature, that allow text and expressions to be composed safely and efficiently, without using the <code>+</code> operator.</p>
<p>Here is an example:</p>
<pre class="brush:java; gutter:false;">
int x = 5, y = 6;
String s = STR."\{x} plus \{y} is equal to \{x + y}";
// evaluates to: "5 plus 6 is equal to 11"
</pre>
<p>In this example, <code>STR</code> is a template processor. The template is <code>\{x} plus \{y} is equal to \{x + y}</code> and <code>\{x}</code> is one of the embedded expressions in the template. The <a href="https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/StringTemplate.html#STR"><code>STR</code></a> template processor is defined in the Java Platform (and is automatically imported into every Java source file), and it performs string interpolation by evaluating the embedded expressions.</p>
<p>More examples:</p>
<pre class="brush:java; gutter:false;">
// you can invoke methods, access fields, use ternaries
String s = STR."\{user.name}: Access \{user.hasAccess() ? "Granted" : "Denied"}";
// multi-line template expression
String xml = STR."""
<book>
<author>\{author}</author>
<title>\{title}</title>
</book>
""";
</pre>
<p><b><font color="#000099">FMT Template Processor</font></b><p>
<p><a href="https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/util/FormatProcessor.html#FMT">FMT</a> is like STR but it also interprets format specifiers which appear to the left of embedded expressions. For example:</p>
<pre class="brush:java; gutter:false;">
double val = 4999.4567;
FormatProcessor.FMT."The value is %,.2f\{val}";
// evaluates to: "The value is 4,999.46"
</pre>
<p>It's quite easy to create your own Template Processor by implementing <a href="https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/StringTemplate.Processor.html"><code>StringTemplate.Processor</code></a>. This is useful if you want to validate inputs before composing the string. It's also possible to return an object of any type, not just <code>String</code>. For instance, a SQL Template processor could first sanitise the input to prevent a SQL injection attack, and then return a <code>PreparedStatement</code> instead of a <code>String</code>.</p>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-82854633008640766852023-09-03T09:00:00.001+01:002023-09-03T09:00:00.149+01:00Linear and Polynomial Regression in kdb+/q<p>In this post, I'll describe how you can implement <a href="https://en.wikipedia.org/wiki/Regression_analysis#Linear_regression">linear</a> and <a href="https://en.wikipedia.org/wiki/Polynomial_regression">polynomial</a> regression in kdb+/q to determine the equation of a line of best fit (also known as a trendline) through the data on a scatter plot.</p>
<p>Consider the following scatter plot:</p>
<img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi41qYrwECPZD3k3iuVQQG-3gOKhtNDezSO_rgC-7APUfvQU6U-pWKVu-_Ch4zSO4nm0ujthgAEqcOcmEXn4Zo4npmqyiq5D0flW19sW_mgLDPevkmebEr7K5YDDn4ElazOQ0o-owIsRKwGV8SSmgFINEZDPF7G3MTk5zqLDe82EseggWJOxd8wuQ/s1600/scatterplot.png"/>
<p>Our aim is to estimate a function of a line that most closely fits the data.</p>
<p>The vector of estimated polynomial regression coefficients (using ordinary least squares estimation) can be obtained using the following formula (for information about how this formula is derived, see <a href="https://en.wikipedia.org/wiki/Regression_analysis">Regression analysis [Wikipedia]</a>):</p>
<h1><font color="green"><code>
b = (X<sup>T</sup>X)<sup>−1</sup>X<sup>T</sup>y
</code></font></h1>
<p>This can be translated into q as follows:</p>
<pre style="font-family: Consolas,Courier New,Courier,mono,serif; font-size:12px;background:black;color:white">
computeRegressionCoefficients:{
xt:<font color="00FFFF">flip</font> x;
xt_x:xt <font color="00FFFF">mmu</font> x;
xt_x_inv:<font color="00FFFF">inv</font> xt_x;
xt_y:xt <font color="00FFFF">mmu</font> y;
xt_x_inv <font color="00FFFF">mmu</font> xt_y}
</pre>
<p><b><font color="#000099">Linear:</font></b><br/>
In order to perform linear regression, we have to first create a matrix X with a column of 1s and a column containing the x-values. The output of linear regression will be a vector of 2 coefficients and the equation of the trendline will be of the form: <code>y = b<sub>1</sub>x + b<sub>0</sub></code></p>
<pre style="font-family: Consolas,Courier New,Courier,mono,serif; font-size:12px;background:black;color:white">
computeLinearRegressionCoefficients:{
computeRegressionCoefficients[<font color="00FFFF">flip</font> (1f;x);y]}
</pre>
<img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5Z6hr2zIITyNaexm4Yj6Thj6HGVw4N6A7D3U5gQD3IKgBZ7bCuCPIkCiQIthHugFUMYNrzk6IER9ChHovbuVunDpSoh62iOUgMEhT-iJrYGeINTPomju9qU1Yhl4iWvPTtl61Whaw9I_ckYtHYHzC128mGSj-Uxq5QJ9exSCQXjlzX8zDsFS0dg/s1600/scatterplot_linear.png"/>
<p><b><font color="#000099">Polynomial:</font></b><br/>
In order to fit a polynomial line, all we have to do is take the matrix X from the linear regression model and add more columns corresponding to the order of the polynomial desired. For example, for quadratic regression, we will add a column for x<sup>2</sup> on the right side of the matrix X. The output of quadratic regression will be a vector of 3 coefficients and the equation of the curve will be of the form: <code>y = b<sub>2</sub>x<sup>2</sup> + b<sub>1</sub>x + b<sub>0</sub></code></p>
<pre style="font-family: Consolas,Courier New,Courier,mono,serif; font-size:12px;background:black;color:white">
<font color="orange">// quadratic</font>
computeQuadraticRegressionCoefficients:{
computeRegressionCoefficients[<font color="00FFFF">flip</font> (1f;x;x*x);y]}
<font color="orange">// cubic</font>
computeCubicRegressionCoefficients:{
computeRegressionCoefficients[<font color="00FFFF">flip</font> (1f;x;x*x;x*x*x);y]}
<font color="orange">// generalisation of polynomial regression for any order</font>
computePolynomialRegressionCoefficients:{[x;y;order]
computeRegressionCoefficients[<font color="00FFFF">flip</font> x <font color="00FFFF">xexp</font>/: <font color="00FFFF">til</font> order+1;y]}
</pre>
<img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwHQrw0PCiHkGlDiVBOW2ezepZbvo7tC7M4-x7spGaQK34Dco5YqHp4fIwODvj-uOI0PStpBb5cXtUlnQP3gFQxG6tuJ10yQQJQH-1W5h_ZlbEUjzA8n1PEisM-Df_DLwGN_DoxRibMeuVh4qqJlfNplZfRKt9gj-10X_rk7kvs8qMsx-Cx87Avw/s1600/scatterplot_quadratic.png"/>
<p><b><font color="#000099">Related post:</font></b><br/>
<a href="https://fahdshariff.blogspot.com/2023/08/matrix-operations-in-kdbq.html">Matrix Operations in kdb+/q</a></p>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-10006815707977835882023-09-02T09:00:00.003+01:002023-09-02T09:00:00.137+01:00Python: Printing the Stdout/Stderr of a Subprocess<p>This is how you can run a subprocess in python and print out its stdout and stderr:</p>
<pre class="brush:python; gutter:false;">
import subprocess
proc = subprocess.Popen(["/path/to/myscript", "arg1", "arg2"],
stderr=subprocess.STDOUT, stdout=subprocess.PIPE)
for line in proc.stdout:
print(line.decode().rstrip())
proc.wait()
if proc.returncode != 0:
print("Command failed with status:", proc.returncode)
</pre>
<p>Check out the <a href="https://docs.python.org/3/library/subprocess.html"><code>subprocess</code></a> documentation for more information.</p>
<p>
<b><font color="#000099">Related post:</font></b><br/>
<a href="https://fahdshariff.blogspot.com/2010/09/python-cheat-sheet.html">Python Cheat Sheet</a></p>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-51240426328913946582023-08-27T10:00:00.001+01:002023-08-27T10:00:00.137+01:00Matrix Operations in kdb+/q<p>In q, a matrix (an array of m x n numbers) is represented as a list of lists. For example, here is a matrix with 2 rows and 3 columns:</p>
<pre style="font-family: Consolas,Courier New,Courier,mono,serif; font-size:12px;background:black;color:white">
q)<font color="#4AF626">A:(1 2 3;4 5 6)</font>
q)<font color="#4AF626">A</font>
1 2 3
4 5 6
</pre>
<p><b><font color="#000099">Matrix Addition and Subtraction</font></b><br/>
If A and B are matrices of the same size, then they can be added and subtracted. To find the entries of A + B, you simply add the corresponding entries of A and B. To find A - B, subtract corresponding entries. If A and B have different sizes, you will get a <code>'length</code> error.</p>
<pre style="font-family: Consolas,Courier New,Courier,mono,serif; font-size:12px;background:black;color:white">
q)<font color="4AF626">A:(1 2;3 4)</font>
q)<font color="4AF626">B:(5 6;7 8)</font>
q)<font color="4AF626">A<font color="00FFFF">+</font>B</font>
6 8
10 12
q)<font color="4AF626">B<font color="00FFFF">-</font>A</font>
4 4
4 4
q)<font color="4AF626">C:(1 1 1;2 2 2)</font>
q)<font color="4AF626">A<font color="00FFFF">+</font>C</font>
'length
[0] A+C
^
</pre>
<p><b><font color="#000099">Scalar Multiplication</font></b><br/>
If A is a matrix and k is a scalar, then the matrix kA is obtained by multiplying each entry of A by k.</p>
<pre style="font-family: Consolas,Courier New,Courier,mono,serif; font-size:12px;background:black;color:white">
q)<font color="4AF626">A:(1 2;3 4)</font>
q)<font color="4AF626">k:2</font>
q)<font color="4AF626">k<font color="00FFFF">*</font>A</font>
2 4
6 8
</pre>
<p><b><font color="#000099">Matrix Multiplication</font></b><br/>
First, in order to multiply two matrices A and B, the number of columns of A must match the number of rows of B. If A is m x n and B is n x p, then the size of the product matrix AB will be m x p. In order to calculate AB, you have to take the dot product (multiply corresponding numbers and then add them up) of each row vector in A and the corresponding column vector in B. This can be done in q using the <a href="https://code.kx.com/q/ref/mmu/"><code>mmu</code></a> (or <code>$</code>) operator.</p>
<pre style="font-family: Consolas,Courier New,Courier,mono,serif; font-size:12px;background:black;color:white">
q)<font color="4AF626">A:(1 2f;3 4f)</font>
q)<font color="4AF626">B:(5 6f;7 8f)</font>
q)<font color="4AF626">A</font>
1 2
3 4
q)<font color="4AF626">B</font>
5 6
7 8
q)<font color="4AF626">A <font color="00FFFF">mmu</font> B</font>
19 22
43 50
</pre>
<p><b><font color="#000099">Identity Matrix</font></b><br/>
This is a square matrix with 1s on the diagonal and 0s everywhere else.</p>
<pre style="font-family: Consolas,Courier New,Courier,mono,serif; font-size:12px;background:black;color:white">
q)<font color="4AF626">I:<font color="00FFFF">{`float${x=/:x}til x}</font></font>
q)<font color="4AF626">I 3</font>
1 0 0
0 1 0
0 0 1
</pre>
<p><b><font color="#000099">Matrix Inverse</font></b><br/>
The inverse of A is A<sup>-1</sup> if AA<sup>-1</sup>=A<sup>-1</sup>A=I, where I is the identity matrix. Use the <a href="https://code.kx.com/q/ref/inv/"><code>inv</code></a> function to find the inverse of a matrix.</p>
<pre style="font-family: Consolas,Courier New,Courier,mono,serif; font-size:12px;background:black;color:white">
q)<font color="4AF626">A:(1 2f;3 4f)</font>
q)<font color="4AF626"><font color="00FFFF">inv</font> A</font>
-2 1
1.5 -0.5
q)<font color="4AF626">A <font color="00FFFF">mmu inv</font> A</font>
1 1.110223e-016
0 1
</pre>
<p><b><font color="#000099">Matrix Tranpose</font></b><br/>
The tranpose A<sup>T</sup> of a matrix A is a flipped version of the original matrix which is obtained by changing its rows into columns (or equivalently, its columns into rows). This can be done by using the <a href="https://code.kx.com/q/ref/flip/"><code>flip</code></a> operation in q.
<pre style="font-family: Consolas,Courier New,Courier,mono,serif; font-size:12px;background:black;color:white">
q)<font color="4AF626">A:(1 2 3;4 5 6)</font>
q)<font color="4AF626"><font color="00FFFF">flip</font> A</font>
1 4
2 5
3 6
</pre>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-32532362315736465642023-08-26T16:03:00.002+01:002023-08-26T16:03:18.776+01:00Using "flock" to Prevent Multiple Instances of a Script from Running<p>In a <a href="https://fahdshariff.blogspot.com/2014/02/using-lockfile-to-prevent-multiple.html">previous post</a>, I wrote about how you can use the <code>lockfile</code> command to ensure that only one instance of a script is running at a time. An alternative to <code>lockfile</code> is the <a href="https://linux.die.net/man/1/flock"><code>flock</code></a> command, which is used as follows:</p>
<pre class="brush:bash; gutter:false">
flock /path/to/mylockfile cmd
</pre>
<p>By default, if the lock cannot be immediately acquired, <code>flock</code> will wait indefinitely until it becomes available. However, you can use the <code>--nonblock</code> (or <code>-n</code>) flag if you want <code>flock</code> to fail (with an exit code of 1) rather than wait if the lock cannot be immediately acquired. You can also specify how long <code>flock</code> should wait by passing in a <code>--timeout</code> in seconds.</p>
<p>A convenient form of <code>flock</code> often used within shell scripts is to use a file descriptor, as follows:</p>
<pre class="brush:bash; gutter:false">
(
flock -n 9 || exit 1
# ... commands executed under lock ...
) 9>/path/to/mylockfile
</pre>
<p>If you want to prevent multiple instances of a shell script from running simultaneously, add the following boilerplate at the top of your script, which will cause the script to lock itself automatically on first run:</p>
<pre class="brush:bash; gutter:false">
[ "${FLOCKER}" != "$0" ] && exec env FLOCKER="$0" flock -en "$0" "$0" "$@" || :
</pre>
<p>
<b><font color="#000099">Related posts:</font></b><br/>
<a href="https://fahdshariff.blogspot.com/2014/02/using-lockfile-to-prevent-multiple.html">Using "lockfile" to Prevent Multiple Instances of a Script from Running</a><br/>
<a href="http://fahdshariff.blogspot.com/2014/02/retrying-commands-in-shell-scripts.html">Retrying Commands in Shell Scripts</a><br/>
<a href="http://fahdshariff.blogspot.com/2013/08/executing-shell-command-with-timeout.html">Executing a Shell Command with a Timeout</a><br/>
</p>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-37165790228975905482023-04-08T09:00:00.001+01:002023-04-08T09:00:00.180+01:00kdb+/q - Converting a CSV String into a Table<p>I often find myself wanting to create quick in-memory tables in q for testing purposes. The usual way to create a table is by specifying lists of column names and values, or flipping a dictionary, as shown below:</p>
<pre class="brush:java; gutter:false;">
([] name:`Alice`Bob`Charles;age:20 30 40;city:`London`Paris`Athens)
// or, using a dictionary:
flip `name`age`city!(`Alice`Bob`Charles;20 30 40;`London`Paris`Athens)
</pre>
<p>As you can see, it's quite difficult to visualise the table being created using this approach. That's why I sometimes prefer to create a table from a multi-line CSV string instead, using the <a href="https://code.kx.com/q/ref/file-text/#0-file-text">0: operator</a>, as shown below:</p>
<pre class="brush:java; gutter:false;">
("SIS";enlist",") 0:
"name,age,city
Alice,20,London
Bob,30,Paris
Charles,40,Athens
"
name age city
------------------
Alice 20 London
Bob 30 Paris
Charles 40 Athens
</pre>
<p>Note that you can also load from a CSV file:</p>
<pre class="brush:java; gutter:false;">
("SIS";enlist",") 0: `$"/path/to/file.csv"
</pre>
<b><font color="#000099">Related post:</font></b><br/>
<a href="https://fahdshariff.blogspot.com/2016/04/kdbq-reading-and-writing-csv-file.html">kdb+/q - Reading and Writing a CSV File</a>
Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-38077012992010054742023-04-02T16:36:00.002+01:002023-04-02T16:36:53.273+01:00Java 20: Record Patterns in For Loops<p><a href="https://fahdshariff.blogspot.com/2022/11/java-19-record-patterns.html">Previously</a>, I wrote about "Record Patterns" introduced in Java 19, that allow you to "deconstruct" records and access their components directly.</p>
<p>In <a href="https://jdk.java.net/20/release-notes">Java 20</a> (released a couple of weeks ago!), <a href="https://openjdk.org/jeps/432">record patterns</a> have been enhanced so that they can also be used in <code>for</code> loops.</p>
<p>Here is an example that uses a nested record pattern in a <code>for</code> loop to print out a list of records:</p>
<pre class="brush:java; gutter:false;">
record Author(String firstName, String lastName) {}
record Book(String title, Author author, double price) {}
static void printBooks(List<Book> books) {
for (Book(var title, Author(var firstName, var lastName), var price): books) {
System.out.printf("%s by %s %s for %.2f\n", title, firstName, lastName, price);
}
}
</pre>
<b><font color="#000099">Related post:</font></b><br/>
<a href="https://fahdshariff.blogspot.com/2022/11/java-19-record-patterns.html">Java 19: Record Patterns</a>
Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-31570028867217241982023-01-06T15:16:00.001+00:002023-01-06T15:16:59.377+00:00fahd.blog in 2022<p><b><font color="#000099">Happy 2023, everyone!</font></b></p><p>I'd like to wish everyone a great start to an even greater new year!</p>
<p>In keeping with tradition, here's one last look back at <a href="http://fahdshariff.blogspot.com">fahd.blog</a> in 2022.</p>
<p>During 2022, I posted 7 new entries on <a href="http://fahdshariff.blogspot.com">fahd.blog</a>. I am also thrilled that I have more readers from all over the world! Thanks for reading and especially for giving feedback.</p>
<b><font color="#000099">Top 3 posts of 2022:</font></b>
<ul>
<li><a href="https://fahdshariff.blogspot.com/2022/05/java-18-simple-web-server.html">Java 18: Simple Web Server</a></li>
<li><a href="https://fahdshariff.blogspot.com/2022/11/java-19-record-patterns.html">Java 19: Record Patterns</a></li>
<li><a href="https://fahdshariff.blogspot.com/2022/12/java-19-virtual-threads.html">Java 19: Virtual Threads</a></li>
</ul>
<p>I'm going to be writing a lot more this year, so stay tuned for more great techie tips, tricks and hacks! :)</p>
<b><font color="#000099">Related posts:</font></b>
<ul>
<li><a href="https://fahdshariff.blogspot.com/2022/01/fahdblog-in-2021.html">fahd.blog in 2021</a></li>
<li><a href="https://fahdshariff.blogspot.com/2021/01/fahdblog-in-2020.html">fahd.blog in 2020</a></li>
<li><a href="https://fahdshariff.blogspot.com/2020/01/fahdblog-in-2019.html">fahd.blog in 2019</a></li>
<li><a href="https://fahdshariff.blogspot.com/2019/01/fahdblog-in-2018.html">fahd.blog in 2018</a></li>
<li><a href="https://fahdshariff.blogspot.com/2018/01/fahdblog-in-2017.html">fahd.blog in 2017</a></li>
<li><a href="https://fahdshariff.blogspot.com/2017/01/fahdblog-in-2016.html">fahd.blog in 2016</a></li>
<li><a href="https://fahdshariff.blogspot.com/2016/01/fahdblog-in-2015.html">fahd.blog in 2015</a></li>
<li><a href="https://fahdshariff.blogspot.com/2015/01/fahdblog-in-2014.html">fahd.blog in 2014</a></li>
<li><a href="https://fahdshariff.blogspot.com/2014/01/fahdblog-in-2013.html">fahd.blog in 2013</a></li>
<li><a href="https://fahdshariff.blogspot.com/2013/01/fahdblog-in-2012.html">fahd.blog in 2012</a></li>
<li><a href="https://fahdshariff.blogspot.com/2012/01/fahdblog-in-2011.html">fahd.blog in 2011</a></li>
<li><a href="https://fahdshariff.blogspot.com/2011/01/fahdblog-in-2010.html">fahd.blog in 2010</a></li>
</ul>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-53752344293797267702022-12-27T12:07:00.003+00:002022-12-27T12:07:33.822+00:00Java: Collecting a Stream into an Existing Collection<p>The following snippet shows how you can collect a stream into an existing collection using <a href="https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collectors.html#toCollection-java.util.function.Supplier-"><code>Collectors.toCollection</code></a>:</p>
<pre class="brush:java; gutter:false;">
stream.collect(Collectors.toCollection(() -> existingCollection));
</pre>
<p>You may also be interested in reading my <a href="https://fahdshariff.blogspot.com/2018/06/java-10-collecting-stream-into.html">previous post</a> about collecting a stream into an unmodifiable collection.</p>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-66831003209391843582022-12-24T17:29:00.002+00:002022-12-24T17:29:34.573+00:00Java 19: Virtual Threads<p>Java 19 introduces <a href="https://bugs.openjdk.org/browse/JDK-8277131">Virtual Threads</a>, which are lightweight threads designed to improve application throughput. This is a preview language feature so must be enabled using <code>--enable-preview</code>.</p>
<p>As you know, the JDK implements platform threads (<code>java.lang.Thread</code>) as thin wrappers around operating system (OS) threads. OS threads are expensive to create, and the number of threads is limited to the number of OS threads that the underlying hardware can support. A platform thread captures the OS thread for the code's entire lifetime.</p>
<p>On the other hand, a virtual thread is an instance of <code>java.lang.Thread</code> that is not tied to a particular OS thread and does not capture the OS thread for the code's entire lifetime. A virtual thread consumes an OS thread only while it performs calculations on the CPU. This means that several virtual threads can run their Java code on the same OS thread, effectively sharing it. When code running in a virtual thread calls a blocking I/O operation, the JVM performs a non-blocking OS call and automatically suspends the virtual thread until it can be resumed later.</p>
<p>Virtual threads help to improve the throughput of thread-per-request style server applications in particular because such applications consist of a great number of concurrent tasks that are not CPU bound and spend much of their time waiting.</p>
<p>Here is an example which creates 10,000 virtual threads; however, the JDK runs the code on perhaps only one OS thread. If we were using 10,000 platform threads (and thus 10,000 OS threads) instead, the program might crash, depending on the hardware available. Virtual threads are cheap and plentiful, and there is no need to pool them.</p>
<pre class="brush:java; gutter:false;">
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
IntStream.range(0, 10_000).forEach(i -> {
executor.submit(() -> {
Thread.sleep(Duration.ofSeconds(1));
return i;
});
});
}
</pre>
<p>The <a href="https://docs.oracle.com/en/java/javase/19/docs/api/java.base/java/lang/Thread.html"><code>java.lang.Thread</code></a> class has been updated with new methods to create virtual and platform threads, such as:</p>
<pre class="brush:java; gutter:false;">
// create a new unstarted virtual thread named "foo".
Thread.ofVirtual().name("foo").unstarted(runnable);
// create and start a virtual thread
Thread.startVirtualThread(runnable)
</pre>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-52743775040110186742022-11-12T10:00:00.001+00:002022-11-12T10:00:00.181+00:00Java 19: Record Patterns<p>Java 19 introduces <a href="https://openjdk.org/jeps/405">Record Patterns</a>, a <a href="https://openjdk.org/jeps/12">preview language feature</a>, which allow us to "deconstruct" records and access their components directly.</p>
<p>Here is an example:</p>
<pre class="brush:java; gutter:false;">
record Book(String title, String author, double price) {}
static void print(Book b) {
if (b instanceof Book(String title, String author, double price)) {
System.out.printf("%s by %s for %.2f", title, author, price);
}
}
</pre>
<p><code>Book(String title, String author, double price)</code> is a record pattern that decomposes an instance of a <code>Book</code> record into its components: <code>title</code>, <code>author</code> and <code>price</code>.</p>
<p><code>null</code> does not match any record pattern.</p>
<p>Record patterns can be used in switch statements and expressions as well:<p>
<pre class="brush:java; gutter:false;">
static double getPrice(Publication p) {
return switch(p) {
case Book(var title, var author, var price) -> price;
case Magazine(var title, var publisher, var price) -> price;
default -> throw new IllegalArgumentException("Invalid publication: " + p);
};
}
</pre>
<p>Note that I have used <code>var</code> inside the record patterns, allowing the compiler to infer the type of each component, thus saving me from having to explicitly state it.</p>
<p>Record patterns can also be nested inside one another, in order to decompose complicated object graphs. For example:</p>
<pre class="brush:java; gutter:false;">
record Author(String firstName, String lastName) {}
record Book(String title, Author author, double price) {}
static void print(Book b) {
if (b instanceof Book(var title, Author(var firstName, var lastName), var price)) {
System.out.printf("%s by %s %s for %.2f", title, firstName, lastName, price);
}
}
</pre>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-84417967415550782582022-11-05T19:47:00.000+00:002022-11-05T19:47:41.238+00:00Java 19: Guarded Patterns in Switch<p><a href="https://fahdshariff.blogspot.com/2021/09/java-17-pattern-matching-for-switch.html">Previously</a>, I wrote about how <code>switch</code> statements and expressions had been enhanced to match on type patterns, and also how "guarded patterns" can be used to refine a pattern so that it is only matched on certain conditions.</p>
<p>In <a href="https://openjdk.org/jeps/427">Java 19</a>, the syntax of the guarded pattern has been changed so that instead of using <code>&&</code>, you need to use a <code>when</code> clause, as shown in the example below.</p>
<pre class="brush:java; gutter:false;">
static String guardedPattern(Collection<String> coll) {
return switch(coll) {
case null ->
"Collection is null!";
case List list
when list.size() > 10 ->
"I am a big List. My size is " + list.size();
case List list ->
"I am a small List. My size is " + list.size();
default ->
"Unsupported collection: " + coll.getClass();
};
}
</pre>
<p>As an aside, it's worth pointing out how <code>null</code>s are handled within the switch block. The <code>default</code> label does NOT match <code>null</code>s, so you need to explicitly add a <code>case null</code>, otherwise you will get a <code>NullPointerException</code>. This is for backwards compatibility with the current semantics of <code>switch</code>.</p>
<b><font color="#000099">Related post:</font></b><br/>
<a href="https://fahdshariff.blogspot.com/2021/09/java-17-pattern-matching-for-switch.html">Java 17: Pattern Matching for Switch</a>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-23680838231593243372022-05-31T12:41:00.001+01:002022-05-31T12:41:36.937+01:00Java 18: Simple Web Server<p>Java 18 offers an out-of-the-box <a href="https://openjdk.java.net/jeps/408">simple web server</a> (<code>jwebserver</code>) that serves static files only (no servlet-like functionality or CGI). This tool is useful for prototyping, ad hoc coding, and testing.</p>
<p>To start the server, simply run:</p>
<pre style="font-family: Consolas,Courier New,Courier,mono,serif; font-size:12px;">
<font color="green">$ jwebserver</font>
Binding to loopback by default. For all interfaces use "-b 0.0.0.0" or "-b ::".
Serving C:\Users\fahd\blog and subdirectories on 127.0.0.1 port 8000
URL http://127.0.0.1:8000/
127.0.0.1 - - [31/May/2022:10:37:31 +0100] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [31/May/2022:10:37:33 +0100] "GET /2022/ HTTP/1.1" 200 -
</pre>
<p>By default, the server binds to localhost:8000 and serves the current working directory. Every request is logged to the console.</p>
<p>You can change the bind address, port number, directory and logging format using the options shown below:</p>
<pre style="font-family: Consolas,Courier New,Courier,mono,serif; font-size:12px;">
<font color="green">$ jwebserver --help</font>
Usage: jwebserver [-b bind address] [-p port] [-d directory]
[-o none|info|verbose] [-h to show options]
[-version to show version information]
Options:
-b, --bind-address - Address to bind to. Default: 127.0.0.1 (loopback).
For all interfaces use "-b 0.0.0.0" or "-b ::".
-d, --directory - Directory to serve. Default: current directory.
-o, --output - Output format. none|info|verbose. Default: info.
-p, --port - Port to listen on. Default: 8000.
-h, -?, --help - Prints this help message and exits.
-version, --version - Prints version information and exits.
To stop the server, press Ctrl + C.
</pre>
<p>To programmatically start the web server from within a java application, you can use the <code><a href="https://docs.oracle.com/en/java/javase/18/docs/api/jdk.httpserver/com/sun/net/httpserver/SimpleFileServer.html#createFileServer(java.net.InetSocketAddress,java.nio.file.Path,com.sun.net.httpserver.SimpleFileServer.OutputLevel)">SimpleFileServer.createFileServer</a></code> method:</p>
<pre class="brush:java; gutter:false;">
import java.net.InetSocketAddress;
import java.nio.file.Path;
import com.sun.net.httpserver.SimpleFileServer;
import com.sun.net.httpserver.SimpleFileServer.OutputLevel;
final var server = SimpleFileServer.createFileServer(
new InetSocketAddress(8080),
Path.of("C:\\Users\\fahd\\blog"),
OutputLevel.VERBOSE);
server.start();
</pre>
<p>Alternatively, use <code><a href="https://docs.oracle.com/en/java/javase/18/docs/api/jdk.httpserver/com/sun/net/httpserver/HttpServer.html#create">HttpServer.create</a></code> if you wish to pass in your own HTTP handler and filter:</p>
<pre class="brush:java; gutter:false;">
import java.net.InetSocketAddress;
import java.nio.file.Path;
import com.sun.net.httpserver.HttpServer;
import com.sun.net.httpserver.SimpleFileServer;
import com.sun.net.httpserver.SimpleFileServer.OutputLevel;
final var server = HttpServer.create(
new InetSocketAddress(8000),
10,
"/context/",
SimpleFileServer.createFileHandler(Path.of("C:\\Users\\fahd\\blog")),
SimpleFileServer.createOutputFilter(System.out, OutputLevel.INFO));
server.start();
</pre>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-38887440512302682392022-01-03T16:23:00.000+00:002022-01-03T16:23:45.313+00:00Advent of Code 2021<p>At the end of last year, I took part in <b><a href="https://adventofcode.com">Advent of Code</a></b>, a programming competition that takes place in December every year. It is an <a href="https://en.wikipedia.org/wiki/Advent_calendar">Advent calendar</a> of programming puzzles - a new puzzle is released every day from 1-Dec to 25-Dec - and is a great way to test your programming skills and brush up on those algorithms that you don't use very often (like <a href="https://en.wikipedia.org/wiki/Dijkstra%27s_algorithm">Djikstra's</a>!). There were some really challenging problems and I am pleased that I managed to answer them all using Java. I actually surprised myself on a couple of them because when I first read the question, I didn't think I'd be able to do it. However, I persevered (sometimes even spending the whole day on the problem) and took insipiration from other programmers on the <a href="https://www.reddit.com/r/adventofcode/">reddit</a> board.</p>
<p>The questions are still available so I would encourage you to have a go!</p>
<p>Bring on Advent of Code 2022!</p>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-64928895551417521212022-01-01T17:58:00.000+00:002022-01-01T17:58:37.938+00:00fahd.blog in 2021<p><b><font color="#000099">Happy 2022, everyone!</font></b></p><p>I'd like to wish everyone a great start to an even greater new year!</p>
<p>In keeping with tradition, here's one last look back at <a href="http://fahdshariff.blogspot.com">fahd.blog</a> in 2021.</p>
<p>During 2021, I posted 6 new entries on <a href="http://fahdshariff.blogspot.com">fahd.blog</a>. I am also thrilled that I have more readers from all over the world! Thanks for reading and especially for giving feedback.</p>
<b><font color="#000099">Top 3 posts of 2021:</font></b>
<ul>
<li><a href="https://fahdshariff.blogspot.com/2021/04/java-16-streamtolist.html">Java 16: Stream.toList()</a></li>
<li><a href="https://fahdshariff.blogspot.com/2021/03/creating-temporary-files-with-junit-5.html">Creating Temporary Files with JUnit 5</a></li>
<li><a href="https://fahdshariff.blogspot.com/2021/04/kdbq-display-table-as-tree.html">kdb+/q - Display a Table as a Tree</a></li>
</ul>
<p>I'm going to be writing a lot more this year, so stay tuned for more great techie tips, tricks and hacks! :)</p>
<b><font color="#000099">Related posts:</font></b>
<ul>
<li><a href="https://fahdshariff.blogspot.com/2021/01/fahdblog-in-2020.html">fahd.blog in 2020</a></li>
<li><a href="https://fahdshariff.blogspot.com/2020/01/fahdblog-in-2019.html">fahd.blog in 2019</a></li>
<li><a href="https://fahdshariff.blogspot.com/2019/01/fahdblog-in-2018.html">fahd.blog in 2018</a></li>
<li><a href="https://fahdshariff.blogspot.com/2018/01/fahdblog-in-2017.html">fahd.blog in 2017</a></li>
<li><a href="https://fahdshariff.blogspot.com/2017/01/fahdblog-in-2016.html">fahd.blog in 2016</a></li>
<li><a href="https://fahdshariff.blogspot.com/2016/01/fahdblog-in-2015.html">fahd.blog in 2015</a></li>
<li><a href="https://fahdshariff.blogspot.com/2015/01/fahdblog-in-2014.html">fahd.blog in 2014</a></li>
<li><a href="https://fahdshariff.blogspot.com/2014/01/fahdblog-in-2013.html">fahd.blog in 2013</a></li>
<li><a href="https://fahdshariff.blogspot.com/2013/01/fahdblog-in-2012.html">fahd.blog in 2012</a></li>
<li><a href="https://fahdshariff.blogspot.com/2012/01/fahdblog-in-2011.html">fahd.blog in 2011</a></li>
<li><a href="https://fahdshariff.blogspot.com/2011/01/fahdblog-in-2010.html">fahd.blog in 2010</a></li>
</ul>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-70599574216272071252021-09-18T18:01:00.001+01:002021-09-18T18:01:52.630+01:00Java 17: Pattern Matching for Switch<p>In <a href="https://jdk.java.net/17/release-notes">Java 17</a> (released only a few days ago), <a href="https://openjdk.java.net/jeps/406">Pattern Matching for switch</a> has been introduced as a preview language feature, which allows case labels with patterns rather than just constants. Here is an example showing how you can match on <b>type patterns</b>:</p>
<pre class="brush:java; gutter:false;">
public static String typedPatternMatching(Object o) {
return switch(o) {
case null -> "I am null";
case String s -> "I am a String. My value is " + s;
case Integer i -> "I am an int. My value is " + i;
default -> "I am of an unknown type. My value is " + o.toString();
};
}
// Output:
> typedPatternMatching("HELLO")
"I am a String. My value is HELLO"
> typedPatternMatching(123)
"I am an int. My value is 123"
> typedPatternMatching(null)
"I am null"
> typedPatternMatching(0.5)
"I am of an unknown type. My value is 0.5"
</pre>
<p>You can also use a <b>guarded pattern</b> in order to refine a pattern so that it is only matched on certain conditions, for example:</p>
<pre class="brush:java; gutter:false;">
public static String guardedPattern(Collection<String> coll) {
return switch(coll) {
case List list && (list.size() > 10) ->
"I am a big List. My size is " + list.size();
case List list ->
"I am a small List. My size is " + list.size();
default ->
"Unsupported collection: " + coll.getClass();
};
}
</pre>
<p>If you have a <a href="https://fahdshariff.blogspot.com/2020/10/java-15-sealed-classes.html">Sealed Class</a> (made a permanent language feature in Java 17), the compiler can verify if the switch statement is complete so no <code>default</code> label is needed. For example:</p>
<pre class="brush:java; gutter:false;">
sealed interface Vehicle permits Car, Truck, Motorcycle {}
final class Car implements Vehicle {}
final class Truck implements Vehicle {}
final class Motorcycle implements Vehicle {}
public static String sealedClass(Vehicle v) {
return switch(v) {
case Car c -> "I am a car";
case Truck t -> "I am a truck";
case Motorcycle m -> "I am a motorcycle";
};
}
</pre>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-334953344814809872021-08-14T15:49:00.001+01:002021-09-18T16:05:17.924+01:00Java 16: Stream.mapMulti<p>Java 16 introduces a new <a href="https://docs.oracle.com/en/java/javase/16/docs/api/java.base/java/util/stream/Stream.html#mapMulti(java.util.function.BiConsumer)"><code>Stream.mapMulti</code></a> method which allows you to replace elements in a stream with multiple elements.</p>
<p>The example below shows how you can use <code>mapMulti</code> to replace each string in a stream with its uppercased and lowercased versions:</p>
<pre class="brush:java; gutter:false;">
Stream.of("Twix", "Snickers", "Mars")
.mapMulti((s, c) -> {
c.accept(s.toUpperCase());
c.accept(s.toLowerCase());
})
.forEach(System.out::println);
Output:
TWIX
twix
SNICKERS
snickers
MARS
mars
</pre>
<p>The same thing can also be achieved using <code>flatMap</code> like this:
<pre class="brush:java; gutter:false;">
Stream.of("Twix", "Snickers", "Mars")
.flatMap(s -> Stream.of(s.toUpperCase(), s.toLowerCase()))
.forEach(System.out::println);
</pre>
<p>So what is the difference between <code>mapMulti</code> and <code>flatMap</code>? According to the <a href="https://docs.oracle.com/en/java/javase/16/docs/api/java.base/java/util/stream/Stream.html#mapMulti(java.util.function.BiConsumer)">javadocs</a>:</p>
<blockquote><i>
<p>This method is preferable to flatMap in the following circumstances:</p>
<ul>
<li>When replacing each stream element with a small (possibly zero) number of elements. Using this method avoids the overhead of creating a new Stream instance for every group of result elements, as required by flatMap.</li>
<li>When it is easier to use an imperative approach for generating result elements than it is to return them in the form of a Stream.</li>
</ul></i>
</blockquote>
<p>Inspecting the code for <code>multiMap</code>, we can see that it delegates to <code>flatMap</code>, however, it makes use of a <code>SpinedBuffer</code> to hold the elements before creating the stream, thus avoiding the overhead of creating new streams per group of result elements.</p>
<pre class="brush:java; gutter:false;">
default <R> Stream<R> mapMulti(BiConsumer<? super T, ? super Consumer<R>> mapper) {
Objects.requireNonNull(mapper);
return flatMap(e -> {
SpinedBuffer<R> buffer = new SpinedBuffer<>();
mapper.accept(e, buffer);
return StreamSupport.stream(buffer.spliterator(), false);
});
}
</pre>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-26700942205549994542021-04-10T08:00:00.000+01:002021-04-10T08:00:00.446+01:00Java 16: Stream.toList()<p>Java 16 introduces a handy new <a href="https://download.java.net/java/early_access/jdk16/docs/api/java.base/java/util/stream/Stream.html#toList()"><code>Stream.toList()</code></a> method which makes it easier to convert a stream into a list. The returned list is unmodifiable and calls to any mutator method will throw an <code>UnsupportedOperationException</code>.</p>
<p>Here is some sample code:</p>
<pre class="brush:java; gutter:false;">
import java.util.stream.Stream;
import static java.util.stream.Collectors.*;
// Java 16
stream.toList(); // returns an unmodifiable list
// Other ways to create Lists from Streams:
stream.collect(toList());
stream.collect(toCollection(LinkedList::new)); // if you need a specific type of list
stream.collect(toUnmodifiableList()); // introduced in Java 10
stream.collect(
collectingAndThen(toList(), Collections::unmodifiableList)); // pre-Java 10
</pre>
<p><b><font color="#000099">Related post: </font></b><a href="http://fahdshariff.blogspot.com/2018/06/java-10-collecting-stream-into.html">Java 10: Collecting a Stream into an Unmodifiable Collection</a></p>Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-20265178437591978732021-04-05T08:00:00.002+01:002021-04-05T08:00:00.223+01:00Java 16: Records and Pattern Matching for instanceof<p>In <a href="https://www.oracle.com/java/technologies/javase/16-all-relnotes.html">Java 16</a>, Records and Pattern Matching have been made a final and permanent feature of the Java language!</p>
<p>I blogged about them when they were first released as preview language features back in Java 14 here:</p>
<ul>
<li><a href="http://fahdshariff.blogspot.com/2020/04/java-14-records.html">Java 14: Records</a></li>
<li><a href="http://fahdshariff.blogspot.com/2020/04/java-14-pattern-matching-for-instanceof.html">Java 14: Pattern Matching for instanceof</a></li>
</ul>
Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-17908060809163898272021-04-02T18:12:00.002+01:002021-04-06T10:54:43.273+01:00kdb+/q - Display a Table as a Tree<p>This post shows how you can convert a keyed table to a hierarchical tree format in kdb+/q. This could be useful if you want to display data as a tree widget in a front-end.</p>
<p>Consider the following keyed table of world populations:</p>
<pre class="brush:java; gutter:false;">
continent country city | population
---------------------------------------------| ----------
North America United States New York City | 8550405
North America United States Los Angeles | 3971883
North America Mexico Mexico City | 8918653
Europe United Kingdom London | 9126366
Europe Russia Moscow | 12195221
Europe Russia Saint Petersburg| 5383890
Africa Nigeria Lagos | 14862000
Africa Egypt Cairo | 9908788
Africa Egypt Giza | 8800000
Asia China Shanghai | 22315474
Asia India Mumbai | 12691836
Asia China Beijing | 11716620
</pre>
<p>We would like to display it as a tree of continent > country > city, as shown below (similar to a pivot table in Excel):</p>
<pre class="brush:java; gutter:false;">
node | population
----------------------------| ----------
Total | 128441136
Asia | 46723930
China | 34032094
Shanghai | 22315474
Beijing | 11716620
India | 12691836
Mumbai | 12691836
Africa | 33570788
Egypt | 18708788
Cairo | 9908788
Giza | 8800000
Nigeria | 14862000
Lagos | 14862000
Europe | 26705477
Russia | 17579111
Moscow | 12195221
Saint Petersburg| 5383890
United Kingdom | 9126366
London | 9126366
North America | 21440941
United States | 12522288
New York City | 8550405
Los Angeles | 3971883
Mexico | 8918653
Mexico City | 8918653
</pre>
<p>In order to achieve this, we need to aggregate the data with different groupings, then combine the resultant tables and format it into a tree.</p>
<p><b><font color="#000099">1. Grouping the data</font></b></p>
<p>First, we will add a dummy <code>Total</code> column to the table and then aggregate the table with the following groupings:</p>
<ul>
<li>Total</li>
<li>Total, continent</li>
<li>Total, continent, country</li>
<li>Total, continent, country, city</li>
</ul>
<p>The code for this is shown below:</p>
<pre class="brush:java; gutter:false;">
// add Total column to the table. (td is a keyed table)
td:(`Total,keys[td]) xkey update Total:`Total from td;
keyCols:keys td;
// create a list of groupings
groupings:(1+til count keyCols) sublist\: keyCols;
// aggregate the table with each grouping
// this gives us a list of keyed tables (one per grouping)
tds:?[td;();;c!(sum;)each c:cols value td] each {x!x} each groupings;
// this step is optional but it's nice to sort each table on population
tds:`population xdesc'tds;
</pre>
<p><b><font color="#000099">2. Joining the data</font></b></p>
<p>Next, we need to join the tables that were obtained as a result of the groupings. We do this by unkeying the tables and then using <code>uj</code>:</p>
<pre class="brush:java; gutter:false;">
td:keyCols xkey (uj/) 0!'tds;
</pre>
<p><b><font color="#000099">3. Formatting the data</font></b></p>
<p>Now let's add a Path column by <a href="http://fahdshariff.blogspot.com/2016/04/kdbq-concatenating-columns.html">concatenating the key columns</a>:</p>
<pre class="brush:java; gutter:false;">
td:![td;();0b;enlist[`Path]!enlist(`$sv';">";(string;(each;{x except `};(flip;enlist,keyCols))))];
td:(`Path,keyCols) xkey td;
</pre>
<p>This is what our tree looks like so far:</p>
<pre class="brush:java; gutter:false;">
Path Total continent country city | population
---------------------------------------------------------------------------------------------------| ----------
Total Total | 128441136
Total>Asia Total Asia | 46723930
Total>Africa Total Africa | 33570788
Total>Europe Total Europe | 26705477
Total>North America Total North America | 21440941
Total>Asia>China Total Asia China | 34032094
Total>Africa>Egypt Total Africa Egypt | 18708788
Total>Europe>Russia Total Europe Russia | 17579111
Total>Africa>Nigeria Total Africa Nigeria | 14862000
Total>Asia>India Total Asia India | 12691836
Total>North America>United States Total North America United States | 12522288
Total>Europe>United Kingdom Total Europe United Kingdom | 9126366
Total>North America>Mexico Total North America Mexico | 8918653
Total>Asia>China>Shanghai Total Asia China Shanghai | 22315474
Total>Africa>Nigeria>Lagos Total Africa Nigeria Lagos | 14862000
Total>Asia>India>Mumbai Total Asia India Mumbai | 12691836
Total>Europe>Russia>Moscow Total Europe Russia Moscow | 12195221
Total>Asia>China>Beijing Total Asia China Beijing | 11716620
Total>Africa>Egypt>Cairo Total Africa Egypt Cairo | 9908788
Total>Europe>United Kingdom>London Total Europe United Kingdom London | 9126366
Total>North America>Mexico>Mexico City Total North America Mexico Mexico City | 8918653
Total>Africa>Egypt>Giza Total Africa Egypt Giza | 8800000
Total>North America>United States>New York City Total North America United States New York City | 8550405
Total>Europe>Russia>Saint Petersburg Total Europe Russia Saint Petersburg| 5383890
Total>North America>United States>Los Angeles Total North America United States Los Angeles | 3971883
</pre>
<p><b><font color="#000099">4. Reordering the rows</font></b></p>
<p>The tree looks okay so far and you can stop there if you want but it would look better if child nodes were directly under their parents e.g. Shanghai should appear under China. In order to do this, we cannot simply use <code>uj</code> to combine our tables but we need to use the <a href="https://code.kx.com/q4m3/6_Functions/#676-over-for-accumulation">Over (/) accumulator</a> to build the tree instead.</p>
<p>In order to get the row ordering correct, we add an id to each row, which will be a combination of the parent id and the row id. These id's look like this: 0, 0.0, 0.1, 0.1.1 etc. and will be used to sort the tree so that children appear under their parents.</p>
<p>Here is the final version of the code:</p>
<pre class="brush:java; gutter:false;">
// Converts a table into a tree.
// @param td - a keyed table
// @param sortCol - the column to sort on
// @returns a table with a tree column
table2tree:{[td;sortCol]
// add Total column to the table
td:(`Total,keys[td]) xkey update Total:`Total from td;
keyCols:keys td;
// create a list of groupings
groupings:(1+til count keyCols) sublist\: keyCols;
// aggregate the table with each grouping
// this gives us a list of keyed tables (one per grouping)
tds:?[td;();;c!(sum;)each c:cols value td] each {x!x} each groupings;
// sort the tables
if[not null sortCol;tds:sortCol xdesc'tds];
// initial tree only has the Total row
tree:update id:"0",node:enlist "Total" from 0!first tds;
// build the tree using the over accumulator
tree:{[tree;td]
keyCols:keys td;
// join the parent id to the current table
td:td lj k xkey ?[tree;();0b;{x!x}(k:-1_keyCols),`id];
// update the id by concatenating the parent id to the row id
// we need to left-pad the row id so that sorting works correctly
// e.g. 1.3 should come before 1.10
td:update id:`$"."sv'flip(string id;(-1*count string count td)$string i) from td;
// add a node column which corresponds to the value of the last key column
td:![td;();0b;enlist[`node]!enlist last keyCols];
// add indentation to the node based on the depth (i.e. number of key columns)
indentation:(4*-1+count keyCols)#" ";
td:update node:(indentation,/:string node) from td;
// now add the table to tree
tree uj 0!td
}/[tree;1_tds];
// sort the tree on id
(`node,keyCols) xkey `id xasc tree}
</pre>
<p>This is what our final tree looks like:</p>
<pre class="brush:java; gutter:false;">
q) data:3!("SSSI";enlist",") 0: `$"population.csv";
q) select node,population from table2tree[data;`population]
node population
-----------------------------------------
Total 128441136
Asia 46723930
China 34032094
Shanghai 22315474
Beijing 11716620
India 12691836
Mumbai 12691836
Africa 33570788
Egypt 18708788
Cairo 9908788
Giza 8800000
Nigeria 14862000
Lagos 14862000
Europe 26705477
Russia 17579111
Moscow 12195221
Saint Petersburg 5383890
United Kingdom 9126366
London 9126366
North America 21440941
United States 12522288
New York City 8550405
Los Angeles 3971883
Mexico 8918653
Mexico City 8918653
</pre>
<p>I also played around with adding lines to connect nodes of the tree but it got complicated very fast!</p>
<p>Can you think of a better way to do this? Let me know in the comments below!</p>
Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0tag:blogger.com,1999:blog-32637828.post-87961239383501438512021-03-06T08:00:00.001+00:002021-03-06T08:00:05.532+00:00Creating Temporary Files with JUnit 5<p>This post shows you how to perform unit testing using temporary files with <a href="https://junit.org/junit5/">JUnit 5</a>. If you're still on JUnit 4, please check out my <a href="http://fahdshariff.blogspot.com/2013/03/junit-creating-temporary-files-using.html">previous post</a>!</p>
<p>In JUnit 5, the <a href="https://junit.org/junit5/docs/5.4.0/api/org/junit/jupiter/api/io/TempDir.html"><code>@TempDir</code></a> annotation is used to indicate that a field or method parameter of type <code>Path</code> or <code>File</code> is a temporary directory. Each test will use its own temporary directory and when the test method has finished executing, the directory and all its contents will be deleted. (If you want to share a temporary directory between tests, you need to make the field <code>static</code>.)</p>
<p>Here is an example:</p>
<pre class="brush:java; gutter:false;">
import static org.hamcrest.CoreMatchers.*;
import static org.hamcrest.MatcherAssert.*;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
public class MyTest {
@TempDir
Path tempDir;
@Test
public void testWrite() throws IOException {
// Create a temporary file.
// This is guaranteed to be deleted after the test finishes.
final Path tempFile = Files.createFile(tempDir.resolve("myfile.txt"));
// Write something to it.
Files.writeString(tempFile, "Hello World");
// Read it.
final String s = Files.readString(tempFile);
// Check that what was written is correct.
assertThat("Hello World", is(s));
}
}
</pre>
<p><b><font color="#000099">Related post: </font></b><a href="http://fahdshariff.blogspot.com/2013/03/junit-creating-temporary-files-using.html">JUnit: Creating Temporary Files using the TemporaryFolder @Rule</a></p>
Fahd Shariffhttp://www.blogger.com/profile/00919911016127601294noreply@blogger.com0