## Sunday, October 23, 2011

### Finding the Maximum using Relational Algebra

We know that if you want to get the maximum value of a column in SQL, you can simply use the `MAX` function as shown below:
```SELECT MAX(value) FROM T
```
You can also do it without the MAX function as follows:
```SELECT T.* FROM T
MINUS
SELECT T.* FROM T, T as T2 WHERE T.value<T2.value
```
or:
```SELECT T.* FROM T
LEFT JOIN T as T2 ON T.value<T2.value
WHERE T2.value IS NULL
```
Relational Algebra:
Using Relational Algebra (RA) syntax, this would be:
```\project_{value}(T)
\diff
\project_{value} (
\select_{value < value2}(
\project_{value}(T)
\cross
\rename_{value2}(\project_{value}(T))
)
)
```
where:
• `\cross` is the relational cross-product operator
• `\diff` is the relational diff operator
• `\project_{attr_list}` is the relational projection operator
• `\rename_{new_attr_name_list}` is the relational renaming operator
• `\select_{cond}` is the relational selection operator

## Sunday, October 16, 2011

### Validating XML with xmllint

The following commands show you how to validate an XML file against a DTD or XSD using `xmllint`.

To validate an XML file against:

• a DTD stored in the same file:
• ```xmllint --valid --noout fileWithDTD.xml
```
• a DTD stored in a separate file:
• ```xmllint --dtdvalid DTD.dtd --noout fileWithoutDTD.xml
```
• an XSD stored in a separate file:
• ```xmllint --schema schema.xsd --noout file.xml
```
The `--noout` option suppresses the output of the xml file.

Example
To validate:

```<countries>
<country name="Afghanistan" population="22664136" area="647500">
<language percentage="11">Turkic</language>
<language percentage="35">Pashtu</language>
<language percentage="50">Afghan Persian</language>
</country>
<country name="Albania" population="3249136" area="28750"/>
<country name="Algeria" population="29183032" area="2381740">
<city>
<name>Algiers</name>
<population>1507241</population>
</city>
</country>
</countries>
```
against:
```<!ELEMENT countries (country*)>
<!ELEMENT country (language|city)*>
<!ATTLIST country name CDATA #REQUIRED>
<!ATTLIST country population CDATA #REQUIRED>
<!ATTLIST country area CDATA #REQUIRED>
<!ELEMENT language (#PCDATA)>
<!ATTLIST language percentage CDATA #REQUIRED>
<!ELEMENT city (name, population)>
<!ELEMENT name (#PCDATA)>
<!ELEMENT population (#PCDATA)>
```
use the command:
```xmllint --dtdvalid countries.dtd --noout countries.xml
```

## Saturday, October 08, 2011

### Splitting a large file into smaller pieces

If you have a large file and want to break it into smaller pieces, you can use the Unix split command. You can tell it what the prefix of each split file should be and it will then append an alphabet (or number) to the end of each name.

In the example below, I split a file containing 100,000 lines. I instruct `split` to use numeric suffixes (`-d`), put 10,000 lines in each split file (`-l 10000`) and use suffixes of length 3 (`-a 3`). As a result, ten split files are created, each with 10,000 lines.

```\$ ls
hugefile

\$ wc -l hugefile
100000 hugefile

\$ split -d -l 10000 -a 3 hugefile hugefile.split.

\$ ls
hugefile                hugefile.split.005
hugefile.split.000      hugefile.split.006
hugefile.split.001      hugefile.split.007
hugefile.split.002      hugefile.split.008
hugefile.split.003      hugefile.split.009
hugefile.split.004

\$ wc -l *split*
10000 hugefile.split.000
10000 hugefile.split.001
10000 hugefile.split.002
10000 hugefile.split.003
10000 hugefile.split.004
10000 hugefile.split.005
10000 hugefile.split.006
10000 hugefile.split.007
10000 hugefile.split.008
10000 hugefile.split.009
100000 total
```

## Sunday, October 02, 2011

### Better Bash Completion for Tmux

In my previous post, I wrote about how awesome tmux is for managing multiple terminals. However, even though it is widely used, I haven't been able to find a good Bash completion script for it. The tmux package does come with `bash_completion_tmux.sh` but this does not complete command options or command aliases. So I wrote a better version which completes `tmux` commands, aliases and their options. However, there is still room for improvement. It would be nice if it could complete session and window names too, but I haven't found the time to implement this yet.

Here is a demo:

```\$ tmux lis[TAB]
list-buffers   list-clients   list-commands
list-keys      list-panes     list-sessions  list-windows

\$ tmux list-windows -[TAB]
-a -t

\$ tmux list-windows -a
sharfah:0: less [180x82] [layout f0de,180x82,0,0]
sharfah:1: tmp [180x82] [layout f0de,180x82,0,0] (active)
sharfah:2: isengard [180x82] [layout f0de,180x82,0,0]
sharfah:3: java [180x82] [layout f0de,180x82,0,0]
```
My completion script is shown below. You need to source it in your Bash profile. Alternatively, save it to your Bash completion directory e.g. `~/.bash/.bash/.bash_completion.d` and it should automatically get picked up.

The script is also available in my GitHub dotfiles repository. If you can improve it, fork it and send me a pull request!

```#
# tmux completion
# by Fahd Shariff
#
_tmux() {
# an array of commands and their options
declare -A tmux_cmd_map
tmux_cmd_map=( ["attach-session"]="-dr -t target-session" \
["bind-key"]="-cnr -t key-table key command arguments" \
["break-pane"]="-d -t target-pane" \
["capture-pane"]="-b buffer-index -E end-line -S start-line -t target-pane" \
["choose-buffer"]="-t target-window template" \
["choose-client"]="-t target-window template" \
["choose-session"]="-t target-window template" \
["choose-window"]="-t target-window template" \
["clear-history"]="-t target-pane" \
["clock-mode"]="-t target-pane" \
["command-prompt"]="-I inputs -p prompts -t target-client template" \
["confirm-before"]="-p prompt -t target-client command" \
["copy-mode"]="-u -t target-pane" \
["delete-buffer"]="-b buffer-index" \
["detach-client"]="-P -s target-session -t target-client" \
["display-message"]="-p -c target-client -t target-pane message" \
["display-panes"]="-t target-client" \
["find-window"]="-t target-window match-string" \
["has-session"]="-t target-session" \
["if-shell"]="shell-command command" \
["join-pane"]="-dhv -p percentage|-l size -s src-pane -t dst-pane" \
["kill-pane"]="-a -t target-pane" \
["kill-server"]="kill-server" \
["kill-session"]="-t target-session" \
["kill-window"]="-t target-window" \
["last-pane"]="-t target-window" \
["last-window"]="-t target-session" \
["link-window"]="-dk -s src-window -t dst-window" \
["list-buffers"]="list-buffers" \
["list-clients"]="-t target-session" \
["list-commands"]="list-commands" \
["list-keys"]="-t key-table" \
["list-panes"]="-as -t target" \
["list-sessions"]="list-sessions" \
["list-windows"]="-a -t target-session" \
["load-buffer"]="-b buffer-index path" \
["lock-client"]="-t target-client" \
["lock-server"]="lock-server" \
["lock-session"]="-t target-session" \
["move-window"]="-dk -s src-window -t dst-window" \
["new-session"]="-d -n window-name -s session-name -t target-session -x width -y height command" \
["new-window"]="-adk -n window-name -t target-window command" \
["next-layout"]="-t target-window" \
["next-window"]="-a -t target-session" \
["paste-buffer"]="-dr -s separator -b buffer-index -t target-pane" \
["pipe-pane"]="-t target-pane-o command" \
["previous-layout"]="-t target-window" \
["previous-window"]="-a -t target-session" \
["refresh-client"]="-t target-client" \
["rename-session"]="-t target-session new-name" \
["rename-window"]="-t target-window new-name" \
["resize-pane"]="-DLRU -t target-pane adjustment" \
["respawn-pane"]="-k -t target-pane command" \
["respawn-window"]="-k -t target-window command" \
["rotate-window"]="-DU -t target-window" \
["run-shell"]="command" \
["save-buffer"]="-a -b buffer-index" \
["select-layout"]="-np -t target-window layout-name" \
["select-pane"]="-lDLRU -t target-pane" \
["select-window"]="-lnp -t target-window" \
["send-keys"]="-t target-pane key " \
["send-prefix"]="-t target-pane" \
["server-info"]="server-info" \
["set-buffer"]="-b buffer-index data" \
["set-environment"]="-gru -t target-session name value" \
["set-option"]="-agsuw -t target-session|target-window option value" \
["set-window-option"]="-agu -t target-window option value" \
["show-buffer"]="-b buffer-index" \
["show-environment"]="-g -t target-session" \
["show-messages"]="-t target-client" \
["show-options"]="-gsw -t target-session|target-window" \
["show-window-options"]="-g -t target-window" \
["source-file"]="path" \
["split-window"]="-dhvP -p percentage|-l size -t target-pane command" \
["start-server"]="start-server" \
["suspend-client"]="-t target-client" \
["swap-pane"]="-dDU -s src-pane -t dst-pane" \
["swap-window"]="-d -s src-window -t dst-window" \
["switch-client"]="-lnp -c target-client -t target-session" \
["unbind-key"]="-acn -t key-table key" \
["unlink-window"]="-k -t target-window" )

declare -A tmux_alias_map
tmux_alias_map=( ["attach"]="attach-session" \
["detach"]="detach-client" \
["has"]="has-session" \
["lsc"]="list-clients" \
["lscm"]="list-commands" \
["ls"]="list-sessions" \
["lockc"]="lock-client" \
["locks"]="lock-session" \
["new"]="new-session" \
["refresh"]="refresh-client" \
["rename"]="rename-session" \
["showmsgs"]="show-messages" \
["source"]="source-file" \
["start"]="start-server" \
["suspendc"]="suspend-client" \
["switchc"]="switch-client" \
["breakp"]="break-pane" \
["capturep"]="target-pane]" \
["displayp"]="display-panes" \
["findw"]="find-window" \
["joinp"]="join-pane" \
["killp"]="kill-pane" \
["killw"]="kill-window" \
["lastp"]="last-pane" \
["last"]="last-window" \
["linkw"]="link-window" \
["lsp"]="list-panes" \
["lsw"]="list-windows" \
["movew"]="move-window" \
["neww"]="new-window" \
["nextl"]="next-layout" \
["next"]="next-window" \
["pipep"]="pipe-pane" \
["prevl"]="previous-layout" \
["prev"]="previous-window" \
["renamew"]="rename-window" \
["resizep"]="resize-pane" \
["respawnp"]="respawn-pane" \
["respawnw"]="respawn-window" \
["rotatew"]="rotate-window" \
["selectl"]="select-layout" \
["selectp"]="select-pane" \
["selectw"]="select-window" \
["splitw"]="[shell-command]" \
["swapp"]="swap-pane" \
["swapw"]="swap-window" \
["unlinkw"]="unlink-window" \
["bind"]="bind-key" \
["lsk"]="list-keys" \
["send"]="send-keys" \
["unbind"]="unbind-key" \
["set"]="set-option" \
["setw"]="set-window-option" \
["show"]="show-options" \
["showw"]="show-window-options" \
["setenv"]="set-environment" \
["showenv"]="show-environment" \
["confirm"]="confirm-before" \
["display"]="display-message" \
["clearhist"]="clear-history" \
["deleteb"]="delete-buffer" \
["lsb"]="list-buffers" \
["loadb"]="load-buffer" \
["pasteb"]="paste-buffer" \
["saveb"]="save-buffer" \
["setb"]="set-buffer" \
["showb"]="show-buffer" \
["if"]="if-shell" \
["lock"]="lock-server" \
["run"]="run-shell" \
["info"]="server-info" )

local cur="\${COMP_WORDS[COMP_CWORD]}"
local prev="\${COMP_WORDS[COMP_CWORD-1]}"
COMPREPLY=()

# completing an option
if [[ "\$cur" == -* ]]; then
#tmux options
if [[ "\$prev" == "tmux" ]]; then
COMPREPLY=( \$( compgen -W "-2 -8 -c -f -L -l -q -S -u -v -V" -- \$cur ) )
else
#find the tmux command so that we can complete the options
local cmd="\$prev"
local i=\$COMP_CWORD
while [[ "\$cmd" == -* ]]
do
cmd="\${COMP_WORDS[i]}"
((i--))
done

#if it is an alias, look up what the alias maps to
local alias_cmd=\${tmux_alias_map[\$cmd]}
if [[ -n \${alias_cmd} ]]
then
cmd=\${alias_cmd}
fi

#now work out the options to this command
local opts=""
for opt in \${tmux_cmd_map[\$cmd]}
do
if [[ "\$opt" == -* ]]; then
len=\${#opt}
i=1
while [ \$i -lt \$len ]; do
opts="\$opts -\${opt:\$i:1}"
((i++))
done
fi
done
COMPREPLY=(\$(compgen -W "\$opts" -- \${cur}))
fi
else
COMPREPLY=(\$(compgen -W "\$(echo \${!tmux_cmd_map[@]} \${!tmux_alias_map[@]})" -- \${cur}))
fi
return 0
}
complete -F _tmux tmux
```
Related posts:
Managing Multiple Terminals with Tmux Writing your own Bash Completion Function

## Saturday, October 01, 2011

### Managing Multiple Terminals with Tmux

I've started using tmux, which is a "terminal multiplexer", similar to screen. It allows you to manage a number of terminals from a single screen. So, for example, instead of having 5 PuTTY windows cluttering up your desktop, you now have only one window, containing 5 terminals. If you close this window, you can simply open a new one and "attach" to your running tmux session, to get all your terminals back at the same state you left them in.

There are lots of cool things you can do with tmux. For example, you can split a terminal window horizontally or vertically into "panes". This allows you to look at files side by side, or simply watch a process in one pane while you do something else in another.

I took the following screenshot of tmux in action:

The status bar along the bottom shows that I have 5 terminal windows open. I am currently in the one labelled "1-demo" and within this window I have 4 panes, each running a different command.

There are quite a few key bindings to learn, but once you have mastered them you will be able to jump back and forth between windows, move them around and kill them without lifting your hands off the keyboard. You can also set your own key bindings for things you do frequently. For example, my `Ctrl-b /` binding splits my window vertically and opens up a specified `man` page on the right. My `Ctrl+b S` binding allows me to SSH to a server in a new window.

Here is my tmux configuration taken from `~/.tmux.conf` which shows my key bindings and colour setup. You can download this file from my GitHub dotfiles repository.

```bind | split-window -h
bind - split-window -v
bind _ split-window -v
bind R source-file ~/.tmux.conf \; display-message "tmux.conf reloaded!"

bind / command-prompt -p "man" "split-window -h 'man %%'"
bind S command-prompt -p "ssh" "new-window -n %1 'exec ssh %1'"
bind h split-window -h  "man tmux"

set -g terminal-overrides 'xterm*:smcup@:rmcup@'

set -g history-limit 9999

# Terminal emulator window title
set -g set-titles on
set -g set-titles-string '#S:#I.#P #W'

# notifications
setw -g monitor-activity on
setw -g visual-activity on

# auto rename
setw -g automatic-rename on

# Clock
setw -g clock-mode-colour green
setw -g clock-mode-style 24

# Window status colors
setw -g window-status-bg colour235
setw -g window-status-fg colour248
setw -g window-status-alert-attr underscore
setw -g window-status-alert-bg colour235
setw -g window-status-alert-fg colour248
setw -g window-status-current-attr bright
setw -g window-status-current-bg colour235
setw -g window-status-current-fg colour248

# Message/command input colors
set -g message-bg colour240
set -g message-fg yellow
set -g message-attr bright

# Status Bar
set -g status-bg colour235
set -g status-fg colour248
set -g status-interval 1
set -g status-left '[#H]'
set -g status-right ''

set -g pane-border-fg white
set -g pane-border-bg default
set -g pane-active-border-fg white
set -g pane-active-border-bg default
```