Saturday, November 22, 2014

Ext JS - Caching AJAX Responses to HTML5 Web Storage for Better Performance

This post shows how you can improve performance of your Ext JS applications by using a "Caching AJAX Proxy". This proxy saves URL responses to HTML5 Web Storage (e.g. session storage or local storage), which means that when the same URL is requested multiple times, a cached response is returned, instead of sending a request to the server each time. This makes the application more responsive and also reduces load on the server handling the requests.

/**
 * A Caching Ajax Proxy which uses AJAX requests to get data from a server and
 * then stores the data to HTML5 Web Storage. If the storage fills up, it removes
 * entries from the cache until space is available.
 * (Compatible with Ext JS 4.2)
 */
Ext.define('App.data.proxy.CachingAjax', {
  extend: 'Ext.data.proxy.Ajax',
  alias: 'proxy.cachingajax',

  // use session storage, but can be configured to localStorage too
  storage: window.sessionStorage,

  // @Override
  doRequest: function(operation, callback, scope) {
    var cachedResponse = this.getItemFromCache(this.url);
    if (!cachedResponse) {
        this.callParent(arguments);
    }
    else {
        console.log('Got cached data for: ' + this.url);
        this.processResponse(true, operation, null, cachedResponse,
                             callback, scope, true);
    }
  },

  // @Override
  processResponse: function(success, operation, request, response,
                            callback, scope, isCached) {
    if (success === true && !isCached) {
        this.putItemInCache(this.url, response.responseText);
    }
    this.callParent(arguments);
  },

  /**
   * @private
   * Returns the data from the cache for the specified key
   * @param {String} the url
   * @return {String} the cached url response, or null if not in cache
   */
  getItemFromCache: function(key) {
    return this.storage ? this.storage.getItem(key) : null;
  },

  /**
   * @private
   * Puts an entry in the cache.
   * Removes a third of the entries if the cache is full.
   * @param {String} the url
   * @param {String} the data
   */
  putItemInCache: function(key, value) {
    if (!this.storage) return;
    try {
      this.storage.setItem(key, value);
    } catch (e) {
      // this might happen if the storage is full.
      // Remove a third of the items and retry.
      // If it fails again, disable the cache quietly.
      console.log('Error putting data in cache. CacheSize: ' + this.storage.length +
                  ', ErrorCode: ' + e.code + ', Message: ' + e.name);

      while (this.storage.length != 0) {
        var toRemove = this.storage.length / 3;
        for (var i = 0; i < toRemove ; i++) {
          var item = this.storage.key(0);
          if (item) this.storage.removeItem(item);
          else break;
        }
        console.log('Removed one-third of the cache. Cache size is now: ' + this.storage.length);
        try {
          this.storage.setItem(key, value);
          break;
        } catch (e) {
          console.log('Error putting data in cache again. CacheSize: ' + this.storage.length +
                      ', ErrorCode: ' + e.code + ', Message: ' + e.name);
        }
      }
      if (this.storage.length == 0) {
        console.log("Cache disabled");
        this.storage = null;
      }
    }
  }
});
Usage:
var store = Ext.create('Ext.data.Store', {
  model: 'User',
  proxy: {
    type: 'cachingajax',
    url : 'http://mywebsite/path'
  }
});

Obviously, you should only use this caching proxy when the server-side data is static, because if it is changing frequently your application will end up displaying stale, cached data.

This proxy can also be extended in the future to remove cached entries after specific time intervals or clear out the entire cache when the application starts up.

Saturday, October 25, 2014

stackoverflow - 90k rep

Five months after crossing the 80k milestone, I have now reached a reputation of 90k on stackoverflow!

The following table shows some stats about my journey so far:

0-10k 10-20k 20-30k 30-40k 40-50k 50-60k 60-70k 70-80k 80-90k Total
Date achieved 01/2011 05/2011 01/2012 09/2012 02/2013 07/2013 12/2013 05/2014 10/2014
Questions answered 546 376 253 139 192 145 66 58 32 1807
Questions asked 46 1 6 0 1 0 0 0 2 56
Tags covered 609 202 83 10 42 14 11 14 4 989
Badges
(gold, silver, bronze)
35
(2, 10, 23)
14
(0, 4, 10)
33
(2, 8, 23)
59
(3, 20, 36)
49
(0, 19, 30)
65
(2, 26, 37)
60
(5, 22, 33)
50
(2, 24, 24)
50
(7, 21, 25)
418
(23, 154, 241)

I'm a bit disappointed that I only managed to answer 32 questions over the last 5 months. It's because work has been keeping me so busy! For me, stackoverflow has not simply been a quest for reputation, but more about learning new technologies and picking up advice from other people on the site. I like to take on challenging questions, rather than the easy ones, because it pushes me to do research into areas I have never looked at before, and I learn so much during the process.

Next stop, 100k!

Tuesday, September 30, 2014

Ext JS CSV Data Reader

In a previous post, I showed how you can convert CSV to JSON in JavaScript. In this post, I will show how you can display CSV data in an Ext JS grid panel using a custom Reader.

Currently, there are three kinds of Readers available in Ext JS 4.2.2:

A CSV reader isn't available but is quite easy to create, simply by extending the JSON reader. The code for my CSV Reader is shown below. It first converts the CSV response from the server into JSON format and then invokes the parent JSON reader.

/**
 * The CSV Reader is used by a Proxy to read a server response
 * that is sent back in CSV format.
 */
Ext.define('CsvReader', {
    extend: 'Ext.data.reader.Json',
    alias : 'reader.csv',

    // converts csv into json
    toJson: function(csvData){
      var lines = csvData.split("\n");
      var colNames = lines[0].split(",");
      var records = [];
      for(var i = 1; i < lines.length; i++) {
        if (lines[i] == "") continue;
        var record = {};
        var bits = lines[i].split(",");
        for (var j = 0; j < bits.length; j++) {
          record[colNames[j]] = bits[j];
        }
        records.push(record);
      }
      return records;
    },

    // override
    getResponseData: function(response) {
        try {
            return this.readRecords(response.responseText);
        } catch (ex) {
            error = new Ext.data.ResultSet({
                total  : 0,
                count  : 0,
                records: [],
                success: false,
                message: ex.message
            });
            this.fireEvent('exception', this, response, error);
            console.log(error);
            return error;
        }
    },

    // override
    readRecords: function(strData) {
        var result = this.toJson(strData);
        return this.callParent([result]);
    }
});

Now let's use it to display an example CSV file containing book data in a grid.

author,title,publishDate
Dan Simmons,Hyperion,1989
Douglas Adams,The Hitchhiker's Guide to the Galaxy,1979

Here is some sample code to read the CSV file into a grid using the CSV reader defined above:

Ext.define('Book', {
    extend: 'Ext.data.Model',
    fields: [
        {name: 'author', type: 'string'},
        {name: 'title', type: 'string'},
        {name: 'publishDate', type: 'string'}
    ]
});

var bookStore = Ext.create('Ext.data.Store', {
    model: 'Book',
    autoLoad: true,
    proxy: {
        type: 'ajax',
        reader: 'csv',
        url: 'data.csv'
    }
});

Ext.application({
    name: 'MyApp',
    launch: function() {
        Ext.create('Ext.Viewport', {
            layout: 'fit',
            items: [{
                xtype:'grid',
                title: 'Books',
                store: bookStore,
                columns: [
                   { text: 'Author', dataIndex: 'author' },
                   { text: 'Title', dataIndex: 'title', flex:1 },
                   { text: 'Publish Date', dataIndex: 'publishDate' }
                ],
                height: 200,
                width: 400
            }],
            renderTo: Ext.getBody()
        });
    }
});

Related posts:
Converting CSV to JSON in JavaScript

Sunday, August 31, 2014

My Git Aliases

A git alias gives you the ability to run a long or hard-to-remember git command using a simple name. They are configured in your .gitconfig file.

One of my favourite aliases is git ls which lists all your commits in a nice format. In addition, git ll shows you what files were committed in each commit.

My git aliases are shown below. (For the latest version of my .gitconfig, visit my GitHub dotfiles repository):

[alias]
    st = status
    co = checkout
    br = branch
    df = diff
    ci = commit
    ca = commit -a --amend -C HEAD
    desc = describe
    rb = rebase -i master --autosquash
    cp = cherry-pick

    who = shortlog -s --
    ls = log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)[%an]%Creset' --abbrev-commit --date=relative
    ll = log --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)[%an]%Creset' --decorate --numstat
    ld = log --pretty=format:'%C(red)%h %Cgreen%ad%C(yellow)%d %Creset%s%C(bold blue) [%cn]%Creset' --decorate --date=short

    # list aliases
    la = "!git config -l | grep alias | cut -c 7-"

If you have any useful aliases, please share them in the comments section below.

You might also like:
My Bash Profile
My Bash Aliases

Saturday, August 30, 2014

Converting CSV to JSON in JavaScript

This post shows how you can convert a simple CSV file to JSON in JavaScript.

Consider the following sample CSV:

author,title,publishDate
Dan Simmons,Hyperion,1989
Douglas Adams,The Hitchhiker's Guide to the Galaxy,1979

The desired JSON output is:

[{"author":"Dan Simmons","title":"Hyperion","publishDate":"1989"},
{"author":"Douglas Adams","title":"The Hitchhiker's Guide to the Galaxy","publishDate":"1979"}]

The following JavaScript function transforms CSV into JSON. (Note that this implementation is quite naive and will not handle quoted fields containing commas!)

function toJson(csvData) {
  var lines = csvData.split("\n");
  var colNames = lines[0].split(",");
  var records=[];
  for(var i = 1; i < lines.length; i++) {
    var record = {};
    var bits = lines[i].split(",");
    for (var j = 0 ; j < bits.length ; j++) {
      record[colNames[j]] = bits[j];
    }
    records.push(record);
  }
  return records;
}

A simple test:

csv="author,title,publishDate\nDan Simmons,Hyperion,1989\nDouglas Adams,The Hitchhiker's Guide to the Galaxy,1979";
json = toJson(csv);
console.log(JSON.stringify(json));

To read a CSV file in JavaScript and convert it to JSON:

var rawFile = new XMLHttpRequest();
rawFile.open("GET", "books.csv", true);
rawFile.onreadystatechange = function () {
  if (rawFile.readyState === 4) {
    if (rawFile.status === 200 || rawFile.status == 0) {
      var allText = rawFile.responseText;
      var result = toJson(allText);
      console.log(JSON.stringify(result));
    }
  }
}
rawFile.send(null);

You might also like:
Converting XML to CSV using XSLT 1.0