index.ndg

Unveil.js
================================================================================

Unveil is a data exploration and visualization toolkit that utilizes data-driven
software design.


Note:

This documentation is generated on the fly using the "Ndogen"[http://github.com/oliver----/ndogen] documentation generator.
Since this instant doc-generation part is under heavy development, you'll
experience some glitches. We're working on it! Please have a look at the "raw markup"[http://github.com/michael/unveil/blob/master/index.ndg] too.


Collection API
================================================================================

Collections are the main building block of the toolkit. A Collection is a simple
data abstraction format where a data-set under investigation conforms to a 
collection of data items that describes all facets of the underlying data in a 
simple and universal way. You can think of a `Collection` as a table of data, 
except it provides precise information about the data contained (meta-data). 

Collections are represented as a JSON string, which serves as an exchange format.
Since JSON is heavily used in web-services, its perfectly suited for usage in a
cloud-like scenario. Your collections can be constructed on the fly using live
data from web services that either provides data directly in the Collection format
or through a translator service that serves as an adapter for an existing service.

After passing the raw format to the `Collection` constructor, they are
internally represented as a network of nodes (a graph so to say), that allows
efficient operations on the Collection by traversing the graph instead of 
iterating over all items, values etc. Also operations like filtering, grouping
etc. are already supported.

Construction
--------------------------------------------------------------------------------

$ var countries = new Collection({
$   "properties": "properties": {
$     "name": {
$       "name": "Country Name",
$       "type": "string",
$       "unique": true
$     },
$     "official_language": {
$       "name": "Official language",
$       "type": "string",
$       "unique": true
$     },
$     "form_of_government": {
$       "name": "Form of governmennt",
$       "type": "string",
$       "unique": false
$     },
$     "currency_used": {
$       "name": "Currency used",
$       "type": "string",
$       "unique": true
$     },
$     "population": {
$       "name": "Population",
$       "type": "number",
$       "unique": true
$     },
$     "gdp_nominal": {
$       "name": "GDP nominal",
$       "type": "number",
$       "unique": true
$     },
$     "area": {
$       "name": "Area",
$       "type": "number",
$       "unique": true
$     },
$     "date_founded": {
$       "name": "Date founded",
$       "type": "date",
$       "unqiue": true
$     }
$   },
$   "items": {
$     "arg": {
$       "name": "Argentina",
$       "official_language": "Spanish Language",
$       "form_of_government": [
$         "Federal republic",
$         "Presidential system"
$       ],
$       "currency_used": "Argentinian Peso",
$       "population": 39745613,
$       "gdp_nominal": 338700000000.0,
$       "area": 2780403.0,
$       "date_founded": "1816-07-09"
$     },
$     ...
$   },
$ });

The Collection constructor takes a collection represented as a JavaScript object,
which conforms to a parsed JSON collection string. Once created you can query
the information contained.

The API features a simple object model for working with such in-memory
collections. The collection object gives you access to the `Properties` and 
`Items` contained. 

Please make yourself familiar with the Node API, which is the data
structure used for modeling the internal structure of a collection, which is a
graph. Be aware that `Collection`, `Property`, `Item` and `Value` objects inherit
from Node, so the full Node API is available on such objects.


Properties
--------------------------------------------------------------------------------

Meta-data (data about data) is represented as a set of properties that belongs
to a collection. A property (cmp. a column in a table) holds a key, a name 
(cmp. a header of a column) and a type (telling whether the data is numeric or
textual, etc.).

Access the Collection's properties:

$ var properties = countries.all('properties'); // => SortedHash

You can also access them directly:

$ var population = countries.get('properties', 'population');

And you can use property inspection:

$ population.type; // => 'string'
$ population.name; // => 'Population'
$ population.unique; // => true

Items
--------------------------------------------------------------------------------

An item of the collection conforms to a row in a data table, except one ‘cell’
can have arbitrary many values (non-unique attributes).

List all items:

$ var items = countries.all('items'); // => SortedHash containing Item objects

Or access them directly:

$ var argentina = countries.get('items', 'arg'); -> Item#arg

Inspect item values:

$ argentina.value('name'); // => 'Argentina'
$ argentina.value('currency_used'); // => 'Argentinian Peso'

Non-unique properties have more that one value:

$ argentina.values('form_of_government');
$ // => [String#Federal_Republic, String#Presidential_system]

You can easily iterate over values using `SortedHash#each`:

$ argentina.values('form_of_government').each(function(index, gf) {
$   gf; // => e.g. 'Presidential System'
$ });

Values
--------------------------------------------------------------------------------

You can not only view values on a raw level but also as Nodes, which provide
useful meta-information about connections etc.

List all items:

$ var items = countries.all('items'); // => SortedHash containing Item objects

inspect the value 'Federal republic':

$ federal_republic = argentina.all('form_of_government').first();
$ // => Node#Federal_Republic

Which other countries also have this government form?

$ federal_republic.all('items'); // => [Node#austria, Node#usa]


Set operations
--------------------------------------------------------------------------------

Since OrderedHashes conform to a set in mathematical respect, some interesting
operations can be performed.

$ var english = countries.all('item').select(function(key, i) {
$   return i.value('official_language') === 'English'; 
$ });
$   
$ var french = countries.all('item').select(function(key, i) {
$   return i.value('official_language') === 'French'; 
$ });
$ 
$ // english and french speaking countries
$ var english_and_french = english.intersect(french); // => [Node#canada]
$ 
$ // english or french speaking countries
$ var english_or_french = english.union(french);
$ // => [Node#uk, Node#france, Node#Switzerland, ...]


Transformers                                                     [$TRANSFORMERS]
--------------------------------------------------------------------------------

Transformers are used to do operations on the desired collection. A commonly
used one is the group transformation.

The following code does a grouping by the `official_language` property and sums
up all numeric properties (like population, area). What you get is a modified
collection that shows aggregated values for all occurring languages.

$ var languages = countries.transform('group', {
$   property: 'official_language',
$   aggregator: 'SUM'
$ });

The resulting collection looks like this

$ {
$   "properties": "properties": {
$     "official_language": {
$       "name": "Official language",
$       "type": "string",
$       "unique": true
$     },
$     "population": {
$       "name": "Population",
$       "type": "number",
$       "unique": true
$     },
$     "gdp_nominal": {
$       "name": "GDP nominal",
$       "type": "number",
$       "unique": true
$     },
$     "area": {
$       "name": "Area",
$       "type": "number",
$       "unique": true
$     }
$   },
$   "items": {
$     "English": {
$       "population": 397445613,
$       "gdp_nominal": 4341700000000.0,
$       "area": 2780403.0,
$     },
$     "German": {
$       "population": 91560000,
$       "gdp_nominal": 4341700000000.0,
$       "area": 2780403.0,
$     },
$     ...
$   }
$ };


Transformers are non-destructive. The original collection is not modified unless
you re-assign the result.

Implement your own Transformer
--------------------------------------------------------------------------------

All you have to do is to assign a new property on the pv.Collection.transformers
object, which holds a transformer function. This function takes an input
collection and a params hash.

$ pv.Collection.transformers.group = function(c, params) {
$   // hardcore grouping action that yields a modified output collection
$   return outputColletion;
$ };

Have a look at the implementation of the "Group Transformer"[http://github.com/michael/unveil/blob/master/src/collection/transformers/group.js], to see how it's done.

Transformer specification:

You need to define a specification for your transformation, which gives it a
human readable name and describes the parameters your transformation takes.

$ pv.Collection.transformers.group.label = "Group By";
$ pv.Collection.transformers.group.params = {
$   properties: {
$     name: "Property",
$     type: "property_list"
$   },
$   aggregator: {
$     name: "Aggregator Function",
$     type: "aggregator"
$   }
$ }

Available Collections
--------------------------------------------------------------------------------

Some sample collections are available through Collectionize, a dedicated
aggregator service, that translates interesting web services to the collection
format.

Currently available:

* "Countries fetched from Freebase.com"[http://collections.quasipartikel.at/countries]
* "Last.fm Playlists"[http://collections.quasipartikel.at/playlists]


Visualization API                                                         [$VIS]
================================================================================

The visualization API provides a simple abstraction layer for visualizations to ease
the process of creating re-usable data-driven visualizations. 

The appearance of the result is determined by the underlying data rather than by user
defined plotting options. Visualizations directly access data trough a well defined
interface, the Collection API, so there's no longer a gap between domain data and
data used by the visualization engine.

Such visualization can be re-used in terms of putting in arbitrary data in. 
This works as long as the data is a valid Collection and satisfies the
visualization specification (some visualizations exclusively use numbers as their
input, others use dates (e.g. Timeline plots), and so on...).


Usage
--------------------------------------------------------------------------------

$ // load some data (represented as a Collection)
$ c = new uv.Collection(countries);
$ 
$ // construct a visualization based on that data
$ vis = new uv.Linechart(c, {
$   measures: ['population'],
$   params: {scale: 'linear'}
$ });
$ vis.render();


Creating visualizations
--------------------------------------------------------------------------------

In order to create your own visualizations you need to setup a new class that
inherits from uv.Visualization.

$ var MyChart = uv.Visualization.extend({
$   constructor: function(collection, options) {
$     uv.Visualization.call(this, collection, options);
$   },
$   render: function() {
$     // hardcore rendering action
$   }
$ });

All you have to do is to imlement `render()`. It's totally up to you whether you
want to use a visualization library (I can recommend "Protovis"[http://protovis.org] and "Processing.js"[http://processingjs.org]) or
work with the native API's (Canvas, SVG).


Visualization Specification:

There are various types of visualizations. Some exclusively use numbers as their
input, some use dates (e.g. Timeline plots) others visualize relationships between
data items (e.g. which countries share the same currency). There are further
visualizations that use various combinations of measure types.

In order to verify that the current selection of measures can be displayed by the
chosen visualization you have to define a specification like this:

$ // Displays 1..n number properties
$ uv.Barchart.spec = {
$   measures: [
$     {
$       types: ['number'],
$       unique: true,
$       cardinality: 1      
$     },
$     {
$       types: ['number'],
$       unique: true,
$       cardinality: "*"
$     }
$   ]
$ };

For reference have a look at the implementations of `Scatterplot` and `Linechart`.

  * "Scatterplot"[http://github.com/michael/unveil/tree/master/src/visualization/visualizations/scatterplot.js]
  * "Linechart"[http://github.com/michael/unveil/tree/master/src/visualization/visualizations/linechart.js]


Examples
--------------------------------------------------------------------------------

  * Stacks[http://quasipartikel.at/unveil/examples/stacks.html]
  * Random Bars[http://quasipartikel.at/unveil/examples/random_bars.html]
  * Scatterplot[http://quasipartikel.at/unveil/examples/scatterplot.html]
  * Linechart[http://quasipartikel.at/unveil/examples/linechart.html]

Scene API                                                               [$SCENE]
================================================================================

Inspired by various great visualization libraries (Protovis, Processing.js) I
created my own rather low level visualization framework that is basically an
implementation of a SceneGraph on top of the HTML5 Canvas API. 

I focussed on two core principles:

  * Object oriented
  * Declarative

Object oriented in terms of thinking in graphical objects and modularizing 
code, declarative in terms of using, configuring and combining existing actors
(graphical objects) and attach them to the scene graph.


Creating a scene is easy:

$ var scene = new uv.Scene({
$       width: 800,
$       height: 400
$     });


Actors
--------------------------------------------------------------------------------

Once you have constructed your scene object you can start adding actors to the scene.
Actors can be primitive graphical objects (like a bar, a line, or a dot) or higher 
level objects that combine lower level ones (e.g. a Hyperbolic Tree). Actors
typically take properties in their constructor. However for higher level objects
you probably want them to be constructed with real domain data instead of simple
graphical oriented properties (width, height etc.). You can decide on your own
how you want to organize them. 


Add an Actor (Bar) to the scene:

$ var bar = new uv.Bar({
$   x: 30,
$   y: 50,
$   width: 30,
$   height: 80,
$   fillStyle: 'darkblue'
$ });

Add an additional bar as a child of the bar just created:

$ bar.add(new uv.Bar({
$   x: 50,
$   y: 20,
$   width: 20,
$   height: 80,
$   fillStyle: 'lightblue'
$ }));

The x and y coordinates are relative to the parent object. So an object does not
know where it is located in the coordinate-space. It just renders itself at 
position 0,0. The positioning is done through matrix transformations, where for
each object the current context (transformation matrix) is stored.
That's how a SceneGraph works.

Start the scene:

$ // defaults to 60 frames per second
$ scene.start();

Creating visualizations using the Scene API most often means implementing your own
Actors. Let's re-create the Bar Actor from scratch, and add some interaction and
animation later:

$ uv.Bar = function(properties) {
$   // super call
$   uv.Actor.call(this);
$   
$   // define your default properties
$   _.extend(this.properties, {
$     width: 40,
$     strokeWeight: 2,
$     strokeStyle: '#ccc'
$   }, properties);
$ };

Every Actor inherits from uv.Actor:

$ uv.Bar.prototype = Object.extend(uv.Actor);


Implement a draw method, which takes a Canvas 2D context:

$ uv.Bar.prototype.draw = function(ctx) {
$   ctx.fillRect(0, 0, this.properties.width, this.properties.height);
$ };


Interaction
--------------------------------------------------------------------------------

Interaction is key in visualizations. Therefore Unveil.js aims to provide an
abstraction for implementing interaction on your Actors. Unlike in SVG, with
the Canvas API you can't attach event handlers on shapes directly. Instead
you need to detect on your own, which objects are currently under the cursor.
Usually you'd do this with some math, but there's a simpler approach that
utilizes `isPointInPath()`:

With `isPointInPath()` you can check if the current mouse-position is inside
the current working path.

To make use of this you need to equip your Actors with an additional
`drawMask()` method. 

Let's add some interaction to our Bar:

$ uv.Bar.prototype.drawMask = function(ctx) {
$   ctx.beginPath();
$   
$   ctx.moveTo(0, 0);
$   ctx.lineTo(this.properties.width, 0);
$   ctx.lineTo(this.properties.width, this.properties.height);
$   ctx.lineTo(0, this.properties.height);
$   ctx.lineTo(0, 0);
$ };

This simply draws an invisible rectangle. If you have a more complex object like
a Star-Shape you can use a rectangle (also known as a bounding-box) as well, to
mask your object for interaction. That's for the lazy folks. ;-)

Actors that have a `drawMask()` implementation can then be easily checked
against the current cursor position.

That's what is going on behind the curtain:

$ uv.Actor.prototype.checkActive = function(ctx, mouseX, mouseY) {
$   if (this.drawMask && ctx.isPointInPath) {
$     this.drawMask(ctx);
$     if (ctx.isPointInPath(mouseX, mouseY))
$       this.active = true;
$     else
$       this.active = false;
$   }
$ };



Animation
--------------------------------------------------------------------------------

It's easy to add animation to your objects too. I've added a "Tween class"[http://quasipartikel.at/tween/]
I created some time ago to animate properties over time.

To animate the height property of a Bar you simply use a Tween for that:

$ uv.Bar = function(properties) {
$   ...
$   this.t = new uv.Tween(this.properties, "height", 
$     uv.Tween.strongEaseInOut, this.properties.height, 
$     this.properties.height, 2);
$ };

The `updateHeight()` method is used to trigger the Tween:

$ uv.Bar.prototype.updateHeight = function(newHeight) {
$   this.t.continueTo(newHeight, 1.5);
$ };


For now you need to manually trigger a Tween tick using the Actor#update() method:

$ uv.Bar.prototype.update = function() {
$   this.t.tick();
$ };

In your sketch you can use setTimeout() or setInterval() to trigger the 
`updateHeight()` method.

$ setInterval(function() {
$   bar.updateHeight(70);
$ }, 1000);

Here's an example of "Moving Random Bars"[http://quasipartikel.at/unveil/examples/random_bars.html],
that feature interaction and animation.


Hints
--------------------------------------------------------------------------------

The Scene API related code is pretty small (~ 200 LOC). I don't plan to provide
a full-featured Visualization library with helper functions, since there are
existing libraries, that do a perfect job. I recommend using Protovis along with
the Scene API. Actually, this library was created because Protovis does not
support the canvas element. I also wanted an easier way to deal with interaction.

To prevent from verbose and messy code, I do not provide any backward
compatibility for older browsers. I'm trying to stay on the edge of technology.
Therefore I focus on supporting the most recent versions of Google Chrome,
Firefox, Safari. Currently mouse interaction is not working in Firefox. Please
use a webkit-based browser for the moment.


Node API                                                                 [$NODE]
================================================================================

Node (not to be confused with Node.js) is a JavaScript Graph implementation that
hides graph complexity from the interface. It introduces properties, which group
types of edges together. Therefore multi-partit graphs are possible without any
hassle. Every Node simply contains properties which conform to outgoing edges.
It makes heavy use of hashing through JavaScript object properties to allow
random access whenever possible.

The Node API heavily relies on SortedHash, please have a look at
the documentation before you start.

Construction
--------------------------------------------------------------------------------

$ // Initialize a plain Node
$  var austria = new Node(),
$      germany = new Node(),
$      uk = new Node();
$  
$ // initialize with raw Value (raw values are typically stored in leave nodes)
$ var eu = new Node({value: 'European Union'}),
$     austrian = new Node({value: 'Austrian'}),
$     english = new Node({value: 'English'}),
$     german = new Node({value: 'German'}),
$     barroso = new Node({value: 'Barroso'});


Connect nodes through properties
--------------------------------------------------------------------------------

$ austria.set('languages', 'at', austrian);
$ austria.set('languages', 'ger', german); 
$ 
$ eu.set('president', 'barroso', barroso);
$ 
$ // Backlinks
$ german.set('spoken_in', {
$   'at': austria,
$   'de': germany
$ });

Get connected nodes
--------------------------------------------------------------------------------

$ austria.all('languages') // => [Node#austrian, Node#german]
$ eu.first('president') // => [Node#barroso]
$ german.all('spoken_in') // => [Node#austria, Node#germany]

Iteration
--------------------------------------------------------------------------------

each:

$ austria.all('languages').each(function(index, node) {
$   node; // => Node#at | Node#ger
$   index; // => 0 | 1
$ });

eachKey:

$ austria.all('languages').eachKey(function(key, node) {
$   node; // => Node#at | Node#ger
$   key; // => 'at' | 'ger'
$ });

Sorted Hash API                                                    [$SORTEDHASH]
================================================================================

Unveil.js features a `SortedHash` data structure that provides a simple layer of
abstraction for managing Sorted Hashes in JavaScript. It's heavily used
throughout the framework.

Insertion
--------------------------------------------------------------------------------

$ var items = new SortedHash();
$ items.set('at', 'Austria');
$ items.set('de', 'Germany');

Access
--------------------------------------------------------------------------------

Hash Semantics:

$ items.get('at') // => 'Austria';
$ items.get('de') // => 'Germany';

Array Semantics:

$ items.at(0); // => 'Austria'
$ items.at(1); // => 'Germany'
$ items.length; // => 2

Iteration
--------------------------------------------------------------------------------

each:

$ items.each(function(index, value) {
$   value; // => 'Austria', 'Germany'
$   index; // => 0, 1
$ });

eachKey:

$ items.eachKey(function(key, value) {
$   value; // => 'Austria', 'Germany'
$   key; // => 'at', 'de'
$ })

Operations
--------------------------------------------------------------------------------

sort:

$ var desc = function(item1, item2) {
$       return item1.value === item2.value
$         ? 0 : (item1.value > item2.value ? -1 : 1);
$     },
$     sortedItems;
$ 
$ sortedItems = items.sort(descendingOrder);


map:

$ var mappedItems = items.map(function (item) {
$   return item.slice(0, 3);
$ });
$ 
$ // leave original SortedHash untouched
$ mappedItems.at(0) // => 'Aus';
$ mappedItems.at(1) // => 'Ger';

This returns a SortedHash in descending order, while the original remains
untouched.


select:

$ var selectedItems = items.select(function (key, item) {
$       return item === 'Austria';
$     });
$ 
$ selectedItems.at(0) // => 'Austria';
$ selectedItems.length; // => 1;


intersect:

$ var items2 = new SortedHash(),
$     intersectedItems;
$ 
$ items2.set('fr', 'France');
$ items2.set('at', 'Austria');
$ 
$ intersectedItems = items.intersect(items2);
$ 
$ intersectedItems.length; // => 1
$ intersectedItems.get('at'); // => 'Austria'


union:

$ var items2 = new SortedHash(),
$     unitedItems;
$ 
$ items2.set('fr', 'France');
$ items2.set('at', 'Austria');
$ 
$ intersectedItems = items.intersect(items2);
$ 
$ unitedItems.length; // => 3
$ unitedItems.get('at'); // => 'Austria'
$ unitedItems.get('de'); // => 'Germany'
$ unitedItems.get('fr'); // => 'France'

Method chaining:

$ var mappedAndSortedItems = items.map(function (item) {
$       return item.slice(0, 3);
$     })
$     .sort(descendingOrder);
$ 
$ selectedItems.at(0) // => 'Ger';
$ selectedItems.at(2) // => 'Aus';