JSEN: JavaScript Expression Notation

That idea I was talking about yesterday? Storing mathematical expressions as JSON? I went ahead and made it as a TypeScript project and released it on GitHub:

JavaScript Expression Notation (JSEN)

Still need to complete the unit test coverage and add a couple more features. I made a change from my original post to the syntax for namespace references. (The reason? I realized I needed to be able to use "*" as a local identifier for multiplication.) They work within Namespace declaration blocks, but I need to make them work at the higher level of Namespaces declaration blocks as well. (Done.) I also want to allow functions to be used as namespaces. (Done.)

This is possible right now:

jsen.decl('my-fake-namespace', {
   'js': 'http://ecma-international.org/ecma-262/5.1',

   'x': 10,
   'y': ['js:Array', 1, 2, 3],
   'z': ['js:[]', 'y', 1]
});

jsen.eval('my-fake-namespace', 'x'); // 10
jsen.eval('my-fake-namespace', 'y'); // [1, 2, 3]
jsen.eval('my-fake-namespace', 'z'); // 2

jsen.expr('my-fake-namespace', 'x'); // 10 // Deprecated
jsen.expr('my-fake-namespace', 'y'); // Deprecated
    // ["http://ecma-international.org/ecma-262/5.1:Array", 1, 2, 3]
jsen.expr('my-fake-namespace', 'z'); // Deprecated
    // ["http://ecma-international.org/ecma-262/5.1:[]", "y", 1]

Eventually something like this will be possible as well:

Mathematical expressions as JSON (and phyloreferencing)

For Names on Nodes I did a lot of work with MathML (specifically MathML-Content), an application of XML for representing mathematical concepts. But now, as XML wanes and JSON waxes, I've started to look at ideas for porting Names on Nodes concepts over to JSON.

I've been drawing up a very basic and extensible way to interpret JSON mathematically. Each of the core JSON values translates like so:

• Null, Boolean, and Number values are interpreted as themselves.
• Strings are interpreted as qualified identifiers (if they include ":") or local identifiers (otherwise).
• Arrays are interpreted as the application of an operation, where the first element is a string identifying the operation and the remaining elements are arguments.
• Objects are interpreted either as:
• a set of declarations, where each key is a [local] identifier and each value is an evaluable JSON expression (see above), or
• a namespace, where each key is a URI and each value is a series of declarations (see previous).

Examples

Here's a simple object declaring some mathematical constants (approximately):

{
    "e": 2.718281828459045,
    "pi": 3.141592653589793
}

Supposing we had declared some operations (only possible in JavaScript, since JSON doesn't have functions) equivalent to those of MathML (whose namespace URI is "http://www.w3.org/1998/Math/MathML"), we could do this:

{
    "x":

        ["http://www.w3.org/1998/Math/MathML:plus",

            1,

            2

        ],
    "y":

        ["http://www.w3.org/1998/Math/MathML:sin",

            ["http://www.w3.org/1998/Math/MathML:divide",

                "http://www.w3.org/1998/Math/MathML:pi",

                2

            ]

        ]
}

Once evaluated, x would be 3 and y would be 1 (or close to it, given that this is floating-point math).

Now for the interesting stuff. Suppose we had declared Names on Nodes operations and some taxa using LSIDs:

{
    "Homo sapiens": "urn:lsid:ubio.org:namebank:109086",
    "Ornithorhynchus anatinus": "urn:lsid:ubio.org:namebank:7094675",
    "Mammalia":

        ["http://namesonnodes.org/ns/math/2013:clade",

            ["http://www.w3.org/1998/Math/MathML:union",

                "Homo sapiens",

                "Ornithorhynchus anatinus"

            ]

        ]

}

Voilá, a phylogenetic definition of Mammalia in JSON!

I think this could be pretty useful. My one issue is the repetition of  long URIs. It would be nice to have a mechanism to import them using shorter handles. Maybe something like this?

{
    "mathml":   "http://www.w3.org/1998/Math/MathML:*",
    "namebank": "urn:lsid:ubio.org:namebank:*",
    "NoN":      "http://namesonnodes.org/ns/math/2013:*",

    "Mammalia":

        ["NoN:clade",

            ["mathml:union",

                "namebank:109086",

                "namebank:7094675"

            ]

        ]
}

Something to ponder. Another thing to ponder: what should I call this? MathON? MaSON?

Using TypeScript to Define JSON Data

JSON has gradually been wearing away at XML's position as the primary format for data communication on the Web. In some ways, that's a good thing: JSON is much more compact and readable. In other ways, it's not so great: JSON lacks some of XML's features.

One of these features is document type definitions. For XML, there are a variety of formats (DTD, XML Schema, RELAX NG, etc.) for specifying exactly what your XML data looks like: what are the tag names, possible attributes, etc. JSON is a lot more loosey-goosey here.

Okay, that's not entirely true: there is JSON Schema. I've never known anyone to use it, but it's there. It's awfully verbose, though. (So are the definitional formats for XML, but it's XML — you expect it!)

I was thinking about this the other day, and I realized that there is actually a great definitional format for JSON already in existence: TypeScript! If you haven't heard of it, TypeScript is a superset of JavaScript which introduces optional strict typing. And since JSON is a subset of JavaScript, TypeScript is applicable to JSON as well.

One of the great features of TypeScript is that interface implementation is implicit. In Java or ActionScript, you have to specifically say that a type "implements MyInterface". In TypeScript, if it fits, it fits. For example:

interface List

{

    length: number;

}

function isEmpty(list: List): bool

{

    return list.length === 0;

}

console.log(isEmpty("")); // true

console.log(isEmpty("foo")); // false

console.log(isEmpty({ length: 0 })); // true
console.log(isEmpty({ length: 3 })); // false
console.log(isEmpty({ size: 1})); // Compiler error!

(Note: for some reason that I can't fathom, isEmpty() doesn't work on arrays. Well, TypeScript is still in development — version 0.8.2 right now. Update: I filed this as a bug.)

Note that you can use interfaces even on plain objects. So of course you can use it to describe a JSON format. Here's an example from a project I hope to release before too long:

interface Model

{

    uid: string;

}

interface Name extends Model

{

    citationStart?: number;

    html?: string;

    namebankID?: string;

    root?: bool;

    string?: string;

    type?: string;

    uri?: string;

    votes?: number;

}

interface Taxon

{

    canonicalName?: Name;

    illustrated?: bool;

    names?: Name[];

}

Now, for example, I can declare that an API search method will return data as an array of Taxon objects (Taxon[]). And look how compact and readable it is!

Note that there is one drawback here: there is no way to enforce this at run-time. JSON Schema might be a better choice if that's what you need. But for compile-time checking and documentation, it's a pretty great tool.

Saving Bootstrap Settings

The popular web page framework Bootstrap recently added a web form whereby you can customize visual settings (color scheme, fonts, etc.). Unfortunately they didn't add a way to save those settings, so if you later decide you need to tweak them and you didn't happen to just leave that web page open, you're screwed. You either have to reinvent them, go from memory, or dig through the generated files and hope you didn't miss anything.

I'm sure they plan to address this eventually, but in the meantime I created some JavaScript code to work around this: https://gist.github.com/4628506

To use this code:

1. Go to: http://twitter.github.com/bootstrap/customize.html
2. Run the script in the JavaScript console. (If you don't use a browser with a JavaScript console, you're beyond my help.)
3. Fill out the customization form.
4. You can record your settings into an object by running: var settings = record()
5. You can grab those as JSON by running: JSON.stringify(settings)
6. You can reinstate those settings later by running: play(settings)
7. You can save your settings to local storage by running: save()
8. You can retrieve your settings from local storage by running: retrieve()

I haven't fully tested this, so let me know if you run into any issues.

SIDE NOTE: This is my first gist!

Introducing Pictish, an image-processing library for web browsers

Boy, between RaphaëlTS, SHA-1, and Haeckel, I've been releasing an awful lot of TypeScript/JavaScript libraries lately, haven't I? Anyway, here's another!

Pictish

A Library for Processing Binary Image Data

Pictish takes advantage of canvas elements and Typed Arrays to provide fast routines for processing raster image data. Here is a rundown of the currently available functions:

• createImageData()
• fromFile()
• fromHTMLImage()
• crop()
• flipX()
• flipY()
• scaleDown()
• quarter()
• silhouettize()

All of these functions have been tested and optimized. More information is available in the documentation at the BitBucket site. Note that since canvas elements and Typed Arrays are based on more recent specifications, not all browsers support it. (For what it's worth, I've been testing in Chrome.)

(The sharp-eyed may notice that last function and wonder if it might have something to do with another project of mine. The answer is yes.)

Next planned step is to create a PNG file encoder — that could take a while, though.

SHA-1 with Typed Arrays

There are already SHA-1 implementations for JavaScript, but all the ones I found use strings as input. This is fine for generating hashes from small amounts of data, but not so great for large binary files. So I've created a SHA-1 library that optionally accepts ArrayBuffer objects.

SHA-1 TypeScript/JavaScript Library

Note that this will not work on browsers that do not support Typed Arrays.

I ran some tests and found that in Chrome it takes less than half a second to hash 10Mb of data.

Hope someone finds this useful! (At the very least, I will.)

Haeckel: A Code Library for Browser-Based Evolutionary Diagrams

For a while now I've been writing posts with diagrams like this one, showing the evolution of cranial capacity in mangani over the past seven million years:

How did I make them? Originally it was all ad-hoc ActionScript code, but more recently I've begun to organize the code into a library and translate it into TypeScript (which, in turn, is automatically translated into JavaScript). Although this library is still in progress, I've decided it's at a stage where I can open it to the general public.

This library includes functionality for:

• Modeling scientific concepts such as taxa, phylogeny, character states, stratigraphy, and geography.
• Processing scientific data (notably calculating morphological distance and inferring unknown character states).
• Rendering data into charts as Scalable Vector Graphics, using RaphaëlJS.

For a while I struggled with what to call this library. It's neither purely about science nor purely about graphics. Finally I got my inspiration from RaphaëlJS, a graphics library named after a great artist. I named my library after a man who was both a great artist and a great biologist:

RaphaëlJS + TypeScript = RaphaëlTS

RaphaëlJS is a library, written in JavaScript, for creating vector graphics on the web. It's commonly used because it supports all web browsers, including the good ones (which use SVG) and Internet Explorer (which uses VML). Among other things, it's the rendering engine of the ExtJS and Sencha frameworks.

TypeScript is a new web language that extends JavaScript, adding optional strict typing. This is a big improvement on JavaScript because it allows more errors to be caught at compile-time rather than at run-time (for example, if you pass a number to a function that expects an array). However it's not as obnoxious as some strictly-typed languages, because the typing is optional and often implicit.

I've spent some time recently playing around with TypeScript and my verdict: it's wonderful. I only want to use this from now on (at least for large projects). It's a great language, and, since it's an extension (or superset) of JavaScript, all existing JavaScript code works just fine in it. (This is a big advantage it has over Dart, an otherwise excellent language as well.)

But while you can just include a JavaScript library in your TypeScript projects, you don't get the full benefit of the compile-time checking (or the code-hinting) unless that library's entities have been declared. TypeScript allows for ambient declarations: defining or partially defining an entity (class, interface, method, variable, etc.) without actually creating it. As an example (and a very useful one at that), here is a TypeScript declaration file for jQuery: jQuery.d.ts

Today I wondered if anyone had made something like this for RaphaëlJS. I wasn't able to find one, so ... I made one. Here you go: https://bitbucket.org/keesey/raphaelts/src

If you're using the Visual Studio plugin or the TypeScript playground (other editors are bound to come out soon), you can now program type-safe RaphaëlJS code with auto-complete (=IntelliSense).

I've verified that it compiles, but I haven't fully tested it—if anyone experiences issues please let me know. (Or feel free to fork the project).

Hope someone else finds it useful ... I know I will!

How similar are we, anatomically, to other primates?

There is no objective way to measure anatomical similarity, but you can get a sense by converting character matrices into distance matrices. I've done this for the matrix used by Diogo & Wood (2011), which looked at soft tissue anatomy. Here is a bar chart showing how similar each taxon is to humans:

Dangit, 2011, not 2010. I'll fix it later.
Click for full size.

As you can see, the distances for great apes are well-marked and exactly what you'd expect based on phylogeny, but past that it gets a bit fuzzy. Moving outward from the great apes we get to Old World monkeys, then gibbons (from phylogeny you'd expect gibbons first, but the difference is so minor I'm sure it's meaningless), then a mixture of non-catarrhine primates, and finally non-primates.

This figure was generated using a JavaScript library I'm developing. I'll say more later, but rest assured it will be free and open source.

Expect to see some more stuff like this on A Three-Pound Monkey Brain soon.

References

• Diogo & Wood (2011). Soft-tissue anatomy of the primates: phylogenetic analyses based on the muscles of the head, neck, pectoral region and upper limb, with notes on the evolution of these muscles. J. Anat. 219:273–359. doi:10.1111/j.1469-7580.2011.01403.x

To Flash or not to Flash

I love building stuff with Flash technologies. I think ActionScript 3.0 is an excellent language and Flex 4 is a very good framework. I'm not particularly enamored of the alternatives. I find JavaScript to be a mediocre language (albeit with some excellent libraries). I don't like wrangling CSS more than I have to (although Less makes it much nicer). I find HTML 5 to be a pretty immature technology so far. SilverLight's days are probably numbered. (I haven't yet delved into mobile operating systems, like Android and iOS, so I can't speak for those. And those aren't complete alternatives, anyway.)


Because I love Flash, I have a tendency to want to build everything in it. But over the years I've learned that this tendency must be curbed whenever possible. There are definite downsides to Flash, and especially to doing entire websites in Flash. (The now-defunct March of Man website and some abandoned versions of the Dinosauricon are testaments to this.)


For PhyloPic, it was pretty clear to me that there would be no significant advantage to building it in Flash. Load times would be increased without any functionality enhancement. I wouldn't be able to use it on my iPhone. And all the functionality I needed was easily available in plain old HTML/JavaScript/CSS.


With one exception.


The Submission Tool is built as a Flex app. There is one primary reason for this: image processing. Processing the silhouettes on the server side was an option, but one that could have potentially bogged the server down. (It's already starting to buckle a bit as is, pending some optimizations.) But, by using Flash's BitmapData class, I can do that bit of work on the client side before the silhouettes are shipped off to the database.


Of course there are some other benefits as well. In descending order of importance:

• Flash allows for a more unified experience for the submitter. No page reloads and no cross-browser differences.
• SPAM bots are much more capable of cracking HTML forms than cracking custom AMF web services. SWF files are generally opaque to them.
• It was easier for me to build and test.

Had it just been those three reasons, there might still have been a good argument to do it as HTML/AJAX. The image processing requirement is what really tipped the scales. One hundred submitters contrasting, cropping, and rescaling bitmaps is much nicer if they're doing it on their own machines than if they're all doing it on the server. (Okay, I've barely had even two simultaneous submitters so far, but I can dream....)

There may be other Flash tools in PhyloPic's future. For example, I think it is the best technology for the Cladogram Generator. But for the rest of the site, plain old HTML/JavaScript/CSS is certainly sufficient—better, even.

UPDATE (2012 Jan 26): The Submission Page still uses Flash but is not a Flex app.

PhyloPainter: Happy Little Trees

The whole Flash/Apple fracas has been rather distasteful to me. But I'm not going to dwell on that right now. Instead, I am trying to keep an open mind by trying out some of the technologies that are competing with my favored development tools. First up: HTML 5.

I'll probably write more on the topic later, but suffice to say for now that working HTML 5 feels like I've traveled in time back to 2001, the days of ActionScript 1.0. JavaScript is a poor language for anything complicated. Canvas has covered the basics of vector drawing well, but little else. That said, I see potential and I'm pretty certain the tools will improve.

For my first HTML 5 app, I ported some basic functionality from Names on Nodes, namely, the ability to read Newick tree strings and the ability to draw graphs. I give you:

PhyloPainter

It's a bit rough right now. For one thing, it doesn't work in Internet Explorer (despite the inclusion of a workaround JavaScript tool—the current version of IE doesn't support HTML 5 Canvas). But it's a start.

Give it a try—paint some happy little trees!