Asynchronous RSA key generation in Javascript

Source on GitHub – https://github.com/KevBurnsJr/rsasync

Generating RSA keys is a characteristically CPU intensive operation. This presents problems when operating on weak devices (such as the iPhone 3G) in environments under strict computation restrictions (such as safari mobile’s 10 second javascript execution timeout).

What is needed is an RSA key generation library that operates asynchronously in order to chug through the 2+ minutes of computation time required to generate a 512 RSA key on a weak device without bumping against the computation restrictions enforced by the safari mobile execution environment.

Some nearly suitable libraries do exist, but all of them fall short in some fashion.

  • Probably the closest is this asynchronous keygen from Atsushi Oka
    http://ats.oka.nu/titaniumcore/js/crypto/readme.txt
    However, the interface for Ats Oka’s library is not simple and the architecture of the code leaves something to be desired.

  • Cryptico is another library featuring RSA key generation which is also touted as a sort of all-in-one solution
    http://code.google.com/p/cryptico/
    However, this library really just glues together a bunch of already-available libraries and packages them as a unit.

  • jsbn is the underlying RSA key generator packaged with Cryptico and is available on its own
    http://www-cs-students.stanford.edu/~tjw/jsbn/
    This library has a fairly simple interface and is relatively fast and compact meeting most of my requirements.
    However, this library doesn’t do asynchronous key generation.

    But we can fix that.

jsbn RSA keygen times out after 11 seconds on the iPhone 3G for even a 256 bit key but with a little fenangling and a lot of setTimeouts, we can get it to handle a key of virtually any size for which the user has the patience to wait.

Here’s an example of the new async interface:

key = new RSAKey();
key.generateAsync(512, "03", function(){
    var pubKey = hex2b64(key.n.toString(16));
    alert(pubKey);
});

This was a great exercise in how to turn synchronous javascript into asynchronous javascript. Taking procedural code and breaking those for loops into recursive functions was a mind bender but once I figured out how it generally ought to work, each function became easier to port.

Originally I did it all inline, but later I ripped it all out into a separate file which extends Tom Wu’s jsbn.

// Copyright (c) 2011  Kevin M Burns Jr.
// All Rights Reserved.
// See "LICENSE" for details.
//
// Extension to jsbn which adds facilities for asynchronous RSA key generation
// Primarily created to avoid execution timeout on mobile devices
//
// http://www-cs-students.stanford.edu/~tjw/jsbn/
//
// ---
 
(function(){
 
// Generate a new random private key B bits long, using public expt E
var RSAGenerateAsync = function (B, E, callback) {
    //var rng = new SeededRandom();
    var rng = new SecureRandom();
    var qs = B >> 1;
    this.e = parseInt(E, 16);
    var ee = new BigInteger(E, 16);
    var rsa = this;
    // These functions have non-descript names because they were originally for(;;) loops.
    // I don't know about cryptography to give them better names than loop1-4.
    var loop1 = function() {
        var loop4 = function() {
            if (rsa.p.compareTo(rsa.q) <= 0) {
                var t = rsa.p;
                rsa.p = rsa.q;
                rsa.q = t;
            }
            var p1 = rsa.p.subtract(BigInteger.ONE);
            var q1 = rsa.q.subtract(BigInteger.ONE);
            var phi = p1.multiply(q1);
            if (phi.gcd(ee).compareTo(BigInteger.ONE) == 0) {
                rsa.n = rsa.p.multiply(rsa.q);
                rsa.d = ee.modInverse(phi);
                rsa.dmp1 = rsa.d.mod(p1);
                rsa.dmq1 = rsa.d.mod(q1);
                rsa.coeff = rsa.q.modInverse(rsa.p);
                setTimeout(function(){callback()},0); // escape
            } else {
                setTimeout(loop1,0);
            }
        };
        var loop3 = function() {
            rsa.q = nbi();
            rsa.q.fromNumberAsync(qs, 1, rng, function(){
                rsa.q.subtract(BigInteger.ONE).gcda(ee, function(r){
                    if (r.compareTo(BigInteger.ONE) == 0 && rsa.q.isProbablePrime(10)) {
                        setTimeout(loop4,0);
                    } else {
                        setTimeout(loop3,0);
                    }
                });
            });
        };
        var loop2 = function() {
            rsa.p = nbi();
            rsa.p.fromNumberAsync(B - qs, 1, rng, function(){
                rsa.p.subtract(BigInteger.ONE).gcda(ee, function(r){
                    if (r.compareTo(BigInteger.ONE) == 0 && rsa.p.isProbablePrime(10)) {
                        setTimeout(loop3,0);
                    } else {
                        setTimeout(loop2,0);
                    }
                });
            });
        };
        setTimeout(loop2,0);
    };
    setTimeout(loop1,0);
};
RSAKey.prototype.generateAsync = RSAGenerateAsync;
 
// Public API method
var bnGCDAsync = function (a, callback) {
    var x = (this.s < 0) ? this.negate() : this.clone();
    var y = (a.s < 0) ? a.negate() : a.clone();
    if (x.compareTo(y) < 0) {
        var t = x;
        x = y;
        y = t;
    }
    var i = x.getLowestSetBit(),
        g = y.getLowestSetBit();
    if (g < 0) {
        callback(x);
        return;
    }
    if (i < g) g = i;
    if (g > 0) {
        x.rShiftTo(g, x);
        y.rShiftTo(g, y);
    }
    // Workhorse of the algorithm, gets called 200 - 800 times per 512 bit keygen.
    var gcda1 = function() {
        if ((i = x.getLowestSetBit()) > 0){ x.rShiftTo(i, x); }
        if ((i = y.getLowestSetBit()) > 0){ y.rShiftTo(i, y); }
        if (x.compareTo(y) >= 0) {
            x.subTo(y, x);
            x.rShiftTo(1, x);
        } else {
            y.subTo(x, y);
            y.rShiftTo(1, y);
        }
        if(!(x.signum() > 0)) {
            if (g > 0) y.lShiftTo(g, y);
            setTimeout(function(){callback(y)},0); // escape
        } else {
            setTimeout(gcda1,0);
        }
    };
    setTimeout(gcda1,10);
};
BigInteger.prototype.gcda = bnGCDAsync;
 
// (protected) alternate constructor
var bnpFromNumberAsync = function (a,b,c,callback) {
  if("number" == typeof b) {
    if(a < 2) {
        this.fromInt(1);
    } else {
      this.fromNumber(a,c);
      if(!this.testBit(a-1)){
        this.bitwiseTo(BigInteger.ONE.shiftLeft(a-1),op_or,this);
      }
      if(this.isEven()) {
        this.dAddOffset(1,0);
      }
      var bnp = this;
      var bnpfn1 = function(){
        bnp.dAddOffset(2,0);
        if(bnp.bitLength() > a) bnp.subTo(BigInteger.ONE.shiftLeft(a-1),bnp);
        if(bnp.isProbablePrime(b)) {
            setTimeout(function(){callback()},0); // escape
        } else {
            setTimeout(bnpfn1,0);
        }
      };
      setTimeout(bnpfn1,0);
    }
  } else {
    var x = new Array(), t = a&7;
    x.length = (a>>3)+1;
    b.nextBytes(x);
    if(t > 0) x[0] &= ((1<<t)-1); else x[0] = 0;
    this.fromString(x,256);
  }
};
BigInteger.prototype.fromNumberAsync = bnpFromNumberAsync;
 
})();

I’m also adding a few functions for sleeping a public or private RSA key object to a simple JSON Transport Object (for storage) and waking it again.

(function(){
 
// Cast to private Transport Object
var RSAPrivTPO = function () {
    return {
        'n'     : hex2b64(this.n.toString(16)),
        'e'     : hex2b64(this.e.toString(16)),
        'd'     : hex2b64(this.d.toString(16)),
        'p'     : hex2b64(this.p.toString(16)),
        'q'     : hex2b64(this.q.toString(16)),
        'dmp1'  : hex2b64(this.dmp1.toString(16)),
        'dmq1'  : hex2b64(this.dmq1.toString(16)),
        'coeff' : hex2b64(this.coeff.toString(16))
    }
}
RSAKey.prototype.privTPO = RSAPrivTPO;
 
// Hydrate from private Transport Object
var RSAFromPrivTPO = function (tpo) {
    this.setPrivateEx(
        b64tohex(tpo.n), 
        b64tohex(tpo.e), 
        b64tohex(tpo.d), 
        b64tohex(tpo.p), 
        b64tohex(tpo.q), 
        b64tohex(tpo.dmp1), 
        b64tohex(tpo.dmq1), 
        b64tohex(tpo.coeff)
    );
    return this;
};
RSAKey.prototype.fromPrivTPO = RSAFromPrivTPO;
 
// Cast to public Transport Object
var RSAPubTPO = function () {
    return {
        'n' : hex2b64(this.n.toString(16)),
        'e' : hex2b64(this.e.toString(16))
    }
};
RSAKey.prototype.pubTPO = RSAPubTPO;
 
// Hydrate from public Transport Object
var RSAFromPubTPO = function (tpo) {
    this.setPublic(
        b64tohex(tpo.n), 
        b64tohex(tpo.e)
    );
    return this;
};
RSAKey.prototype.fromPubTPO = RSAFromPubTPO;
 
})();

Source on GitHub – https://github.com/KevBurnsJr/rsasync

Domain specific media types and REST

Below is a thread posted to rest-discuss. I’ve reposted my response here for easy linking and reading.
http://tech.groups.yahoo.com/group/rest-discuss/message/17650

Daniel Roussel posts:

Hi,

I’ve been reading a lot about how to do “proper” REST this week and the more I read, the more I’m lost, especially the HATEOAS part I fear.

First, to give some context, the company I work for develops mobile applications for clients. Most of the time, they want to get an iPhone native application, an Android application and a traditional Web based Application to cover the other mobile phones out there.

The way we are currently doing things is the good old (bad?) RPC over HTTP way. We define a bunch of URI which are coded inside the different apps, we exchange data as JSON, etc. This week, trying to do things in a better way, I’ve begin a more serious study of REST and how to do it properly.

What I really can’t wrap my head around is how, technically, have HATEOAS in a native application? I mean, when building a native application, I have tables to display lists, buttons to do some things, etc. My understanding is that all those should be displayed based on the data (hypermedia) received from the server. Is that right?

A concrete example would be a hotel room rental service. The person would open the application and have fields to enter the from/to dates. It would then tap a “Get Available Rooms”. The app would call the server and get back a list of rooms along with prices and other details. From there the person could select one room and rent it.

The RPC way of coding this is obvious to me but I have no idea how I’d do that in a proper REST way! What bugs me is that every way I look at it, the client application would still be tightly coupled to the service. I understand how I would only need to GET the http://rent-a-room.com URI hardcoded and then in the response I would have the http://rent-a-room.com/available-rooms URI given. But… My application would expect each “call” to return some pre-defined data and “rel”, those can’t appear out of the blue?!

I guess what I’m trying to say is that both the business process and the data exchanged must be known to my client application at the moment of coding it, and those can’t change without breaking existing clients. But reading about REST, every is talking about loose coupling and not breaking clients… I just don’t see it.

What am I missing?

Thanks a lot and sorry if it is a stupid question!

Daniel, Here are 3 ways you might use self-descriptive messages in your API

1) Create many domain specific media types (one for each view)

Content-Type: application/rent-a-room+xml

2) Create one domain specific media type

Content-Type: application/vnd.hotels.com+xml

3) Create zero domain specific media types

Content-Type: application/json
Link: </schema/rent-a-room>; rel="describedBy"

All three of these approaches could be seen as satisfying the self-descriptive messages constraint.

If you create many DSMs (domain specific media types), your application might bind the media type to the view class via some sort of client-side configuration.

"application/rent-a-room+xml"   =>   RentARoomView

If you create one DSM, your media type might specify the semantics by which a representation specifies details about itself which could be used in rendering the representation in a GUI.

{"_type": "rent-a-room", ... }

… which you might then bind to a view …

"rent-a-room"   =>   RentARoomView

If you create zero DSMs, your application might bind the value of the describedBy link header to a view in the gui.

"/schema/rent-a-room"   =>   RentARoomView

An alternative approach would be to create one DSM with a richer semantics which would effectively allow you to compose the interface from the server side using code-on-demand and/or more granular views

{"_links":[
    {"rel":"view","type":"text/javascript","href":"/views/RentARoomView.js"}
    {"rel":"commentable","type":"text/javascript","href":"/attributes/commentable.js"}
]}

This Code-on-demand approach would take greatest advantage of the constraints of REST to create a highly evolvable service by never binding anything directly to a view class within the application. Instead, your application would become a user agent, parsing representations and fetching additional computational resources as necessary to render the view.

Code-on-demand may be significantly less feasible if your client is written in object C, but perhaps it’s something to think about. The embedded links might not be javascript or CSS, but perhaps some other language used for GUI composition, such as XUL or a simple DSL.

Finally I’m sure it goes without saying that whatever way you wind up rendering a representation for a view, the UI would contain links which you would click to navigate to new screens which are built using the data and metadata from the representation of the resource identified by the link.

And there you have a few takes on creating an engine of application state with self-descriptive messages and code-on-demand.

Three Line Ternaries

These little buggers are so much more readable than their single-line brethren:

$yourmom['fat_joke'] = $yourmom['dinner'] instanceof Elephant
	? "so fat she ate Dumbo" 
	: "so skinny Africa sends her food";

Anonymous Functions in Mustache

When using Anonymous Functions in Mustache, it’s important to note this line from the spec:
https://github.com/mustache/spec/blob/master/specs/~lambdas.yml#L82

Lambdas used for sections should receive the raw section string.

This means that the inputs to your lambda will not be pre-rendered. The output of the lambda will be rendered.

So I had a lambda that looked like this…

function($timestamp) {
	return str_replace("T", " ", $timestamp);
}

But It was giving jibberish.
bobthecow (author of Mustache.php) helped me realize that I needed to render the inputs myself…

function($timestamp) use($view) {
	return str_replace("T", " ", $view->render($timestamp));
}

The lesson is this:
The next time I run into a problem, I’ll be sure to read the tests.

Thanks, Justin!

The Cloud is not a silver bullet.

Many decisions are made at the outset of a brand new product/company. Recently I participated in an all-hands meeting to kick off development of a company’s first product (CEO guy, Product guy, Engineering guy [me]). We talked about the product and the architeture and came around to the infrastructure. It was decided that we would use AWS for its handy scalability.

During the conversation, the product guy (who has some tech chops) said something that caught me off guard. His quip was something like: “Cloud is the way to go. We’re definitely going to do everything in the cloud.”

Below is my lengthy response.

Hi Chris,

You mentioned something in that first meeting which I’ve been wanting to elaborate on and partially refute.

I’d like to caution that The Cloud is not a silver bullet.
While AWS may be an excellent solution for the immediate future, there may come a point where moving to a datacenter proves to be more reliable and cost-effective.

There are a number of properties of the infrastructure involved in cloud computing which guarantee that some cloud resources will never match the performance and reliability of bare metal.

See for instance the recent Reddit outage in which AWS EBS volumes became unavailable for several hours, resulting in serious data problems.
http://blog.reddit.com/2011/03/why-reddit-was-down-for-6-of-last-24.html

In AWS, the hard drives (EBS) are connected to the machines (EC2) over a shared 1Gb/s ethernet connection.
In a real server, the hard drives are connected to the machine over a dedicated 6Gb/s SATA cable.

There are some ways to help mitigate these bottlenecks
http://victortrac.com/EC2_Ephemeral_Disks_vs_EBS_Volumes

1) you can set up multiple EBS volumes in a RAID configuration to increase throughput
2) you can load balance traffic across multiple availability zones to help mitigate network congestion and availability issues like those experienced by Reddit

But also keep in mind that there will come a point at which renting a cabinet (42U) in a datacenter will become far more cost effective than running a server farm in the cloud.
A full cabinet with 15Amps of power and a 100Mb/s connection runs $500/mo with a signed 3 year lease.
A quarter cabinet (7U) with 2Amps of power and a 100Mb/s connection runs $200/mo with a signed 3 year lease.

So if we took one of these $200 PE 1850s I have sitting here and plugged it in to a quarter cabinet at Hurricane Electric, we would already have more processing power and it would save us $40/mo over our current us-west m1.large instance.

But there are 2 advantages to the cloud:
1) If we need more capacity, we can provision it immediately without having to provision hardeware
2) If we need capacity in a different location, we can easily spin up a cluster in Japan or Europe or the East Coast without having to physically be there.

In the long term I hope we’ll come to a hybrid approach where we can
- leverage the best of real-world datacenters (for backups and heavy number crunching)
- and leverage the cloud for all its advantages (for geo dispersion, fault tolerance and early stage ramp-up)

Remember the words of Devops Borat
http://twitter.com/DEVOPS_BORAT/status/40816140632596480

“Formation of hybrid cloud: 1) roll in datacenter 2) SHIT! roll to cloud 3) SHIT! roll half back to datacenter.”

- Kev

Thanks, Chris for suggesting I post this to the blog.

PHP Fatal Error 500

How to force PHP to return an empty 500 response when it encounters a fatal error:
Add this to the very beginning and very end of your bootstrap file

register_shutdown_function(function(){	
	if(!defined('REQUEST_SUCCEEDED')) {
		header("HTTP/1.1 500 Internal Server Error");
		if(getenv('APP_ENV') != 'dev') {
			ob_clean();
		}
	}
});
 
// ...
// ... your whole application ...
// ... from top to bottom ...
// ...
 
define('REQUEST_SUCCEEDED', true);

How to prevent a leak like Tumblr’s

Tumblr took a bit of a tumble today.
http://news.ycombinator.com/item?id=2343330

One of Tumblr’s engineers (presumably) deployed a file to production which contained a critical flaw.
The first character of the file was replaced with an `i` instead of a `<`.

i?php
    require_once('chorus/Utils.php');
    require_once('chorus/Kestrel.php');
    require_once('chorus/DataService.php');
    require_once('chorus/Shard.php');
 
    Database::set_defaults(array(
        'user'     => 'tumblr3',
        'password' => 'm3MpH1C0Koh39AQD83TFhsBPlOM1Rx9eW55Z8YWStbgTmcgQWJvFt4',
        'database' => 'tumblr3',
    //    'write_lock_tables' => '*',
        'extended_log' => (idate('G') == 17 && intval(idate('i')) == 56 && trim(`hostname`) == 'web10.tumblr.com')
    ));
 
    if (__FILE__ == '/var/www/apps/tumblr/config/config.php' || __FILE__ == '/data/tumblr/config/config.php') {
        define('ENVIRONMENT',      'production');
        if (! defined('DEFAULT_DATABASE')) define('DEFAULT_DATABASE', 'primary');
        define('S3_BUCKET',        'data.tumblr.com');
        define('ENABLE_PANTHER',   true);
        define('ENABLE_MEDIA_CDN', true);
        define('ASSETS_URL',       (ENABLE_MEDIA_CDN && ! (isset($_SERVER['HTTPS']) && $_SERVER['HTTPS']) ? 'http://assets.tumblr.com' : ''));
        define('MEMCACHE_HOST',         '10.252.0.68');
        define('MEMCACHE_VERSION_HOST', '10.252.0.67');
        define('VALIDATION_FAILURE_LOG', BASE_PATH . '/validate.log');
        # <snip>

Yes, that is tumblr’s production database password.

Full source here

Once this error was introduced to production, people viewing any page would see a dump of the first 749 lines of the config file along with some PHP errors. THEN, GoogleBot came along and indexed the whole mess which is why you can still see it in Google’s search results

Learning from the mistakes of others

How can we keep this from happening to us?

First of all, go to any project on your local dev environment, replace the first `<` with `i` in any included file in your project (hint config.php) and see what happens. Chances are you'll see exactly the same thing that happened to Tumblr.

The only solution I see to this is pre-commit syntax checking for committed PHP files.

Here's a tutorial for php syntax checking in SVN and here’s a pre-commit hook script for php syntax checking in GIT.

Basically the way it works is that if you ever commit a PHP file which contains a syntax error, your commit will be blocked and you will have to amend it before you are allowed to commit it to the repository. If Tumblr had done this, they might never have leaked their config file.

[UDPATE] Apparently both PHP’s built-in Syntax Checking and the PEAR package PHP_CodeSniffer won’t pick up on errors such as this. Searching for a valid solution now…

[UDPATE] The best solution I’ve seen so far is to always return 500 responses to clients with a generic error message in production, thus preventing errors like this from bubbling up at the Apache level.

[UDPATE] Turns out PHP returns 200 when it encounters a fatal error. Inconceivable.

[UDPATE] Here’s how to force PHP to return a 500 when it encounters a fatal error. Add this to a prepend file or make it the first 5 lines of your bootstrap file.

function die_with_honor() {
	header("HTTP/1.1 500 Internal Server Error");
	ob_clean();
}
register_shutdown_function('die_with_honor');

A letter from momtrepreneur

Hi, its me again ;-).

I was thinking about what you said about focusing on your stuff in the US. I was driving along the road to Supetar today, passing the olive orchards and thinking of you.

I was thinking about how one’s 20s are the time for learning and exploring, honing skills and interests, finding out what you like and don’t like, learning what gives you satisfaction and a sense of purpose – for your professional and personal life. One’s 30s are an important time to focus what you have learned and build the foundation for life. I think that means choosing how you want to work and what you want to do as well as building long lasting professional, personal and family relationships.

In my 20s I worked in a lot of areas and learned as much as I could technically as well as about business, constantly challenging myself to try new things until I found what I was best at and what gave me the most sense of satisfaction and purpose. It took a long time ;-). I must have taken every type of class offered at school, just for the enjoyment of it. Once I discovered the computer industry, there was an endless supply of things to learn, which I had never dreamed of. I hadn’t even seen a computer ever when I started my first computer business. I was stunned by the possibility of it when I went to my first computer club meetings where they were talking about the Apple I computer and Local Area Networks (ethernet to be specific).

I also learned after having you that I wanted to have as much control and flexibility over my time so I could focus on you children and the family. This is why I went into consulting. I felt I could have rewarding work, would learn a lot from working with many companies, but also not be overwhelmed by the demands of a job where someone else decided when and where. I think I built a very good foundation in my 30s for this and was able in my 40s to have even more flexibility and further develop my business capabilities. Of course, I would have liked to have spent even more time at home because once I focused on business, the demands from clients made it increasingly difficult to manage my time. The conundrum of balancing work, getting new business and family life was much harder than I thought it would be.

The first time I tried consulting on my own (right after I had Deirdre), I was unsuccessful. I didn’t want to go back to work with the two of you at home. I was 30. But it turned out I didn’t have the contacts or ability to get clients, so I didn’t have much work – we had invented email on the PC and Macintosh, but nobody really used it. And the web had not been commercialized, so the only people that used it were universities and research companies like SRI.

I was really lucky that a friend of mine who worked at Apple called me and asked if I wanted to work on a project for Apple because one of the consultants that was working on it quit right in the middle and had not done any of the work. It turns out he didn’t know what to do. So, I said yes and wound up starting a consulting firm with the 3 people who were working on the project together. . . so many serendipitous events, which really shaped my life. I think what I learned from it was that in order to get to a point where I could do what I wanted, I had to have some foundations put into place. About 5 years later, I was able to start my own firm, quite successfully – I had learned how to run a consulting business and I had the network to get work.

We haven’t spoken in a long time about what you want and what you are thinking about, I am very interested in your thoughts and feelings about what you are doing.

Lots of love,

Mom

Mixing it up

I have 6 books on my nightstand, each on a different topic:

Lean Thinking process engineering
Metaphors We Live By role of language in psychology
Just Enough Software Architecture software architecture
Drive psychology of motivation
Web Operations operations
Leading Geeks management
Programming Erlang programming

This gives me the opportunity to jump around to follow whatever impulse is fueling me at any given time. If I’m feeling burned after working around some bogus proprietary API, I will jump into a book on Software Architecture to remind me that there is still a path toward sanity. If I’ve been teching out for days and feel like I’ve lost the forest for trees, I’ll pick up a book on psychology to remind me that software does still have the potential to assist human beings in a large number of not-necessarily-obvious ways. If I’ve spent all day writing emails and haven’t had time to touch a piece of code, I’ll jump into a book on process or management to give me new ideas on how to talk less and do more. If I’m feeling adventurous, I’ll pick up a book on a new language and type along for a few hours to advance my understanding of the universe.

And I do the same thing with projects:

Teambo A generic ticket tracking application
ToroPHP A PHP Framework
ripple-php A Riak ODM for PHP (ported from Ruby)
riak-php-client Riak’s PHP client which I’m extending to include support for Protocol Buffers

If I’m burnt on churning out HTML and CSS for days, I’ll step back and jump into something esoteric like adding a new transport protocol to an open source database client. If I’ve been crunching on a hard CS problem and start lacking steam, I’ll jump over to an application I’ve got on the back burner and knock out a few quick features to get the gears turning again. If I’m losing sleep over a cool new use case I’ve been turning over, I’ll flip on the lights and crank out a few test cases to put my ideas on paper as fodder for tomorrow’s session.

This sort of autonomy is crucial to remaining happy and productive.

Plant a thousand projects and let them flourish as they may.

Pretty JSON : Pipe to pj



    google image search for
    pipe and pjs

If you wind up doing a lot of curl from the command line (like you will using Riak), add this line to the bottom of ~/.bashrc

# ...
alias pj='python -mjson.tool'

Now when you’re curling, you can just pipe curl output to pj:

$ curl -s http://localhost:8098/riak/stats | pj

And this …

{"props":{"name":"stats","n_val":3,"allow_mult":false,
"last_write_wins":false,"precommit":[],"postcommit":[]
,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_s
td_keyfun"},"linkfun":{"mod":"riak_kv_wm_link_walker",
"fun":"mapreduce_linkfun"},"old_vclock":86400,"young_v
clock":20,"big_vclock":50,"small_vclock":10,"r":"quoru
m","w":"quorum","dw":"quorum","rw":"quorum"}}

Becomes this …

{
    "props": {
        "allow_mult": false,
        "big_vclock": 50,
        "chash_keyfun": {
            "fun": "chash_std_keyfun",
            "mod": "riak_core_util"
        },
        "dw": "quorum",
        "last_write_wins": false,
        "linkfun": {
            "fun": "mapreduce_linkfun",
            "mod": "riak_kv_wm_link_walker"
        },
        "n_val": 3,
        "name": "stats",
        "old_vclock": 86400,
        "postcommit": [],
        "precommit": [],
        "r": "quorum",
        "rw": "quorum",
        "small_vclock": 10,
        "w": "quorum",
        "young_vclock": 20
    }
}

Self-descriptive hypermedia in Riak

So… tell me about yourself.

GET /riak/people/old-macdonald
Accept: application/json

Not much to say, really. My name is Old MacDonald. I have a farm.

HTTP/1.1 200 Okay
Content-Type: application/json
Link:  </riak/people>; rel="up", \
    </riak/schema/person>; riaktag="describedby", \
    </riak/animals/trixie>; riaktag="pet", \
    </riak/locations/macdonalds-farm>; riaktag="farm"

{
    "name": "Old MacDonald"
}

Ya, I’ve heard the song. But lets pretend like I’m an alien.


WHAT IS A PERSON? BEEP BOOP

GET /riak/schema/person
Accept: application/json

A person is an animal of the species Homo Sapien.

HTTP/1.1 200 Okay
Content-Type: application/schema+json
Link:  </riak/schema>; rel="up"

{
    "id": "/riak/schema/person",
    "extends": "/riak/schema/animal",
    "type": "Person",
    "description": "A Homo Sapien",
    "properties": {
        "species": "homo sapien",
        "arms": { "type": "number", "default": 2 },
        "legs": { "type": "number", "default": 2 }
    }
}

What is an animal?

GET /riak/schema/animal
Accept: application/json

An animal is a member of the animal kingdom.

HTTP/1.1 200 Okay
Content-Type: application/schema+json
Link:  </riak/schema>; rel="up"

{
    "id": "/riak/schema/animal",
    "type": "Animal",
    "description": "A member of the animal kingdom.",
    "properties": {
        "species": { "type": "string" },
        "name": { "type": "string" }
    }
}

Okay, I get it. So tell me more about that `farm` you mentioned.

GET /riak/locations/macdonalds-farm
Accept: application/json

Dude. My farm is insane. You would not believe how many animals I have.

HTTP/1.1 200 Okay
Content-Type: application/json
Link:  </riak/people>; rel="up", \
    </riak/schema/location>; riaktag="describedby", \
    </riak/animals/wilbur>; riaktag="animal", \
    </riak/animals/bessy>; riaktag="animal", \
    </riak/animals/mr-ed>; riaktag="animal", \
    </riak/animals/donald>; riaktag="animal", \
    </riak/mortgages/macdonalds-farm>; riaktag="mortgage", \
    </riak/mortgages/macdonalds-farm-2>; riaktag="mortgage"

{
    "name": "Old MacDonalds Farm",
    "geo": { "lat": "30.000635", "lng": "-95.225313" }
}

What is a location?

GET /riak/schema/location
Accept: application/json

LOL wut? A location is a location. Are you high right now?

HTTP/1.1 200 Okay
Content-Type: application/schema+json
Link:  </riak/schema>; rel="up"

{
    "id": "/riak/schema/location",
    "type": "Location",
    "description": "A location",
    "properties": {
        "name": { "type": "string" },
        "geo": {
            "type": "object",
            "properties": {
                "lat": { "type": "number" },
                "lng": { "type": "number" }
            }
        }
    },
    "links": [
        {"map": "http://maps.google.com/maps?ll={geo.lat},{geo.lng}&z=18"}
    ]
}

No dude, I’m an alien. Remember? BEEP BOOP BEEP

Tell me about this animal on your farm.

GET /riak/animals/bessy
Accept: application/json

Bessy is a dairy cow. She goes `moo`.

HTTP/1.1 200 Okay
Content-Type: application/json
Link:  </riak/people>; rel="up", \
    </riak/schema/cow>; riaktag="describedby"

{
    "name": "Bessy the bovine",
    "type": "dairy"
}

What is a `cow`?

GET /riak/schema/cow
Accept: application/json

A cow is a producer of butter or a precursor to steak or an object of worship.

HTTP/1.1 200 Okay
Content-Type: application/schema+json
Link:  </riak/schema>; rel="up"

{
    "id": "/riak/schema/cow",
    "extends": "/riak/schema/animal",
    "type": "Cow",
    "description": "A Cow is a bovine animal.",
    "properties": {
        "type": { "type": "string", "enum": [
            "steer",
            "dairy",
            "sacred"
        ]},
        "legs": { "type": "number", "default": 4 }
    }
}

Tell me about your pet.


I still don’t know that she’s a dog. Or an animal for that matter.

I am programmed to treat URLs as opaque.

GET /riak/animals/trixie
Accept: application/json

Trixie’s a 3-legged Australian Shepherd. She got hit by a car when she was
young and lost the use of her front left leg.


Wait, how did you know she is a dog if you didn’t know she is a dog?

HTTP/1.1 200 Okay
Content-Type: application/json
Link:  </riak/people>; rel="up", \
    </riak/schema/dog>; riaktag="describedby"

{
    "name": "Trixie",
    "breed": "Australian Shepherd",
    "legs": 3
}

What are you talking about? I don’t even know what a dog is.

I’m an alien, remember? BEEP BOOP BOP BEEP

GET /riak/schema/dog
Accept: application/json

Whatever, dude. This whole routine is gettin’ kinda played out.

HTTP/1.1 200 Okay
Content-Type: application/schema+json
Link:  </riak/schema>; rel="up"

{
    "id": "/riak/schema/dog",
    "extends": "/riak/schema/animal",
    "type": "Dog",
    "description": "A Dog is a Canis lupus familiaris.",
    "properties": {
        "species": "Canis lupus familiaris",
        "breed": { "type": "string" },
        "legs": { "type": "number", "default": 4 }
    }
}

Ya, you’re right. Lemme just try 1 more thing.

DELETE /riak/animals/bessy

Dude.

HTTP/1.1 204 No Content

Hi, I’m a PHP Developer.

This year in #rest on Freenode



Full Size (1920×1100)
pdf also available

The more treacherous paths are yet unsolved

PBF Cube

Illustration used without permission from
The Perry Bible Fellowship Almanack
http://www.pbfcomics.com

We tend to see success as a constant. As though the puzzle of our existence were a common mystery. In our understanding we misunderstand that a success for some may easily be viewed as a failure by others who see the world in different terms. Living in a world such as ours in the top 1% of the planet’s wealthiest, it is difficult for us not to lead a life which has a carbon footprint one thousand times the average.

The chaos of a disenchanted existence while inglorious holds at least a candle to the dark reality fueling our unsustainable existence. This is an illustration of the indetermination expressed by those who accept their block with two yellows as an absolution of the responsibility they have to their contribution to this world. We all have double yellows at some places and levels of our being. So this is also an illustration of the ignorance expressed by those who feel shame in seeing that which makes them unique. Hoping nobody will see that their whites are touching their yellows.

A dark portrait of what our society has become.

That’s what I call rap

munin-node on EC2

If you’re trying to set up multiple munin nodes in EC2, you need to use the internal hostname to reference slave nodes.

munin.conf

[web01]
    address domU-22-31-38-12-25-04.compute-1.internal
    use_node_name yes

munin-node.conf

host_name web01
allow ^.*$

Voila!

Also be sure to open 4949 if you’ve got a security group set up.

Talent Acquisition

Talent Acquisition

compile php-cgi

If you’re compiling PHP and you can’t find the php-cgi binary, remove ‐‐with-aspx2 from the config params and you should be good to go.

php ubuntu error: xml2-config not found.

You need to install libxml2-dev

sudo apt-get install libxml2-dev

_trackEvent not working

label can not be an integer or Google’s _trackEvent function fails silently.

pageTracker._trackEvent('VLP', 'click-out', 12345);

Fixed as

pageTracker._trackEvent('VLP', 'click-out', '12345');

Next Page »