After resisting exceptions for a while, I started to come around. It started to become clear that even if code is perfect (which it rarely is) there is still a pile of network and system related errors that can occur. While one way to handle these errors is to add return codes to all functions that happen to touch the outside world, a perhaps cleaner way is to throw an exception. All of the sudden exceptions became cool.

Knowing that exceptions have a time and place in a program, I started designing all kinds of complex exception handling machinery. My code was destine to handle all exceptions with grace and ease, and easily recover. An exception handler would in turn try another body of code that could throw an exception, which would enter a wait state before retrying and catching another exception and so on. It was quite a mess, and the more I developed these techniques, the more I became acutely aware that this wasn’t really the right way to do things either. Applications simply can’t be able to recover from every possible scenario.

A few years and a couple of paradigm shifts later I’m finally starting to see what the point of catching an exception is, why they are useful, and what to do when one occurs. The bottom line is the point of an exception is to limit damage when something unexpected happens in your code, get back to a known state if possible, or abort if not. Exceptions do not make your code bullet-proof, however they do a great job of mending wounds and keeping the system up, despite unexpected occurrences. Nowadays my exceptions follow a pretty rigid two-phase pattern, restoring state and restoring flow control, and this works reasonably well.

Restoring State

The most immediate goal of an exception is to return the application to a known state. This involves performing any necessary maintenance on modified data structures, managing any unmanaged resources that need to be released, and deleting any temporary files that had happened to be created during the execution of the routine.

I’ve been doing a lot of server work lately, and this usually consists of another try block to close any open database connections, and an iteration through a list of temporary files. As certain methods execute in my program that use intermediate files they keep a list of created temporary files, and my error handling routine iterates through this list and deletes any files still in existence.

In other cases I end up deleting items from a HasMap structure, where the value of the entry points to an incompletely initialized object. Either way, after this part of the exception routine is done, the application should be in a state identical to before it entered the try block.

Restoring Flow Control

After restoring the application’s state, the next problem is where to transfer flow control to. I’ve found that exceptions usually break down into two pretty clean categories.

Temporary exceptions occur when servers go down, or network requests fail. They are often transient, and upon retrying might succeed. For these kinds of exceptions I usually implement some sort of retry loop, with a fixed number of retries allotted based on the timing requirements of the function I’m in.

Permanent exceptions involve events that are not likely to ‘fix themselves’ if given time. File permissions issues, logic errors, and most other non-IO exceptions fall into this category. When I encounter this kind of exception the show is basically over. If the exception occurs in a reasonably compartmentalized task, as is often the case, the entire program doesn’t need to cease execution, we after all did restore it to a known state, but we do need to abort the current task and present a failure message to the user.

When a temporary exception times out after a certain number of retries it too becomes a permanent exception.

Exception Hierarchy

In this model it’s actually really reasonable to throw in a hierarchical approach to exceptions as well. It’s common in the Java world to catch an exception, only to throw another one. For example a class named MagicMoneyWorker might throw MagicMoneyWorkerException exceptions as a result of network timeouts when performing remote procedure calls. It has wrapped the more low-level exception with a high level one.

The reasons for doing this fit well with the object oriented approach to programming. The MagicMoneyWorker class probably knows best how to handle a network timeout. It knows how to preserve it’s inner state, and recover, and probably even has some retry and reconnect logic built in to it. By the time it’s done handling the exception, if an error occurs, it’s a permanent exception and it means that the task at hand failed. This needs to be handled now by the caller function.

Wrapping exceptions in this manor allows you to keep the primitive exceptions hidden, and preserves the level of abstraction you’re operating with. If you’re writing a function that deals with MagicMoneyWorkers on a high level, you don’t want to have to dive into the bowels of the implementation. It makes much more sense for an exception with a matched level of abstraction to be thrown, and for your function to act on this exception.

Simply my Personal Approach

I doubt this approach will work well for all systems or applications, but I’ve found it works reasonably well for my development lately. Throughout my computer science education there hasn’t been much focus on how to handle exceptions. I think they are considered more of an engineering problem, because the code we deal with is so algorithmic in nature, and brushed under the rug as something you’ll pick up over time. Developing a personal framework for how to handle an exception, and giving a rigid procedure, purpose, and ordering to the process has tremendously helped hone my skills in developing real world applications. If you don’t have a clear idea of how to deal with exceptions in your own projects, I would definitely recommend spending some time to mediate on them and figure out what the goals of catching an exception are for you.

I’m just finishing up a trip to IPSN’12 in Beijing this week where I presented a demonstration of my current research project. It’s a pretty cool piece of work. We wanted to develop a device that plugs into the headset port of a mobile phone and allows for capture of analog signals. To that end we developed a hardware and software system called AudioDAQ that does just that.

The system consists of a small square-inch form factor piece of hardware and server side software for processing. It allows for arbitrary analog waveforms to be exported over the microphone line in the audible range, and is powered entirely from the microphone bias voltage. This allows for data to be captured by the built-in voice recording application of the phone, and makes the system compatible with virtually every handset on the market today with no software needed on the phone-side.

Data captured in a voice recording is sent to a remote server, where an algorithm extracts the original signal and produces a plot of the data, along with a comma-separated-value file of the data points.

I’ll dive into a little bit of the design of this system. I think it’s a really good example of something that’s cleverly simple. Despite being fairly simple, we encountered significant design challenges while fine-tuning system parameters to optimize power transfer and data recovery.

Delivering Power

The AudioDAQ hardware consists of a small number of analog and digital components, that require small amounts of power to operate. Additionally we wanted to provide enough power for small active sensors to operate. Most transducers only require a small number of active op-amps to operate which, if efficiently designed, will only draw a couple hundred microwatts of power. Because of this the microphone bias voltage was a suitable candidate for powering the system.

The microphone bias voltage is traditionally used to power small amounts of amplification circuitry in modern microphones found in hands-free pieces. It has been found to be around two DC volts on most surveyed handsets, and sits behind a high-impedance resistor that prevents too much current from flowing (R1 in the photo above). Because of this resistor it does not provide too much power, usually on the order of hundreds of microwatts.

We feed the microphone bias voltage into a small ultra low dropout linear regulator to ensure it is a consistent 1.8V. This becomes important because we also use this voltage as a reference voltage for our system.

Capturing Data

Next we must capture data for recording. The microphone port cannot simply be fed a DC-valued analog signal. It has a high-pass filter, formed by C1 and R2 in the first diagram, that prevents DC values from making their way through the system. To overcome this we go with a simple, effective solution: installing an analog multiplexer to switch between system ground and the signal at a speed within the audible passband.

This multiplexer creates a square wave with an amplitude reflective of the DC value of the original signal. Phone analog front ends all have different characteristic values however, and while the magnitude is proportional to the voltage of the analog sensor signal, it does not have an implicit scale. To fix this we additionally export the reference voltage across the microphone port. By switching between ground, the signal, and a reference voltage we can determine where the signal sits between the reference voltage and ground, and scale accordingly.

To further extend the system we can add multiple channels of inputs as shown in the diagram above. This gives us the ability to simultaneously capture multiple analog signals.

An interesting design challenge was managing the tradeoff between signal fidelity and energy delivery. The amplitude of a microphone signal is approximately 10mV, which is quite small. Adding a linear regulator and a small amount of capacitance will easily drown out that signal by adding additional noise and attenuating it. To negate this we installed an additional resistor between the linear regulator and the microphone line. Sizing it carefully yields a good tradeoff between power delivery, and the isolation of the microphone line from noise from the linear regulator.

Data is captured with the voice recording app on the phone and e-mailed to our server for decoding. Because almost every phone manufactured recently has a built-in voice memo application the AudioDAQ platform is compatible with a large base of existing devices.

Processing the Data

Finally we must reconstruct the signal. Currently a small piece of Python code does this. The code grabs the multiplexed, encoded audio data, and extracts from it framing information. You can see the intermediate steps of the algorithm in the figure. It first detects the edges, finds a mean value to represent the steps between the edges, measures these values, and finally reconstructs the original signal from them. In practice this works wonderfully, with the original signal being easily recovered.

In Closing

AudioDAQ works really well and has been a big focus of my work for the past few months. You can read the published demo paper here. Signal reconstruction obtains good quality results and the technology is fairly well developed. If you have any interest in using AudioDAQ in your projects, feel free to contact me and I’d be more than happy to send you some schematics and code!

To that end I’ve started looking at characterizing TCP delivery on my mobile phone, in the hopes that from this characterization I can develop some more efficient algorithms, and just get an intuitive sense for what can and cannot be offloaded to the server. A big idea that has started gaining some traction lately is heterogeneous computing, where computationally expensive tasks are offloaded from your phone to the cloud for processing. Knowing the abilities of the network data channel will play a big roll in determining the feasibility of this, and many other ideas.

Let’s Transfer Some Images

My current side project is centered around manipulating images on the mobile phone, or in the cloud, so a good starting point in this study was to see how long it takes to upload and download images of various sizes, over HTTP. I used three image sizes:

• Small – 11KB
• Medium – 51KB
• Large – 86KB

I took these sizes from the three thumbnail sizes Facebook’s Haystack image store uses to present images to the end user, as those sizes seem to be working reasonably well for them. A large image is by no means a full-sized image, but it’s more than big enough to display on any smartphone screen, with some zooming supported.

Because this is a first pass, the implementation is pretty rough, and leaves room for optimization. My goal was not to see what the most optimal way to send an image across the wire was, but it was to characterize the most obvious way a typical developer might do it. I used the pre-bundled HTTP classes, and did everything the obvious way. I wrote two functions and wrapped them up in an Android service, with some logging facilities.

The functions themselves looked like this:

Downloading

        private int doDownload(String path) {
            long startTime = System.currentTimeMillis();

            try {
                URL url = new URL(path);
                URLConnection connection = url.openConnection();

                // getting file length, will cause the first
                // read of the HTTP headers
                int lenghtOfFile = connection.getContentLength();

                // create an input stream to grab data from the connection
                InputStream input = new BufferedInputStream(connection.getInputStream(), 8192);

                // do a read to null basically and grab the entire image, forcing
                // the internals of the stream implementations to do what they must
                // do to push the image to us.
                byte data[] = new byte[1024];
                long total = 0;
                int count;
                while ((count = input.read(data)) != -1) {
                    total += count;
                }

                // close the connection
                input.close();
            } catch (Exception e) {
                return -1;
            }

            long stopTime = System.currentTimeMillis();

            return (int) (stopTime - startTime);
        }

Uploading

        private int doUpload(String filePath, String path) {
            long startTime = System.currentTimeMillis();

            try {
                // No simple way to do a multipart post, so we
                // build it from scratch in a sense.
                HttpClient httpClient = new DefaultHttpClient();
                HttpContext localContext = new BasicHttpContext();
                HttpPost httpPost = new HttpPost(path);

                MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
                entity.addPart("image", new FileBody(new File (filePath)));

                httpPost.setEntity(entity);
                HttpResponse response = httpClient.execute(httpPost, localContext);
            } catch (IOException e) {
                e.printStackTrace();
                return -1;
            }

            long stopTime = System.currentTimeMillis();

            return (int) (stopTime - startTime);
        }

Quite frankly, they leave a lot of questions to be answered. To really understand what’s going on, we need to dive into the internals of the BufferedInputString class, and do some TCP dump captures of the packet data. It’s of great interest to see how the request headers and body are broken up, controlling TCP packet segmentation could have big gains when transferring images. As I said before however, this is only preliminary exploration, so we’ll leave all that to be answered in the future.

Results

I left my sample application service running on my phone as I went for a run over the course of two hours in the city. I figured this would give me a decent spread of typical mobile conditions, in a diverse set of areas, and give decent data. I collected approximately 1500 data points and produced some histograms of the data.

Taking a look at the data already yields some really interesting insights. Images are generally transfered within 2-3 seconds, that’s the common case, with an extremely long tail of extended delivery. Interestingly enough I didn’t have a single timeout, but some images took over a minute to transfer, which is completely unacceptable for the end-user experience, considering the size of the images involved.

The difference in size as compared to transfer time was approximately linear. Uploading of the small image seemed to exceed expectations, and didn’t actually have much of a long tail like other transfers. I’m guessing the length of the tail is a function of the number of round-trips required to successfully transfer the data. Examining the packet flow of these images would yield some interesting insight into that relationship.

Conclusions

So this really just creates a bunch of further questions. It’s a really high-level view of delivery times for small files on the phone, but closer examination could yield some more interesting information. I’m really curious about applications of this knowledge in designing efficient image uploading algorithms. Relaxing some of the guarantees of TCP might allow for significantly faster image upload, and data transfers more suited for mobile apps. Expect to see more along this vector in the blog in the future!

Java really is a nice language for all the criticism it receives. While it doesn’t quite match the ‘batteries-included’ model of Python, the built-in standard library is excellent, striking a nice balance between generality and performance. The standard containers are well-matched and don’t try to hide the underlying data structures, which is appreciated. There’s a vast number of libraries available that can perform almost any task available for you. Even concurrency support is decent, with the excellent java.util.concurrent library providing many thread-safe collections, and monitor/lock semantics for tightly coupled threading.

The garbage-collected nature of the runtime environment, while sacrificing some performance, takes a lot of focus away from low-level details and memory management, which is not easy or directly related to the goals we often wish to accomplish as programmers. The syntax is a little verbose, but it does the job and scores major points for being not Visual Basic or Cobol.

Java by itself is really not that bad, however, anytime I’ve gone near a certain corner of the Java world I’ve found myself irritated. In my opinion, by far the largest problem with the Java language is the attachment of this entire enterprise mindset. The entire Java enterprise ecosystem feels so bloated by frameworks and technologies that it’s almost unusable. At multiple points in my life I’ve spent a few hours trying to wrap my head around this strange world, but even creating a definition for what exactly the composition of a J2EE application is seems like a venture in self-torment. I feel as if a lot of this stuff exists for the sole purpose of making the ecosystem more complex and raising the barrier to entry to ensure that developers need additional training to work competently within it.

Coming from the recent work I’ve done in the Node.js ecosystem, where developers focus on keeping libraries intentionally small and compact, and making obvious, light-weight APIs, the world of heavy Java frameworks seems silly. I do not need a GenericToasterDataSourceFactorAdapter, I need a Toaster, and would prefer to instantiate one directly.

So far I’ve started to take a minimalistic approach to Java programming with some great success. I’m not using a framework to manage my database connections, or message passing servers, I’m using the connection objects directly. I’ve written thin wrappers around them to handle basic reconnection logic, and connection pooling using existing libraries such as 3CP0 when needed. When a connection-level error occurs it’s much more obvious what should be done this way. I can take context-specific actions based on the demands of my application when a service goes down and perform actions such as simply blocking until the service becomes available, or silently dropping the message in cases of logging and other areas that only need soft delivery guarantees. Threading is done with a thread-pool, that can be tuned to the needs of my application and messages are passed directly from the RabbitMQ object, with some glue code managing loss of connection for me as well. This all is working really, really well, and I’ve been thrilled with the performance and reliability of the service in practice.

My original approach involved picking a framework and building on top of it. I found myself spending so much time learning the specifics of that framework, and looking up the syntax and design patterns, when what I wanted to accomplish was not nearly as complicated as the intricate dances I had to play with the framework itself. I really wouldn’t recommend this to anyone, it was tedious and frustrating.

What I would recommend is keeping your application reasonably decoupled. Services like RabbitMQ allow for really nice decoupling of the various aspects of application development, while solving a lot of the typical problems for you. Use messaging like this, and build small, purposeful services for executing specific tasks. Don’t be an enterprise architect and end up with a GenericFactoryWorkerApplication. Build the application itself, and fall back on frameworks and other enterprisey stuff if it becomes obvious the requirements are so great than your ability to implement them in light-weight code is not sufficient. In practice you’ll be surprised how seldom this happens.

To this end, a great deal of the code written will be dealing with edge cases, and in my opinion one of the best measures of the maturity of a web-based development effort is how well it handles odd or malformed input. One of the best examples of a situation where this is especially evident is how well a service handles e-mail addresses. RFC 2822 gives the formal definition for what is allowed in an e-mail address, and the brave have even implemented this described standard in a machine-readable fashion, creating a beautiful, legible regular expression:

(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\[(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?|[a-z0-9-]*[a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])

Fortunately most e-mail addresses only exist in a subset of this standard, containing mostly alphanumeric identifiers, with the occasional period thrown in, and many services limit e-mail addresses to this subset. For many e-mail services there’s also a feature that allows for sub-addressing of an e-mail address with a plus or hyphen operator. The most widely used case of this is GMail allowing for a plus sign and any identifier to be appended to your e-mail address. For example if my address was joe.smith@gmail.com I could also use joe.smith+work@gmail.com to direct e-mail to my inbox, and optionally do some filtering based on the To address field, using that tag.

The plus sign is an interesting case that’s often handled poorly. I’ve seen behaviors ranging from rejecting the perfectly valid e-mail address (which is almost justifiable- users aliasing their address with a sub-address are more than most likely trying to create multiple accounts), to allowing the account to be created, but never allowing one to log into the account. My most recent experience with this odd behavior has come from PayPal.

I don’t typically consider PayPal a company that is on top of things technologically speaking, their service is slow with requests often timing out, and their APIs are terrible (see Stripe for an example of payment APIs done right), but I generally expect above all else for their services to be correct in their behavior.

For unknown reasons my previous PayPal account was placed on hold pending confirmation of my address. This confirmation process can only be done via home phone or by shipping some sort of post card to my residence. I don’t really care enough to complete this process, I’d rather just register a new account, almost every bank account and credit card attached to the previous account has expired or been closed, so I decided to take a shot in the dark and try the e-mail subaddressing trick, and swiftly added a ‘+1′ to my e-mail account and registered again. Surprisingly PayPal allowed this sketchy behavior, and merrily sent me on my way. However, when they sent me the activation e-mail things didn’t go quite so well:

It turns out that the activation link contains the e-mail address I registered with as part of the query string. It’s standard practice when parsing a query string to substitute plus signs with spaces, which PayPal does appropriately, and in turn breaks the URL.

Obviously this isn’t a huge security flaw, or really much more than an inconvenience, but it’s a little odd to see a company who’s business is performing secure transactions not even escape a query string correctly.

At 30,000 Feet

I’ll recommend taking a look at the sixth tutorial in the RabbitMQ documentation for a really nice introduction to the mechanics of a RPC request. I’ll shamelessly steal their diagram and present it below:

The sequence looks something like this:

1. The client, upon connecting to the broker server, creates a queue with a server-generated name for receiving the eventual replies to RPCs. This queue name is stored in a variable called responseQueue, and a receiving function is bound to it to handle a response from a RPC. We also create a field called correlationId to store an incrementing counter, which allows us to later match up requests with responses.
2. When a RPC request needs to be made a message is published to a request queue, in the message headers we set the replyTo and correlationId parameters, using the established values above. We store a callback function in a map, keyed with the current correlationId, and also set a timeout timer to ensure that an action is taken if the RPC request isn’t processed in time.
3. A server bound to the requestQueue received the request, does any necessary processing, constructs an appropriate response to be sent to the responseQueue, which has been specified in the replyTo header, and finally acknowledges receipt of the request.
4. Assuming the response is received in time, the client will finally look up the correlationId in the map of stored callbacks, disable the timeout timer, and finally fire off the callback function

Node.js couples really nicely with this model, there is very little impedance mismatch between this ideal model and the typical callback pattern observed in Node, so implementation is really straight forward. Let’s take a look at how this is put together.

Connecting in Node to a RabbitMQ Server

First thing is first, let’s create a connection. I’ll be using the excellent node-amqp library for communication. Setting up a connection to a RabbitMQ broker involves a function with a ready callback:

var responseQueue = null;
var correlationId = 0;
var rpcRequestMap = {};
var connection = null;

function connect() {
    // Setup the connection object, with associated callbacks for connection events.
    connection = amqp.createConnection();

    connection.on('ready', function() {
        console.log('message.connect: ampq connection established');
        // by not specifying a queue name, the server will assign us one randomly
        // by specifying the exclusive option we ensure the queue will be cleaned up
        // upon application exit
        connection.queue('', {exclusive: true}, function(queue) {
            console.log('message.connect: rpc queue created: ' + queue.name);
            queue.subscribe(handleRpcResponse);
            responseQueue = queue;
        });
    });    

    connection.on('error', function() {
        console.log('message.connect: connection error');
    });

}

The node-amqp library provides default connection parameters for a standard RabbitMQ installation. You’ll notice that upon creating the queue we call the subscribe event with a callback function. We’ll come back to that later, first let’s send a request off to an RPC server:

function doRpc(requestQueue, payload, callback) {
    var thisId = correlationId;
    correlationId = correlationId + 1;

    // setup the object in the callback map for
    // storing our callback function and the unique timer id
    // node assigns
    rpcRequestMap[thisId] = {};
    rpcRequestMap[thisId].callback = callback;

    // setTimeout is part of the node core and will return
    // a unique timer handle that we can use to control
    // the created timer. we set up this timer in case the
    // rpc server isn't running or has crashed
    rpcRequestMap[thisId].timer = setTimeout(function() {
        console.log('rpc timeout');
        var fn = rpcRequestMap[thisId].callback;
        delete rpcRequestMap[thisId];
        fn('rpc timeout', null);
    }, 5000);

    client.publish(requestQueue, payload,
        {replyTo: rpcQueue.name, 'correlationId': String(thisId), mandatory: true});

    console.log('message.doRpc: message published, requestMap binding created, id:' +
        String(thisId));
}

This function takes in a set of function arguments in the payload variable, a request queue name, which can be thought of as the RPC function name, and a callback function, which must follow the typical Node pattern of function(err, data). This will send off the request to the broker, which, assuming a RPC server is running, will deliver the message.

Take notice of the implied and explicit queue and message parameters. Out queue is setup with the exclusive parameter defined, ensuring that only this instance of our client can consume messages from it, and ensuring that it will be deleted when the connection is closed. Our messages are sent without persistence flags and with the mandatory flag set. If the RPC server is unavailable it’s unlikely we’d want the message to be processed, so this allows the broker to drop it instead.

Using the official Java API for Rabbit, let’s build the server now:

package fusao.tangifo.backend;

import java.io.IOException;
import com.rabbitmq.client.*;

public class Processor {
    public static void main(String [] args) {
        System.out.println("Processor: initializing");
        // Our main AMQP connection, we'll open
        // a channel per thread later with a threadpool.
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUsername("guest");
        factory.setPassword("guest");
        factory.setHost("localhost");
        factory.setPort(5672);

        // Let's connect and setup our basic queues.
        System.out.println("connecting to AMQP server...");
        Connection conn = null;
        final Channel channel;

        try {
            conn = factory.newConnection();
            channel = conn.createChannel();
        } catch (IOException e) {
            System.out.println("failed to create channel or connect to server");
            e.printStackTrace();
            return;
        }

        System.out.println("connected.");

        try {
            // Setup the queue, if it's not already declared this will
            // create it.
            // durable - false
            // exclusive - false
            // autoDelete - true
            // arguments - none
            channel.queueDeclare("image", false, false, true, null);

            // Add a callback for when messages arrive at the queue
            // autoAck - false
            channel.basicConsume("image", false,
                 new DefaultConsumer(channel) {
                     @Override
                     public void handleDelivery(String consumerTag,
                                                Envelope envelope,
                                                AMQP.BasicProperties properties,
                                                byte[] body)
                         throws IOException
                     {
                         String routingKey = envelope.getRoutingKey();
                         String contentType = properties.getContentType();
                         String correlationId = properties.getCorrelationId();
                         String responseQueue = properties.getReplyTo();
                         long deliveryTag = envelope.getDeliveryTag();

                         String message = new String(body);
                         System.out.println("message received");
                         System.out.println("correlationId: " +
                             correlationId +
                             " responseQueue: " +
                             responseQueue);
                         System.out.println(message);                         

                         AMQP.BasicProperties b = (new AMQP.BasicProperties.Builder())
                             .correlationId(correlationId)
                             .build();

                         channel.basicPublish("", responseQueue, b, "{}".getBytes("UTF-8"));
                         channel.basicAck(deliveryTag, false);
                     }
                 });
        } catch (IOException e) {
            e.printStackTrace();
            System.out.println("Something went horribly wrong.");
            return;
        }
    }
}

The Java implementation isn’t terribly exciting, take a look at where we extract the correlationId and responseQueue from in the DefaultConsumer delivery handler. We publish the message to the default exchange, and just serialize an empty JSON object.

This server will display the parameters to the console, and issue a response. The final piece of the puzzle is the response handler in Node:

var handleRpcResponse = function (message, headers, deliveryInfo) {
    console.log(headers);
    console.log(deliveryInfo);
    if (!deliveryInfo.hasOwnProperty('correlationId') ||
        !rpcRequestMap.hasOwnProperty(deliveryInfo.correlationId) ||
        rpcRequestMap[deliveryInfo.correlationId] === null) {
        console.log('message.handleRpcResponse: stray rpc message received');
        return;
    }
    var thisId = deliveryInfo.correlationId;
    clearTimeout(rpcRequestMap[thisId].timer);
    var cb = rpcRequestMap[thisId].callback;
    delete rpcRequestMap[thisId];
    cb(null, message);
};

We delete the map item once we’ve received the response, call the callback, and log some debug parameters to the console.

So there you have it, this is a really trimmed down version of a RPC pattern for Node.js and Java. I’ve been using it in testing for a few days now with good results to sling JSON requests back and forth between my frontend and backend server. I really like Node.js for building RESTful APIs, and with a solid messaging layer and backend it forms a really nice stack that has a fast response time, and can scale horizontally with a nice decoupling between heavy-lifting backend services, and the web-facing frontend.

Even more alarming is that as the codebase grows in complexity, my confidence in it shrinks. When the code is small, it is easy to maintain. I understand the entire system, and see it for what it is. When I don’t like something about the code, I simply refactor the entire thing, and again I am happy. Once the project grows past a couple thousand lines refactoring gets tougher, and no longer do I feel like I have a good grip on the code. It starts to feel sloppy, and thrown together. I start feeling like there should be a better way, and that my code isn’t up to standard, and won’t be maintainable. In essence, I start to think my code sucks.

At first this really concerned me, if I couldn’t write more than a few thousand lines of code before everything turned into unorganized spaghetti, then I surely was a poor programmer. I started looking through the old code, trying to figure out where I went wrong, and why I felt it was written so poorly.

Taking a look at all of this written code, it turns out that there really wasn’t anything wrong. Sure, there’s minor pieces that could be written differently, with a better trick I picked up only after that module was completed, but there’s nothing terrible about it. No matter where I looked, I couldn’t find the lines of code that I was so sure sucked. This being the case, if the architectural decisions were decent, and implementations were clean then why did I feel so terrible about the project?

I think this answer has to do with what happens when a program exceeds your ability to fit it in your head. There’s a point where you can no longer easily hold your entire program in your head, and it happens pretty early on in a large project. There’s just too much code to keep all of the right assertions and intuitive reasoning patterns needed to fluently work in your head at once, and as a result you have to page out the ones that aren’t immediate to your current task.

This feeling has bothered me for quite some time, I couldn’t understand it before. I think the only solution is to just come to terms with it, and accept that anytime a project grows above a certain threshold you will inevitably feel like there’s some smells coming from already written code. Writing smaller modules, and ensuring things are self-contained with a consistent exception framework has really helped to contain this feeling for me. If you can write code that doesn’t surprise you later on when invoked, and which performs small, obvious functions then you’re doing something right. Focusing on creating abstractions with a point of avoiding leakage and planning upfront for code extension has also significantly helped.

If you feel this way too I wouldn’t worry too much, it only means that you’re pushing the limits of what you’re capable of doing, and creating complex systems that actually solve real world problems! Very few projects are small enough to fit in one mind, so simply focus on always creating tidy code when focusing on a specific part of your application, and making a consistent style for error handling and other generic aspects of your code.

I noticed a lot of services will return a slightly encoded ID such as ch_byLy9Gy1cUkZte, where the prefix will give some sort of indication as to the object type, and the rest is a random looking alphanumeric key. I set out to design something like this, and be able to easily map it back to database identifiers.

Using the OpenSSL-based encryption libraries in Node, I came up with something pretty decent. We encrypt the identifiers using a stored passphrase, along with a prefix, and return it as a base-64 encoded string to the user.

    var encrypt = function(id, prefix) {
        var encrypted = null;
        try {
            var cipher =
                crypto.createCipher('aes-256-cbc', config.obfuscater.key);
            cipher.update(prefix + '_' + String(id), 'ascii', 'base64');
            var cryptString1 = cipher.final('base64').replace(/\=/g, '');
            encrypted = prefix + '_' + cryptString1;
        }
        catch (err) {
            return null;
        }

        return encrypted.replace(/\//g,'.').replace(/\+/g,'-');
    };

There’s a couple strengths to this approach:

• The incrementing column id is not recoverable without knowing the passphrase used to encrypt the data.
• By encrypting the prefix with the id we ensure that one can’t take a generated ID from a different object type and modify it to access another object. For example, a malicious user can’t grab pref1_abc and create pref2_abc, the system would reject it upon decoding and realizing the prefixes don’t match.
• We base-64 encode the data, and replace URL-unsafe parameters, leaving us with a string that can easily be transmitted within a GET request.

When using this approach, we’ll generate IDs like this:

i: 0 eString: ord_KU.cXCev42ulIW3rc78NxQ
i: 1 eString: ord_Ba1REDoYHMDXfaPj-lEB5A
i: 2 eString: ord_zTevdc8HP52J9wIQB-.42A
i: 3 eString: ord_vIpAhysJx0iqOTSztCz0KA
i: 4 eString: ord_xzxCz0Dzkp73buVd2ASPeA
i: 5 eString: ord_GnZM3eLF6El7rLOIywXt2w
i: 6 eString: ord_ZpHin3pOEg0B3d9rmNOJUw
i: 7 eString: ord_UGCaf9zbnQpqwphHjuSCDg
i: 8 eString: ord_i.0dpeDqY5-lW6ZaylQplQ
i: 9 eString: ord_a7X8upiCXDEMwW8Sd1RwhQ

Not too shabby! The end-user has little chance to figuring out the pattern behind the resource locator, and it’s easily decrypted server side.

I’ve been developing a RESTful API with the wonderful node-restify framework and found that error handling is actually quite graceful. There’s a really powerful pattern that emerges when using the callback paradigm that makes handling errors quite simple.

In Node almost everything happens with a callback, and if you value your sanity you’ll not nest these callbacks into a 10 level deep soup, you’ll instead build stacks of tasks, each of which takes in a callback and subsequently executes it upon completion. The really cool thing about these tasks is that they force you to break up your work into small, atomic functions with a clearly defined goal. For example, in my application when a user PUTs a new address there’s a number of small tasks that must happen. The user’s session is validated, authentication details are retrieved from the database and verified, we validate they supplied a valid ID for an address entry, we do the update, and finally send back a generic HTTP-200 response.

At any step something could go wrong, and the request could no longer continue. In the real world things like database errors can happen, as can poor user input, or nonexistent database entries. To support handling all of these errors, each function has the opportunity to set an error code in the req.error field, and drop directly out of the callback stack.

By making the actions reasonably self-contained, and adding a degree of atomicness to operations by lastly doing database updates and setting status fields, we actually have a really nice error handling framework. The callback model lends to a clean abstraction, and to breaking things down in this manor, so it’s almost effortless.

Implementation

I’ve extended the routing engine inside Restify a little bit to support a flexible, stack-based approach to callbacks. I found the default routing engine to be too limiting, it forces us down a predefined path and as a believer in indeterminism I wanted the flexibility to determine the destination of my route as it was being processed, in response to query parameters and database properties. To do this we setup a route inside the server like this:

server.put('/addresses/:id', routes.address.put);

Instead of using the built-in routing, we defer most of the handling to my custom routing engine, where the route is setup like this:

    var routes.address.put = function(req, res, next) {
        req.stack = Array();

        // Standard validation and session stack
        req.stack.push(validator.requireSession);
        req.stack.push(db.session.setup);
        req.stack.push(db.session.updateLastUsed);
        req.stack.push(validator.requireAuthentication);

        // Address specific code
        req.stack.push(validator.requireId);
        req.stack.push(db.address.update);
        req.stack.push(api_responses.session);

        return helper.pop(req, res, next);
    };

Not too shabby! Looks really clean, where’s the error handling even located? One of the requirements of all top-level routing functions is that they can fail, and are atomically clean operations that don’t leave the global application state (database, file storage, etc) in an inconsistent state. I’ve taken care to ensure that each function does something that doesn’t require further processing once completed to return the application to a consistent state. Inside each function, it has the opportunity to set an error code, and then the machinery that links these functions together handles termination of the invokation chain, and printing of the error message. Let’s take a look at one of the most basic versions of this function:

    var validator.requireId = function(req, res, fn) {
        if(!checkPreconditions(req, ['id'])) {
            req.error = new restify.MissingParameterError('Missing parameters.'));
        }
        return fn();
    };

It has access to the typical req and res objects exposed by Express or Restify, and a callback function. This one simply does error checking, but if an error is found it sets up an error in the req object. Upon returning we break into the following function, which was first called at the end of the routing function I defined above.

var pop = function(req, res, next) {
        if(req.hasOwnProperty('error')) {
            console.log(req.error);
            return next(req.error);
        }

        var fn = req.stack.shift();
        if(fn) {
            return fn(req, res, next, function() {
                module.exports.pop(req, res, next);
            });
        }
        next();
    };

A semi-clever function that simply shifts elements off the stack of tasks, runs them, and passes itself as the callback, with a localized scope. The error handling magic happens when we check the existence of req.error at every entry, at which point in time we invoke Restify’s built-in error processor.

Combining this with the node-logging library to give a primitive form of function tracing, and some nice database and validation wrappers I’ve created a really clean error solution. When the database is unavailable, HTTP-500 errors are bubbled up appropriately, and user input validation as well as existence checks are handled in the same manor. It’s a really refreshing break from threaded programming to see such a clean way to handle errors, and while the callback approach can be tedious at times, seeing these kinds of clean abstractions fall out of it so often really suggests that it’s a very appropriate way to write request processing servers, not to mention the performance benefits of the callback concurrency model when faced with IO-bound requests.

Realizing this, and reading about the history and origin of some programming languages in article I found, Research in Programming Languages, I’ve become a little frustrated with these papers. As a good example, half of the languages we use today were designed in a few days by someone outside of the academic world, yet power the majority of the modern web. I’ve read a couple papers about scheduling algorithms, that promise great advances over the current threading mechanisms we use, but despite being published years ago we still use the same, simple threading mechanisms. I just feel like a lot of these system designs are unnecessary and needly complex, for the sake of complexity itself, while not realizing significant performance increases. In this spirit I hereby christen a new adjective to describe some of these systems, in honor of their ridiculous complexity, which is surpassed only by their irrelevancy.