Moving Forward

Recently I became very interested in generating some statistics from a SVN repo. In our research group we have a repository for all the currently in progress papers, which are written in LaTeX, and doing some rudimentary reporting on the number of committed lines by author sounded like a fun way to gamify the process of writing. You can see below one of the highlights of this reporting. As would be expected by a graduate student research lab, a large number of commits happen late in the night, with a large void during working business hours.

I found a great tool to generate some statistics from SVN repos, appropriately called StatSVN. It’s decent out of the box, but lacked some customizability, and automation.

The way it works by default is you invoke it as shown below, and it uses a generated output file from SVN, along with the path to a checked out local repo, to generate a pile of HTML reports and figures tallying various commit statistics. It automatically invokes subversion, and requests the diffs between commits, storing data in a local cache file.

java -jar statsvn.jar papers/logfile.log papers -include "**/*.tex" -config-file config.txt

This works pretty well, but to really create some fun statistics we need to work a little harder. I wanted to filter out some of the larger bulk-commits that don’t accurately reflect actual work, and I wanted to customize the generated report. Naturally I fired up vim and started writing some Python…

Filtering Out Certain Revisions

The first problem was that this repository is pretty new, and a lot of the first commits involved setting up templates and doing other administrative tasks. I want to collect statistics on who produced the most content, not who can push the metaphorical broom hardest in cleaning up templates and moving directories around, so I needed a method to filter out certain commits. The way StatSVN works is by first parsing an exported svn log file, containing a list of commits. What I found is that by simply removing the associated log entry for a commit StatSVN will simply ignore it.

A Sample Log Entry from the SVN Log

<logentry revision="172">
<author>androbin</author>
<date>2012-02-15T19:06:10.225746Z</date>
<paths>
<path kind="file" action="M">/papers/mobicom12-audio/tex/design.tex</path>
</paths>
<msg>Fixed broken paper by updating design.tex</msg>
</logentry>

Python Code to Perform an Update and Generate the Log

print 'Updating SVN repo'
os.system('cd papers; svn up')

print 'Running XML export from SVN repo'
os.system('cd papers; svn log -v --xml > logfile.log')

Before removing it, we update the repository, which I’ve checked out into a directory called papers/, and generate a fresh log file. Next using lxml we load the log file, and an exclude list, and perform the deletion.

Removing Revisions from Statistics based on Number

listToExclude = []
with open('exclude-list.txt', 'r') as f:
    listToExclude = map(lambda x: x.strip(), f.readlines())

print 'Exclude list: ' ,
print listToExclude 

doc = le.parse('papers/logfile.log')
elementsToRemove = []
for pat in listToExclude:
    for elt in doc.findall('logentry[@revision=\'' + pat + '\']'):
        print 'Removing element...'
        elt.getparent().remove(elt)

print 'Writing fille back to disk...'
with open('papers/logfile.log', 'w') as f:
    f.write(le.tostring(doc))

exclude-list.txt simply consists of revision numbers, separate by newlines.

After we’ve modified the logfile we invoke the statistics generation program manually.

Invoking StatSVN

print 'Invoking graph generation software...'
os.system('java -jar statsvn.jar papers/logfile.log papers -include "**/*.tex" -config-file config.txt')

Of interest here is the fact that we’ve passed it a configuration file. I’ve identified three key graphs I’d like to include in my final repo, and resizing them to appropriately fit in the spaces I’ve allocated for them is a little challenging, so I’ve used StatSVN’s ability to specify a config file to resize them and pump up the plot lineStroke to be a little more readable.

StatSVN Config File

chart.loc_per_author.lineStroke=4
chart.loc_per_author.width=600
chart.loc_per_author.height=300

chart.activity_time.width=600
chart.activity_day.width=600
chart.activity_time.height=370
chart.activity_day.height=408

Making an Aggregate Report

So now I’ve filtered out all the commits I don’t care about, but I’m not that happy with the default reports. My goal is to load these stats on a display-case monitor, and none of the default reports are attractive enough, or contain the right information, to make the cut. The approach I decided to take here was to use BeautifulSoup to extract the information I wanted from each of the reports, and then composite it into one report using a template file. This works really well in practice, since the report software’s format won’t change BeautifulSoup has no problems selecting the elements of interest.

HTML Template for the Final Report

<html>
<head>
<title>Group Dangerzone Paper Log</title>
<link rel="stylesheet" href="ocss.css" type="text/css">
</head>

<body>
<h1>Dangerzone Paper Commit Log</h1>
<table width="100%">
<tr>
    <td valign="top" width="70%">
    <table width="100%">
        <tr>
            <td valign="top">[A]</td>
            <td align="right"><img src="loc_per_author.png" /></td>
        </tr>
    </table>
    <br><br><br>
    <table width="100%">
        <tr>
            <td valign="top"><img src="activity_time.png" /></td>
            <td><img src="activity_day.png" /></td>
        </tr>
    </table>
    <h2>Commit Message Tag Cloud</h2>
    [T]
    </td>
    <td>
        [C]
    </td>
</tr>
</table>
</body>
</html>

In the template shown above we use placeholders [C], [T], and [A] for the commit log, tag cloud, and list of author contribution by percentage respectively. Below the python script will extract those elements from the generated reports, and push them into the template, before writing it to output.html.

Making a Pretty Report

print 'Generating output HTML...'

def getSoup(fileName):
    with open(filename, 'r') as f:
        return BeautifulSoup(f.read())

template = ''
with open('template.html', 'r') as f:
    template = f.read()

developers = getSoup('developers.html')
index = getSoup('index.html')
clog = getSoup('commitlog.html')

authorTable = developers.html.body.table
template = template.replace('[A]', str(authorTable))

tagCloud = index.html.body.findAll('div')[2].p
template = template.replace('[T]', str(tagCloud))

commitList = clog.html.body.findAll('dl')[1]
for i in range(24,len(commitList.contents)):
    commitList.contents[len(commitList.contents) - 1].extract()
template = template.replace('[C]', str(commitList))

with open('output.html', 'w') as f:
    f.write(template)

The End Result

This whole script is saved in a file, set to run with a cron job every half-hour, and a line is added to the template file to cause the browser to refresh the page every so often. The finished product is shown below.

I use a Macbook Pro for my day to day tasks, but quite frequently find myself needing a Linux environment to compile and test code. Like many others I use VMware Fusion to run a virtual machine, booting the latest version of Ubuntu. When I started this practice I was worried about the battery life of my Macbook decreasing significantly, so I started looking for some hacks to make things a little more efficient, and just to make my life a little easier.

Disable X.org and Use SSH Exclusively

This step netted me the biggest single improvement in battery life. Since I almost exclusively SSH into my VM it doesn’t make sense to keep the login prompt running. To kill the desktop manager, light in new versions of Ubuntu, simply adding ‘manual’ to the end of the upstart config file will work. Upstart is a relatively recent innovation in the world of Linux, replacing the traditional rc.d startup procedure and properly parallelizing the startup of independent services in Linux.

echo manual >> /etc/init/lightdm

Assign a Static NAT IP Address

For network settings I’ve found that the best possible configuration is using the NAT routing, and statically assigning an IP address. Many corporate networks (and coffee shops!) won’t allow your computer to obtain two DHCP leases, so running NAT is the easiest solution to keep things working no matter where you are.

To tell the VMWare DHCP server to assign a static address we modify /Library/Application Support/VMware Fusion/vmnet8/dhcpd.conf.

Add a line corresponding to the name of your VM to statically assign an address. In this case my VM is simply named Ubuntu.

# Assign a static IP to ubuntu VM
host Ubuntu {
	hardware ethernet 00:0c:29:49:d1:2c;
	fixed-address 192.168.39.5;
}

You’ll also need to record the MAC address given to your VM, easily available using ipconfig. After setting up a static IP address I add an entry to my /etc/hosts file to give my VM an easy hostname for SSHing and testing network services.

192.168.39.5    vm-ubuntu

Setup Key-Based SSH Authenciation

This one is mostly for convenience, but I’d recommend setting up public-key login for SSH. It’s tedious to have to repeatedly type your passphrase when logging into your VM. This is really super easy, so I’d recommend doing it.

Generate a DSA keypair if you haven’t already (make SURE you use a password)

ssh-keygen -t dsa

Copy the DSA public key to your Ubuntu Machine

scp ~/.ssh/id_dsa.pub vm-ubuntu:.ssh/authorized_keys2

This guide has more details, setting up SSH keys has been covered a million times, so I won’t elaborate much further.

Kill the VMwave Window Once Booted Up

Previous versions of VMwave Fusion supported this as an additional menu item that could be enabled, but with later versions the feature has been removed. Once your Ubuntu VM is up and running there’s not much of a reason to keep the window in your dock, so I’d recommend killing the viewer process.

The easiest way to do this is Control+Option right click on the VMware icon in your dock and goto Force Quit. The VM will happily continue running in the background, even without the window.

At this point to further conserve battery life, I’d recommend installing gfxCardStatus and forcing your Macbook into integrated mode, unless you need discrete mode for 3D graphics and rendering. The vmx-server process will force it into discrete mode, as will a number of other proceses (I’m looking at you, Google Chrome), and forcing integrated mode will easily net another 20-30 minutes of runtime.

Following these steps, I can achieve decent battery life with my Macbook Pro. Typically when I’m just reading books in Kindle Reader, or doing light web browsing I’ll see 4-5 hour battery life, with it quickly dropping down to 2-3 hours once I start doing heavy work again such as compiling and working on test cases.

It’s that time of year again. At the beginning of every year I like to set a few goals for myself in the programming world, some would call them new years resolutions. Last year I set out to read a few of the classical books in computer science such as The Mythical Man Month by Fred Brooks, Code Complete by Steve McConnell, and Showstopper, the book about David Cutler and the Windows NT kernel development saga. I also wanted to further my understanding of algorithms by finishing up the latter chapters of my favorite algorithms book, known to many as the CLR. Finally I wanted to get a firm handle on a new language or two.

Luckily I managed to accomplish all these goals and more during 2011. I picked up some really solid experience in both Python and Java during the year, implementing some large projects using both languages, read a lot of core computer science books, and wrote a lot of code. I ventured into solving non-trivial problems with the fundmental concepts behind current AI research, and learned a fair deal about networking by exploring the 6LoWPAN stack.

With all that under my belt, I’d like to outline my list of goals for the new year.

• Learn and Practice Common Design Patterns – I want to ensure my code has a structure to it that makes it easy to reuse, extend, and understand. Design patterns represent solutions that have withstood the test of time, and are a familiar language to all developers. While I’m familiar with the usage of common patterns such as factories and listeners, I’d like to expand outwards and practice the implementation of more diverse patterns.
• Become Competent in Functional Programming – Many believe that in order to effective make use of parallel computing architectures we will need to rely more and more on functional programming. This style of programming also has benefits when it comes to proving correctness, and has claimed to reduce the number of lines of code required to achieve a solution. Towards the end of the year I started working with Haskell, I’d like to implement at least one large project in a functional language this year, and examine how they can be applied to embedded systems, where most operations are typically IO related and many resource constraints exist.
• Further my Understanding of Compilers – Fundamental to programming are the tools we use to translate code into machine instructions. I’m planning on developing my understanding of compilers further during the year
• Publish More Often – While I produced a fair amount of content in 2011, I’d like to ensure that I give even more back. There’s so much in the world of computer science to write about and discover. In 2012 I hope to finish off more write-ups and go the last mile often required to turn exploratory research into publications- both blog posts and research papers

It works! The system is reasonably secure, and certainly fun to use.

Be sure to check out part 1 and 2 to see details of how the system was built:
Part One – The Hardware
Part Two – The Software

Future improvements could include a better GUI and more integration with other services in our lab. The system is written in such a way that it’s reasonably modular, so doing something like tacking a Twitter account on it would be trivial at this point.

Continuing our long journey, the next step in hacking the Keurig B40 is writing the software that will drive this secure coffee maker of the future. I’ll start with the Arduino code, and build our way up the stack.

Writing the Arduino code required overcoming a few annoyances. I needed to simulate a user-button press, which is an event that typically lasts for a few hundred milliseconds, and detect if the system was ready to brew, which isn’t indicated by a single LED, but rather the blinking of an LED. These are pretty simple challenges to overcome.

Sending Compact Status Data

The Arduino code uses a bit-struct to pack the values, which will be unpacked by Python on the host side of things. Here’s what that bit structure looks like:

union send_t {
  struct {
   byte  isOn : 1;
   byte  isEightOzButtonPressed : 1;
   byte  isTenOzButtonPressed : 1;
   byte  isWarmedUp : 1;
   byte  isNeedWater : 1;
   byte  isReadyToBrew : 1;
  };
  byte value;
};

This is one of my favorite bit tricks in C. Unions can be used all over the place to avoid unnecessary casting. In this case, I’ve use a bit-packed anonymous struct, which are really useful when writing types to set bit-masks, and unioned it with a byte, to allow us to get and set the entire raw value at once. While we could of used bit-shifting to get the same result, this has the advantage of being 100% readable.

Reading Coffee Maker Status

Next we create a function to handle reading the inputs, and outputting this data over the UART channel:

send_t toSend;

int outputCounter = 0;

// Counts toggles, > 3 = READY TO BREW.
int toggleCounter = 0;

// We could use outputCounter to keep track of this value,
// but breaking it out into a separate variable makes it
// a little clearer what exactly is going on.
int toggleTimeCounter = 0;

// The last value, used to determine toggle.
byte lastValue = 0;

void handleOutput() {
  // We want to catch these events no matter what, so we simply
  // AND them with every poll, that way if a button press was detected
  // at any point it's recorded.
  toSend.isEightOzButtonPressed =
    toSend.isEightOzButtonPressed | ((PIND >> 4) & 1);
  toSend.isTenOzButtonPressed =
    toSend.isTenOzButtonPressed | ((PIND >> 5) & 1);

  // We've detected a toggle.
  if(lastValue != ((PIND >> 6) & 1)) {
    // If it's been 500ms - 550ms it's probably the light blinking
    // to symbolize that brewing is ready.
    toggleCounter++;
    toggleTimeCounter = 0;
  }

  // If we've been waiting more than 600ms for a toggle let's give up.
  if(toggleTimeCounter > 600) {
     toggleCounter = 0;
  }

  lastValue = ((PIND >> 6) & 1);

  if(outputCounter % 500 == 0) {
    toSend.isOn = ((PIND >> 3) & 1) ^ 1;
    toSend.isNeedWater = ((PIND >> 7) & 1) ^ 1;
    if(toggleCounter > 3) {
      toSend.isReadyToBrew = 1;
    }

    toSend.isWarmedUp = (lastValue == 0) || toSend.isReadyToBrew || toggleCounter > 0;
    Serial.write(toSend.value);
    toSend.value = 0;
  }
  outputCounter++;
  toggleTimeCounter++;
}

I’ve gone with an event-loop style loop here, so everything happens on counter intervals. In this block we measure the time between transitions of the brew button, read the state of the buttons on the front to record any user actions, and finally read the other fairly static parameters and send them along the wire.

I read user input in a sticky manor, every 500ms a UART packet is generated and if the user pressed one of the brew buttons at any point in time it will record it for that frame.

In this code I don’t use the digitalRead and Write functions. I don’t feel that the thin wrappers the Arduino code base provides are really that useful. Having direct access to the ports on the AVR really isn’t a grand leap.

Handling Inbound Commands

Left to do is handle commands sent from the host pc to the Arduino. This is also a pretty simple process.

unsigned int commandCounter = 0;

void handleInput() {
  // Here we handle button presses-
  // We simulate a ~200ms press asyncronously
  // to simulate user input. 

  // If commandCounter == 0 we look for UART data,
  // else we block on this specific command and increment
  // up to 200 (~200ms)

  // If a lot of commands were sent at once this would
  // potentially overflow and drop some commands, but
  // I don't think that's a realistic situation.
  if(commandCounter > 0) {
    commandCounter++;
    if(commandCounter == 200) {
      // Set PORTB into high-z state, since the buttons
      // use an analog resistor network we don't want
      // stray current messing with the PIC on the coffee
      // maker
      DDRB = 0;
      // Disable internal pullup resistors
      PORTB = 0;
      // Reset commandCounter
      commandCounter = 0;
    }
  } else if(Serial.available() > 0) {
    switch(Serial.read()) {
      case 'p':
        // Press the power button (Pin 
        PORTB = (1<<0);
        DDRB = (1<<0);
        commandCounter = 1;
      break;
      case 't':
        // Press the 10oz brew button (Pin 9)
        PORTB = (1<<1);
        DDRB = (1<<1);
        commandCounter = 1;
      break;
      case 'e':
        // Press the 8ox brew button (Pin 10)
        PORTB = (1<<2);
        DDRB = (1<<2);
        commandCounter = 1;
      break;
    }
  }
}

We read bytes as they come down the wire, relying on the serial buffer to hold any pending button press commands while the current ones are executing. The code will toggle the button for approximately 200ms.

We’ve designated three commands, ‘p’ to power the unit on or off, ‘t’ to perform a 10oz brew, and ‘e’ to perform a 8oz brew. The commands are single ASCII characters so no delimiting or framing was required.

Of interest is the fact that we deliberately keep all the buttons in a high impedance state. Because the Keurig reads buttons using a resistive network, pushing or pulling stray current into it could easily cause the coffee maker to fail to register button presses. We do this by setting the pins to inputs when they are not being actively driven, and disabling the internal pull ups.

Finally we have the often-found Arduino sketch components, loop and main:

void setup() {
  Serial.begin(9600);
}

void loop() {
  handleInput();
  handleOutput();
  delay(1);
}

That covers the Arduino code, next we hook this up to a Python interface to handle reading status, writing commands, taking a photo of the user, and handling RFID events.

Reading data from the RFID Reader and Coffee Maker

Reading serial data in python is actually surprisingly easy, with the pySerial module it really is as simple as it can get. First we open up two serial ports simultaneously:

rfidSer = serial.Serial('/dev/ttyUSB0', 9600, timeout=1)
coffeeSer = serial.Serial('/dev/ttyUSB1', 9600, timeout=1)

Next we parse data from both the ports.

For the RFID Reader:

	if rfidSer.inWaiting() > 1:
		data = rfidSer.readline().strip()[1:-2]
		if data in authorizedUsers:
			handleUserAuth(authorizedUsers[data])
		else:
			handleAccessDenied()
		time.sleep(1)
		rfidSer.flushInput()

I wait a second and flush the buffer after reading the line. The reader uses ASCII data but puts some start text/stop text characters at the beginning and end that python doesn’t parse naturally. Just clearing the buffer is sufficient to get rid of them.

For reference the RFID reader is the ID-12 sold by Sparkfun, available here.

For the coffee maker

	if coffeeSer.inWaiting() > 0:
		# Read the new status, detect differences, and fire
		# events accordingly.
		incomingByte = ord(coffeeSer.read())
		newCoffeeMakerStatus = {
			'isOn' : ((incomingByte >> 0) & 1) == 1,
			'isEightOzButtonPressed' : ((incomingByte >> 1) & 1) == 1,
			'isTenOzButtonPressed' : ((incomingByte >> 2) & 1) == 1,
			'isWarmedUp' : ((incomingByte >> 3) & 1) == 1,
			'isNeedWater' : ((incomingByte >> 4) & 1) == 1,
			'isReadyToBrew' : ((incomingByte >> 5) & 1) == 1
		}

This simply looks like a reversal of the C++ Arduino code I wrote earlier. I use a Python dict to keep track of the coffee maker state, and then test for differences after receiving each byte.

Taking a Photo

Last, but certainly not least, I wanted the python script to take a photo of the coffee user. Luckily there’s a command line tool available. We simply take a photo, save it to a directory, and append it to a running HTML document.

def takePhoto():
	fileName = '-'.join((str(time.time()), currentUser))
	os.system('streamer -q -o cphotos/%s.jpeg &> /dev/null' % fileName)
	f = open('userListAppend.html', 'w')
	f.write("""<table><tr><td align="center"><img width="330" src='cphotos/""" + fileName + """.jpeg'></td></tr><tr><td><b>User: </b> """ + currentUser + """<br><b>Date:</b> """ + str(datetime.datetime.now()) + """</td></tr></table>""")
	f.close()
	os.system('cat userList.html >>userListAppend.html & cp userListAppend.html userList.html')

That’s pretty much it for the software section of this hack. There’s some glue code here and there I can make available upon request, but all the hard parts are laid out here.

In my implementation I went ahead and integrated our coffee maker with an existing system designed to track sensor events. The coffee maker posts to our event server, and the events are displayed on a large screen outside our lab.

The Keurig B40 coffee maker is a masterpiece of engineering marvel. Inside you will find 2 pumps, 3 solenoids, 4 circuit boards, and an exorbitant number of tubes, sensors, and connectors. As a weekend project I’ve taken one of these beasts apart, documented it, and subsequently modified it to solve the complete lack of physical security and auditing, which I consider a serious flaw in the original design. K-cups are a precious commodity, with costs far exceeding that of traditional bulk-purchased coffee, and their usage must be carefully monitored to control costs. Previously we had no way to attribute usage to individuals as to ensure proportional, fair contributions were made to the coffee fund that accurately reflected individual consumption. In this day and age it is frightening to imagine that such a common device lacks the proper physical security interfaces required to properly solve this problem.

To rectify this situation I added an Arduino hardware interface and RFID scanner to track usage of the machine. We had previously distributed RFID devices to all lab members as part of another project to create a keyless entry system, so using RFID was the natural choice. The original brewing interface (push button on the machine) has been secured by being disconnected, and attached to the Arduino, where input is received and processed by the netbook, which performs authentication, auditing, and authorizes brewing.

Disassembly

WARNING: Proceed at your own risk. A coffee maker, by definition, mixes water and electricity, which has generally been considered a bad thing. This coffee maker, in addition to making delicious, bold blends, could easily kill you or permanently damage whatever you might connect to it.
Read the rest of this entry »

After reading this article on installing Speex for Android I realized that it was severely lacking. The author never presents the JNI interface, and some of the ndk-build flags are incorrect for my platform.

Here’s a set of steps to setup Speex for Android, and building a simple wrapper, assuming you have the Android NDK installed. If you don’t have the NDK installed, download it from the Android homepage and add the bin directory to your path statement.

1. Download the latest Speex source tarball from the downloads page.
2. Extract the files and copy the include and libspeex directories into your JNI directory in a current Android project.
3. Create a file called native.c and add the following code listing to it, this is the missing JNI interface:

   #include <jni.h>
   #include "speex/speex.h"
   
   #define FRAME_SIZE 320
   
   int nbBytes;
   /*Holds the state of the encoder*/
   void *state;
   /*Holds bits so they can be read and written to by the Speex routines*/
   SpeexBits bits;
   int i, tmp;
   
   void Java_fusao_awesome_TestAppActivity_init(JNIEnv * env, jobject jobj) {
      /*Create a new encoder state in narrowband mode*/
      state = speex_encoder_init(&speex_wb_mode);
   
      /*Set the quality to 8*/
      tmp=8;
      speex_encoder_ctl(state, SPEEX_SET_QUALITY, &tmp);
   
      /*Initialization of the structure that holds the bits*/
      speex_bits_init(&bits);
   }
   
   jbyteArray Java_fusao_awesome_TestAppActivity_encode(JNIEnv * env, jobject jobj, jshortArray inputData) {
   		jbyteArray ret;
   
   		jshort * inputArrayElements = (*env)->GetShortArrayElements(env, inputData, 0);
   
   		/*Flush all the bits in the struct so we can encode a new frame*/
   		speex_bits_reset(&bits);
   
   		/*Encode the frame*/
   		speex_encode_int(state, inputArrayElements, &bits);
   		/*Copy the bits to an array of char that can be written*/
   		nbBytes = speex_bits_nbytes(&bits);
   
   		ret = (jbyteArray) ((*env)->NewByteArray(env, nbBytes));
   		jbyte * arrayElements = (*env)->GetByteArrayElements(env, ret, 0);
   
   		speex_bits_write(&bits, arrayElements, nbBytes);
   
   		(*env)->ReleaseShortArrayElements(env, inputData, inputArrayElements, JNI_ABORT);
   		(*env)->ReleaseByteArrayElements(env, ret, arrayElements, 0);
   		return ret;
   }
   

   Notice that the function names need to be modified to fit your project. In this case I was working in the fusao.awesome namespace, building an activity called TestAppActivity, modify your code to match what you’re doing.

   JNI is not an Android-specific technology, there are plenty of JavaDoc and Oracle resources available for writing the JNI interfaces. At first it’s not intuitive, but the documentation should help you to write your own interfaces. I’ve listed some additional resources at the end of this post for further customization.

   I’ve implemented a really simple encode function, and initializer. The encode function simply takes a frame (20ms at 16kHz) of audio, and encodes it using the default wideband options. It should easily be modified to accommodate additional functions, such as preprocessing to remove non-voice data.

4. Create a file called Android.mk, to handle the build configuration. Paste in the following configuration:

   
   LOCAL_PATH := $(call my-dir)
   
   include $(CLEAR_VARS)
   
   LOCAL_MODULE    := libspeex
   LOCAL_CFLAGS = -DFIXED_POINT -DUSE_KISS_FFT -DEXPORT="" -UHAVE_CONFIG_H
   LOCAL_C_INCLUDES := $(LOCAL_PATH)/include
   
   LOCAL_SRC_FILES :=  \
   ./libspeex/bits.c \
   ./libspeex/buffer.c \
   ./libspeex/cb_search.c \
   ./libspeex/exc_10_16_table.c \
   ./libspeex/exc_10_32_table.c \
   ./libspeex/exc_20_32_table.c \
   ./libspeex/exc_5_256_table.c \
   ./libspeex/exc_5_64_table.c \
   ./libspeex/exc_8_128_table.c \
   ./libspeex/fftwrap.c \
   ./libspeex/filterbank.c \
   ./libspeex/filters.c \
   ./libspeex/gain_table.c \
   ./libspeex/gain_table_lbr.c \
   ./libspeex/hexc_10_32_table.c \
   ./libspeex/hexc_table.c \
   ./libspeex/high_lsp_tables.c \
   ./libspeex/jitter.c \
   ./libspeex/kiss_fft.c \
   ./libspeex/kiss_fftr.c \
   ./libspeex/lpc.c \
   ./libspeex/lsp.c \
   ./libspeex/lsp_tables_nb.c \
   ./libspeex/ltp.c \
   ./libspeex/mdf.c \
   ./libspeex/modes.c \
   ./libspeex/modes_wb.c \
   ./libspeex/nb_celp.c \
   ./libspeex/preprocess.c \
   ./libspeex/quant_lsp.c \
   ./libspeex/resample.c \
   ./libspeex/sb_celp.c \
   ./libspeex/scal.c \
   ./libspeex/smallft.c \
   ./libspeex/speex.c \
   ./libspeex/speex_callbacks.c \
   ./libspeex/speex_header.c \
   ./libspeex/stereo.c \
   ./libspeex/vbr.c \
   ./libspeex/vq.c \
   ./libspeex/window.c \
   ./native.c
   
   include $(BUILD_SHARED_LIBRARY)
   

5. Open up jni/include/speex/speex_config_types.h (create it if not already present) and add the following bit:

   #ifndef __SPEEX_TYPES_H__
   #define __SPEEX_TYPES_H__
   typedef short spx_int16_t;
   typedef unsigned short spx_uint16_t;
   typedef int spx_int32_t;
   typedef unsigned int spx_uint32_t;
   #endif
   

6. In a command line of the main directory of your project run ndk-build, ensure that it completes successfully.
7. In the activity you’re creating add the following framework inside the class to tell Java what the function definition looks like for your native interface:

   	native byte[] encode(short [] inputData);
   	native void init();
   
   	static {
   		System.loadLibrary("speex");
   	}
   

8. Ensure you build your project again, now you should be able to envoke your JNI functions:

       	short [] inputArray = new short[320];
           // Write data to inputArray.
   	byte [] encodedBuffer = encode(inputArray);
   

And that’s all there is to it!

Additional Resources
JavaDocs on Array JNI Function Specifications
A JNI Example from Oracle
A nice tutorial on building a default JNI applicatoin

Next time I’ll be presenting a complete code listening that demonstrates taking this interface, wrapping it with Google ProtoBuffers, sending it over a socket, and reconstructing the Speex packets on a server to play back on a personal computer!

I’m working on a JNI wrapper for the popular Speex codec for VoIP recording as part of a project to create a continuously available background audio stream. Setting up the Android audio interface can be a little unintuitive and I couldn’t find a decent example that demonstrated this in a concise fashion, so without further ado:

import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;

import android.app.Activity;
import android.media.AudioFormat;
import android.media.AudioRecord;
import android.media.AudioRecord.OnRecordPositionUpdateListener;
import android.media.MediaRecorder;
import android.os.Bundle;
import android.os.Environment;
import android.util.Log;
import android.content.Context;

public class TestAppActivity extends Activity {

    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
        Log.i("Main", "Starting main audio capture loop...");

        // Capture mono data at 16kHz
        int frequency = 16000;
        int channelConfiguration = AudioFormat.CHANNEL_CONFIGURATION_MONO;
        int audioEncoding = AudioFormat.ENCODING_PCM_16BIT;

        // The minimal buffer size CANNOT be merely 20ms of data, it must be
        // at least 1024 bytes in this case, this is most likely due to a MMIO
        // hardware limit.
        final int bufferSize = AudioRecord.getMinBufferSize(frequency, channelConfiguration, audioEncoding);

        // Setup the audio recording machinery
        AudioRecord audioRecord = new AudioRecord(MediaRecorder.AudioSource.VOICE_RECOGNITION,
        frequency, channelConfiguration,
        audioEncoding, bufferSize);

        // The short and file buffers, this might not be the most
        // efficient way to do things, but since we're planning on
        // redirecting this data into an encoder in a later version
        // of this project, we're not worried about it.

        // 320 = 16kHz * 20ms - Number of frames of audio required.
        short[] buffer = new short[320];
        byte[] fileBuffer = new byte[320 * 2];
        audioRecord.startRecording();

		FileOutputStream f = null;
        File sdCard = Environment.getExternalStorageDirectory();
        File dir = new File(sdCard.getAbsolutePath() + "/audioTest");
        dir.mkdirs();
        File file = new File(dir, "testAudio.wav");
        try {
			f = new FileOutputStream(file, true);
		} catch (FileNotFoundException e) {
			e.printStackTrace();
		}

        // Blocking loop uses about 40% of the CPU to do this.
        int sampleNumber = 0;

        // We'll capture 3000 samples of 20ms each,
        // giving us 60 seconds of audio data.
        while(sampleNumber < 3000) {
        	audioRecord.read(buffer, 0, 320);

			for(int i = 0; i < buffer.length; i++) {
				fileBuffer[i*2] = (byte)(buffer[i] & (short)0xFF);
				fileBuffer[i*2 + 1] = (byte)(buffer[i] >> 8);
			}

			try {
				f.write(fileBuffer);
			} catch (IOException e) {
				e.printStackTrace();
			}

			sampleNumber++;
        }

		try {
			f.close();
		} catch (IOException e) {
			e.printStackTrace();
		}

    }
}

When initially writing this code I attempted to use the periodic callback features in the AudioRecord class with little success. Others online have reported mixed results, with the consensus being that capturing audio data in a blocking loop in a separate thread was sufficient. and occurred no additional CPU usage.

The code is setup to receive data in 20ms chunks, which will eventually be fed into the native Speex encoder for sending across the network.

The project I’m working on is power-sensitive, so I did some benchmarking to determine how the buffer size affects CPU usage, which is directly correlated to power consumption.

Interestingly enough CPU usage can be lowered by extending the buffer and periodically writing the samples, the summarized data is listed below:

The take away is that for applications where background processing is occurring, and the results do not need to be near-instantaneous, a much better power consumption profile can be achieved by using a larger buffer.

I still have some questions about the hardware interface to the audio driver. It would be really interesting to know if it’s a memory mapped system, that triggers an interrupt when a certain buffer size is achieved, or if the CPU busy-waits on the audio device and fills a buffer. Without knowing the answer to this question I think it’s tough to create really effective audio listeners. As always, with API abstraction it’s hard to nail down what the most efficient implementation would look like without knowing the details underneath the hood.

DigiKey has done a wonderful thing for those of us who tinker with electronics. It allows for us to purchase small quantities of literally millions of different parts. They have an extremely comprehensive database full of almost every electronic part in production today, however what they are lacking is a proper API.

To this end I have created a simple wrapper around the DigiKey HTML interface suitable for scraping pricing and stock levels for items, based on DigiKey part number. I’ve posted this code in a GitHub repo here.

For an example on how to use this API see below:

using DigikeyApi;

Product p = Scraper.scrapePart("PART NUMBER HERE-ND");
if (p != null) {
    Console.WriteLine("{0}- {1} in stock @ ${2} for 100.",
                    p.partNumber, p.quantityAvailiable, p.calculatePrice(100));
}
else {
    errorParts.Add(part);
    Console.WriteLine("Part lookup failed.");
}

I’ve dove head-first into Python lately for part of a new natural language processing project. Part of this project involves collecting tweets and inserting them into the database for later analysis. To accomplish this goal I set out to find a good API already written. I found Tweepy on GitHub and it seemed to do the trick. The problem is most of the code was written before OAuth became a requirement, and while they supported accessing the Twitter stream, there was no solid example. To that end I’ve examined the source and written an example that does just that.

Ensure you clone or download the code directly from the Git repo. The current stable release does not include support for OAuth in the stream module.

OAuth Authentication

The first step in using OAuth is to obtain your API keys. They can be obtained on the Twitter App Developers site. You’ll need both a consumers key, and an access token, with their respective secrets to successfully communicate. Once you have downloaded Tweepy and obtained the keys you can start a new python script and create an instance of the api module.

import tweepy

auth1 = tweepy.auth.OAuthHandler('CONSUMER KEY','CONSUMER SECRET')
auth1.set_access_token('ACCESS TOKEN','ACCESS TOKEN SECRET')
api = tweepy.API(auth1)

At this point you have a fully authenticated API module. To test things out you can post a tweet to your account.

api.update_status('This is a test!')

If all is good, you’ll see this tweet appear when you visit your homepage on Twitter.

Creating a Stream Listener

The Twitter stream operates by holding open an HTTP connection and continuously sending JSON packets across it containing a structure that represents the most recent tweets. This stream is consumed by the Tweepy module asynchronously and is acted upon by a callback class implementation called StreamListener. In order to process tweets we’ll have to implement this class. The following example implements the on_status method, and simply inserts the tweet into a database. For simple data collection purposes this should be adequate. Additionally we’ll display the tweet on the screen using the TextWrapper class for debugging and observational reasons.

class StreamListener(tweepy.StreamListener):
    status_wrapper = TextWrapper(width=60, initial_indent='    ', subsequent_indent='    ')
    conn = mdb.connect('localhost', 'dbUser','dbPass','dbBase')
    def on_status(self, status):
        try:
            cursor = self.conn.cursor()
            cursor.execute('INSERT INTO tweets (text, date) VALUES (%s, NOW())' ,(status.text))
            print self.status_wrapper.fill(status.text)
            print '\n %s  %s  via %s\n' % (status.author.screen_name, status.created_at, status.source)
        except Exception, e:
            # Catch any unicode errors while printing to console
            # and just ignore them to avoid breaking application.
            pass

Tying it all together

Finally we use the stream API and start capturing tweets. With all the framework put in place it’s a simple matter of setting a list of keywords to filter with and calling the filter method of stream.

l = StreamListener()
streamer = tweepy.Stream(auth=auth1, listener=l, timeout=3000000000 )
setTerms = ['hello', 'goodbye', 'goodnight', 'good morning']
streamer.filter(None,setTerms)

Alternatively to get a sample of all incoming tweets (something like 1% of total) you can use the streamer.sample() method. The filter method accepts, with default API permission levels, up to 400 keywords to filter on. The first parameter accepts a list of interesting people to follow. Using both parameters together results in an OR of the terms.

After running your stream listener for a few days you’ll have more than enough data to do some natural language processing on the data using the nltk!