Moving Forward

In our lab we have a very nice Tektronix scope. It has many fancy dials and more features than you can shake a stick at. It is truly a great piece of technology, and makes our lives much easier when we’re working on analyzing and troubleshooting circuits.

Unfortunately, and I’ve found this holds true for almost every piece of lab equipment I’ve used, when it comes to actually collecting data from the device for plotting in paper-quality graphs everything falls to pieces. What was previously your 4-channel best friend, now becomes your mortal enemy, and you must do battle with this device to achieve great research success.

One of the biggest struggles I’ve had as I start my academic career has been creating an efficient pipeline for data collection and plotting. As a computer scientist, it’s in my nature to avoid solving the specific problem at hand, and instead spend my time trying to solve the general class of problems that it belongs to. To that end I’ve started to amass a small collection of data processing tools that I’ve found useful, and some thoughts I’ve had on what works well, and what doesn’t work so well, that I will share with you as we embark down this path. I’ll do this in a couple different articles, as to avoid information overload, and segment things at appropriate boundaries.

Data Collection?

In their most basic form data sets are nothing more than big piles of numbers. We collect and worship these numbers because we feel that by looking at them the right way we’ll be able to draw some sort of meaningful conclusion from them, or that they will perhaps help us prove some hypothesis, or demonstrate the validity of a concept. By themselves data sets have almost no meaning or value. A big pile of numbers can represent almost anything, and prove almost nothing, and this makes keeping track of the context is one of the most important aspects of data collection. While I can create a reasonably useful pipeline for plotting data, if I don’t remember why or to what ends I took a data set I might as well throw it away, it is of no use to me.

Data sets have a tendency of feeling rather vague and ill-defined at times. Almost immediately after taking a data set we begin to forget why we took it, and it starts to entropy. To combat this gradual increase in entropy you must document everything you can. Write memos in the folders containing your data that border on the point of obsessive compulsive, and try to capture the entire essence of your work.

Your goal in data collection is to tell a story. You want to collect the data you need, filter out any irrelevant data, and crystalize the essence of your message by displaying the data in a way meaningful to your reader. There’s essentially three pieces to the data collection puzzle I’ll cover.

• Getting Data Out of Devices – The art of retrieving data from devices that are designed with seemingly intentional malice.
• Processing that Data – The science of taking the pile of wrongly formatted, poorly scaled, and generally inconvenient data that you finally manage to coax out of the device, and formatting it such that it might be useful.
• Plotting It – The black magic behind gnuplot and others, a brief overview of the esoteric incantations required to transform your data set into a story