I've accumulated a large amount of data from Mt Erebus. Most of it is temperature, but it is unfortunately in proprietary formats and turning it into useful data for analysis in python's scipy and also in matlab has been a slight annoyance in the case of the HOBO and Tinytag data and so far impossible with the FLIR data. This is a progress report which might help other researchers deal with these proprietary formats and the evil business strategies employed by their creators.

Onset Computing HOBO U12-008

I initially looked into reverse-engineering the .hobo format. For my U12-008 four-channel loggers, which had three channels plus temperature enabled, the file structure seemed to consist of a pattern that nearly repeated itself every eleven bytes. I begun to attempt to figure out which bit was what, but abandoned efforts when I realized that an update to the Hoboware Pro software we purchased enabled a bulk csv export feature. Phew. I'd still like to see this format reverse engineered.

Gemini Tinytag II

I couldn't make any sense of Tinytag's .ttd format in a hex editor. I wonder if it is encrypted or compressed. I was stuck for a while since the Tinytag Explorer software only allows csv export of individual input files at a time. Then I noticed that in the windows file manager, right-clicking on .ttd files revealed a handy "convert to .csv" option. Selecting a large number of files and choosing this proved a quick and easy way to get all of my files into csv.

FLIR I7

I forgive Gemini and Onset for their proprietary formats, since their standard $100-ish software allows escape from those formats as described above. FLIR, on the other hand, rips off their customers in the worst way when it comes to data. We purchased the I7 for several thousand dollars (most of their cameras are in the tens of thousands). It produces .jpg images unusuable for quantitative analysis. They are stamped with several watermarks including the FLIR logo, a crosshairs, scale, etc. Additionally, the sliding temperature / color scale means that each image has a different color to temperature mapping.

FLIR is aware of this and stores the raw data in a non-standard EXIF tag called App1-FLIR. The only way to decode it is to purchase their poorly-documented, windows-only SDK for $1000. There is a thread here where a FLIR employee explains contemptfully to researchers that $1000 is cheap and noone uses unix anyway. I have had a look at this in a hex editor briefly and so far have had no luck decoding it. It would be a practical and moral victory to hack this one.

I just returned from an excellent day of squelching around in Predjamska Jama, the "Castle Cave," with Matej, Nina, Yaros, Mario, and Sten. A few others, including my Canadian friend Tim came for the first part of the trip and then headed back.

After two weeks of en mass caving with 40+ people on concrete paths, It was great to be in a wild cave with a reasonably sized group. Actinomycete colonies shimmered on the walls: tiny water droplets colored gold and silver. In some places I had trouble distinguishing between these bacteria colonies and "normal" condensation droplets and was a little leery that the distinction was real. Then on one wall I found a dried substance like a greenish gold lichen which I supposed was a failed or dormant Actinomycete colony; Sten agreed.

I cleaned my sneakers in the scalloped stream passages. These were Yorkshire-esque except for two things. First, there was hardly a meander at all; the stream shot downwards straight and true. Secondly, it was clear there was some very high energy flow and plenty of flooding going on here. The stream banks were lined with coarse gravel and bits of trash were caught high up in the passage. In several places bits of plastic bag hung from soda straws, and we found a cooking pot embedded in the gravel.

It seems that when you use django-tagging with the administrative interface, each time the "save and continue editing" button, a deeper structure of blank tag containers is formed in the tags field. I think this only happens when no tags are entered.

Edit, 29-June: This is fixed now on my site. I think it had something to do with a conflict between the tags_descriptor and tag field, which wasn't detected because my tag field was in an abstract base class...?

I just experienced a mental glitch which happens constantly to me and probably to others, and yet I'd never really thought about it before. I was convinced I was giving the program a correct path, and it couldn't find the file. Flummoxed, I tried pasting the path in the non-working python script into the "open" dialog in a text editor. Of course, when the text editor also failed to find the file I immediately saw my obvious mistake in the path I'd originally typed and which I had been so certain was correct.

Why was it that I couldn't see the path was wrong until I requested a second opinion from the computer?

Being an Ubuntu user is very exciting -- in addition to a constant stream of useful updates, we get an entirely new release once or twice a year. Every release speeds up the startup time, improves usability, and switches the pretty graphics, animations, and other snazziness with new snazziness.

However, I've tried the new Ubuntu 10.04 (the release numbers are the year and month) on my server and decided I won't upgrade my laptop. Here's why:

- For no reason whatsoever, the close, maximize, minimize buttons have been moved to the left hand side of the screen, which I find highly irritating. Apparently Mark Shuttleworth, Ubuntu's "Self Appointed Benevolent Dictator for Life," is more interested in wooing Mac-heads than maintaining the usability of his product. This is of course configurable, but having to change the default every time I'm on a new system or new user is not desirable.

- Control of daemons / services has been switched from the old Unix-style init.d system to a new system called Upstart. I think this is a great idea and improves command-line usability. For example, restarting your Apache 2 server is now "service apache2 restart" instead of "/etc/init.d/apache2 restart," which I find more intuitive. However, there's a problem -- Ubuntu used to have a perfectly good GUI for managing services at System --> Preferences --> Services. With the switch to Upstart, that GUI was removed and replaced with ... nothing.

- With the new indicator-applet, the mail icon is glued to the volume control! I don't want a mail icon on my taskbar, but I do want volume control. In the previous version, I could remove the mail icon and keep the volume icon (which was gnome-volume-control).

But the real reason is that I'm very happy with Ubuntu 9.04. It's pretty much exactly what I want. It ain't broke, I ain't a-fixin' it.

I noticed recently that my mercurial repository was no longer detecting new files. "hg status" and "hg commit -A" were returning "nothing changed" even though that was patently false. My .hgignore file seemed like a likely culprit, but it took me a while to figure out the problem.

To ignore OpenOffice temporary files, which end in #, I had added the lines:

syntax: glob

• #

to my .hgignore file. It turns out that # is the comment character for .hgignore file. After escaping the # with a backslash:

• \#

things started working perfectly.

I spent quite some time today trying to figure out where the rsync configuration file that defined an rsync module in use by MEVO was located (on a Solaris machine). There was an rsyncd.conf file in /etc/ , with modules in it, but not the ones I was looking for! Turns out it was in /usr/local/etc/rsync.d . Hope that helps someone out there!

Last night I did my best to hear the humming and whining of the tent guy lines as a lullaby rather than a disturbing din. It was my first really windy night in a scott tent.

This morning more symptoms of the storm became apparent. First the urinal froze up and we replaced it with a system of pee bottles and a big empty orange mogas drum. Anna marked this "pee here pee-lease."

Then the AN8-burning stove for heating the main hut went out. This was really a result of bad planning rather than bad weather -- noone had filled it before the storm. Weather precluded using snowmobiles so after hunting around for a full AN8 barrel, Marie, Harry and I manhandled it towards the hut. On hands and knees we wrestled with the barrel until we'd rolled it up next to the tank on the side of the hut and then struggled to lift it upright. Marie and I fetched the heavy "hurdy-gurdy" pump and I cranked it until to my relief the hut tank was full. Harry ruined the moment by pointing out that we should fill the garage tank as well.

Finally, we trudged back into the hut, only to find the stove flooded. Marie and Harry carefully dished out the fuel with a Parmesan cheese container and everyone relaxed once it was finally lit. Outside, the snowdrift about four feet from the window was still barely visible.

This morning I woke up looking forward to the 9am helicopter scheduled to bring up my survey and datalogging equipment, which would be delivered in a large 67-pound case dangling from a sling. Untying the entrance tube of the Scott tent, I was immediately disappointed -- howling winds and low visibility clearly made it impossible for the helicopter to land.

Munching on cornflakes and yogurt mixed with last night's couscous and dried apricots in the cozy hut helped take the edge off of my disappointment. Clive, amused by my breakfast, got out the Roland fish sauce, tumeric, raspberry vinegar, and vegemite, suggesting they would make a tasty addition to an already colorful bowl.

We had left the snowmobiles out without covers on the night before, and all trudged out into the tempest to cover them and prevent snow-filled engines. It was invigorating working in the raw elements, at least so close to shelter.

It surprised me at first that the United States Antarctic Program pushes drugs through intimidation. In the altitude training course I was given, it was made clear that we were expected to take Diamox, Dexamethasone, or Viagra (recently clinically proven for altitude use) to facilitate our acclimation to altitude. After a well-taught hour of high altitude physiology covering everything from Chain-Stokes Breathing to High Altitude Cerebral Edema a doctor was trundled in apparently for the sole purpose of telling with authority horror stories of what happens to people who go to field camp without taking their pills. When I went to pick up my "Dex," I asked if I could take a lower dose or delay taking it until I noticed symptoms of altitude. The nurse seemed angry and offended that I would suggest such misconduct and I was told in no uncertain terms to take the pills as directed.

It may seem strange, but this is certainly the best practice. USAP has to evacuate people every year with cerebral or pulmonary edema from various high-altitude camps. Not only are their grantees working outside of the normal environmental range for human beings, but travel is generally by helicopter, slamming the body immediately into a low pressure, oxygen starved environment. Such unnatural procedure requires intervention with unnatural drugs.

Right now, I'm at 3700m and feel great. Other than tiring a little quicker than usual when building walls of snow blocks, I have no symptoms of altitude. I'm thankful for the tools (Dex) and planning (two and a half days of acclimation at Fang Ridge) that USAP provided to keep me comfortable and healthy, and yet I can't help wondering how I would fare without the drugs.

For the last three days, the first wave of G-081ers (Nial, Marie, Harry, Anna, Clive, and me), acclimatizing to altitude at 3000m on Fang Ridge, have been tantalized by glimpses of a column of puffy steam floating above the snowy rise to the East. Last night communications on the Iridium phone foretold a helicopter arrival this morning at 9am, to carry us up towards that sight. After a hearty meal of spinach and pasta with Harry and Anna in Nial's and my tent, we tried to sleep despite the loud booms coming from the ice beneath our heads. Eventually I saw Nial get up, snowbibs and all.

"Are you hearing those crazy booming noises?!," he demanded.

I'd just read a book on the Transantarctic Expedition, and had an idea.

"Yeah, icequakes, maybe?"

Nial hadn't heard of them. I couldn't remember the exact mechanics, but I thought it had something to do with cracks forming as the ice cooled overnight. We did have somewhat of a night, when the sun was mostly behind Erebus. He returned with tales of the otherworldly echoes bouncing off of Fang Ridge.

When our alarms went off at 6:15, we speculated what we'd see outside the tent, convinced we'd be marooned on a pinnacle of ice, the rest of the glacier having fallen away. Or perhaps Erebus had been lobbing bombs our way, in its lazy strombolian style and we'd be surrounded by smoldering volcanics. Outside, the volcano denied having been up to anything untoward during the night. It was the third beautiful day in a row, and the verdict was still out on the new Pope.

Soon we had all of our gear in a cargo line and a red Bell 212 thumped down from the sky. After a good deal of fumbling with my big crash helmet I could just about make out the chatter between Anna and our pilot Paul and tested my mic, which didn't seem to work. Verbal communication was quickly superseded by sight and smell as Paul looped around the crater in a series of tight swoops not 30m above the edge. The yawning yellow jaws jutted vertically up at us as the smell of rotten eggs filled the helicopter. I grabbed my camera and shoved it to Marie, who had a better shot out out of her window.

Ducking under the spinning rotors we ferried bags to the Lower Erebus Hut. Soon we were defrosting the pee tube in the urinal, digging steps, lighting stoves and filling pots with snow for water, and digging snow anchors to pitch tents. Nial and I stood up on the roof of the hut and looked off towards the nearby "hut fumaroles," gnarled dwarven ice towers as a glittery snow began to fall in the brilliant sunlight.

After a "brew" or two in the hut, Harry led the charge back out into the snow to chop up snow blocks and build walls around everyone's tents to block the wind. Our altitude classes encourage "active rest" and sawing up 40 pounds blocks, carrying and stacking them certainly qualified as "active." Nial and Harry invested a fruitless half an hour trying to start a chainsaw to ease the process while I watched in amusement.

We retired to the hut for more brews and a surprisingly tasty meal of couscous, falafel, cheese, and other dehydrated rabbit food courtesy of Anna and Nial. Both Anna and Marie were laid low by altitude sickness but looked much better after hits on the O2 tanks.

My first night in LEH was cold to begin with, and a big patch of rime ice grew on my sleeping bag surrounding my head. Melting ice running down my face around 3am informed me that the air in the tent was finally above freezing.

To improve the quality of communication on IRC, I propose the integration of the ubiquitous "upvote / downvote" system (like that on reddit, stackoverflow, and slashdot) with a web chat client. Each statement could be individually rated by any interested user.

I then picture the statement being displayed in a tone of greyscale representative of the votes on that statement. The most poorly rated statements would appear nearly white, while relevant ones would be a solid black.

Perhaps the reputations of a user's statements would accumulate and affect the reputation of the user herself. This would make it easy to visually ignore lamers and n00bs.

A system like this would be useless in sparsely attended rooms, but might be a good solution to the static coming through in big rooms like #python. It might help increase the signal to noise ratio in that loud environment.