Monthly Archives: December 2012

Apache Redirect & Varnish

Apache’s mod_rewrite is an awesome module providing all sorts of impressive capabilities, however it’s sometimes overkill when all you want is to redirect an entire site or path to a different one. In these situations, the mod_alias’s Rewrite function is often the ideal solution, with it’s simple syntax.

Redirect permanent /  http://newhost.example.com/
Redirect permanent /test.html http://newhost.example.com/test.php

Unfortunately I’ve found a situation where certain syntax can cause failures in some environments – consider the following:

Redirect permanent /  http://newhost.example.com

This example will work happily when a user is accessing Apache directly, however it will fail horribly when accessing via a Varnish cache, eg:

$ wget oldhost.example.com
 --2012-11-27 11:26:54--  http://oldhost.example.com/
 Resolving oldhost.example.com (oldhost.example.com)... 172.16.1.1
 Connecting to oldhost.example.com (oldhost.example.com)|172.16.1.1|:80... connected.
 HTTP request sent, awaiting response... 301 Moved Permanently
 Location: unspecified
 ERROR: Redirection (301) without location.

This was the source of a lot of headaches since the site still *worked* correctly when connecting directly to it, but kept failing whenever accessed via the varnish caches.

The cause is simply a missing trailing / in the redirect – configuration should actually look like:

Redirect permanent /  http://newhost.example.com/

It’s a nasty trap for the unwary sysadmin, since it works perfectly when accessing the server directly – I’m not entirely sure whether it’s an issue with Apache allowing this syntax, or whether it’s an issue with Varnish not understanding how to proceed – maybe a bit of both.

RFC 2616 isn’t clear, it states that if the abs_path is empty, it’s the same as abs_path being equal to /, and that when requesting a URL with an undefined abs_path, it should be treated as /… but it doesn’t clarify whether an application serving a URL should or shouldn’t append the trailing /.

I guess ideally Apache should be patched to serve up URLs with a trailing / and Varnish should accept redirects with or without it to protect both applications from blame.

Android ICS & the MTP pain

4 Replies

With Android 3.0, Google switched the way of accessing files on Android devices from USB mass storage device behaviour, to Media Transfer Protocol, a standard initially created by Microsoft, but later developed into part of the USB standard for media devices.

For most Android users, new ICS phones are where they will first encounter this new approach to sharing file content. Phones previously upgraded from Android 2.x may still provide the conventional mass storage device behaviour, but any new ICS devices force the option of either MTP or the camera-focused PTP.

MTP is one of those technologies designed with the best of intentions – the fundamental design around it is that it abstracts the underlying filesystem, enabling the phone/device to store the files in whatever way suits it best and then exposing them to the computer.

Unlike the traditional mass storage mount, the files can be available to both the phone and the computer at the same time, whereas previously Android applications would lose access to the “sdcard” storage space when mounted on the computer.

It’s a great idea, except that it’s nowhere near as compatible with computers as a generic mass storage device is. Whilst Windows has native support for MTP, thanks to MS being the original creator of the standard, MacOS has no native support and requires a download, and Linux systems are all kinds of fail when it comes to MTP.

Considering I use a Linux laptop, this situation is very unhelpful – reading online suggests various options (the arch wiki was quite helpful):

Use of GNOME with GVFS or KDE’s KIO to connect the storage using the GUI’s device handling functionality. Unfortunately with my tests performed using Nautilus (GNOME’s file manager) on Debian Testing (recent app versions), I wasn’t able to connect to the device, instead experiencing a lot of timeout errors.
Linux has a FUSE driver that is able to read MTP, with corresponding usertools called mtpfs – however in my tests, this driver has some serious usability, stability and trust issues, thanks to it repeatedly failing and randomly unmounting, as well as hiding many files that are known to exist.
An alternative driver called “go-mtpfs” (written in Go) which provides user space connectivity to MTP Android devices at reasonable speeds and a reasonable UI for mounting the drives.
Failing these, a network transfer over WiFi could be an option, such as Samba or SSH. Bluetooth is also a possibility, but the performance is quite poor for numerous file transfers.

I’ve ended up using go-mtpfs which appears to work quite nicely, at least with my ICS Google Nexus phone, however it’s annoying that it’s not in the Debian repositories yet.

Whilst I understand the design decisions and issues solved by Google moving to using MTP rather than USB Mass Storage, the sudden deprecation of a feature that previously existed is a real pain for a user like myself who is wondering why their new phone won’t connect to their computer…

Personally I think Google should have included a legacy USB Mass Storage option for at least another release (ie till Android 5) to have provided time for desktop OS developers to improve the handling of this protocol.

It would also have been useful if they had included a “Install Android Driver” option to expose a small USB mass storage drive (eg 20MB) that includes drivers/apps for common platforms such as MacOS and Linux, so that users could connect and load on drivers without needing

For now, go-mtpfs seems to work OK, meanwhile one of the GVFS/GNOME developers is re-working the MTP handling to properly support Android devices, so hopefully we won’t have the headaches around this protocol for too much longer…

Great server crash of 2012

Leave a reply

In a twist of irony, shortly after boarding my flight in Sydney for my trip back to Wellington to escape the heat of the AU summer, my home NZ server crashed due to the massive 30 degree heatwave experienced in Wellington on Christmas day. :-/

I have two NZ servers, my public facing colocation host, and my “home” server which now lives at my parent’s house following my move. The colocation box is nice and comfy in it’s aircon controlled climate, but the home server fluctuates quite significantly thanks to the Wellington climate and it’s geolocation of being in a house rather than a more temperature consistent apartment/office.

After bringing the host back online, Munin showed some pretty scary looking graphs:

localhost flew too close to the sun and plummeted to it's doom

localhost flew too close to the sun and plummeted to his doom

I’ve had problems with the stability of this system in the past. Whilst I mostly resolved this with the upgrades in cooling, there are still the odd occasions of crashing, which appears to be linked with summer months.

The above graphs are interesting since they show a huge climb in disk temperatures, although I still suspect it’s the CPU that lead to the actual system crash occurring – the CPU temperature graphs show a climb up towards 60 degrees, which is the level where I’ve seen many system crashes in the past.

What’s particularly annoying is that all these crashes cause the RAID 6 to trigger a rebuild – I’m unsure as to why exactly this is, I suspect that maybe the CPU hangs in the middle of a disk operation that has written to some disks, but not all.

Having the RAID rebuild after reboot is particularly nasty since it places even more load and effort onto an already overheated system and subjects the array to increased failure risk due to the loss of redundancy. I’d personally consider this a kernel bug, if a disk operation failed, the array should still have a known good state and be able to recover from that – fail only the blocks that are borked.

Other than buying less iffy hardware and finding a cooler spot in the house, there’s not a lot else I can do for this box…. I’m pondering using CPU frequency scaling to help reduce the temperature, by dropping the clock speed of the CPU if it gets too hot, but that has it’s own set of risks and issues associated with it.

In past experiments with temperature scaling on this host, it hasn’t worked too well with the high virtualised workload causing it to swap frequently between high and low performance, leading to an increase in latency and general sluggishness on the host. There’s also a risk that clocking down the CPU may just result in the same work taking longer on the CPU potentially still generating a lot of heat.

I could attack the workload somewhat, the VMs on the host are named based on their role, eg (prod-, devel-, dr-) so there’s the option to make use of KVM to suspend all but key production VMs when the temperature gets too high. Further VM type tagging would help target this a bit more, for example my minecraft VM is a production host, but it’s less important than my file server VM and could be suspended on that basis.

Fundamentally the host staying online outweighs the importance of any of the workloads, on the simple basis that if the host is still online, it can restart services when needed. If the host is down, then all services are broken until human intervention can be provided.

Google Convoy

2 Replies

Spotted this when doing street view around Pyrmont Sydney – there’s at least two Google cars since one has captured this picture, looks like there might even be a third….

The watchers are watching the watchers.

Unsure why there would be a convoy of cars, the few occasions when I’ve seen Google imaging cars in the past, they’ve always been standalone. Having said that, Pyrmont contains Google’s Australian headquarters, so maybe there’s a few special projects on the go.

linux.conf.au 2013 plans

Leave a reply

It’s nearing that important time of year that the NZ-AU open source flock congregate that important and time honoured tradition of linux.conf.au. I’ve said plenty about this conference in the past, going to make an effort to write a lot more this year about the conference.

There’s a bit of concern this year that there might not be a team ready to take up the mantle for 2014, unfortunately linux.conf.au is a victim of it’s own success – as each year has grown bigger and better, it’s at the stage where a lot of volunteers consider it too daunting to take it on themselves. Hopefully a team has managed to put together a credible bid for 2014, it would be sad to lose this amazing conference.

As I’m now living in Sydney, I can actually get to this year’s conference via a business class coach service which is way cheaper than flying, and really just as fast once taking the hassles of getting to the airport, going through security and flying into account. Avoiding the security theatre is a good enough reason for me really – I travel a lot, but I actually really hate all the messing about.

If you’re attending the conference and departing from Sydney (or flying into Sydney from NZ to then transfer to Canberra), I’d also suggest this bus service – feel free to join me on my booked bus if you want a chat buddy:

Depart Sydney, Sunday 27th Jan at 11:45 on bus GX273.
Depart Canberra, Saturday 2nd Feb at 14:00 on bus GX284.

The bus has WiFi and power and extra leg room, so should be pretty good if you want to laptop the whole way in style – for about $35 each way.

Museum of Contemporary Art Australia

Leave a reply

On one of our trips down to Circular Quay, we recently discovered the Museum of Contemporary Art Australia.

It’s a free museum with a range of interesting and unique exhibits and worth visiting if you have a couple of hours time to spare – since it’s right on Circular Quay it’s really easy to get to, plus it’s also right by The Rocks which is worth a visit.

Circular Quay area

This is what happens when you erect scaffolding in a flight path…

Mail order plane anyone?

OCD alcoholism: drunken night of beer bottle smashing followed by having to re-assemble them all again.

This is a visual representation of my inbox.

This is a visual representation of emails pouring into my inbox. :-/

At what stage are artists just really taking the piss?

The colour! OMG!

The best work of art in the whole place :-P

Of course not all of Sydney is as excited about artwork….

Is this sign art itself?

The MCA was a really good visit, well worth checking out if you have time in Sydney – plus it’s free, which is a miracle for Sydney really. ;-)

Based on how much I enjoyed MCA, I’m even more keen to make a trip down to Tasmania in the near future to go visit MONA – I sadly missed it on my last trip as it hadn’t quite opened at that stage, but hearing lots of awesome stuff about it.

Sydney’s Powerhouse Museum

Leave a reply

The Powerhouse Museum in Sydney is a science/tech/design museum offering a range of exhibits including space, robotics, history, fashion and other geeky and design related topics.

I went there on a special event day so the usual $12 entrance fee had been halved (yay!) and spent a few hours having a good look around the museum.

No tech museum would be complete without a steam exhibit – the powerhouse actually has some of the engines in a powered state, although there wasn’t a whole lot going on when I was there.

Before data centers, these were the power houses of industry.

Before data centers, these were the power houses behind the world’s industry.

Old destination board from a railway station.

The man walking in front of a steam engine with a red flag to limit it’s speed seems about as hopeless as the RIAA/MPAA wanting to stop digital downloads…. you can’t restrict new technology for long.

There’s also a good exhibit of space technology, including an actual F-1 rocket engine, the most powerful liquid fuelled rocket ever developed and the machine responsible for powering the Saturn V which took humanity to the moon.

(from the left) F-1 rocket engine, a sounding rocket (research), several models of famous space craft and satellites and more.

F-1 Engine! These things are NOT small!

$\m/ squeeeee$

\m/ :-D

Rocket thruster used in command modules.

1/3 scale Soyuz pair coupled together.

Retro computer inside the space station module mockup.

Replica Mars Rover – The Soviets sure made some weird looking hardware.

There are a range of robotics exhibits, including some neat demonstrations of industrial robotic arms that are bit more uncommon to see.

All hail the robotic overlords!

Everyone loves hexapods!

Plus a bunch of other random bits:

Weird looking aircraft

Electrifying Touch

At times the selection of exhibits feels a bit disjointed, things certainly don’t flow quite as well as some of the other science and technology museums I’ve been to and some areas a bit worn and dated – having said that, they are in the process of renovations, so it might be fairer to re-evaluate it in a year or so.

That being said, it’s worth a visit, just for some of the awesomeness they have there – plus how often can you take a picture of you and your partner standing underneath an F1 rocket engine? :-)

Houston, set engines to snuggly!

Bondi Beach

Leave a reply

Seeing as I’m living in Sydney, I felt compelled to head out to visit Bondi Beach – it’s a pretty famous location, so figured it’s worth getting out there to check it out at least once.

All set up for a day at the beach, geek style!

Getting out to Bondi is a bit of a mission, since unfortunately the rail link doesn’t go the whole way – instead you catch the train to Bondi Junction, and then switch to bus in order to get to the beach.

You can buy a (pricey) ticket at any of the city train station machines that includes both the train and bus fare – costing about $8.80 return at an off-peak time.

The bus is the worst part of the trip, in future I’ll probably consider just walking it as it’s only 3.2km from the railway station – whilst the buses *do* have air con, any benefits are lost with the squeeze with numerous tourists squished into it, personally I’d rather be hot outside than hot squeezed into a metal tin with random strangers….

Bondi Beach. It has sand. Waves. Usual beach things really.

People are really the biggest issue with Bondi – it’s a nice beach in it’s own right, but the volume of tourists and shops around it kind of ruins the enjoyment somewhat, particularly when combined with the transport hassles. In summer the crowds can be massive, as much as 30 thousand+ – and it’s not a particularly large beach really.

The smaller beaches in walking distance to the south like Tarramara and Bronte look more appealing if you’re more of an introverted type, or take the ferry over to Manly for some of the nice beaches over there.

If you’re a tourist on holiday, you’re probably better off spending your time in Sydney to explore the city and waterfront and save the beach trips for when visiting Brisbane’s gold coast.

Having said that, if you’re in Sydney and don’t mind lots of people, it’s a nice beach and looks great for swimming and surfing, plus it’s well patrolled by surf life savers.

One of the perks of Bondi Beach is being able to meet pretty ladies. Although they won’t always appreciate you taking pictures of them. ;-)

Aside from the beach, there are a large number of eateries along the beach and in the surrounding streets – (like Funky Pies for some delicious vegetarian/vegan goodness), as well as a walk up over the rocky cliffs.

Walkway from the southern end of Bondi Beach.

Looking back at Bondi

It was worth the visit, but I don’t think Bondi will become one of my regular visited locations, the transport hassles make it a bit difficult to get there – it’s easier for me to get out to Manly on the ferry than it is to get to Bondi on the train/bus.

It would be a different story if living locally or having a private transport option- some of my colleagues at work get there by bike which seems to work quite well for them.

Das Keyboard Ultimate Silent

8 Replies

With the recent move to Sydney, I’ve had to leave my beloved IBM Model M keyboards back in New Zealand – sadly they’re a bit heavy and large to effectively pack into my luggage without sacrificing some much needed clothes.

Even if I was to bring them over here to Australia with me, the Model M’s are too loud for me to use in a shared office environment – my Model M was previously banned from my last employers office after they could hear it through two walls and down a phone at the other end….

Instead I’ve brought a Das Keyboard Ultimate Silent. I’ve been a fan of the Das Keyboard idea for a while – just like the IBM Model M they’re traditional clicky mechanical keyboards, but have modern features such as USB, lighter bodies and (love them or hate them) Windows keys that are useful for both Windows and MacOS users.

Das Keyboards come in both labelled (Professional) and unlabelled (Ultimate) revisions, and the option of either standard loud clicky keys or the “silent” model – considering I’m working in a shared office space, I elected to go for the silent edition.

Mmmmm sleek black sexiness.

I’ve been using the Ultimate Silent for about two months now, general impressions are:

It’s an excellent keyboard that’s well worth the $150 AUD price tag. I’ve had heaps of comments from co-workers on how great it feels to type on, command line power geeks can’t be wrong. ;-)
The keys still have the tactile feedback of a mechanical clicky keyboard. Whilst the responsive spring-back is a little more subdued than on the Model Ms, it still delivers a very nice feel.
The blank keys are AWESOME. People who try to use my computer are always really put off at first, however if you’re a touch typist it won’t take long to get used to it.
It’s not exactly silent -“quieter” is a more accurate term, I certainly have the loudest keyboard in the office, but it’s nowhere near as loud as an actual Model M is. The sound is more a subdued tap as the keys hit the bottom of the keyboard when typing, rather than the audible click of a traditional clicky keyboard.
My colleagues are a pretty good bunch of people since they haven’t murdered me for loudly typing and stretching the “silent” label to the limits. ;-)
I have occasional issues with finding a particular symbol key (things like ^ or &), but I can touch type almost any of the 104 keys on it without an issue.

Personally I’ll keep using the IBM Model Ms as my personal keyboard, they’re great quality keyboards and I love the fact I can keep using something designed in 1980s (mine were manufactured in 1994) on my computer for possible the rest of my life, but I’d be very content with using a Das Keyboard personally as well as professionally if I didn’t already have the Model M.

It always amazes me how often geeks will spend huge money on their computers and then neglect the keyboard or buy something that features lots of flashy lights and special keys, but ignore the most important requirement of good typing ergonomics and performance.

I haven’t tried the clicky version of the Das Keyboard myself so I can’t really compare it – I expect you’d find that the clicky version has a even nicer feedback (like the Model M) but the silent is the better investment if you work near anyone else.

I brought mine from AusPCMarket who have local stock and it arrived in a couple of working days without an issue, otherwise buy direct from Das Keyboards.