Monthly Archives: November 2012

Light Rail of Sydney

Ending up living in Sydney rather than Melbourne has brought one unfortunate side effect – a lack of awesome trams spread around the city. Thankfully it turns out one doesn’t have to go all the way to Melbourne just to get a light rail fix.

Sydney has two forms of light rails – the monorail which is to be decommissioned in 2013 and the more recent tram link that runs from Central station, through Pyrmont and out into Glebe.

mmmmmm light rail here, light rail there, light rail everywhere!

I haven’t been on the Sydney trams yet, based on the location of my apartment, central station and my office in Pyrmont, the tram tends to take me away from the direction I want to actually travel in – although that being said it may be useful if I ever want to go quickly from work to Central station and then transit onto the commuter rail network.

I’m hoping that Sydney considers adopting more light rail – whilst the commuter rail network is very effective at linking main centers, there’s a lot of Sydney that’s only linked by buses, which aren’t particularly fast and seem to be at their limits in regards to capacity.

Extending the tram lines out to places like Surry Hills and Bondi would be a huge plus, the linking of commuter rail and light rail has been very successful for Melbourne and there’s no reason why it couldn’t be replicated here as well.

Sydney Harbour Bridge Walk

Leave a reply

One of my biggest annoyances about living in Auckland is that the only way to cross the Auckland Harbour Bridge was via car or bus – so whilst my apartment would have been within feasible walking distance to the city, it was utter impossible to walk it.

Sydney has at least got this right and their famous harbour bridge is crossed by car, train, cycle and foot, which offers an excellent chance to see the bridge up close and get some amazing views over the city and harbour.

Views don’t get much more iconic than this. Amazingly hot dude and some building thingy in the background.

It’s an easy walk to get to if you’re a tourist – you just need to take the Northern Rail line Central, Wynward or Town Hall train stations and depart at the Milsons Point railway station on the North Shore, which is right where the bridge crossing starts for both pedestrians and cyclists.

Both the train and driving across the bridge offer views, but it’s nowhere near as excellent as checking it out on foot – and of course driving across will incur you some lovely tolls.

It’s hard to get a sense of scale – for reference there are two rail lines running through the right side tower, each capable of fitting a double decker train though it – or in the case of the left side, a road lane and a pedestrian path.

If I find my current 1 bedroom CBD apartment rent painful, I’d hate to think how much these places would be costing…

Sydney Opera House!

There’s one metric fuckload of steel in this.

It’s also really high!

Circular Quay, the international passenger terminal for cruise ships and the rocks.

I love these apartments at the city end of the bridge for being a neat varying stack of towers and roof gardens, but at the same time hate it’s blocky concreteness.

Interspersed concrete and greenary awaits at the end of the crossing – descend the steps to end up in “The Rocks”, the heritage part of Sydney

The crossing is well worth doing, it’s a real shame that Auckland hasn’t added a walk and cycleway of their own – not only is it a great recreational addition, but it also makes the lifestyle choice of walking or cycling to work from the North Shore feasible.

Australian National Maritime Museum

Leave a reply

As part of my daily walk to work in Pyrmont, I pass the impressive Australian National Maritime Museum – considering they have both a Destroyer and a submarine parked in the harbor outside, I could hardly resist a visit for long. :-)

The museum has the usual range of maritime artifacts, exhibits about life at see, immigration, ship guns and steam engines, as well as a few neat items such as a maritime helicopter and small yachts and motor boats suspended inside.

Weird helicopter – kind of looks like some weird bug when viewed from underneath.

Of course for me, the real attraction was what was parked outside…

Above or below, either way we have your warfare needs sorted. :-)

Nowhere near as large as the cruise ships that now dock at Sydney, but still impressive when looking at them up close.

The best part is that yes, you can board both the destroyer and the submarine and explore them for yourselves! Unfortunately the day that I was there the submarine was closed, but I’m sure I’ll be back at some point and able to go inside it to explore more.

I had a good wander around the HMAS Vampire, a 1956 darling class destroyer built in Australia, and the last generation of warships before weapon systems moved from traditional cannons/big guns to missile based systems.

If you filmed a porno here, the title would be “one shade of gray”.

Military budgets don’t seem to extend towards having normal height ceilings.

Bit obsolete in the age of the drones and smart bombs, but nothing is quite as impressive as some big cannons.

I love the sci-fi look to these AA cannons.

Pyrmont Bridge with the wooden boat festival in the foreground.

Fuck your NBN and your UFB, fibre to the warship is where it’s at!

Server rooms have come a long way since. :-)

More computer users need to keep this in mind.

Early generation Nagios outage notification device?

Boom, boom, boom, boom, boom.

I can’t resist some good thick chains…. ;-)

Sir, we have a firing solution for the Apple store from the front turret!

I love warning signs like these “this shit be bad bro, try not to hang around here”

The rounds fired by these machine guns must be massive… probably tear a small boat apart.

Standing on the deck of the weapon of the 20th century looking out at the new weapons of the 21st century…. mega corporations.

If your target is too quick to move, there’s also the smaller patrol boats you can use to chase them down. They can’t have been too comfortable to spend time on, even sitting at the dock on a calm day the boat had a noticeable rocking to it.

The little offspring of the destroyer?

Whilst I couldn’t get inside on this trip, I got some great pictures of the HMAS Oberon diseal-electric submarine parked up alongside the destroyer. Whilst it’s not hard to find warships at maritime museums around the world, near-operational submarines are a little less common.

Long, hard, black… however now devoid of any seamen.

The little access hut looks a bit out of place compared with the sleek design of the rest of the machine.

Various snorkels and periscopes.

If you’re looking for some less recent history, the museum also has two large wooden ships – replica of the endeavor as well as a restored tallship.

A clone of the ship HMS Endevour, used by James Cook for Britain’s “discovery” of New Zealand, Australia and other pacific locations.

I might have found something even more uncomfortable than economy class for traveling around the world on…

Some excellent views of Darling Harbor

Pyrmont bridge and Darling Harbor

If you have several hours to spare in Sydney, the museum is well worth a visit, especially if you love complex machines such as warships.

Debian Testing with Cinnamon

7 Replies

I’ve been running Debian Stable on my laptop for about 10 months for a number of reasons, but in particular as a way of staying away from GNOME 3 for a while longer.

GNOME 3 is one of those divisive topics in the Linux community, people tend to either love it or hate it – for me personally I find the changes it’s introduced impact my workflow negatively, however if I was less of a power user or running Linux on a tablet, I can see the appeal of the way GNOME 3 is designed.

Since GNOME 3 was released, there have been a few new options that have arisen for users craving the more traditional desktop environment offered – two of the popular options are Cinnamon and MATE.

MATE is a fork of GNOME 2, so duplicates all the old libraries and applications, where as Cinnamon is an alternative GNOME Shell, which means that it uses the GNOME 3 libraries and applications.

I’m actually a fan of a lot of the software made by the GNOME project, so I decided to go down the Cinnamon path as it would give me useful features from GNOME 3 such as the latest widgets for bluetooth, audio, power management and lock screens, whilst still providing the traditional window management and menus that I like.

As I was currently on Debian Stable, I upgraded to Debian Testing which provided the required GNOME 3 packages, and then installed Cinnamon from source – pretty easy since there’s only two packages and as they’ve already packaged for Debian, just a dpkg-buildpackage to get installable packages for my laptop.

So far I’m pretty happy with it, I’m able to retain my top & bottom menu bar setup and all my favorite GNOME applets and tray features, but also take advantages of a few nice UI enhancements that Cinnamon has added.

All the traditional features we know and love.

One of the most important features for me was a functional workspace system that allows me to setup my 8 different workspaces that I use for each task. Cinnamon *mostly* delivers on this – it correctly handles CTL+ALT+LEFT/RIGHT to switch between workspaces, it provides a taskbar workspace switcher applet and it lets me set whatever number of workspaces I want to have.

Unfortunately it does seem to have a bug/limitation where the workspace switcher doesn’t display mini icons showing what windows are open on which workspace, something I often use for going “which workspace did I open project blah on?”. I also found that I had to first add the 8 workspaces I wanted by using CTL+ALT+UP and clicking the + icon, otherwise it defaulted to the annoying dynamic “create more workspaces as you need them” behavior.

On the plus side, it does offer up a few shinier features such as the graphical workspace switcher that can be opened with CTL+ALT+UP and the window browser which can be opened with CTL+ATL+DOWN.

You can never have too many workspaces! If you’re similarly anal-retentive as me you can go and name each workspace as well.

There’s also a few handy new applets that may appeal to some, such as the multi-workspace window list, allowing you to select any open window across any workspace.

Window applet dropdown, with Nautilus file manager off to the left.

I use Rhythmbox for music playback – I’m not a huge fan of the application, mostly since it doesn’t cope well with playing content off network shares over WAN links, but it does have a nice simple UI and good integration into Cinnamon:

Break out the tweed jackets and moleskins, you can play your folk rock in glorious GTK-3 graphics.

The standard Cinnamon theme is pretty decent, but I do find it has an overabundance of gray, something that is quite noticeable when using a window heavy application such as Evolution.

Didn’t you get the memo? Gray is in this year!

Of course there are a lot of other themes available so if the grayness gets to you, there are other options. You also have the usual options to change the window border styles, it’s something I might do personally since I’m finding that the chunky window headings are wasting a bit of my laptop’s very limited screen real estate.

Overall I’m pretty happy with Cinnamon and plan to keep using it for the foreseeable future on this laptop – if you’re unhappy with GNOME 3 and preferred the older environment, I recommend taking a look at it.

I’ve been using it on a laptop with a pretty basic Intel GPU (using i810 driver) and had no issue with any of the accelerated graphics, everything feels pretty snappy – there is also a 2D Cinnamon option at login if your system won’t do 3D under any circumstance.

Point & click Procmail with MailGuidance

Leave a reply

Procmail is a rather old, but still very useful Unix/Linux application commonly used for writing mail filter rules on Linux servers. I typically use it for user-level filtering, such as defining mailbox filters for all my emails.

It’s also useful for handling shared email addresses, such as support mailboxes receiving a range of emails. Procmail allows these emails to be re-directed to multiple people, different folders or almost any other action desirable.

To make it easier to manage Procmail rule sets in this scenario, I built a tool called “MailGuidance”. It’s an open source PHP/MySQL application which allows a user to create Procmail filters in a web environment and having it then generate the appropriate configuration in the background on the server.

Define whom in your organisation should be getting emails for each matching filter.

MailGuidance is intended for small organisations or an individual seeking a web-based way of managing their procmail rules, it’s intentionally simple and does limit the power of procmail somewhat in exchange for making an easy to use experience for users.

Easy web based interface where filters can be enabled/disabled per user.
User “holiday mode” where all emails to that user get redirected to another until they return, so that nothing gets forgotten.
Optional email archiving into different folders.
Configurable behavior for archiving and unmatched mail.
Works perfectly with IPv6. :-)

Configurable behaviors.

Going away? Send all that albino monkey porn you’ve subscribed to through to your colleague instead!

The best use case for MailGuidance so far has been for handling server log and error emails, by filtering and then redirecting them to the appropriate people/teams to avoid spamming system administrators with irrelevant messages.

I spent some time this weekend tweaking it a bit more and have now packaged some releases and opened up the repository publicly – you can download stable version 1.0.0 or read more about it on my project page here. RPMs are available for users of RHEL/clones.

Radius Rapid Rotate (R3)

Leave a reply

I’ve been spending a bit of time lately going through my private source code repositories and tidying up things for public release.

A while ago I had a customer who required their FreeRadius traffic accounting logs to be collected from a few servers and saved onto a mounted network drive. It’s a simple enough problem, however there’s a few requirements that make it slightly trickier than it sounds:

Extremely important that log files weren’t lost or corrupted in any circumstance.
The archive location was a mounted network drive, this means no guarantee that the filesystem would always be mounted and writable.
The rotated files need to be named with the server hostname so that files from multiple servers could be collated in a single location without clashing.
Regular frequent execution period, eg every 5 or 10 minutes.

The solution I wrote was “Radius Rapid Rotate” (or R3 since I’m a lazy typist). This utility rotates FreeRadius log files in a manner which meets all the above requirements.

It would have been possible to write this all into an existing application, such as logrotate, however logrotate isn’t intended for such frequent execution and won’t do log rotation onto a network mount in a manner that will handle a dodgy network mount gracefully.

Whilst this application is FreeRadius focused, it would be easy to port to use for other purposes if suitable.

Introducing FlatTraffic

2 Replies

FlatTraffic is an AGPL web interface for analyzing NetFlow records and showing statistics designed to make it clear and easy to determine which hosts of the network are consuming data.

It’s still in beta stage, the application is functional and is documented, but may have bugs and need a few tweaks here and there to bring it up to a stable grade… I’m releasing now so that people can start using and breaking it to get a well tested piece of code to enable a 1.0.0 release.

I’d be lying if I said this was a complete list of my computers….

As you are probably aware, New Zealand (and Australia to a lesser degree) are victims of the much hated internet data cap, an unfortunate response to the economic pressures of providing internet services in our markets.

This is a particular issue when you have situations such as flatmates sharing a connection or a a collection of servers behind an internet link which are hungrily consuming the data cap every second.

To help keep the peace with flatmates I started writing this application when I was back in Wellington to report on traffic usage, using a SQL DB of NetFlow records collected by the gateway. It got put on hold somewhat after moving to Auckland and getting a fat DSL plan from Snap NZ, however it recently got resurrected so that I could track down which host on my home server was chewing through the much smaller data cap at it’s new home at my parents place (sadly my full tower beauty wouldn’t fit into my plane luggage).

FlatTraffic is focused at being a geek home/small server environment tool rather than a general purpose NetFlow analyzer – there are more powerful tools already available for that, my design focus with FlatTraffic is simplicity and doing one job really well.

FlatTraffic assumes you’re using it in a conventional ISP customer situation and allows you to configure the monthly date that your service renews on, so that it will show data usage periods that match your billing period. You can also configure other key options such as 1000 vs 1024 bytes and what automatic DB truncating options should be turned on.

Graphical configuration options, eat your heart out Microsoft developers.

There are currently four reports defined in FlatTraffic:

Traffic consumed by protocol.
Traffic consumed by host (with reverse DNS lookup resolution of host IPs)
Traffic consumed per day.
Traffic consumed by configured network range.

Helpful daily totals, aligned with your ISP’s billing period.

FlatTraffic doesn’t replace a NetFlow collector, you still need to understand the principles of setting up NetFlow traffic accounting and configuring a collector that stores records into a SQL database.

I’ve included some sample scripts for use with flowd (from the flow-tools collection) however I’m going to work on adding support for some better collectors. There’s also work needed for IPv6, since whilst the app UI is IPv6 compatible, the NetFlow reporting is strictly IPv4 only currently.

(Unfortunately I also have issues in that the iptables module I’m using to generate NetFlow records don’t seem to have an ip6tables version, so I’m a bit stuck for generating IPv6 records currently without adding a device between my server and the WAN connection :-( ).

In my own environment I hand out static DHCP leases to all my systems along with having configured reverse DNS so when doing a host report I can clearly see which host is responsible for what usage – if you have dynamically addressed hosts doing lots of traffic, things won’t be too helpful until you fix the leases for at least the high users.

To keep performance reasonable when working with huge NetFlow databases, FlatTraffic queries summary data for the selected date period and then caches into MySQL MEMORY tables to make subsequent reports quick and non resource intensive.

Please sir, can I have some more flow records?

I’m currently using it with NetFlow DBs with several months worth of data without issue, but it needs further and wider testing to determine how scalable it really is. I’ve worked to avoid putting much memory hungry logic in PHP, instead FlatTraffic tries to do as much as possible inside MySQL itself and uses some easily indexable queries.

To get started with FlatTraffic, visit the project page and install from either RPM, Source Tarball or direct from SVN – and send me feedback, good or bad. If you’re using another type of NetFlow collector other than flowd and would like support take a look at this page. Also note that there’s no reason why FlatTraffic couldn’t end up using other sources of data, it’s not architecturally limited to just NetFlow if you can get similar traffic details in some other form that would do fine.

If you end up using this application, please let me know how you find, always good to know what is/isn’t useful for people.

Munin 2.0.x on EL 5/6 with IPv6

2 Replies

I’ve been looking forwards to Munin 2 for a while – whilst Munin has historically been a great monitoring resource, it’s always been a little bit too fragile for my liking and the 2.x series sounds like it will correct a number of limitations.

Munin 2.0.6 packages recently became available in the EPEL repository, making it easy to add Munin to your RHEL/CentOS/OracleEL 5/6 servers.

Unfortunately the upgrade managed to break value collection for all my hosts, thanks to the fact that I run a dual-stack IPv4/IPv6 network. :-(

Essentially there were two problems encountered:

Firstly, the Munin 2.x master attempts to talk to the nodes via IPv6 by default, as it typical of applications when running in a dual stack environment. However when it isn’t able to establish an IPv6 connection, instead of falling back to IPv4, Munin just fails to connect.
Secondly, the Munin nodes weren’t listing on IPv6 as they should have been – which is the cause of the first problem.

The first problem is an application bug, or possibly a bug in one of the underlying libraries that Munin-node is using. I haven’t gone to the effort of tracing and debugging it at this stage, but if I get some time it would be good to fix properly.

The second is a packaging issue – there are two dependency issues on EL 5 & 6 that need to be resolved before munin-node will support IPv6 properly.

perl-IO-Socket-INET6 must be installed – whilst it may not be a package dependency (at time of writing anyway) it is a functional dependency for IPv6 to work.
perl-Net-Server as provided by EPEL is too old to support listening on IPv6 and needs to be upgraded to version 2.x.

Once the above two issues are corrected, make sure that the munin configuration is correctly configured:

host *
allow ^127\.0\.0\.1$
allow ^192\.168\.1$
allow ^fdd5:\S*$

I configure my Munin nodes to listen to all interfaces (host *) and to allow access from localhost, my IPv4 LAN and my IPv6 LAN. Note that the allow lines are just regex rather than CIDR notation.

If you prefer to allow all connections and control access by some other means (such as ip6tables firewall rules), you can use just the following as your only allow line:

allow ^\S*$

Once done, you can verify that munin-node is listening on an IPv6 interface. :-)

ipv4host$ netstat -na | grep 4949
tcp 0 0 0.0.0.0:4949 0.0.0.0:* LISTEN
ipv6host$  netstat -na | grep 4949
tcp 0 0 :::4949 :::* LISTEN

I’ve created packages that solve these issues for EL 5 & EL 6 which are now available in my repos – essentially an upgraded perl-Net-Server package and an adjusted EPEL Munin package that includes the perl-IO-Socket-Net package as a dependency.

Commuting in Sydney

Leave a reply

I’ve now been in Sydney for 5 weeks, settling into a new job, a new lifestyle and an entirely new city. Still very much in the tourist phase, there’s heaps we still need to see and do and only just starting to get settled really.

Sydney opera house!

The first two weeks here were spent staying with some of Lisa’s relatives out in Hornsby Heights – nice suburban area, but it takes a bus and a train in order to get into the CBD, which is a 1.5 hour per-way trip – 3 hours a day, or even more depressingly 15 hours a week just to get to and from work.

Because of this commute we haven’t really done much in the first two weeks whilst here, most of my time was either traveling or looking for a place to live with Lisa.

Sydney residents seem to complain about the train service, but it’s actually one of the best I’ve used, really the only thing that lets it down is the lack of a smart card system like Melbourne’s Myki

Instead it uses magnetic stripe tickets which are purchased via ticket machines at every station. These tickets can be for single trips, returns, weekly, monthly or even yearly – I had enough trouble keeping the paper ticket in one piece for a week, so unsure how well monthly or yearly tickets are going to last. :-/

Buttons! All the buttons!

The trains do vary in quality, some of the ones being run are a bit beat up and graffitied, but they always seem to be on time and pretty reliable.

It’s the first city I’ve been in which runs double decker trains – Sydney tends to run them as two pairs of locomotives, each with 4 carriages – effectively 8 carriages, or 16 if you count the fact that they’re double decker and probably fit about twice of the average carriage.

Inside one of the newer trains – note the upper and lower levels!

The suburban train network has the best views, most city training tends to be underground into subway stations, which do tend to be quite hot and cramped – thankfully Sydney has seemed to learn to build big, the newer stations in some suburbs are massive spacious underground caverns.

Unfortunately this large station entrance is an exception to the rule…

Of course you don’t necessarily have to take the underground rail….

MONORAIL!!! I get to walk past it every day on the way to work, so cool.

There’s an active cycling scene in Sydney, particularly around where I live and work, although I’m not sure how anyone survives cycling in hot Sydney days which are pretty horrific survival experiences at times. :-/

Whilst there are cycle lanes, they can be a bit scary as a pedestrian as a lot of cyclists seem to consider themselves immune to the cyclist traffic lights and will sometimes ride right at you against the red light whilst pedestrians are crossing…. There’s also a few wonderful design failures such as shared pedestrian/cyclist zones that are no larger than 1 bike each way at a time leading to people riding a bit too close for comfort.

Thankfully we have now found a place and we’ve finally settled in somewhat – now living in an apartment on Clarence St, right in the middle of the CBD which makes my job in Pyrmont only a short 20minute walk, meaning I can actually spend time enjoying my evenings.

Tree lined home street!

There’s a few pretty awesome perks to my commute, which takes me over Darling Harbour via Pyrmont Bridge and offers some pretty neat views.

Pyrmont bridge in the evening – note the monorail track which goes over it.

Harbor view – just off to the left is the rear end of a warship and a submarine at the maritime museum – but I’ll post more about these later on….

Generally things are going pretty well here, quite a culture shock compared to NZ, but we are getting out and about learning new places and things to see.

Wellington in New Years

Leave a reply

For those of you back in NZ, I will be flying to Wellington for New Years and will be in the city from 25th Dec until 4th of January. My lovely fiancee will be there for a subset of that with me, as she is spending some time with family visiting Sydney before flying into NZ.

If you’re around it will be great to catch up over coffee/beer/other for a few days whilst I’m there.

Unfortunately there won’t be an Auckland visit this trip, so if you’re up there, why not come down to Wellington for a few days instead? You know you want to. :-)