I’ve been working with Ben Raymond of Grevillea colour distribution fame on an entry to the GBIF Ebbe Nielsen Challenge, which aims to inspire scientists, informaticians, data modelers, cartographers and other experts to create innovative applications of open-access biodiversity data.
You can see a sneak peak at our entry at peterneish.github.io/gbif-soundscape/. Its a system that pulls together sound recordings of animals to build up a soundscape from various localities.
Below are the slides from a talk I gave at VALA 2014 on how we went about trialling Linked Data at the Victorian Parliamentary Library. There is also a longer paper that goes along with the talk.
Here’s another visualisation of some data from the 2013 Australian Federal Election.
I wanted to see how consistent voting patterns were across booths within an electorate. From handing out how to vote cards at my local polling booth I had the feeling that not every booth is the same.
Fortunately the Australian Electoral Commission publishes a live feed of all data by booth. Its the same data that the news outlets use, so its pretty good. It follows the Election Markup Language standard, so extracting the data was not too hard. The AEC had added in their own elements, but they used namespaces which made it fairly simple to process using a fairly basic perl script.
Data is based on first preferences only. Because of the way heatmaps work, if booths are closer together the intensity increases, so to some extent the heat map is determined by the layout of the booths. However, distinct patterns are discernible if you explore the data. Only parties that registered votes in at least 200 booths have been included.
Absentee voting at capital cities booths are mapped at the booth rather than in the electorate that the vote was for, so capital cities tend to show high results for every party.
You can see the live demo and the source code is at GitHub.
The Australian Electoral Commission have released the data for preference allocation and makes a good subject for a visualisation.
Using a PERL script I was able to group the ticket preferences into groups and then create a matrix in a format suitable for input to a d3js chord diagram.
Getting useful data is always a compromise. Some parties have multiple tickets (ie they split their preference flows in two or more tickets). Independents do not have parties, but work in a group. I grouped independents by using the highest preferenced candidate for that ticket. The coalition have more than one party, so I had to combine these under coalition.
To measure how highly a party was preferenced, I took the average position on the ballot for each member of the party and average these if there were more than one ticket. For some parties with split tickets this meant that they might end up without preferencing any party particularly highly.
Next was how to visualise the data. A chord diagram seemed the natural way to show how parties preference each other. The problem is that there is too much data to show every preference allocation (by definition every party preferences every other candidate). So I needed to draw the line on how much data to show. I arbitrarily decided that any party that averaged a position in the top 25% of the ballot order was highly preferenced by the other party.
The visualisation shows some interesting things. Whether there is a symmetrical or asymmetrical relationship between parties can easily be seen. Als,o the wider the party is around the circle, the more other parties preference it. The ALP and coalition have fairly narrow widths while the bullet train party and family first are relatively wide.
Of course preferences are a lot more complex than shown here. The order of preferences and how they flow once quotas are allocated can have a subtle and profound affect on the election outcome. If you are considering voting below the line Antony Green has some good advice.
My visualisation is available at http://peterneish.github.io/preferences/ and the code can be found on GitHub.
At the moment it is fairly basic and is not entirely intuitive (no back-button or bookmarking support, inconsistent actions), but these things are fairly easy to add. D3js takes a while to get your head around, but the combination of a well thought out API from the Atlas of Living Australia combined with d3js and twitter bootstrap has allowed me to produce this application relatively easily.
The demo application and source are hosted on GitHub.
A quick note on installing Linux Mint on an Acer eMachines netbook (em350) for anyone going down the same path.
This netbook is a few years old and the installed OS Â (Windows 7 starter) had slowed almost to a crawl. After some painful session trying to work out what was causing excessive load under Windows I decided that installing Linux was the best option.
Continue reading Linux Mint on an Acer em350 netbook
The Horizon LMS from SirsiDynix provides a utility to export marc records from the system. This is quite robust, except if your bib records are quite large and have a lot of subject tems. The following VBscript allows you to export smaller batches of marc records and skip over known bad records.
Continue reading Exporting records from Horizon
Yesterday I spent a really interesting morning at Museum Victoria attending and presenting at a workshop on Linked Open Data in Libraries, Archives and Musuems (LOD-LAM) organised by Culture Victoria. As usual with these things, there were varying levels of technical and background knowledge on the topic. However, I think the level of the presentations was spot on and judging by the number of questions and discussions happening there was a lot of interest in this topic.
Mia Ridge (@mia_out) gave an excellent introduction to Linked Data and how it is used in libraries, galleries and museums. She’s put together a wiki with links to all sorts of useful stuff.
Below are the slides from my talk on activities at the Victorian Parliamentary Library (astute viewer will notice they are mostly lifted straight out of my VALA talk).
Here’s the slides from another talk from VALA2012 where I talked about how we’ve been using OpenCalais at the Victorian Parliamentary Library to add tags and semantic data to one of our databases. You can also see the talk here or download a longer paper that goes with the talk.
I was lucky enough to attend VALA2012 and also present a couple of papers. The first was about some Open Source software that we created at the Victorian Parliamentary Library in order to track reference requests. Its not quite finished, but as soon as its wrangled into a coherent jumble of code, I’ll be putting it up on http://github.com/pov.
I’m quite happy with the system and it got some good comments from people at VALA. Here are the slides I presented that show a bit of the user interface. Programmed using the Play! Framwork.