Pachube and Yahoo Pipes part 1: show me the data

I’ve just been playing around with Yahoo Pipes for the first time in ages to see if it might be useful in mangling data feeds from Pachube. (Would have been useful to have tried this out before homecamp but never mind!) Early signs looked good- falling off a log looks difficult in comparison to getting Pachube data into a Yahoo Pipe. Having said that, finding data feeds to start working with is not so easy. If I want to start doing something with power data, I have to manually find the relevant feeds on the Pachube site with a tag search. It would be great to be able to get a feed of data feeds; that would be useful for keeping up with new data feeds that get added to Pachube as well as being a useful input to a Yahoo Pipe.

So the simple example pipe for part 1 does just this kind of aggregation except that, because it’s a hack, it only includes a specified set of feeds. The default user input list is just the feed IDs I manually copied from that tag search for ‘power’:

• Fetch Pachube Feeds Feed

It should work using my API key if you don’t have one, but you’ll need your own if you clone the feed to do something more interesting.

Coming up in part 2: doing something useful with the data.

Advertisement

Location, location, location

I discovered a very nice alternative to Rightmove for finding houses on the web this weekend. As the saying goes, the three most important things to take in to account when looking for a house are location, location and location, so using a map to find properties for sale seems like an obvious choice:

http://ononemap.com/

I like maps anyway, so I’m hooked. It also beats Rightmove with support for RSS feeds. How can Rightmove not have a feed? Maybe I’m just missing it. Still, there is one tiny problem with OnOneMap: it doesn’t have as many properties on it. Even our house isn’t on there, although I’m planning to pester the estate agent on Monday to see if I can persuade them to give it a try.

Using Yahoo pipes on the Freeview OTA update schedule

Having a Sony Freeview box that has a tendency to lock up when the software update feature enabled means I should check the Engineering Channel Download Schedule every week. I don’t. Seems like an ideal exercise for my first play with Yahoo pipes!

My first impression of using pipes has been pretty good, although pulling out the right information from the source for a web page is a bit of a pain (don’t look too closely at the regular expressions!!) but that’s not Yahoo’s fault. (Maybe some more filters tailored to sorting out the wheat from the HTML chaff would be nice though.) I was also trying to use the last updated text on the site as the pubDate, but have so far failed to get that to work; I could get a separate feed of one item with a nicely formatted date in, but haven’t yet worked how to add that to each item in another feed.

Still, I did get something that looks like it works, although my guess is that I might need to tweak it over the next few weeks as new schedules come out. So, if you want to keep up with Freeview updates, or can suggest any improvements to my regular expressions, here’s the ‘Freeview OTA updates‘ pipe!

Update: found an example that could work for getting the date into the pipe. Also tinkering with a new pipe, which got off to a false start with a “failed to parse non HTML page” message for the Romanse real time traffic page. Luckily the BBC traffic site seems more cooperative.

Update: sorry about the small blockage in the Freeview pipe. I called dynarod out and it seems to be flowing nicely again now! (12 Feb 2008)