Yahoo has done the world a favour by introducing a hosted “interactive feed aggregator and manipulator”, called Pipes . While the name will go down well with propellerheads, it’s a dreadful choice for the target audience. The service majors on the ordinary, not very techie user.
From a visual editor, users can drag and drop RSS and Atom feeds (others are planned), user inputs and various operations straight into a working area and, by wiring them together, create a new output as an RSS feed. It’s a “data mashup” or “feed remixer”.
All the clever bits are constructed from modules containing text boxes and drop-down menus, minimising the chance of failure. At the moment, there are 25 functional modules, including Flickr (search), Google base (search), Sort, BabelFish (translate), Union (merge feeds), Unique (de-dupe), Yahoo Local (classified listings) and Yahoo Search .
Here’s an example of Pipes in action: Craig Cmehil, a web developer and evangelist for the SAP Developer Network (SDN), wanted to pick up external blogs containing specific tags but only if the blogger was a member of SDN. He used Pipes to take three feeds from the Technorati blog tracker where the tags included blogger and either sdn, bpx or sap. These were filtered to eliminate any who weren’t genuine SDN bloggers. The posts in the resulting aggregated feed were then de-duped and sorted by title. The results can be viewed at Cmehil’s hosted Pipe or the feed picked up in RSS, RDF, Atom and JSON (JavaScript Object Notation) formats.
Cmehil created this non-trivial Pipe as an experiment to see how easy it was to build. The answer appears to be “very”. Okay, he’s an experienced web developer but, it’s not rocket science. Even if it took you four times as long, a genuine application would still pay back the time investment.
A quick way to get going would be to search pipes.yahoo.com/pipes/ for a Pipe that approximates what you want to do. If you like the way it works, copy it and tweak it to suit. You’ll need a (free) Yahoo ID to do this.
Right now, Pipes is hosted by Yahoo so you won’t be able to work on internal feeds. But enhancements are already planned, including programmatic access to the Pipes engine. This may provide opportunities for tighter integration with internal information feeds. Yahoo is planning other input formats, and new output formats, such as maps. It will also add new processing modules of its own as well as those developed by third parties.
Pipes appears to be part of a wider movement to give knowledge workers new powers over their working environment. IBM’s QEDwiki lets users compile applications out of a library of widgets and information sources, all wrapped up in a wiki-based collaborative environment.
Teqlo is as easy to use as Pipes, but is more powerful, being a kind of platform for widgets of all kinds. It orchestrates the cooperative working of elements of application functionality – spreadsheets, calendars and maps, for example.
Large chunks of application development are being put within the grasp of users, which could mark a step-change in productivity, just as spreadsheets did all those years ago.
All