RSS is the new radar

As you know, the amount of information reaching your doors, both from within and outside the organisation, is growing rapidly. Yet you are expected to keep on top of it and discover the most valuable elements to pass on to your internal clients.

Software and systems have evolved to help you master traditional information sources such as email, but the business of handling RSS and Atom feeds is still in its infancy.

For those unfamiliar with the terms, Atom and RSS are dialects of the XML (Extensible Markup Language) data format for publishing information feeds. The information is usually added to periodically and readers tipped off when something new appears. You see them on blogs, news channels, auctions and podcasts to name but four.

The official RSS icon these days is a square with rounded corners, usually orange, containing a dot and two concentric arcs. A click on the icon or a right-click to pick up the feed address is the first step to receiving regular updates from that feed. Some sites also list reader-specific “chicklets”, which automatically add the feed to your preferred reader.

A huge variety of RSS feedreaders are available. Some are standalone, some run from the browser, some are hosted externally and some internally. Some are smart, some have little intelligence. Which you choose is a matter of personal preference and the functionality you seek.

In recent years most people might have kicked off with Bloglines or NewsGator. Nowadays, Google’s Reader has powered to the fore, together with standalone readers like GreatNews and RSSBandit, and those embedded in modern email and browser software.

But reading everything that comes your way can be an onerous way of working. In fact, it’s a direct route to a nervous breakdown. Far better to use software to do some filtering, aggregating and digesting on your behalf. Similarly, software makes it easy to publish your own feeds and discoveries to your clients.

The remainder of this article is a walk through the RSS process from the birth of new information to its delivery as part of a new feed. But remember: even though you can republish the content of a feed, you still need to be sensitive to copyright and authorisation issues.

INFORMATION AND FEEDS
Creating information is the easy bit. People and machines churn it out like there’s no tomorrow. First websites, then blogs were responsible for a continuous and ongoing explosion of new data – let’s not dignify it all with the word “information”.

The Burton Group recently produced a report entitled Techniques to Address Attention Fatigue and Info-Stress in the Too-Much-Information Age. The report says: “Searching for the nuggets of valuable information among the overwhelming mass of information an organisation gathers can be compared to prospecting for gold by panning in a river. Good prospectors create value from the nuggets they find, not the silt they discard.”

That, in essence, is your challenge as an information professional.

Useful, sometimes vital, information is buried inside the data streams. All you have to do is find it. And that’s where RSS (including Atom) comes to the rescue.

A feed encapsulates new nuggets of information inside an XML-formatted file. This sits at a persistent location to which other software can connect. The consistent format – a header followed by a series of information items, each with a series of predictable fields – makes it easy for the information to be extracted and processed or presented to a user.

Feeds can be created by blogs, wikis, content management systems, enterprise syndication systems, feed creators and even by hand.

Reading software can be given feed addresses explicitly or it can find them automatically by looking for a specific pointer in the HTML page header. Some reading software is able to discover feeds that match certain criteria supplied by the user. These typically call on external services such as Technorati, Del.icio.us or Google Blog Search.

You can set up your own searches by providing your criteria to an external service and receiving an RSS URL for each set of results. The external service then runs the searches at periodic intervals to keep the feed updated. Technorati popularised this approach with its Watchlists. Google’s news alerts can be delivered through RSS or Atom but its blog alerts can only be delivered by email. You could, if your email system publishes unread mails to RSS, create a new email address for each alert and pick up new blog posts that way.

A service called MySyndicaat is a neat way to gather feeds together in a river of news as a prelude to some serious filtering. It merges items from all the different feeds by date, automatically removing duplicates according to your rules. It also allows you to do a certain amount of content editing on the fly, plus some filtering. MySyndicaat can deliver a very clean RSS 2.0 feed. It is ideal, in fact, for pushing to a service like FeedRinse, which then allows you to apply multiple filtering criteria to any of the fields before publishing the result as a new feed.

Yahoo’s recently announced Pipes “feed remixer” promises much. It is a hosted service which gives users a way to program their own feed manipulations. It both overlaps and complements the services already mentioned. It can take in multiple feeds, dedupe them, remove or retain them according to criteria, sort the results and create a new feed. It gives the user intimate control over the preparation process, providing they’re prepared to get a tiny bit geeky. It would help, for example, to know how to construct advanced search URL lines for the major search engines.

Some organisations are already implementing enterprise products such as KnowNow Enterprise Syndication Solution, Attensa Feed Server and NewsGator Enterprise Server. They all offer organisations the ability to gather, manage and distribute internal and external RSS feeds from the centre, while still providing management and users with a degree of discretion over who receives which feeds.

If you are the one responsible for dealing with the incoming information stream, you will still have work to do. Marjolein Hoekstra is an RSS professional who has a background as a library automation specialist with the National Library in the Netherlands. She now blogs about RSS as CleverClogs (how apt!). She says that if she were setting up an RSS-based feed management system today: “I would set up topic radars, market radars, and people radars. I would make these available on the corporate network or to specific groups within my company. I’d assist in implementing feed readers, desktop alert systems and other means of instant notification.”

But the job doesn’t end with the creation of the feeds. She warns: “The major concern is that the list of topics might get outdated. Information professionals need to work with their clients to keep the radars current, perhaps by having mutual access to topic lists.”

This seems an ideal use for a wiki, where participants can subscribe to an RSS feed of recent changes or, indeed, to individual topic pages.

If you can find ways of monitoring feed usage, you have another weapon at your disposal when it comes to cleaning up the feeds or encouraging people to change their behaviour. A corporate system is bound to contain statistical and profiling information to see what gets attention and what gets ignored. But if you’re going it alone, you may need to analyse the popularity of your various feeds. This might require some sweet-talking of the IT department to get hold of the log files and some suitable analysis software.

Public services like FeedBurner, which can take your feeds and make them presentable to all manner of readers, are also able to show this kind of information. But you need to be careful. If you are mixing confidential internal information into the feeds, it must remain behind the organisation’s firewall.

HUMANS CREATE NEW FEEDS
Once you have cleaned up your incoming feeds and identified what you need to publish, you are ready to start working on the feeds you’re going to create and the various channels and protection mechanisms you’re going to put in place.

If you’re a bit of a masochist, you can construct your own feeds by hand or by using local software to automate the process. Blogs and wikis have feed-generation built in, or you can use a feed generator like Dan Brickin’s ListGarden. This seems to be the most usable of the more technical alternatives. Or you can get the web folk in your organisation to set you up with some server-based generation software such as the Planet reader/generator.

Always bear in mind you cannot deliver information to anyone who is not entitled to see it. The usual rules apply and it’s your job to protect the integrity of some feeds, possibly even encrypting them, as they flow in and out of your department.

You may want to create new RSS feeds or publish reading lists for people to import to their feed readers, so they can pick and choose from the feeds on offer. BlogBridge Feed Library offers a highly visual and friendly way of displaying feeds individually, while allowing your clients to pick up the OPML (Outline Processor Markup Language) for any individual folder.

Most, if not all, feed aggregators can export OPML. Some export all the feeds, others allow you some control over the process. BlogBridge allows you to arrange your feeds in Guides and export one or more at a time.

OPML offers a quick way of passing collections of feeds around. The problem with passing the files themselves around is that it reflects the feeds at one moment in time. It would be better if the reader could subscribe directly to an OPML file. Then, when you make changes to your feeds, you simply republish the OPML to the same URL to give users the most up-to-date version whenever they look there.

Even better for your users is a program like Grazr. This offers a great way to view OPML files. It always reads the file in real time, and shows the entire content of an OPML file in the form of an outline, a slider system or a traditional three-pane layout. What makes Grazr special is that it will recognise all possible components of an OPML file, not just the RSS feeds. So it can display plain text, HTML links, RSS feeds and include links to other OPML files. Used recklessly, this could cause information overload but it’s a very powerful way of sharing hierarchical reading lists.

Some enterprise systems can prioritise their feeds and send them to the most appropriate channel – email, instant message, mobile phone and so on. If you are constructing your RSS feed system by hand, you may find xFruits to your liking. Among other things, it will pump RSS to a mobile phone, email, the web, PDF and OPML. It will also pump OPML to a mobile phone.

RSS and Atom feeds can offer “always on” sensitivity to important information in the internal and external worlds. The clever bit is discovering, discriminating and delivering the right feeds to the right people in time for it to be useful. Many tools are out there to help you with this vital work, and you can be certain that more are in the pipeline.

In case you were wondering, this is real. The Home Office, for example, already monitors over 3,000 RSS feeds daily, many containing multiple posts. Staff apply business and political knowledge plus their information professional skills to reduce this to some 40 press office alerts a day and about 400 current awareness alerts a fortnight.

Like this story? Spread the news by clicking below:

 del.icio.us     Digg this     reddit!