So have you been hearing about mashups recently? A data mashup is basically a web app that combines data from several sources into a single source. An example would be to combine a photo feed and google maps. The result is a map with points showing exactly where your photos where taken. Sounds cool hu? Well yahoo has created a visual tool that makes creating mashups relatively easy and fun called Yahoo Pipes. And of course Yahoo Pipes uses the YUI AJAX library so it's full of drag-n-drop goodness. Below is screenshot of the interface
So you maybe thinking, OK this looks pretty neat, but I still don't really get it. You can think of Yahoo Pipes as it's own functional programming language. It can take input through various means such as RSS feeds, CSV files, Web Pages, or Web Services. It also contains operators and helper functions to manipulate your data. Finally you can output your data to something a web browser can read (like an RSS feed). It also has a built in debugger so you can debug your "application" each step of the way.
Still fuzzy? Let me give you an example of a Pipe I just created to choose a winner on my Flickr Challenge Group. The winner must meet the following criteria:
Sounds simple enough, but it is an extremely tedious process that just gets worse as more people join and there is no automated way on flickr to do this. Here is how I automated the process:
Below are the results in an RSS feed format. By clicking the link, you get taken to the photo.
You can checkout and view the source from this page:
http://pipes.yahoo.com/pipes/pipe.info?_id=e9e03c4f1214d97f5453aa825b9aab3e
So this is just one example of what can be done, but think of all the possibilities. And because it supports web services, you can even deal we real data objects! So next time your thinking of regexping a web page to mine some data, try automating it with a Pipe first.