Since discovering Yahoo Pipes a couple of months ago it has really been off my to-do-list. Whether I am progressing is debatable but there is a feeling that the bits are starting to form a shape, if not necessarily fitting together easily.
Following the news from the head of the stream I have been playing with Pipes to create an effective monitoring capacity, with some success. On the face of it it seems simple enough but the truth seems a little less clear. There are a selection of examples other have put together and as the general instructions are not so good it seems best to reverse engineer one of these.
In a future post I am planning to concentrate on one of the Pipe apps, Open Calais semantic tagging tool. Its a brilliant concept and with time I am determined to make use of it.
The other element I want to use is a Google Docs spreadsheet, which I understand can receive RSS feeds. Going right to the end of the ‘food-chain’ I also need some sort of admin. This might need to be custom derived using something like PHP / MySQL or Drupal(all about which I know very little).
The one thing I am determined to do is blog about the experience as these new tools and are all free and very current I hope it will useful information.
Michael:
Tom Tague from Calais here.
First – Great! We love it when people think up ways to experiment with getting end user value out of Calais. Best of luck in your experiments.
Second – you are going to find some limitations with the Pipes implementation of Calais. Pipes is a great framework – but it puts some pretty onerous limits on how long a Pipes process can work. Basically – taking in a lengthy RSS feed requires that our Pipes connector go fetch the original content, run it through the Calais system and return the results to you. And Pipes does this serially for each item in the feed.
The net result is sometimes timeouts by Pipes – which can be frustrating.
If you want to do some high performance testing the best way is to hook up with a developer and use the Calais web service itself. This can easily handle many (many) thousands of transactions and give you results quite quickly.
As a back off position you may want to take a look at some of the other tools you can find in the Calais gallery at http://www.opencalais.com – in particular there are some batch submission tools that may do the trick for you.
Regards
Thanks Tom, I am excited about what the Calais could do however I am also coming to terms with my limitations, particularly with programing. Hence, I have started a search for a local developer to help out.