Thursday, May 20, 2010

Feeding your content through the pipe

Assuming you want to implement my blog-journey hack badly enough to have set up a feed and read an introduction to Yahoo Pipes: don't turn back now.

Here is the quick-and-dirty.
  1. Go to the pipe for my hack.
  2. If you've set your feed to Short, use Blog Journey (Short); if Full or Jump-Break, Blog Journey (Full or Jump Break).
  3. If you are using a Short feed, make a clone of the pipe--this will create your own editable copy of the pipe in your account. You will be prompted to log in to Yahoo or create a Yahoo account. (Sorry!)
      You don't need to do this for the other feed types or if all you are after is a list of your blog-post titles. (Short vs. Full vs. Jump Break here.)
  4. Put your feed url in the little box where indicated.
  5. Run the pipe to see what you've done.
  6. Click "Get as RSS" and pay no attention if you then see an incomprehensible babel of code. (Some browsers don't format the feed, but it doesn't matter.) That is not intended for human eyes.
  7. What you are after is the url (the web address in your browser bar) for that page. Copy it and port it over to a handy rss-to-JavaScript converter.
There is sometimes an additional step. Pipes sometimes generates RSS-feed urls that begin with "feed://" instead of "http://"

If this happens to you, correct the feed by deleting and replacing "feed://" with "http://" and paste that into the rss-to-JavaScript converter.

Update: The pipe for Full and Blog-Journey feeds now has an option for start and end dates to let you segment your blog journey by date. Read about that here.

Fall 2011 Update: Yahoo has tinkered with Pipes in a very unhelpful way. If you have, or expect to have, more than 100 posts in your blog, see this post for a description of the problem and workaround.


  1. Just want to make sure I am following you... I submitted my URL into you pipe and got a lengthy URL as following.


    I assume this is the RSS url?

    The problem I have now... only 25 items are listed...

  2. Hi, I might have figured out why only 25 items are retrieved... think it might have something to do with AdSense.

    So I created another blog without Adsense activated and now it returns 500 items- though it doesn't show all items in one page.

    The next problem I got is that, when trying to build a feed using Feed2JS. It shows only items on the first page.

    Any thought?

  3. Rat: Adsense interferes with the feed? Does it entail a Feedburner redirect or something? There ought to be a workaround for that. It's certainly not satisfactory to have to chose between Adsense, if you want that, and showing your feed.

    There have been several changes to blogger and other web services recently that complicate my blog-journey hack.

    First, Yahoo Pipes recently decided to paginate its feeds, with only 100 items per "page." The workaround is to make a script for each page and paste them serially. You can get the feed for each page by adding the "&page=" parameter to the end of the url for the feed. See this thread at the blogger help forum for a little more on that.

    Second, and potentially more seriously, Blogger now limits its feeds to the most recent 500 items. This is undocumented and may been been in effect for a while unbeknown to me. I am still hopeful that there is a workaround, but maybe not.

    This is obviously frustrating and, potentially, a deal breaker for folks looking for a solution that works, however tricky it may be to set up in the first place.

    I know I should write a post about this. Its just that I don't completely have a handle on every issue yet and I hate to give bad advice.

    If you learn anything else, I'd like to know. I hope you'll post anything you discover here, or email me via my profile page.

    In the mean time the solutions at MS-Potilas's blog are looking better all the time! Let's hope he gets the bugs ironed out of his approach.