OpenLinkz

I had a bunch of trouble toward the end of last week with my live stock quote widget that retrieves stock prices from Yahoo every 30 seconds throughout the day.

Having run into problems with the WidgetBox proxy service that allows widgets to retrieve data from other domains I had converted the widget into a Google Gadget. I then wrapped the Google Gadget within the WidgetBox widget so all those folks who were using the WidgetBox widget wouldn't be left with a non-functioning widget.

Last week, the Google Gadget began acting up. Using FireFox and Firebug, I was able to determine that the response from Yahoo in a simple CSV format was still functioning properly but the callback wasn't being executed within the gadget to actually read the response. I use the simple _IG_FetchContent function to execute the request and process the response.

In my experimenting, I discovered that using a Yahoo Pipe and rendering the output as JSON worked perfectly. Still, this seemed like too much overhead for such a simple request.

As it turns out, I had to make an adjustment to the URL at Yahoo that is used to request the stock price data. Where previously I used the following URL:

http://finance.yahoo.com/d/quotes.csv

Now I have to use this URL:

http://download.finance.yahoo.com/d/quotes.csv

If you type these URLs into your browser (with the requested symbols and request parameters), both of them would return the requested data. I can only think that some kind of redirection was taking place on the Yahoo site that made the Google Gadget function reject the response.

In summary, it seems these cross-domain calls can be awfully touchy. I apologize to any users who experienced an outage as a result of the problems in this widget. Hopefully, both Yahoo and Google will avoid making any changes for a while.

In my work to maintain a widget that retrieves stock quote data continuously throughout the day I have ended up experimenting with several different approaches.

As discussed in my last post, WidgetBox on its own will not provide the required functionality so we migrated to Google gadgets. In working with the Google gadget libraries, I initially used their XML-HTTP request as a work-around by preventing the test for an XML structure in the response. Eventually, I found that this approach was actually not as reliable as I had hoped. In looking for an alternative, I found that Google also had a function that retrieves simple text data across domains.

Since the raw stock quote data from Yahoo! is CSV text, this text retrieval function turns out to be the most efficient in terms of speed, being light weight and, most importantly, will run continuously and reliably as long as a user is viewing the page, even if it's all day.

The Google gadget libraries are clearly more flexible and provide someone willing to dig through the documentation greater control and power over the functionality within a widget/gadget.

I recently wrote about WidgetBox breaking my live stock quote widget by disallowing data in CSV text format when using their proxied XML-HTTP request functionality. (See "WidgetBox tightens up AJAX interface")

As an alternative, I set up a Yahoo! pipe to supply the data in RSS format, which seemed to work well.

The problem is when a person stays on the page hosting the widget in order to watch the quotes update throughput the day. After half an hour or so, the widget stops working. There appears to be some kind of strange caching going on that prevents any data from being returned. I tried the usual work-around: appending in the query string a key/value pair containing a random number but to no avail.

My eventual solution was to create a Google gadget with the same logic that retrieved data from the original source in CSV text format. This gadget was then embedded inside the WidgetBox widget.

With a little fooling around, I was able to pass the user configuration data through the WidgetBox widget to the Google widget and, once again, things seem to be working fine.

The bottom line is that RSS just does not seem to lend itself to continuous data retrieval for applications like real-time quotes. It is hard to be sure exactly where the problem lies but either WidgetBox or Yahoo! or both treat RSS as an information feed, not a raw data feed. As such, the assumption is that one wouldn't want to continuously retrieve an RSS feed as I have been trying to do.

So far, it appears the Google libraries offer more flexibility than the WidgetBox approach but at the cost of more programming effort. For those with good Javascript expertise, however, programming gadgets should not be too hard.