::::: : the wood : davidrobins.net

Using the Goodreads API

Technical ·Monday March 30, 2009 @ 23:51 EDT (link)

I was playing around with Goodreads' API. Their only example uses Ruby, which is way to schizo a language for my tastes. So I looked around on CPAN and found Net::OAuth, which I had some trouble with, but then I found OAuth::Lite, which was easier to use. OAuth is "an open protocol to allow secure API authorization in a simple and standard method from desktop and web applications." It's only necessary for calls that need to modify data; read-only requests just need a developer key.

I ended up using WWW::Mechanize (and WWW::Mechanize::Gzip) since I needed to do some screen-scraping, or thought I did. One thing I noticed when looking at the source is that Goodreads' site is much better organized (cleaner, better URLs, less badly-written Javascript) than weRead. I looked at the source of the Ruby OAuth module to find out about some defaults, and came up with this OAuth::Lite constructor call:

my $oalc = OAuth::Lite::Consumer->new(
  consumer_key       => YOUR KEY HERE,
  consumer_secret    => YOUR SECRET HERE,
  site               => 'http://www.goodreads.com',
  request_token_path => '/oauth/request_token',
  access_token_path  => '/oauth/access_token',
  authorize_path     => '/oauth/authorize',
);

I had a lot of trouble since Goodreads was randomly failing my authentication attempts; first I thought it was timing out the access key really fast, or that I was sending invalid requests, but then I determined (through much experimentation, and showing that the same request can both fail and succeed) that it just randomly fails (401 Invalid OAuth Request, without a WWW-Authenticate header). When that happens, you just need to keep trying.

For the first (OAuth) request, you need to get a request token which can be parlayed into an access token:

my $reqt = $oalc->get_request_token;
my $url = $oalc->url_to_authorize(token => $reqt->token);

The user has to go to that URL and approve your application (if it's a web application, you can redirect them there and pass a callback URL for it to send the user to afterward). When it's approved, get an access token:

$oalc->get_access_token;

This token (available as $oalc->access_token) can be persisted (e.g. to a file or database: save the token and secret data) and then next time used to create a token to use to authenticate:

my $acct = OAuth::Lite::Token->new(token => $token, secret => $secret);
$oalc->access_token($acct);

Now you can make requests, using code like this, which returns a hash of information (id, link, name) about the current user:

sub get_user {
  my $res = $oalc->request(url => 'http://www.goodreads.com/api/auth_user');
  unless ($res->is_success) {
    print "Failed to get user: ".$res->status_line."\n".$res->content."\n";
    return;
  }
  my $u = $xs->parse_string($res->content)->{user};  # id, link, name
  return $u;
}

Even after the first successful request, subsequent requests can randomly fail. I have no idea why this is happening; I've asked Goodreads in the developer forum and elsewhere, but they aren't saying anything, although they were very helpful with some other requests, e.g. documenting some undocumented functionality for adding a read date to a review.