Ready to Learn?Ex Libris products all provide open APIs

Tech Blog

 

Making Parallel API Calls to Alma

Josh Weisman on December 19th, 2018

It's often helpful to make several Alma API calls in parallel, for example when using APIs to make changes to lots of data, or to bring data from several sources for an online application. Writing parallel code has always been challenging. It used to be the realm of programmers with specialized expertise and experience. But recent features in many languages have made the task easier. We can leverage some of those new features to make it easier to perform parallel requests in Alma.

Asynchronous Programming in JavaScript

The asynchronous programming model in JavaScript has always relied on callbacks. Both in Node.js and on the client, the model has been to pass a callback to the asynchronous function which often accepts an error as the first parameter. So using the Request Node library to call the Alma API to retrieve and act on data might look something like this:

request(uri, function (error, response, body) {
  if (error) { 
    // handle error 
  } else {
    // do something to the data
    request({ uri: uri, method: 'PUT' }, 
      function(error, response, body) {
        if (error) {
          // handle error
        } else {
          return true;
        }
      }
    );
  }
}


The introduction of Promises and later the async/await syntax in ECMAScript 8 can be used to simplify the code. These patterns and syntax allow asynchronous JavaScript code to be written in a procedural manner, avoiding nested callbacks.

So our pseudocode above can be simplified to the following (using helper functions which will be described below):

try {
  var data = await alma.get(uri);
  // change data
  await alma.put(uri, data)
} catch (e) {
  // handle error
}

There are many introductions and primers on the web that do a great job of explaining how to use these features.

Parallel Alma API Requests

For our purposes, we'll use the Node.js asynchronous programming model to call an Alma API to retrieve a list of users, and then execute an action on the entire list in parallel.

Note: We'll need to remain aware of the concurrent request threshold and be sure not to open too many requests simultaneously. So we'll limit our bulk size to 25.

alma.js

The file alma.js contains several utility methods which simplify the calling of Alma APIs from Node.js. It exposes method for both “classic” JavaScript asynchronous functions with callbacks (i.e. get, put, post), and functions which return Promises that can be used with the await syntax (i.e. getp, putp, postp).

parallel.js

In the parallel.js script, we call the Get Users API in chunks of 25.

var data = await alma.getp('/users?limit=25);

Then we map the array of users to an array of Promises. This allows us to get a "handle" on all of the API calls which will be executed simultaneously, and then await for them all to complete before continuing with our code.

var actions = data['user'].map(async item => {
  return alma.getp(item.link+'?&view=brief').then(data => i++).catch(err => e++);
});

In our example, we simply increment a counter for each call, successful or otherwise. Of course in your code you'll likely perform a more meaningful action. We tell the program to wait until all of the simultaneous calls have been completed with this syntax:

await Promise.all(actions);

To continue processing additional records, we can loop on the "Get Users" call and bring the next set using the offset parameter.

There are many use cases for making parallel requests, and given the tools available today it's easier than ever.

The full code sample, including the utility functions in alma.js, is available in this Github Gist.