Recycling Pending Promises in Node.js
With the package “pending-promise-recycler”, we can save precious resources and avoid performing the same operation again and again by recycling pending promises.
Imagine the following scenario.
We have an API that sometimes makes calls to a 3rd party service; for example, to retrieve a list of stores. Let’s assume that retrieving this list of stores is a very expensive and slow operation, averaging a response time of about 12 seconds.
As soon as we get the response, we probably want to cache it somehow so that we don’t make our API consumers wait for so long every time they need to get a list of stores.
But what would happen when 1,000 of our API consumers fire a call to our store list endpoint at the same time?
- The first request comes in. It realizes there isn’t a store list in the cache yet, so it fires the call. Just to remind you, this call will take around 12 seconds.
- The second request comes it one millisecond later. There is still no store list in the cache, so it fires another call. This will take another 12 seconds.
- The other 998 requests will come in, not finding anything in the cache. Another 998 calls will be fired.
- There will be 1,000 operations to save the store list in the cache, 999 of them being completely unnecessary.
- All the 1,000 requests will take around 12 seconds to respond.
- The 3rd party API that provides us with a store list will probably not very happy about our hammering of their service.
Wouldn’t it be nice if our API would somehow be aware that there is an ongoing, pending promise to a list of stores created by the first request, and all the other 999 requests would be fulfilled with that same single promise?
Introducing pending-promise-recycler
To facilitate the recycling of pending promises I’ve written the Node.js module pending-promise-recycler.
Let’s imagine we have an API like the one described before, and a very slow or expensive operation similar to the one to retrieve a list of stores:
With pending-promise-recycler, we can “wrap” our fetchSomethingExpensive function to make it aware of any other pending version of itself:
In this example with four concurrent executions of recyclableFetch(), our very expensive fetchSomethingExpensive function gets only executed and resolved once, but satisfies all the four executions.
How does it work?
The module pending-promise-recycler keeps track of all ongoing, pending promises using a Map.
const registry = new Map();We need to somehow identify each possible function that might be wrapped. By default, they will be individually identified by their function name and a concatenation of their arguments. Of course, this might work for simple functions and systems, but ideally we will define our own identified by either using a string or a function — the behavior of which can be found in a separate section in the documentation.
const identifier = typeof options.keyBuilder === 'function'
? options.keyBuilder(func, ...args) : options.keyBuilder;We will check if there is a pending promise for the given function in the registry. If there is one, we will return it.
if (registry.has(identifier)) {
return registry.get(identifier);
}If there is no pending promise in the registry, we will start the execution of the function and store it in the registry.
const res = func(...args);
registry.set(identifier, res);We will wait for it to resolve (or be rejected!) and then we will remove it from the registry and return its result.
try {
await res;
} finally {
registry.delete(identifier);
}
return res;
