Overview
I had the opportunity to use pagination in IIIF collections with IIIF Presentation API v2, so this is a memo of the process.
Background
IIIF collections allow you to provide a list of multiple manifest files (and collections) as shown below.
https://iiif.io/api/presentation/2.1/#collection
When the number of target manifest files becomes large, it becomes difficult to deliver them in a single IIIF collection.
To address this, there is a pagination specification available at the following link, which we will use.
https://iiif.io/api/presentation/2.1/#paging
Pagination
The above page introduces examples like the following.
First, the first JSON indicates the total number of manifest files in the collection with total, and provides a link to the first subset of manifest files with first.
The collection specified by first is the second JSON, which uses startIndex and next to indicate the current position and a link to the next subset of manifest files.
Implementation Example Using Drupal and Elasticsearch
Here is an implementation example of the above pagination using a combination of Drupal and Elasticsearch. This assumes that Drupal and Elasticsearch are connected using drupal/elasticsearch_connector as introduced in the following article.
In Elasticsearch, you can specify an offset using from, but it does not work correctly when exceeding max_result_window (default: 10,000 records).
To address this, you can use search_after to retrieve content following a specific item.
For example, under certain sort conditions, you can obtain the ID (field_id) of the 20,000th content item and use a query like the following to retrieve content from the 20,001st item onward. The searchAfter contains the ID of the 20,000th content item.
However, when using Elasticsearch alone with the default max_result_window under specific sort conditions, some ingenuity is needed to obtain the ID (field_id) of, for example, the 20,000th content item.
Therefore, for retrieving the Nth content item under specific sort conditions, we used Drupal’s REST API. Using Drupal’s Views, we allowed users to specify the offset and configured it to return a single content item.
Example: /api/xxx?offset=20000
This allowed us to obtain the content ID needed for search_after, and by combining it with the Elasticsearch query described above, we were able to build IIIF collections with pagination.
Notes
We tested the IIIF collection implemented above with the Presentation API Validator.
https://presentation-validator.iiif.io/
As a result, the first JSON representing the entire collection showed the following warnings.
Also, the second JSON representing a subset of the collection showed the following warnings.
Furthermore, the first JSON representing the entire collection could not be displayed correctly in Universal Viewer or Mirador.
I would like to continue investigating whether there is an issue with the implementation introduced here.
Summary
I hope this article serves as a helpful reference when presenting a large number of manifest files (or sub-collections) using IIIF collections.