Adding Annotations to Videos Using iiif-prezi3

Overview

This is a note on how to add annotations to videos using iiif-prezi3.

Adding Annotations

Amazon Rekognition’s label detection is used.

https://docs.aws.amazon.com/rekognition/latest/dg/labels.html?pg=ln&sec=ft

Sample code is available at the following link.

https://docs.aws.amazon.com/ja_jp/rekognition/latest/dg/labels-detecting-labels-video.html

In particular, by setting the aggregation in GetLabelDetection to SEGMENTS, you can obtain StartTimestampMillis and EndTimestampMillis.

However, please note the following.

When aggregated by SEGMENTS, information about detected instances with bounding boxes is not returned.

Data Used

The video “Prefectural News Vol. 1” (Nagano Prefectural Library) is used.

https://www.ro-da.jp/shinshu-dcommons/library/02FT0102974177

Reflecting in the Manifest File

Assume that a manifest file has already been created, referring to the following article.

The following script adds a VTT file to the manifest file.

The following manifest file is generated.

https://d1u7hq8ziluwl9.cloudfront.net/sdcommons_npl-02FT0102974177/manifest_label_seg.json

Viewer Display Example

Here is a display example in a viewer.

https://d1u7hq8ziluwl9.cloudfront.net/mirador/index.html?manifest=https://d1u7hq8ziluwl9.cloudfront.net/sdcommons_npl-02FT0102974177/manifest_label_seg.json

The source code published in “IIIF Video Annotation” by UTokyo Digital Humanities is used.

https://dh.l.u-tokyo.ac.jp/activity/iiif/video-annotation

References

By setting the aggregation in GetLabelDetection of Amazon Rekognition to TIMESTAMP, bounding boxes can be obtained for some annotations.

Using this, annotations can be displayed together with rectangles as shown below.

Summary

I hope this is helpful for adding annotations to video files using IIIF.