S3

Overview In the following article, I introduced the “Image Collection Management” tool designed for easily trying out IIIF features. This time, I will introduce the technologies used behind this tool. Introduction The Image Collection Management tool is a web application for managing and publishing image collections in the IIIF (International Image Interoperability Framework) format, an international standard. This article explains the technical implementation of this tool, with a focus on the IIIF specification implementation and handling of geospatial information. ...

Overview This article describes how to add mdx.jp object storage to GakuNin RDM storage. Procedure mdx.jp Submit a usage application for mdx.jp object storage and note down the access key and secret key. GakuNin RDM Enable S3 Compatible Storage. Select mdx S3DS as the S3-compatible service and enter the access key and secret key you noted earlier. A list of buckets will be displayed. Select the bucket you want to connect to. ...

Overview This is a memo on how to store Omeka S files in mdx.jp object storage. Base Module We use the following module as a base, which enables integration with Amazon S3. https://omeka.org/s/modules/AmazonS3/ This module provides an extension for storing media files such as images and videos handled by Omeka S in Amazon S3. However, since it did not support specifying an endpoint, it was not possible to target object storage services like mdx.jp. ...

Overview This is a note on my experiment with serving IIIF images using mdx.jp object storage and IIP Image (IIIF Image Server). This is a continuation of the following article. Docker Version of IIP Image A Docker image for the IIPImage server is available at the following link, so I will use that. https://hub.docker.com/r/iipsrv/iipsrv Refer to the following article and others for installing Docker. https://qiita.com/Marron-chan/items/570c7c7baaae3b4d6b11 Execution Following the previous article, mount the mdx.jp object storage as follows. ...

Overview I had the opportunity to mount mdx.jp object storage as a file system using s3fs, so this is a memo of the process. 1. Prerequisites This guide targets Ubuntu. Installing s3fs s s u u d d o o a a p p t t u i p n d s a t t a e l l s 3 f s Setting up authentication credentials ...

Overview I had the opportunity to use IIIF manifest files stored in mdx.jp object storage from NestJS, so here are my notes. Background After a quick check of mdx.jp object storage, it appeared that CORS settings could not be configured, making it difficult to use IIIF manifest files uploaded to mdx.jp object storage directly from other viewers. /en/posts/ad76f58db4e098/#注意（corsの許可） Therefore, the approach is to load the IIIF manifest files uploaded to object storage from NestJS and return them. ...

Overview This article explains how to use processing_config in Archivematica transfers. Background In Archivematica transfers, you can select a processing_config. The following shows that you can choose from three options: “automated,” “default,” and “mdx.” This can be configured in “Processing configuration” under the “Administration” menu. For example, the following is a configuration example designed for interacting with mdx.jp’s S3-compatible storage. By selecting the target storage for “Store AIP location” as shown below, when this processing configuration is selected, the AIP will be saved to that storage. ...

Overview I had the opportunity to add mdx.jp object storage to Archivematica, so this is a note for reference. Background In the following article, I described how to configure Amazon S3 as both a processing target and AIP storage destination for Archivematica. This time, based on those steps, I tried connecting mdx.jp object storage. Configuration Method Configure as follows. For S3 Endpoint URL, set https://s3ds.mdx.jp. For Access Key ID to authenticate and Secret Access Key to authenticate with, use the Access Key and Secret Key obtained from the following. ...

Overview This is a note on how to connect GakuNin RDM with Amazon S3 and process files with Archivematica. https://rcos.nii.ac.jp/service/rdm/ Background In the following article, I described how to use Amazon S3 as a processing target in Archivematica. This allows you to upload files and folders to a specified bucket and use them as processing targets in Archivematica to create AIPs and DIPs. However, this approach required creating an IAM user for each project member. ...

Overview This is a 2024 edition article on building an IIIF Image Server using AWS serverless applications. Background The following repository called serverless-iiif is publicly available. Using this repository, it is claimed that a cost-effective and infinitely scalable IIIF Image Server can be built using AWS services. https://github.com/samvera/serverless-iiif I introduced how to use it as of 2022 in the following article, but today’s service has become more user-friendly. Method There are several build methods, but for a GUI-based approach, refer to the following. Basic setup follows the instructions on the site below. Here, I introduce the procedure including custom domain setup with CloudFront and Route 53. ...

Overview This is a personal note on how to upload multiple files to mdx.jp object storage. It references the following video. https://youtu.be/IN_4NS9hO2Y Preparation Working on macOS. b r e w i n s t a l l s 3 c m d Configuration (please refer to the video for details.) s 3 c m d - c o n f i g u r e Batch Registration (Sync) The following syncs files in the local rekion folder to s3://rekion/iiif/. ...

Overview This is a personal note on how to deliver IIIF images using mdx.jp object storage and Cantaloupe Image Server, one of the IIIF image servers. Background In the following article, I introduced how to deliver images using mdx.jp object storage. In the following article, I introduced how to deliver images stored in Amazon S3 using Cantaloupe Image Server. By combining these approaches, we aim to address the cost challenges of IIIF image delivery in digital archives. ...

Overview I had the opportunity to use mdx’s object storage, so this is a memo. https://mdx.jp/ Pricing The pricing for fiscal year 2024 is as follows. https://mdx.jp/guide/charge It costs 0.01 points (yen) per GB per day, which is approximately 0.3 yen per GB per month. Application Method & Usage with s3cmd The following official tutorial video was helpful. https://www.youtube.com/watch?v=IN_4NS9hO2Y Using Cyberduck The video above introduces file operations using command-line tools. ...

Overview I needed to redirect from one URL to another and was able to accomplish this using Amazon S3 and Route 53, so this is a memo of the process. Method This method uses an S3 bucket for the redirect and Route 53 for DNS configuration. The steps are explained below. Step 1: Amazon S3 Bucket Configuration Create a new bucket in Amazon S3. The bucket name should match the domain name you want to redirect (e.g., example.com). In the bucket properties, select “Static website hosting.” In the “Static website hosting” options, choose “Redirect requests” and enter the redirect destination URL (e.g., http://example.net). Step 2: DNS Configuration in Route 53 In Route 53, open the hosted zone for the domain name you want to redirect. Create a new record set. Select A as the record type. Set “Alias” to “Yes.” As the alias target, select the static website hosting endpoint of the S3 bucket configured in Step 1 (e.g., example.com.s3-website-us-east-1.amazonaws.com). With this setup, when someone accesses the specified domain, they will be redirected to the configured URL. This method is simple yet effective for redirecting from one domain to another URL. ...

Overview I configured Strapi with the following plugin to store media in S3. https://www.npmjs.com/package/@liashchynskyi/strapi-provider-upload-s3-cloudfront At that time, the following error occurred and images were not displayed. R e f u s e d t o l o a d t h e i m a g e ' h t t p s : / / x x x / u p l o a d s / y y y . j p g ' b e c a u s e i t v i o l a t e s t h e f o l l o w i n g C o n t e n t S e c u r i t y P o l i c y d i r e c t i v e : " i m g - s r c ' s e l f ' d a t a : b l o b : d l . a i r t a b l e . c o m " . I was able to resolve this issue by modifying ./config/middleware.js, as described in the following article. ...

To list S3 buckets using AWS CLI and delete buckets based on a specific pattern, you can follow the steps below. Here, we explain how to delete buckets whose names start with wby. Prerequisites AWS CLI is installed. Appropriate AWS credentials and access permissions are configured. Step 1: List Buckets First, use the installed AWS CLI to list all S3 buckets: a w s s 3 l s Step 2: Delete Matching Buckets To delete buckets starting with wby, use a shell script to filter matching buckets and delete them. ...

Overview This is a memo on how to configure Amazon S3 files and folders as processing targets in Archivematica, and save the resulting AIPs to S3. Using S3 as storage is expected to facilitate integration with other systems and expand options for long-term AIP preservation. The following article from Wellcome Collection was helpful. https://docs.wellcomecollection.org/archivematica/administering-archivematica/bootstrapping Amazon S3 Configuration Create a bucket. This time, I created a bucket named archivematica.aws.ldas.jp in the us-east-1 region. ...

Overview I introduced how to use the Docker version of Cantaloupe in the following article. To use this Docker-based Cantaloupe in a (non-large-scale) production environment, connection to Amazon S3 and SSL support are required. Here is an example of how to do this. Connecting to Amazon S3 The official documentation is available at: https://cantaloupe-project.github.io/manual/5.0/sources.html#S3Source The following Japanese article is also available: For the Docker version covered here, information was found at: ...

Overview This is a note on how to serve images stored in Amazon S3 using Cantaloupe Image Server, one of the IIIF image servers. As an alternative method for serving images stored in Amazon S3, I also introduced an approach in the following article. (The tools may have been updated since the article was written, so the instructions may not work exactly as described.) Configuration The official manual is available at the following link. ...

I tried to access a specific AWS S3 bucket using Cyberduck, referencing the following article. https://dev.classmethod.jp/articles/specify_s3_folder_iam_cyberduck/ However, when I opened the macOS version of Cyberduck and pressed the “New Connection” button at the top of the screen, the form for entering bucket information and other details was not displayed. After investigating, I found the following issue. https://github.com/iterate-ch/cyberduck/issues/11154 It indicated that you should open a bookmark instead, as follows: Please refer to Access third party buckets. To set a default path, create a new bookmark instead of choosing Open Connectoin. ...