Mdxjp

概要 I created an “Image Collection Management” tool for easily trying out IIIF features. https://pocket.webcatplus.jp https://pocket.webcatplus.jp/ This article introduces how to use this tool. Collection Management Creating a Collection Click the “New Collection” button on the dashboard Enter the required information: Collection name (required): Give it a descriptive name Description (optional): Describe the collection contents Visibility setting: Choose public/private Click “Create” Editing a Collection Click the “…” menu on the collection you want to edit from the collection list Select “Edit” Update the information and click “Save” Deleting a Collection Click the “…” menu on the collection you want to delete from the collection list Select “Delete” Click “Delete” in the confirmation dialog Note: Deleting a collection will also delete all items it contains ...

Overview This is a memo about hosting TEI/XML files on S3-compatible object storage. Specifically, we target the mdx I object storage. https://mdx.jp/mdx1/p/about/system Background We are building a web application (Next.js) that loads TEI/XML files and visualizes their content. When the number and size of files were small, they were stored in the public folder, but as these grew larger, we considered hosting them elsewhere. There are many options for storage locations, but this time we target mdx I’s S3-compatible object storage. ...

Overview I investigated how to implement IP address restrictions for MDX.jp’s object storage. The following article was written by AI after verifying the implementation. Introduction This article explains how to configure access restrictions by specific IP addresses for the DDN EXAScaler S3-compatible object storage service provided by MDX.jp. Object Storage Security Layers DDN EXAScaler S3-compatible storage has three main security layers: Access Key and Secret Key: Basic authentication credentials Bucket Policy: Bucket-level access control Access Control List (ACL): Object-level access control To implement IP address restrictions, “Bucket Policy” is used. ...

Overview This is a memo on building a proxy server for mdx I’s object storage. Background The mdx I user guide explains the following. https://docs.mdx.jp/ja/index.html#オブジェクトストレージ This is a DataDirect Networks manual describing the API specifications handled by the S3 Data Service (DDN EXAScaler S3 Data Service) provided by mdx. Please review it together with the object storage usage examples in Tips. https://docs.mdx.jp/ja/_downloads/b5d961f2c152387fa10ed951d5278f47/S3 Data Services 5.2.7 API Reference Guide.pdf I loaded this into Google NotebookLM and checked the CORS implementation status. The following is the answer. ...

Overview This article describes how to add mdx.jp object storage to GakuNin RDM storage. Procedure mdx.jp Submit a usage application for mdx.jp object storage and note down the access key and secret key. GakuNin RDM Enable S3 Compatible Storage. Select mdx S3DS as the S3-compatible service and enter the access key and secret key you noted earlier. A list of buckets will be displayed. Select the bucket you want to connect to. ...

Overview This is a memo on how to store Omeka S files in mdx.jp object storage. Base Module We use the following module as a base, which enables integration with Amazon S3. https://omeka.org/s/modules/AmazonS3/ This module provides an extension for storing media files such as images and videos handled by Omeka S in Amazon S3. However, since it did not support specifying an endpoint, it was not possible to target object storage services like mdx.jp. ...

Overview This is a note on my experiment with serving IIIF images using mdx.jp object storage and IIP Image (IIIF Image Server). This is a continuation of the following article. Docker Version of IIP Image A Docker image for the IIPImage server is available at the following link, so I will use that. https://hub.docker.com/r/iipsrv/iipsrv Refer to the following article and others for installing Docker. https://qiita.com/Marron-chan/items/570c7c7baaae3b4d6b11 Execution Following the previous article, mount the mdx.jp object storage as follows. ...

Overview This is an example of how to create AIPs using Archivematica for files in Alfresco. Below is a demo video of the deliverable. https://youtu.be/7WCO7JoMnWc System Configuration For this project, I used the following system configuration. There is no particular significance to using multiple cloud services. Alfresco was built on Azure, referencing the following article. Archivematica and object storage use mdx.jp, and the analysis environment uses GakuNin RDM. ...

Overview I confirmed the workflow for processing Dataverse data with Archivematica, so here are my notes. Background Archivematica provides a feature to input data from Dataverse. https://www.archivematica.org/en/docs/archivematica-1.17/user-manual/transfer/dataverse/ I learned about this feature at the following lecture, so I decided to try it out. https://www.kulib.kyoto-u.ac.jp/bulletin/1402322 Dataverse I used the Demo Dataverse that was also used in the following article. I uploaded the following data. https://demo.dataverse.org/dataset.xhtml?persistentId=doi:10.70122/FK2/IHQZL3 From here, download both the image data itself and the JSON data. Go to the Metadata tab and select JSON from Export Metadata. ...

Overview I had the opportunity to use IIIF manifest files stored in mdx.jp object storage from NestJS, so here are my notes. Background After a quick check of mdx.jp object storage, it appeared that CORS settings could not be configured, making it difficult to use IIIF manifest files uploaded to mdx.jp object storage directly from other viewers. /en/posts/ad76f58db4e098/#注意（corsの許可） Therefore, the approach is to load the IIIF manifest files uploaded to object storage from NestJS and return them. ...

Overview This article explains how to use processing_config in Archivematica transfers. Background In Archivematica transfers, you can select a processing_config. The following shows that you can choose from three options: “automated,” “default,” and “mdx.” This can be configured in “Processing configuration” under the “Administration” menu. For example, the following is a configuration example designed for interacting with mdx.jp’s S3-compatible storage. By selecting the target storage for “Store AIP location” as shown below, when this processing configuration is selected, the AIP will be saved to that storage. ...

Overview I had the opportunity to build a character detection model using YOLOv11x and the Japanese Classical Character (Kuzushiji) Dataset, so this is a memo of the process. http://codh.rois.ac.jp/char-shape/ References Previously, I performed a similar task using YOLOv5. You can check the demo and pre-trained models at the following Spaces. https://huggingface.co/spaces/nakamura196/yolov5-char Below is an example of application to publicly available images from the “National Treasure Kanazawa Bunko Documents Database.” ...

Overview I had the opportunity to train YOLOv11 classification (kuzushiji recognition) using mdx.jp, so here are my notes. Dataset The following “Kuzushiji Dataset” is used as the target. http://codh.rois.ac.jp/char-shape/book/ Dataset Creation The dataset is formatted to match the YOLO format. First, data separated by book title is flattened and merged. # c | l a e s x s d p e o C f r l t a c # f # f s r i o s e " l o r i a . e u f t . s t f c o i # o s i e / p i l u f s h c _ d = u l s t p . u a d a t e p o r m t t a t g _ = u s c i a i i t a l d i t . o n k l o a / o i n f _ p n t e . n s * b r i f a t ( d c : e / ( t l i t i f i o t c i = q e l h n " r p ( h n d . e . u C s y s a p " m s e e o ( ( e r u . ( p = x p f f l a t . f l i y " i f c _ / i i f s i { l , t f d l t " t n o e e i a e ( { s g u , i r l t s " o ( t n s e a ) / u o { p o p / _ / : " t u f u u u * p d ) p t i t t t / a a [ u p l _ p _ * t t - t u e d u f . h a 2 _ t } i t i j ) s ] d _ r _ l p e i f t } f e g t r i o / i _ " " } l { l p / e { c e a { ) o l ) t c : u s h l t } , s p " } u , o / t u { _ e t f f x p i i i u l l s t e e t _ . } _ d s " o i p ) k r l = ) i T : t r ( u ' e / ) ' ) [ - 1 ] } " Next, the dataset is split using the following script. ...

Overview I had the opportunity to run a local LLM using mdx.jp’s 1GPU pack and Ollama, so this is a memo of the process. https://mdx.jp/mdx1/p/guide/charge References I referred to the following article. https://highreso.jp/edgehub/machinelearning/ollamainference.html Downloading the Model Here, we target llama3.1:70b. After the download is complete, it becomes selectable as shown below. Usage Example We use the following “Shibusawa Eiichi Biographical Materials.” https://github.com/shibusawa-dlab/lab1 Using the API Documentation was found at the following location. ...

Overview I had the opportunity to add mdx.jp object storage to Archivematica, so this is a note for reference. Background In the following article, I described how to configure Amazon S3 as both a processing target and AIP storage destination for Archivematica. This time, based on those steps, I tried connecting mdx.jp object storage. Configuration Method Configure as follows. For S3 Endpoint URL, set https://s3ds.mdx.jp. For Access Key ID to authenticate and Secret Access Key to authenticate with, use the Access Key and Secret Key obtained from the following. ...

Overview This is a personal note on how to deliver IIIF images using mdx.jp object storage and Cantaloupe Image Server, one of the IIIF image servers. Background In the following article, I introduced how to deliver images using mdx.jp object storage. In the following article, I introduced how to deliver images stored in Amazon S3 using Cantaloupe Image Server. By combining these approaches, we aim to address the cost challenges of IIIF image delivery in digital archives. ...

Overview I had the opportunity to use mdx’s object storage, so this is a memo. https://mdx.jp/ Pricing The pricing for fiscal year 2024 is as follows. https://mdx.jp/guide/charge It costs 0.01 points (yen) per GB per day, which is approximately 0.3 yen per GB per month. Application Method & Usage with s3cmd The following official tutorial video was helpful. https://www.youtube.com/watch?v=IN_4NS9hO2Y Using Cyberduck The video above introduces file operations using command-line tools. ...

Update History 2024-05-22 Added “Adding the user who executes Docker commands to the docker group.” Overview mdx is a data platform for industry-academia-government collaboration co-created by universities and research institutions. https://mdx.jp/ This time, we will use an mdx virtual machine to run NDL Classical Japanese OCR. https://github.com/ndl-lab/ndlkotenocr_cli Project Application This time, I selected “Trial” as the project type. With “Trial,” one GPU pack was allocated. Creating a Virtual Machine Deploy This time, I selected “01_Ubuntu-2204-server-gpu (Recommended).” ...