Archivematica

In Archivematica, metadata schemas other than Dublin Core (DC) can be embedded into the AIP’s METS.xml. This guide explains how to include non-DC metadata such as EAD and MODS in a Transfer using source-metadata.csv, and verifies via API that they are correctly stored in the AIP. Table of Contents Background and Purpose How source-metadata.csv Works XML Validation Feature Verification 1: MODS-Only Metadata Registration Verification 2: Simultaneous EAD + MODS Registration Storage Format of Non-DC Metadata in METS.xml Verification 3: Metadata Addition via Reingest Summary Background and Purpose In a standard Archivematica Transfer, Dublin Core metadata described in metadata/metadata.csv is stored as <dmdSec> in METS.xml. However, in actual digital archive operations, there are use cases where metadata schemas other than DC need to be handled: ...

Overview This is a memo on how to add Normalization rules in Archivematica’s Preservation planning. Background When ingesting images with the .jpg extension into Archivematica, there were cases where tif files were not created for preservation, despite having a rule to create tif files for items with Format of JPEG as shown below. I checked the task details from the history screen shown below. The results were as follows. ...

Overview This is an example of how to create AIPs using Archivematica for files in Alfresco. Below is a demo video of the deliverable. https://youtu.be/7WCO7JoMnWc System Configuration For this project, I used the following system configuration. There is no particular significance to using multiple cloud services. Alfresco was built on Azure, referencing the following article. Archivematica and object storage use mdx.jp, and the analysis environment uses GakuNin RDM. ...

Overview I confirmed the workflow for processing Dataverse data with Archivematica, so here are my notes. Background Archivematica provides a feature to input data from Dataverse. https://www.archivematica.org/en/docs/archivematica-1.17/user-manual/transfer/dataverse/ I learned about this feature at the following lecture, so I decided to try it out. https://www.kulib.kyoto-u.ac.jp/bulletin/1402322 Dataverse I used the Demo Dataverse that was also used in the following article. I uploaded the following data. https://demo.dataverse.org/dataset.xhtml?persistentId=doi:10.70122/FK2/IHQZL3 From here, download both the image data itself and the JSON data. Go to the Metadata tab and select JSON from Export Metadata. ...

Overview This article explains how to use processing_config in Archivematica transfers. Background In Archivematica transfers, you can select a processing_config. The following shows that you can choose from three options: “automated,” “default,” and “mdx.” This can be configured in “Processing configuration” under the “Administration” menu. For example, the following is a configuration example designed for interacting with mdx.jp’s S3-compatible storage. By selecting the target storage for “Store AIP location” as shown below, when this processing configuration is selected, the AIP will be saved to that storage. ...

Overview I created a Python library for visualizing the contents of Archivematica METS files. For example, it visualizes aggregated results of processes (premis:event) performed during AIP creation, as shown below. Background In the following article, I introduced METSFlask, a web application for exploring Archivematica METS files in a human-friendly way. What I created this time is a library version of the functionality provided by METSFlask, making it easier to use outside of Flask. ...

Overview I had the opportunity to add mdx.jp object storage to Archivematica, so this is a note for reference. Background In the following article, I described how to configure Amazon S3 as both a processing target and AIP storage destination for Archivematica. This time, based on those steps, I tried connecting mdx.jp object storage. Configuration Method Configure as follows. For S3 Endpoint URL, set https://s3ds.mdx.jp. For Access Key ID to authenticate and Secret Access Key to authenticate with, use the Access Key and Secret Key obtained from the following. ...

Overview This is a note on how to connect GakuNin RDM with Amazon S3 and process files with Archivematica. https://rcos.nii.ac.jp/service/rdm/ Background In the following article, I described how to use Amazon S3 as a processing target in Archivematica. This allows you to upload files and folders to a specified bucket and use them as processing targets in Archivematica to create AIPs and DIPs. However, this approach required creating an IAM user for each project member. ...

Overview When registering metadata to folders in Archivematica, I found that Japanese folder names need to be avoided, so here are my notes. Metadata By preparing a /metadata/metadata.csv file like the following, you can include metadata in the AIP. filename dc.type objects/aaa Folder objects/aaa/MARBLES.TGA Image At this point, if aaa is prepared with a Japanese name, the metadata for that record was not registered. Summary I hope this serves as a useful reference for those experiencing similar issues. ...

Overview Archivematica sample data is stored in the following repository. https://github.com/artefactual/archivematica-sampledata Notes Archivematica supports multiple input types, including Standard, Zipped directory, and Zipped bag. The data in the above repository is helpful as a reference for what files and folders to prepare for each of these types. Example: Registering with a CSV File Containing Metadata The manual documentation is at the following location. https://www.archivematica.org/en/docs/archivematica-1.16/user-manual/transfer/transfer/#transfers-with-metadata A sample is available here. https://github.com/artefactual/archivematica-sampledata/tree/master/SampleTransfers/CSVmetadata ...

Overview For starting Archivematica, using Docker was the easiest method. Installation instructions are documented at the following link. https://github.com/artefactual/archivematica/tree/qa/1.x/hack Following the steps worked well for installation, but I am documenting the points where I encountered failures as a memo. Elasticsearch container When starting the Elasticsearch container, instructions were given to set vm.max_map_count. https://github.com/artefactual/archivematica/tree/qa/1.x/hack#elasticsearch-container If this setting is forgotten, the Elasticsearch container would not start, as shown below. d [ A e e e d o + ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ t l l l e c ] t a a a p k C C C C C C C C C C a s s s e e R o o o o o o o o o o c t t t n r u n n n n n n n n n n h i i i d n t t t t t t t t t t i c c c e c n a a a a a a a a a a n s s s n o i i i i i i i i i i i g e e e c m n n n n n n n n n n n a a a y p g e e e e e e e e e e t r r r o r r r r r r r r r r o c c c f s 1 h h h a e 0 a a a a a a a a a a a - - - i / m m m m m m m m m m r 1 1 1 l u 0 - - - - - - - - - - c e p m a n c e f g a a a h d y r g l l i e r r r i s c i a a t a c c c v t q h n m s s r h h h e o l i x a t - m i i i m - v - v i 1 a v v v a s 1 e 1 d c n e e e t t m - s d m m m i a a 1 e - a a a c r t a 1 t t t a t i r i i i - : c c c c c d a h a a a a c - - - - - s o s 1 m d m h n t c a c b t o p s p o a r - h - a | i a c b s r n g l o e d [ E [ e e i a r - 2 R 1 r - e r v 1 0 R ] s n d e , 2 O : a e t - r 4 R m r - 1 - a - : m - v 1 1 r 0 a e i c 6 [ x l c h - 1 a e i 1 ] v s - v 8 i t 1 e T b r i m 0 o t c a 2 o u s R R R R C R R R C R t : t a e u u u u r u u u r u i 1 s l a n n n n e n n n e n c 6 t r n n n n a n n n a n a : r m c i i i i t i i i t i - 4 a e h n n n n e n n n e n m 0 p m - g g g g d g g g d g c , o 1 p 3 c r - 7 h y e c 3 e x l ] c a i i [ k r t e I s e e n N a d t F f s - O a ( 1 i v 7 , ] l m 8 [ e . ) a o d m r . a c e x h . _ i e m v . a e N p m o _ a d c t e o i E u c n n a v t - i m r [ c o 6 p n 5 - m 5 s e 3 e n 0 r t ] v e i r s - 1 ] t , o [ o a a r m l c - o h n w i o , v d e e i m ] n a c t u r i s e c i a a n s - g e s t [ t o 1 o r ] a a g d t e a - t l s a e e a r p s v a t i t c h [ e s 2 - , 6 1 2 , m 1 o 4 c u 4 l n ] a t m s a v [ d [ - / 1 u , s r e / l s a h s a t r i e c / s e e l a a r s c t h i - c 1 s , e a f r i c t h s / - d 1 a , t a g e a / r d m e a v n / d r - o 1 o , t ) m ] y ] s , q l n - e 1 t , u n s g a i b n l x e - _ 1 s p a c e [ 1 1 . 4 g b ] , n e t t o t a l _ s p a c e [ 4 8 . 2 g b ] , t y p e s [ e x t 4 ] 0 0 0 0 0 0 0 0 0 0 . . . . . . . . . . 0 0 0 0 0 0 0 0 0 0 s s s s s s s s s s Summary Regarding Docker startup, it is noted that it is designed for developers, and care should be taken when using it in production environments. We hope this serves as a useful reference for using Archivematica. ...

Overview When inputting files with Japanese filenames into Archivematica with default settings, a filename like “ユースケース公募提案書.docx” is converted as follows: yu-suke-suGong_Mu_Ti_An_Shu_.docx This article explains how to customize this filename conversion. Overview The filename conversion is performed in the following file: https://github.com/artefactual/archivematica/blob/qa/1.x/src/MCPClient/lib/clientScripts/change_names.py Specifically, the following line: d e c o d e d _ n a m e = u n i d e c o d e ( b a s e n a m e ) An example of running this in Google Colab is available here: ...

Overview I will try out the following METSFlask. https://github.com/tw4l/METSFlask It is described as follows: A web application for human-friendly exploration of Archivematica METS files Usage You can try it on the following site. http://bitarchivist.pythonanywhere.com/ Here is the result of uploading a METS file. In this case, since only one Word file was stored, information about one original file is displayed. Clicking the View button navigates to the detail screen. ...

Overview In this article, I try out the following tool. https://github.com/artefactual-labs/AIPscan This tool is described as follows: AIPscan was developed to provide a more in-depth reporting solution for Archivematica users. It crawls METS files from AIPs in the Archivematica Storage Service to generate tabular and visual reports about repository holdings. It is designed to run as a stand-alone add-on to Archivematica. It only needs a valid Storage Service API key to fetch source data. ...

Overview I had the opportunity to add Japanese translation to a documentation site built with sphinx, so this is a note for reference. The target is the following. https://github.com/artefactual/archivematica-storage-service-docs Method First, fork the target repository. Next, clone it. g c i d t a c t l o o m n - e d o h c t s t p s : / / g i t h u b . c o m / n a k a m u r a 1 9 6 / a r c h i v e m a t i c a - s t o r a g e - s e r v i c e - d o c s Here, we create a Python virtual environment. ...

Overview This is a memo on how to configure Amazon S3 files and folders as processing targets in Archivematica, and save the resulting AIPs to S3. Using S3 as storage is expected to facilitate integration with other systems and expand options for long-term AIP preservation. The following article from Wellcome Collection was helpful. https://docs.wellcomecollection.org/archivematica/administering-archivematica/bootstrapping Amazon S3 Configuration Create a bucket. This time, I created a bucket named archivematica.aws.ldas.jp in the us-east-1 region. ...

Introduction In the following article, I described how to set up Archivematica on EC2. This time, we configure a custom domain and enable HTTPS. Custom Domain Configuration This time, we assign the domains matica.aws.ldas.jp and storage.aws.ldas.jp to the IP address. We use Route 53. Obtaining an SSL Certificate s y y e u u u r d m m t o b i i o s n n t u s s t t c a a e l l r l l t o e c n p e l e r y l t - b r o - e t w l e e b a r s o e o t - w / u s r / s h a r e / n g i n x / h t m l - d m a t i c a . a w s . l d a s . j p - d s t o r a g e . a w s . l d a s . j p Web Server Configuration: Nginx Installation v i / e t c / n g i n x / c o n f . d / a r c h i v e m a t i c a - a n d - s t o r a g e . c o n f Configuration: ...

Background I was able to perform the process from Transfer through AIP download using the Archivematica API, so I am documenting it here. Previously, I wrote separate articles about using the Archivematica API and the Storage Service API. This time, I combine the above to perform the process from Transfer through AIP download. Method I documented the method in the following notebook. https://colab.research.google.com/github/nakamura196/ndl_ocr/blob/main/ArchivematicaのAPIを使ってみる.ipynb Summary I hope this serves as a helpful reference for using the Archivematica API. ...

Overview I encountered a bug where clicking Browse in Archivematica did not allow me to view folders or files. Here I introduce the cause and solution. /transfer/ Symptom /administration/storage/ E r r o r r e t r i e v i n g l o c a t i o n s : i s t h e s t o r a g e s e r v e r r u n n i n g ? P l e a s e c o n t a c t a n a d m i n i s t r a t o r . ...

The AIP created by Archivematica includes a README.html file. I translated this file using DeepL (with some manual corrections). There may be many errors, but I hope you find it helpful. Archivematica AIP Structure This Readme file describes the basic structure of the Archival Information Package (AIP) generated by Archivematica. Acronyms AIP = Archival Information Package METS = Metadata Encoding and Transmission Standard OAIS = Open Archival Information System PDI = Preservation Description Information ...