[Rdap] MOVING Content: Institutional Tools and Strategies for Fedora 3 to 4 Upgrations

[Carol Minton Morris]

June 15, 2015

Read it online: http://duraspace.org/articles/2606

MOVING Content: Institutional Tools and Strategies for Fedora 3 to 4
Upgrations

The Fedora team has made tools that simplify content migration from Fedora
3 to Fedora 4 available to assist institutions in establishing production
repositories. Using the Hydra-based Fedora-Migrate tool — which was built
in advance of Penn State’s deadline to have Fedora 4 in production, before
the generic Fedora Migration Utilities was released —  Penn State’s
ScholarSphere [1] moved all data from production instances of Fedora 3 to
Fedora 4 in about 20 hours. The new ScholarSphere repository now takes
advantage of Fedora 4’s processing of large files. The ScholarSphere
service was collaboratively developed by The University Libraries and
Information Technology Services (ITS) to offer coordinated digital
stewardship support for Penn State e-research and e-science initiatives.

Fedora-Migrate [2] and Fedora Migration Utilities [3] are now available for
community testing to ensure that Fedora community members have the tools
they need to begin work toward establishing Fedora 3 to 4 repository tests
and production workflows.

Most pilot institutions tested Fedora-Migrate and Fedora Migration
Utilities while taking slightly different approaches to using the migration
tools to determine how to move towards establishing production Fedora 4
repositories.

Last year the National Library of Wales [4], The University of New South
Wales [5], York University [6], and Columbia University [7]developed pilot
projects to test the migration of content from Fedora 3 to Fedora 4 under a
variety of scenarios in preparation for the launch of production
repositories with superior Fedora 4 features.  Each institution took a
slightly different approach to scoping their test project and defining its
successful implementation.

Like Penn State’s ScholarSphere, Columbia University’s Academic Commons
took advantage of the Fedora-Migrate tool to determine if Fedora 3 data
could be modeled in Fedora 4. The Fedora-Migrate tool, which is designed
primarily for Hydra implementations with an emphasis on those based on
Sufia, iterates over existing Fedora 3 objects using the Rubydora gem. For
each object it finds, it creates a new object with the same id in Fedora 4
and proceeds to migrate each datastream, including versions if they are
defined, and verifies the checksum of each. Permissions and relationships
are migrated as well but using different procedures due to the changes to
permissions and relationships in Fedora 4. Developers at Columbia noted
that future migrations would require additional modifications to
Fedora-Migrate because more Fedora 3 features are used as part of Columbia
content models.

The National Library of Wales (NLW) tested the general Migration Utilities
to process their Fedora 3 FOXML documents and convert them to Fedora 4
resources. The NLW pilot project tested the migration of a large Newspaper
title (187,331 objects) and a modern digitized Journal with complex rights
issues. The NLW team found that the the Migration Utilities were easy to
use. The team tested the utilities and were able to provide valuable
feedback to the developers by logging issues in the JIRA bug tracker. This
feedback allows for improvement of the migration utilities to support a
broader range of use cases.

University of New South Wales (UNSW) directed their pilot efforts towards
coming up with a strategy for upgrading the Library’s Fedora 3-based
repositories. A key part of this strategy was the development of a Fedora 4
data model that would remain compatible with existing UNSW repositories but
would also align with the Portland Common Data Model [8]. The project has
established a test Fedora 4 instance that implements the preliminary data
model, and later in 2015 the first UNSW legacy repository will be migrated
to Fedora 4.

York University Libraries has developed YUDL which is an institutional
Islandora repository that runs on the latest version of all Islandora
Foundation modules. The repository is as close to a stock/generic Islandora
instance as possible. The aim of the project was to serve as a basis for a
generic Islandora Fedora 3 to Fedora 4 upgration using Fedora Migration
Utilities. To this end, the project contributed greatly to the development
of the Fedora Migration Utilities, including detailed mappings from Fedora
3 to Fedora 4 data models.

These Fedora 4 migration tools, Fedora-Migrate [2] and Fedora Migration
Utilities [3], are now available thanks to community efforts. Please
download, test and provide feedback to help improve them!

[1] https://scholarsphere.psu.edu/
[2] https://github.com/projecthydra-labs/fedora-migrate
[3] https://github.com/fcrepo4-labs/migration-utils
[4] https://wiki.duraspace.org/display/FF/Upgration+Pilot+-+NLW
[5] https://wiki.duraspace.org/display/FF/Upgration+Pilot+-+UNSW
[6] https://wiki.duraspace.org/display/FF/Upgration+Pilot+-+York+University
[7]
https://wiki.duraspace.org/display/FF/2015-04-20+Upgration+Pilot+Update#id-2015-04-20UpgrationPilotUpdate-ColumbiaUniversity
<https://wiki.duraspace.org/display/FF/Upgration+Pilot+-+Columbia>
[8] https://wiki.duraspace.org/display/FF/Portland+Common+Data+Model
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kunverj.com/pipermail/rdap/attachments/20150615/3cc766db/attachment.html>