When one starts working with migrations, it is easy to be overwhelmed by so many modules providing migration functionality. Throughout the series, we presented many of them trying to cover one module at a time. We did this to help the reader understand when a particular module is truly needed and why. But we only scratched the surface. Today’s article presents a list of migration related Drupal modules, all good for Drupal 8, for quick reference. Let’s get started.

Core modules

At the time of this writing, Drupal core ships with four migration modules:

• Migrate: provides the base API for migrating data.
• Migrate Drupal: offers functionality to migrate from other Drupal installations. It serves as the foundation for upgrades from Drupal 6 and 7. It also supports reading configuration entities from Drupal 8 sites.
• Drupal Migrate UI: provides a user interface for upgrading a Drupal 6 or 7 site to Drupal 8.
• Migrate Drupal Multilingual: is an experimental module required by multilingual translations. When they become stable, the module will be removed from Drupal core. See this article for more information.

Migration runners

Once the migration definition files have been created, there are many options to execute them:

• Migrate Tools: provides Drush commands to run migrations from the command line. It also exposes a user interface to run migrations created as configuration entities. It offers support for migration groups and tags. The module depends on Migrate Plus.
• Migrate Run: provides Drush commands to run migrations from the command line. It does not offer support for migration groups, but tags are supported.  The module does not depend on Migrate Plus.
• Migrate Upgrade: provides Drush support for upgrading a Drupal 6 or 7 site to Drupal 8.
• Migrate Manifest: provides a Drush command for running migrations using a manifest file. See this article for an example of using this module for Drupal upgrades.
• Migrate Scheduler: integrates with Drupal Core’s Cron API to execute migrations on predefined schedules.
• Migrate Cron: exposes a user interface to execute migrations when Cron is triggered. At the time of this writing, the module does not execute dependent migrations. Follow this issue for updates.
• Migrate source UI: provides a form for uploading files to use as source for already defined CSV, JSON, and XML migrations. At the time of this writing, it seems that JSON and XML migrations are not being detected. Follow this issue for updates.

Source plugins

The Migrate API offers many options to fetch data from:

• Migrate (core module): provides the `SqlBase` abstract class to help with fetching data from a database connection. See this article for an example. It also exposes the `embedded_data` plugin which allows the source data to be defined inside the migration definition file. It was used extensively in the example migrations of this series. It also offers the `empty` plugin which returns a row based on provided constants. It is used in multilingual migrations for entity references.
• Migrate Plus: combining various plugins, it allows fetching data in JSON, XML, and SOAP formats. It also provides various plugins for parsing HTML. See this article by Benji Fisher for an example. There is also a patch to add support for PDF parsing.
• Migrate Source CSV: allows fetching data from CSV files.
• Migrate Google Sheets: leverages Migrate Plus functionality to allow fetching data from Google Sheets.
• Migrate Spreadsheet: allows fetching data from Microsoft Excel and LibreOffice Calc files.
• Migrate Source YAML: allows fetching data from YAML files.
• WP Migrate: allows fetching data from a WordPress database.

Destination plugins

The Migrate API is mostly used to move data into Drupal, but it is possible to write to other destinations:

• Migrate (core): provides classes for creating content and configuration entities. It also offers the `null` plugin which in itself does not write to anything. It is used in multilingual migrations for entity references.
• Migrate Plus: provides the `table` plugin for migrating into tables not registered with Drupal Schema API.
• CSV file: example destination plugin implementation to write CSV files. The module was created by J Franks for a DrupalCamp presentation. Check out the repository and video recording.

Development related

These modules can help with writing Drupal migrations:

• Migrate (core): provides the `log` process plugin. See this article for an example of its use.
• Migrate Devel: offers Drush options for printing debug information when executing migrations. It also provides the `debug` process plugin. See this article for an example of its use.
• Migrate Process Vardump: provides the `vardump` plugin. It works like the `debug` plugin.

Field and module related

• Migrate Media Handler: provides migration process plugins to facilitate the migration into Drupal 8 media entities. The source can be Drupal 7 file or image fields. It also supports inline file embeds in rich text. It leverages the DOM parsing plugins provided by Migrate Plus.
• Media Migration: provides an upgrade path from Drupal 7 to Drupal 8 media entities. The source can be image fields and fields attached to media and file entities.
• Migrate File Entities to Media Entities: migrates Drupal 8.0 file entities to Drupal 8.5 media entities.
• Migrate Files: provides process plugins for migrating files and images.
• Webform Migrate: provides a plugin to help migrating from the Drupal 6 and 7 versions of the Webform module.
• Migrate HTML to Paragraphs: turns HTML markup into paragraph entities.
• Commerce Migrate: offers a general purpose migration framework for bringing store information into Drupal Commerce.
• Address: offers a process plugin to migrate data into fields of type address. It also provides an upgrade path from Drupal 7’s Address Field module. See this article for an example.
• Geofield: offers a process plugin to migrate data into fields of type geofield. See this article for an example.
• Office Hours: offers a process plugin to migrate data into fields of type office hours.
• Workbench Moderation to Content Moderation: migrates configuration from one module to the other.

Modules created by Tess Flynn (socketwench)

While doing the research for this article, we found many useful modules created by Tess Flynn (socketwench). She is a fantastic presenter who also has written about Drupal migrations, testing, and much more. Here are some of her modules:

• Migrate Directory: imports files from a directory into Drupal as managed files.
• Migrate Process S3: downloads objects from an S3 bucket into Drupal.
• Migrate Process URL: provides a process plugin to make it easier to migrate into link fields.
• Migrate Process Vardump: helps with debugging migrations.
• Many process plugins that wrap PHP functions. For example: Migrate Process Array, Migrate Process Trim, Migrate Process Regex, Migrate Process Skip, and Migrate Process XML.

Miscellaneous

• Feeds Migrate: it aims to provide a user interface similar to the one from Drupal 7’s Feeds module, but working on top of Drupal 8’s Migrate API.
• Migrate Override: allows flagging fields in a content entity so they can be manually changed by side editors without being overridden in a subsequent migration.
• Migrate Status: checks if migrations are currently running.
• Migrate QA: provides tools for validating content migrations. See this presentation for more details.

And many, many more modules!

What did you learn in today’s blog post? Did you find a new module that could be useful in current or future projects? Did we miss a module that has been very useful to you? Share your answers in the comments. Also, I would be grateful if you shared this blog post with others.

Next: Introduction to Drupal 8 upgrades

This blog post series, cross-posted at UnderstandDrupal.com as well as here on Agaric.coop, is made possible thanks to these generous sponsors: Drupalize.me by Osio Labs has online tutorials about migrations, among other topics, and Agaric provides migration trainings, among other services.  Contact Understand Drupal if your organization would like to support this documentation project, whether it is the migration series or other topics.

Thank you for being awesome!  Use this page to register with your discount. If you want to attend more than one training, fill out the form once per training.

Ben and Micky spoke at DrupalCon Seattle 2019 and New England Drupal Camp 2019 about scaling community conversations and decision-making.  Rather than having slides of fine print and links, we're posting resources here in this blog post.

Here's a short-enough-to-write-on-a-business-card link for this page – agaric.coop/scaling-community – for easier sharing these resources with others.

350.org is a global climate justice organization that helped organize the largest climate strike in history. Together we improved their Climate Justice Action Map (CJAM) in the lead up to the strike so that activists could better mobilize their communities.

Punctuating Lists

If an article has multiple authors (using a user reference field or a content reference field), Drupal core displays each on their own line.

Like this ugly, rigid presentation:

As big tech continues to violate our privacy, thwart worker power, collude with militaries and deportation agencies, we need to move to ethical tech tools we can trust.

However, it can be intimidating to make the switch to new, unfamiliar tools.

Micky is leading two workshops to help people make the switch.

“How to Switch to Free Software: Email” 4:45-6:45pm Wednesday, December 4

Micky Metts of Boston’s Agaric Tech Cooperative will help us get out of the fix. The key to regaining security and control of the internet is switching to free software equivalents to proprietary/corporate software for communicating – among many uses.

This first workshop focuses on communications using email and texting software. The moderate inconvenience of switching is worth the privacy of strong encryption and the freedom of a new world of user-controlled communications and organizing. This first workshop in Sala Quetzal focuses on email and texting with encrypted software under user control. Bring your phone.

“How to Switch to Free Software: Browsers Etc” 2-4pm Thursday, December 5

A professional software developer, Micky, will focus on this second workshop, also in Sala Quetzal, on browsing, searching, and virtual meeting software. She will show how free software opens up the world of “platform co-ops” – think of Uber under drivers’ control and AirBnB under homeowners’ control.

Micky will be assisted by at-the-elbow translator/helpers. Bring your laptop and your phone.

Get notified of trainings

Learn when we have new opportunities for learning (two to four announcements a year).

IV. Movements and Networks

Free Software Movement

Free/Libre software is software that asserts the 4 fundamental software freedoms to every user. From Drupal modules to entire platforms, everything we build is Free Software because we believe that individuals and communities alike should have as much ownership as possible over the software they use. We even advise our clients in our blog on how to perform daily business operations using Free Software. Agaric's are members and supporters of the Free Software Foundation.

Design Justice Network

We design sites according to the principles of Design Justice. This means that the user experience of marginalized individuals is central to our design considerations. As a part of this network we share ideas and learn techniques to provide equitable online experiences for all. You may not know what it is like to be left out, until you are. 

MayFirst Movement Technology

MayFirst Movement Technology (MFMT) is a cooperative, of movement organizations and activists, that provides web hosting, email services and other free software tools to its members. MFMT has a bold and progressive mission and agrees not turn over your data to third parties under any circumstances.

As members of MFMT, we support the organization by raising awareness on ethical hosting and free software community standards. We encourage people and organizations to join and host their websites through MFMT and gain access to a suite of free software tools and services. Micky is an active board member, involved in discussions of governance as well as the free software tools and platforms that MayFirst offers. Your web hosting does matter, and so does your voice!

US Federation of Worker-Owned Cooperatives

USFWC is a non-profit 501c3 that is active in supporting worker-owned cooperatives while building the solidarity economy together. Currently we are building deeper ties with other tech coops within the USFWC to help advance economic justice in the tech industry as we build technology and support social justice movements, enhance trust and work with them. The federation works closely with DAWI - Democracy at Work Institute to educate and support members having a voice in their workplace.

U.S. Solidarity Economy Network

The USSEN is dedicated to exploring equitable and ethical economies that benefit community members and can interact on a larger scale to define methods and processes for promoting the growth and sustainability of each community as it relates to the world around it. As an organization, we are a member of the North American chapter of RIPESS which is part of an international organization that is an umbrella for groups like USSEN, worldwide. We also create Resist and Build  workshops that support the Solidarity movement.

Tech Coop Network

The Tech Coop Network was formed at a USFWC conference in Los Angeles in 2017 by members of several worker-owned tech and web development cooperatives. We are currently discussing bylaws and governance to prepare for inviting other cooperatives to join us. We meet monthly and are establishing our communications infrastructure.

International Workers of the World

Some Agaric members have joined the Boston Chapter of the IWW. This chapter has a horrible and ignoble past that the current members are determined to overcome. Current members are committed to rebuilding this branch to exemplify the very causes it was based upon before it was destroyed from within. We stand with marginalized people and communities. We represent the disenfranchised worker and support non-conforming individuals rights. An injury to one is an injury to all!

The Digital Fourth

As members of the Boston Chapter, we stay informed on new laws and changes to current laws that could effect our online presence as well as our real life freedoms. Agaric enjoys keeping up with the digital legal strategies and will voice our opinion on whether digital citizens will be affected negatively or positively by pending legislation in our state and sometimes nationally. Artificial intelligence and machine learning are two complex areas of technology and they need to be addressed by groups like us, with diverse knowledge. Surveillance by the Government is also a priority and we publish our findings and recommendations. We cannot change the things that we do not know about. D4th is a part of the nationwide coalition Restore The Fourth.

Science for the People

Scientists and citizens convene to discuss and create ways for the general population to get involved in all things scientific. We believe that you should not need a degree to have access to people in fields of scientific study. We work to remove barriers and make connections between scientists and average citizens people with good ideas and questions that could benefit scientific understanding. Agaric is very interested in  connecting people, projects and promoting discussions in this group. Science for the People has online meetings and is open to all.