Translation provided by Colette Morya
Sometimes it is necessary to add extra information in our entities to migrate the content to Drupal 8.
Let’s say that I want to migrate the articles content type from Drupal 7 to Drupal 8 and as part of that I want to migrate the tags with my articles.
There are two options:
migration_lookup plugin.I will go with option two in this example. To do that I’m going to use the Migrate Plus module, which provides some handy plugins such as entity_generate.
One of Drupal’s biggest strengths is its data modeling capabilities. You can break the information that you need to store into individual fields and group them in content types. You can also take advantage of default behavior provided by entities like nodes, users, taxonomy terms, files, etc. Once the data has been modeled and saved into the system, Drupal will keep track of the relationship between them. Today we will learn about migration dependencies in Drupal.
As we have seen throughout the series, the Migrate API can be used to write to different entities. One restriction though is that each migration definition can only target one type of entity at a time. Sometimes, a piece of content has references to other elements. For example, a node that includes entity reference fields to users, taxonomy terms, and images. The recommended way to get them into Drupal is writing one migration definition for each. Then, you specify the relationships that exist among them.

When you break up your migration project into multiple, smaller migrations they are easier to manage and you have more control of process pipeline. Depending on how you write them, you can rest assured that imported data is properly deleted if you ever have to rollback the migration. You can also enforce that certain elements exist in the system before others that depend on them can be created. In today’s example, we are going to leverage the example from the previous post to demonstrate this. The portraits imported in the file migration will be used in the image field of nodes of type article.
You can get the full code example at https://github.com/dinarcon/ud_migrations The module to enable is UD migration dependencies introduction whose machine name is ud_migrations_dependencies_intro. Last time the udm_dependencies_intro_image was imported. This time udm_dependencies_intro_node will be executed. Notice that both migrations belong to the same module. Refer to this article to learn where the module should be placed.
To keep things simple, the example will only write the node title and assign the image field. A constant will be provided to create the alternative text for the images. The following snippet shows how the source section is configured:
source:
constants:
PHOTO_DESCRIPTION_PREFIX: 'Photo of'
plugin: embedded_data
data_rows:
- unique_id: 1
name: 'Michele Metts'
photo_file: 'P01'
- unique_id: 2
name: 'David Valdez'
photo_file: 'P03'
- unique_id: 3
name: 'Clayton Dewey'
photo_file: 'P04'
ids:
unique_id:
type: integerRemember that in this migration you want to use files that have already been imported. Therefore, no URLs to the image files are provided. Instead, you need a reference to the other migration. Particularly, you need a reference to the unique identifiers for each element of the file migration. In the process section, this value will be used to look up the portrait that will be assigned to the image field.
The destination section is quite short. You only specify that the target is a node entity and the content type is article. Remember that you need to use the machine name of the content type. If you need a refresher on how this is set up, have a look at the articles in the series. It is recommended to read them in order as some examples expand on topics that had been previously covered. The following snippet shows how the destination section is configured:
destination:
plugin: 'entity:node'
default_bundle: articleTo be able to reuse the previously imported files, the migrate_lookup plugin is used. Additionally, an alternative text for the image is created using the concat plugin. The following snippet shows how the process section is configured:
process:
title: name
field_image/target_id:
plugin: migration_lookup
migration: udm_dependencies_intro_image
source: photo_file
field_image/alt:
plugin: concat
source:
- constants/PHOTO_DESCRIPTION_PREFIX
- name
delimiter: ' '
In Drupal, files and images are entity reference fields. That means, they only store a pointer to the file, not the file itself. The pointer is an integer number representing the file ID (fid) inside Drupal. The migration_lookup plugin allows you to query the file migration so imported elements can be reused in node migration.
The migration option indicates which migration to query specifying its migration id. Additionally, you indicate which columns in your source match the unique identifiers of the migration to query. In this case, the values of the photo_file column in udm_dependencies_intro_node matches those of the photo_url column in udm_dependencies_intro_image. If a match is found, this plugin will return the file ID which can be directly assigned to the target_id of the image field. That is how the relationship between the two migrations is established.
Note: The migration_lookup plugin allows you to query more than one migration at a time. Refer to the documentation for details on how to set that up and why you would do it. It also offers additional configuration options.
As a good accessibility practice, an alternative text is set for the image using the alt subfield. Other than that, only the node title is set. And with that, you have two migrations connected between them. If you were to rollback both of them, no file or node would remain in the system.
The node migration depends on the file migration. That it, it is required for the files to be migrated first before they can be used to as images for the nodes. In fact, in the provided example, if you were to import the nodes before the files, the migration would fail and no node would be created. You can be explicit about migration dependencies. To do it, add a new configuration option to the node migration that lists which migrations it depends on. The following snippet shows how this is configured:
migration_dependencies:
required:
- udm_dependencies_intro_image
optional: []
The migration_dependencies key goes at the root level of the YAML definition file. It accepts two configuration options: required and optional. Both accept an array of migration ids. The required migrations are hard prerequisites. They need to be executed in advance or the system will refuse to import the current one. The optional migrations do not have to be executed in advance. But if you were to execute multiple migrations at a time, the system will run them in the order suggested by the dependency hierarchy. Learn more about migration dependencies in this article. Also, check this comment on Drupal.org in case you have problems where the system reports that certain dependencies are not met.
Now that the dependency among migrations has been explicitly established you have two options. Either import each migration manually in the expected order. Or, import the parent migration using the --execute-dependencies flag. When you do that, the system will take care of determining the order in which all migrations need to be imported. The following two snippets will produce the same result for the demo module:
$ drush migrate:import udm_dependencies_intro_image
$ drush migrate:import udm_dependencies_intro_node
$ drush migrate:import udm_dependencies_intro_node --execute-dependenciesIn this example, there are only two migrations, but you can have as many as needed. For example, a node with references to users, taxonomy terms, paragraphs, etc. Also note that the parent entity does not have to be a node. Users, taxonomy terms, and paragraphs are all fieldable entities. They can contain references the same way nodes do. In future entries, we will talk again about migration dependencies and provide more examples.
The core Migrate API offers another mechanism to execute multiple migrations at a time. You can tag them. To do that you add a migration_tags key at the root level of the YML definition file. Its value an array of arbitrary tag names to assign to the migration. Once set, you run them using the migrate import command with the --tag flag. You can also rollback migrations per tag. The first snippet shows how to set the tags and the second how to execute them:
migration_tags:
- UD Articles
- UD Example
$ drush migrate:import --tag=UD Articles,UD Example
$ drush migrate:rollback --tag=UD Articles,UD ExampleIt is important to note that tags and dependencies are different concepts. They allow you to run multiple migrations at a time. It is possible that a migration definition file contains both, either, or neither. The tag system is used extensively in Drupal core for migrations related to upgrading to Drupal 8 from previous versions. For example, you might want to run all migrations tagged ‘Drupal 7’ if you are coming from that version. It is possible to specify more than one tag when running the migrate import command separating each with a comma (,).
Note: The Migrate Plus module offers migration groups to organize migrations similarly to how tags work. This will be covered in a future entry. Just keep in mind that tags are provided out of the box by the Migrate API. On the other hand, migrations groups depend on a contributed module.
What did you learn in today’s blog post? Have you used the migration_lookup plugin to query imported elements from a separate migration? Did you know you can set required and optional dependencies? Have you used tags to organize your migrations? Please share your answers in the comments. Also, I would be grateful if you shared this blog post with your colleagues.
Next: Migrating taxonomy terms and multivalue fields into Drupal
This blog post series, cross-posted at UnderstandDrupal.com as well as here on Agaric.coop, is made possible thanks to these generous sponsors: Drupalize.me by Osio Labs has online tutorials about migrations, among other topics, and Agaric provides migration trainings, among other services. Contact Understand Drupal if your organization would like to support this documentation project, whether it is the migration series or other topics.
The value of a directory are the listings within it. It is truly a community effort to assemble enough accurate and up-to-date resources in a single place for it to be useful. For Cambridge, the service providers that work at government agencies and nonprofits are the lifeblood of the directory. Without them, there would be no Find It Cambridge.
For these service providers (many already pressed for time), Find It is a system that is easy enough to take the time to enter information into, and which conveniently structures the data so that it can easily be searched and filtered on by community-members looking for opportunities.
Through user research, we mapped the information architecture to the mental models that service providers hold for their events and programs.
Most service providers thought of their events in terms of what the event is about, how to contact the organizers, who it is for, when it is happening, where it is happening, how much it costs, and if there is any sort of registration required. So we organized fields into working tabs to match: About, Contact, For whom, When, Where, Cost, and Signup.
Even with fields grouped by tabs, the form can take some time to complete. That is why we introduced autosave and soft save features. When working on a form, the site automatically saves the current draft every few seconds. Service providers can also save a draft to return to later. Fields that are required for publishing, are optional for a draft.
Service providers have many responsibilities to juggle. It is important that they can start creating an event or program, save it and return to it later before publishing it.
Drupal has powerful workflow states, which we have put to use to help service providers clearly know the status of their content.
A service provider can either save their content as a draft or publish it immediately. If saved as a draft, a banner appears on the page clearly indicating that the event or program is not yet published.
Authors can also create a draft alongside a published version. This allows new versions to be worked on, while maintaining the current page for site visitors.
There are particular ways to write and format content on events and programs to make the most of Find It's features. We provide help text along the way to clue providers in on the best ways to write their content. We also include a character count so providers know if they are staying within the recommended limits for certain text fields.
Certain fields have many options. In some cases the majority of them apply. For example, many educational events are for all ages up to 18. In that scenario, having a "Select All" option speeds up the data entry process. The Selectize JavaScript library adds elegant toggle and check all options to multi-value fields. We created the CheckboxesJS Drupal project so that other Drupal sites can easily incorporate these features on fields of their choosing.
Some fields on Event and Programs only need to show under certain conditions. For example, if an event does not require registration, then there is no need to worry service providers with a registration link field. Using conditional logic keeps forms simple and streamlined.
There was a lot of discussion on whether to support repeating rules or instead allow multiple dates. We decided on multiple dates, as experience has shown that even repeating events oftentimes have exceptions (and because the events we import from Cambridge Public Libraries are a list of arbitrary dates rather than a recurring rule, and somehow no one in computer science has created a library to produce a best-effort recurring rule from a list of dates).
Find It search is powered by Apache Solr, a popular open-source enterprise search platform. We use its numerous features to make the search results as relevant as possible for site visitors. It is an ongoing process of tweaks; here are some of the things we have done so far.
On content with lots of data like the events and programs of Find It, certain fields carry more importance than others. The title of an event, for example, is one of the most important. The transportation notes, on the other hand, carries less significance in search queries. When someone types the keyword "music lesson", an event with music lesson in the title or summary shows up before a program for English lessons.
When someone searches "childcare" but a program uses "child care", the search engine should know these are equivalent. The same is true for "STEM" and "science education."
Find It supports synonyms. The site manager can define synonyms so that when site visitors search for a certain term, results with matching synonyms show up as well.
We used the results of our user research to show the critical information people need to pin point the right opportunities: title, neighborhood, and a short summary.
Filters help users narrow a search query down to specific criteria. In our testing, we found that age and neighborhood were most important, especially for low-income caregivers. For those of us that rely on public transportation, events and programs need to be nearby. We placed these filters accordingly towards the top of the page.
Naming conventions in Cambridge are unique, which is true for other cities too. Residents might not know the official name of their neighborhood or live at the border between two. We have included a labeled, clickable map to help users choose the right neighborhood. We built this so that other Find It platforms can upload their own SVG map to show their neighborhood.
Find It comes out of the box with four different types of opportunities: Events, Places, Organizations and Programs.
The organization serves as the foundation for opportunities posted on a Find It page. Every event and program posted to Find It, belongs to an organization. This helps an organization's page serve as a mini-website. When an event or program is published, it automatically shows up on its organization page.
Organizations can also have "child" organizations, which is helpful for larger groups that might have distinct sub-committees or departments that have sub-departments.
An event is an opportunity with a clear start and end date. When an event is published it shows up on the Homepage, Events page, Search page and on the organization's page.
Visitors can sort opportunities by start date to find upcoming events.
A program is similar to an event. In fact, most fields are shared between the two. A program though, implies more longevity and commitment than an event. Rather than requiring a specific date or dates, a program can simply be "ongoing." There is the option to include specific dates though.

In the first version of Find It Cambridge, a new opportunity surfaced that did not quite fit into the event, program, or organization categories. Parks, neighborhood pools, and other destinations were a good fit for Find It's library of opportunities. They have open hours, but many of the event fields were irrelevant. The same went for Programs. In fact, sometimes these places have events or programs happening at them.
These are community-minded destinations people can go to. In other words, places.

Curated by members of the Cambridge Kid's Council, Find It Cambridge was designed to help busy parents stay informed about the plethora of educational opportunities available for their children.
Curated by Immigrant Family Services Institute, the Immigrant Navigator was designed to provide information on educational events and social services for new Haitian Immigrants in order to ease their transition into life in the United States.
We are Agaric, a web development collective eager to understand where you want to go with your web presence and to apply our knowledge and skills to get you there.
Because we are a worker-owned cooperative, every person you interact with at Agaric is an owner of the company. Because we are part of the Free Software movement, we build on tools and frameworks created and tested by thousands of people. Our goal is to give you power over your web sites and online technology.
Have something you think we may be able to help with? Just ask.
I had a great time at BioRAFT Drupal Nights, on January 16th, 2014. Originally Chris Wells (cwells) was scheduled to speak, unfortunately he was down with the flu. Michelle Lauer (Miche) put an awesome presentation together on really short notice. With help from Diliny Corlesquet (Dcor) there was plenty to absorb intellectually along with the delicious Middle Eastern food. I love spontaneity, so it was fun to hop over to BioRAFT in Cambridge, MA, to be a member of a panel of Senior Drupal Developers.
I joined Seth Cohn, Patrick Corbett, Michelle Lauer and Erik Peterson to present some viable solutions to performance issues in Drupal.
Several slides covered the topic - Drupal High Performance and a lively discussion ensued. The panel members delved into some of the top issues with Drupal performance under several different conditions. The discussion was based on the book written by Jeff Shelton, Naryan Newton, and Nathaniel Catchpole - High Performance Drupal - O'Reilly - http://shop.oreilly.com/product/0636920012269.do .
The room at BioRAFT was comfortably full with just the right amount of people to have an informal Q and A, and to directly address some concerns that developers were actually working on. Lurking in the back I spotted some of the top developers in the Drupal community (scor, mlncn, kay_v just to name a few) that were not on the panel, but they did had a lot of experience to speak from during Q and A.
The video posted at the bottom is packt with great information and techniques - Miche even shared some code snippets that will get you going if you seek custom performance enhancements.
One of the discussions was around tools that can be used to gauge performance or tweak it.
I also found an online test site: http://drupalsitereview.com
We also talked a lot about caching and the several options that exist to deal with cache on a Drupal site.
There are options to cache content for anonymous users and there are solutions and modules for more robust sites with high traffic.
PHP 5.5 offers a good caching solution with integrated opcode caching. They are a performance enhancement and extension for PHP. Some sites have shown a 3x performance boost by using opcode caching. Opcode caching presents no side-effects beyond extra memory usage and should always be used in production environments. Below are a couple of the suggestions we discussed that are listed in the Modules listed on Drupal.org:
The panel agreed that there are many things to consider when seeking to improve the sites performance:
A few more are discussed in the video below...
Sending out Be Wells to cwells! We look forward to a future presentation, and we are also pleased that on such short notice it did not seem too hard to gather a panel of several senior Drupal developers to discuss High Performance Drupal. See the whole discussion on YouTube
According to the documentation:
- @variable: When the placeholder replacement value is:
- A string, the replaced value in the returned string will be sanitized using \Drupal\Component\Utility\Html::escape().
- A MarkupInterface object, the replaced value in the returned string will not be sanitized.
- A MarkupInterface object cast to a string, the replaced value in the returned string be forcibly sanitized using\Drupal\Component\Utility\Html::escape().
Using the @ as the first character escapes the variable if it is a string but if we pass a MarkupInterface object it is not going to be sanitized, so we just need to return an object of this type.
And Drupal 8 has a class that implements MarkupInterface called: Markup, so the code above for Drupal 8 will look like this:
public function content() {
$markup = new Markup();
$links[] = Link::fromTextAndUrl('Google', Url::fromUri('http://www.google.com'))->toString();
$links[] = Link::fromTextAndUrl('Yahoo', Url::fromUri('http://www.yahoo.com'))->toString();
$types = $markup->create(implode(', ', $links));
return [ '#markup' => t('Links %links', ['%links' =>$types]), ];
}
And that's it, our string will display the links.
If you want to see a real scenario where this is helpful, check this Comment Notify code For Drupal 7 and this patch for the D8 version.
Sign up if you want to know when Mauricio and Agaric give a migration training:
Benjamin Melançon of Agaric helped with a patch for the Drupal 7 version of Insert module.
Here at Agaric we work a lot with install profiles and, more often than not, we have to provide default content. This is mostly taxonomy terms, menu links, and sometimes even nodes with fields. Recently, I have started to use Migrate to load that data from JSON files.
Migrate is usually associated with importing content from a legacy website into Drupal, either from a database or files. Loading initial data is just a special case of a migration. Because it handles many kinds of data sources with a minimum of configuration effort, Migrate is well suited for the task.
Here is an example from our project Find It Cambridge. It is a list of terms I would like to add to a vocabulary, stored in a JSON file.
[
{
"term_name": "Braille",
"weight": 0
},
{
"term_name": "Sign language",
"weight": 1
},
{
"term_name": "Translation services provided",
"weight": 2
},
{
"term_name": "Wheelchair accessible",
"weight": 3
}
]
If you do not need a particular order for the terms in the vocabularies, you can skip the weight definition. In the next code snippet we specify a default value of 0 for the weight. In such a case, Drupal will list the terms alphabetically.
Migrate does almost all the work for us— we just need to create a Migration class and configure it using the constructor. For a single JSON file the appropriate choice for the source is MigrateJSONSource. The destination is an instance of MigrateDestinationTerm. Migrate requires a data source to have a primary key which is provided via MigrateSQLMap. In this case term_name is defined as the primary key:
class TaxonomyTermJSONMigration extends Migration {
public function __construct($arguments) {
parent::__construct($arguments);
$this->map = new MigrateSQLMap(
$this->machineName,
array(
'term_name' => array(
'type' => 'varchar',
'length' => 255,
'description' => 'The term name.',
'not null' => TRUE,
),
),
MigrateDestinationTerm::getKeySchema()
);
$this->source = new MigrateSourceJSON($arguments['path'], 'term_name');
$this->destination = new MigrateDestinationTerm($arguments['vocabulary']);
$this->addFieldMapping('name', 'term_name');
$this->addFieldMapping('weight', 'weight')->defaultValue(0);
}
}
This migration class expects to find the vocabulary machine name and the location of the JSON file in the $arguments parameter of the constructor. Those parameters are passed to Migration::registerMigration. Registration and processing can be handled during installation of the profile. Because there are several vocabularies to populate I have defined a function:
function findit_vocabulary_load_terms($machine_name, $path) {
Migration::registerMigration('TaxonomyTermJSONMigration', $machine_name, array(
'vocabulary' => $machine_name,
'path' => $path,
));
$migration = Migration::getInstance($machine_name);
$migration->processImport();
}
This function is called in our profile's implementation of hook_install with the path and vocabulary machine name for each vocabulary. The file is stored at profiles/findit/accessibility_options.json relative to the Drupal installation directory. The following snippet is an extract from our install profile that demonstrates creating the vocabulary and using above function to add the terms.
function findit_install() {
…
$vocabularies = array(
…
'accessibility_options' => st('Accessibility'),
…
);
…
foreach ($vocabularies as $machine_name => $name) {
findit_create_vocabulary($name, $machine_name);
findit_vocabulary_load_terms($machine_name, dirname(__FILE__) . "/data/" . $machine_name.json);
}
…
}
Executing drush site-install findit will set up content types, vocabularies, and create the taxonomy terms.
In the past I have used Drupal's API to create taxonomy terms, menu links, and other content, which also works well and does not add the mental overhead of another tool. But the Migrate approach has one key benefit in my opinion: it provides a well defined way of separating data from the means to import it and enables the developer to easily handle more complex tasks like nodes with fields. Compare the above approach of importing taxonomy terms to the following equivalent code:
$terms = array(
array('vocabulary_machine_name' => 'accessibility_options', 'name' => 'Braille', 'weight' => 0),
array('vocabulary_machine_name' => 'accessibility_options', 'name' => 'Sign language', 'weight' => 1),
array('vocabulary_machine_name' => 'accessibility_options', 'name' => 'Translation services provided', 'weight' => 2),
array('vocabulary_machine_name' => 'accessibility_options', 'name' => 'Wheelchair accessible', 'weight' => 3),
);
foreach ($terms as $term_data) {
$term = (object) $term_data
taxonomy_term_save($term);
}
Even though one can write the code in a style that takes care of separating code and data, it gets more complicated with less intuitive APIs. My preference is to have content in a separate file and rely on a well tested tool for importing it. Using Migrate and JSON files is a convenient and powerful solution to this end. What is your approach to providing default content?
Agaric organiza una reunión semanal en línea conocida como "Show and Tell". Los participantes comparten consejos y trucos que hemos aprendido y plantean preguntas a otros desarrolladores sobre tareas o proyectos en los que estamos trabajando. Cada semana pedimos a la gente que nos envíe un poco de información sobre lo que les gustaría presentar. Esto no es un requisito previo, sólo una sugerencia. Tener un aviso previo de las presentaciones nos permite correr la voz a otros que puedan estar interesados, pero usted puede simplemente presentarse, y lo más probable es que haya tiempo para presentar durante 5-10 minutos. Puede inscribirse en la lista de correo de "Show and Tell" y será notificado de los próximos eventos.
Recientemente hemos abierto el chat de "Show and Tell" para vincularnos con otras cooperativas que hacen trabajos de desarrollo web. Agaric fue contactada por miembros de Fiqus.coop en Argentina, ya que ellos ya habían comenzado con una iniciativa para conocer a otros desarrolladores de cooperativas y asi, compartir valores y metas. Nadie había enviado un aviso de presentación, así que cambiamos el tema del chat para que fuera más un encuentro y un saludo para conocernos mejor con el objetivo de poder compartir nuestro trabajo en los proyectos. El valor de la reunión se hizo evidente de inmediato cuando nos adentramos en la conversación con algunos miembros de Fiqus.
A continuación, invitamos a más desarrolladores a participar en la discusión y se abrieron las puertas para compartir más profundamente y conectar. Esta semana nuestra reunión fue exagerada! Nicolás Dimarco nos guió a través de una corta presentación de diapositivas que reveló un proceso y flujo de trabajo federado para compartir el desarrollo con los miembros de múltiples cooperativas. El plan es tan simple que todos lo entendieron inmediatamente. La conversación que siguió fue convincente, las preguntas fueron indicativas de dónde necesitamos educar a los demás sobre los principios cooperativos vs. las tácticas corporativas. Necesitamos más discusión sobre la confianza y la amistad. Hay tantos desarrolladores en trabajos corporativos que me han preguntado cómo funciona una cooperativa de desarrollo web y cómo funciona un proyecto sin un gerente. Primero, me gusta explicar que los proyectos tienen gerentes, pero ellos están manejando el trabajo, no a la gente. Tomarse el tiempo para conocer las habilidades y pasiones de cada uno sobre la programación es una parte esencial para poder trabajar juntos en una Federación. Fiqus.coop ha hecho que sea sencillo para todos ver el camino para compartir el trabajo en los proyectos!
Aquí está el enlace a la grabación de video del chat donde Nicolás Dimarco de Fiqus.coop presenta la fórmula de trabajo federado entre cooperativas. Aquí hay un enlace a las notas de la reunión del 20/3/2019 y algunas reuniones pasadas de Show and Tell.
Más información sobre el Show and Tell.
Algunas tiendas de Drupal ya trabajan juntas en proyectos y podemos ayudar a que crezcan compartiendo nuestras experiencias. Nos encantaría saber cómo trabajan y los procesos que han descubierto que hacen que compartir el trabajo en los proyectos sea un éxito!