I am very happy the bug is fixed and this blog post will be obsolete in mere days! Usually this sort of technical noodlings get relegated to our raw notes, currently hosted through GitLab, but figured at least a few other Drupal developers would want to know what has been going on with their toolbars.

Image credit: "Too Many Tabs" by John Markos O'Neill is licensed with CC BY-SA 2.0.

Conclusion

With the In Other Words module, it is now possible to keep your structured content and display it naturally. The logic of how a list should display in various scenarios is boiled down to an intuitive user interface for site builders, with no coding required. Try it for yourself! Get the module at drupal.org/project/inotherwords.

In the previous entry, we wrote our first Drupal migration. In that example, we copied verbatim values from the source to the destination. More often than not, the data needs to be transformed in some way or another to match the format expected by the destination or to meet business requirements. Today we will learn more about process plugins and how they work as part of the Drupal migration pipeline.

Syntactic sugar

The Migrate API offers a lot of syntactic sugar to make it easier to write migration definition files. Field mappings in the process section are an example of this. Each of them requires a process plugin to be defined. If none is manually set, then the get plugin is assumed. The following two code snippets are equivalent in functionality.

process:
  title: creative_title

process:
  title:
    plugin: get
    source: creative_title

The get process plugin simply copies a value from the source to the destination without making any changes. Because this is a common operation, get is considered the default. There are many process plugins provided by Drupal core and contributed modules. Their configuration can be generalized as follows:

process:
  destination_field:
    plugin: plugin_name
    config_1: value_1
    config_2: value_2
    config_3: value_3

The process plugin is configured within an extra level of indentation under the destination field. The plugin key is required and determines which plugin to use. Then, a list of configuration options follows. Refer to the documentation of each plugin to know what options are available. Some configuration options will be required while others will be optional. For example, the concat plugin requires a source, but the delimiter is optional. An example of its use appears later in this entry.

Providing default values

Sometimes, the destination requires a property or field to be set, but that information is not present in the source. Imagine you are migrating nodes. As we have mentioned, it is recommended to write one migration file per content type. If you know in advance that for a particular migration you will always create nodes of type Basic page, then it would be redundant to have a column in the source with the same value for every row. The data might not be needed. Or it might not exist. In any case, the default_value plugin can be used to provide a value when the data is not available in the source.

source: ...
process:
  type:
    plugin: default_value
    default_value: page
destination:
  plugin: 'entity:node'

The above example sets the type property for all nodes in this migration to page, which is the machine name of the Basic page content type. Do not confuse the name of the plugin with the name of its configuration property as they happen to be the same: default_value. Also note that because a (content) type is manually set in the process section, the default_bundle key in the destination section is no longer required. You can see the latter being used in the example of writing your Drupal migration blog post.

Concatenating values

Consider the following migration request: you have a source listing people with first and last name in separate columns. Both are capitalized. The two values need to be put together (concatenated) and used as the title of nodes of type Basic page. The character casing needs to be changed so that only the first letter of each word is capitalized. If there is a need to display them in all caps, CSS can be used for presentation. For example: FELIX DELATTRE would be transformed to Felix Delattre.

Tip: Question business requirements when they might produce undesired results. For instance, if you were to implement this feature as requested DAMIEN MCKENNA would be transformed to Damien Mckenna. That is not the correct capitalization for the last name McKenna. If automatic transformation is not possible or feasible for all variations of the source data, take notes and perform manual updates after the initial migration. Evaluate as many use cases as possible and bring them to the client’s attention.

To implement this feature, let’s create a new module ud_migrations_process_intro, create a migrations folder, and write a migration definition file called udm_process_intro.yml inside it. Follow the instructions in this entry to find the proper location and folder structure or download the sample module from https://github.com/dinarcon/ud_migrations It is the one named UD Process Plugins Introduction and machine name udm_process_intro. For this example, we assume a Drupal installation using the standard installation profile which comes with the Basic Page content type. Let’s see how to handle the concatenation of first an last name.

id: udm_process_intro
label: 'UD Process Plugins Introduction'
source:
  plugin: embedded_data
  data_rows:
    -
      unique_id: 1
      first_name: 'FELIX'
      last_name: 'DELATTRE'
    -
      unique_id: 2
      first_name: 'BENJAMIN'
      last_name: 'MELANÇON'
    -
      unique_id: 3
      first_name: 'STEFAN'
      last_name: 'FREUDENBERG'
  ids:
    unique_id:
      type: integer
process:
  type:
    plugin: default_value
    default_value: page
  title:
    plugin: concat
    source:
      - first_name
      - last_name
    delimiter: ' '
destination:
  plugin: 'entity:node'

The concat plugin can be used to glue together an arbitrary number of strings. Its source property contains an array of all the values that you want put together. The delimiter is an optional parameter that defines a string to add between the elements as they are concatenated. If not set, there will be no separation between the elements in the concatenated result. This plugin has an important limitation. You cannot use strings literals as part of what you want to concatenate. For example, joining the string Hello with the value of the first_name column. All the values to concatenate need to be columns in the source or fields already available in the process pipeline. We will talk about the latter in a future blog post.

To execute the above migration, you need to enable the ud_migrations_process_intro module. Assuming you have Migrate Run installed, open a terminal, switch directories to your Drupal docroot, and execute the following command: drush migrate:import udm_process_intro Refer to this entry if the migration fails. If it works, you will see three basic pages whose title contains the names of some of my Drupal mentors. #DrupalThanks

Chaining process plugins

Good progress so far, but the feature has not been fully implemented. You still need to change the capitalization so that only the first letter of each word in the resulting title is uppercase. Thankfully, the Migrate API allows chaining of process plugins. This works similarly to unix pipelines in that the output of one process plugin becomes the input of the next one in the chain. When the last plugin in the chain completes its transformation, the return value is assigned to the destination field. Let’s see this in action:

id: udm_process_intro
label: 'UD Process Plugins Introduction'
source: ...
process:
  type: ...
  title:
    -
      plugin: concat
      source:
        - first_name
        - last_name
      delimiter: ' '
    -
      plugin: callback
      callable: mb_strtolower
    -
      plugin: callback
      callable: ucwords
destination: ...

The callback process plugin pass a value to a PHP function and returns its result. The function to call is specified in the callable configuration option. Note that this plugin expects a source option containing a column from the source or value of the process pipeline. That value is sent as the first argument to the function. Because we are using the callback plugin as part of a chain, the source is assumed to be the last output of the previous plugin. Hence, there is no need to define a source. So, we concatenate the columns, make them all lowercase, and then capitalize each word.

Relying on direct PHP function calls should be a last resort. Better alternatives include writing your own process plugins which encapsulates your business logic separate of the migration definition. The callback plugin comes with its own limitation. For example, you cannot pass extra parameters to the callable function. It will receive the specified value as its first argument and nothing else. In the above example, we could combine the calls to mb_strtolower() and ucwords() into a single call to mb_convert_case($source, MB_CASE_TITLE) if passing extra parameters were allowed.

Tip: You should have a good understanding of your source and destination formats. In this example, one of the values to want to transform is MELANÇON. Because of the cedilla (ç) using strtolower() is not adequate in this case since it would leave that character uppercase (melanÇon). Multibyte string functions (mb_*) are required for proper transformation. ucwords() is not one of them and would present similar issues if the first letter of the words are special characters. Attention should be given to the character encoding of the tables in your destination database.

Technical note: mb_strtolower is a function provided by the mbstring PHP extension. It does not come enabled by default or you might not have it installed altogether. In those cases, the function would not be available when Drupal tries to call it. The following error is produced when trying to call a function that is not available: The "callable" must be a valid function or method. For Drupal and this particular function that error would never be triggered, even if the extension is missing. That is because Drupal core depends on some Symfony packages which in turn depend on the symfony/polyfill-mbstring package. The latter provides a polyfill) for mb_* functions that has been leveraged since version 8.6.x of Drupal.

What did you learn in today’s blog post? Did you know that syntactic sugar allows you to write shorter plugin definitions? Were you aware of process plugin chaining to perform multiple transformations over the same data? Had you considered character encoding on the source and destination when planning your migrations? Are you making your best effort to avoid the callback process plugin? Please share your answers in the comments. Also, I would be grateful if you shared this blog post with your colleagues.

Next: Migrating data into Drupal subfields

This blog post series, cross-posted at UnderstandDrupal.com as well as here on Agaric.coop, is made possible thanks to these generous sponsors. Contact Understand Drupal if your organization would like to support this documentation project, whether is the migration series or other topics.

Agaric builds tools for medical and scientific communities to advance their work, enhance collaboration, and improve outcomes.  And we've been doing this—helping healthy discussion about science and medicine flourish online—since 2008.

Reusable platforms for advanced online collaboration

The Collaboratory

The National Institute for Children's Health Quality partnered with Agaric to build the Collaboratory—a platform designed specifically to help healthcare improvement teams collaborate, innovate, and make change. During this partnership, begun in 2015, Agaric built a collaborative analytics tool that allows healthcare quality teams to visualize, compare, and benchmark data, identify opportunities for improvement, and celebrate their successes. We were proud to be NICHQ's 2020 partners in making the most of the digital health revolution.

Platform for Experimental Collaborative Ethnography (PECE)

In 2015, we began contributing to PECE, an open source digital platform that supports multi-sited, cross-scale ethnographic and historical research.  PECE is built as a Drupal distribution that can be improved and extended like any other Drupal project.  Agaric's contributions include building an API integration between PECE's bibliographic citation capabilities and Zotero's open source reference management and collaborative bibliography tools.

We have been brought back for a larger role to realize the full upgrade of this distribution and platform to Drupal 10.

Partners In Health Drug Resistant Tuberculosis Network and EndTB.org

Beginning in 2010, Agaric took over the development of the DRTB Network platform for Partners In Health, the famed international nonprofit public health organization, and the TB Care II initiative.  The core of this work was connecting practitioners in the field with experts through a natural yet structured response process complete with careful editorial review.  This crucial work lives on with endTB.org, a partnership between Partners In Health, Médecins Sans Frontières, and Interactive Research & Development.  All of this work for PIH is in Drupal.

Science Collaboration Framework

Agaric was the lead developer for the Science Collaboration Framework, a project of Harvard's Initiative in Innovative Computing. Working with researchers from Harvard University and Massachusetts General Hospital, we built a reusable platform for collaboration and communication in biomedical research, enriching the contributions of scientists and the biomedical online community with semantic data, highlighting advanced, structured relationships between contributed resources, and facilitating structured community discourse around biomedical research. We even earned a writeup in a scientific journal for our work!

As part of SCF, Agaric led the work of building the website for an online community of Parkinson's disease researchers and research investors, on the Science Collaboration Framework, for the Michael J. Fox Foundation for Parkinson's Research.

Ask Agaric for help building your world-bettering community today!

Micky speaks at events and hosts workshops online and around the world on topics including Free Software, Personal Digital Security, Drupal and Platform Cooperatives. She is also a host of a monthly cyber security webinar and a weekly series, Show and Tell with Agaric and friends.

We are a worker-owned cooperative specializing in building tools and websites that respect your freedom. We also provide training and consultation to meet your goals. Everything we do is part of our purpose to help all people gain the most power possible over their own lives.

Sign up to be notified when Agaric gives a migration training:

Welcome to Drutopia! We hope you are enjoying the features we have built. Everything you are using is open-source and free for good people like yourself to use.

We invite you to give your input on the project and contribute where you can by becoming a member of Drutopia. For as little as $10 a year you can become a member who votes for our leadership team, suggests features for our roadmap and is part of a community building tools for the grassroots.

Learn more about membership at drutopia.org

You can define your own permissions for the Drupal permissions page (/admin/people/permissions in modern Drupal, Drupal 8, 9, 10, and beyond) and then add conditional options to your code to do different things based on the role of the user and the permissions configured by a site administrator.

Here's how.

Create a modulename.permissions.yml file

This simple file has the permission machine name (lower case with spaces) and a title (Sentence case) with an optional description.

For our module, which has a particularly long name, that file is drutopia_findit_site_management.permissions.yml and its contents are like so:

access meta tab:
  title: 'Access meta tab'
  description: 'Access meta information (author, creation date, boost information) in Meta vertical tab.'

You can repeat lines like these in the same file for as many permissions as you wish to define.

Check for that permission in your code

The process for checking permissions is simply to use a user object if that's handed into your code, or to load the current user if it's not, and use the hasPermission() method which returns TRUE if that user has permission and FALSE if not.

For example, in a form alter in our drutopia_findit_site_management.module file:

/**
 * Implements hook_form_BASE_FORM_ID_alter() for node_form.
 *
 * Completely hide the Meta vertical tab (field group) from people without permission.
 *
 */
function drutopia_findit_site_management_form_node_form_alter(&$form, FormStateInterface $form_state, $form_id) {
  // If the current user has the permission, do not hide the Meta vertical tab.
  if (\Drupal::currentUser()->hasPermission('access meta tab')) {
    return;
  }
  // Code to hide the meta tab goes here, and is only reached if the user lacks the permission. 
  // ...
}

See all this code in context in the Find It Site Management module.

To learn more about defining permissions in modern Drupal, including dynamic permissions, you can see the change record for when the new approach replaced hook_permission().

Functionality designed to your life is the Agaric Design signature. Utilizing open source, free software from around the world, Agaric Design websites are impeccably crafted with a modern, sophisticated and understated spirit.

Our guiding philosophy is to bring open source and usability that works effortlessly in your day to day life.

I know we will become a favorite in your contacts list. And thank you! I consider that the ultimate compliment.

Dan Hakimzadeh

Sign up to be notified when Agaric gives a migration training:

Learning online has become a necessity for schools and independent teachers and tutors. Agaric can provide a Learning Management System (LMS) to support your teaching skills and knowledge integrated with video chat and whiteboard features. Every component of our platform is 100% Free/Libre Software because we believe in protecting the privacy of our students!

Let us know what you need and we can set up a time to discuss your unique situation.

Here you will find the details of our online learning tools.

Learning Management System

The Canvas learning management platform allows you or your organization to build the digital learning environment that meets the unique challenges faced by your institution. Canvas LMS allows teachers to create courses and grade students. Developed in 2011, Canvas was designed to better engage people in teaching and learning processes. 

Canvas is integrated with video chat software called BigBlueButton and it works to seamlessly allow teachers and students, if given the permission, to create live conferences with many features for moderation and engagement for all participants. Both Canvas and BigBlueButton are free open source software because our mission includes  privacy for all and the ability to freely modify the software to suit your needs.

Canvas LMS Features:

• Practice Tests, or Quizzes – These are assignments, groups of questions, that can be used to challenge student understanding and assess comprehension.  Question types include multiple choice, true/false, fill in the blanks, multiple answers, matching, numerical answers, formulas, and long text (freeform/essay).
• Question Banks – Group quiz questions in question banks to create grade, topic, institutional, or departmental question repositories.
• Outcomes – Outcomes (also called standards or competencies) describe what a learner should be able to do and are used to measure knowledge and ability.
• Mastery Paths – This tool allows course content to automatically be released to a learner based on performance, providing differentiation to students.
• Canvas Data – Canvas Data allows instructors to parse Canvas-generated data quickly.
• Canvas Commons – Canvas Commons is a repository that lets users easily share content.
• Integrated Media Recorder – This tool allows users to record audio and video messages for the digital classroom.
• Web conferencing – Set a meeting time for the whole class or one-on-one synchronous online discussions.
• Mastery Gradebook – Helps instructors assess the Outcomes being used in Canvas courses and measure student learning for accreditation or standards-based grading.
• Canvas Parent – Canvas Parent allows parents to engage with their children’s education by reviewing upcoming or past assignments, checking grades and receiving course announcements.
• Canvas Polls – An app (available on iOS and Android) that gauges students’ comprehension of material without “clicker” devices.

Videochat and whiteboard

The BigBlueButton video and whiteboard software is integrated into the Canvas dashboard and allows teachers and students to meet in a live learning environment that brings interactivity to the forefront. BigBlueButton is a free open-source web conferencing system designed for online classes, workshops and lectures. BigBlueButton allows teachers to create breakout rooms for different levels of learning.

Big Blue Button Features:

• Accounts and Profile
  • Sign up / Login
  • Profile
• Conference Rooms
  •  Create New Conference Rooms
  •  Breakout Rooms
  •  Conference Settings
  •  Rename Conference
  •  Manage Access to Conferences
•  Record Conference
  •  View Recordings
  •  Manage Recordings
  •  Modify Recordings
  •  Sort and Search Recordings

Drupal content management platform

You can provide a simple website for your school by building on the Drupal platform, the Canvas learning management platform automatically has access to a large number of features that are built into Drupal.  We describe some important ones here, though there are many more.

Content editing with Drupal

• Build your website with basic pages, landing pages, news postings, and more.
• Create instructional guides and reference links to relevant information.
• Engage the students in making additions to the site and maintaining the content.
• Optionally, use a content moderation workflow in which a content editor can approve draft content.
• Restore deleted content (retrieve from trash), so student exercises involving creating content on the site are not lost.
• Administrators can set permissions on any content and allow access in a granular way.

What will you provide for your students?