In preparation for Agaric's migration training at DrupalCon Seattle, we've seen again that getting a development environment suitable for working on Drupal 8 with Composer is a big road block. For that reason, here are instructions for getting it all going, suitable for our training or for working on Drupal in a code sprint, with only one hard requirement: PHP.

Getting a local development going is a problem for everyone, including programmers and people who should be skilled at this. Vagrant and DrupalVM were created to make identical development environments work on any computer. Unfortunately, that's more a pipe dream than a promise, as our own experiences and support requests we've received can attest,

Docker is another attempt to make local development match production environments, and a newer generation of Drupal-adjacent development workflow tools, such as DDEV, Lando, Docksal, and Humpback, build on it. Using an online development environment is also an option, but has too much reliance on working WiFi for conference situations.

The goal is any Drupal development environment—any of the above or anything else—where you can go to your project root directory in a terminal and type, say, composer update and have If you don't have a local development environment of any kind, here is the minimalist approach. All the steps start with opening your terminal.

Goal

The most power possible for all people over our own lives.

Building blocks of freedom

• Free software
• Cooperative platforms
• Solidarity economy (examples: Ujima Boston, Cooperation Jackson)
• Personal power (ask Micky)

Power is coordination

“Knowledge is not power,” said Mariame Kaba (prisonculture), who knows a thing or two about it as a longtime organizer focusing on ending violence, dismantling the prison industrial complex, practicing transformative justice, and supporting youth leadership development.

Power is organization.

Ben tends to fixate on the injustices caused by an organized, unaccountable minority making decisions affecting a much larger but unorganized majority, but there are plenty of problems that have nothing or little to do with the problem of concentrated benefit and distributed cost.  They're just still problems because we, society, hasn't gotten our act together.  Traffic jams affect millions of people who, while mostly not in the 1% of income or wealth, are easily in the top couple quintiles.

Coordination requires not doing everything oneself

Ben notes: I strongly believes that we, people generally, need to have the ultimate say in the things that affect our lives, including our own workplaces, our Internet connection, our home water supply, mediating platforms like Facebook and Google, and our medical care.

That does not mean I or anyone else wants to be micromanaging my coworkers, laying cables and arguing protocols, inspecting plumbing, reviewing commits and moderating posts, and ordering doctors and nurses around.

Even various approaches to direct democracy don't mean people do all the work directly.  But how can we be the ultimate decision-makers in so many aspects of society?

"The market" is attractive here: vote with your money.

A regular systematic redistribution of wealth could make that a feasible answer, but for now it means a certain few dozen people have more decision-making power than a few billion people.

So we need to try some other way of delegating authority.

On the problem of self-selected leaders

The major problem — one of the major problems, for there are several — one of the many major problems with governing people is that of whom you get to do it; or rather of who manages to get people to let them do it to them.

To summarize: it is a well-known fact that those people who must want to rule people are, ipso facto, those least suited to do it. To summarize the summary: anyone who is capable of getting themselves made President should on no account be allowed to do the job. To summarize the summary of the summary: people are a problem.

Chapter 28, The Restaurant at the End of the Universe, by Douglas Adams in the Hitchhiker's Guide to the Galaxy series.

Solutions to the problem of self-selected leaders

• Citizens' Assembly: "a representative group of citizens who are selected at random from the population to learn about, deliberate upon, and make recommendations in relation to a particular issue or set of issues. Other similar bodies have operated in parts of Canada – notably British Columbia and Ontario – and there is a citizens’ assembly currently operating in Ireland. This website [citizensassembly.co.uk] draws together experiences of Citizens Assemblies in the UK."

Democracy at scale: Historically possible

almost everyone nowadays insists that participatory democracy, or social equality, can work in a small community or activist group, but cannot possibly ‘scale up’ to anything like a city, a region, or a nation-state. But the evidence before our eyes, if we choose to look at it, suggests the opposite. Egalitarian cities, even regional confederacies, are historically quite commonplace. Egalitarian families and households are not.

David Graeber and David Wengrow, "How to change the coures of human history (at least, the part that's already happened)" (2 March 2018) accessed 2018 December.

Tyranny of structurelessness

During the years in which the [open source] movement has been taking shape, a great emphasis has been placed on what are called leaderless, structureless groups as the main -- if not sole -- organizational form of the movement.  The source of this idea was a natural reaction against the over-structured [corporations and universities] in which most of us found ourselves, and the inevitable control this gave others over our lives, and the continual elitism of the Left and similar groups among those who were supposedly fighting this overstructuredness.

The idea of "[openness/structurelessness]," however, has moved from a healthy counter to those tendencies to becoming a goddess in its own right. The idea is as little examined as the term is much used, but it has become an intrinsic and unquestioned part of [open source philosophy]. For the early development of the movement this did not much matter. It early defined its main goal, and its main method, as [writing working code and] consciousness-raising, and the "structureless" [open] group was an excellent means to this end. The looseness and informality of it encouraged participation in discussion, and its often supportive atmosphere elicited personal insight.  If nothing more concrete than personal [itch-scratching and] insight ever resulted from these groups, that did not much matter, because their purpose did not really extend beyond this.

Jo Freeman (aka Joreen) famous essay The Tyranny of Structurelessness (with the references to the women's liberation movement replaced with references to free software).

See also Cathy Levine's response, "The Tyranny of Tyranny".  An excerpt:

Contrary to the belief that lack of up-front structures lead to insidious, invisible structures based on elites, the absence of structures in small, mutual trust groups fights elitism on the basic level — the level of personal dynamics, at which the individual who counters insecurity with aggressive behaviour rules over the person whose insecurity maintains silence. The small personally involved group learns, first to recognise those stylistic differences, and then to appreciate and work with them; rather than trying to either ignore or annihilate differences in personal style, the small group learns to appreciate and utilise them, thus strengthening the personal power of each individual.

Related

Meritocracy

The word meritocracy comes from a political satire. It was never meant to be something we should aspire to. It was the opposite, actually, a warning about how we rationalize what we believe we've "earned". If that sentence doesn't seem to you applicable to the tech industry and our cyclical discussions about sexism, racism, and even occasionally classism,

... then read that blog post, "you keep using that word" (content warning: cuss words).

Rough consensus

Common in technical projects, a particular approach of "rough consensus"—a decision moves forward when all objections have been resolved in a way that everyone can live with or at least have been seriously considered and judged not to be showstoppers, whether or not anyone is still actively making the objection—is probably best described as the Internet Engineering Task Force's ideal in an informational document by Pete Resnick, On Consensus and Humming in the IETF.

Decision-making in Drupal

Drupal is not a democracy

The governance group itself, along with its assignment, is a product of the very power structure it's tasked with reworking. The task force was personally approved by the dictator for life. It reports to the dictator for life. Any decision on its recommendations will be made by the dictator for life.

Nedjo Rogers, Drupal and governance.

Governance group agrees

Having a BDFL model means loyalty, time, and attention is divided. There is often frustration as pressure for change and decisions reach an individual bottleneck. There is a strong feeling that any community change or action requires Dries’ approval before commencing let alone expanding. The project is bigger than one individual; it’s time to recognise that and place a community group at the center.

The Drupal Governance Task Force 2018 Proposal; see the issue posted for this in the Drupal Community Governance project.

Background reading

In addition to the works quoted above, here are some that informed our presentation.

Articles

• Drupal and governance
• Six principles for conversational transformation
• Drupal as a political act
• The dehumanizing myth of meritocracy
• Drupal's Governance
• Zapatista - Caracoles - Resistance and Autonomy
• Murray Bookchin and the Kurdish resistance

Books

• Collective Courage: A History of African American Cooperative Economic Thought and Practice, by Jessica Gordon Nembard
• Life and Ideas: The Anarchist Writings of Errico Malatesta
• The Anarchist Collectives: Workers’ Self-Management in the Spanish Revolution, 1936–1939 edited by Sam Dolgoff (full text online)

Other links of interest

• Loomio's Decision-making Handbook
• Cooperative Resources
• Privacy and Security in the Digital era

Federated/P2P/decentralized services.

https://redecentralize.org
https://prism-break.org/en/all/
https://secushare.org/comparison

Projects:

https://p2pfoundation.net/ https://gnunet.org/en/
https://tox.chat/about.html
https://net2o.de
https://ipfs.io https://github.com/cjdelisle/cjdns

All Improvements Are Shared Freely as Free Software

Everything we do at Agaric uses software that is free as in freedom - free to install, use and repurpose for your own needs.

Our work with Portside has already translated into improving and even creating the following Drupal modules:

* Give - on-site donation forms and reporting
* Social Post Twitter - Re-post content from your website to your Twitter account
* Social Share Facebook - Re-post contnt from your website to your Facebook page
* Minimal HTML - A text format handy for short text fields

So, when you donate to Portside's fundraising campaign, you are both supporting independent journalism and the open-source software that benefits other independent outlets and websites.

Donate to Portside

La gente a menudo pregunta por las herramientas de software libre que Agaric utiliza para administrar su negocio cooperativo. En este artículo, compartiremos los recursos que elegimos para las tareas diarias de oficina y administración, así como para las comunicaciones en nuestras operaciones comerciales.   

Agaric utiliza software libre siempre que es posible. Por esta razón, creamos sitios web con Drupal, una plataforma  de gestión de contenido que destaca por la calidad de su código y por su amplia variedad de funcionalidades.  Como miembros de la Comunidad Drupal, buscamos contribuir activamente con los grupos que trabajan en encontrar opciones para su mejora y ampliación. 

La necesidad de una tecnología responsable - Parte 1 

Quizás podrán preguntarse por qué decimos "Software Libre" y no "Código Abierto", ya que en ambos casos esencialmente es lo mismo. Nosotros preferimos este término porque incluye los principios éticos sobre el respeto a la libertad del usuario. La ética del software libre dice que los usuarios merecen control sobre el código que usan, mientras que el uso del término "Código abierto" se refiere solo  a permitir que los usuarios participen en el desarrollo. Nosotros apoyamos la idea de un software que proteja nuestros derechos y nos evite estar vulnerables. 

Worker owned cooperatives are businesses owned and controlled by the people who work in them. This is a collection of resources that will help you navigate the world of cooperatives and opportunities.
This list will continue to grow and connect with cooperative networks around the globe.

Boston area:

• WORC'N, the Worker-Owned and Run Cooperative Network of Greater Boston is a network of worker-owned cooperatives, co-op developers, and those seeking support to start worker cooperatives.
• Boston Center for Community Ownership

National/international:

• US Federation of Worker Cooperatives
• Democracy at Work aims to build a social movement for a new society whose productive enterprises (offices, factories, and stores) will mostly be Worker Self-Directed Enterprises (WSDEs), a true economic democracy.
• North American Technology Worker Cooperatives
• Tech Coop How To - .pdf

Find worker cooperatives:

• Member directory of the US Federation of Worker Cooperatives
• Find.coop (open directory of all types of coops)
• ica.coop (open directory of all types of coops)
• Drupal Worker Cooperative group and overall Drupal coop group. See in particular Wiki of worker coops and resources.

Legal resources

• Sustainable Economies Law Center

News articles about cooperatives:

• TheNews.coop
• Guardian Co-operative group revolution

Presentations and speakers:

• Think Outside the Boss
• Commonbound speakers and sessions
• Conference.coop speakers

Cooperative Resources:

• Citizens Command Center

Some great events happened in 2014 based around building cooperatives and collectives. These events should have videos online of the speakers and events: California Cooperative Conference and Chicago Freedom Summer

V. Auxiliary Tech Projects

CoopGuide

Working together on projects is one of the most important goals of our tech coop. It is also the 6th Cooperative Principle, Cooperation among Cooperatives.  CoopGuide is a work in progress being built to facilitate team members in locating each other and getting familiar with each other's skills. The idea for this site came from discussions at Agaric's weekly Show and Tell gatherings. 

SnowDrift

A crowdmatching platform that is a fund raising source for public goods. The first project will be to fund the building of Snowdrift itself. Micky is on the board and is working with the team to guide the formation of this cooperative effort.

IEEE.org SA Open

The SA Open is a platform, by the IEEE and managed by LeadingBit, that
aims to create a set of agreed upon standards to help people, both technical and organic, co-work more effectively to build Free Software with proper documentation and support materials on using and changing it. Agaric volunteers as a member of the Community Advisory Group, where we offer our expertise on this very subject. You can get involved with SA Open and help make it happen.

meet.coop

Meet.coop is an online meeting co-operative dedicated to hosting BigBlueButton video chat servers in multiple locations around the world. Agaric is a member.  We meet weekly in different teams to work out the governance and member issues. There is an operational team that builds and maintains the hardware and software for providing an excellent video conference network structured in Sociocracy, also called dynamic governance. Meet.coop has forums where you can mingle with members and get up to date information on the development and learn more about the project.

Mastodon

Mastodon is a federated social network made up of affinity groups. The functionality is much like Twitter, where you can message other members and share information or "Toots" as they are called, in a group or one-on-one. You can join a sub-community or create your own. Agarics are members of the social.coop sub-community. Members can contact members of other sub-communities and there is a directory where you can browse the groups in the whole federation.

Mass Mesh

Mass Mesh is a high-tech social club building community-owned mesh networks throughout Boston. These networks enable home Internet access that is fairer, more secure, and more locally resilient than traditional cable. The purpose of this wiki is to provide you with all of the information you need to join the mesh by setting up your own mesh node. All of the information on their site and all of the code in their software is free to use share and edit.

Today we are going to learn how to migrate users into Drupal. The example code will be explained in two blog posts. In this one, we cover the migration of email, timezone, username, password, and status. In the next one, we will cover creation date, roles, and profile pictures. Several techniques will be implemented to ensure that the migrated data is valid. For example, making sure that usernames are not duplicated.

Although the example is standalone, we will build on many of the concepts that had already been covered in the series. For instance, a file migration is included to import images used as profile pictures. This topic has been explained in detail in a previous post, and the example code is pretty similar. Therefore, no explanation is provided about the file migration to keep the focus on the user migration. Feel free to read other posts in the series if you need a refresher.

Getting the code

You can get the full code example at https://github.com/dinarcon/ud_migrations The module to enable is UD users whose machine name is ud_migrations_users. The two migrations to execute are udm_user_pictures and udm_users. Notice that both migrations belong to the same module. Refer to this article to learn where the module should be placed.

The example assumes Drupal was installed using the standard installation profile. Particularly, we depend on a Picture (user_picture) image field attached to the user entity. The word in parenthesis represents the machine name of the image field.

The explanation below is only for the user migration. It depends on a file migration to get the profile pictures. One motivation to have two migrations is for the images to be deleted if the file migration is rolled back. Note that other techniques exist for migrating images without having to create a separate migration. We have covered two of them in the articles about subfields and constants and pseudofields.

Understanding the source

It is very important to understand the format of your source data. This will guide the transformation process required to produce the expected destination format. For this example, it is assumed that the legacy system from which users are being imported did not have unique usernames. Emails were used to uniquely identify users, but that is not desired in the new Drupal site. Instead, a username will be created from a public_name source column. Special measures will be taken to prevent duplication as Drupal usernames must be unique. Two more things to consider. First, source passwords are provided in plain text (never do this!). Second, some elements might be missing in the source like roles and profile picture. The following snippet shows a sample record for the source section:

source:
  plugin: embedded_data
  data_rows:
    - legacy_id: 101
      public_name: 'Michele'
      user_email: 'micky@example.com'
      timezone: 'America/New_York'
      user_password: 'totally insecure password 1'
      user_status: 'active'
      member_since: 'January 1, 2011'
      user_roles: 'forum moderator, forum admin'
      user_photo: 'P01'
  ids:
    legacy_id:
      type: integer

Configuring the destination and dependencies

The destination section specifies that user is the target entity. When that is the case, you can set an optional md5_passwords configuration. If it is set to true, the system will take an MD5 hashed password and convert it to the encryption algorithm that Drupal uses. For more information password migrations refer to these articles for basic and advanced use cases. To migrate the profile pictures, a separate migration is created. The dependency of user on file is added explicitly. Refer to these articles more information on migrating images and files and setting dependencies. The following code snippet shows how the destination and dependencies are set:

destination:
  plugin: 'entity:user'
  md5_passwords: true
migration_dependencies:
  required:
    - udm_user_pictures
  optional: []

Processing the fields

The interesting part of a user migration is the field mapping. The specific transformation will depend on your source, but some arguably complex cases will be addressed in the example. Let’s start with the basics: verbatim copies from source to destination. The following snippet shows three mappings:

mail: user_email
init: user_email
timezone: user_timezone

The mail, init, and timezone entity properties are copied directly from the source. Both mail and init are email addresses. The difference is that mail stores the current email, while init stores the one used when the account was first created. The former might change if the user updates its profile, while the latter will never change. The timezone needs to be a string taken from a specific set of values. Refer to this page for a list of supported timezones.

name:
  - plugin: machine_name
    source: public_name
  - plugin: make_unique_entity_field
    entity_type: user
    field: name
    postfix: _

The name, entity property stores the username. This has to be unique in the system. If the source data contained a unique value for each record, it could be used to set the username. None of the unique source columns (eg., legacy_id) is suitable to be used as username. Therefore, extra processing is needed. The machine_name plugin converts the public_name source column into transliterated string with some restrictions: any character that is not a number or letter will be converted to an underscore. The transformed value is sent to the make_unique_entity_field. This plugin makes sure its input value is not repeated in the whole system for a particular entity field. In this example, the username will be unique. The plugin is configured indicating which entity type and field (property) you want to check. If an equal value already exists, a new one is created appending what you define as postfix plus a number. In this example, there are two records with public_name set to Benjamin. Eventually, the usernames produced by running the process plugins chain will be: benjamin and benjamin_1.

process:
  pass:
    plugin: callback
    callable: md5
    source: user_password
destination:
  plugin: 'entity:user'
  md5_passwords: true

The pass, entity property stores the user’s password. In this example, the source provides the passwords in plain text. Needless to say, that is a terrible idea. But let’s work with it for now. Drupal uses portable PHP password hashes implemented by PhpassHashedPassword. Understanding the details of how Drupal converts one algorithm to another will be left as an exercise for the curious reader. In this example, we are going to take advantage of a feature provided by the migrate API to automatically convert MD5 hashes to the algorithm used by Drupal. The callback plugin is configured to use the md5 PHP function to convert the plain text password into a hashed version. The last part of the puzzle is set, in the process section, the md5_passwords configuration to true. This will take care of converting the already md5-hashed password to the value expected by Drupal.

Note: MD5-hash passwords are insecure. In the example, the password is encrypted with MD5 as an intermediate step only. Drupal uses other algorithms to store passwords securely.

status:
  plugin: static_map
  source: user_status
  map:
    inactive: 0
    active: 1

The status, entity property stores whether a user is active or blocked from the system. The source user_status values are strings, but Drupal stores this data as a boolean. A value of zero (0) indicates that the user is blocked while a value of one (1) indicates that it is active. The static_map plugin is used to manually map the values from source to destination. This plugin expects a map configuration containing an array of key-value mappings. The value from the source is on the left. The value expected by Drupal is on the right.

Technical note: Booleans are true or false values. Even though Drupal treats the status property as a boolean, it is internally stored as a tiny int in the database. That is why the numbers zero or one are used in the example. For this particular case, using a number or a boolean value on the right side of the mapping produces the same result.

In the next blog post, we will continue with the user migration. Particularly, we will explain how to migrate the user creation time, roles, and profile pictures.

What did you learn in today’s blog post? Have you migrated user passwords before, either in plain text or hashed? Did you know how to prevent duplicates for values that need to be unique in the system? Were you aware of the plugin that allows you to manually map values from source to destination? Please share your answers in the comments. Also, I would be grateful if you shared this blog post with others.

Next: Migrating users into Drupal - Part 2

This blog post series, cross-posted at UnderstandDrupal.com as well as here on Agaric.coop, is made possible thanks to these generous sponsors. Contact Understand Drupal if your organization would like to support this documentation project, whether it is the migration series or other topics.

Those could be separated by a comma using CSS. For example, "By Clayton Dewey, Ben Melançon, David Valdez."

A more natural display, though, would be "By Clayton Dewey, Ben Melançon, and David Valdez." If, instead of three people, two people wrote the article (as is the case), we would want to display "By Clayton Dewey and Ben Melançon."

Before, a sitebuilder would have to abandon structured content for a text field which an editor would use instead. In other words, instead of using a reference field to display the authors (even if present on the edit form), the sitebuilder would add a regular text field for the content editor to write out the list in a more human-friendly way and that is what would be output, or the content editor would write the byline into the body field.

Now, displaying lists of structured content naturally is possible with the In Other Words module.

(More featureful listings are also possible with Twig templates, but that does not empower sitebuilders— it bogs down themers!)

For text list fields and entity reference fields, In Other Words provides the In Other Words: List field formatter. Its settings page looks like this: