Blog

Get notified of trainings

Learn when we have new opportunities for learning (two to four announcements a year).

IV. Movements and Networks

Free Software Movement

Free/Libre software is software that asserts the 4 fundamental software freedoms to every user. From Drupal modules to entire platforms, everything we build is Free Software because we believe that individuals and communities alike should have as much ownership as possible over the software they use. We even advise our clients in our blog on how to perform daily business operations using Free Software. Agaric's are members and supporters of the Free Software Foundation.

Design Justice Network

We design sites according to the principles of Design Justice. This means that the user experience of marginalized individuals is central to our design considerations. As a part of this network we share ideas and learn techniques to provide equitable online experiences for all. You may not know what it is like to be left out, until you are. 

MayFirst Movement Technology

MayFirst Movement Technology (MFMT) is a cooperative, of movement organizations and activists, that provides web hosting, email services and other free software tools to its members. MFMT has a bold and progressive mission and agrees not turn over your data to third parties under any circumstances.

As members of MFMT, we support the organization by raising awareness on ethical hosting and free software community standards. We encourage people and organizations to join and host their websites through MFMT and gain access to a suite of free software tools and services. Micky is an active board member, involved in discussions of governance as well as the free software tools and platforms that MayFirst offers. Your web hosting does matter, and so does your voice!

US Federation of Worker-Owned Cooperatives

USFWC is a non-profit 501c3 that is active in supporting worker-owned cooperatives while building the solidarity economy together. Currently we are building deeper ties with other tech coops within the USFWC to help advance economic justice in the tech industry as we build technology and support social justice movements, enhance trust and work with them. The federation works closely with DAWI - Democracy at Work Institute to educate and support members having a voice in their workplace.

U.S. Solidarity Economy Network

The USSEN is dedicated to exploring equitable and ethical economies that benefit community members and can interact on a larger scale to define methods and processes for promoting the growth and sustainability of each community as it relates to the world around it. As an organization, we are a member of the North American chapter of RIPESS which is part of an international organization that is an umbrella for groups like USSEN, worldwide. We also create Resist and Build  workshops that support the Solidarity movement.

Tech Coop Network

The Tech Coop Network was formed at a USFWC conference in Los Angeles in 2017 by members of several worker-owned tech and web development cooperatives. We are currently discussing bylaws and governance to prepare for inviting other cooperatives to join us. We meet monthly and are establishing our communications infrastructure.

International Workers of the World

Some Agaric members have joined the Boston Chapter of the IWW. This chapter has a horrible and ignoble past that the current members are determined to overcome. Current members are committed to rebuilding this branch to exemplify the very causes it was based upon before it was destroyed from within. We stand with marginalized people and communities. We represent the disenfranchised worker and support non-conforming individuals rights. An injury to one is an injury to all!

The Digital Fourth

As members of the Boston Chapter, we stay informed on new laws and changes to current laws that could effect our online presence as well as our real life freedoms. Agaric enjoys keeping up with the digital legal strategies and will voice our opinion on whether digital citizens will be affected negatively or positively by pending legislation in our state and sometimes nationally. Artificial intelligence and machine learning are two complex areas of technology and they need to be addressed by groups like us, with diverse knowledge. Surveillance by the Government is also a priority and we publish our findings and recommendations. We cannot change the things that we do not know about. D4th is a part of the nationwide coalition Restore The Fourth.

Science for the People

Scientists and citizens convene to discuss and create ways for the general population to get involved in all things scientific. We believe that you should not need a degree to have access to people in fields of scientific study. We work to remove barriers and make connections between scientists and average citizens people with good ideas and questions that could benefit scientific understanding. Agaric is very interested in  connecting people, projects and promoting discussions in this group. Science for the People has online meetings and is open to all.

As it was for much of the world, 2018 was a combination of extremes for Agaric and the free and open web. Happily, we expanded our team, launched new sites, and empowered our clients through libre software. Unhappily, many of us and our communities endured health issues, political instability, and the effects of climate change.

For the open web, we disappointedly saw the United States officially end Net Neutrality while we excitedly watched the European Union begin enforcing comprehensive privacy laws with its General Data Protection Regulation. We were disgusted by tech giants like Facebook and Palantir diverting and deflecting from the abuses they carry out, but we were also inspired by workers at companies like Amazon and Google forcing their bosses to do better.

In looking back, we celebrate the victories and learn from the challenges—with our eyes set on serving our clients better, expanding the open web, and building an economy based on solidarity rather than exploitation.

To that end, here are the highlights of our work from last year and our intentions for the new year.

We have already covered two of many ways to migrate images into Drupal. One example allows you to set the image subfields manually. The other example uses a process plugin that accomplishes the same result using plugin configuration options. Although valid ways to migrate images, these approaches have an important limitation. The files and images are not removed from the system upon rollback. In the previous blog post, we talked further about this topic. Today, we are going to perform an image migration that will clear after itself when it is rolled back. Note that in Drupal images are a special case of files. Even though the example will migrate images, the same approach can be used to import any type of file. This migration will also serve as the basis for explaining migration dependencies in the next blog post.

File entity migrate destination

All the examples so far have been about creating nodes. The migrate API is a full ETL framework able to write to different destinations. In the case of Drupal, the target can be other content entities like files, users, taxonomy terms, comments, etc. Writing to content entities is straightforward. For example, to migrate into files, the process section is configured like this:

destination:
  plugin: 'entity:file'

You use a plugin whose name is entity: followed by the machine name of your target entity. In this case file. Other possible values are user, taxonomy_term, and comment. Remember that each migration definition file can only write to one destination.

Source section definition

The source of a migration is independent of its destination. The following code snippet shows the source definition for the image migration example:

source:
  constants:
    SOURCE_DOMAIN: 'https://agaric.coop'
    DRUPAL_FILE_DIRECTORY: 'public://portrait/'
  plugin: embedded_data
  data_rows:
    - photo_id: 'P01'
      photo_url: 'sites/default/files/2018-12/micky-cropped.jpg'
    - photo_id: 'P02'
      photo_url: ''
    - photo_id: 'P03'
      photo_url: 'sites/default/files/pictures/picture-94-1480090110.jpg'
    - photo_id: 'P04'
      photo_url: 'sites/default/files/2019-01/clayton-profile-medium.jpeg'
  ids:
    photo_id:
      type: string

Note that the source contains relative paths to the images. Eventually, we will need an absolute path to them. Therefore, the SOURCE_DOMAIN constant is created to assemble the absolute path in the process pipeline. Also, note that one of the rows contains an empty photo_url. No file can be created without a proper URL. In the process section we will accommodate for this. An alternative could be to filter out invalid data in a source clean up operation before executing the migration.

Another important thing to note is that the row identifier photo_id is of type string. You need to explicitly tell the system the name and type of the identifiers you want to use. The configuration for this varies slightly from one source plugin to another. For the embedded_data plugin, you do it using the ids configuration key. It is possible to have more than one source column as identifier. For example, if the combination of two columns (e.g. name and date of birth) are required to uniquely identify each element (e.g. person) in the source.

You can get the full code example at https://github.com/dinarcon/ud_migrations The module to enable is UD migration dependencies introduction whose machine name is ud_migrations_dependencies_intro. The migration to run is udm_dependencies_intro_image. Refer to this article to learn where the module should be placed.

Process section definition

The fields to map in the process section will depend on the target. For files and images, only one entity property is required: uri. Its value should be set to the file path within Drupal using stream wrappers. In this example, the public stream (public://) is used to store the images in a location that is publicly accessible by any visitor to the site. If the file was already in the system and we knew the path the whole process section for this migration could be reduced to two lines:

process:
  uri: source_column_file_uri

That is rarely the case though. Fortunately, there are many process plugins that allow you to transform the available data. When combined with constants and pseudofields, you can come up with creative solutions to produce the format expected by your destination.

Skipping invalid records

The source for this migration contains one record that lacks the URL to the photo. No image can be imported without a valid path. Let’s accommodate for this. In the same step, a pseudofield will be created to extract the name of the file out of its path.

psf_destination_filename:
  - plugin: callback
    callable: basename
    source: photo_url
  - plugin: skip_on_empty
    method: row
    message: 'Cannot import empty image filename.'

The psf_destination_filename pseudofield uses the callback plugin to derive the filename from the relative path to the image. This is accomplished using the basename PHP function. Also, taking advantage of plugin chaining, the system is instructed to skip process the row if no filename could be obtained. For example, because an empty source value was provided. This is done by the skip_on_empty which is also configured log a message to indicate what happened. In this case, the message is hardcoded. You can make it dynamic to include the ID of the row that was skipped using other process plugins. This is left as an exercise to the curious reader. Feel free to share your answer in the comments below.

Tip: To read the messages log during any migration, execute the following Drush command: drush migrate:messages [migration-id].

Creating the destination URI

The next step is to create the location where the file is going to be saved in the system. For this, the psf_destination_full_path pseudofield is used to concatenate the value of a constant defined in the source and the file named obtained in the previous step. As explained before, order is important when using pseudofields as part of the migrate process pipeline. The following snippet shows how to do it:

psf_destination_full_path:
  - plugin: concat
    source:
      - constants/DRUPAL_FILE_DIRECTORY
      - '@psf_destination_filename'
  - plugin: urlencode

The end result of this operation would be something like public://portrait/micky-cropped.jpg. The URI specifies that the image should be stored inside a portrait subdirectory inside Drupal’s public file system. Copying files to specific subdirectories is not required, but it helps with file organizations. Also, some hosting providers might impose limitations on the number of files per directory. Specifying subdirectories for your file migrations is a recommended practice.

Also note that after the URI is created, it gets encoded using the urlencode plugin. This will replace special characters to an equivalent string literal. For example, é and ç will be converted to %C3%A9 and %C3%A7 respectively. Space characters will be changed to %20. The end result is an equivalent URI that can be used inside Drupal, as part of an email, or via another medium. Always encode any URI when working with Drupal migrations.

Creating the source URI

The next step is to create assemble an absolute path for the source image. For this, you concatenate the domain stored in a source constant and the image relative path stored in a source column. The following snippet shows how to do it:

psf_source_image_path:
  - plugin: concat
    delimiter: '/'
    source:
      - constants/SOURCE_DOMAIN
      - photo_url
  - plugin: urlencode

The end result of this operation will be something like https://agaric.coop/sites/default/files/2018-12/micky-cropped.jpg. Note that the concat and urlencode plugins are used just like in the previous step. A subtle difference is that a delimiter is specifying in the concatenation step. This is because, contrary to the DRUPAL_FILE_DIRECTORY constant, the SOURCE_DOMAIN constant does not end with a slash (/). This was done intentionally to highlight two things. First, it is important to understand your source data. Second, you can transform it as needed by using various process plugins.

Copying the image file to Drupal

Only two tasks remain to complete this image migration: download the image and assign the uri property of the file entity. Luckily, both steps can be accomplished at the same time using the file_copy plugin. The following snippet shows how to do it:

uri:
  plugin: file_copy
  source:
    - '@psf_source_image_path'
    - '@psf_destination_full_path'
  file_exists: 'rename'
  move: FALSE

The source configuration of file_copy plugin expects an array of two values: the URI to copy the file from and the URI to copy the file to. Optionally, you can specify what happens if a file with the same name exists in the destination directory. In this case, we are instructing the system to rename the file to prevent name clashes. The way this is done is appending the string _X to the filename and before the file extension. The X is a number starting with zero (0) that keeps incrementing until the filename is unique. The move flag is also optional. If set to TRUE it tells the system that the file should be moved instead of copied. As you can guess, Drupal does not have access to the file system in the remote server. The configuration option is shown for completeness, but does not have any effect in this example.

In addition to downloading the image and place it inside Drupal’s file system, the file_copy also returns the destination URI. That is why this plugin can be used to assign the uri destination property. And that’s it, you have successfully imported images into Drupal! Clever use of the process pipeline, isn’t it? ;-)

One important thing to note is an image’s alternative text, title, width, and height are not associated with the file entity. That information is actually stored in a field of type image. This will be illustrated in the next article. To reiterate, the same approach to migrate images can be used to migrate any file type.

Technical note: The file entity contains other properties you can write to. For a list of available options check the baseFieldDefinitions() method of the File class defining the entity. Note that more properties can be available up in the class hierarchy. Also, this entity does not have multiple bundles like the node entity does.

What did you learn in today’s blog post? Had you created file migrations before? If so, had you followed a different approach? Did you know that you can do complex data transformations using process plugins? Did you know you can skip the processing of a row if the required data is not available? Please share your answers in the comments. Also, I would be grateful if you shared this blog post with your colleagues.

Next: Introduction to migration dependencies in Drupal

This blog post series, cross-posted at UnderstandDrupal.com as well as here on Agaric.coop, is made possible thanks to these generous sponsors. Contact Understand Drupal if your organization would like to support this documentation project, whether it is the migration series or other topics.

We had a fantastic and fun 2019! Agaric team members attended many conferences and hosted multiple trainings while building sites with some amazing clients. We also took on maintenance for a few sites, helping them gain more audience and stability as well as ease of use for visitors and administrators.

Platform Cooperativism Conference

Wow - more than a month has gone by since the New year began! Heads down, we now surface for some news an announcements on the who, what, where and when of the past year and the beginning of 2020. At the end of last year, Agaric members hosted a Town Hall style session at a conference titled: "Who Owns the World?" This was the yearly Platform Cooperativism event convened by Trebor Scholz at the New School in NYC. It was an incredible way to end the year. We met so many people involved in so many worthy projects that it was a bit overwhelming! 

Attending the 2019 Platformcoop event in NYC was an incredible way to end 2019. We met so many people involved in so many worthy projects that it was a bit overwhelming!  The biggest take-aways were learning about how many successful bicycle delivery platform cooperatives there are in many countries. We also found out that worker-owned cooperatives are sparse in places like Indonesia and Japan even though they have a healthy cooperative ecosystem. Cooperative governance is still a wide-open discussion going on amongst many cooperatives. It was nice to have some discussions with other cooperatives on how they go about creating their governance. We have been in touch with several people that we met at the conference this year.

Since the Platformcoop event in NYC, Agaric has been busy working hard on building two platforms. The first one to mention is the Find It platform, a robust opportunity locator, a participatory searchable directory of events, programs and organizations.

Find It Platform

Last year we connected with the City of Cambridge, Massachusetts as they were met with the challenge of getting their residents more engaged in community activities and increasing usage of public spaces. We worked closely with the Kids Council in Cambridge to coordinate an effort between government, the residents of the city, and local coding groups to design a platform that would provide residents the means to more easily navigate a directory of public events based on their age and interests.

The resulting platform is named Find It, and it was developed by our team of worker-owners at Agaric.

To give you an idea of the value the platform can bring to the residents of a city, here is a PSA on Find It to the residents of Cambridge.

After seeing how successful Find It has been in enriching the communities in Cambridge, we are eager to invite other cities to join in the collaboration. FindIT is built on Drupal, free and open-source software that ensures adopters a powerful and affordable tool that can be altered to suit the needs of the city it serves. Better yet, surveys and interview questions for city residents are already prepared from the Find It Cambridge collaboration. As more cities get involved in the Find It project, we will be able to increase its capabilities all the while decreasing the cost of the service.

Drutopia

A budding platform cooperative that Agaric has collaboratively built with other Drupal shops and freelance contributors is, Drutopia, an ecosystem of Drupal distributions, built and managed as a software cooperative. Soon Agaric will be offering what we call a Libre Software as a Service program, which will allow people to host their site through us, similar to online website builders but governed democratically and with the power of Drupal behind it all. You can find more information on the Drutopia website: https://drutopia.org

Contributing Back to Drupal

Working with Drupal, Agaric has led some data migration training workshops at some Drupal camps. We hosted a migration training in San Francisco at BADcamp - Bay Area Drupal Camp as well as a session on growing the community and scaling democracy. The team hosted a full day, sold-out training for 45 people at DrupalCon Seattle in 2019. We are engaged to host more trainings at the upcoming 2020 DrupalCon in Minnesota in May of 2020. We recently hosted a migration training at DrupalCamp New Jersey at Princeton and these are our upcoming scheduled data migration trainings:

• March 18 MidCamp - Drupal 8 Upgrade All Day Training 

• May 18 DrupalCon Minneapolis - Drupal 8 Content Migrations All Day Training 

• May 19 DrupalCon Minneapolis - Upgrading to Drupal 8 Using the Migrate API All Day Training 

You can see the list and join us  - https://agaric.coop/blog/make-2020-year-you-begin-upgrade-drupal-8-upcoming-trainings-and-resources

We have also been busy working on contributed modules. Currently, Gnuget is working on making https://www.drupal.org/project/filefield_sources stable and at the same time compatible with Drupal 9.

Forging Connections in Mexico

Some Agarics are also headed back to San Miguel de Allende, Mexico to continue a series of lectures and workshops on security, privacy, surveillance, and Platform Cooperativism at the local Biblioteque, in mid-February. The lecture is titled: Platform Cooperativism, Surveillance Capitalism, Predictive Analysis and You.

In 2018, we made our first visit to the Center for Global Justice in San Miguel de Allende when Micky was asked to speak at an Encuentro (gathering) in Mexico City to celebrate the victory of Mexico's electrical workers' union (SME). SME workers were fired en masse in 2009 but stayed united and fought back. They now provide electricity to central Mexico as a co-op under democratic workers' control!  This visit activated an idea to organize trips to Mexico to host workshops and lectures specifically on software freedom and security with an introduction to protecting your privacy online. The next Workshop/Lecture tour will happen February 2020 - We hosted some workshops earlier this year and you can read about them here: https://agaric.coop/blog/micky-lead-free-software-trainings-mexico 

2019 was an exciting year for Agaric as we traveled to teach and to learn. Micky presented as a Keynote speaker at two major events for developers in New England, NerdSummit and the Free Software Foundation's yearly event, LibrePlanet. She will be back again this year with a presentation on Surveillance Capitalism, Predictive Analysis and YOU. View the LibrePlanet Keynote here:

MayFirst Movement Technology

As active members of MayFirst Movement Technology, an organization providing web hosting, email services, and other free software tools to its members. MFMT also advocates for an open web and ethical technology. This year, Micky was elected by the members to serve on the board of the new cooperative formed by the members. Together, we are building online and offline workshops and discussions on freedom and raising awareness on the need to have a web host that is ethical and that you truly trust to protect you and your data.

Agaric continues to host a weekly online gathering where everyone shares methods and processes and ideas for sharing our knowledge and learning from others. You can join us any Wednesday for Show and Tell - lurk, engage or contribute! We also have some new Agaric initiatives and we have launched some new client websites as well as redesigns of other sites. We continue to reach out to share the knowledge we have gained while including as many people as we can in the process of learning to use free software to make a better planet and to make a better life for all!

Creating an issue fork to contribute to a Drupal module, without getting confused.

(This cheat sheet is for myself. Fellow cheaters welcome.)

Check the branch that the issue is against— even if you filed the issue yourself, it should have defaulted to the branch people should be working on.

Go to the module page and scroll down, if you are lucky there will be a dev release toward the bottom, for example "Development version: 2.0.x-dev", which you can click into and copy the composer command.

(If there is not a dev release, you'll have to look at Development in the sidebar, click through "Source code" and copy from the Code dropdown either the URL under Clone with SSH or Clone with HTTPS, and git clone that into a modules.

Press the Create issue fork button and wait a minute.

Press the "Show commands" link, and copy-paste the commands in the Add & fetch this issue fork’s repository

And then copy-paste the commands from Check out this branch for the first time

Assign to yourself in "Issue metadata fieldset and save the issue form with any comment about how you plan to work on this that you hope somebody might provide feedback on or as a reminder to your later self.

On my quest to improve a client's Drupal site performance I considered installing the Alternative PHP Cache. It reduces the overhead of compiling the PHP sources into opcode on each request by caching the compiled code¹. 2bits posted a very good case study about PHP opcode caches a while ago.

I have seen significant performance improvements from opcode caches in past Drupal projects. But every site is different. Usually the relative efficiency of an opcode cache correlates with the share the bootstrap process takes of total page rendering time. This can be easily measured with a profiling tool like Xdebug and a visualization software (Kcachegrind is an excellent free software product, for MacOS X there's MacCallGrind).

 

 

 

Visualization of Xdebug profiling results with MacCallGrind

After the first profiling run, it became clear that the bootstrap process only amounted to about 10% of total page rendering time. Since the full rendering period was an order of magnitude greater than the whole bootstrapping period, any performance improvement in the bootstrapping process would have no perceivable performance impact. Nevertheless I took some time to make some observations. I was especially curious about comparing APC running with apc.stat enabled and disabled.

For the purpose of this test I put the site on a Virtual Box machine running Debian Wheezy: PHP 5.3.10 with Xdebug 2.1.3, MySQL 5.1.58, PHP APC 3.1.2, Apache 2.2.22. Some initial test runs were made to establish the amount of memory APC needed to avoid cache resets, the actual experiment used a value of 128 MB for apc.shm_size. To warm up the APC cache two requests to the page were made before starting each test series. A test series consists of requesting the homepage with profiling enabled and noting the total execution time of drupal_bootstrap and the share it has of the total page rendering time — repeated 10 times.

 

 

 

Execution time of drupal_bootstrap with APC cache disabled

 

 

 

Execution time of drupal_bootstrap with APC cache enabled

The results show a reduction in execution time between 25% and 60% taking the standard deviation into account. Disabling apc.stat had no measurable effect.

1. PHP has been designed for adding dynamic content to web pages by embedding snippets of code (<?php ... ?>) in HTML markup. Still the most popular way of running PHP is by means of Apache with mod_php which is originally written for that use case. Each time a request comes to a page containing PHP that code is parsed and executed in fractions of a second. The growth of the PHP community and increasing complexity of problems being solved with PHP has led to the development of ever more complex software - like Drupal. If you look at the source of Drupal most of the files contain purely PHP with a little bit of HTML here and there, primarily in the themes. Now the cost of the parser loading and compiling the sources into byte code to be executed by the interpreter has become a challenge. An opcode cache like APC saves time because it remembers the compiled code in memory thereby reducing the overhead per request.

This covers 90% of use cases, but what if we need to add a button for a new network?

Creating a Custom Social Simple Button

The Social Simple module already supports custom buttons, we just need to let the module know that we want to add one.

What we need to do is:

• Create a class that implements SocialNetworkInterface.
• Register this class in our services file.
• Add the tag social_simple_network to our service.

For our example we are going to create a basic Mail button. We start by creating a custom module. Inside our module let's create a Mail php file inside of the src/SocialNetwork folder:

mkdir -p src/SocialNetwork cd src/SocialNetwork touch Mail.php

The next step is to create a class and implement the SocialNetworkInterface which interface has the following methods:

• getShareLink: This is the most important method. It must return a rendered array which later Drupal will use to create the button.
• getLabel: Here we will need to provide the name of our button. In our case Mail.
• getId: The ID of the button. We can choose any ID here, we just need to make sure that it is unique. Let's use mail for our example.
• getLinkAttributes: These attributes are going to be passed to the link. We can add custom parameters to the link in this part.

Our class looks like this:

namespace Drupal\social_simple\SocialNetwork; use Drupal\Core\Entity\EntityInterface; 

use Drupal\Core\StringTranslation\StringTranslationTrait;
use Drupal\Core\Url; 

/** * The Mail button. */
class Mail implements SocialNetworkInterface { 

use StringTranslationTrait; 

/** * The social network base share link. */
const MAIL = 'mailto:'; 

/** * {@inheritdoc} */
public function getId() {
  return 'mail';
} 

/** * {@inheritdoc} */
public function getLabel() {
  return $this->t('Mail');
} 

/** * {@inheritdoc} */
public function getShareLink($share_url, $title = '', EntityInterface $entity = NULL, array $additional_options = []) {
  $options = [
    'query' => [
      'body' => $share_url,
      'subject' => $title,
    ],
    'absolute' => TRUE,
    'external' => TRUE,
  ]; 

  if ($additional_options) {
    foreach ($additional_options as $id => $value) {
      $options['query'][$id] = $value;
    }
  }
  $url = Url::fromUri(self::MAIL, $options);
  $link = [
    'url' => $url,
    'title' => ['#markup' => '' . $this->getLabel() . ''],
    'attributes' => $this->getLinkAttributes($this->getLabel()),
  ]; return $link;
} 

/** * {@inheritdoc} */
public function getLinkAttributes($network_name) {
  $attributes = [ 'title' => $network_name, ];
  return $attributes;
  }
}

The next step is to let the social network know about our new button and we do this by adding this class as a service in our module.services.yml. If you are not familiar with this file, you can read the structure of a service file documentation..

Basically we need to add something like this:

services: 
  social_simple.mail: 
    class: Drupal\custom_module\SocialNetwork\Mail 
    tags: - { name: social_simple_network, priority: 0 }

Next, rebuild the cache. Now when we visit the social simple configuration we will see our new button there, ready to be used.

The only thing that we need to pay extra attention to is that the Social Simple module will just search the services with the tag social_simple_network otherwise our class will not be found.

If you want to see how the whole thing is working, you can check this patch that I made as a part of a project: https://www.drupal.org/project/social_simple/issues/2899517. As a bonus, I made an initial integration with the Forward module.

The Views module provides a flexible method for Drupal site builders to present data. On a recent project we needed to filter a view's result set in a way we could not achieve by means of the module's UI. How do you programmatically alter a view's result set before rendering? Let's see how to do it using the hooks provided by the module.

The need surfaced while working on the web site for MIT's Global Studies and Languages department, which uses Views to pull in data from a remote service and display it. The Views module provides a flexible method for Drupal site builders to present data. Most of the time you can configure your presentation needs through the UI using Views and Views-related contributed modules. Notwithstanding, sometimes you need to implement a specific requirement which is not available out of the box. Luckily, Views provides hooks to alter its behavior and results. Let’s see how to filter Views results before they are rendered.

Assume we have a website which aggregates book information from different sources. We store the book name, author, year of publication, and ISBN (International Standard Book Number). ISBNs are unique numerical book identifiers which can be 10 or 13 characters long. The last digit in either version is a verification number and the 13 character version has a 3-character prefix. The other numbers are the same. A book can have both versions. For example:

ISBN10:    1849511160
ISBN13: 9781849511162

In our example website, we only use one ISBN. If both versions are available, the 10-character version is discarded. We do this to prevent duplicate book entries which differ only in ISBN as shown in the following picture:

To remove the duplicate entries, follow this simple two step process:

1. Find the correct Views hook to implement.
2. Add the logic to remove unwanted results.

After reviewing the list of Views hooks, hook_views_pre_render is the one we are going to use to filter results before they are rendered. Now, let’s create a custom module to add the required logic. I have named my module views_alter_results so the hook implementation would look like this:

/**
 * Implements hook_views_pre_render().
 */
function views_alter_results_views_pre_render(&$view) {
  // Custom code.
}

The ampersand in the function parameter indicates that the View object is passed by reference. Any change we make to the object will be kept. The View object has a results property. Using the devel module, we can use dsm($view->results) to have a quick look at the results.

Each element in the array is a node that will be displayed in the final output. If we expand one of them, we can see more information about the node. Let’s drill down into one of the results until we get to the ISBN.

The output will vary depending on your configuration. In this example, we have created a Book content type and added an ISBN field. Before adding the logic to filter the unwanted results, we need to make sure that this logic will only be applied for the specific view and display we are targeting. By default, hook_views_pre_render will be executed for every view and display unless otherwise instructed. We can apply this restriction as follows:

/**
 * Implements hook_views_pre_render().
 */
function views_alter_results_views_pre_render(&$view) {
  if ($view->name == 'books'
    && $view->current_display == 'page_book_list') {
    // Custom code.
  }
}

Next, the logic to filter results.

/**
 * Implements hook_views_pre_render().
 */
function views_alter_results_views_pre_render(&$view) {
  if ($view->name == 'books'
    && $view->current_display == 'page_book_list') {
    $isbn10_books = array();
    $isbn13_books = array();
    $remove_books = array();

    foreach ($view->result as $index => $value) {
      $isbn = $value->field_field_isbn[0]['raw']['value'];
      if (strlen($isbn) === 10) {
        // [184951116]0.
        $isbn10_books[$index] = substr($isbn, 0, 9);
      }
      elseif (strlen($isbn) === 13) {
        // 978[184951116]2.
        $isbn13_books[$index] = substr($isbn, 3, 9);
      }
    }

    // Find books that have both ISBN10 and ISBN13 entries.
    $remove_books = array_intersect($isbn10_books, $isbn13_books);

    // Remove repeated books.
    foreach ($remove_books as $index => $value) {
      unset($view->result[$index]);
    }
  }
}

To filter the results we use unset on $view->result. After this process, the result property of the view object will look like this:

And our view will display without duplicates book entries as seen here:

Before wrapping up, I’d like to share two modules that might help you achieve similar results: Views Merge Rows and Views Distinct. Every use case is different, if neither of these modules gets you where you want to be, you can leverage hook_views_pre_render to implement your custom requirements.

Update #1 Tue, 06/02/2015

As indicated by Leon and efpapado this approach only works for views that present all results in a single page. That was the original use case. The altering presented here only affects the current page and the pager will not work as expected.

The grocery store was open for a brief time, but it was never a cooperative. I know, because I joined as a member the day it first opened in 2017, on August 11. (I first e-mailed to ask about becoming a member ten months earlier, but that required meeting in person and this was the first time it was feasible.)

On 2018, June 12, after Wirth Cooperative Grocery had been closed for two months, I received my only formal communication as a member owner: an e-mail acknowledging that the store was closed, noting that the current grocery market is extremely competitive, and saying they had plans to re-open by the end of the month.

The e-mail did not ask for decisions. It did not ask for volunteers or help of any kind. While addressed to us as member owners, it did not afford us any opportunity to affect the future of the store. It did not provide any information on which we might try to act. Instead it told us to wait to be notified when an opening date was set. An opening date was never set.

Although I'm certain some staff and volunteers worked hard behind the scenes, from my perspective as a member owner the grocery store went down without a fight.

That's why it's so important for cooperatives to be true cooperatives. The seven cooperative principles aren't a "pick any three" sort of deal.

The first principle specifies that membership in a cooperative be open to all people willing to accept the responsibilities, but a person cannot accept responsibilities which aren't offered.

The second principle is democratic member control, but people cannot exercise direct control or hold representatives accountable without information and without a means to communicate with one another.

Likewise for the next three cooperative principles: the third, that members democratically control capital; the fourth, that external agreements preserve democratic control; and the fifth, that a cooperative educate, train, and inform members so they can contribute to it effectively. An organization with no mechanisms for discussion nor for democracy violates these principles, too.

Principles six and seven, cooperation among cooperatives and concern for community, are likely to be hollow without functioning internal democracy— and certainly cannot be realized if the business fails.

A cooperative can't exist only on good intentions.

When I hear this sentiment expressed by experienced cooperators—founders and developers of cooperatives—it usually means that there needs to be a solid business model for a cooperative, because a cooperative that isn't economically viable can't fulfill any of its goals.

A more fundamental meaning is that a business can't be a cooperative if it merely intends to be; it must act like a cooperative. Otherwise, it's just another business with some co-op lip service gloss— and given the greater success of cooperatives compared to other businesses it's less likely to be around as a business at all, if it does not live up to its cooperative principles.

I'm not trying to use technicalities to dodge counting the failed Wirth grocery store as a failure for "team cooperative". On the contrary, this is a wakeup call for everybody who supports cooperatives, one that must rouse us, because fully-functioning cooperatives are bigger than the cooperative movement. Cooperatives can prefigure true democracy. We need to show that economic democracy works in our voluntary organizations; we need to give people in a world which is already ruled unjustly, and threatening to spiral out of control, a promise and a practice for how whole communities can take collective control.

In my experience as a member owner, Wirth Cooperative Grocery was not a co-op beyond its name and some vague intentions. Now I know that my experience matched everyone else's, thanks to Cirien Saadeh's reporting, both last year and in the most recent issue of North News (an invaluable local non-profit enterprise).

What worries me, then, is that no one quoted in these articles called out this failure to meet the basic requirements of being a cooperative. Minneapolis and Minnesota have perhaps the highest rate of cooperative membership in the United States, with about half of all residents estimated to belong to at least one cooperative of one kind or another. If we, here, don't have the awareness and interest to note when a cooperative doesn't act like a cooperative, who will?

More important than calling out failures is providing pathways to success. There are many programs and organizations supporting cooperatives in Minnesota and beyond, but none put ensuring member control first.

The bias of my profession and my passion is to lead with an online tool: ensure member owners can communicate with one another. Although a technological fix can't solve political problems, people who are affected need a way to talk among themselves to come up with a solution.

Efforts like a local cooperative grocery are small enough that a Loomio group or any mailing list would mostly work for member owner discussion. A better software application would work for collaborative groups of any size: members would filter messages for quality without ceding control to any outside group. This self-moderation for groups of equals, including cooperatives, is a goal of Visions Unite.

Are you in a position to provide resources to cooperatives and other groups seeking to grow and be democratic? Are you starting or do you want to start a cooperative or group? Do you have other ideas on how to help new and established cooperatives live by the foundational cooperative principles? I would like to hear from you!