I have watched in sadness and sometimes anger as large non-profit after large non-profit collectively poured enough money into Raiser's Edge and other Blackbaud licenses and consulting services to fund many feature enhancements for the main FLOSS alternative, CiviCRM— improvements which would then be free for everyone, forever.

I have never met anyone who actually likes Blackbaud products and services. However, many organizations felt they were the only safe option, in the sense of claiming to have everything an enterprise needs.

Now, Blackbaud failed to secure its servers sufficiently and large amounts of its clients' donor data, including personally identifying information, was obtained in a ransomware attack. This was back in May. Blackbaud ultimately paid the ransomer to allegedly destroy the data they obtained— and only late in July finally told their customers what happened.

As the American Civil Liberties Union wrote to all its supporters, current and past (including myself), this is a rotten situation:

In all candor, we are frustrated with the lack of information we've received from Blackbaud about this incident thus far. The ACLU is doing everything in our power to ascertain the full nature of the breach, and we are actively investigating the nature of the data that was involved, details of the incident, and Blackbaud's remediation plans.

We are also exploring all options to ensure this does not happen again, including revisiting our relationship with Blackbaud.

Fortunately, none of Agaric's clients are affected. But we hope everyone using or considering using Blackbaud and other proprietary services for their most important data will look at free/libre open source solutions. Code you (or your technology partner) can see and contribute to means you truly can do anything. And if you put aside the money that would be gouged out of your organization by the eTapestry, Kintera, and Convio-swallowing monopolist Blackbaud, you probably can afford to.

At Agaric, we have recently been working with CiviCRM more recently (building on experience dating back fifteen years!) and we know our friends at Palante Technology Cooperative and myDropWizard are well-versed in CiviCRM, as are many others. Please consider this when weighing your options for maintaining a strong, ethical relationship with your supporters, and let us know if you have any thoughts or questions!

Last week we were asked to disable the "threads" in the comments and display them as a plain list in one of our projects, so checking in the Comment field settings we found an option which says:

"Threading: Show comment replies in a threaded list."

Which basically display the comments as a threaded instead of a list of comments. This option is marked on by default, so it seemed that it was just a matter of unchecking this option, but there was a problem with this, that setting did display the comments as a plain list, but in the code they were still being saved as a thread.

The main problem with this is if a comment is deleted and it has replies (even if the option of the thread is unchecked) all the replies are going to be deleted as well.

This is not a new problem there is an 11 year old issue and it seems that this has been happening since Drupal 4. The way to fix this so far was by using a contrib module called: Flat Comments The problem is that this module hasn't been ported to D8 yet.

We decided to fix that and port the module to help others with this problem. You can check the code here: https://github.com/agaric/flat_comments and we are in contact with the module's maintainer to create the D8 branch in drupal.org soon.

Expecting that this can help someone else.

All Improvements Are Shared Freely as Free Software

Everything we do at Agaric uses software that is free as in freedom - free to install, use and repurpose for your own needs.

Our work with Portside has already translated into improving and even creating the following Drupal modules:

* Give - on-site donation forms and reporting
* Social Post Twitter - Re-post content from your website to your Twitter account
* Social Share Facebook - Re-post contnt from your website to your Facebook page
* Minimal HTML - A text format handy for short text fields

So, when you donate to Portside's fundraising campaign, you are both supporting independent journalism and the open-source software that benefits other independent outlets and websites.

Donate to Portside

Agaric is facilitating a full day training at DrupalCon Seattle to help you understand how to import content into your to Drupal 8 website.  It's the latest in our series of migration trainings.

This training is open for attendees with intermediate experience with Drupal– familiarity with installing a Drupal site and installing modules. We will use the Migrate API and related modules, which allows users to migrate content without writing any code.

With six instructors we will ensure no one gets left behind. Instead everyone will get the attention they need.

Attendees will learn to:

• Import data from CSV and JSON files.
• Transform data to populate taxonomy, date, image, file, and address fields.
• Get content into Paragraphs.
• Debug migration errors.

The training price is $450. Space is limited so we do encourage registering as soon as possible to ensure your spot.

As a web development cooperative that champions free software, we're passionate about migrations. It is a way to better understand Drupal's codebase, tap into the power of new features and build community. We have successfully migrated multiple sites to Drupal 8 and look forward to sharing our experience with you.

This training has sold out but please get in touch to be the first to know about future opportunities!

In the previous entry, we wrote our first Drupal migration. In that example, we copied verbatim values from the source to the destination. More often than not, the data needs to be transformed in some way or another to match the format expected by the destination or to meet business requirements. Today we will learn more about process plugins and how they work as part of the Drupal migration pipeline.

Syntactic sugar

The Migrate API offers a lot of syntactic sugar to make it easier to write migration definition files. Field mappings in the process section are an example of this. Each of them requires a process plugin to be defined. If none is manually set, then the get plugin is assumed. The following two code snippets are equivalent in functionality.

process:
  title: creative_title

process:
  title:
    plugin: get
    source: creative_title

The get process plugin simply copies a value from the source to the destination without making any changes. Because this is a common operation, get is considered the default. There are many process plugins provided by Drupal core and contributed modules. Their configuration can be generalized as follows:

process:
  destination_field:
    plugin: plugin_name
    config_1: value_1
    config_2: value_2
    config_3: value_3

The process plugin is configured within an extra level of indentation under the destination field. The plugin key is required and determines which plugin to use. Then, a list of configuration options follows. Refer to the documentation of each plugin to know what options are available. Some configuration options will be required while others will be optional. For example, the concat plugin requires a source, but the delimiter is optional. An example of its use appears later in this entry.

Providing default values

Sometimes, the destination requires a property or field to be set, but that information is not present in the source. Imagine you are migrating nodes. As we have mentioned, it is recommended to write one migration file per content type. If you know in advance that for a particular migration you will always create nodes of type Basic page, then it would be redundant to have a column in the source with the same value for every row. The data might not be needed. Or it might not exist. In any case, the default_value plugin can be used to provide a value when the data is not available in the source.

source: ...
process:
  type:
    plugin: default_value
    default_value: page
destination:
  plugin: 'entity:node'

The above example sets the type property for all nodes in this migration to page, which is the machine name of the Basic page content type. Do not confuse the name of the plugin with the name of its configuration property as they happen to be the same: default_value. Also note that because a (content) type is manually set in the process section, the default_bundle key in the destination section is no longer required. You can see the latter being used in the example of writing your Drupal migration blog post.

Concatenating values

Consider the following migration request: you have a source listing people with first and last name in separate columns. Both are capitalized. The two values need to be put together (concatenated) and used as the title of nodes of type Basic page. The character casing needs to be changed so that only the first letter of each word is capitalized. If there is a need to display them in all caps, CSS can be used for presentation. For example: FELIX DELATTRE would be transformed to Felix Delattre.

Tip: Question business requirements when they might produce undesired results. For instance, if you were to implement this feature as requested DAMIEN MCKENNA would be transformed to Damien Mckenna. That is not the correct capitalization for the last name McKenna. If automatic transformation is not possible or feasible for all variations of the source data, take notes and perform manual updates after the initial migration. Evaluate as many use cases as possible and bring them to the client’s attention.

To implement this feature, let’s create a new module ud_migrations_process_intro, create a migrations folder, and write a migration definition file called udm_process_intro.yml inside it. Follow the instructions in this entry to find the proper location and folder structure or download the sample module from https://github.com/dinarcon/ud_migrations It is the one named UD Process Plugins Introduction and machine name udm_process_intro. For this example, we assume a Drupal installation using the standard installation profile which comes with the Basic Page content type. Let’s see how to handle the concatenation of first an last name.

id: udm_process_intro
label: 'UD Process Plugins Introduction'
source:
  plugin: embedded_data
  data_rows:
    -
      unique_id: 1
      first_name: 'FELIX'
      last_name: 'DELATTRE'
    -
      unique_id: 2
      first_name: 'BENJAMIN'
      last_name: 'MELANÇON'
    -
      unique_id: 3
      first_name: 'STEFAN'
      last_name: 'FREUDENBERG'
  ids:
    unique_id:
      type: integer
process:
  type:
    plugin: default_value
    default_value: page
  title:
    plugin: concat
    source:
      - first_name
      - last_name
    delimiter: ' '
destination:
  plugin: 'entity:node'

The concat plugin can be used to glue together an arbitrary number of strings. Its source property contains an array of all the values that you want put together. The delimiter is an optional parameter that defines a string to add between the elements as they are concatenated. If not set, there will be no separation between the elements in the concatenated result. This plugin has an important limitation. You cannot use strings literals as part of what you want to concatenate. For example, joining the string Hello with the value of the first_name column. All the values to concatenate need to be columns in the source or fields already available in the process pipeline. We will talk about the latter in a future blog post.

To execute the above migration, you need to enable the ud_migrations_process_intro module. Assuming you have Migrate Run installed, open a terminal, switch directories to your Drupal docroot, and execute the following command: drush migrate:import udm_process_intro Refer to this entry if the migration fails. If it works, you will see three basic pages whose title contains the names of some of my Drupal mentors. #DrupalThanks

Chaining process plugins

Good progress so far, but the feature has not been fully implemented. You still need to change the capitalization so that only the first letter of each word in the resulting title is uppercase. Thankfully, the Migrate API allows chaining of process plugins. This works similarly to unix pipelines in that the output of one process plugin becomes the input of the next one in the chain. When the last plugin in the chain completes its transformation, the return value is assigned to the destination field. Let’s see this in action:

id: udm_process_intro
label: 'UD Process Plugins Introduction'
source: ...
process:
  type: ...
  title:
    -
      plugin: concat
      source:
        - first_name
        - last_name
      delimiter: ' '
    -
      plugin: callback
      callable: mb_strtolower
    -
      plugin: callback
      callable: ucwords
destination: ...

The callback process plugin pass a value to a PHP function and returns its result. The function to call is specified in the callable configuration option. Note that this plugin expects a source option containing a column from the source or value of the process pipeline. That value is sent as the first argument to the function. Because we are using the callback plugin as part of a chain, the source is assumed to be the last output of the previous plugin. Hence, there is no need to define a source. So, we concatenate the columns, make them all lowercase, and then capitalize each word.

Relying on direct PHP function calls should be a last resort. Better alternatives include writing your own process plugins which encapsulates your business logic separate of the migration definition. The callback plugin comes with its own limitation. For example, you cannot pass extra parameters to the callable function. It will receive the specified value as its first argument and nothing else. In the above example, we could combine the calls to mb_strtolower() and ucwords() into a single call to mb_convert_case($source, MB_CASE_TITLE) if passing extra parameters were allowed.

Tip: You should have a good understanding of your source and destination formats. In this example, one of the values to want to transform is MELANÇON. Because of the cedilla (ç) using strtolower() is not adequate in this case since it would leave that character uppercase (melanÇon). Multibyte string functions (mb_*) are required for proper transformation. ucwords() is not one of them and would present similar issues if the first letter of the words are special characters. Attention should be given to the character encoding of the tables in your destination database.

Technical note: mb_strtolower is a function provided by the mbstring PHP extension. It does not come enabled by default or you might not have it installed altogether. In those cases, the function would not be available when Drupal tries to call it. The following error is produced when trying to call a function that is not available: The "callable" must be a valid function or method. For Drupal and this particular function that error would never be triggered, even if the extension is missing. That is because Drupal core depends on some Symfony packages which in turn depend on the symfony/polyfill-mbstring package. The latter provides a polyfill) for mb_* functions that has been leveraged since version 8.6.x of Drupal.

What did you learn in today’s blog post? Did you know that syntactic sugar allows you to write shorter plugin definitions? Were you aware of process plugin chaining to perform multiple transformations over the same data? Had you considered character encoding on the source and destination when planning your migrations? Are you making your best effort to avoid the callback process plugin? Please share your answers in the comments. Also, I would be grateful if you shared this blog post with your colleagues.

Next: Migrating data into Drupal subfields

This blog post series, cross-posted at UnderstandDrupal.com as well as here on Agaric.coop, is made possible thanks to these generous sponsors. Contact Understand Drupal if your organization would like to support this documentation project, whether is the migration series or other topics.

Ayudamos a las organizaciones a cumplir sus objetivos y fortalecer el movimiento de software libre al proporcionar consultoría en estrategia de tecnología en línea, al crear y personalizar software de alta calidad, capacitar a personas y hablar en eventos.

Pregunte a un Agárico

Estrategía

Utilizamos los principios de justicia de diseño para ayudar a que su presencia en línea alcance sus objetivos y tenga un impacto en el mundo real.
 

• Formación analítica
• Gobernanza del contenido
• Estrategia de contenido
• Operaciones de contenido
• Estrategia de redes sociales

Desarrollo

Construimos con un software probado que le da poder y control sobre su sitio web y presencia en línea.
 

• Desarrollo de la gestión de relaciones constituyentes (CRM)
• Migración de contenido
• Desarrollo drupal
• Desarrollo de Django
• Optimización de la donación
• Posicionamiento en buscadores (SEO orgánico)
• Integración de redes sociales
• Seguridad del sitio web y CRM
• Mantenimiento del sitio web
• Afinación del rendimiento del sitio web
• Actualización del sitio web

Usamos y contribuimos con el software libre siempre que sea posible, Creative Commons licencia nuestra documentación y trabajamos bajo un modelo de organización abierta.

Capacitación

Asesoramos y enseñamos, aprovechando su experiencia actual.
 

• Asesoramiento personalizado uno a uno
• Introducción a Drupal
• Desarrollo del módulo drupal.
• Migración drupal
• Seguridad en linea

Obtenga más información sobre las oportunidades para recibir capacitación de Agaric.

Speaking

Compartimos nuestros conocimientos y promovemos el software libre hablando en eventos.

Los temas incluyen:

• Cómo participar en el Común Digital
• Financiamiento efectivo y gobernanza para el software libre
• Ética en la tecnología
• Software Libre = Sociedad Libre
• Economía solidaria
• Desarrollo cooperativo
• Inteligencia Artificial y Vigilancia
• Poder personal
• Drupal - Comunitario y Técnico

Reserve con nosotros para un próximo evento.

La gente a menudo pregunta por las herramientas de software libre que Agaric utiliza para administrar su negocio cooperativo. En este artículo, compartiremos los recursos que elegimos para las tareas diarias de oficina y administración, así como para las comunicaciones en nuestras operaciones comerciales.   

Agaric utiliza software libre siempre que es posible. Por esta razón, creamos sitios web con Drupal, una plataforma  de gestión de contenido que destaca por la calidad de su código y por su amplia variedad de funcionalidades.  Como miembros de la Comunidad Drupal, buscamos contribuir activamente con los grupos que trabajan en encontrar opciones para su mejora y ampliación. 

La necesidad de una tecnología responsable - Parte 1 

Quizás podrán preguntarse por qué decimos "Software Libre" y no "Código Abierto", ya que en ambos casos esencialmente es lo mismo. Nosotros preferimos este término porque incluye los principios éticos sobre el respeto a la libertad del usuario. La ética del software libre dice que los usuarios merecen control sobre el código que usan, mientras que el uso del término "Código abierto" se refiere solo  a permitir que los usuarios participen en el desarrollo. Nosotros apoyamos la idea de un software que proteja nuestros derechos y nos evite estar vulnerables. 

Micky speaks at events and hosts workshops online and around the world on topics including Free Software, Personal Digital Security, Drupal and Platform Cooperatives. She is also a host of a monthly cyber security webinar and a weekly series, Show and Tell with Agaric and friends.

Sign up to be notified when Agaric gives a migration training:

I've always had a passion for good design and healthy coding, even back in the days of owning a web site cart in downtown Natick. Back then, my business partner and I made all natural HTML roll-up web sites and, as an incentive for customers to wait in line, we baked Drupal into different flavored designs.

The Drupal became remarkably popular and before we knew it, the Agaric collective was born.

Today, you can enjoy Agaric's many great flavors. They are baked, all natural and totally delicious. So treat yourself well, and treat yourself often.

Visit http://www.agaric.com to find more about our other sites, discover great recipes, and stay in touch.

Dan Hakimzadeh
Co-Founder

V. Auxiliary Tech Projects

CoopGuide

Working together on projects is one of the most important goals of our tech coop. It is also the 6th Cooperative Principle, Cooperation among Cooperatives.  CoopGuide is a work in progress being built to facilitate team members in locating each other and getting familiar with each other's skills. The idea for this site came from discussions at Agaric's weekly Show and Tell gatherings. 

SnowDrift

A crowdmatching platform that is a fund raising source for public goods. The first project will be to fund the building of Snowdrift itself. Micky is on the board and is working with the team to guide the formation of this cooperative effort.

IEEE.org SA Open

The SA Open is a platform, by the IEEE and managed by LeadingBit, that
aims to create a set of agreed upon standards to help people, both technical and organic, co-work more effectively to build Free Software with proper documentation and support materials on using and changing it. Agaric volunteers as a member of the Community Advisory Group, where we offer our expertise on this very subject. You can get involved with SA Open and help make it happen.

meet.coop

Meet.coop is an online meeting co-operative dedicated to hosting BigBlueButton video chat servers in multiple locations around the world. Agaric is a member.  We meet weekly in different teams to work out the governance and member issues. There is an operational team that builds and maintains the hardware and software for providing an excellent video conference network structured in Sociocracy, also called dynamic governance. Meet.coop has forums where you can mingle with members and get up to date information on the development and learn more about the project.

Mastodon

Mastodon is a federated social network made up of affinity groups. The functionality is much like Twitter, where you can message other members and share information or "Toots" as they are called, in a group or one-on-one. You can join a sub-community or create your own. Agarics are members of the social.coop sub-community. Members can contact members of other sub-communities and there is a directory where you can browse the groups in the whole federation.

Mass Mesh

Mass Mesh is a high-tech social club building community-owned mesh networks throughout Boston. These networks enable home Internet access that is fairer, more secure, and more locally resilient than traditional cable. The purpose of this wiki is to provide you with all of the information you need to join the mesh by setting up your own mesh node. All of the information on their site and all of the code in their software is free to use share and edit.