We can look at the recent popularity of some widely used platforms like Zoom and ask ourselves some questions as to why we still use them when we know a lot of terrible things about them. Agaric prefers to use a free/libre video chat software called BigBlueButton for many reasons, the first one being the licensing, but there are many reasons.

Zoom has had some major technology failures, which the corporation is not liable to disclose. At one point, a vulnerability was discovered in the desktop Zoom client for MacOS that allowed hackers to start your webcam remotely and launch you into a meeting without your permission. The company posted a note saying that they fixed the issue. Unfortunately, the Zoom source code is proprietary and we are not even allowed to look at it. There is no way for the community to see how the code works or to verify that the fix was comprehensive.

The Zoom Corporation stated early on that the software was encrypted end-to-end (E2EE) from your device to the recipient's device. This was untrue at the time, but the company states that it has been corrected for users on their client app. While it is no longer true that E2EE is unsupported, it does require that you use the proprietary Zoom client for E2EE to work. Without E2EE, any data that is retrieved on its way from your computer to a server can be accessed! The only real security is knowing the operators of your server. This is why Agaric uses trusted sources like MayFirst.org for most of our projects and we have a relationship with our BigBlueButton host. The Intercept also revealed that Zoom users that dial in on their phone are NOT encrypted at all

BigBlueButton does not have a client app and works in your browser, so there is no E2EE. The idea for E2EE is that with it,  you "do not have to trust the server operator and you can rely on E2EE" because the model implies that every client has keys that are protecting the transferred data. However: you MUST still use a proprietary client in order to get the benefits of E2EE support, so once again you MUST trust Zoom as you have no permission to examine the app to determine that the keys are not being shared with Zoom.

Of course there is always the fact that hackers work day and night to corrupt E2EE and a Corporation is not obligated to tell you the customer every time there has been a security breach, and this information is usually buried in the terms of service they post - sometimes with a note saying the terms are subject to change and updates. A Corporation is not obligated to tell you, the customer when there has been a security breach" unless any personal information is exposed. There are now mandatory timely disclosure requirements for all states: https://www.ncsl.org/research/telecommunications-and-information-technology/security-breach-notification-laws.aspx ...Can Zoom really be trusted? As with some laws, the fine that is applied is low and affordable and subject to the interpretation of the courts and the status of knowledge your lawyer is privvy to - meaning most Corporations normally have a battery of lawyers to interpret the law and drag the case out until you are... broke.

In the case of BigBlueButton encryption, E2EE would only make sense if there are separate clients using an API to connect to the BBB server so a user does not have to trust the BBB server operator. If the user trusts the server operator, then there would be no need for E2EE." Lesson learned: It is always best practice to know and trust your server hosts as they are the ones that have the keys to your kingdom. 

Some technology analysts consider Zoom software to be malware. Within companies that use Zoom, employers are even able to monitor whether or not you are focusing on the computer screen during meetings which seems excessively intrusive. Speaking of intrusive, the Zoom Corporation also shares your data with FaceBook, even if you do not have a FB account - that could be a whole blog in itself, but just being aware of some of the vulnerabilities is a good thing to pass on. Some of the bad stuff remains even if you uninstall the Zoom app from your device! Even though a class action suit was filed over privacy issues, the company stock still continued to rise.

Those are many reasons why we do not support Zoom. But there are also many reasons why we prefer BBB over Zoom. Besides, BBB has many great features that Zoom lacks:

1. Easily see who is speaking when their name appears above the presentation.

2. Chat messages will remain if you lose your connection or reload and rejoin the room.

3. Video is HD quality and you can easily focus on a persons webcam image.

4. Collaborative document writing on a shared Etherpad.

5. Easily share the presenter/admin role with others in the room.

6. Write closed captions in many languages, as well as change the language of the interface.

7. An interactive whiteboard for collaborative art with friends!

One huge advantage of free software, like BBB, is that you can usually find their issue queue where you can engage with the actual developers to report bugs and request feature enhancements. Here is a link to the BigBlueButton issue queue.

So, why do people keep using a platform like Zoom, even though there are many features in BigBlueButton that are much better? 

There is very little publicity for free software and not many know it exists and that there are alternative solutions. You can find some great suggestions of software and switch to it by using this site called switching.software. The marketing budget for Zoom is large and leads you to believe it has everything you will need. Sadly their budget grows larger everyday with the money people pay for subscriptions to the platform. As a result, many people go with it as it is already used by their friends and colleagues, even though there are reports of irresponsible behavior by the Zoom Corporation. This is why the New York school system does not use Zoom and many organizations are following suit. The company gives people a false sense of security as it is widely used and very popular.

Of course, there are reasons to avoid other proprietary chat platforms too...

Agaric offers BigBlueButton for events and meetings. Check out our fun BBB website at CommunityBridge and test drive the video chat yourself!

If this discussion interests you, please share your thoughts with us in the comments.

Looking to learn more about problems with Zoom? There are a lot of articles about Zoom scandals.

• If you need to leak a zoom video without being identified, you need to be very careful!
• Zoom censorship of Palestine seminars sparks fight over academic freedom
• Zoom encryption is not suited for secrets
• Zoom is unclear about whether or not the app is actually end-to-end encrypted, which it isn't.
• And, yes - the wikipedia page has a good summary of some issues
• Stallman provides resources on what is bad about Zoom also

Looking to learn more about protecting your privacy online? These links have some helpful information and videos for tech-savvy people and organic folks alike!

• The Hated One
• Rob Braxman Tech
• Safe Services

2021 could be the year we all begin to STOP supporting the Corporations that oppress us. 

Special thanks to Keegan Rankin for edits!

Websites powered by Find It connect people to all that their communities have to offer.  Developed under the guidance of the Cambridge Kids' Council, Find It ensures that people from all backgrounds can easily find activities, services, and resources.

Find It's design was informed by hundreds of hours of research, including over 1,250 resident surveys and over 120 interviews with service providers. Six years of research and development and continuous improvements has resulted in the Find It approach and platform, ready to be implemented in cities and communities around the world.

What is Free Software?

A program is free software if the program's users have the four essential freedoms:

1. The freedom to run the program as you wish, for any purpose (freedom 0).
2. The freedom to study how the program works, and change it, so it does your computing as you wish (freedom 1).
3. The freedom to redistribute copies so you can help others (freedom 2).
4. The freedom to distribute copies of your modified versions to others (freedom 3).

Access to the source code is a precondition for this. To determine whether the source code is free, see what license it carries and then check the GNU project's list of licenses.

We do a few tasks that we do not have free software for. So, we use non-free programs for them. Sometimes we use specific non-free software that a client insists on including in the web site. By using non-free programs, we sacrifice some of our freedom; some activists would refuse to do that. We do not like the compromise, so we help develop free replacements for those programs. Whenever possible, we use software that reflects our values.

Operating Systems

GNU/Linux

We have chosen to use GNU/Linux as our default system for our local development. When we take on a new student, we install a GNU/Linux distribution. We always give the option of installing a different distribution, or if a student wishes to do so, they may. These are the favored GNU/Linux distributions in use by our cooperative team *members:

• Debian
• Ubuntu
• Pop! OS (an Ubuntu derivative)

These are not the best versions of GNU/Linux in regards to being completely free; you should consult the list of free distributions on the Free Software Foundation website.

* Currently one team member is using the proprietary but BSD-based Mac OS X, which is compliant with the Unix 03 / POSIX standard which GNU/Linux distributions also meet.

Browsers

Firefox

As developers, we have to test client sites in all browsers, but for working and building sites, we use Mozilla Firefox. Although the source code of Firefox is free software, they include some icons, trademarks, and logos in the download that make it non-free. You can easily remove these as has been done with IceCat, the GNU version of the Firefox browser. It has great performance, developer tools, and community. The plus side of having a community around the software we use is having access to a large pool of people with experience and guidance as we learn and build together.

Tor Browser

As citizens, we are not fond of being tracked, so we use a free anonymizing web browser that does not allow tracking. It is called Tor.

Micky uses and loves the Brave browser on Android and her GNU/Linux laptop. It blocks ADs right out of the box!  https://brave.com/

Without looking deeply at Qwant, it looks pretty decent - https://www.qwant.com/

We came across Qwant via https://iridiumbrowser.de/ - a more secure chromium release, but probably has some things we may not know about or want...

File Storage and Calendar

NextCloud

NextCloud is a collection of utilities suited for the De-Googlization process. It provides most of the popular tools in Google Drive. Agaric uses a hosted version of NextCloud on MayFirst.org servers that is inclusive of:

• document and file storage
• shared document editing
• multi-device synchronization
• image galleries
• calendar
• contacts

See a comparison of the features NextCloud offers vs. the proprietary options like GoogleDocs/Drive.

Finance, Accounting, and Bookkeeping

GNUcash

Accounting software that we use for our bookkeeping.

You can see a review of GNUcash vs. Quickbooks and decide if it works for you. We have found a few bookkeeping cooperatives that do accounting with GnuCash.

Real-time Chat

As a team, we rely on different tools to communicate with each other and with clients about daily activities and long term project goals. We have a distributed team at locations around the world and must maintain contact especially when pair-programming or during a migration which calls for all-hands-on-deck, as well as sharing some long informational text notes and documents that include administrative information.

Freenode

IRC - Internet Relay Chat - Realtime Text Chat: Yes, we still use IRC, and you can find us on irc.freenode.net server in the #devs-r-us channel

Our preferences here are as varied as our team members: some use irssi via a remote, always-on virtual server, many use desktop clients, such as HexChat or Konversation, and still, others prefer the web-based solution "The Lounge."

Email

MayFirst.org hosts Agaric.com email

Email Client: Thunderbird

An email client from Mozilla, which also makes Firefox, and is available for your phone. It also has an encryption plugin called EnigMail that works well and is not super tricky to get set up.

Hosted Email: RiseUp: Encrypted services run by anonymous volunteers and you must be invited to have a membership.

MayFirst offers three web-based email solutions.

1. Roundcube which has a friendly and simple web interface, making it the easier of the two programs to use.
2. SquirrelMail is an option that is Javascript-free!
3. Horde, on the other hand, offers more than just email - you can share calendars, to-dos and more with other members of your group.

Hosted Email

Protonmail: An email service that is hosted and encrypted.

Email Lists:

We use email list servers for mailing lists based on groups and topics. It allows group mailing to people that sign up for a specific list.

MayFirst Email Server

RiseUp Email Server

Social Media

Mastodon: Publish anything you want: links, pictures, text, video. All on a platform that is community-owned and ad-free.

Social.coop: A community similar to Twitter, the main difference is that it is owned by the members. For as little as $1 a month you can become an owner/member and take part in shaping the future of the platform. You can find and follow Agaric in social.coop, a coop-run corner of the fediverse, a cooperative and transparent approach to operating a social platform

Live Streaming

MayFirst Live Streaming: MayFirst membership includes live streaming.

Conference Calls and Online Meetings

Some Agaric team members are using Jitsi recognizing that it is a work in progress and there may be technical failures at times - such as we have also found using Google Hangouts - lag time, cut-offs, poor sound quality and issues with screen sharing. At times we have found that we need to use a proprietary solution that seems to work reliably as we continue to support development efforts and bug fixes with Jitsi. At the heart of Jitsi are Jitsi Videobridge and Jitsi Meet, which let you have conferences on the internet, while other projects in the community enable other features such as audio, dial-in, recording, and simulcasting.

You can self-host an instance of Jitsi or choose a hosted version. You can use http://meet.jit.si or an instance is also available for public use at https://meet.mayfirst.org We do encourage you to become a MayFirst member and have access to all of the free software tools they offer. The Jitsi project needs volunteers to use and test Jitsi so it can get better swiftly!

Currently, Agaric was using and paying for, the proprietary Zoom audio/video conference call service and software. Since October we have been using BigBlueButton a free software alternative. BBB has been working well for us and we have worked it into being our go-to solution for protecting our privacy.

BigBlueButton

BigBlueButton.org: Agaric uses BigBlueButton as our video-chat meeting platform. We also offer free chatroom hosting for those that cannot afford to pay for this service. 

Phone Calls and Text Messages

Signal: Agaric uses Signal to encrypt SMS text messages and phone calls. Encrypted phone and instant messaging found to be secure and recommended by Edward Snowden as the only truly encrypted messaging app that is not able to be decrypted by anyone. Note that security is an arms race and this could become false at any time.

Collaborative Note Taking

Etherpad: When hosting an online meeting we usually open a shared note pad so that everyone may contribute to getting the important bits logged. Etherpad text is synchronized as you type so that everyone viewing the page sees the same text. This allows you to collaborate on documents with large or small teams seamlessly! We use the hosted version, but you are welcome to host it yourself. We have tried a few online pads and settled on Etherpad as the most reliable.

Collaborative Ongoing Discussion

With some collaborators, particularly people involved with the Drutopia initiative, we use Mattermost rather than IRC. Mattermost can be more useful for ongoing discussions; it is similar to Slack and offers a threaded conversation. The community version is free software.

Notes and ToDo Lists

TomBoy A tiny app that lets you take notes while it conveniently makes hyperlinks out of titles and allows synchronization over SSH and more.

Password Management

KeePass A password management system that takes most of the worry, distraction and thinking out of storing and retrieving your login information for multiple projects and sites.

Text Document Editing, Spreadsheets, and Presentations

Libre Office: A suite of office tools similar to Microsoft Office, Documents, Spreadsheets, Slides. We use LibreOffice tools that come as core software in the distributions of GNU/Linux we are using. You may have heard of OpenOffice; it is now called LibreOffice. It consists of basic publishing and calculating software for doing office tasks. These are the ones we use most often:

1. LibreOffice Calc - Similar features and functions of a calculating software to make spreadsheets, such as MicroSoft Excel

2. LibreOffice Writer - Similar features and functions of a word processor such as MicroSoft Word

3. LibreOffice Impress - We use this tool to build slide decks and presentations using text/graphics and videos; it is similar to Microsoft PowerPoint in features.

Project Management and Issue Tracking

*GitLab: This tool is a web-based and self-hosted Git-repository manager with a wiki and issue-tracking features. We also use Gitlab for cooperative development on our projects.

*Although GitLab isn't fully free software, it does offer a self-hosted version that is. The Enterprise hosted version has extra features and is proprietary.

Redmine: A free program that you can run locally or on your own server for use as a project management and issue tracking tool. Before finding GitLab, we used a self-hosted instance of Redmine which is free software.

Decision Making and Voting

Loomio: A hosted service available at http://loomio.org

Loomio offers distributed decision-making system where you can make groups that can have discussions and make decisions without an in-person meeting. Decide yes or no, or that you need more information.

Note that Loomio also has built a great cooperative resource on at their other URL - http://loomio.coop

Customer Relationship Management

CiviCRM: Agaric is working with the developers at MyDropWizard to take a look at CiviCRM with Drupal 8.

CiviCRM is a free software to manage client relationships and memberships. We have not deployed it yet.

Resources and Free Software Directories

• Free Software Directory
• Awesome-list of self-hosted applications
• You can search for Free software alternatives. - be sure to search for Free/Open Source, not the free trial software, and LOOK at the license.

You can contribute to groups working towards solutions, there are many roles, and you do not have to be a developer. As an example, *IndieWeb and Jitsi are projects that we make time to support with development, testing, outreach, and feedback.

*With IndieWeb, you can take control of your articles and status messages can go to all services, not just one, allowing you to engage with everyone. Even replies and likes on other services can come back to your site, so they’re all in one place.

Framasoft: A large collection of free software tools where we use the calendar and polling software most often. We are experimenting with several other FramaSoft tools and may adopt them in the future.

If this has been a helpful read, please pass it on and let us know in the comments how it helped you. A follow-up post will list the tools we use for development purposes. Please be sure to mention any free software you have found and are using now.

I will leave you with an excellent TEDx talk where Richard Stallman explains Free Software:

Sign up to be notified when Agaric gives a migration training:

Join us for an interactive day devoted to Drupal in healthcare— a relaxed and friendly close to DrupalCon with learning, networking, and discussing. Whether you are in a pharmaceutical company, a state department of health, a non-profit hospital, a public health organization, or anyplace else in the broad healthcare space, there are unique needs in ensuring security, accessibility, compliance, and availability of important information and tools.

Healthcare is a crucial, dynamic space for web development today. Online communication and emerging technologies promise improved access and capabilities for patients and professionals. Crafting inspiring and useful digital experiences, however, must be built on a solid foundation of accessibility, security, and compliance. Come listen to industry leaders share their experience solving these challenges in healthcare.

Get tickets to go to DrupalCon and the Healthcare Summit!

What You Will Get by Attending

• Andy Waldrop will talk about scalable healthcare content in Drupal— with lessons from building WebMD!
• Georgiana Masgras and Jesse Meece will talk about using Drupal to makes it easy to maintain our sites.
• Jon Stewart will talk about how to leverage Drupal for web application development— the Open Web in healthcare!
• Two table talk sessions will give everybody a chance to dive deeply into key topics, including the application of large language models in healthcare and biggest takeways from DrupalCon and the summit. Bring your needs and we will embark on facilitated peer-to-peer problem solving with others who are affected and tech and healthcare industry experts.
• Our sponsors, Evolving Web and Acquia (by way of their partner Phase2), will have special presentations
• We will be closing the day with lightning talks. Invited topics include: Future of Drupal at your company (and how to improve it), multisite platforms, knowledge bases, provider directories, and the healthcare take on layout builder becoming experience builder versus paragraphs.

We will be all wrapped up by 4pm.

Attend DrupalCon and the Healthcare Summit!

Who Should Attend

Everybody interested in hearing and discussing how companies and the community are creating rich digital experiences in the healthcare space. All levels of colleagues in the pharma, medical, clinical, hospital, payers, caregivers, advocates, and healthcare professional space should go to DrupalCon and the Healthcare Summit!

COVID-19 Safer Space

The Oregon Convention Center controls all rooms per ASHREA 62.1 standards, using sensors to monitor occupancy based on CO₂ levels and bring outside air into the space accordingly. They report at the most levels at 800–1000 ppm during normal occupancy, and use MERV-13 filters for any filtration, which is appropriate for COVID-19.

Agaric will have high-quality N95 masks available to anyone who wants them and will have our own CO₂ monitors.

More about the Healthcare Industry Summit

The Healthcare Summit at the 2024 Portland, Oregon DrupalCon is organized by Jeanne Cost, George Matthes, and Pooja Jeeva. I am glad to be playing a part in coordinating this summit as well, especially given Agaric's involvement in and commitment to health and science communities.

Healthcare is a complex, regulation-filled industry that poses great challenges to building vibrant and compliant digital experiences for patients. Whether you are a pharma working towards the next product, a hospital treating someone with cancer, or an insurance company making sure a patient has access to information and proper care, there is much for you to gain from this summit. 

Hear from experts on areas of accessibility, compliant patient experiences, new regulations and standards that can affect your teams and customers, success stories, and much more.

Sponsored by: Acquia and Evolving Web

Sign up to go to the Healthcare Industry Summit at DrupalCon Portland, OR 2024 this coming May 6th through May 9th for the summit

We partnered with the City of Cambridge to redesign Find It Cambridge, an online opportunity locator serving city residents, in a unique way— designing and developing out in the open, and releasing the software under an open-source license so that other cities can spin up their own Find It platforms.

Cambridge, like many cities, has a wide array of programs and events happening to serve residents. However, it can be difficult for people to find and compare the many offerings out there. The city website has a calendar of events and list of departments. These are limited to government-run programs, though. There are myriad nonprofits and community groups that run programs and events that go unlisted. Social media platforms like Facebook have filled the gap in some areas, but promotion of these opportunities relies in many ways on people's social connections— leaving those most in need of services out of the loop. Find It Cambridge solves this problem by aggregating the many different opportunities happening into one website.

The upgrade and migration from Drupal 7 to Drupal 8 (ready for Drupal 9!) afforded Cambridge and Agaric the opportunity to make the directory of organizations, events, and programs better for families and their children and all involved, grounded in research and testing.

Intuitive, Structured Authoring Experience for Service Providers

The value of a directory are the listings within it and it's truly a community effort to assemble enough accurate and up-to-date resources in a single place, for it to be useful. For Cambridge, the service providers that work at government agencies and nonprofits are the lifeblood of the directory. Without them, there would be no Find It Cambridge.

The challenge then, is building a system that is easy enough for people (many already pressed for time) to take the time to enter their information into, while structuring the data to be easily searched and filtered on.

Through user research, we mapped the information architecture to the mental models that service providers hold for their events and programs.

Input Fields Grouped by Key Questions

Most service providers thought of their events in terms of what the event is about, how to contact the organizers, who it is for, when it is happening, where it is happening, how much it costs, and if there is any sort of registration required. So we organized fields into working tabs to match: About, Contact, For whom, When, Where, Cost, and Signup.

Autosave and Soft Required Fields

Even with fields grouped by tabs, the form can take some time to complete. That's why we introduced autosave and soft save features. When working on a form, the site automatically saves the current draft every few seconds. Service providers can also save a draft to return to later. Fields that are required for publishing, are optional for a draft.

Draft States to Save Work for Later

Service providers have many responsibilities to juggle. It's important that they can start creating an event or program, save it and return to it later before publishing it.

Drupal has powerful workflow states, which we've put to use to help service providers clearly know the status of their content.

A service provider can either save their content as a draft or publish it immediately. If saved as a draft, a banner appears on the page clearly indicating that the event or program is not yet published.

Authors can also create a draft alongside a published version. This allows new versions to be worked on, while maintaining the current page for site visitors.

Help text and character counts for guidance

There are particular ways to write and format content on events and programs to make the most of Find It's features. We provide help text along the way to clue providers in on the best ways to write their content. We also include a character count so providers know if they're staying within the recommended limits for certain text fields.

Bulk Select for Quick Data Entry

Certain fields have many options. In some cases the majority of them apply. For example, many educational events are for all ages up to 18. In that scenario, having a "Select All" option speeds up the data entry process. The Selectize JavaScript library adds elegant toggle and check all options to multivalue fields. We created the CheckboxesJS Drupal project so that other Drupal sites can easily incorporate these features on fields of their choosing.

Conditional Fields to Show Only What is Relevant

Some fields on Event and Programs only need to show under certain conditions. For example, if an event doesn't require registration, then there's no need to worry service providers with a registration link field. Using conditional logic keeps forms simple and streamlined.

Multiple Dates

There was a lot of discussion on whether to support repeating rules or instead allow multiple dates. We decided on multiple dates as experience has shown that even repeating events oftentimes have exceptions (and because the events we import from Cambridge Public Libraries are a list of arbitrary dates rather than a recurring rule, and somehow no one in computer science has created a library to produce a best-effort recurring rule from a list of dates).

Easy but Powerful Search for Residents

Find It search is powered by Apache Solr, a popular open-source enterprise search platform. We use its numerous features to make the search results as relevant as possible for site visitors.  It's an ongoing process of tweaks; here are some of the things we've done so far.

Weighted Fields for Relevance

On content with lots of data like the events and programs of Find It, certain fields carry more importance than others. The title of an event, for example, is one of the most important. The transporation notes, on the other hand, carries less significance in search queries. When someone types the keyword "music lesson", an event with music lesson in the title or summary shows up before a program for English lessons.

Synonym Matching

When someone searches "childcare" but a program uses "child care", the search engine should know these are equivalent. The same is true for "STEM" and "science education."

Find It supports synonyms. The site manager can define synonyms so that when site visitors search for a certain term, results with matching synonyms show up as well.

Key Information in Search Results

We used the results of our user research to show the critical information people need to pin point the right opportunities: title, neighborhood, and a short summary.

Filters for Sophisticated Queries

Filters help users narrow a search query down to specific criteria. In our testing, we found that age and neighborhood were most important, especially for low-income caregivers. For those of us that rely on public transportation, events and programs need to be nearby. We placed these filters accordingly towards the top of the page.

Naming conventions in Cambridge are unique, which is true for other cities too. Residents might not know the official name of their neighborhood or live at the border between two. We've included a labeled, clickable map to help users choose the right neighborhood. We built this so that other Find It platforms can upload their own SVG map to show their neighborhood.

Informative Opportunity Pages

Find It comes out of the box with four different types of opportunities: Events, Places, Organizations and Programs.

Organization

The organization serves as the foundation for opportunities posted on a Find It page. Every event and program posted to Find It, belongs to an organization. This helps an organization's page serve as a mini-website. When an event or program is published, it automatically shows up on its organization page.

Organizations can also have "child" organizations, which is helpful for larger groups that might have distinct sub-committees or departments that have sub-departments.

Event

An event is an opportunity with a clear start and end date. When an event is published it shows up on the Homepage, Events page, Search page and on the organization's page.

Visitors can sort opportunities by start date to find upcoming events.

Program

A program is similar to an event. In fact, most fields are shared between the two. A program though, implies more longevity and commitment than an event. Rather than requiring a specific date or dates, a program can simply be "ongoing." There is the option to include specific dates though.

Place

In the first version of Find It Cambridge, a new opportunity surfaced that didn't quite fit into the event, program, or organization categories. Parks, neighborhood pools, and other destinations were a good fit for Find It's library of opportunities. They have open hours, but many of the event fields were irrelevant. The same went for Programs. In fact, sometimes these places have events or programs happening at them.

These are community-minded destinations people can go to. In other words, places.

Bring Find It to Your City!

Find It is helping Cambridge residents connect with activities and services to improve their lives. We would love to help do the same for other cities, counties, and other communities. The platform is open-source and flexible so that communities can customize it to their needs.

Whether you are city IT staff, a developer that works with cities, or are a resident that could use a Find It in your community, we'd love to talk.

"Wirth Co-op won't reopen", a tiny heading on the front page of North News announced. The story ran inside on page four.

The grocery store, right in my neighborhood, set out to build community wealth and provide healthy food in north Minneapolis. The loss of its promise is crushing.

Wirth Grocery's failure to act as a cooperative—with its own existence at stake—is even more upsetting. Cooperatives are one of the few spaces where people have a chance to take collective control of something that matters to them.

One epic month of blog post tutorials about migrating into Drupal 8 and 9 by Mauricio who also is the driving force behind Agaric's migration trainings and Agaric's upgrade services.

One man.  One month.  Many migrations.

Would you give $20 for that feature/fix/documentation you've always wanted in a module/theme/distribution?

Yes? Great! Would you pledge $20 for the chance to give that $20? Wonderful! Please make your tax deductible contribution, which will only be charged if we raise enough to do this, in the next three days!

If not, why not? Serious question— the goal is not to extort money from you, the goal is to extort either money or information (or both). What would make you unwilling to pledge money toward something you think Drupal needs? Please let me know by contact form or in the comments.

So you have goals to evaluate for worthiness and practicality, we intend that Snowball (the platform to get community initiatives rolling) will:

• Allow asynchronous aggregation of wants and needs. We don't want a mad rush of fundraising campaigns (like this one); instead we want to see what people are offering to put time or resources toward, and then help an actionable plan take shape.
• Provide a platform for the instigators, the organizers, the advocates, who are crucial to making things happen in any community.
• Give people with more money than time or talent (for a particular task, so that means most of us) a direct way to contribute.
• Get your most-wanted improvement to Drupal made... provided we find enough other likeminded people who act on their convictions.
• Make the world a better place.

Is this a fool's errand? If so, i have the credentials. but please say why Snowball as a concept doesn't hold together. If not a fool's errand, i think we can continue to broaden those involved in this "initiative to make more initiatives possible" until it is possible itself. Join in and even pledge a few bucks for the effort on CrowdTilt while every cent (if the funding level is reached at all) will go to build it.

Sign up if you want to know when Mauricio and Agaric give a migration training:

Today we will learn how to migrate content from Microsoft Excel and LibreOffice Calc files into Drupal using the Migrate Spreadsheet module. We will give instructions on getting the module and its dependencies. Then, we will present how to configure the module for spreadsheets with or without a header row. There are two example migrations: images and paragraphs. Let’s get started.

Getting the code

You can get the full code example at https://github.com/dinarcon/ud_migrations.

The module to enable, as in yesterday's post in which we imported Google Sheets, is UD Google Sheets, Microsoft Excel, and LibreOffice Calc source migration whose machine name is ud_migrations_sheets_sources. It comes with four migrations: udm_google_sheets_source_node.yml, udm_libreoffice_calc_source_paragraph.yml, udm_microsoft_excel_source_image.yml, and udm_backup_csv_source_node.yml. The image migration uses a Microsoft Excel file as source. The paragraph migration uses a LibreOffice Calc file as source. The CSV migration is a backup in case the Google Sheet is not available. To execute the last one you would need the Migrate Source CSV module.

You can get the Migrate Spreadsheets module using composer: composer require drupal/migrate_spreadsheet:^1.0. This module depends on the PHPOffice/PhpSpreadsheet library and many PHP extensions including ext-zip. Check this page for a full list of dependencies. If any required extension is missing the installation will fail. If your Drupal site is not composer-based, you will not be able to use Migrate Spreadsheet, unless you jump through a lot of hoops.

Understanding the example set up

This migration will reuse the same configuration from the introduction to paragraph migrations example. Refer to that article for details on the configuration. The destinations will be the same content type, paragraph type, and fields. The source will be changed in today's example, as we use it to explain Microsoft Excel and LibreOffice Calc migrations. The end result will again be nodes containing an image and a paragraph with information about someone’s favorite book. The major difference is that we are going to read from different sources.

Note: You can literally swap migration sources without changing any other part of the migration.  This is a powerful feature of ETL frameworks like Drupal’s Migrate API. Although possible, the example includes slight changes to demonstrate various plugin configuration options. Also, some machine names had to be changed to avoid conflicts with other examples in the demo repository.

Understanding the source document and plugin configuration

In any migration project, understanding the source is very important. For Microsoft Excel and LibreOffice Calc migrations, the primary thing to consider is whether or not the file contains a row of headers. Also, a workbook (file) might contain several worksheets (tabs). You can only migrate from one worksheet at a time. The example documents have two worksheets: UD Example Sheet and Do not peek in here. We are going to be working with the first one.

The spreadsheet source plugin exposes seven configuration options. The values to use might change depending on the presence of a header row, but all of them apply for both types of document. Here is a summary of the available configurations:

• file is required. It stores the path to the document to process. You can use a relative path from the Drupal root, an absolute path, or stream wrappers.
• worksheet is required. It contains the name of the one worksheet to process.
• header_row is optional. This number indicates which row containing the headers. Contrary to CSV migrations, the row number is not zero-based. So, set this value to 1 if headers are on the first row, 2 if they are on the second, and so on.
• origin is optional and defaults to A2. It indicates which non-header cell contains the first value you want to import. It assumes a grid layout and you only need to indicate the position of the top-left cell value.
• columns is optional. It is the list of columns you want to make available for the migration. In case of files with a header row, use those header values in this list. Otherwise, use the default title for columns: A, B, C, etc. If this setting is missing, the plugin will return all columns. This is not ideal, especially for very large files containing more columns than needed for the migration.
• row_index_column is optional. This is a special column that contains the row number for each record. This can be used as unique identifier for the records in case your dataset does not provide a suitable value. Exposing this special column in the migration is up to you. If so, you can come up with any name as long as it does not conflict with header row names set in the columns configuration. Important: this is an autogenerated column, not any of the columns that come with your dataset.
• keys is optional and, if not set, it defaults to the value of row_index_column. It contains an array of column names that uniquely identify each record. For files with a header row, you can use the values set in the columns configuration. Otherwise, use default column titles like A, B, C, etc. In both cases, you can use the row_index_column column if it was set. Each value in the array will contain database storage details for the column.

Note that nowhere in the plugin configuration you specify the file type. The same setup applies for both Microsoft Excel and LibreOffice Calc files. The library will take care of detecting and validating the proper type.

Migrating spreadsheet files with a header row

This example is for the paragraph migration and uses a LibreOffice Calc file. The following snippets shows the UD Example Sheet worksheet and the configuration of the source plugin:

book_id, book_title, Book author
B10, The definitive guide to Drupal 7, Benjamin Melançon et al.
B20, Understanding Drupal Views, Carlos Dinarte
B30, Understanding Drupal Migrations, Mauricio Dinarte

source:
  plugin: spreadsheet
  file: modules/custom/ud_migrations/ud_migrations_sheets_sources/sources/udm_book_paragraph.ods
  worksheet: 'UD Example Sheet'
  header_row: 1
  origin: A2
  columns:
    - book_id
    - book_title
    - 'Book author'
  row_index_column: 'Document Row Index'
  keys:
    book_id:
      type: string

The name of the plugin is spreadsheet. Then you use the file configuration to indicate the path to the file. In this case, it is relative to the Drupal root. The UD Example Sheet is set as the worksheet to process. Because the first row of the file contains the header rows, then header_row is set to 1 and origin to A2.

Then specify which columns to make available to the migration. In this case, we listed all of them so this setting could have been left unassigned. It is better to get into the habit of being explicit about what you import. If the file were to change and more columns were added, you would not have to update the file to prevent unneeded data to be fetched. The row_index_column is not actually used in the migration, but it is set to show all the configuration options in the example. The values will be 1, 2, 3, etc.  Finally, the keys is set the column that serves as unique identifiers for the records.

The rest of the migration is almost identical to the CSV example. Small changes were made to prevent machine name conflicts with other examples in the demo repository. For reference, the following snippet shows the process and destination sections for the LibreOffice Calc paragraph migration.

process:
  field_ud_book_paragraph_title: book_title
  field_ud_book_paragraph_author: 'Book author'
destination:
  plugin: 'entity_reference_revisions:paragraph'
  default_bundle: ud_book_paragraph

Migrating spreadsheet files without a header row

Now let’s consider an example of a spreadsheet file that does not have a header row. This example is for the image migration and uses a Microsoft Excel file. The following snippets shows the UD Example Sheet worksheet and the configuration of the source plugin:

P01, https://agaric.coop/sites/default/files/pictures/picture-15-1421176712.jpg
P02, https://agaric.coop/sites/default/files/pictures/picture-3-1421176784.jpg
P03, https://agaric.coop/sites/default/files/pictures/picture-2-1421176752.jpg

source:
  plugin: spreadsheet
  file: modules/custom/ud_migrations/ud_migrations_sheets_sources/sources/udm_book_paragraph.ods
  worksheet: 'UD Example Sheet'
  header_row: 1
  origin: A2
  columns:
    - book_id
    - book_title
    - 'Book author'
  row_index_column: 'Document Row Index'
  keys:
    book_id:
      type: string

The plugin, file, amd worksheet configurations follow the same pattern as the paragraph migration. The difference for files with no header row is reflected in the other parameters. header_row is set to null to indicate the lack of headers and origin is to A1. Because there are no column names to use, you have to use the ones provided by the spreadsheet. In this case, we want to use the first two columns: A and B. Contrary to CSV migrations, the spreadsheet plugin does not allow you to define aliases for unnamed columns. That means that you would have to use A, B in the process section to refer to these columns.

row_index_column is set to null because it will not be used. And finally, in the keys section, we use the A column as the primary key. This might seem like an odd choice. Why use that value if you could use the row_index_column as the unique identifier for each row? If this were an isolated migration, that would be a valid option. But this migration is referenced from the node migration explained in the previous example. The lookup is made based on the values stored in the A column. If we used the index of the row as the unique identifier, we would have to update the other migration or the lookup would fail. In many cases, that is not feasible nor desirable.

Except for the name of the columns, the rest of the migration is almost identical to the CSV example. Small changes were made to prevent machine name conflicts with other examples in the demo repository. For reference, the following snippet shows part of the process and destination section for the Microsoft Excel image migration.

process:
  psf_destination_filename:
    plugin: callback
    callable: basename
    source: B # This is the photo URL column.
destination:
  plugin: 'entity:file'

Refer to this entry to know how to run migrations that depend on others. In this case, you can execute them all by running: drush migrate:import --tag='UD Sheets Source'. And that is how you can use Microsoft Excel and LibreOffice Calc files as the source of your migrations. This example is very interesting because each of the migration uses a different source type. The node migration explained in the previous post uses a Google Sheet. This is a great example of how powerful and flexible the Migrate API is.

What did you learn in today’s blog post? Have you migrated from Microsoft Excel and LibreOffice Calc files before? If so, what challenges have you found? Did you know the source plugin configuration is not dependent on the file type? Share your answers in the comments. Also, I would be grateful if you shared this blog post with others.

Next: Defining Drupal migrations as configuration entities with the Migrate Plus module

This blog post series, cross-posted at UnderstandDrupal.com as well as here on Agaric.coop, is made possible thanks to these generous sponsors. Contact Understand Drupal if your organization would like to support this documentation project, whether it is the migration series or other topics.

When continuing development of a web site, big changes occur every so often. One such change that may occur, frequently as a result of another change, is a bulk update of URLs. When this is necessary, you can greatly improve the response time experienced by your users—as they are redirected from the old path to the new path—by using a handy directive offered in Apache's mod_rewrite called RewriteMap.

At Agaric we regularly turn to Drupal for it's power and flexibility, so one might question why we didn't leverage Drupal's support for handling redirects. When we see an opportunity for our software/system to respond "as early as it can", it is worth investigating how that is handled. Apache handles redirects itself, making it entirely unnecessary to hand-off to PHP, never mind Drupal bootstrapping and retrieving up a redirect record in a database, just to tell a browser (or a search engine) to look somewhere else.

There were two conditions that existed making use of RewriteMap a great candidate. For one, there will be no changes to the list of redirects once they are set: these are for historical purposes only (the old URLs are no longer exposed anywhere else on the site). Also, because we could make the full set of hundreds of redirects via a single RewriteRule—thanks to the substitution capability afforded by RewriteMap—this solution offered a fitting and concise solution.

So, what did we do, and how did we do it?

We started with an existing set of URLs that followed the pattern: http://example.com/user-info/ID[/tab-name]. Subsequently we implemented a module on the site that produced aliases for our user page URLs. The new patten to the page was then (given exceptions for multiple J Smiths, etc via the suffix): http://example.com/user-info/firstname-lastname[-suffix#][/tab-name]. The mapping of ID to firstname-lastname[-suffix#] was readily available within Drupal, so we used an update_hook to write out the existing mappings to a file (in the Drupal public files folder, since we know that's writable by Drupal) . This file (which I called 'staffmapping.txt') is what we used for a simple text-based rewrite map. Sample output of the update hook looked like this:

# User ID to Name mapping:
1 admin-admin
2 john-smith
3 john-smith-2
4 jane-smith

The format of this file is pretty straight-forward: comments can be started on any line with a #, and the mapping lines themselves are composed of {lookupValue}{whitespace}{replacementValue}.

To actually consume this mapping somewhere in our rules, we must let Apache know about the mapping file itself. This is done with a RewriteMap directive, which can be placed in the Server config or else inside a VirtualHost directive. The format of the RewriteMap looks like this: RewriteMap MapName MapType:MapSource. In our case, the file is a simple text file mapping, so the MapType is 'txt'. The resulting string added to our VirtualHost section is then: RewriteMap staffremap txt:/path/to/staffmapping.txt This directive makes this rewrite mapping file available under the name "staffremap" in our RewriteRules. There are other MapTypes, including ones that uses random selection for the replacement values from a text file, using a hash map rather than a text file, using an internal function, or even using an external program or script to generate replacement values.

Now it's time to actually change incoming URLs using this mapping file, providing the 301 redirect we need. The rewrite rule we used, looks like this:

RewriteRule ^user-detail/([0-9]+)(.*) /user-detail/${staffremap:$1}$2 [R=301,L]

The initial argument to the rewrite rule identifies what incoming URLs this rule applies to. This is the string: "^user-detail/([0-9]+)(.*)". This particular rule looks for URLs starting with (signified by the special character ^) the string "user-detail/", then followed by one or more numbers: ([0-9]+), and finally, anything else that might appear at the end of the string: "(.*)". There's a particular feature of regex being used here as well: each of search terms in parenthesis are captured (or tagged) by the regex processor which then provides some references that can be used in the replacement string portion. These are available with $<captured position>—so, the first value captured by parenthesis is available in "$1"—this would be the user ID, and the second in "$2"—which for this expression would be anything else appearing after the user ID.

Following the whitespace is our new target URL expression: "/user-detail/${staffremap:$1}$2". We're keeping the beginning of the URL the same, and then following the expression syntax "${rewritemap:lookupvalue}", which in our case is: "${staffremap:$1}" we find the new user-name URL. This section could be read as: take the value from the rewrite map called "staffremap", where the lookup value is $1 (the first tagged expression in the search: the numeric value) and return the substitution value from that map in place of this expression. So, if we were attempting to visit the old URL /user-detail/1/about, the staffremap provides the value "admin-admin" from our table. The final portion of the replacement URL (which is just $2) copies everything else that was passed on the URL through to the redirected URL. So, for example, /user-detail/1/about includes the /about portion of the URL in the ultimate redirect URL: /user-detail/admin-admin/about

The final section of the sample RewriteRule is for applying additional flags. In this case, we are specifying the response status of 301, and the L indicates to mod_rewrite that this is the last rule it should process.
That's basically it! We've gone from an old URL pattern, to a new one with a redirect mapping file, and only two directives. For an added performance perk, especially if your list of lookup and replacement values is rather lengthy, you can easily change your text table file (type txt) with a HashMap (type dbm) that Apache's mod_rewrite also understands using a quick command and directive adjustment. Following our example, we'll first run:

$> httxt2dbm -i staffrepam.txt -o staffremap.map

Now that we have a hashmap file, we can adjust our RewriteMap directive accordingly, changing the type to map, and of course updating the file name, which becomes:

RewriteMap staffremap dbm:/path/to/staffremap.map

RewriteMap substitutions provide a straight-forward, and high-performance method for pretty extensive enhancement of RewriteRules. If you are not familiar with RewriteRules generally, at some point you should consider reviewing the Apache documentation on mod_rewrite—it's worthwhile knowledge to have.

Agaric proposes the creation of a new kind of workplace, essentially a Drupal commune, but really more like an open source free software idea & brainstorming commune, kind of along the same lines as an artist's or writer's colony.

Imagine a network of state-of-the-art green living spaces spread across the planet in the most beautiful locations that mother nature has to offer. Places filled with the best and the brightest drupalistas living and playing and growing together, and together developing Drupal and the various technologies that help it work. As an open source community, Drupal already possesses many of the cooperative qualities that would make this sort of commune possible. Just think of any Drupal meetup you've attended in the past. Some show up to demo their latest work, others to ask for help, others to give help... What if we had spaces where this sort of thing was constantly happening?

The Drupal commune would be open to whomever wanted to come join in for a period of time and help advance the cause. Every member of the community would provide a special skillset to the team, be it coding, theming, graphic design or documentation. Some people could be permanent residents at such places and others could come and go based on work available and projects being worked on. Of course at first, we think it would have to start as more of a couch-surfing type thing. People who have the space can offer it up, recruit and house a few drupalistas for the duration of a project, like contributed work, or payed client work. From there, teams could assemble all around the world, do the necessary work, and do it better than they could alone, all while being able to enjoy and explore the world. Then disband, these communes or colonies don't have to be permanent, although that is the eventual goal.

• The individual contributor gets a combination code sprint and couch surf experience.
• Drupal gets fantastic functionality or process improvements made in ideal productive circumstances.
• And everyone gets satisfaction of making something awesome and worthwhile, in community, while getting to see and explore planet Earth and do fun stuff.

This idea is nothing new and has recently been advocated by the brilliant physicist and surfer Garret Lisi. In his own words he says these places... "would essentially be large houses in beautiful locations where theorists could live and work." Citing his own experience living in Maui and the mountains of Tahoe and Colorado, Lisi says that for theoretical research it is good to have opportunities for hiking and things to do outside in attractive environments. Describing the idea more formally, Lisi says:

The physical requirements for conducting scholarly research have changed dramatically with the rise of the internet. It is now viable for researchers with laptop computers to work autonomously -- with access to current articles and communication channels on par with the resources available at large universities. These new circumstances motivate the creation of a new kind of research enterprise: a Science Hostel. By providing places to live and work with other researchers, in beautiful locations, a Science Hostel could increase creative productivity and overall quality of life for scholars in the internet age.

We believe the Drupal world currently developing before our eyes is especially well poised to embrace such an idea. We are a growing community of able, intelligent, free thinking, and like minded people. We all think Drupal has a lot to offer people, and we've all been making a living developing with Drupal, and we also all our own personalities and hobbies outside of work. We see a lot of people in this world showing up to "just a day job" or a "9 to 5". We don't like seeing Drupal applied to this model. We believe that satisfying work comes from truly embracing what you do and doing it to it's fullest. So it's not just your day job, but your lifestyle, something that is an essential part of your being. When focused on certain subjects with such passion, originality and creativity can flourish, sparks happen. Our proposition is that we really pursue this idea of combining our work lives with our personal ones, evolving things into a lifestyle of the future.

These communities could also include people who are interested in sustainable living, and/or off-the-grid living and promote these ideas as well. So not just Drupalers, but there could be specialists doing other things like growing food with aquaponics, creating mesh wireless networks, or harnessing solar and wind energy. Proving that people can create and shape their universe when they come together and cooperate.

The way we see it, this thing would be community managed and totally open, the way Drupal has always been, except now we'd really be starting to shape the material world using Drupal, creating a free culture. There are a lot of folks who are tirelessly spending nights and weekends coding masterpieces like CCK and Views and giving us the power to feed and clothe ourselves. We love seeing this happen, we love thinking about it. A person driven to create is a beautiful thing. We want to see more community forming around this stuff. Why should development be done in cooped up cubicle type spaces? Why have we let so many convince us that our time is best spent, and we are most useful and productive in offices and sitting in traffic? Would things evolve more organically, and would Drupal advance exponentially with the future of humanity if places like this existed? They don't even need to be permanent, we could organize and create them for certain collaborative projects and disband after the work is done and has been contributed back to the community.

There is a lot more that can be written about this, and all the details still need to be worked out. But we're confident that the community could organize enough to essentially allow for these "corporate retreats on acid" or "software development communes" to happen. Drupalistas could apply for internship type positions and get voted in, most likely through the Drupal Association or similar organizations. Teams of independent contractors could come together, share projects, split up the work, and live the good life. We could become the renaissance men and women of the future and break free of all the old, tired, hierarchical, institutionalized ways of being productive that have been shoved down our throats in the past. We can create community, developing our ideas, doing the work we love to do, and seeing and shaping the world into something better during our time here.

Sure it's not for everybody, but we have a feeling their might be other Drupal nomads out there...