Blog

Only Section 3 is particularly Drupal/Drush specific, but still might give you a hint about running remote commands that use ssh from behind their shiny CLI.

TLDR: just give me what works, NOW (please?)

Since you asked so politely: if you are already familiar with hosting options, bash shell configuration, and Drush, this somewhat lengthy article can be summed up quite quickly, really. For those that might not be pro's in any one of these, you can take this as the quick intro, and keep reading to learn why these situations exist, as well as the detailed fix.

1. If you are on a hosting provider, they normally will have a UI to select the desired PHP version. If you are doing your own hosting, install Ondrej Sury's PHP PPA/Apt repo, and install PHP-FPM (not mod_php). Adjust your SetHandler (Apache), or fastcgi_pass (NGINX) in each site configuration to the socket for the pool of the version of PHP you want.
2. Configure a local ~/bin folder and place a link to the php version you want (e.g. ln -s /usr/bin/php8.1 ~/bin/php) there. Be certain ~/bin is also in your PATH.
3. When using Drush aliases to run commands on remote instances, this still won't work until you add an env-vars: key to your site.yml file and place a PATH in there.

Jump to the "Part" below if you only need help with one or more of these.

Part 1: All things hosting (well, at least Apache or NGINX, plus FPM)

Let's start where most people do: setting up the hosting itself. In most cases, a managed hosting provider will have some way to select the appropriate version of PHP for your individual site. If that's the case - pick your target, and you should be good to go (move on to Part 2, you lucky dog)! If you do your own hosting, there will be some additional steps to take, which I'll give an overview of, and some additional resources to get you over this first hurdle.

If you are running "A-PAtCHy" web server (possible name change coming?) you will not be able to use mod_php for your PHP duties, as this method does not allow the web server to directly serve content of different virtual hosts using different versions of PHP. Instead, I recommend using PHP's "FastCGI Process Manager" service - aka FPM. This is a stand-alone service that Apache and NGINX will speak to using new-age technology from 1996 called FastCGI. It's still technically CGI, only, like, Fast (seriously, it works really well). Your web server hands off to this service with it's related FastCGI/proxy module.

The process is quite similar for both web servers, and an article over at Linode covers the basics of this method for each, but wait! Finish reading at least this paragraph before you jump over there for both a caveat emptor and then some Debian specific derivations, if you need those (the article is Ubuntu-specific). In the article, they utilize the excellent PHP resources offered by Ondřej Surý. From this PPA/Debian APT resource, you can run concurrent installations of any of the following PHP versions (listed as of this writing): 5.6, 7.0, 7.1, 7.2, 7.3, 7.4, 8.0, 8.1, and 8.2. Do keep in mind, however, that versions prior to 8.0 are [per php.net] now past their supported lifetime and no longer actively developed (see also this FAQ for more details). Debian-specific instructions for setting up this repository in apt (as opposed to Ubuntu PPA support) are also found within Ondřej's instructions. The remainder of the Linode process should still apply with that one change. OK, run along and get the PHP basics talking to your web server. If you'll be running Drupal (why wouldn't you?) then you'll want to ensure you have the version-specific php modules that it requires (this is for Drupal 9+, but links to earlier revisions also).  I'll wait...

Assuming you have now progressed as far as having both versions of PHP you need installed, and followed the article from Linode above or whatever your favorite substitute source was, you likely noticed the special sauce that connects the web server to a particular PHP FPM service. In Apache, we have: SetHandler "proxy:unix:/var/run/php/php8.0-fpm.sock|fcgi://localhost", and in NGINX flavor, it's: fastcgi_pass unix:/var/run/php/php8.0-fpm.sock; These directives point to the Unix socket as defined for in the default PHP "pool" for the particular version. For reference, these are defined in /etc/php/{version}/fpm/pool.d/www.conf and therein will look like: listen = /var/run/php/php8.1-fpm.sock. So - all that's necessary to select your PHP version for your web server is to point to whichever socket location for the version of PHP you want.

The Linode article does not go into handling multiple host names, and I won't go too deep here either as I've already navigated headlong into a bit of a scope-creep iceberg. The quick-and-dirty: for Apache, add another site configuration (as in, add another your_site.conf in /etc/apache/sites-available, and link to it from sites-enabled) repeating the entire VirtualHost and everything inside it, however, use a different listen port, or the same port, and add the ServerName directive to specify the unique DNS name. Likewise, with NGINX, except you here you repeat the full server block in another configuration, and changing the listen and/or server_name bits. Oh yeah - you'll probably be changing the folder location of the Drupal installation in there too, that should definitely help reduce some confusion.

Phew - we should have the web server out of the way now!

Part 2: BASH-ing the `php` command

Next up: PHP on your command line. Here, I'm referring to what you get when you type php on your command line once logged in (via ssh) as the user that manages the web site. In this section, I'm assuming that per best-practices, there are different users for each site. This method does not help much if you have but one user...though I guess it can help if they're both using the same non-default version of php, in contrast to, say, other users on the server.

Which PHP is which?

When a server has multiple versions of PHP, only one of them at a given time will ever live at the path /usr/bin/php. Ordinarily, this is what you get when you type just php at the command line. This also is what you'll get whenever you run a file with a shebang of #!/usr/bin/env php, meaning if you run drush (or wp-cli, for our Wordpress friends), you'll get whatever PHP is found there as well. You can run which php if you'd like to see where php is found.

At this point, definitely check php --version. If you are getting the version you want, you're are done with this article! Well, maybe not - you just want to switch to the account where you do require a different version of PHP than this gives you.

So, php --version gives one version, but there should be multiple PHP executables on the system now, right? (You should have installed multiple, or else have multiple versions available at this point). So where are those? These can be directly executed by running them using their versions in their name. For example php8.1 or php7.4. So, what is happening here that we just get "one proper php"? Well, a couple things.

Environmental conditions...

What, the smog? No...well, yes, that's a problem, but some good things happened in 2022. In this instance, our first issue comes from an environment variable. In particular: the venerable PATH, which contains a list of locations that the shell will use to look for a given executable as you enter a command name (or, again, as specified by a shebang). The PATH variable is a colon-delimited list, typically looking about like this: /usr/local/bin:/usr/bin:/bin:. The shell simply looks for the first occurrence of your command as it peeks in each directory in sequence (left-to-right). Is there a /usr/local/bin/php? That's what you'll get. If not, how about /usr/bin/php? And so on, until it finds one, or else you've mistyped pph and end up with command not found instead. You can see your path with echo $PATH (or try which agaric. I'm guessing you won't have an agaric program, this will tell you where it looked for it when it fail to find one).

What's the 'alternative': links all the way down

The second part of this equation is what is going on with the /usr/bin/php that was found. This alleged "The PHP" is actually a soft link to the current system-level default version of PHP that's installed. You can see how this situation is resolved with a command such as readlink -f /usr/bin/php. This command basically says "read the symbolic link (recursively, due to the -f, try it again without the -f!) and show what it's [ultimately] pointing to". This link (and those it links to) come from an "alternatives" system used by Debian-like systems that connects such things as the canonical name of an executable to a specific installed version. You can learn more about how how this is set up (for PHP, anyway) from...you guessed it: Ondřej's FAQ.

So...the fix then?

Now, where we have multiple versions of PHP installed, it's generally impractical to change all the shebang lines to something else, but that is technically one way to do things. We're also assuming you want to use multiple versions simultaneously here - so updating via the alternatives system isn't a great option either - if you can even do that. There is a simple method to make this work, even as an unprivileged user: make your own link called php and make sure the shell can find it in the PATH.

What we will do, is create our own ~/bin folder, and make our link there. Then, we just make sure that ~/bin is in our path, and comes before other locations that have a php file. There's no shortage of places for customizing the PATH (and bash, generally), and quite frankly, since I'm not positive what the canonical location is, I'll happily follow the Debian manual which says ~/.bashrc. The particular file you'll want to use can be influenced by the type of shell you request (a login vs non-login, and interactive vs non-interactive). In the manual, part of their example looks like this:

# set PATH so it includes user's private bin if it exists
if [ -d ~/bin ] ; then
  PATH="~/bin${PATH+:$PATH}"
fi
export PATH

Curious what that odd-looking ${PATH+:$PATH} syntax is about? Most people just refer to PATH as $PATH when they want it, like this PATH=~/bin:$PATH, right? Well, yes, and that will probably work just fine, but that weird reference does have some smarts. These are something called parameter expansions. Note that in proper bash parlance, the thing we've been calling a variable this whole time is referred to as a parameter. Go figure...that certainly didn't help me find this reference documentation. If you are interested in shell programming (which I clearly think everyone is, or should be) these can be very helpful to know. You'll bump into these when performing various checks and substitutions on varia-er, parameters. Check out the bash documentation to figure this one out.

OK, this looks good! Let's go ahead and add that to the ~/.bashrc file in your ssh user's home folder. Now, you either need to source the updated file (with . ~/.bashrc), or just reconnect. Sourcing (abbreviated with that previous ., otherwise spelled out as source ~/.bashrc) essentially executes the referenced file in a way that can it can modify the current shell's context. If you were to just run ~/.bashrc without sourcing it, it happily sets up an environment, but all that gets wiped out when the script ends.

Now, let's get moving with this plan again. Make a bin folder in our ssh user's home folder: mkdir ~/bin. Finally, link the php version you want in there. Here, I'll do php8.1: ln -s /usr/bin/php8.1 ~/bin/php. Voila! Now when you run php --version, you'll get PHP 8.1!

Another diversion, really?

For the impatient: reconnect and skip to the paragraph. For the curious (or when you have time to come back): it turns out our shell has a bit of memory we never expected it to. If you've typed php sometime earlier in your session, and bash last found that it was the one in /usr/bin/php that came up first, it's now remembered that so it doesn't have to find it again. While you can just re-login - again! - you might also wan to take the reins of your system and try typing hash -d php (see help hash to learn what that does - the help command covers shell built-in functionality, like hash). At last, php really works the way we wanted! No more odd little shell corners hiding dusty references on us.

OMG, that works?

Finally...despite my droning on, we're making progress! At this point, when you call upon your drush status, it should actually run without errors (some things under php7.x just don't work now - as expected) and show that it's using the correct php version.

Part 3: Drush does speak SSH, but not YOUR SSH.

The Drush boffins graced us with doodads called site aliases that allow us to readily send commands to various sites we control. If you don't know about those, you'll have to read up on them first. The prolific Moshe Weitzman (along with some 310 contributors, and counting) didn't give us these because they were getting bored after 20 years of Drupal; They're pretty essential to working with Drupal effectively.

Assuming you have a grasp of aliases under your belt, let's try a Drush command from our local environment to the remote system: drush @test status. OK - something is wrong here. My Drush just said I'm still using php7.4 and not my beloved 8.1...again! Does Drush have a memory too? No, it just doesn't care how you want your ssh sessions to work; those are left to you and Drush does it's own thing. The ssh connection that Drush opens has it's own context set-up, unlike the one we have when we've ssh'd in interactively. Thankfully, there's a quick fix for this - they key Drush needs to know is how to set the PATH up so it also gets our targeted PHP version. Let's add our modified PATH to the site aliases configuration, so Drush also knows what the cool kids are doing.

live:
  host: example.com
  paths:
    drush-script: /home/example-live/vendor/bin/drush
  env-vars:
    PATH: /home/example-live/bin:/usr/bin:/bin
  root: /home/example-live/web
  uri: 'https://example.com/'
  user: example-live

Note the specified PATH of /home/example-live/bin:/bin:/usr/bin places the bin directory in home at the beginning again. At last, when we run drush @test status, it's telling us the PHP it uses is the one that we use. We can share, Drush. We made it!

Thanks for stopping by!

That wraps it up for this one. Hopefully you now feel a little more confident you have some tools helping you master your environment. The version of PHP you need is now at your command. Now you can go get busy with your composer updates and other TODO's. By the way, if all this configuration headache is just not for you - check out our hosted Drutopia platform. We run an instance of this distribution, and provide all these niceties as part of our hosting setup, so reach out if interested. Either way, thanks for coming by!

Martin Owens is a committed programmer that lives for making great code that helps people. He fully supports the Free Software philosophy and likes to publish online as much of his own code as possible. Martin hopes to work for a company where his talents can be used to make great products for users, where he can be stretched and encouraged to learn new things as well as being able to commit to the wider open source landscape where possible.

test

"Word as Image" by Ji Lee is licensed under CC BY-NC 4.0.

So often an organization's online donation system is divorced from their website. For staff, it means one more tool to use and manage. For supporters, it can result in a donation experience that is tedious or clunky. There are free software tools like CiviCRM and CommitChange which do a good job of feeling integrated into your site, despite being a separate system. However, for smaller groups these tools can be overkill. We built the Give module to provide a lightweight donation solution built right into your Drupal site.

2. We do recommend moving directly to Drupal 9 (which was released on June 3rd of 2020), however:

3. Moving to Drupal 8 or to Drupal 9 is much the same. Drupal 8 starts what i call the "modern Drupal" era. Whereas for going from Drupal 5 to 6 or 6 to 7 or 7 to 8 broke backward compatibility and might as well be a full rebuild (so we would often recommend hopping a version, say, stay on Drupal 6 and wait for Drupal 8 to be ready) going from Drupal 8 to 9 is closer to going from Drupal 8.8 to 8.9— an in-place upgrade from 8.9 to 9.0. Going from 9 to 10 will work the same, and that's the plan and promise for Drupal 8 on out.

4. All that said, if anything significant needs fixing on your current Drupal 7 site, or you are looking to make any improvements, you'll want to do that on Drupal 8+ or Drupal 8/9 as we phrased it back when Drupal 9 was still a pretty recent release, but now we can just say Drupal 9— or, as i call it to emphasize the decreased importance of major version numbers, modern Drupal.

Agaric is always happy to discuss more! Mostly what i'm saying here is the useful things to talk about are the specific goals for the sites—when you want to accomplish what—because the official support cycles are a distraction in the current context of Drupal. So make sure your current site is maintained, but take your time to get clear on your objectives, and contact Agaric or the Drupal professionals of your choice when you think it might make sense to upgrade your site into the era of modern Drupal.

This blog post series, cross-posted at UnderstandDrupal.com as well as here on Agaric.coop, is made possible thanks to these generous sponsors. Contact Understand Drupal if your organization would like to support this documentation project, whether the migration series or other topics.

Today we will learn how to migrate dates into Drupal. Depending on your field type and configuration, there are various possible combinations. You can store a single date or a date range. You can store only the date component or also include the time. You might have timezones to take into account. Importing the node creation date requires a slightly different configuration. In addition to the examples, a list of things to consider when migrating dates is also presented.

Getting the code

You can get the full code example at https://github.com/dinarcon/ud_migrations The module to enable is UD date whose machine name is ud_migrations_date. The migration to execute is udm_date. Notice that this migration writes to a content type called UD Date and to three fields: field_ud_date, field_ud_date_range, and field_ud_datetime. This content type and fields will be created when the module is installed. They will also be removed when the module is uninstalled. The module itself depends on the following modules provided by Drupal core: datetime, datetime_range, and migrate.

Note: Configuration placed in a module’s config/install directory will be copied to Drupal’s active configuration. And if those files have a dependencies/enforced/module key, the configuration will be removed when the listed modules are uninstalled. That is how the content type and fields are automatically created.

PHP date format characters

To migrate dates, you need to be familiar with the format characters of the date PHP function. Basically, you need to find a pattern that matches the date format you need to migrate to and from. For example, January 1, 2019 is described by the F j, Y pattern.

As mentioned in the previous post, you need to pay close attention to how you create the pattern. Upper and lowercase letters represent different things like Y and y for the year with four-digits versus two-digits, respectively. Some date components have subtle variations like d and j for the day with or without leading zeros respectively. Also, take into account white spaces and date component separators. If you need to include a literal letter like T it has to be escaped with \T. If the pattern is wrong, an error will be raised, and the migration will fail.

Date format conversions

For date conversions, you use the format_date plugin. You specify a from_format based on your source and a to_format based on what Drupal expects. In both cases, you will use the PHP date function's format characters to assemble the required patterns. Optionally, you can define the from_timezone and to_timezone configurations if conversions are needed. Just like any other migration, you need to understand your source format. The following code snippet shows the source and destination sections:

source:
  plugin: embedded_data
  data_rows:
    - unique_id: 1
      node_title: 'Date example 1'
      node_creation_date: 'January 1, 2019 19:15:30'
      src_date: '2019/12/1'
      src_date_end: '2019/12/31'
      src_datetime: '2019/12/24 19:15:30'
destination:
  plugin: 'entity:node'
  default_bundle: ud_date

Node creation time migration

The node creation time is migrated using the created entity property. The source column that contains the data is node_creation_date. An example value is January 1, 2019 19:15:30. Drupal expects a UNIX timestamp like 1546370130. The following snippet shows how to do the transformation:

created:
  plugin: format_date
  source: node_creation_date
  from_format: 'F j, Y H:i:s'
  to_format: 'U'
  from_timezone: 'UTC'
  to_timezone: 'UTC'

Following the documentation, F j, Y H:i:s is the from_format and U is the to_format. In the example, it is assumed that the source is provided in UTC. UNIX timestamps are expressed in UTC as well. Therefore, the from_timezone and to_timezone are both set to that value. Even though they are the same, it is important to specify both configurations keys. Otherwise, the from timezone might be picked from your server’s configuration. Refer to the article on user migrations for more details on how to migrate when UNIX timestamps are expected.

Date only migration

The Date module provided by core offers two storage options. You can store the date only, or you can choose to store the date and time. First, let’s consider a date only field. The source column that contains the data is src_date. An example value is '2019/12/1'. Drupal expects date only fields to store data in Y-m-d format like '2019-12-01'. No timezones are involved in migrating this field. The following snippet shows how to do the transformation.

field_ud_date/value:
  plugin: format_date
  source: src_date
  from_format: 'Y/m/j'
  to_format: 'Y-m-d'

Date range migration

The Date Range module provided by Drupal core allows you to have a start and an end date in a single field. The src_date and src_date_end source columns contain the start and end date, respectively. This migration is very similar to date only fields. The difference is that you need to import an extra subfield to store the end date. The following snippet shows how to do the transformation:

field_ud_date_range/value: '@field_ud_date/value'
field_ud_date_range/end_value:
  plugin: format_date
  source: src_date_end
  from_format: 'Y/m/j'
  to_format: 'Y-m-d'

The value subfield stores the start date. The source column used in the example is the same used for the field_ud_date field. Drupal uses the same format internally for date only and date range fields. Considering these two things, it is possible to reuse the field_ud_date mapping to set the start date of the field_ud_date_range field. To do it, you type the name of the previously mapped field in quotes (') and precede it with an at sign (@). Details on this syntax can be found in the blog post about the migrate process pipeline. One important detail is that when field_ud_date was mapped, the value subfield was specified: field_ud_date/value. Because of this, when reusing that mapping, you must also specify the subfield: '@field_ud_date/value'. The end_value subfield stores the end date. The mapping is similar to field_ud_date except that the source column is src_date_end.

Note: The Date Range module does not come enabled by default. To be able to use it in the example, it is set as a dependency of demo migration module.

Datetime migration

A date and time field stores its value in Y-m-d\TH:i:s format. Note it does not include a timezone. Instead, UTC is assumed by default. In the example, the source column that contains the data is src_datetime. An example value is 2019/12/24 19:15:30. Let’s assume that all dates are provided with a timezone value of America/Managua. The following snippet shows how to do the transformation:

field_ud_datetime/value:
  plugin: format_date
  source: src_datetime
  from_format: 'Y/m/j H:i:s'
  to_format: 'Y-m-d\TH:i:s'
  from_timezone: 'America/Managua'
  to_timezone: 'UTC'

If you need the timezone to be dynamic, things get a bit harder. The from_timezone and to_timezone settings expect a literal value. It is not possible to read a source column to set these configurations. An alternative is that your source column includes timezone information like 2019/12/24 19:15:30 -07:00. In that case, you would need to tweak the from_format to include the timezone component and leave out the from_timezone configuration.

Things to consider

Date migrations can be tricky because they can be affected by things outside of the Migrate API. Here is a non-exhaustive list of things to consider:

• For date and time fields, the transformation might be affected by your server’s timezone if you do not manually set the from_timezone configuration.
• People might see the date and time according to the preferences in their user profile. That is, two users might see a different value for the same migrated field if their preferred timezones are not the same.
• For date only fields, the user might see a time depending on the format used to display them. A list of available formats can be found at /admin/config/regional/date-time.
• A field can always be configured to be presented in a specific timezone. This would override the site’s timezone and the user’s preferred timezone.

What did you learn in today’s blog post? Did you know that entity properties and date fields expect different destination formats? Did you know how to do timezone conversions? What challenges have you found when migrating dates and times? Please share your answers in the comments. Also, I would be grateful if you shared this blog post with others.

Next: Migrating addresses into Drupal

This blog post series, cross-posted at UnderstandDrupal.com as well as here on Agaric.coop, is made possible thanks to these generous sponsors. Contact Understand Drupal if your organization would like to support this documentation project, whether it is the migration series or other topics.

Find It makes it easier for caretakers to find opportunities for their children online so that they can stay focused on taking care of themselves and their children.

Join us on Agaric's Meet.coop BigBlueButton videochat & screenshare for presentation and discussion, February 2, Sunday, at 15:00 UTC (10am Eastern).

https://meet.agaric.coop/rooms/a8m-x61-skh-ift/join

How can a group of thousands of people talk about and decide anything? How's this 'community' concept supposed to work at scale, even in theory?

Any Free/Libre Open Source Software project will have elements of do-ocracy (rule of those who do the work), but not all decisions should devolve to implementors. A better ideal is that decisions should be made by the people who are most affected.

Particularly when a decision strongly impacts more than those who carry it out, we need better ways of making decisions that give everyone their say. This starts by letting people by heard by everyone else. Fortunately, we can scale conversations and decisions in a fair and truly democratic way.

I attended BarCamp9 this year and had a more than wonderful time. The sessions were off the chart, everything from a Public Speaking 101 - Overview & Workshop, to a lecture on Immortality, a GNU/Linux sysadmin introduction, and a lively discussion on the state of mental health in tech, mixed in with Dating for Nerds.

First, I love the idea of the un-conference where nothing is scheduled in advance and everyone is encouraged to participate on some level. Most cons and camps I have been to are a "smash and grab" of info, BarCamp9 and the un-conference style bust that wide open in a non-intimidating way for people. The attendees I spoke to found themselves going to sessions they would never have thought to sign up for.  The non-schedule atmosphere invites people to explore in a non-threatening way. This un-conference was one of the best I have been to because the organization was smooth and the volunteers made the event really personal. The sessions ranged from highly technical discussions and presentations to down to earth heartfelt meetings. One session in particular was intriguing in that it drew a capacity crowd for the room. The gender balance was pretty much 50/50 and everyone was engaged in the conversation... the topic: "Things you wish you knew earlier". 

I tossed my sticky note up on the board and proceeded to do a presentation/discussion: Cooperatives and Collectives—Think Outside the Boss. A moderate sized group of people filled the room and I started the slides. When discussing the difference between a freelancer, an employee and a worker-owner, Gonzalos Chomon, a cooperative member of evovelo.com took part and gave great details of the different laws that affect European cooperatives. In Spain it seems like coops are given the benefits of the law as corporations are in the US. Twenty percent of a cooperatives earnings go into a fund to support new and growing cooperatives—this is mandatory.  Sometimes the audience makes the presentation come alive! I have posted my slides.

My colleague Fernandes Paredo Garcia, of Dilygent, based in Peru, translated my presentation to Spanish and presented it at an event in Guatemala—some of the attendees started the formation of a new cooperative that night!

I would recommend that you go to the next barCamp  in your area, or any un-conference near you - you might be suprised by what you find, and by what you may contribute!

Mighty thanks to all the sponsors that made this event possible http://www.barcampboston.org 

Here is the schedule for Saturday and then for Sunday, eclectic or what? There should be a lot more unconferences!  Agaric is working on PowerToConnect.org to facilitate conferences and events of all types—more on that later.

Note of interest: I heard in passing that the name barCamp came from a play on words in response to fooCamp, a programmers play on the term foo-bar, which is yet another play on the acronym F.U.B.A.R.

Agaric builds tools for medical and scientific communities to advance their work, enhance collaboration, and improve outcomes.