10 minute read Published: 17 Aug, 2021 Author: Matt Parker
Drupal Planet , Migrations , Drupal

Easy commit credits with migrations, part 2: Can we get an RTBC?

This is the second in a series of blog posts on writing migrations for contrib modules:

In part 1, we set up a simple core migration.
In this part, I’ll cover how to review and manually test patches.
In part 3, I demonstrate how to write automated migration tests.
In part 4, I discuss migrating simple configuration.
In part 5, I explain how to declare a module’s migration status to the migrate wizard.
In part 6, I show how to migrate data from a custom database table.

Stay tuned for more in this series!

Background

While migrating off Drupal 7 Core is very easy, there are still many contrib modules without any migrations. Any sites built using a low-code approach likely use a lot of contrib modules, and are likely blocked from migrating because of contrib. But — as of this writing — Drupal 7 still makes up 60% of all Drupal sites, and time is running out to migrate them!

If we are to make Drupal the go-to technology for site builders, we need to remember that migrating contrib is part of the Site Builder experience too. If we make migrating easy, then fewer site builders will put off the upgrade or abandon Drupal. Plus, contributing to migrations gives us the opportunity to gain recognition in the Drupal community with contribution credits.

Problem / motivation

As a maintainer of several modules, it really helps me when other members of the community review and test patches, and if they work, mark them with the issue status RTBC (“Reviewed and Tested by the Community”)!

Proposed resolution

One of the easiest ways that you can contribute is by testing migration patches as thoroughly as you can, reviewing the code, and marking the issue as RTBC if everything checks out.

Aside: Engaging with the community

Any time you’re engaging with the Drupal community, and especially in the issue queue, it is worth keeping a few things in mind:

The Drupal code of conduct — “be considerate; be respectful; be collaborative; when we disagree, we consult others; when we are unsure, we ask for help; and; step down considerately”;
The Drupal.org issue etiquette — “dos” and “dont’s” for making issues flow smoother; and;
The strategic initiative that you are working towards — in this case, improving the migration experience for Site Builders.

Steps to complete

These steps assume that you have followed the Steps to complete in in part 1 of this blog series at least once.

First, we will need to find a contrib module with a migration patch in “Needs review” status. Read the issue with the patch in detail. Doing so will provide you with some insight into the intended scope of the patch, and also what to test. I suggest taking notes.

For your convenience, here is a link to a search for the word migration across all projects, filtered to issues in “Needs review” status (but since this blog series is about contrib migrations, you can ignore the results for the project “Drupal core”)¹.

For example, let’s suppose that I found issue #3024040 in this list.
Next, install the latest 7.x release of the contrib module you chose into the Drupal 7 site you set up in part 1 of this blog series (it is reasonable to assume that Site Builders looking to migrate are running the latest release of a contrib module).

If you haven’t worked much with Drupal 7, the recommended place to install contrib modules is inside sites/all/modules/.

Using our example issue #3024040; because it is a patch for the Tablefield module, I would install tablefield-7.x-3.6 into my D7 test site (because that was the latest recommended D7 version of the module at time-of-writing).
Then, git clone the contrib module you chose into the web/modules/ folder of the Drupal 9 site you set up in part 1 of this blog series.

You need to clone the branch specified in the “Version” field of the issue with the patch.

For your convenience, Drupal.org can generate a git clone command for you to copy-paste: go to the module’s project page, and look at the top for a “Version control” tab… click that tab, choose the “Branch to work from”, and click the “Show” button.

Using our example issue #3024040, the “Version” field in that issue’s metadata shows 8.x-2.x-dev, i.e. the 8.x-2.x Git branch. If we then click the module name (“Tablefield”) in the issue’s breadcrumb bar, then its Version control tab, then choose 8.x-2.x, and click “Show”, Drupal.org gives us the command git clone --branch '8.x-2.x' https://git.drupalcode.org/project/tablefield.git
Next, we want to apply the most-recent patch in the issue — but before we do that, we should create a branch to apply the patch on (creating a branch will make it easier to generate interdiff files if we need to submit our own patch).

If the issue is using an Issue fork instead of patches, then click “Show commands”, follow the instructions to “Add & fetch this issue fork’s repository”, then follow the instruction to “Check out this branch”, and skip ahead to the next step — the rest of the instructions in this step are for issues using patches.

If the issue is using patches, then I usually create a branch named after the issue ID and comment number that I got the patch from. In our example issue #3024040, the most recent patch at time-of-writing is in comment #8, so I would name the branch 3024040-8, i.e.: I would run git checkout -b 3024040-8

Now we can apply the patch.

Important note: if you’re following along with the example in issue #3024040, be aware that at some point in the future, the maintainers of the Tablefield module will likely accept and commit the patch — trying to apply the patch after it has been committed will fail.

If the patch applies successfully, commit the changes in the patch to the new branch. There’s no need to come up with a fancy commit message because we won’t be pushing it anywhere: I use branch name as the commit message (e.g.: git commit -m "3024040-8")
Now, we run through essentially the same process we used in part 1 of this blog series to test the migration. That is to say:
1. (Re-)install the D7 site using the Standard install profile.
2. (Re-)install the D9 site using the Standard install profile.
3. On the D7 site, install the Tablefield module, and set it up (i.e.: add a Tablefield to a node type). Then, create some Tablefield nodes as test migration content.
4. On the D9 site, install the core Migrate, Migrate Drupal, and Migrate Drupal UI modules; and also install the Tablefield module.
5. Make a database backup of the D9 site (so you can easily re-run the migration).
6. On the D9 site, run through the migration wizard at /upgrade.
7. When the upgrade is complete, check the migrated Tablefield nodes to ensure they contain the test migration content you set up on the D7 site.
If the test migration content you set up on the D7 site did not correctly migrate onto the D9 site, see the “What to do if something goes wrong” section below.
If the migration appeared to go correctly, then read the patch in more detail.

Future blog posts in this series should make the patch easier to understand, but even now, you can probably get a vague sense of what is being migrated, and how it is being done.

In particular, if you notice that the patch migrates some things that you did not test, it would be worth reverting to the database backup you made, and trying the migration again, so you can test those new things.

If you find coding style issues in a contrib patch, I would refrain from pointing them out — let the module maintainer do that if they feel strongly enough about it! Many Contrib maintainers have their own style, or don’t feel strongly about coding style: the coding standards used for Drupal Core are only suggestions for Drupal Contrib. Furthermore, some module maintainers will accept the patch but fix the style issues in the patch when they commit it (this is what I do for modules that I maintain).

Remember the strategic initiative we are working towards: we want to improve the experience for Site Builders — attaining coding standards perfection will delay the patch and prevent it from helping the Site Builders who need it!
Finally, if you are satisfied with the patch after reading it and testing the migration, then it is time to add a comment to the issue:
1. In the Issue metadata, set the “Status” field to Reviewed and tested by the community, and make sure the “Assigned” field is set to Unassigned.
2. Don’t forget to “Attribute this contribution”.
3. In the “Comment” field, clearly state that it worked for you, and describe what you tested.
4. Finally, click “Save” to post your comment.

Note that the patch to Tablefield in issue #3024040 is just used as an example — please do not leave comments in that issue unless you have something specific and constructive to add.

What to do if something goes wrong

If the test migration doesn’t go the way you expect, this may not necessarily indicate a problem with the patch! For example:

The D9 version of the module may operate or store data in a different way than the D7 version does;
The D9 version of the module may have fewer or different features from the D7 version;
The issue that you got the patch from intentionally leaves certain migrations out-of-scope; or;
Your expectations might be wrong (this happens to me a lot!).

So, before leaving a comment in the issue, take some time to:

Read the issue in depth (including earlier patch versions and interdiffs, if applicable), to understand what is, and is not, in scope;
Skim the D7 and the D9 versions of the module’s code, to understand the differences between the two versions of the module and how they work; and;
Read the patch, to understand what it is trying to migrate and to try to pinpoint the problem.

If you think that you can pinpoint the problem, then it’s worth posting your own comment on the issue. In your comment:

Describe the steps you took,
Describe how to create the content and/or configuration which did not migrate properly in D7,
Explain what you expected the migrated content and/or configuration to look like in D9,
Explain what you think the problem is.

Be aware that the patch author and/or module maintainer may be okay with things not working perfectly! Recall the Drupal code of conduct: be respectful (of the module maintainer’s decisions), and step down (i.e.: back off) considerately.

Next steps

As mentioned earlier, the best way to find Migration issues that need review is to search for them. As you may know, marking your own patches RTBC is discouraged, so you’ll probably run across patches that I’ve written floating out there in the issue queues!

If you’re reading through an issue, and you find it confusing to keep track of everything that changed, other people probably find it confusing too! You can help move the issue forward by simply updating the issue summary. But, be aware that, like coding standards, following Core’s issue sumamry template is just a suggestion in Contrib.

In the next blog post, we’ll talk about converting some of the testing that you’re doing manually right now.

I’ve proposed officially adopting the migrate issue tag for contrib migration issues, but this needs to be approved by the Drupal.org administrators, so don’t tag issues with it for now. I’ll update this blog post if this proposal is accepted. ↩︎

The article Easy commit credits with migrations, part 2: Can we get an RTBC? first appeared on the Consensus Enterprises blog.

We've disabled blog comments to prevent spam, but if you have questions or comments about this post, get in touch!

We help small teams do big things.