Drupal 9: Sanitising Data With Drush

06 Mar 2022

Drush (the Drupal Shell) is one of the most convenient and functional assistants of all developers. Those who have already worked with it won't let us lie.
Today, our Drupal development team continues to uncover all the power of this command-line utility and UNIX scripting interface for Drupal.
Read on if you're interested in learning how Drupal 9 and Drush work together, what benefits it has, and how easy it is to sanitize data with Drush. Well, let's start!

What Is Drush?

Drush is described by its creator as follows:
"Drush is a command line shell and Unix scripting interface for Drupal. Drush core ships with lots of useful commands and generators. Similarly, it runs update.php, executes SQL queries, runs content migrations, and misc utilities like cron or cache rebuild. Drush can be extended by 3rd party command files."
– Moshe Weitzman

Let's add that Drupal Drush speeds up all the above commands at least twice. That is, what Drupal developers once spent half a day on, today it will take half the time. It replaces a hundred clicks and page refreshes in the UI by using just one or two commands in the terminal.

How to Install Drush?

Step 1: Your site must be built with Composer and Drush listed as a dependency to install Drush.
Step 2: If Drush is not listed as a dependency, you need to fix it quickly. Run composer requires 'drush/drush' to add it.
Step 3: Next, call Drush via 'vendor/bin/drush'.
Step 4: We strongly recommend you install Drush Launcher as well. This program allows you to call Drush from anywhere.
Step 5: Congratulations! You have installed Drush, and now you can quickly sanitize databases or whatever you want.

Why do you need Drush?

You need to install Drush on the Drupal website because it checks for bugs and fixes them quickly, maintains and updates your website regularly, interacts directly with APIs, modules, themes, database, Drupal core, and speeds up web development. Are there enough reasons to install Drupal Shell immediately? Oh no, we feel like you still have doubts, so here are five more Drush benefits for you:

  1. It makes it easier to update and clean up the database
  2. It can generate fake content
  3. It updates the search index of the site
  4. It allows you to edit the administrator password
  5. It helps to make migration easier

and many more cool things.

Sanitizing Data With Drush on Drupal 9

Drush dramatically simplifies the work and management of Drupal sites. To do this, you need to know only a few basic commands. Let's look at an example of what commands can be and how Drupal Drush works.
We have the following task:
We migrated a database from the prod environment to dev or local. So, we must sanitize it to prevent data leaks from the database. That includes passwords, email addresses, contact info, gender, company name, and other private data.

How to sanitize databases with Drupal 9 and Drush?

Sanitization is a mandatory step in the implementation of this task. You are responsible for maintaining the security of users' data, and sanitization will help you with this.
To clear all data using Drupal 9 and Drush, you need to run the following command:

  • drush sql:disinfect

This command is available out of the box, so it's easy to use.
As soon as you call this command, you will see a list of data that matches this criterion and which will be deleted.
Drush will look for all the data in different places throughout the site, including modules. And if you use some module related to data collection (for example, comment module), then Drush will offer to remove data from it. Here's what it will look like:

drush sql:sanitize
The following operations will be performed:

 * Remove comment display names and emails.
 * Truncate sessions table.
 * Sanitize text fields associated with users.
 * Sanitize user passwords.
 * Sanitize user emails.

 Do you want to sanitize the current database? (yes/no) [yes]:

Nothing needs to be done to remove all information from custom fields in Drupal 9. It's built into the command. To do the same in Drupal 7, you need to write your own custom commands.

[success] Comment display names and emails removed.
[success] Sessions table truncated.
[success] user__field_user_name table sanitized.
[success] User passwords sanitized.
[success] User emails sanitized.

The above output shows that all emails, passwords, comments, and sessions have been deleted successfully. After this manipulation, your database will also change. The data that you deleted will disappear from it. Only the username will still be available, and for a good reason. It cannot be deleted because it allows you to log into the test environment and verify that all personal data has been deleted.
NB If you run it in prod, you`ll sanitize the database. You will not be able to recover deleted data. This is a one-way ticket.
There are additional flags in the official Drush command documentation. These few flags allow you to specify what data will be deleted and what will not.

Available Options:

allowlist-fields
You can mark fields that you do not want to be cleared with this flag.

drush sql:sanitize --allowlist-fields=field_biography,field_phone_number

sanitize-email
You can mark those emails that you do not want to clean up with this flag. To not delete emails, set this option to "no".

drush sql:sanitize --sanitize-email=noreply+%uid@example.com

sanitize-password
You can mark those passwords that you do not want to delete with this flag. To not delete passwords, set this option to "no".

drush sql:sanitize --sanitize-password=no

How to extend Drush sanitize?

In addition to the three options described above, there is one more - the Drush sanitize extension. This option helps to remove data from additional fields or tables that are not included in the list of the sanitization process. You need to add a drush.services.yml file to your codebase to get this to work.

services:
  mymodule.sanitize.command:
    class: \Drupal\mymodule\Commands\MyModuleSanitizeCommand
    tags:
      - { name: drush.command }

A real-life example of the MyModuleSanitizeCommand

As the Asian proverb says: "It is better to see once than hear a thousand times."
So you have mymodule_custom_table, which contains users' data. Your task is to sanitize the database using Drupal Shell.
First, you need to inject a database handler into the command. Add the arguments parameter to the drush.services.yml file.

services:
  mymodule.sanitize.command:
    class: \Drupal\mymodule\Commands\MyModuleSanitizeCommand
    arguments: ['@database']
    tags:
      - { name: drush.command }

Next, add a constructor in MyModuleSanitizeCommand class. After this manipulation, your class will look like this:

namespace Drupal\mymodule\Commands;

use Consolidation\AnnotatedCommand\CommandData;
use Drush\Commands\DrushCommands;
use Drush\Drupal\Commands\sql\SanitizePluginInterface;
use Symfony\Component\Console\Input\InputInterface;
use Drupal\Core\Database\Connection;

/**
 * Drush sql-sanitize plugin for sanitising some random data.
 *
 */
class MyModuleSanitizeCommand extends DrushCommands implements SanitizePluginInterface {

  /**
   * The database connection.
   *
   * @var \Drupal\Core\Database\Connection
   */
  protected $database;

  /**
   * MyModuleSanitizeCommand constructor.
   *
   * @param \Drupal\Core\Database\Connection $database
   *   The database.
   */
  public function __construct(Connection $database) {
    parent::__construct();
    $this->database = $database;
  }

}

This modifies the sanitize() method to remove data from your table partially.

public function sanitize($result, CommandData $command_data) {
  $this->database->truncate('mymodule_custom_table')->execute();
}

This option works, but the table remains empty. To remove all personal data, update the information. A faster and easier way to do this is to update any private information fields you wish to remove. For example, take 'mail' fields.

public function sanitize($result, CommandData $command_data) {
  $this->database->update('mymodule_custom_table')
    ->fields(['mail' => 'user+1@localhost.localdomain'])
    ->execute();
}

Or here's another option for you, especially if you have a lot of user data. Call a single update command and run database string replace functions for all the data.
To get a report on the success or failure of sanitizing, you must request the following commands:

$this->logger()->success(dt('MyModule data sanitised.'));

 

$this->logger()->error(dt('An error occurred in MyModule sanitisation step.'));

Sanitize your data with Drupal 9 Drush!

Sanitization is unthinkable without adding the drush sql:sanitize command. Use this method if you need to protect sensitive users' data.
Have a Drupal website and additional questions or ideas? Go to our Drupal services page.

Comments

An
Anonymous