Content migration - Analysis and migration content plan

One of the star services that we provide in La Drupalera is the migration of sites into Drupal 8, regardless of the origin technology. It is a very demanded service and it requires a lot of care for implementing it in an efficient way.

The main lessons that we have learnt in the past years are the ones that we have collected in a series of articles that we begin today, where we will show how to migrate contents to Drupal 8/9.

The main approach is oriented to carry out migrations from a drupal 7, however, some other source systems are mentioned in order to perform them.

The contents of the series will be the following:

1. Migration of content – Analysis and migration plan.
2. Migration of content – Structure of a migration.
3. Migration of content – How the ETL from the migrate module works (Plugins).
4. Migration of content – Managing migrations (Migrate tools).

Let’s get started!

What is a migration in the context of Drupal?

In versions previous to Drupal 8, migrating content and upgrading versions were two concepts that came hand in hand. For this reason, the same database was used, and the tasks were focused on taking the code adapted (download the drupal core, contributed modules, themes, etc.). Once you got that, we used to update the system through a series of routines when executing the update.php.

Furthermore, it is noteworthy that for those versions, there was a contributed module available called migrate. With this module, we could achieve the transference of information or content from different sources of data, among others, Drupal.

Drupal 8 or higher, changes drastically the philosophy of migrating from previous versions to this one (to the actual one). This is due to the change that comes with the newer Drupal version (object-oriented, plugins, services, etc). That is why, in order to upgrade the version, it is necessary to perform two fundamental and differentiated steps.

1. Build the site (not for automatic migrations).
2. Develop and launch the migration of contents.

In these ones, the migrate module is included in the drupal core and it has the advantage in relation to the previous versions, in which configurations can be migrated and not only content.

Automatic migrations

In order to make automatic migrations from Drupal 6 or 7 to newer versions, there is a set of modules (Migrate, Migrate Drupal UI and Migrate drupal). With these modules, migrations can be configured and launched at the interface level, by simply configuring the database, source files folder (files) and searching the relationship in modules for the adaptation of the destination code.

The main limitations for this types of migrations are the following:

• There are modules which do not exist in the actual versions.
• You have to work with clean installations, so you cannot make a site building until you have already launched the migration.
• Migrations are only launched once, and no rollback can be achieved. If you have made changes after having launched the migration and you detect something incoherent, so that you need to relaunch the migration, you will lose those new changes.
• No incremental migrations can be done.
• It is very difficult that they fill all the requisites of your information system.

For all those limitations, it is not recommended the use of these types of migrations. To make a content migration, it needs to be developed.

Definition of a migration

What are we going to migrate to Drupal 8 or higher?

The first thing we need to have in mind is the answer to this question. It is very helpful to have the destination already built, or at least know which data model will be implemented.

• Users
• Content
• Media
• Taxonomy
• Roles
• Blocks
• URL Aliases
• Configurations

This information will help us understand how the source data will be processed, and we will have a starting point to be able to begin analyzing the migration. We will be able to generate entities relationships from source to destination, like an entity/relation diagram.

Source data analysis

In order to ensure the migration and define the development that we are going to perform in the migrations, we will need to have in mind a series of key considerations that will give us information about the development efforts and complexity:

• Which is the source of data (SGBD o data support) from the origin that we are going to migrate (MySQL, CSV, PostgreSQL, etc). There are contributed modules that extend these functionalities, such as Migrate Source CSV.
• Which is the base architecture (Drupal 6, Drupal 7, Wordpress, etc). There are contributed modules that extend the source, such as WordPress Migrate.
• Relationships among data, users, tables, content type entities, etc.
• Files, formats, blob, binary, cdn, etc.
• The generation of documentation about content and source entities that are going to be migrated. We will see an example about this point:

Source/destination relationships of a migration

The relation of entities from a source support to a destination can be diverse:

• Relationship (1,1): A source entity will be associated with a destination entity. For example, in the case of an article content type in origin, it will result in an article content type in destination. Article -> Article.
• Relationship (N, 1): N entities in origin will be associated with a destination entity. For example in the case that in source two content types (publications and news) exist and in destination only one content type of news that makes reference to the previous two: Publications and News -> News.
• Relationship (1, N): A source entity will be associated with various entities in destination. For example, if there are news that has a field to differentiate in origin (news, publications, etc) and there are different types of content for each one in destination. News -> News, Publications, etc.

Generating documentation

The generation of documentation gives us the information of the entities, fields or data mapping that we should execute inside a migration. Then below, an excel of a migration from Drupal 7 to Drupal 8 is shown, in which it is setted out the fields map of various source content types to a destination.

The previous image is a clear example of a N:1 relationship, so in this migration of two source content types (news and publications) they result in only one destination. Through the field field_type we will know if it is a news item or a publication.

The destination content types are shown in the left (there will always be one) and the source content types in the right side, that they will be one or more. The title of each content type will bring an excel page in which we will have more information about the represented entity. We will also have a selector of states (pending, blocked and finished) in order to know about the state in which the migration is.

It should be noted that in the excel page, depending on the quantity of people that are in the migration team, fields can be added for their assignments and so make collaborative work about this. In the same way, it can be extended to add any kind of relevant information about the migration.

Conclusions

To conclude, we point out that the main things that this post attend to are the following:

1. Define what are we going to migrate to Drupal.
2. Analyze the source of the data.
3. Define the destination in order to start guessing and understanding how the data will be processed.
4. Especify the relationships among the two migrations and their types.
5. Generate documentation in order to have everything prepared for it.

In the next post we will see how will the structure of a migration be to start moving data.