Batch processing large data sets

Session track
Code & Development
Experience level
50 min

Learn how to leverage Drupal's batch processing in a variety of ways. We will start with the basics of triggering a batch from a form. We will learn how to utilise the same code with hook update batches, via drush, via views, and even how it could be added on to a migration.


This session is on how to handle batch processing for large amounts of data. You will learn how to structure your code in such a way to build your long-running task for use across the range of batch processing that Drupal Core offers. For instance, you can run a batch process

  1. Via an admin form
  2. On node save
  3. View Views Bulk Operations
  4. Off the back of a migration
  5. Using a drush command
  6. In an update or deploy hook

Coding together

There will be lots of code examples and talking through the code so that you can understand the logic behind it in order to be able to adapt it for your purposes and debug it easier. 

We will use a real life example connecting to an external API to pull in data when triggered by an editor and on sync.

About me

I am the Head of Web Development at Soapbox. We are an agency that builds large sites in WordPress and Drupal. We have nine in house developers as well as a trusted team of Drupal Partners who work full time for us. Some clients we have built or are building sites for in Drupal: United Nations University, International Institute for Sustainable Development, Institute for Fiscal Studies, and many other think tanks big in their areas.


Session video: