How migrations might be slowing down your Laravel tests

Over the years I've read a lot of interesting posts about speeding up Phpunit. The last one being a pretty good summary by Laravel News, Tips to Speed up Your PHPUnit Tests. Remarkably, there's one trick that wasn't mentioned in any of these articles.

 

 

One of the larger test suites I run daily has about 1500 tests in it. It takes just over 4 minutes to complete at an average of 160ms per test. That's pretty good. However, lately I've noticed a delay between starting PHPUnit and running the first test. This delay grew to the point where running a single test would take almost 12 seconds with the setUp method using most of that time on... migrations.

The RefreshDatabase trait

This test suite contains mainly end-to-end tests using a local MySql server as its database. Laravel's RefreshDatabase trait is used to start every test with a freshly migrated database. Under the hood, Laravel runs the following code before every test:

protected function refreshTestDatabase()
{
    if (! RefreshDatabaseState::$migrated) {
        $this->artisan('migrate:fresh');
        
        RefreshDatabaseState::$migrated = true;
    }

    $this->beginDatabaseTransaction();
}

As you can see, the migrations will run only once before the first test. After the first test, database transactions are used to quickly revert back to the initial migrated database state. This saves a lot of time when running multiple tests.

However, the time to run these initial migrations can significantly delay your test results. This is especially annoying if you want to run just one "quick" test (or if your name is GladOS and you just want to run tests the entire time).

Let's take a look at a couple of solutions

The obvious solution might be to merge multiple migrations together into one migration per database table. There's even a couple of tools that can help you doing this (for example this one).

In any case, you'll still be limited to one migration per table. If you have a lot of tables, you'll have a lot of migrations and a lot of waiting to do when running them. Furthermore, some might argue that migrations are like version control for your database and that history should never be changed. Be that as it may, in the early development stages merging migrations together might be the cleanest and simplest solution.

If you're 40 tables deep into a large project, it might not be ideal to rewrite and deploy all migrations from scratch. In that case a solution is exporting the freshly migrated database as an SQL "snapshot". This one file containing all migrations as raw SQL queries will be significantly faster to parse and execute in comparison to the original of migrations.

Setting up faster migrations during tests

Start by clearing the database and running the migrations using php artisan migrate:fresh. Then open up your preferred database client and export (or backup) the empty database. You should be left with a single SQL file. Let's rename that file to migrations_2019_01_10.sql and put it in our app's database directory.

Next up, we'll have to execute this SQL file in the RefreshDatabase trait. You can either copy the entire trait to your own codebase or simply override the the method in your own TestCase.php. That'll end up looking something like this:

abstract class TestCase extends \Illuminate\Foundation\Testing\TestCase
{
    use CreatesApplication;
    use RefreshDatabase;

    protected function refreshTestDatabase()
    {
        if (! RefreshDatabaseState::$migrated) {
            DB::unprepared(file_get_contents(database_path('migrations_2019_01_10.sql')));

            $this->artisan('migrate');

            $this->app[Kernel::class]->setArtisan(null);

            RefreshDatabaseState::$migrated = true;
        }

        $this->beginDatabaseTransaction();
    }
}

As you can see, we've kept the migrate command in there to run any new migrations that might be been added after our migrations.sql snapshot. This way you wont have to export the migrated database every time a migration is added. Just remember to prepare a new snapshot every once in a while.