Prefer Narrow Assertions in Unit Tests

This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.

by Kai Kent

Your project is adding a loyalty promotion feature, so you add a new column CREATION_DATE to the ACCOUNT table. Suddenly the test below starts failing. Can you spot the problem?

TEST_F(AccountTest, UpdatesBalanceAfterWithdrawal) {

  ASSERT_OK_AND_ASSIGN(Account account,

                       database.CreateNewAccount(/*initial_balance=*/5000));

  ASSERT_OK(account.Withdraw(3000));

  const Account kExpected = { .balance = 2000, /* a handful of other fields */ };

  EXPECT_EQ(account, kExpected);

}

You forgot to update the test for the newly added column; but the test also has an underlying problem:

It checks for full equality of a potentially complex object, and thus implicitly tests unrelated behaviors. Changing anything in Account, such as adding or removing a field, will cause all the tests with a similar pattern to fail. Broad assertions are an easy way to accidentally create brittle tests  - tests that fail when anything about the system changes, and need frequent fixing even though they aren't finding real bugs.

Instead, the test should use narrow assertions that only check the relevant behavior. The example test should be updated to only check the relevant field account.balance:

TEST_F(AccountTest, UpdatesBalanceAfterWithdrawal) {

  ASSERT_OK_AND_ASSIGN(Account account,

                       database.CreateNewAccount(/*initial_balance=*/5000));

  ASSERT_OK(account.Withdraw(3000));

  EXPECT_EQ(account.balance, 2000);

}

Broad assertions should only be used for unit tests that care about all of the implicitly tested behaviors, which should be a small minority of unit tests. Prefer to have at most one such test that checks for full equality of a complex object for the common case, and use narrow assertions for all other cases.

Similarly, when writing frontend unit tests, use one screenshot diff test to verify the layout of your UI, but test individual behaviors with narrow DOM assertions.

For testing large protocol buffers, some languages provide libraries for verifying a subset of proto fields in a single assertion, such as:

Prefer Narrow Assertions in Unit Tests

This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.

by Kai Kent

Your project is adding a loyalty promotion feature, so you add a new column CREATION_DATE to the ACCOUNT table. Suddenly the test below starts failing. Can you spot the problem?

TEST_F(AccountTest, UpdatesBalanceAfterWithdrawal) {

  ASSERT_OK_AND_ASSIGN(Account account,

                       database.CreateNewAccount(/*initial_balance=*/5000));

  ASSERT_OK(account.Withdraw(3000));

  const Account kExpected = { .balance = 2000, /* a handful of other fields */ };

  EXPECT_EQ(account, kExpected);

}

You forgot to update the test for the newly added column; but the test also has an underlying problem:

It checks for full equality of a potentially complex object, and thus implicitly tests unrelated behaviors. Changing anything in Account, such as adding or removing a field, will cause all the tests with a similar pattern to fail. Broad assertions are an easy way to accidentally create brittle tests  - tests that fail when anything about the system changes, and need frequent fixing even though they aren't finding real bugs.

Instead, the test should use narrow assertions that only check the relevant behavior. The example test should be updated to only check the relevant field account.balance:

TEST_F(AccountTest, UpdatesBalanceAfterWithdrawal) {

  ASSERT_OK_AND_ASSIGN(Account account,

                       database.CreateNewAccount(/*initial_balance=*/5000));

  ASSERT_OK(account.Withdraw(3000));

  EXPECT_EQ(account.balance, 2000);

}

Broad assertions should only be used for unit tests that care about all of the implicitly tested behaviors, which should be a small minority of unit tests. Prefer to have at most one such test that checks for full equality of a complex object for the common case, and use narrow assertions for all other cases.

Similarly, when writing frontend unit tests, use one screenshot diff test to verify the layout of your UI, but test individual behaviors with narrow DOM assertions.

For testing large protocol buffers, some languages provide libraries for verifying a subset of proto fields in a single assertion, such as: