1 of 33

Why Good Developers�Write Bad Tests

Michael Lynch

PyGotham 2019

​

​

https://mtlynch.page.link/gdbt-pg

2 of 33

Best practices

  • Refactor large functions into smaller ones
  • Eliminate redundancy
  • Avoid magic numbers
  • Strive for brevity

...for production code

3 of 33

Unit tests

  • Test at the smallest possible granularity
  • Verifies a particular function is correct

def test_converts_farenheit_to_celsius(self):

self.assertAlmostEqual(100.0,

converter.farenheit_to_celsius(degrees=212.0),

places=1)

4 of 33

What’s wrong with this test?

  1. Where did the joe123 account come from?
  2. Why do I expect joe123’s score to be 150?

def test_initial_score(self):

initial_score = self.account_manager.get_score(username='joe123')

self.assertEqual(150, initial_score)

5 of 33

The answers you seek are in setUp

def setUp(self):

database = MockDatabase()

database.add_row({

'username': 'joe123',

'score': 150

})

self.account_manager = AccountManager(database)

​

def test_initial_score(self):

initial_score = self.account_manager.get_score(username='joe123')

self.assertEqual(150, initial_score)

​

6 of 33

Test code is not like other code

Production code:

  • Too complex to read all at once

��Good production code:

  • Is well-factored
    • Components are in logical chunks
    • Abstracts away complexity

Test code:

  • Simple enough to read all at once
  • Developers read functions in isolation
  • Must be correct by inspection

Good test code:

  • Maximizes obviousness
  • Minimizes cognitive load

7 of 33

Tests are diagnostic tools

  • Tests need to be at least one level simpler than the thing they’re testing
  • Good tests minimize probability of incorrect measurements

8 of 33

Can another developer diagnose a failure?

  • Imagine yourself as the developer who has to diagnose why this test is suddenly failing

def test_initial_score(self):

initial_score = self.account_manager.get_score(username='joe123')

self.assertEqual(150, initial_score)

9 of 33

Keep the reader in your test function

def test_initial_score(self):

database = MockDatabase()

database.add_row({

'username': 'joe123',

'score': 150

})

account_manager = AccountManager(database)

​

initial_score = account_manager.get_score(username='joe123')

​

self.assertEqual(150, initial_score)

​

Arrange

Act

Assert

10 of 33

The reader should understand your test without reading any other code.

11 of 33

Dare to violate DRY

  • DRY
    • Don’t
    • Repeat
    • Yourself

12 of 33

Dare to violate DRY

def get_usernames():� connection = sqlite3.connect(DB_PATH)� cursor = connection.cursor()� cursor.execute(� 'SELECT username FROM users')� usernames = cursor.fetchall()� connection.close()� return usernames

def get_user_ids():� connection = sqlite3.connect(DB_PATH)� cursor = connection.cursor()� cursor.execute(� 'SELECT id FROM users')� user_ids = cursor.fetchall()� connection.close()� return user_ids

def _fetchall_single_column(column):� connection = sqlite3.connect(DB_PATH)� cursor = connection.cursor()� cursor.execute(� 'SELECT %s FROM users' % column)� values = cursor.fetchall()� connection.close()� return values��def get_usernames():� return _fetchall_single_column('username')��def get_user_ids():� return _fetchall_single_column('id')

13 of 33

Dare to violate DRY

def test_increase_score(self):� database = MockDatabase()� database.add_row({� 'username': 'joe123',� 'score': 150� })� account_manager =� AccountManager(database)�� account_manager.adjust_score(� username='joe123', adjustment=25)

self.assertEqual(175,� account_manager.get_score(� username='joe123'))

def test_initial_score(self):� database = MockDatabase()� database.add_row({� 'username': 'joe123',� 'score': 150� })� account_manager =� AccountManager(database)�� initial_score = � account_manager.get_score(� username='joe123')

self.assertEqual(150, initial_score)

14 of 33

Accept redundancy if it supports simplicity.

15 of 33

What about when you need helper methods?

def test_increase_score(self):� user_database = MockDatabase()� user_database.add_row({� 'username': 'joe123',� 'score': 150� })� privilege_database = MockDatabase()� privilege_database.add_row({� 'privilege': 'upvote',� 'minimum_score': 200� })� privilege_manager = PrivilegeManager(privilege_database)� url_downloader = UrlDownloader()� account_manager = AccountManager(user_database, privilege_manager, url_downloader)�� account_manager.adjust_score(username='joe123', adjustment=25)

self.assertEqual(175, account_manager.get_score(username='joe123'))

16 of 33

Improving your production code

simplifies your test code.

17 of 33

Cardinal sin of test helper methods

  • Never bury critical values in your test helper
  • Critical value
    • Any value the reader needs to know to understand the correctness of your test

18 of 33

Cardinal sin of test helper methods

def test_increase_score(self):� self.account_manager = AccountManager()� self.add_dummy_account()�� self.account_manager.adjust_score(� username='joe123',� adjustment=25)

self.assertEqual(175,� account_manager.get_score(� username='joe123'))

def add_dummy_account(self):� dummy_account = Account(� username='joe123',� name='Joe Bloggs',� email='joe123@example.com',� score=150)�� self.account_manager.add_account(� dummy_account)

​

Helper Method

Test Method

19 of 33

Using helper methods responsibly

def test_increase_score(self):� account_manager = AccountManager()� account_manager.add_account(� make_dummy_account(username='joe123',� score=150))�� account_manager.adjust_score(� username='joe123', adjustment=25)

self.assertEqual(175,� account_manager.get_score(username='joe123'))

​

def make_dummy_account(� self, username, score):� return Account(username=username,� name='Dummy User',� email='dummy@example.com',� score=score)

Helper Method

Test Method

20 of 33

Don’t bury critical information in your test helper methods.

21 of 33

Go crazy with long test names

  • Which function name would you choose in production code?
    • userExistsAndTheirAccountIsInGoodStandingWithAllBillsPaid
    • isAccountActive
  • Key difference in tests
    • You never write calls to test functions

22 of 33

Can you diagnose this failure?

class Tokenizer(object):�� def __init__(self, stream):� self._stream = stream�� def next_token(self):� ...�

======================================================================�FAIL: test_next_token (tests.test_tokenizer.TokenizerTest)�----------------------------------------------------------------------�AssertionError: '' is not None

23 of 33

What about this one?

======================================================================�FAIL: test_next_token_returns_None_when_stream_is_empty (tests.test_tokenizer.TokenizerTest)�----------------------------------------------------------------------�AssertionError: '' is not None

class Tokenizer(object):�� def __init__(self, stream):� self._stream = stream�� def next_token(self):� ...�

24 of 33

Name your tests so well that others can diagnose failures from the name alone.

25 of 33

Embrace magic numbers

  • Magic number
    • A numeric value or string that appears in code without information about what it represents.

employee_pay = calculate_pay(80)

HOURS_PER_WEEK = 40

WEEKS_PER_PAY_PERIOD = 2

employee_pay = calculate_pay(hours=HOURS_PER_WEEK * WEEKS_PER_PAY_PERIOD)

26 of 33

A test that avoids magic numbers

def test_greeting_appends_first_name(self):

GREETING = 'Look who’s here! It’s'

FIRST_NAME = 'Michael'

LAST_NAME = 'Lynch'

greeter = greeting.Greeter(GREETING, FIRST_NAME, LAST_NAME)

expected_message = GREETING + FIRST_NAME

self.assertEqual(expected_message, greeter.welcome_message())

27 of 33

A test that embraces magic numbers

def test_greeting_appends_first_name(self):

greeter = greeting.Greeter(greeting='Look who’s here! It’s',

first_name='Michael', last_name='Lynch')

self.assertEqual('Look who’s here! It’s Michael', greeter.welcome_message())

28 of 33

A test that avoids magic numbers

def test_greeting_appends_first_name(self):

GREETING = 'Look who’s here! It’s'

FIRST_NAME = 'Michael'

LAST_NAME = 'Lynch'

greeter = greeting.Greeter(GREETING, FIRST_NAME, LAST_NAME)

expected_message = GREETING + FIRST_NAME

self.assertEqual(expected_message, greeter.welcome_message())

GREETING + FIRST_NAME = 'Look who’s here! It’sMichael'

29 of 33

Why don’t we like magic numbers?

  • Creates implicit coupling
    • Not an issue in test code
  • Makes the reader wonder why it was chosen
    • In test code, values are often arbitrary
    • The test name should explain choices of inputs

30 of 33

Prefer magic numbers to named constants in test code.

31 of 33

Summary

  • Keep the reader in your test function
  • Accept redundancy if it supports simplicity
  • Refactor production code before adding test helpers
  • Don’t bury critical information in test helpers
  • Go crazy with long test names
  • Embrace magic numbers

32 of 33

Thanks!

33 of 33

Tell me about services you wish existed

  • I’m a solo founder, considering my next project
  • Tasks where you’ve thought:
    • “Why hasn’t anyone built a service that does X?”
  • Maybe I’ll build it
    • michael@mtlynch.io