Testing with Magic Numbers

Magic numbers are evil right? And we should do everything we can to eliminate them from our code.

But what about using magic numbers in tests? Say I want to test that a particular method returns a particular magic number - what’s the best way to do it?

Let's say I have a class; we'll call it Houdini:

class Houdini  
end  

And in it, I have captured some magic numbers using a hash:

MAGIC = {  
  red: 34,
  blue: 19,
  yellow: 106
}

Clearly this is an arbitrary mapping, but it serves our purpose. Now I have some class methods:

magic_number_red()  
magic_number_blue()  
magic_number_yellow()  

defined in Houdini. They are trivial methods that simply return an entry from the constant MAGIC. What’s the best way to test them?

Should I test for the magic number in my test?:

expect(Houdini.magic_number_red).to eq 34  

or test against MAGIC directly?:

expect(Houdini.magic_number_blue).to eq Houdini::MAGIC[:blue]  

or should I be stubbing the constant like this?:

stub_const('Houdini::MAGIC', {yellow: :success})  
expect(Houdini.magic_number_yellow).to eq :success  

The answer is, it really depends on exactly what you want to test.

In particular, it depends on whether the constant MAGIC is an implementation detail or not. In other words, is it important that:

a. magic_number_red returns the value 34, irrespective of how this is achieved? Or:

b. magic_number_blue returns a value that corresponds with MAGIC[:blue], whatever that value might be? Or:

c. magic_number_yellow specifically returns the entry MAGIC[:yellow]?

This is important because you want to create relevant tests that aren't brittle (i.e. tests that don't fail when trivial changes are made to the target code that don't affect the program logic.).

If it's perfectly valid for me to change the program at some point (not while it's running!) so that MAGIC[:red] is 35, then clearly, the first example test is brittle, whereas the style of the second test would not be.

However, if it is vital that magic_number_blue always returns 19, then the second test is inadequate, whereas the style of the first test would be correct.

Finally, if it is vital that magic_number_yellow always returns the specific entry at MAGIC[:yellow], then only the style of the third test will suffice.

So, in conclusion, as with all testing, you should always start by being clear on exactly what you need to test, and write that test.

Arguably, in the first test, we might go a step further to eliminate the magic number 34 in the test by encapsulating it in another constant local to our test file, e.g. :

MAGIC_TEST = {  
  red: 34,
  blue: 19,
  yellow: 106
}

Then test against it like so:

expect(Houdini.magic_number_red).to eq MAGIC_TEST[:red]  

But that may or may not be a premature refactor.