Note To Self: Best Practice for Testing Random Generators

While testing is great, you can’t test non-deterministic (i.e., ‘random’) systems. This is due to their unpredictability, which makes it hard to know what to expect when. Testing works best when a system is deterministic — the system's output only changes when the input changes.

Recently I had to implement functionality that randomly selected values based on a weighted distribution. The process involved implementing a binary search, which was reasonably complex enough to require some tests to ensure I got everything right. However, there was a problem: I was randomly selecting values.

This random selection was based on a random number I generated inside the same binary search function. This made the process non-deterministic, which meant it was impossible to test.

But there’s a solution: don’t put the random number generation inside the function. By bypassing the random number as a parameter instead, we turn a non-deterministic process into a deterministic one. And we can then test the function by passing in specific numbers and verifying we get the expected output for that value.