I recently completed a Udacity course on software testing. It was a quick course, but I learned a few interesting techniques. Mostly, I got the chance to use a couple of tools I had heard about but never actually used.

## Python coverage measurements

Python has a tool called Coverage.py that generates statement and branch coverage metrics for a given set of tests. For instance, we can write the following Fizz Buzz program and set of tests.

``````def fizzbuzz(n):
if n % 15 == 0:
return "FizzBuzz"
if n % 3 == 0:
return "Fizz"
if n % 5 == 0:
return "Buzz"
return str(n)

assert fizzbuzz(3) == "Fizz"
assert fizzbuzz(4) == "4"
assert fizzbuzz(5) == "Buzz"
``````

We can evaluate our tests with the coverage tool.

``````coverage erase
coverage run --branch fizzbuzz.py
coverage html
open htmlcov/index.html
``````

This produces the following output.

Based on this, we can easily see that we don’t have any tests covering the condition where `n` is a multiple of 15. The red line indicates a lack of statement-level coverage, while the yellow line shows a partially covered branch. We can fix both of these issues by adding one more test to our program.

``````assert fizzbuzz(30) == "FizzBuzz"
``````

## Fuzzing a program

My favorite exercise of the course was writing a simple random fuzzer. Random fuzzing is surprising effective across many programs, and although I did not find any bugs I am glad to have practiced the technique.

Random fuzzing involves starting with good inputs, mutating random bytes, and feeding those modified inputs to the software under test. I decided to test `feh`, a photo viewer. The fuzzing code is pretty self-explanatory, which I am excited about: it turns out writing a basic fuzzer is easy!

``````import math
import os
import random
import string
import subprocess
import time

file_dir = "input-photos/"
file_list = os.listdir(file_dir)
app = "/usr/local/bin/feh"
output_dir = "output-photos/"

FuzzFactor = 250
num_tests = 10000

for i in range(num_tests):
file_choice = random.choice(file_list)
with open(file_dir + file_choice, 'rb') as f:
numwrites = random.randrange(math.ceil((float(len(buf)) / FuzzFactor))) + 1

for j in range(numwrites):
rbyte = random.randrange(256)
rn = random.randrange(len(buf))
buf[rn] = rbyte

output = output_dir + str(i) + ".tif"
with open(output, 'w+b') as f:
f.write(buf)

process = subprocess.Popen([app, output], stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL)

time.sleep(2)
crashed = process.poll()
if crashed and crashed != 0 and crashed != 1:
print("Crash on test {}!".format(i))
else:
process.terminate()
``````