During the previous blog post in this series, we introduced a new testing strategy named property-based testing.
To illustrate the usage of this technique we wrote some properties for our sum function written in Swift or Java. But we changed the approach to test our software. From specifying input data and asserting that the result value is the expected one, to using randomly generated input data and check if there are some properties we can validate. The SDKs we used were SwiftCheck and JUnit-QuickCheck.
To go deeper into the usage of this testing strategy we are going to implement a new application. The software we are going to test this post models the behavior of our office, the Karumi headquarters. In the Karumi HQs, there is just one rule, the number of maxibons (an awesome icecream) in the fridge has to be always greater than two. The Java and Swift solution can be found in our GitHub profile.
This is the exercise:
Summer is coming and sometimes our small team needs Maxibons to work better. But in the Karumi HQs finding a Maxibon is not always easy. We start every week with 10 Maxibons but once there are just 2 Maxibons or less we need to buy more.
Karumi developers can consume zero or a positive number of maxibons. The Karumi team is composed of five engineers and everytime some of these engineers go to the kitchen they grab some maxibons as follows:
If the developer is Pedro, he grabs three maxibons.
If the developer is Davide, he does not grab any maxibon.
If the developer is Fran, he grabs one maxibon.
If the developer is Jorge, he grabs one maxibons.
If the developer is Sergio, he grabs two maxibon.
When a Karumi engineer goes to the kitchen, they can go in group if needed, and there are just 2 maxibons or less left they have to send a message through the Slack API saying "Hi guys, I'm <NAME OF THE DEVELOPER>. We need more maxibons!". And the number of maxibons available will be automatically incremented by 10 :). If the number of maxibons left is lower than the number of maxibons the developer tries to get they will get just the number of maxibons available.
Based on the problem statement we can easily implement our software as you can see in the Swift or Java solution where the main classes we are going to use are: Developer, KarumiHQs, and Chat.
Based on this sofware how can we start writing our properties? First, we need to identify the properties we need to evaluate. We can start checking an easy one: "a Karumi developer can't grab a negative number of maxibons".
Swift solution:
class DeveloperSpec: XCTestCase {
func testAll() {
property("The number of maxibons a developer can grab can't be negative")
<- forAll { (numberOfMaxibons: Int) in
let developer = Developer(name: "Pedro", numberOfMaxibonsToGet: numberOfMaxibons)
return developer.numberOfMaxibonsToGet >= 0
}
}
}
Java solution:
@RunWith(JUnitQuickcheck.class) public class DeveloperProperties {
@Property public void theNumberOfMaxibonsAssignedIsPositiveOrZero(int numberOfMaxibons) {
Developer developer = new Developer("Pedro", numberOfMaxibons);
assertTrue(developer.getNumberOfMaxibonsToGrab() >= 0);
}
}
If we add a print trace inside our properties we will see how the variable numberOfMaxibons will take many different values from the maximum integer to the minimum integer representable when our tests are executed.
But what about the most important property in our app? "The number of maxibons in the fridge has to be always greater than 2". To be able to know if our software matches this property we need to first write it down. For our previous properties, we used just a bunch of integers as input parameters, but our KarumiHQs class receives instances of Developer instead (see the methods named openFirdge).
To be able to use Developer instances randomly generated we need to provide the testing framework a mechanism to instantiate these values. In Swift, we need to extend Developer to implement the Arbitrary protocol. In the Java version, we are going to create a new class extending Generator<T>.
Swift version:
extension Developer: Arbitrary {
public static var arbitrary: Gen<Developer> {
return Gen<Int>.fromElementsIn(Int.min...Int.max).map {
let name = String.arbitrary.generate
return Developer(name: name, numberOfMaxibonsToGet: $0)
}
}
}
Java version:
public class DevelopersGenerator extends Generator<Developer> {
public DevelopersGenerator() {
super(Developer.class);
}
@Override public Developer generate(SourceOfRandomness random, GenerationStatus status) {
String name = RandomStringUtils.randomAlphabetic(random.nextInt(16));
int numberOfMaxibons = random.nextInt(Integer.MIN_VALUE, Integer.MAX_VALUE);
return new Developer(name, numberOfMaxibons);
}
}
These are the generators we are going to use in our properties as described below.
Swift version:
class KarumiHQsSpec: XCTestCase {
func testAll() {
property("The number of maxibons left can not be lower than two")
<- forAll { (developer: Developer) in
let karumiHQs = KarumiHQs()
karumiHQs.openFridge(developer)
return karumiHQs.maxibonsLeft > 2
}
}
Java version:
@RunWith(JUnitQuickcheck.class) public class KarumiHQsProperties {
private KarumiHQs karumiHQs;
@Before public void setUp() {
karumiHQs = new KarumiHQs();
}
@Property public void theNumberOfMaxibonsIsAlwaysGreaterThanTwo(
@From(DevelopersGenerator.class) Developer developer) {
karumiHQs.openFridge(developer);
assertTrue(karumiHQs.getMaxibonsLeft() > 2);
}
}
What if we want to use a list of developers instead of just one Developer instance? We can write our properties as follows:
Swift version:
property("The number of maxibons left can not be lower than two")
<- forAll { (developers: ArrayOf<Developer>) in
let karumiHQs = KarumiHQs()
karumiHQs.openFridge(developers.getArray)
return karumiHQs.maxibonsLeft > 2
}
Java version:
@Property public void theNumberOfMaxibonsIsAlwaysGreaterThanTwo(
List<@From(DevelopersGenerator.class) Developer> developers) {
karumiHQs.openFridge(developers);
assertTrue(karumiHQs.getMaxibonsLeft() > 2);
}
The list of developer will be full of random developer instances or completely empty. Depending on the random input data generation performed by the testing framework. Take a look at the usage of List<@From(DevelopersGenerator.class) Developer> developers and ArrayOf<Developer>. These two constructors are provided by the framework, there is no need to create or own list generators.
Henceforth when any random Developer instance is needed, we will use our DeveloperGenerator or ArbitraryDeveloper implementations as the property generator. If for any reason we need to customize how Developer instances are generated we can create a new generator or customize how the arbitrary values are generated.
Swift version:
extension Developer: Arbitrary {
public static var arbitrary: Gen<Developer> {
return Gen<Int>.fromElementsIn(Int.min...Int.max).map {
let name = String.arbitrary.generate
return Developer(name: name, numberOfMaxibonsToGet: $0)
}
}
public static var arbitraryHungry: Gen<Developer> {
return Gen<Int>.fromElementsIn(8...Int.max).map {
let name = String.arbitrary.generate
return Developer(name: name, numberOfMaxibonsToGet: $0)
}
}
}
Java version:
public class HungryDevelopersGenerator extends Generator<Developer> {
public HungryDevelopersGenerator() {
super(Developer.class);
}
@Override public Developer generate(SourceOfRandomness random, GenerationStatus status) {
String name = RandomStringUtils.randomAlphabetic(random.nextInt(16));
int numberOfMaxibons = random.nextInt(8, 10);
return new Developer(name, numberOfMaxibons);
}
}
The complete generators documentation for Java or Swift can be found in the JUnit-QuickCheck or SwiftCheck documentation.
In the next blog post, we review how we can combine property-based testing with test doubles to check if our software is working or not. Take a look!
This post is just a small part of the content shown during our testing training. If you are interested in performing a training in your company or to be part of our open trainings, all the information can be found in our web site.