Property-based testing for Swift & Java. Part 3
During the previous blog post, we reviewed the usage of a testing strategy named property-based testing.
To illustrate the usage of this technique, we wrote some properties declaring that the number of ice creams in the Karumi headquarters has to always be greater than two.
Our previous properties were quite straightforward to write down because they were based on an in-memory variable our KarumiHQs instance keeps. We could even rewrite our openFridge methods to be implemented as pure functions if needed. However, we still have one requirement we haven't tested yet, and that is that we have to send a message using the chat. This is an extract of said specification:
"When a Karumi engineer goes to the kitchen, they can go in group if needed, and there are just 2 maxibons or less left they have to send a message through the Slack API saying "Hi guys, I'm . We need more maxibons!"."
As we are developing using the object oriented paradigm, side effects are part of our design. Based on this, at some point, we will need to check if messages are being sent as a side effect computed inside the openFridge method. To do this we are going to combine property-based testing with the usage of test doubles.
We are going to combine the usage of a custom generator (that we implemented in the previous post) with the usage of a mock in order to check if we are effectively sending a message when a developer grabs more than 8 maxibons.
Swift solution:
class KarumiHQsSpec: XCTestCase {
func testAll() {
property("If there are two or fewer maxibons after opening the fridge the developer sends a message to buy more")
<- forAll(Developer.arbitraryHungry) { (developer: Developer) in
let chat = MockChat()
let karumiHQs = KarumiHQs(chat: chat)
karumiHQs.openFridge(developer)
let expectedResult = chat.messageSent == "Hi guys, I'm \(developer). We need more maxibons!"
chat.messageSent = nil
return expectedResult
}
property("If there are more than two maxibons after opening the fridge the developer does not send any message")
<- forAll(Developer.arbitraryNotSoHungry) { (developer: Developer) in
let chat = MockChat()
let karumiHQs = KarumiHQs(chat: chat)
karumiHQs.openFridge(developer)
let expectedResult = chat.messageSent == nil
chat.messageSent = nil
return expectedResult
}
}
}
Java solution:
@RunWith(JUnitQuickcheck.class) public class KarumiHQsProperties {
private KarumiHQs karumiHQs;
private Chat chat;
@Before public void setUp() {
chat = mock(Chat.class);
karumiHQs = new KarumiHQs(chat);
}
@Property public void ifThereAreLessThanTwoMaxibonsLeftAMessageIsSentRequestingMore(
@From(HungryDevelopersGenerator.class) Developer developer) {
karumiHQs.openFridge(developer);
verify(chat).sendMessage("Hi guys, I'm " + developer.getName() + ". We need more maxibons!");
}
@Property public void ifThereAreMoreThanTwoMaxibonsLeftNoMessagesAreSentOrderingMore(
@From(NotSoHungryDevelopersGenerator.class) Developer developer) {
karumiHQs.openFridge(developer);
verify(chat, never()).sendMessage(anyString());
}
}
By customizing the generator used to create instances of hungry developers or not so hungry developers we ensure every evaluation of our property always performs the side effect we need to check using the Chat mock.
The usage of test doubles in Java or Swift is also different. In Java, we use a mocking library named Mockito. Meanwhile, for Swift we use handmade mocks.
When testing our software, knowing what properties we need to validate is not always easy. However, there are some common properties we can take a look when using this technique. Think of a property as a condition your code has to satisfy under any circumstances. This doesn't mean that our properties have to be satisfied for any arbitrary input but that the input for a given property needs to be defined in a generic way. Here you have a list of common properties and their definition:
Idempotence
A function is idempotent if, no matter how many times is applied (but at least one), it always returns the same result. Sorting an array could be an example. No matter how many times an array is sorted, the result will always be the same.
Symmetry
The symmetry property, links together two functions, let's say foo and bar, and is satisfied when the composition of those functions equals to the identity. It might sound hard at first but it's a really easy concept to understand that can be explained with a single line: bar(foo(x)) = x. Basically, bar acts as the inverse function of foo. A few examples of such functions are serialization/deserialization or insertion/deletion.
Model based
Model-based properties work by modeling one attribute of your code and testing its behavior. The models need to be a simplification of your system and should only cover one single aspect of your tested code. Coming back to our sorting example, we can think of the algorithm as a remove/insert process with each element, and model that part as a simple counter. We can then extract a useful property the model has to satisfy, the number of insertions has to be the same as the number of deletions: |insert calls| = |delete calls|
Invariants
Invariants are properties that never change during your code execution. In our sorting algorithm we have several invariants, for example, the list length is always smaller than the original size. We can also define more complex properties like that in each iteration the number of ordered elements increases (non-strictly) or that the sum of all elements in each iteration is never greater than the sum of its elements before starting the algorithm.
To sum up. There are tons of other common properties, even ones unique to your system! You just need to discover them. The important part is to detect those properties and test them to verify that your system behaves correctly under any circumstance.
Remember, this is just a new tool in your toolbox and not a silver bullet. This technique is not replacing the usage of unit or integration tests. You can also find a complete example of the usage of property-based testing in these repositories:
This post is just a small part of the content shown during our testing training. If you are interested in performing a training in your company or to be part of our open trainings, all the information can be found in our website.