Karumi Blog

Replacing creational patterns for testing with property-based testing generators

Pedro Gómez -

Creational patterns are one of the most powerful tools any engineer can use when writing automated tests. But today we are going to review a different approach, we will review the usage of a testing strategy heavily used in functional programming for replacing the usage of creational patterns in our tests.

When writing tests we always need instances of classes with some specific characteristics or state. To solve this problem we use creational patterns in order to obtain instances easily without duplicating the object instantiation code all around our tests.

Let's review an example! Imagine we are working on a fintech company where we need to create instances of a class named Transaction for our tests. A transaction represents a money transfer between two bank accounts.

object model {

 object TransactionType extends Enumeration {
    val Deposit, Withdrawal = Value
  }

  object TransactionSubtype extends Enumeration {
    val ACHCredit, ACHDebit, ManualACHFromRemoteAccount, ManualACHFromLocalAccount, Wire, Check = Value
  }

  object OperationSourceIdentifier extends Enumeration {
    val Admin, Member, Procedure = Value
  }

  object Status extends Enumeration {
    val New, Pending, Posted, Failed, Deleted = Value
  }

  case class Account(number: String, routing: String)

  case class Transaction(id: UUID,
                         creationDate: DateTime,
                         lastStatusUpdateDate: Option[DateTime],
                         memberId: UUID,
                         creatorType: OperationSourceIdentifier.Value,
                         createdBy: Option[UUID],
                         transactionType: TransactionType.Value,
                         transactionSubtype: Option[TransactionSubtype.Value],
                         amount: BigDecimal,
                         remote: Option[Account],
                         finance: Option[Account],
                         status: Status.Value,
                         effectiveDate: Option[DateTime],
                         creatorIpAddress: String)
}

We could need an instance with some characteristics to use it as a stub or as input in our tests. Using creational patterns like Object Mother or a Builder so that we can get the instances as code like this:

val withdrawal = TransactionsMother.postedWithdrawal(amount = -15000)

val deposit = TransactionsMother.depositPendingToBeApproved(amount = 4000)

or

val withdrawal = TransactionsBuilder
  .withAmount(-15000)
  .status(Status.Posted)
  .build()

val deposit = TransactionsBuilder
  .withAmount(4000)
  .status(Status.Pending)
  .build()

Thanks to these patterns we can easily get an instance. The object mother can be more expressive and the builder approach more flexible, but what if we try something different? What if we could use randomly generated data with some characteristics or state defined for the instances? When using a builder or a mother the default values used for the rest of the instance constructor params are hardcoded. We don't actually care for these values at all as long as we configure some specific fields so, what if we just use random values here instead of hardcoded ones? We could write code something like this:

val withdrawal = arbitraryTransaction(
  amount = -19000, 
  status = Status.Posted)

val deposit  = arbitraryTransaction(
  amount = 8000,
  status = Status.Pending)

And this would generate values with randomly generated data for the rest of the transaction fields but with a fixed amount and status. Additionally, we could replace the literal values with generators so the value could vary between different ranges:

val withdrawal = arbitraryTransaction(
  arbitraryAmount = Gen.choose(-9000, -10), 
  arbitraryStatus = Gen.oneOf(Seq(Status.Pending, Status.Posted)))

val deposit  = arbitraryTransaction(
  amount = Gen.const(8000),
  status = Gen.oneOf(Status.values.toSeq))

A simple implementation could be something like this if we use Scalacheck:

def arbitraryTransaction(amount: BigDecimal, status: Status.Value): Gen[Transaction] =
    arbitraryTransaction(arbitraryAmount = Gen.const(amount, arbitraryStatus = Gen.const(status)))
  
  def arbitraryTransaction(arbitraryAmount: Gen[BigDecimal] = arbitraryAmount, arbitraryStatus: Gen[Status.Value] = arbitraryStatus): Gen[Transaction] = {
    val generator = for {
      id                     <- Gen.uuid
      creationDate           <- arbitraryDateTime
      lastStatusUpdate       <- Gen.option(arbitraryDateTime)
      memberId               <- Gen.uuid
      creatorType            <- arbitraryCreatorType
      createdBy              <- arbitraryCreatedBy
      transactionType        <- arbitraryTransactionType
      transactionSubtype     <- arbitraryTransactionSubtype
      amount                 <- arbitraryMoneyAmount
      availableAmount        <- arbitraryMoneyAmount
      virtualBalance         <- arbitraryMoneyAmount
      availableBalance       <- arbitraryMoneyAmount
      remoteAccount          <- Gen.some(arbitraryAccount)
      originAccount          <- Gen.some(arbitraryAccount)
      status                 <- arbitraryStatus
      effectiveDate          <- Gen.option(arbitraryFutureDateTime(new DateTime()))
      nachaFileId            <- Gen.option(Gen.uuid)
      creatorIpAddress       <- arbitraryIpAddress
      synapseFiTransactionId <- arbitrarySynapseFiTransactionId
      synapseFyDashboardUrl  <- Gen.option(arbitraryAlphanumericStringSize(2000))

    } yield
      Transaction(
        id,
        creationDate,
        lastStatusUpdate,
        memberId,
        creatorType,
        createdBy,
        transactionType,
        transactionSubtype,
        amount,
        virtualBalance,
        availableAmount,
        availableBalance,
        remoteAccount,
        originAccount,
        status,
        effectiveDate,
        creatorIpAddress
      )
  }

Thanks to the usage of the default parameters and the overloaded version we can use both generators and literal values in parameters:

val withdrawal = arbitraryTransaction(
  arbitraryAmount = Gen.choose(-9000, -10), 
  arbitraryStatus = Gen.oneOf(Seq(Status.Pending, Status.Posted)))

val deposit  = arbitraryTransaction(
  amount = 8000,
  status = Status.Pending)

There is just one piece missing, we need to be able to get an instance of our transactions so we don't depend on the property-based framework to get instances of our generators already defined. We can easily achieve this goal by using Scala extensions, also known as pimp my library pattern:

implicit class RichGen[T](val gen: Gen[T]) extends AnyVal {
    final def one: T = {
      val sample = gen.sample
      if (sample.isDefined) {
        sample.get
      } else {
        one
      }
    }
  }

If you are a functional programmer this function could break your heart. But don't you worry, we are just trying to be pragmatic and let the rest of the world getting closer to the property-based testing strategy we all love :)

With this extension method we can write tests without using forAll functions as follows:

"Transactions creator" should "never create withdrawals in posted state" in {
  val withdrawal = arbitraryTransaction(
    amount = -9000, 
    status = Status.Posted).one

  val result = createTransaction(withdrawal)

  result.isLeft shouldBe true
}

or using forAll functions:

"Transactions creator" should "always create deposits without taking the state into account" in {
  forAll(arbitraryTransaction(
  arbitraryAmount = Gen.choose(9000, 10))) { deposit =>
    val result = createTransaction(deposit)

    result.isRight shouldBe true
  }
}

We could also reuse these generators in a composable way and create a richer version of the arbitraryTransaction generator creating one just for deposits:

def arbitraryDeposit(arbitraryAmount = Gen[BigDecimal] = Gen.choose(1, 9000)) =
  arbitraryTransaction(arbitraryAmount = arbitraryAmount)

The approach seems interesting, but there are some pros and cons we should know:

  • The definition of the values used in our generators will make us think twice in the data we are going to handle in our application.
  • Generators are highly composable definitions we can combine all over our application. We can reuse them when needed.
  • As we can use generators as part of a property-based testing framework or a regular test we can use both testing strategies based on our needs.
  • Due to the usage of default parameters configured as generators we can easily choose the range of values for just some of our parameters.
  • The usage of random values for the parameters we don't define can help us to find bugs during our tests execution.
  • The usage of random values can make our tests flaky so we would need to improve our assert messages so we can know what the input was when the test failed.
  • Debugging can be harder using this approach.
  • Without using the property-based testing framework we miss one of the killer features, shrinking :(
  • We can automatize data generation using Scalacheck-Shapleless if we don't want to write the generators manually.

If for some reason you'd like to use this approach without Scalacheck you can take a look at this example our friends from CodelyTV wrote.

Some interesting pros and cons, right? Thanks to the usage of generators we can combine the object mother expressiveness and the configurable capabilities of the builder with the power of generators getting the best of every pattern in just one piece of code we can easily combine an reuse. We'd strongly recommend you to try this approach in Scala or any other language supporting generators like Kotlin with KotlinTest and let us know your thoughts in a comment.

Happy testing :)