encodeering

Writing a good test is hard, but, what does good actually mean in testing? Striving for an encompassing answer is not the goal of this post, but to focus on a single topic called state management. However, why should we bother, when many state-related topics do not become apparent in specifications, because of their isolated and predominant single-threaded nature?

Concurrency issues in specifications are not the main concern in terms of state, as they are usually not executed concurrently within a single specification, even though they could be when looking at the whole set. It is not about sharing a state at the same time, but about sharing the same state across multiple tests one by one. There are different ways to handle changes, depending on whether OO or FP concepts and philosophies are preferred, but a general consent could be that effectively immutable structures should be used if a state cannot be avoided or derived. A specification normally comprises over one test, so a state defined inside of one specification:

• might be reused, and accidentally applied side-effects may become visible at random.
• might decrease the overall visibility, and figuring out whether a declaration is used is part of the analysis.
• might introduce different programming styles of specifications, especially when the subject under test is defined by a before handler, but needs to be used differently.

There is always good and bad altogether and so for testing. While Story-alike frameworks give you an excellent way of structuring your tests within a specification, their outlined hello-world examples highlight these issues, because state defined by variables and setup handlers can be defined at any location in the story, including nested definitions. It takes a specification-wide, or spoken differently, non-local analysis of the mentioned points to grasp all preconditions of a test again, which will very likely take effect at a later time.

So, how can we do well?

Let us start with removing the before handlers by using:

• constant named fixtures that serve a general purpose. For example: memorable users with a non-changing, distinct behavior.
  static mary = 'Mary is female and 24 years old.'
• builder functions or factory methods for entities that vary and by calling them at the beginning of a test.

A benefit of this is the ability to analyze the test where it fails, provided that the initialization of used entities and creation of services is at the top of every test, which also leads to a clear uniform specification structure. Wait, were those handlers not meant to be used exactly for not being redundant for every test?

Indeed, but with all that said, mental and maintenance costs are out of all proportion regarding their value of complying with DRY and other principles. These principles should also be first-class citizen for a very simple reason: Production code, that can't be easily changed because of high adoption costs of the specification, will not change frequently, leading to more software erosion. Code refactoring is a crucial part of the software development process.

Before diving into the coding part, it might be valuable to summarize, where we would like to be at the end of this article. A good stateful test should be written that:

• the time required, to get an understanding of all involved components, should be small and not vary much across each test.
• the number of changes required, to change production and test code, should be small.

A drawing of this problem can be presented with the help of Jest, a popular JS testing framework. A typical implementation is given in 1, visualizing the mentioned points, and a proposed solution to all this is done in 2.

Psst: All snippets can also be found on Github and executed locally with the command npm test.

describe ('derivation showcase', () => {

    // 1

    let a; // shared partially
    let b; // shared

    beforeEach (() => {
        a = 'artifact'
        b = 42;
    });

    describe ('with nested tests', () => {

        let d; // shared

        beforeEach (() => {
            d = new Date (b); // mutable state needs to be reset
        });

        it ('uses state from different before handlers', () => {
            expect (b).toEqual (42); // 1st block
            expect (d).toEqual (new Date (42)); // 2nd block
        });



        // 2
        //            A    B      C     D  E
        const setup = f => async ( ) => f ({ a: 'artifact', b: 84, get d () { return new Date (this.b); }});  // no shareable declarations, mutable state is recreated on every call

        it ('shows state from a setup factory', setup (({ b, d }) => { // okay, a is not used in this test!
            expect (b).toEqual (84);
            expect (d).toEqual (new Date (84));
        }));

    });

});

Not only does 2 show all used components on the first line at once, but it also creates a clean setup for every execution. Jest and many other frameworks base their testing API on functions or lambdas and allow the return of promises if asynchrony is required. The latter is usually controlled by the signature of the function or lambda, which means in terms of Jest, that a parameter less function is required to activate the promise-based approach:

• setup is a higher-order function that takes the actual test code (A) and returns a new parameter less function (C), which, when invoked from the test runner (D) with the calculated environment (E), always returns a promise (B) containing any result of (A). The restriction, that the returned function needs to be parameter less is only valid for the outermost function to ensure API compatibility with Jest.

Ohkay, but what if some parts of the environment should be initialized only once because of their initialization costs? This question arise when the testing purpose is moving towards integrating external systems, like databases and two strategies exists:

• external systems start as part of the build job before the test and shut down afterward — transferring the responsibility to an outer process.
• external systems start as part of the specification.

Long story short: Going with the latter seems to be a better option because even if the total time spent is higher, all required components are part of the same logical specification and ensuring that specifications do not interfere during the execution is easier to achieve if they all run their own Docker container.

Next stop on the line: Integration testing with Jest and Docker.

How can we start a container if before handlers shall be avoided? The solution to this is called environment in Jest. An environment is created for every specification and started once before the specification is loaded and shut down in the same manner, surrounding all tests inside with a before-all and after-all semantic.

It is time to connect the dots with an example environment.

const NodeEnvironment = require ('jest-environment-node');

class MysqlNodeEnvironment extends NodeEnvironment {

    constructor (config, context) {
        super   (config, context);
    }

    async setup () {
        await super.setup ();

        /*
            you may want to use one of the following docker libraries

            * npm install --save-dev node-docker-api
            * npm install --save-dev testcontainers 

            to implement these steps cooperatively (see teardown)

            1. create a docker environment with a configuration according to the required environment (mysql in this case), e.g. environment variables, health-check
            2. start the environment
            3. wait until the environment has reached either a `success`, `timeout` or `failure` state and continue or abort the execution, respectively
            4. resolve additional configuration parameters from the container, e.g. network host and port
        */

        this.global.mysql = f => async () => f ({ /* port, host, username, password */ });   // expose a global function that serves connection information of the running database
    }

    async teardown () {
        try {
        /*
            1. cancel any running request to start the environment, e.g poison flag
            2. wait until the container has shut down, or enforce a shutdown after an amount of time
        */
        } finally {
            await super.teardown ();
        }
    }

}

module.exports = MysqlNodeEnvironment;

Wait, someone replaced the actual code with comments?! True, but it is straight forward to implement these steps with one of the mentioned libraries and they left some instructions at least. It is also a good opportunity to come into contact with this topic if seen as an exercise. When all is ready, connect the environment to the specification with the following comment at the beginning of the file:

/**
 * @jest-environment <rootDir>/test/jest/env/mysql.js
 */

It may take some time, depending on the readiness time of an image, to get through the setup completely, but Jest takes care of this. It is a good time to recap, that the execution of the mysql function inside of a test would not call the test code immediately, but return a function that is invoked later — the same behavior as shown with setup from the 2 nd example. There are two things to consider:

• all code executed by the test runner should be rather fast.
  It becomes part of the test execution and counts into the maximum test execution time. So either increase the timeout or prepare a docker image upfront that runs the costly part as part of the container startup, which is handled by the environment, as we already know.
• a container startup on every execution should be avoided while developing a feature, because fast feedback cycles value more.
  A global mysql function, serving valid connection information, is all that is needed by the test. Thus, it is recommend to create a static environment in advance that act's as a replacement during that time.

Up to here we have seen how explicit state management can be replaced by compensating all before and after handlers with different mechanisms and would say "Bingo" at this point if there wouldn't be one more thing to mention: modular expandability.

An application will most likely not communicate directly to external systems, but through API drivers, wrapped by additional service layers and so on or spoken differently:

• a driver translates connection information into a connection.
• a service translates a connection into business value.
• ...

Any of these steps can be expressed by a function with an input and an output and form a pipeline with a signature we are already familiar with:

f => async (...args) => f (...translate (...args))

/**
 * @jest-environment <rootDir>/test/jest/env/mysql-static.js
 */
const log = console.log.bind (console);

describe ('composition showcase', () => {

    // 1

    it ('should provide information from the used environment', mysql (config => log (JSON.stringify (config))));



    // 2

    const sequelize = f => async (config) => {
        const db = await connect (config);

        await migrate (db);
        await truncate (db);

        try {
            return await f (db);
        } finally {
            await disconnect (db);
        }
    };

    it ('should supply a service layer', mysql (sequelize (async ({ teams, sequelize }) => log (`creating an ${await teams.create ('awesome-team')}, still on ${sequelize.whom}`))));



    // 3

    const populate = f => async (db) => {
        const entities = {
            team: await db.teams.create ('awesome-team'),
        };

        return f (db, entities);
    };

    it ('should populate required entities to avoid a before/after setup', mysql (sequelize (populate (async ({ teams, sequelize }, { team }) => log (`:aw_yeah:, my ${team} is here to meet the ${await teams.create ('avengers-team')}, still on ${sequelize.whom}`)))));



    // 4

    const setup = require ('ramda').compose (mysql, sequelize, populate); // imports should stay on the top; used it here to structure this showcase. welcome, ramda

    it ('should be easy to use', setup (async ({ teams, sequelize }, { team }) => log (`:aw_yeah:, my ${team} is here to meet the ${await teams.create ('avengers-team')}, still on ${sequelize.whom}`)));

});



const zzzZ     = 10;
const delay    = (duration = 1000, f) => new Promise (r => setTimeout (r, duration)).then (f); // not cancelable, just for this showcase

// a real implementation targeting a test preparation for sequelize can be found at
// https://medium.com/riipen-engineering/testing-with-sequelize-cc51dafdfcf4

const connect  = ({ database,      host,   port }) => {
    const whom = `${database} on ${host}:${port}`;

    log (`connecting ${whom}`);

    return delay (zzzZ, () => ({
        Sequelize: {},
        sequelize: {
            whom,
        },
        teams: {
            create (whatever) { return delay (zzzZ, () => whatever); },
        }
    }));
};
const disconnect = db => delay (zzzZ, () => log (`disconnecting ${db.sequelize.whom}`));
const migrate    = db => delay (zzzZ, () => log (`migrating ${db.sequelize.whom}`));
const truncate   = db => delay (zzzZ, () => log (`truncating ${db.sequelize.whom}`));

The last part in 4 shows the composition of a pipeline with smaller building blocks. Many of them could be stored separately and reused in different specifications, not to mention how great a company-wide testing module would be.

Happy jest-ing!

The photograph for this article was taken by Scott Trento.