find in path

Testing spring-retry functionality with Cucumber

2019-11-23cucumberspring-retryspring-aopwiremock

This post is intended to be heads up on the benefits in the readability of the test scenarios that come when using Cucumber library.

The Github project spring-retry-cucumber which accompanies this blog post represents a simplisting Github API client that exposes operations for retrieving Github user details.

Feel free to checkout the source code of the project and run the tests by using the command:

mvn clean test

Below is presented a simple demo of the functionality of this Github API client.

    var config = new GithubProperties("https://api.github.com/");
    var github = new Github(config);
    var usersEndpoint = github.users();

    System.out.println(usersEndpoint.getUser("findinpath"));
    System.out.println(usersEndpoint.getUsers(0));

In case that sporadic exceptions may occur on the Github API, through the usage of the spring-retry mechanism built on top of the previously demoed Github API client,these failures should go unnoticed in the client program flow.

The program flow will retrieve successfully the user details even though sometimes one sporadic API call will fail, because the API call will be retried and therefor in the client context, the API call will appear as successful (even though it was actually performed twice).

Cucumber

Cucumber is a software library that enables the usage of Behaviour Driven Development in the projects where it is integrated.

Gherkin is a core component for Cucumber responsible for parsing the executable test specifications is . Gherkin is a set of grammar rules that makes plain text structured enough for Cucumber to understand.

Below can be seen the accuracy & failure test scenarios written for the spring-retry-cucumber demo project.

When having a quick look over the scenarios above, it gets strikingly clear how the Github API client is expected to work.

The demo project spring-retry-cucumber has a few particularities in terms of usage for the Cucumber framework which will be detailed in the lines below.

Support for custom parameter types

In the images above that showcase the accuracy & failures test scenarios for the Github API client, in the scenario steps, there are used the highlighted tokens:

  • GET (corresponds to RequestMethod.GET)
  • INTERNAL_SERVER_ERROR (corresponds to HttpStatus.INTERNAL_SERVER_ERROR)

In the scenario steps they are referenced in the following manner:

WireMockApiSteps.java

  @Then("I have made {int} {requestMethod} calls made towards Github API {string} resource")
  public void checkNumberOfApiCalls(int count, RequestMethod requestMethod, String uri) {
    // ...

UserSteps.java

  @Then("I will receive an {httpStatus} response status instead of the user details")
  public void checkErroneousCall(HttpStatus httpStatus) {
    // ...

Gherkin allows the registration of custom parameter types. See below the corresponding code for registering the custom parameter types:

ParameterTypes.java

    typeRegistry.defineParameterType(new ParameterType<>(
        "requestMethod", // name
        "GET|POST|PUT|DELETE", // regexp
        RequestMethod.class, // type
        (io.cucumber.cucumberexpressions.Transformer<RequestMethod>) s -> {
          RequestMethod requestMethod;
          if ("GET".equals(s)) {
            requestMethod = RequestMethod.GET;
          } else if ("POST".equals(s)) {
            requestMethod = RequestMethod.POST;
          } else if ("PUT".equals(s)) {
            requestMethod = RequestMethod.PUT;
          } else if ("DELETE".equals(s)) {
            requestMethod = RequestMethod.DELETE;
          } else {
            throw new IllegalArgumentException("Unknown value " + s + " for RequestMethod");
          }
          return requestMethod;
        }
    ));

    var regexpHttpStatus = Arrays.stream(HttpStatus.values()).map(Enum::name)
        .collect(Collectors.joining("|"));
    typeRegistry.defineParameterType(new ParameterType<>(
        "httpStatus", // name
        regexpHttpStatus, // regex
        HttpStatus.class, // type
        (io.cucumber.cucumberexpressions.Transformer<HttpStatus>) HttpStatus::valueOf
    ));

Data tables

Data tables is a feature of Gherkin through which can be elegantly wrapped related values as objects in order to be passed the step definitions:

AccuracyCases.feature

    Given I have configured the responses for the Github API
      | uri               | httpStatus | payloadFile                      |
      | /users/findinpath | 200        | api/github/users/findinpath.json |

WireMockApiSteps.java

  @Given("I have configured the responses for the Github API")
  public void configureApiResponses(List<GithubApiResponse> responseList) {
    // ...
  }

  // ...

  public static class GithubApiResponse {

    private String uri;
    private int httpStatus;
    private String payloadFile;

    // ...

Cucumber Spring Integration

Cucumber integration with spring framework has a few tweaks are worth mentioning in the lines below.

The CommonSteps.java class is the only one annotated with @SpringBootTest annotation. This is one of the limitations imposed by the Cucumber framework in the ingration with the spring framework.

@SpringBootTest(classes = SpringDemoTestApplication.class)
@ActiveProfiles("test")
public class CommonSteps {
  //...
}

The step definition classes, the so-called “glue” to the test code get their dependendant spring beans injected by using the @Autowired annotation on the constructor of the step class. Below is presented the code of UserSteps class constructor for showcasing this particularity:

  @Autowired
  public UserSteps(UsersEndpoint usersEndpoint,
      UserSharedScenarioData userSharedScenarioData) {
    this.usersEndpoint = usersEndpoint;
    this.userSharedScenarioData = userSharedScenarioData;
  }

WireMock

The library WireMock is being used in the tests in order to be able to mock the Github API.

Particularly useful, in case of failure tests, was the Stateful Behavior for being able to simulate failures and recoveries when calling the mock API for a specific endpoint. The relevant code from WireMockApiSteps.java for configuring the mocked Github API responses is presented below:

    Given I have configured the responses for the Github API
      | uri               | httpStatus | payloadFile                      |
      | /users/findinpath | 500        |                                  |
      | /users/findinpath | 200        | api/github/users/findinpath.json |
  @Given("I have configured the responses for the Github API")
  public void configureApiResponses(List<GithubApiResponse> responseList) {
    var server = wireMockGithubApi.getWireMockServer();

    var responsesByUriMap = responseList
        .stream()
        .collect(Collectors.groupingBy(githubApiResponse -> githubApiResponse.getUri().trim()));

    responsesByUriMap.forEach((uri, uriResponseList) -> {
      for (int index = 0; index < uriResponseList.size(); index++) {
        var scenarioState =
            (index == 0) ? Scenario.STARTED : "Attempt " + (index + 1) + " for " + uri;

        var scenarioName = uri;
        var githubApiResponse = uriResponseList.get(index);
        var scenarioMappingBuilder = WireMock
            .get(WireMock.urlEqualTo(uri))
            .inScenario(scenarioName)
            .whenScenarioStateIs(scenarioState);
        if (index != uriResponseList.size() - 1) {
          scenarioMappingBuilder = scenarioMappingBuilder
              .willSetStateTo("Attempt " + (index + 2) + " for " + uri);
        }
        if (githubApiResponse.getHttpStatus() == HttpStatus.OK.value()) {
          var response = WireMock.aResponse()
              .withHeader("Content-Type", "application/json")
              .withStatus(githubApiResponse.getHttpStatus());
          if (githubApiResponse.getPayloadFile() != null) {
            response.withBodyFile(githubApiResponse.getPayloadFile());
          }
          server.stubFor(
              scenarioMappingBuilder
                  .willReturn(response)
          );
        } else {
          server.stubFor(
              scenarioMappingBuilder
                  .willReturn(
                      WireMock.aResponse()
                          .withHeader("Content-Type", "application/json")
                          .withStatus(githubApiResponse.getHttpStatus())
                  )
          );
        }
      }
    });
  }

Also very useful has proved to be the ability to browse through the requests made towards the WireMock server.

    And I have made 2 GET calls made towards Github API "/users/findinpath" resource
    But I have a backoff delay between GET requests 1 and 2 made towards Github API "/users/findinpath" resource

Check more details on how to browse through the requests reaching WireMock Server on the blog post

Spring-retry

The spring-retry topic has already been covered in a previous post. Check it out for seeing how to layer several AOP aspects on top of each other in order to get in-depth metrics (including retries) on how much time external API call takes.

The spring-retry functionality is configured in the github-api-aop-config.xml file:

  <aop:config>
    <aop:pointcut id="github-api-calls"
      expression="execution(* com.findinpath.api.github.UsersEndpoint.*(..))  "/>

    <aop:advisor pointcut-ref="github-api-calls"
      advice-ref="githubApiRetryAdvice" order="1"/>

  </aop:config>

Conclusions

As already mentioned in the Cucumber documentation the main benefits of using this libray for testing a software project are:

  • Encouraging collaboration across roles to build shared understanding of the the problem to be solved
  • Working in rapid, small iterations to increase feedback and the flow of value
  • Producing system documentation that is automatically checked against the system’s behaviour

Nevertheless, there are some drawbacks that have to be taken into account before integrating Cucumber in your project:

  • Poorly written tests can easily increase test-maintenance cost
  • Cucumber is based (at the moment of this writing) on JUnit 4. Junit 5 will be supported with the upcoming release of Cucumber 5 (see on Github the Add JUnit 5 Support issue)
  • Cucumber tests are executed in a single-threaded fashion. This may be an incovenience for projects having a lot of tests.

With all these things said, Cucumber is a very viable alternative for end to end tests because it enables collaboration across all the roles in the project (software engineer, quality assurance, project management, requirements analyst) to build shared understanding of the the problem to be solve.