API testing best practices

Priorities

When starting to develop test scenarios, some of us are relatively common to jump right into some of the negative test scenarios (to verify invalid information, validation on special characters, uncommon input, etc.). These types of verification might cause crashes and other interesting problems. In other words, easy bugs might be found quickly with negative test scenarios. Unfortunately, it is not necessarily the best use of a tester’s effort. It is suggested that mainline scenarios should be focus first with higher priority. A basic guideline is to identify the most common parameters and conditions that an end developer will use when calling the APIs and test these scenarios extensively.

Functional and Non-functional testing priorities

API is an essential part of modern web applications, its non-functional aspects such as performance, security, connectivity will affect the system heavily. The behavior makes traditional testing mindset, which focuses on functional testing first, not suitable anymore. Functional and non-functional testing should be performed at the same time with equal priority.

End-points management

A typical product may have hundreds of API/Web services endpoints, which look very similar to each other, and might cause duplication issues to the testing project. How you organize the tests and the endpoints will influence the productivity, effectiveness, and maintenance of your tests.

Test endpoints could be grouped by path (including path parameters), which generally performance action on specific object types. For examples:

api/2/issue: affects to Jira issue
   -- Create issue
   POST /rest/api/2/issue
   -- Create issues
   POST /rest/api/2/issue/bulk
   -- Get issue
   GET /rest/api/2/issue/{issueIdOrKey}

while

api/2/group: affects to Jira Group
   -- Create group
   POST /rest/api/2/group
   -- Get group  DEPRECATED
   GET /rest/api/2/group
   -- Remove group
   DELETE /rest/api/2/group

This method will help us preventing missing test endpoints, and test scenarios.

Endpoint as a template

This endpoint is a part of the data-driven approach, which relates to data models more than input data. One of the most challenging issues in API testing is test data mapping with data models, subsets of data models. For examples:

An API creating new Jira issue requires an input data model having required fields (such as Project Key, Summary) and non-required fields (such as Description, Assignee, etc.). All the combination of required and non-required fields should be tested to ensure that all the test scenarios are covered. For each combination, we then apply data-driven to test with different data set.

This practice suggests that we consider each combination as a test template for the endpoint, then apply data-driven approach for this test template to generate actual executable test scenarios.

Data-driven

The most important feature of any API test tools is the capability of the data-driven approach. Lacking data-driven ability will lead to test data hardcoded, duplicated test scripts, test verification, therefore causing massive effort at maintenance phase.

Together with data-driven, there are some important notes for test data that we should pay attention seriously: data types, blank, empty, null string. With RESTful web services, the input data is typically in JSON format, and missing value of a specific key is considered differently (null, empty) in some cases. To avoid these ambiguous test scenarios, using a subset of input data models are highly suggested.