The figure below shows three different layers of testing called the test pyramid initially coined by Mike Cohn in his book Succeeding with Agile. It has layers representing different types of testing. Despite its being overly simplistic, it offers us a general rule of thumb: it suggests how much testing we should focus on at each layer. As such, API and services tests in the second layer is an important testing activity that we should focus on.
Unit/component tests: This lowest level of testing brings the highest value and ROI. It is mainly performed by developers. The unit/component tests can attain between 70% and 80% of code coverage and require as much effort.
Business rules / Functional tests: This level of testing focuses on business rules of the application under test. The tests are designed to test against user stories to ensure that all implemented functions are working as expected. In Web and mobile applications, data shown in the user interface is often returned from servers via API/services. So API testing is performed to ensure the accuracy of API/services. Testing at this level may need about 20% of the total testing effort.
Workflow Tests (through the UI): functional UI testing is performed via the UI of the application to ensure that its features are built as expected. Due to the fact that automated functional UI testing is brittle, and any changes in the UI can break the tests, test automation teams should focus less on this level of testing.
Commonly, applications have three separate layers or tiers including Presentation Layer or user interface, Business Layer or application user interface for business logic processing, and Database Layer for modeling and manipulating data.
API Testing is performed at the most critical layer, the Business Layer, where business logic processing is carried out, and all transactions between User Interface and Database happen.
API testing is a type of software testing that involves testing application programming interfaces (APIs) directly and as part of integration testing to determine if they meet expectations for functionality, reliability, performance, and security. Since APIs lack a GUI, API testing is performed at the message layer and can validate application logic very quickly and effectively.
API testing is critical for automation testing and CI/CD process because it can cope with short release cycles and frequent changes especially the presentation layer without breaking the test outputs. API testing also requires less maintenance effort compare to UI automation testing which makes it a preferred choice for Agile and DevOps teams.
A side note, for Web and mobile applications, API often means Web services, and API testing refers to the automation test performed to the Web services.
This article aims to provide an overview of API testing with candid answers to the What – When – Why-questions that hopefully shed light on this mysterious land hidden inside the boundary of testing – engineering realm.
You will also find potential challenges with API test implementation – best practices and how to choose right toolset to help you achieve API testing with a higher success chance.
The main feature of the Specflow is that it focuses on Acceptance testing. It made it easy for anyone in the team to read and write test and with this feature it brings business users in to the test process, helping teams to explore and understand requirements.
Feature: Sign up
Sign up should be quick and friendly.
Scenario: Successful sign up New users should get a confirmation email and be greeted personally by the site once signed in.
Given I have chosen to sign up When I sign up with valid details Then I should receive a confirmation email And I should see a personalized greeting message
Scenario: Duplicate email
Where someone tries to create an account for an email address that already exists.
Given I have chosen to sign up But I enter an email address that has already registered Then I should be told that the email is already registered And I should be offered the option to recover my password
Now after a look on the above example code anybody can understand the working of the test and what it is intend to do. It gives an unexpected powerful impact by enabling people to visualize the system before it has been built. Any of the business user would read and understand the test and able to give you the feedback that whether it reflects their understanding of what the system should do, and it can even leads to thinking of other scenarios that needs to be consider too.
SpecFlowis inspired by Cucumber framework in the Ruby on Rails world. Cucumber uses plain English in the Gherkin format to express user stories. Once the user stories and their expectations are written, the Cucumber gem is used to execute those stores. SpecFlow brings the same concept to the .NET worldand allows the developer to express the feature in plain English language. It also allows to write specification in human readable Gherkin format.
TestNG is a Testing framework for Java programming language, inspired by JUnit and NUnit. The design goal is to cover a wider range of test categories: unit, functional, end-to-end, integration, etc., with more powerful and easy-to-use functionalities.
Support for parameterized and data-driven testing.
Support for multiple instances of the same test class (with @Factory).
Test cases can be Grouped & Prioritized more easily.
Flexible execution model. TestNG can be run either by Ant via build.xml, or by an IDE plugin with visual results.
TestNG generates test reports in HTML and XML formats.
TestNG is an open source framework which is distributed under the Apache Software License and is readily available for download.
There are two steps in configure NUnit project environment:
Configure Project with NUnit assemblies
Setup TestRunners which show the results of NUnit test cases
Configure Project with NUnit assemblies
We always creates separate project when creating project for NUnit. According to naming conventions test project name should be [Project Under Test].[Tests]. For example, if we are testing the project name “CustomerOrderService” then test project name should be “CustomerOrderService.Tests”.
Visual Studio -> New Project -> Class Library -> Name: CustomerOrderService
Right click on solution -> Add New Project -> Class Library -> Name: CustomerOrderService.Tests
Now we have two projects in our solution. In CustomerOrderService project, we write code for business logic and in second project CustomerOrderService.Tests we write test cases for CustomerOrderService project.
Our next step is to add Nunit assemblies.
Right click on CustomerOrderService.Tests and choose “Manage NuGet Packages”.
In NuGet search box, Choose Browse tab and type Nunit in search textbox.
Choose NUnit and click on Install button.
NUnit assembly (nunit.framework) is added to our test project.
Add reference of our CustomerOrderService class library to test project.
Choose add reference in test project -> Project – Solution tab -> Mark the checkbox before the CustomerOrderService -> Click on OK button.
Now our test project is configured with Nunit assemblies. Our next step is to add TestRunners to our solution.
For setup TestRunners, we need to add Nunit Test Adapter from NuGet packages. Follow the below steps:
Right click on CustomerOrderService.Tests and choose ‘Manage NuGet Packages’
Choose NUnit3TestAdapter and click on Install button.
Now we have to write sample test case to check whether every thing is setup successfully or not. Write a sample test case in class1 of CustomerOrderService.T
Include namespace NUnit.Framework in namespaces section. Write the above code exactly in Class. We’ll learn more about TestFixture attribute, Test attribute and Assert class in our next posts. Now Build the solution.
Choose visual studio Test Menu -> Windows -> Test Explorer.
Text Explorer shows Test function in Not Run Tests section. Choose Run All button to execute test cases.
In below of Test Explorer, it will show the result of Test1 test result. In the below screenshot, it is showing the result ‘Test Passed – Test1’.
Now our Test project and TestRunner is configured properly. In next posts, we’ll learn more about Nunit attributes and classes.
NUnit is a unit testing framework for .NET. It is the most used framework for writing unit test cases.
We can write testing code in either C# or VB.NET. It is suggested to write testing code in different assemblies called Test Assemblies. These assemblies only contain testing code nothing else. We need to run these test assemblies to check whether all test cases are passed or failed. For that we required Test Runner.
Test Runners are UI tool which actually run NUnit test cases and show the result of test cases whether they are passed or failed. We’ll learn about test runners in Environment Setup in next post.
NUnit is very easy to use. It only provides some custom attributes and some static Assert classes. With the combination of custom attributes and static classes, we can write unit test cases easily.
Custom attributes provides hint to NUnit test runners that these classes or functions contains unit testing code. Assert classes is used to test the conditions whether system under test (SUT) satisfy a condition or not. If condition is satisfied then test is pass else fail.
Some of the custom attributes are:
In the next post, We’ll learn how to setup your environment for writing NUnit test cases.