The main feature of the Specflow is that it focuses on Acceptance testing. It made it easy for anyone in the team to read and write test and with this feature it brings business users in to the test process, helping teams to explore and understand requirements.
Feature: Sign up
Sign up should be quick and friendly.
Scenario: Successful sign up New users should get a confirmation email and be greeted personally by the site once signed in.
Given I have chosen to sign up When I sign up with valid details Then I should receive a confirmation email And I should see a personalized greeting message
Scenario: Duplicate email
Where someone tries to create an account for an email address that already exists.
Given I have chosen to sign up But I enter an email address that has already registered Then I should be told that the email is already registered And I should be offered the option to recover my password
Now after a look on the above example code anybody can understand the working of the test and what it is intend to do. It gives an unexpected powerful impact by enabling people to visualize the system before it has been built. Any of the business user would read and understand the test and able to give you the feedback that whether it reflects their understanding of what the system should do, and it can even leads to thinking of other scenarios that needs to be consider too.
SpecFlowis inspired by Cucumber framework in the Ruby on Rails world. Cucumber uses plain English in the Gherkin format to express user stories. Once the user stories and their expectations are written, the Cucumber gem is used to execute those stores. SpecFlow brings the same concept to the .NET worldand allows the developer to express the feature in plain English language. It also allows to write specification in human readable Gherkin format.
TestNG is a Testing framework for Java programming language, inspired by JUnit and NUnit. The design goal is to cover a wider range of test categories: unit, functional, end-to-end, integration, etc., with more powerful and easy-to-use functionalities.
Support for parameterized and data-driven testing.
Support for multiple instances of the same test class (with @Factory).
Test cases can be Grouped & Prioritized more easily.
Flexible execution model. TestNG can be run either by Ant via build.xml, or by an IDE plugin with visual results.
TestNG generates test reports in HTML and XML formats.
TestNG is an open source framework which is distributed under the Apache Software License and is readily available for download.
There are two steps in configure NUnit project environment:
Configure Project with NUnit assemblies
Setup TestRunners which show the results of NUnit test cases
Configure Project with NUnit assemblies
We always creates separate project when creating project for NUnit. According to naming conventions test project name should be [Project Under Test].[Tests]. For example, if we are testing the project name “CustomerOrderService” then test project name should be “CustomerOrderService.Tests”.
Visual Studio -> New Project -> Class Library -> Name: CustomerOrderService
Right click on solution -> Add New Project -> Class Library -> Name: CustomerOrderService.Tests
Now we have two projects in our solution. In CustomerOrderService project, we write code for business logic and in second project CustomerOrderService.Tests we write test cases for CustomerOrderService project.
Our next step is to add Nunit assemblies.
Right click on CustomerOrderService.Tests and choose “Manage NuGet Packages”.
In NuGet search box, Choose Browse tab and type Nunit in search textbox.
Choose NUnit and click on Install button.
NUnit assembly (nunit.framework) is added to our test project.
Add reference of our CustomerOrderService class library to test project.
Choose add reference in test project -> Project – Solution tab -> Mark the checkbox before the CustomerOrderService -> Click on OK button.
Now our test project is configured with Nunit assemblies. Our next step is to add TestRunners to our solution.
For setup TestRunners, we need to add Nunit Test Adapter from NuGet packages. Follow the below steps:
Right click on CustomerOrderService.Tests and choose ‘Manage NuGet Packages’
Choose NUnit3TestAdapter and click on Install button.
Now we have to write sample test case to check whether every thing is setup successfully or not. Write a sample test case in class1 of CustomerOrderService.T
Include namespace NUnit.Framework in namespaces section. Write the above code exactly in Class. We’ll learn more about TestFixture attribute, Test attribute and Assert class in our next posts. Now Build the solution.
Choose visual studio Test Menu -> Windows -> Test Explorer.
Text Explorer shows Test function in Not Run Tests section. Choose Run All button to execute test cases.
In below of Test Explorer, it will show the result of Test1 test result. In the below screenshot, it is showing the result ‘Test Passed – Test1’.
Now our Test project and TestRunner is configured properly. In next posts, we’ll learn more about Nunit attributes and classes.
NUnit is a unit testing framework for .NET. It is the most used framework for writing unit test cases.
We can write testing code in either C# or VB.NET. It is suggested to write testing code in different assemblies called Test Assemblies. These assemblies only contain testing code nothing else. We need to run these test assemblies to check whether all test cases are passed or failed. For that we required Test Runner.
Test Runners are UI tool which actually run NUnit test cases and show the result of test cases whether they are passed or failed. We’ll learn about test runners in Environment Setup in next post.
NUnit is very easy to use. It only provides some custom attributes and some static Assert classes. With the combination of custom attributes and static classes, we can write unit test cases easily.
Custom attributes provides hint to NUnit test runners that these classes or functions contains unit testing code. Assert classes is used to test the conditions whether system under test (SUT) satisfy a condition or not. If condition is satisfied then test is pass else fail.
Some of the custom attributes are:
In the next post, We’ll learn how to setup your environment for writing NUnit test cases.
Selenium 4 version is all set to release this Christmas. Simon Stewart (founding member of Selenium) has officially announced at the recently held selenium conference at Bangalore. Some major changes in the upcoming Selenium 4 have been revealed. So, it’s time to get ahead of the curve and figure out what is going to be changed, added and deprecated. In this article, we will take a look a few important features and give an insight on updates you can expect for your automation framework.
W3C WebDriver Standardization
The Selenium 4 WebDriver will be completely W3C Standardized. The WebDriver API has grown to be relevant outside of Selenium. It is used in multiple tools for automation. For example, it’s used heavily in mobile testing through tools such as Appium and iOS Driver. The W3C standard will encourage compatibility across different software implementations of the WebDriver API.
Let’s take an example of how Selenium Grid communicates with Driver executables as of now –
A test in Selenium 3.x, at local end (through json wire protocol) communicates with browser at End node.This requires encoding and decoding of API. In Selenium 4, test will directly communicate without any encoding and decoding of API requests (through W3C Protocol) though java bindings will be backward compatible but focusing more on W3C Protocol.
The Selenium IDE support for Chrome is in bucket. As we all know that Selenium IDE is a record and playback tool. It will now be available with much more rich and advance features such as:
New plugin system– Any browser vendor will now be able to easily plug-in to the new selenium IDE. You can have your own locator strategy and can plug in selenium IDE.
New CLI runner – It will be completely based on nodejs and not old html based runner. It will have following capabilities-
WebDriver Playback – The new Selenium IDE runner will be completely based on WebDriver.
Parallel execution- The new CLI runner will also support parallel test case execution and will provide useful information like time taken, number of test cases passed and failed.
Improved Selenium Grid
The one who have worked on Selenium Grid must know how difficult it is to setup and configure.Selenium Grid supports test case execution on different browsers, operating systems and machines. Thus it provides parallel execution capability.
There are two main elements in Selenium Grid- Hub and Node.
Hub- Acts as a server,a central point to control all the test machines in a network. In selenium grid there is only one hub which allocates the test execution to a particular node based on capability matches.
Node- In simple words node is a test machine where test cases actually run.
The typical process to setup selenium grid till now caused testers sometimes to face difficulty in connecting node to hub.
In Selenium 4 the grid experience is going to be easy and smooth. As there will not be any need to setup and start hub and node separately. Once we start selenium server, the grid will act as both hub and node.
Selenium 4 will come up with more stable selenium grid in terms of removing all thread-safety bugs, and a better support for docker.
Better Selenium Grid UI
The Selenium 4 would come up with more user friendly UI of grid having relevant informations about sessions running, capacity etc.
“Passive observability is the ability to do descriptive tracing.” – Simon Stewart
Observability, logging and debugging is no more confined to DevOps in recent times. As part of this release request tracing and logging with hooks will be improved so as to provide automation engineers a hold on debugging.
Documentation plays a key role in any project’s success. Selenium docs were not updated since selenium 2.0 release. In the latest upgrade selenium documentation is also going to be refreshed and detailed [WIP].You can access it on – https://seleniumhq.github.io/docs/
Selenium 4 In a nutshell
Upgrading to the latest version of selenium should not require any code changes. Setting up node and hub will become smooth and the entire grid experience is going to be streamlined. For automation engineers, the latest version should not be challenging and existing automation framework should work with minimal changes.
Many of us are very familiar with the inputs, tools, techniques, and outputs (ITTOS) of traditional project management in a Project Management Institute way. I am making an attempt here to map the Scrum model into ITTOs for anyone’s understanding and easy implementation.
To become expert in any system, one must understand the ecosystem by knowing the inputs to the system, the tools and technologies used to process the inputs, and finally the outputs of the system. In the Agile/Scrum delivery model we have various inputs, tools, and technologies to process, and finally there will be outputs.
Scrum in nutshell
Scrum suggests three roles: the team, ScrumMaster, and product owner; four ceremonies: the sprint planning meeting, Daily Scrum, sprint review meeting, and sprint retrospective meeting; and three artifacts: the product increment, product backlog, and sprint backlog. We will try to understand the various elements that support these ceremonies in terms of ITTOs. The complete Scrum team attends all the ceremonies except the retrospective, which the product owner may or may not attend.
Sprint planning meeting
Timebox: Eight hours for a four-week sprint, proportionately shorter for shorter sprints Attendees: The complete Scrum team, including all roles Most important: Team capacity and DoD (Definition of Done)
Timebox: Fifteen minutes is standard, irrespective of the duration of the sprint length Attendees: The complete Scrum team, including all roles Most important: To speak about any impediments
Sprint review meeting
Timebox: Four hours for a four-week sprint, proportionately shorter for shorter sprints Attendees: The complete Scrum team, including all roles, plus any other stakeholders who are interested in the project success Most important: Demo of working software and assessing the feedback
Sprint retrospective meeting
Timebox: Three hours for a four-week sprint, proportionately shorter for shorter sprints Attendees: The complete Scrum team, including all roles; the product owner’s attendance is optional Most important: To brainstorm and agree on what is working and what is not
I think this helps map Scrum for those familiar with PMI, and any PMP can easily relate the concepts of Agile/Scrum that would be useful in transitioning from a Waterfall delivery model to an Agile delivery model. These ITTOs can be extended to a few other Scrum practices, such as Sprint Zero, Sprint H, and a mid-sprint health check. As these best practices are not prescribed in the Scrum framework, I do not focus on them here.
Following skills are essential to becoming a good software tester. Compare your skill set against the following checklist to determine whether Software Testing is a reality for you-
Analytical skills: A good software tester should have sharp analytical skills. Analytical skills will help break up a complex software system into smaller units to gain a better understanding and create test cases. Not sure that you have good analytical skills – Refer this link – if, if you can solve at least ONE problem you have excellent analytical skills.
Communication skill: A good software tester must have good verbal and written communication skill. Testing artifacts (like test cases/plans, test strategies, bug reports, etc.) created by the software tester should be easy to read and comprehend. Dealing with developers (in the event of bugs or any other issue) will require a shade of discreetness and diplomacy.
Time Management & Organization Skills: Testing at times could be a demanding job especially during the release of code. A software tester must efficiently manage workload, have high productivity, exhibit optimal time management, and organization skills
GREAT Attitude: To be a good software tester you must have a GREAT attitude. An attitude to ‘test to break’, detail orientation, willingness to learn and suggest process improvements. In the software industry, technologies evolve with an overwhelming speed, and a good software tester should upgrade his/her technical skills with the changing technologies. Your attitude must reflect a certain degree of independence where you take ownership of the task allocated and complete it without much direct supervision.
Passion: To Excel in any profession or job, one must have a significant degree of the passion for it. A software tester must have a passion for his / her field. BUT how do you determine whether you have a passion for software testing if you have never tested before? Simple TRY it out and if software testing does not excite you switch to something else that holds your interest.
This list is long, so please bear with me
Basic knowledge of Database/ SQL: Software Systems have a large amount of data in the background. This data is stored in different types of databases like Oracle, MySQL, etc. in the backend. So, there will be situations when this data needs to be validated. In that case, simple/complex SQL queries can be used to check whether proper data is stored in the backend databases.
Basic knowledge of Linux commands: Most of the software applications like Web-Services, Databases, Application Servers are deployed on Linux machines.So it is crucial for testers to have knowledge about Linux Commands.
Knowledge and hands-on experience of a Test Management Tool: Test Management is an important aspect of Software testing. Without proper test management techniques, software testing process will fail. Test management is nothing but managing your testing related artifacts.For example – A tool like Testlink can be used for tracking all the test cases written by your team.There are other tools available that can be utilized for Test Management. So, it is important to have knowledge and working experience of such tools because they are used in most of the companies.
Knowledge and hands-on experience of any Defect Tracking tool- Defect Tracking and Defect life cycle are key aspects of software testing. It is extremely critical to managing defects properly and track them in a systematic manner. Defect tracking becomes necessary because the entire team should know about the defect including managers, developers, and testers. Several tools are used to lock defects including QC, Bugzilla, Jira etc.
Knowledge and hands-on experience of Automation tool: If if you see yourself as an “Automation tester” after a couple of years working on manual testing, then you must master a tool and get in-depth, hands-on knowledge of automation tools.
Note – Only knowledge of any Automation tool is not sufficient to crack the interview, you must have good hands-on experience, so practice the tool of your choice to achieve mastery. Knowledge of any scripting language like VBScript, Java Script, C# is always helpful as a tester if you are looking for a job into automation. Few companies also use Shell/Perl scripting, and there is a lot of demand for testers having knowledge of the same. Again, it will depend on the company and which tools are used by that company.
There is also a lot of scope for Performance Testing tools because applications need to be tested for their performance which is a part of non-functional testing.
Note: That’s it to technical knowledge. Please note you do not need ALL the technical skills listed above. The technical skill sets required vary with the Job Role and company processes.
Testing is important because software bugs could be expensive or even dangerous. Software bugs can potentially cause monetary and human loss, history is full of such examples.
In April 2015, Bloomberg terminal in London crashed due to software glitch affected more than 300,000 traders on financial markets. It forced the government to postpone a 3bn pound debt sale.
Nissan cars have to recall over 1 million cars from the market due to software failure in the airbag sensory detectors. There has been reported two accident due to this software failure.
Starbucks was forced to close about 60 percent of stores in the U.S and Canada due to software failure in its POS system. At one point store served coffee for free as they unable to process the transaction.
Some of the Amazon’s third party retailers saw their product price is reduced to 1p due to a software glitch. They were left with heavy losses.
Vulnerability in Window 10. This bug enables users to escape from security sandboxes through a flaw in the win32k system.
In 2015 fighter plane F-35 fell victim to a software bug, making it unable to detect targets correctly.
China Airlines Airbus A300 crashed due to a software bug on April 26, 1994, killing 264 innocent live
In 1985, Canada’s Therac-25 radiation therapy machine malfunctioned due to software bug and delivered lethal radiation doses to patients, leaving 3 people dead and critically injuring 3 others.
In April of 1999, a software bug caused the failure of a $1.2 billion military satellite launch, the costliest accident in history
In may of 1996, a software bug caused the bank accounts of 823 customers of a major U.S. bank to be credited with 920 million US dollars.