Skip to content

Iptv Assist

Learn More from us

Menu
  • HOW TO
  • Firestick
  • Chromecast
  • PC Apps
  • Lg Smart TV
  • IPTV Services
  • Automation Testing
  • Smart TV
  • Software Testing Tools
  • Contact Us
Menu

How to Have Seamless Script Execution Planning and Reporting for Success of an Automation Project

Posted on February 5, 2023

Best Iptv Service Provider 2023 With 40k+ Channels And 100k+ VOD . 24/7 Suppport . Paypal Supported

So far we have discussed many things related to test automation. We have discussed how to start the automation process in your organization, how to select the tool for automation, what are the types of frameworks we can develop and how we can create scripts in a maintainable manner.

Best Iptv Service Provider 2023 With 40k+ Channels And 150k+ VOD . Hurry Up

Check Here the list of all tutorials in this automation testing series.

Recommended IPTV Service Providers

  1. IPTVGREAT – Rating 4.8/5 ( 600+ Reviews )
  2. IPTVRESALE – Rating 5/5 ( 200+ Reviews )
  3. IPTVGANG – Rating 4.7/5 ( 1200+ Reviews )
  4. IPTVUNLOCK – Rating 5/5 ( 65 Reviews )
  5. IPTVFOLLOW -Rating 5/5 ( 48 Reviews )
  6. IPTVTOPS – Rating 5/5 ( 43 Reviews )

 

We will now discuss the execution plan of scripts.

What You Will Learn:

  • Execution Plan
  • Q #1. What is the execution environment?
  • Q #2. When and by whom these test cases will be executed?
  • Q #3. Should test cases be stopped on the first failure or should they continue execution?
  • Reporting
  • Some other points to remember:
  • Conclusion

Execution Plan

Tips for Successful Script Execution Plan:

Test Execution and Reporting

The most important thing about execution plan is that it has to be planned before the creation of scripts. You need to know some points before creating the scripts.

For example, consider the application under test is a web application. You have to create some scripts to test the login feature of the website.

Now it seems simple at a first glance, to create 8 to 10 scripts to test the login feature covering major aspects. But, once you will try to execute test cases you will face many difficulties. So it is better to ask these questions before starting to create scripts.

These Questions to ask include:

Q #1. What is the execution environment?

This is actually the most important question. The environment means that where test cases will be executed.

For example, you are testing a web application which is hosted on the local environment. You will create test cases, keeping in mind that this website is hosted locally. But in future, if this website is to be hosted on a live server on the Internet, we should know that beforehand. Once we know it, we can rectify that by keeping many values outside our script that can be changed easily.

For example the URL of the website, the usernames, the passwords, the emails and many other things that could be changed with the change in the server. We will keep these changeable values outside of our script (in an excel file or database or a configuration settings file) so that they can be changed easily without having to modify our script.

Secondly, where our tests will be executed? Will they be executed on a VM (Virtual Machine) or on a physical machine? If you will use a VM, how many VMs are needed and how much RAM and other processor requirements on each VM are required? All this need to be answered.

The important thing to know is – Can we execute the test cases without installing the tool on these VMs? Because installing the tool on a VM means buying another license of that tool which will be very expensive. For this purpose, some tools give an execution engine which is a stripped down and less expensive version of the tool and it is only used to execute test cases.

For example, MS Coded UI gives “MS Test Agent” and Test Complete gives “TestExecute” for this purpose.

The advantage of using a VM is that you can start the execution and minimize the VM and continue doing other work on your physical machine. So the time taken by scripts to execute can be utilized to create new scripts.

The third point related to the environment is on how many browsers you want the scripts to be executed? If we need to execute on Chrome, Firefox and IE, we will make sure that the techniques we use to identify elements should work on all 3 browsers. If we are using CSS techniques, we will make sure that these CSS attributes are supported on all browsers for seamless execution.

Q #2. When and by whom these test cases will be executed?

The “When” question seems easy to answer. “On every build”.

But that was not the intention of this question.

Nowadays, companies are using Continuous Integration.

What is continuous integration? In simple words, you can say that – as soon as developer checks in their code, a routine will be executed, which will integrate all the parts of the application and release the build automatically. In some cases, this build will be deployed automatically and will be tested automatically by automated scripts.

The purpose of this question is to determine that whether our automated scripts will be part of a continuous integration? If yes, then we have to make sure that our scripts can properly integrate with the CI servers such as Jenkins, MS Build and Hudson etc.

Some companies use nightly builds. In such cases, we have to make sure that our scripts can be started automatically and executed automatically without any human intervention.

Some companies do not use any build server. They manually release the build and from there, we install the software on our system and execute the automated test case. In this scenario, sometimes the scripts are executed by the automation team.

In some companies, manual testers run these scripts or a dedicated resource is hired to execute scripts. It all depends on the scale of application and number of environments. If a number of environments are higher, I suggest hiring a dedicated resource for script execution.

Q #3. Should test cases be stopped on the first failure or should they continue execution?

Some applications are such that their flow is heavily dependent on previous actions.

For example, Consider you are testing a payroll module from an ERP application. The tests are such that, in first test case you have to create an employee in the database. In the next test case, you have to print a salary check for that employee.

Now consider this situation:

If the employee is not created (due to a bug in an application), you cannot give him any salary. So in this kind of scenarios, if the first test case is failed there is no point to run the next test case.

So before designing the test cases, we have to see the dependency between test cases. If they are dependent on each other, we should order them in a respective manner. We should stop the execution on the first failure and report the bug.

Some applications are such that they have a base state.

For example, a dashboard page from which all links can be opened. So this dashboard page will become the base state in our test case. Every test case will start from that base state and end at this base state. So that the new test case can start without any problem. In this kind of scenarios, we do not stop the execution on failures.

If any test case fails, we mark it failed and return to base state and continue the next test case (it is also called the recovery scenario). At the end of execution, we have a report of how many test cases have failed and how many of them are passed. We debug the failed test cases and identify and report bugs. (The login page scenario, discussed in the beginning of the article also contains a base state).

The above questions must be asked and their answers must be known to automation engineers in order to carry out successful script development and seamless execution. A simple automated login script is easy to create but the execution is a lot tougher if we try to create scripts without considering the above points.

Reporting

Tips for Effective Reporting of Test Execution:

If scripts are great, but reporting is not, then it is difficult to find bugs through automation.

See also => How to Report Test Execution Smartly for manual testing projects

Clear and comprehensive reports help us to reach a conclusion after scripts execution is completed.

The reporting formats are very different in each tool, but I will try to list down the most important aspects that should be present in the report.

1) Report for a batch of scripts:

If multiple test cases are included in a batch and that batch is executed, then the important points to include in the report are:

  1. Total Number of scripts.
  2. List of all test cases in tabular form
  3. Test Result (Passed or Failed in front of every test case)
  4. Duration (in front of every test case)
  5. Machine / Environment Name (in front of every test case)

2) Detailed report for an individual test case:

Now when we click on any test case to see the detailed report, the detailed report of an individual test case should contain the following information.

  1. Name of the test case
  2. The id of that test case if it is connected to test case repository.
  3. Duration of the test case. (min: ss)
  4. Status (Passed or Failed)
  5. Screen Shots (only on failure or on every step)
  6. Where it fails. (The exact line number in the script where it was failing)
  7. Any other helpful logs which we have written in the script to be displayed on the report.

Examples:

Example of Batch Report: (Selenium C# Web Driver and MSTest)

Test automation reporting 1

Example of Detailed Report for a Passed Test Case:

Test automation reporting 2

Example of Detailed Report for a Failed Test Case

(Click on image to view enlarged)

Test automation reporting 3

Some other points to remember:

  1. Duration is an important factor. If the test cases are taking the longer duration to complete, they must be carefully debugged and optimized for speedy performance. The quicker the test case completes, the better it is.
  2. The Screenshot should be taken on failure only to speed up the execution.
  3. The Report should be exportable in a shareable format such as a PDF or Excel File.
  4. Custom Messages should be written on assertions and checkpoints so that in case of Failed Assertion, we exactly know just by seeing the report that what has gone wrong.
  5. A good practice is to Log a line of text before every action. This log will be displayed in the report and a layman will know that on which action the test case has failed.

Conclusion

Seamless Execution and proper reporting are one of the important factors in test automation.

I have tried my best to explain these factors in simple language with my own experience. We have faced many difficulties when we were not having an execution plan in place. But in our next projects, we came up with the proper execution planning and it resulted in much less pain afterwards.

We would love to hear your point of view as well. We will definitely learn something from your comments.

PREV Tutorial #5 | NEXT Tutorial #7

Related

Best Iptv Service Provider 2023 With 40k+ Channels And 150k+ VOD . Hurry Up

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recent Posts

  • IPTV List: Best iptv lista 2023
  • IPTV Premium: Best Premium IPTV Service Provider List And Benefits
  • Nikon IPTV Review: Over 10,000 Live Channels for $12/Month
  • Iptvwings. Com Review: +18000 Live IPTV Channels ,+70000 Movies, +40000 TV show For $15/1 month
  • IPTVUNO Review: More Than 16000 Live TV channels, 55,000 Movies & VOD For $15/Month

Recent Comments

  1. IPTV List: Play lista iptv 2022 - Iptv Assist on Best IPTV Player in 2023 for Watching Live TV
  2. Cola IPTV – Over 18,000 Live Channels for $12/Month - Iptv Assist on FileLinked – How to Install on Firestick/Fire TV and Android Devices
  3. Cola IPTV – Over 18,000 Live Channels for $12/Month - Iptv Assist on 50+ Best IPTV Service Providers for Streaming Live TV 2023
  4. XoomsTV IPTV – Over 11,000 Channels & VOD for Under $13/Month on Best VPN for IPTV in 2023 and How to Install on Firestick/Android
  5. Voodoo Streams IPTV Review – Over 12,000 Channels for $11/Month - Iptv Assist on Dynasty TV IPTV Review – Over 6,000 Channels for $10/Month

Archives

  • February 2023

Categories

  • Activate
  • Agile Testing
  • Alternatives
  • Android
  • APK
  • Apple TV
  • Automation Testing
  • Basics of Software Testing
  • Best Apps
  • Breakfast Hours
  • Bug Defect tracking
  • Career in Software Testing
  • Chromebook
  • Chromecast
  • Cross Platform
  • Database Testing
  • Delete Account
  • Discord
  • Error Code
  • Firestick
  • Gaming
  • General
  • Google TV
  • Hisense Smart TV
  • HOW TO
  • Interview Questions
  • iPhone
  • IPTV
  • IPTV Apps
  • Iptv Service SP
  • IPTV Services
  • JVC Smart TV
  • Kodi
  • Lg Smart TV
  • Manual Testing
  • MI TV
  • Mobile Testing
  • Mod APK
  • newestiptv.com
  • News
  • Nintendo Switch
  • Panasonic Smart TV
  • PC Apps
  • Performance Testing
  • Philips Smart TV
  • PS4
  • PS5
  • Python
  • QA Certifications
  • QA Leadership
  • QA Team Skills
  • Quality Assurance
  • Reddit
  • Reviews
  • Roku
  • Samsung Smart TV
  • Screenshot
  • Selenium Tutorials
  • Sharp Smart TV
  • Skyworth Smart TV
  • Smart TV
  • Soft Skills For Testers
  • Software Testing Templates
  • Software Testing Tools
  • Software Testing Training
  • Sony Smart TV
  • Sports
  • Streaming Apps
  • Streaming Devices
  • Tech News
  • Test Management Tools
  • Test Strategy
  • Testing Best Practices
  • Testing Concepts
  • Testing Methodologies
  • Testing News
  • Testing Skill Improvement
  • Testing Tips and Resources
  • Toshiba Smart TV
  • Tutorials
  • Twitch
  • Types of Testing
  • Uncategorized
  • Vizio Smart TV
  • VPN
  • Web Testing
  • What is
  • Xbox
©2023 Iptv Assist | Design: Newspaperly WordPress Theme