Testim.io Tutorial: Speed-up Authoring and Execution of Automated Tests

In this hands-on tutorial, you will learn how to build stable tests in minutes using Testim automation tool and use machine learning process to speed-up the authoring, execution, and maintenance of automated tests.

When we hear the phrase “Automation Tool”, the major considerations that come to mind with regards to our tests include:

• Authoring and Execution
• Stability
• Reusability
• Extensibility
• Maintenance
• Troubleshooting
• Reporting
• CI/CD Integration

There are many tools in the market which address one or few of the above factors but not all of them. This is where Testim.io comes into the picture.

Testim.io is not a tool but a platform that includes almost all the prime factors like Authoring & Execution, maintenance, troubleshoot, reporting, etc., of automated tests. With the introduction to the concept of Dynamic locators, they focus more few testing types like functional testing, end-to-end testing, and UI testing.

It increases the extensibility and stability of your test suites. And also gives the teams and the organizations, the flexibility to extend the functionalities of the platform using complex programming logic with JavaScript and HTML.

Here is a detailed insight on Testim.io and how it helps to overcome some of the challenges that customers face during automation.

How is this tool different from other tools?

In order to understand how Testim differentiates itself from the other tools let’s take a step back on how automation tools have evolved over the past decade.

In the past 10 years, there have been several vendor tools and open source tools that tried to address the various problems related to automated testing. But, one major problem that no one has addressed until now is the use of static ID’s.

For Example, say a developer changes the ID name or some attribute of an element on the web page. The automated tests immediately break due to this change, as it is still referring to the old ID name. This leads to unstable tests and the testers have to spend a lot of time in maintaining them.

They introduced the concept of Dynamic locators. The Artificial Intelligence (AI) underneath the platform in real time, analyzes all the DOM objects of a page and extracts the objects and its properties.

Finally, the AI decides the best location strategy to locate a particular element based on this analysis.

Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. As a result of this, authoring and execution of automated tests are much faster and more stable.

Testim.io is a Platform and NOT a Record nor a Playback Tool

First of all, this is not a tool but a platform.

It has robust features which not only help to automate your tests but it also takes care of the other important aspects of your tests such as running your tests locally and in the cloud (their private grid or 3rd party grid), integrating with CI/CD systems, capturing logs and screenshots of test runs, giving detailed reporting of the test runs with graphs, statistics and other related information.

And finally making the capture and reporting of bugs very simple by integrating with the widely used bug tracking tools.

Secondly, it gives the organizations and the teams the flexibility to build their own wrappers around the platform by using complex programming logic with JavaScript and HTML. In this way, anyone can extend the functionality of the platform.

Finally, the platform aligns with the Shift-Left paradigm where the Developers and Testers can start writing the tests right from the start by working on different branches without overwriting each other’s changes. It also helps the teams to collaborate and gives the flexibility for anyone to write automated tests via its built-in functionalities.

This way, even a non-technical team member can make valuable contributions to your automated tests. The platform helps to instill the concept that, automated tests should be simple, effective, stable and collaborative.

Addressing the Key Factors

How does Testim Address the key factors that are vital to Automated Tests?

Authoring and Execution

Automated tests need to be easy to author and execute them from anywhere and at any time.

Testim helps to do this in the following ways:

1) Creating a Test

We create a new Test by clicking on “Create New” or “New Test”.

2) Recording and Playing back a test

Once we click the “Record” button, we can record different user actions in our application.

After recording the different actions, click on “Stop Recording” button to finish recording our tests. Use the “Play” button to replay the recorded test and the “Save” button to save the tests.

3) Validations and Assertions

This platform helps to make validation of different attributes of an element and really simple API’s.

Various options for users such as:

• Adding Custom Validations using JavaScript and HTML
• Validate element visibility
• Validate element text
• Pixel level validation
• API level validation

4) Screenshots

While each test is recorded, the platform takes a screenshot of all the Pass and Failed results of each and every step. As a result, the users find it easier to troubleshoot the problems and understand what happens underneath the hood.

For Example, In the below screenshot, we will notice that the attributes of the “Select Destination” button have changed but the test still ran because the tool AI used Dynamic Locators. Also, the functionality of the system did not change but only the attribute of an element.

We also get the screenshot of what exactly happened while running the test by displaying the Expected image (Baseline) and the Actual image.

(Note: Click on the below image for an enlarged view)

5) Feedback on Each Step

The user also gets the feedback on each step in terms of whether the tests Passed or Failed by showing a “Green” or “Red icon” on the top left portion of each step as shown below:

6) Labeling tests

Testim provides the feature to label each and every test that a user creates.

There are 2 reasons for which we may want to label a test:

• Helps to identify the reason for which the test was created in the first place.
• Helps to run the tests with the same label all at once through the CLI feature.

The way we create labels is by clicking on the “Label” button and either select an existing label or create a new one.

7) User Documentation

They provide users with all the documentation that they will need to use the different features of this platform.

Most of the answers about using this platform can be found by clicking on the “Educate” tab and by visiting the  documentation site as shown below:

Due to the above reasons, Testim.io helps to make authoring and execution of tests really fast and simple. Within a matter of seconds, a user can record, replay and save the tests.

Stability

Testim uses Dynamic locators instead of Static ids. Due to this, the tests are more stable and the more we run the tests the smarter the AI becomes in ensuring more stability to the automated tests.

A user will be able to see the different location strategy that their AI uses to locate an element in the IDE, through the Properties panel, for each element that we interact via our automated tests.

As we can notice, the AI parses through all the DOM objects, lists them in the Properties Panel along with the rankings of each and every location strategy for that particular element. In this way, even if the attribute of an element changes, then the AI can use a different location strategy from the already parsed list of DOM objects.

Thus, a user does not have to worry about flaky tests.

Reusability

One of the good practices of writing automated tests is creating reusable components that can be used in different parts of our test suite.

Why is this important?

Creating reusable components is important because:

• It helps to increase the readability of the automated tests.
• Saves effort by not repeating the same set of steps in different parts of the tests.
• Any changes to the reusable step need to be done only in one place and it is reflected throughout the tests, across different projects.
• Makes the automated tests more extensible.

Testim helps to ensure Reusability by “Grouping” and “Parameterization”.

1) Grouping

Any number of related steps can be grouped into one reusable component.

For Example – The “Login” scenario is one of the most commonly used steps in any application. The way in which we can create a reusable “Login” step would be to select the steps that we want to group together and then click on “Add new Group” as shown below:

2) Parameterization

This platform gives the option of testing the application through various input combinations via Parameterization.

This can be achieved in various ways.

One way to do this is to give all the input parameters that we would need to test the application in the form of a JSON file in the Setup step (first step of our tests) as shown below:

Extensibility

This tool gives the flexibility for organizations to extend the functionalities of this platform using JavaScript and HTML. This way, in any functionality, that this platform does not handle, the user can write their own code to build a robust automation framework

For Example – Say we want to validate the “Select Destination” button from our previous examples.

The way to do this would be.

• Click on “Add custom action”.

• Give a name to the New Step and click on “Confirm”.

• Click on “PARAMS” and Select “HTML” for this example.
• Add Custom Code.

• The new step with Custom Code gets added to the list of already existing steps.

Maintenance

One of the most important factors related to automated tests is “Maintenance”.

A lot of effort is spent on maintaining the tests than writing the actual tests. This leads to wastage of valuable effort and time from the resources who could have rather spent that time on testing the actual application.

Testim helps to minimize the effort required to maintain the automated tests in the following ways:

1) Version Control

At any given time, it is important to have the logs of what changes were made to a particular test. This way we can always revert back to the older version of a test as and when required. This platform provides this functionality by showing all the version history by going to the Properties panel of the setup step and clicking on “See old revisions”.

2) Branching

In the “Shift Left Paradigm” where both Development and Testing must start in parallel as early as possible in the software development lifecycle.

Keeping this in mind, that they have provides the functionality to the teams to create separate branches for each team member and work on the same projects and tests. This way, no one can overwrite the changes of the other team members and the teams can work on the same code base at any time.

You just need to select “Fork” to create a new branch and you can also switch between existing branches.

Users have the option of scheduling their tests. This helps to run the tests automatically on a certain day and time without any manual intervention. We can also get notified via email in case of any errors.

Troubleshooting

As testers, we spend a considerable amount of time in troubleshooting the issues. To help in troubleshooting, this platform offers different options to the users to narrow down the scope of the problem.

These options include:

1) Screenshots

The screenshot feature explained in the “Authoring and Execution” section helps the users to know what was the baseline image and what is the actual image found.

2) Properties Panel

The properties panel helps to capture the error messages and display it to the user.

For Example, When an element is not visible and the test fails, we get the below error message in the panel.

3) Test Logs

Logs are a rich source of information on what happened underneath the UI. The tool provides test logs when the user runs the tests on our grid or a 3rd party grid. The option can be found in the properties panel of the setup step.

4) Documentation

Testim put a lot of effort to document most of the features of the tool in the User Documentation that is found under the “Educate” tab.

Reporting

This is one of the most important aspects of your tests to get the visibility on what tests ran, passed, failed, duration of the run, what browser combinations and so on.

Testim.io provides 2 types of the Reporting feature:

1) Reporting of Suite and Test Runs

When users click on “Runs” option in the side menu, they can see all the suite and test runs with all the required details as shown below. This helps to get the visibility on the test runs and helps in easier reporting.

2) Reporting Bugs

We spend a considerable amount of time in a day reporting bugs. This is one of the most critical and time-consuming parts of testing.

Testim.io helps to reduce the bug reporting time and increase the productivity by giving a chrome extension to capture and report the bugs quickly. We can capture screenshots, videos of the bug and record an actual automated test with this extension. It integrates with most of the widely used bug tracking systems.

After capturing the defect and clicking on “Publish”, it automatically populates all the required steps in the Bug Description. It also adds the screenshot, video and the automated tests with the defect.

(Note: Click on the below image for an enlarged view)

CI/CD Integration

This tool integrates with different CI/CD systems such as Jenkins, Travis CI, Circle CI and other build tools. It also integrates with 3rd party grids hosting their own private cloud to run tests. All this is done via the CLI (Command Line Interface) that we provide in the platform.

Clicking on the “Settings” icon from the side menu opens up the CLI.

It is pre-populated with the commands that need to run the tests on CI/CD systems, 3rd party grids or their grids. We just have to click on “Copy” and literally, paste the command in whichever build tool or grid the organization is using.

The CLI also accepts the other parameters to run specific tests, to run tests in parallel, to override the base URL and a lot more functionalities.

Conclusion

In a nutshell, the Testim.io platform helps to overcome some of the major challenges with automation as described in the above sections. With this, you can make automation simpler & more stable.

This platform will help:

• Non-technical team members to get involved in automation and write solid tests.
• Technical team members use complex programming logic to extend the functionalities of this platform.
• Authoring and execution of tests are super fast.
• Tests are more stable with the use of “Dynamic Locators”.

Overall it is a platform that helps in various facets of test automation and they are in the process of building more functionalities to help the agile teams and organizations to collaborate with the motto of ONE TEAM and ONE GOAL.

=> You can explore more about this tool here

Feel free to ask if you have any queries and also share your thoughts in the comments section as we would love to know what challenges you face with automation and how you managed to solve them.