Being a performance tester is steadily becoming one of the hottest specializations within the realm of software testing. The reason is that with digital transformation of our society in full blossom, digital performance is at the heart of everything – customer experience, brand image, revenue generation and other critical parts of any business.

As Ian Molyneaux famously puts it in The Art of Application Performance Testing: “If an end user perceives bad performance from your website, her next click will likely be on your-competition.com.”

=> Click Here For Complete Performance Testing Tutorials Series

So, how can you test performance? In a nutshell, performance testing and optimization of a website (or basically any IT system) require being able to put it under a specific load, measure the results, make changes and repeat. You can do that by using a performance testing tool to create a test scenario, which is then automatically performed by a large number of virtual users.

SmartMeter.io Hands-on Review Tutorial

There are many tools to choose from, each offering a different set of features and approaches towards performance testing. Today, we are going to have a closer look at one such tool – SmartMeter.io.

What is SmartMeter.io?

This is a desktop application created to cover all performance and load testing needs.

Looking at the landscape of current performance testing tools it is apparent that free tools are either too simple or cumbersome requiring a lot of manual work. On the other hand, professional tools tend to be overly complicated, obsolete and expensive.

This tool aims to be an affordable way of combining the best of these two worlds by re-thinking the de-factor open source standard JMeter.

To quote its developers on what was their motivation behind creating the tool: “While we love JMeter for what it offers freely to everybody, we were often annoyed by being stuck or wasting time on things that could be done smarter, faster, more intuitively or even fully automated.”

As the best way to evaluate any tool is to actually use it, let’s get to it.

Installation

This tool is available for Windows, Linux, and MacOS. You can download the tool here.

This is available for all major platforms which are shown in below figure:

When launching the tool for the first time you will be prompted to choose a license file from disc. If you don’t already have a license, you can click on the displayed link to see the pricing options or get a free trial by clicking on the Free evaluation button.

After that, you’re all set.

Below is an example of Licence Screen, where you can either purchase the license or download free trial:

Creating your first test

The basic workflow of performance testing consists of three steps

• Test creation
• Test run
• Test reporting

Let’s break down how to do this in SmartMeter.io.

Test Creation: Recording a Test Scenario

The first thing within test creation is recording a test scenario. The test scenario determines actions (test steps) the virtual users are supposed to perform in the test. These can be any combination of things like visiting a certain page, login, placing an order, payment, etc.

This tool includes a special component called the Recorder, which makes it quite simple to create the test scenario. Let’s see how it works.

On the welcome screen click the Start Recorder button. This will open up the Recorder.

Below is an example of how to Start the Recorder from the welcome screen:

The Create a new test window appears. There are two values you need to set for a basic test recording – Test name and the URL of the tested site.

You can leave other settings unchanged. Continue by clicking Start recording button, which will open the main Recorder window.

Below is an example of Initial settings of the Recorder:

The Recorder is split into two main parts. The left part contains a list of transactions – a step within the test scenario which contains all related HTTP requests, etc. The right part of the Recorder contains an embedded browser window where you can record new transactions.

To create new transaction click the plus button in the upper left corner and then perform the action in the embedded browser (i.e. click on the homepage).

When you recorded all transactions you want to have in your test scenario click the Save test button. When asked if you want to open the recorded test in the editor click Yes.

Below is an example of the main window of the Recorder containing transactions and browser:

Test creation: Editing scenario parameters

The recorded test scenario defines what actions will be performed in the test, the Editor allows setting the other important parameters of the test.

We have opened the editor as the last step of scenario recording. You can also open the Editor from the welcome screen by selecting the test script and clicking the Create/Edit Scenario button.

Below is an example of how to Launch the Editor from the welcome screen:

The Editor is a place where you can design the rest of the test.

In this example the most important parameters are:

• Target concurrency
• Ramp-up time
• Ramp-up steps
• Hold target rate time

Target concurrency – This number defines how many virtuals will be generated in the test (technically it is a number of threads that will run in parallel during a test). The more virtual users is set, the higher the load on the tested system.

Ramp-up time – Every test starts with zero virtual users. This number defines how long it will take until the full load will be reached.

Ramp-up steps – When filled in the virtual users will be added in batches. You can specify the number of these batches, or steps. If not filled in, the virtual users will be added gradually.

Hold target rate Time – This number specifies how long the test should go on after reaching the max number of generated virtual users.

To set these parameters click Virtual Users in the left part of the window and set the parameters according to your needs. There is also a helpful chart visualizing the test settings.

Below is the example of the Editor window and fundamental settings:

(Note: Click on any image for enlarged view)

Test run

When you’re done with the test scenario recording and settings you can start the test. It is possible to start the test straight from the Editor by clicking on the red arrow icon.

Below is an example of how to Start the test from the Editor:

You can also start a saved test from the welcome screen by selecting it in the Test Script drop-down menu and clicking the Start test button.

Below is an example of how to Start the test from the welcome screen:

When the test starts, the so-called Runner window appears. After a moment, the graphs will start to fill with data as first results arrive.

In the left part of the window, there are other metrics (also called listeners). You can drag and drop these to the main part of the window.

In the upper right corner, there are other real time metrics such as duration of the test, a number of warnings, errors, and a number of virtual users (threads) currently present in the test.

Below is an example of a Test run in the Runner window:

It is useful to see the results in real time as you can decide to stop the test before its end, for example when there are too many errors. To end the test, click the stop button in the upper left corner.

Below is an example of Force stopping the test run:

Test reporting

Test reporting is a crucial part of performance testing as it makes it possible to analyze the state of the system performance and discover potential bottlenecks, which is after all the objective of a performance tester.

In SmartMeter.io, the test report is generated automatically at the end of the test. Once the test is done just click on the Report icon in the upper left corner.

Generating test report after the test is done:

The report contains many statistical values and graphs. It provides information about metrics such as:

• Hits per second
• Response times
• Response codes
• Virtual users (Threads state)
• Transactions per second
• Bytes throughput etc…

The SmartMeter.io report contains the Target tool to help you easily compare these metrics with each other and determine their relationship at any given time. You can, for instance, compare Transactions per second with Virtual users and Response time.

You can, for instance, compare Transactions per second with Virtual users and Response time.

As we mentioned before, a transaction is basically a step in a test scenario such as visit homepage, product purchase, failed or successful login, etc. In a perfect case, the Transactions per the second graph would increase the same way as the number of virtual users.

If not, you can determine how many users it took for the system to start having a slow response just by comparing the graph values using the Target tool. And this is just one of the possible cases of results analysis.

Below is an example of the Target tool in report:

There is also a Trend analysis function built into the report. It makes it possible to compare results of different test runs at a glance.

This is a classic use case in performance testing. After the test is done and related changes by developers were made, it is important to test whether there was any improvement or not.

The Trend analysis function makes this comparison very simple. It is, however, necessary to keep test settings of individual test runs unchanged.

Below is an example of the Trend analysis function

Conclusion

So, does SmartMeter.io delivers on its promises as mentioned in the introduction of this post?

Although the tool might seem daunting at a first glance it actually provides a whole range of features to make the performance testing process complex and thorough yet straightforward.

It is important to note that this brief hands-on review has barely scratched the surface of what SmartMeter.io is capable of doing as the main goal was to see what it takes to go from nothing to some actual results. As it turns out, it takes little time and effort to do so.

It is mainly thanks to its Recorder module allowing for test scenario creation just by clicking in a browser. The same goes for its advanced automatic reporting functions which can save an incredible amount of time (as any JMeter user would know).

All in all, SmartMeter.io makes the greatly compelling choice for both testing novices and professionals alike and provides great value for its price. The best way to make up your own mind is to try it out for yourself.

Let us know if you have any questions/suggestions.

=> Visit Here For Complete Performance Testing Tutorials Series