Continuous Integration Process: How to Improve Software Quality and Reduce Risk

Continuous Integration Process Tutorial:

Our previous tutorial explained about Shift left testing in detail.

Earlier, we understood the nuances of Continuous Delivery (CD) where we have production-ready software at any point via continuous feedback loops. Continuous integration (CI) is the real meat behind the CD process and is the reason that makes Continuous Delivery possible.

To understand CI, let’s take the terms at face value and deduce a basic definition. The first word means “ongoing” or “frequent” and the second “merged” or “made part of”. So CI is a process where something is being “merged”-“frequently”.

Logically the next question is: What is the something being merged and where is it merged?

Considering that the CD is just the conceptual extension of CI, the answer is sort of obvious, isn’t it?

The “something” that is being merged is code and the “where” is a repository or Version control.

Thus, Continuous integration is a process where the code is checked into a repository very frequently.

Additionally, as discussed in the previous article, tests are run immediately to catch any errors each time a code check-in happens thereby setting up the necessary feedback loop for a Continuous Delivery.

Suggested Reading => Excellent DevOps Training Tutorials

Continuous Integration Process

Let’s retrace back to the CD pipeline diagram discussed previously. The red circle will be our focus in this article in order to understand the CI process.

Even though the CI process may seem very development centric, it’s vital for QA engineers to get an overall picture and adapt accordingly.

Before we proceed any further, the following terminology is important to know:

• Source code or version control system has all the code related to a project/feature.
• Mainline: The most recent state of the code in a version control/source code system
• Local Copy: If you have had experience with working with an Eclipse-like IDE, you will know that you can import a project’s codebase into your local machine at a particular location. That location in your local system is the “local copy”.
• Check out: When a developer begins to work, the common practice is to import the latest source code onto the local copy. This activity is a “check out”.

Say there are two or three development engineers working on a feature and are using Continuous Integration.

This is how the sequence of events would appear:

1) On the local copy, the developer builds his code for the new feature.
2) After coding is done, he might write the required automated unit tests to test his code.
3) A local build is run, to ensure that the newly added code doesn’t break anything.
4) Once the build succeeds the developer checks if any of his peers made any new check-ins.
5) In case there are new incoming changes, he has to first accept those incoming changes to make his local copy most current.
6) Synching with the mainline might result in some conflicts due to the newly merged local copies.
7) If a conflict arises, it is fixed so that the changes are in sync with the mainline code
8) After step 7, the code changes are ready to be checked in. This is called “code commit”.
9) After commit, another build pertaining to the source code (mainline) is run on an integration system.
10) This now is ready to be consumed by the next stages.

Continuous Integration Tools

It is dependent on each organization on what kind of CI tools they use.

While some of the tools like Hudson, CruiseControl, Jenkins are popular choices, there are many other tools that provide similar capabilities in addition to their own unique features.

The decision of choosing a CI tool depends on a lot of factors, such as:

• Being able to integrate with configuration management tools
• Easy, customizable reporting
• Integration with automation tools, etc.

=> Here is the complete list of CI tools with features

CI Benefits

#1) Errors detected early: If it is an error in the local copy or code checked indirectly without being synching to the mainline, a build failure will occur at the appropriate stage. It forces the developer to fix the bug before proceeding further. QA teams will also be benefited from this as they will mostly be working on stable builds.

#2) Decreases bug accumulation: Bugs are inevitable; however with the use of CI the piling of bugs is reduced greatly. Based on how effective the automation is, the bugs are easy to find early and fix, greatly reducing risks.

#3) Setting the stage for Continuous Delivery: CI reduces manual intervention because build, sanity, and other tests are all supposed to be automated. This paves way for a successful continuous delivery.

#4) Increased transparency: CI brings in a greater level of transparency into the overall development and QA processes. There is, at all times, a clear indication of what tests are failing, causes for failure, defects, etc. enabling you to make factual decisions on where and how to improve efficiency.

#5) Cost-effectiveness: Based on the above points of early error detection, less bug accumulation, more automation and clarity in the overall system, it is needless to say, that cost is optimized.

CI in QA – A Point Of View

This is a natural and logical extension to our previous discussion. These factors have to be in place to be able to use CI for testing.

#1) Initial tests: We left off at a point where we now have a good build after a code commit. The code commit should trigger some automated tests – smoke or sanity checks certifying that the build is now ready for QA.

#2) Automation Framework: To stay true to CI, every QA team should invest in building a test automation framework that automatically triggers tests that uncover not only feature specific shortcomings but also identify framework enhancement requirements (for current and new tests).

#3) Parallel testing using Automation: A robust automation framework facilitates parallel testing and replication of production with various configurations. It also yields better test coverage and fewer bug escapes.

For instance, if support for two or more browsers on certain Operating Systems is required, then tests that simulate the needed configurations can be set up and run in parallel, thereby drastically increasing test efficiency.

#4) Automation across different types of testing: Continuing the previous point, automated test coverage should include different types of testing – functional and non-functional tests – such as stress, load, performance, regression, database, acceptance, etc.

#5) Bugs: This is particularly interesting because even the logging of bugs can be automated with a CI system! You can poll for certain kinds of errors coming up in the logs and on encountering them, a bug is automatically logged. A typical example of this situation is auto-logging bugs for nullPointerExceptions observed in a server trace log.

---

Recommended Tool:

#1) LambdaTest

Software Testing Help recommends LambdaTest to perform automated web testing on a combination of 2000+ desktop & mobile browsers.

You can even run tests in parallel to reduce overall testing time, mark bugs from LambdaTest, and share it with teams via integrations like JIRA, Asana, Monday. LambdaTest supports all major CI/CD tools like Jenkins, CirlceCI, Teamcity, Bamboo, etc.

---

CI Implementation And Best Practices

Continuous Integration aims to have a drastic drop in the degree of errors during software development through feedback mechanisms, automation, and quick bug fix turnaround.

Although it may seem too ambitious for a process to achieve all of this, it can certainly be a reality with some of the continuous integration best practices described below:

#1) Shared repository to maintain code: With Agile evolving rapidly, it is a given that there are multiple developers working on different or same features of a product. It is therefore absolutely necessary to have one repository that will be able to capture the timeline of changes that all developers are making.

Version or source control tools help you create different development streams and better prepare the teams to be able to respond to these needs. They also help in keeping all the artifacts needed to perform a complete build (libraries, properties, environment variables, test scripts, etc.) in one place.

#2) Trigger automated builds through CLI: Builds need to be automated to a point that they can be triggered through a CLI.

For example, you can use ANT and Maven to spin a build. This means that one should be able to connect to the build server, load the required assets from an online or local repository, compile and run the whole build with a simple Sometimes in a large build, some of the artifacts would have already been downloaded as part of a previous build. Therefore, the build tool must be able to gauge and download only the needed resources.

A build may successfully compile but that doesn’t mean it would be suitable for testing. Therefore, it’s also important to have some tests incorporated as part of the build, in order to discover any obvious bugs early.

#3) Frequent code-commits: The beauty of this system is that even with multiple developers, code conflicts are easily caught. This is because each developer has to update his local copy before he can commit. If this is done daily or as often as possible: The more commits = the more updated local copy is with the mainline. The more updated the local copy, fewer conflicts. Fewer conflicts = stability!

#4) Quick Build time: A longer build time defies the whole purpose of Continuous Integration because it won’t be possible to get ongoing fast feedback. Secondly, frequent code commits will become increasingly difficult. How do we combat this? This takes us to the next best practice of having Staged builds.

#5) Staging builds: In order to expedite the build process, the build pipeline could be broken down into smaller chunks and executed in parallel.

#6) Run mainline build on integration machine: In the Continuous Integration process, we had talked about running a second build pertaining to the mainline code. This build happens on an integration machine. You might wonder why? As the testers, we encounter situations where bugs are seen only on a particular environment and not in another. This is exactly the reason a mainline build is run on an integration machine.

Sometimes integration machine will spring up surprises that didn’t exist in a developer’s local system. Human errors such as not synching your code with the mainline will also show up here. Therefore only once it builds successfully here, the commit can be declared

#7) Create a duplicate of Production System: As the testers, we’re so familiar with environment-related defects. Production systems have their own configurations in terms of database levels, Operating system, OS patches, libraries, networking, storage, etc.

It is a good practice to have the test environment as close as possible to the production system, if not an exact replica. Any major discrepancies and risks can be easily identified this way before it actually hits production systems.

#8) Automating Deployment: In order to get to the point of running different kinds of tests in a CI model, the test setup has to be done as a prelude. As a best practice, you could have scripts that automatically setup the needed runtime environments for testing.

#9) Publish build locations: With frequent builds getting spun, it is important to make all the consumers of the build know where the latest build can be found. A repository where builds are published can be shared with all those involved. Likewise, build, and initial sanity results should also be published, so that people can see what integrations or fixes have gone in.

Conclusion

Application of CI in test organizations proves to be invaluable for its automation scope.

As we’ve seen above CI reduces testing effort with a promise of greater accuracy. This is much needed for the Continuous Delivery process as a whole.

Continuous Delivery and Continuous Integration are definitely evangelizing Agile. Adopting these procedures will take you a step closer to respond to rapidly volatile and dynamic markets.

Let us know if you have any other best practices/suggestions for continuous integration.

PREV Tutorial | NEXT Tutorial