Applitools Eyes, an easy-to-integrate automated visual UI testing solution, was used by Wix.com‘s Test Automation Team for inspecting aspects in their HTML 5 Website Editor and generated sites. Benefiting from Applitools Eyes algorithm that mimics the human eye, the Wix team was able to reduce the number of regressions & UI bugs as well as cut down manual testing efforts.
Wix Automation team is using several test solutions, written in Java, JS, angularJS, using Selenium test framework, and triggered by Team City CI server, for testing products that are built using a variety of coding languages.
The Wix platform’s core product – its code-free HTML5 Website Editor – is a complex and feature-rich solution, enabling users to create a beautiful and dynamic web presence themselves. It includes thousands of commands and features, all of which are being tested daily by the company’s QA and Automation departments.
When the company’s Automation team realized the necessity of adding frontend visual UI testing, providing a screenshot comparison of the app’s viewer side, to the overall set of tests that covers Wix’s web products, a visual UI testing solution was developed in house. The goal was to enable QA to compare “before and after” screenshots to easily detect regressions and visual defects in the Editor.
However this solution didn’t answer the company’s needs. Tests often created “false positives”: any pixel change – even those undetectable by human eyes – caused tests to fail. As a result QA had to deal with not only real bugs and regressions, but also with tests that failed for no meaningful reason.
The Wix Automation team decided to implement Applitools Eyes which automatically validates all UI elements across devices, browsers and form factors. Applitools Eyes differs from other screenshot comparison tools due to its unique image matching algorithm that mimics the human eye: this means that tests are robust, do not generate “false positives” and are easy to create, maintain, and update.
Implementing Applitools Eyes in Wix’s existing Selenium test automation framework and TeamCity Build Server was quick and within days the Automation team was utilizing it to its full extent. Wix’s teams quickly realized the value of Applitools Eyes’ robust, automated test suite: it saved valuable resources by automatically detecting UI, layout and functionality issues that needed attention, while ignoring those that are insignificant to human users.
Two features that proved most valuable to Wix are “Ignore Regions”, allowing to label regions to ignore (usually where dynamic data is placed), and “Floating Regions” (used to mark areas that can move within a page). Regions placed on any UI page are automatically applied to other UI pages that include similar elements thereby substantially reducing maintenance times.
Furthermore, when Wix makes a change in the Editor that affects multiple pages in the same way (e.g. change in navigation), they only need to review and accept only a single occurrence of this change; Applitools Eyes scans all other pages and automatically approves similar changes.
Following the addition of Applitools Eyes Automated Visual UI Testing Solution to Wix Selenium-based Test Automation infrastructure, automated visual UI testing is now a daily part of the Wix test suite. This substantially improved QA ability to catch regressions & UI bugs, while radically cutting manual testing efforts & development resources.
With Applitools Eyes automated visual UI testing solution, Wix managed to substantially reduce the amount of UI regression bugs, while drastically reducing manual testing efforts. In addition, Wix reported substantial time saving for all departments involved: developers, test engineers and DevOps.
Manual regression cycles are now leaner and more agile allowing the QA team to focus on deep functionality and progression testing instead of acceptance and visual UI testing.
