Robot Framework for Mobile Test Automation
Robot Framework is a test automation framework for acceptance test-driven development and...
The war for “best mobile application testing framework” is still raging. Looking at the market, the predominant tool is still Appium, but native frameworks like Espresso (for Android) and XCTest/XCUITest (for iOS) are beginning to gain ground for a variety of reasons.
Looking for the best mobile application testing framework for your business case?
When deciding to start your journey towards mobile test automation, it’s important to have all of the information in front of you before choosing your automation tooling. If you (like many other teams out there) are currently trying to decide between Appium, Espresso, and XCTest / XCUITest, it’s very helpful to weigh the pros and cons of each approach. To try and make this easier, we have done the hard work for you and compiled information around what each solution is and what the pros and cons are.
XCUITest is a native framework for UI testing on iOS mobile devices. Since XCUITest is baked right into Xcode IDE, it provides the fastest possible testing for iOS devices. However, there are several limitations to be aware of in addition to the several major benefits.
Like XCTest, Espresso is a native framework for UI testing. However, instead of iOS devices, Espresso is pure Android. Espresso is baked right into the Android SDK and integrated into Android Studio. Espresso has major benefits for teams looking to develop and test on Android devices. However, there are a few limitations to keep in mind. Let’s explore the pros and cons.
There are several major pros to using Espresso for your mobile automation testing:
Appium is an open-source automation framework, or collection of client “automation libraries” as Appium calls it, used primarily to test both Android and iOS native mobile applications, as well as hybrid and web applications. Appium is part of the Selenium WebDriver family, thus utilizes Selenium WebDriver API that specifies a client-server protocol that uses vendor-provided frameworks under the hood. With no need to compile a third-party framework or code, you’ll be able to deploy the same app you have tested across cross-platform devices with confidence.
There are several major pros to using Appium for your mobile automation testing:
While we believe that teams should choose whatever framework suits them best (and we will help make Appium, Espresso, and XCTest / XCUITest better), we have focused heavily on the Appium automation engineer. This year, we started releasing what we call “AI-enhanced Appium,” and this makes your Appium automation easier to author, more comprehensive in its coverage, faster in its execution (hallelujah!), and easier to remediate for when issues are encountered.
It all starts with test authoring. With Kobiton, we’ll let you inject our NOVA AI Engine into your existing Appium automation scripts, but we can also help you get to that script faster to begin with by leveraging our auto-Appium generation or Appium Export technology.
Kobiton’s Appium Export technology allows you to run a manual test on a Kobiton real device, tell our AI to auto-generate an Appium script, and then export that script into your IDE for quick “base script” authoring. Not only is this easy, but it’s also a huge time saver for teams who are trying to quickly build out their libraries of Appium automation.
Beyond just getting to your Appium “base script,” we also make it easy to author assertions into your Appium script. We actually just released the Beta version of our “Inject NOVA Visual Assertions into your Appium” capability, and will be releasing the General Availability version in our April 19th, 2021 release. This allows you to take advantage of the Visual Assertions and Visual Testing capabilities previously unique to the Kobiton Scriptless Automation flow and easily inject them into your Appium scripts. And, unlike other solutions that do similar things to this, Kobiton allows you to do this by simply making a quick change to your Desired Capabilities and also doesn’t require setting a new baseline for each and every mobile device. With Kobiton, you can run a Visual Test using one device’s visual output (let’s say a Samsung Galaxy S9) as a baseline and compare that baseline across multiple different device types (let’s say LG and Moto G4 devices).
Yes, that means no more making API calls, no more source code instrumentation, and no more endless baseline screenshots.
After the test is authored, it’s got to execute quickly. That’s why Kobiton offers “souped-up” Appium that can run greater than two times as fast as a standard Appium execution. In addition to this, injecting NOVA into your script also allows for advanced adaptation and auto-correction capabilities that make your Appium much more reliable and stable when running in parallel across different device types.
Okay, so you’ve run NOVA AI-enhanced Appium and it’s been able to run across a bunch of different devices while also invoking Visual assertions to capture visual issues and discrepancies.
With Kobiton, you’ll actually be able to easily remediate those visual issues identified by Appium with our Assertion Overview and Remediation interface. With this interface, you can easily see where NOVA identified visual discrepancies across device types and/or application versions, and you can even give NOVA instructions on how to better execute your tests going forward with directions around the strict and relaxed comparison.
To see this all in action, check out this snippet of a "visual assertions in Appium" demo run by Leslie Tan, Global Director of Solutions Engineering at Kobiton:
There are many different test automation solutions out there, but only a handful that are truly optimized for mobile application automation. While we understand Kobiton’s offering as providing the most feature-rich, comprehensive, and flexible solution, we recognize that teams might have specific needs that fit better alongside other solutions. This is why we recommend taking a thorough look at your team’s make up, skill-set, and needs before evaluating and ultimately choosing a solution.