Sauce Labs review
Ranked #4 / 6 in Browser testing
— Last updated: 2021-12-14
Sauce Labs has a global presence and is ideal for large businesses. It provides a large number of devices and browsers and it is also possible to purchase private devices for testing. However, it has only a limited number of integrations and suffers from high latency during live testing.Visit Sauce Labs
Sauce Labs offers plans starting at $49/month for manual testing and $199/month and $249/month for automated virtual device and real device testing.
Device & browser support4.5/5
Sauce Labs provides cloud infrastructure for testing on thousands of desktop and mobile browser/OS combinations around the world.
Desktop testing can be performed on multiple versions of Windows, macOS, and Linux operating systems. They claim to deliver new versions within 48 hours of release, however, when we checked (2 months after launch), Windows 11 and macOS 12 were still not available.
Mobile testing can be performed on emulators as well as 240+ real Android and iOS devices. Recently launched “Google Pixel 6” and “iPhone 13” are also available. On desktop, users can choose between many versions of Chrome, Firefox, Safari, and Edge; but on mobile devices, users are stuck with pre-installed browsers, i.e. Chrome on Android and Safari on iOS devices.
It is possible to set up, manage, and view test results from within a continuous integration server. CI/CD tools like Bamboo, Bitbucket, Jenkins, TeamCity, Travis CI, and CircleCI can be integrated directly.
The saucectl CLI is a framework agnostic tool for orchestrating tests via a command-line interface. A REST API is also available which allows downloading, updating, and deleting tests and reports.
Debugging & error reporting4/5
As a developer, the availability of the browser console logs and network call logs helps to reduce time spent analyzing root causes. This helps to gain insight into network requests and browser performance. All the commands issued during the test can be reviewed in test reports available in the dashboard. It also shows a video recording of the test, test logs, and test metadata. Long-term trends are presented as graphs in the dashboard to visualize pass/fail rate, number of errors etc.
Test results can be posted to Slack, integrated into other systems via a webhook or shared directly via a link. Reports shared via a link can be set to be viewed by public, private, team audiences. Any bugs discovered in the report can be reported to Jira directly from the dashboard.
What we like
Sauce Labs supports multiple testing frameworks (Selenium, Cyprus, Playwright, Puppeteer etc.) for automated testing. Sample scripts (in multiple languages) for supported frameworks are available on GitHub. This is helpful for novice users and speeds up the setup process for veterans. Visual component testing can be performed using Sauce Visual which tests React, Vue, and Angular components across different browsers and resolutions. This makes it possible to test UI components in isolation and enables components to be tested individually and in mocked states. Enterprise customers can perform functional testing on headless browsers to speed up automated testing of non UI elements.
Sauce Labs provides “Sauce Connect Proxy” (a proxy server) and “Sauce IPSec Proxy” (an IPsec VPN) which securely connects web applications hosted on a private network (or local machine) to the Sauce Labs cloud network - this network can be used by virtual machines and real devices for testing. Compared to the traditional approach of whitelisting certain IP address ranges, this solution provides time-based access controls. It is possible to connect several tunnels at once and group them together to be used as a single high availability connection.
It also provides the tester the ability to distribute the test traffic (using round-robin load-balancing) between different tunnels. This feature is unique to Sauce Labs and not something we saw with the other browser testing tools we reviewed.
There are many reasons one would use Sauce Labs over other tools. The following features set this tool apart:
- Availability of thousands of browser configurations.
- Option to buy private devices for testing.
- Framework independent CLI tool.
Many interesting device vitals such as internet traffic, number of process threads, and CPU/Memory usage are available with live testing. Having access to these device metrics alongside detailed device logs is useful when debugging. It is possible to send and retrieve text snippets to and from the clipboard on virtual machines as well as to share Live testing sessions in real-time. Neither BrowserStack nor CrossBrowserTesting provides device vitals, if this feature is important for your use case then Sauce Labs may be the only option.
Performance metrics such as Time to First Meaningful Paint, Time to First Interactive, Page Weight, Speed Index etc. can be captured to analyze performance bottlenecks.
Sauce Insights is an AI-assisted failure analysis tool that looks for patterns, for example, specific Selenium commands in a test suite that frequently fail.
Sauce Labs provides a powerful API and almost anything which is possible via the dashboard can be performed via the API. The saucectl CLI can be used to access the tests written with one of the supported frameworks (Cypress, Playwright, TestCafe, Puppeteer, Espresso, XCUITest) and run them. This CLI can be configured to work with Jetbrains IDEs and Visual Studio Code.
What we don’t like
Sauce labs is one of the oldest browser testing tools but it provides integrations only with other well-established tools and services. Jenkins, Travis CI, CircleCI, etc. are supported but lesser-known tools like Spinnaker and Buildbot can not be used with Sauce Labs. Similarly, for ticketing systems, Jira is supported but BugZilla and MantisBT are not. This makes working with unsupported tools harder than it needs to be and might limit the number of tools you can work with in the future. All the Sauce Labs alternatives can be integrated with Slack in one way or another, but Sauce Labs does not yet offer this - it’s coming soon.
We found that scrolling and typing actions during live testing is difficult due to high network latency; when playing videos, the frame rate is very poor. Users have to regularly wait 1-2 seconds after each input to see changes on the screen, this makes the testing process slow and irritating. On mobile devices, each device only has one browser available (Safari on iOS and Chrome on Android). If having multiple browsers on mobile devices is a strong necessity, then we suggest using BrowserStack.
Sauce Labs claims to reset devices after testing concludes, but during our usage, we encountered some issues which might contradict this. e.g. we were testing in English but the devices defaulted to Arabic. Restarting the test fixed this. This may indicate that devices are not wiped between tests or settings are not consistent between devices. If the latter statement is true, then it might make reproducing bugs difficult.
Sauce Labs has data centers in four global locations (US West, EU Central, US-East, APAC Southeast (BETA)), each of these is independent from the other and the user must use different API endpoints and URLs for each case. Even the dashboard is different for each location and tests are segregated accordingly, i.e. tests executed in one location can only be viewed on its respective dashboard. Although this approach is unique, it adds unwanted complexity. While some users may enjoy additional control, it provided no clear benefit to us and only added confusion. Other tools use a unified dashboard and allow changing location in test configurations, which in our opinion is a better approach. LambdaTest even allows changing network locations during live testing and makes it easy to test geographical features.
In July 2021 , Sauce Labs acquired Backtrace which contains most of the advanced debugging features. Features such as data redaction, querying, sorting and grouping by the captured log attributes are bundled into an independent service which needs to be purchased separately. BrowserStack provides some of these features (like data redaction) at no additional cost. We hope to see these services merge functionality over time, but for now they are essentially separate tools.
Console developer perspective
Browser testing tool comparison
In our review of the best browser testing tools for developers, we highlight the best 6 browser testing tools for developers in 2022. See how they compare to Sauce Labs in the table below.
Sauce Labs alternatives
Interesting tools by email
Our free weekly newsletter picks out the 2-3 most interesting tools. See the latest email.