Test Automation

    Test automation services

    Manual regression testing slows every release and still misses things. Agnotic Technologies builds test automation your team can trust: the right tests at the right layer, wired into CI as quality gates, with flaky tests under control and a suite your team owns.

    Web, mobile and APICI quality gatesFlaky test controlSuite handed over

    Trusted by global innovators

    Benchmark
    Chibasco
    Fundency
    Lantimer
    Lauren
    Lera
    One Minute
    Pento Pix
    TAP
    Xtrium
    Healthevolve
    Benchmark
    Chibasco
    Fundency
    Lantimer
    Lauren
    Lera
    One Minute
    Pento Pix
    TAP
    Xtrium
    Healthevolve
    Benchmark
    Chibasco
    Fundency
    Lantimer
    Lauren
    Lera
    One Minute
    Pento Pix
    TAP
    Xtrium
    Healthevolve
    Benchmark
    Chibasco
    Fundency
    Lantimer
    Lauren
    Lera
    One Minute
    Pento Pix
    TAP
    Xtrium
    Healthevolve

    QA automation

    What is test automation, and what do we build?

    Test automation is replacing repeated manual checks with code that tests your product on every change. Agnotic Technologies builds QA automation for web, mobile and API products: we choose what to automate first, pick frameworks per platform, wire tests into CI as quality gates, control flaky tests and hand the suite to your team.

    Automation does not replace testers. It takes over the checks that repeat every release, so people can spend their time on exploratory testing, new features and the cases a script cannot judge.

    Our wider QA services, including manual testing, are on the QA and testing page. Security and penetration testing are a separate service.

    Where to start

    What should you automate first?

    The order we follow on most products, so the first gates pay off quickly.

    1. Rank flows by risk and change

      We list user flows and business rules by what breaks most often and what costs most when it does.

    2. Start below the interface

      API tests for the riskiest rules come first, because they are fast, stable and cheap to maintain.

    3. Add a small UI smoke suite

      A handful of end-to-end journeys that earn money or keep users safe, such as sign-up, checkout or booking.

    4. Gate merges

      Unit and API tests block a merge when they fail; UI tests block a release.

    5. Grow UI coverage where manual work repeats

      We automate further only where your team repeats the same manual checks every release.

    6. Keep judgement manual

      Exploratory testing, visual review and usability stay with people.

    Decision table

    Which test layer catches what, and which CI gate does it feed?

    Each layer has a job. Putting a check at the lowest layer that can catch it keeps the suite fast.

    Test layerWhat to automateTypical tool familyCI gate it feeds
    UnitBusiness rules, calculations and edge casesThe language's own test runnerEvery commit; blocks the merge
    API and contractEndpoints, access per role, response schemas and contracts between servicesHTTP test clients and consumer-driven contract toolsEvery pull request; blocks the merge
    Web UIA few critical journeys, end to endPlaywright, Cypress or Selenium WebDriverDeploy to staging; blocks the release
    Mobile deviceSign-in, core flows, permissions and offline behaviourXCUITest, Espresso or Appium on real and virtual devicesNightly and before store submission
    AccessibilityThe WCAG 2.2 checks that rules can detectRule-based scanners run inside UI testsWarns on pull requests; blocks new failures at release
    PerformanceResponse times of key endpoints under expected loadLoad testing tools against a staging stackBefore major releases, not every commit

    Automated accessibility checks catch only part of WCAG. We pair them with manual checks before major releases.

    End to end testing

    How do end-to-end tests cover journeys that cross other systems?

    End to end testing of journeys that cross other systems works when the suite stays small and controlled: a few high-stakes journeys, partner sandboxes where they exist, stubs only at the outer boundary, fresh data on both sides for every run and the same journeys checked again in production. Agnotic Technologies builds them in that order.

    01

    Pick the few journeys that matter

    • Journeys where a failure loses revenue or harms a user, such as a payment, an identity check or a record sent to a partner
    • Each journey traced across every system it touches
    • Everything else stays at the unit, API and contract layers
    02

    Use each partner's sandbox

    • Payment, identity and EHR vendors often provide test environments with test accounts
    • Sandbox limits, such as rate limits and reset schedules, written into the test plan
    • A sandbox outage marks the run as blocked, not as a product failure
    03

    Stub only at the outer boundary

    • Your own services run for real, from the first screen to the last write
    • A stub replaces a partner only when it has no sandbox, or when an action cannot be undone, such as a real payout
    • Stubs replay recorded partner responses, so they stay close to the real thing
    04

    Seed and reset data on both sides

    • Matching records created in your system and in the partner sandbox before each run
    • Every record tagged with the run ID, so cleanup finds it afterwards
    • No run depends on data left behind by an earlier one
    05

    Wait on events, not timers

    • Webhooks, queue messages and emails awaited by checking for the event itself
    • Each wait has a timeout that names the system that did not answer
    06

    Run the same journeys in production

    • The journeys rerun as synthetic checks against production on a schedule
    • Synthetic accounts are flagged so they never count as real billing, analytics or partner activity
    • A failed check alerts the on-call engineer with the step that broke

    Which checks should stay manual, and which should be automated?

    Manual vs automated testing is decided per test type. Exploratory and usability testing stay manual; smoke, regression and data-driven checks are automated; accessibility, cross-browser and performance testing need both. Agnotic Technologies marks every check with that decision and its reason, and automates a manual check only once it is stable and repeats.

    Exploratory testing: manual

    A tester follows hunches into new paths. A script only repeats what someone already thought of.

    Usability testing: manual

    Whether a person understands a screen is a judgement only people can make.

    Accessibility with assistive technology: both

    Rule checks run in the suite; screen reader and keyboard testing against WCAG 2.2 is done by a person.

    Smoke tests: automated

    A short set that runs on every deploy, where speed matters more than depth.

    Regression testing: automated

    The same checks every release are where manual effort repeats the most.

    Data-driven tests: automated

    Many input combinations through one flow are slow and easy to get wrong by hand.

    Cross-browser testing: both

    Functional checks run on each browser automatically; a person reviews layout on the browsers your users use most.

    Performance testing: both

    Tools generate the load; an engineer reads the results and decides what is acceptable.

    Frameworks

    Which test framework fits each platform?

    We pick per platform and per team, and keep the number of frameworks small.

    Web

    • Playwright: several browsers, auto-waiting and traces for debugging
    • Cypress: runs in the browser, suits front-end teams
    • Selenium WebDriver: the W3C standard protocol, widest language choice

    iOS

    • XCUITest in Xcode for native iOS apps

    Android

    • Espresso for native Android, with built-in synchronization
    • UI Automator for flows that leave the app

    Cross-platform mobile

    • Appium when one suite must cover iOS and Android
    • Framework test tools for React Native or Flutter apps

    API

    • Code-based HTTP tests in the service's own language
    • Contract tests between services
    • Schema checks against your OpenAPI definition

    Codeless or scripted

    Codeless or scripted test automation, and when should you replace Selenium?

    Codeless vs scripted testing automation is a choice about who owns the tests. Codeless tools let testers record and edit tests visually; scripted frameworks keep tests as code in your repository, reviewed like product code. Agnotic Technologies usually recommends scripted tests for long-lived products, and replaces a Selenium suite only when its upkeep blocks releases.

    CriterionCodeless toolScripted framework
    Who writes and maintains testsTesters and analysts without coding skills, in the tool's editorEngineers and test engineers, in the same languages as the product
    Where tests are storedIn the tool's platform, in its own formatAs code in your repository, next to the product code
    Code reviewChanges reviewed inside the tool, where it supports reviewsPull requests with the same review rules as product code
    Complex data and logicLimited to the steps and plug-ins the tool offersAnything the language can express, including test data set up through APIs
    CI integrationThrough the tool's own runner or APIRuns in your CI system like any other test job
    Leaving the toolExport options differ by tool, so check them before you commitThe tests stay in your repository and run on open source runners

    Moving a Selenium WebDriver suite happens in four stages: (1) new tests go into the new framework, (2) the smoke suite is ported first, (3) both suites run in CI until their results match, (4) the old runner is retired.

    Flaky tests

    How do we keep flaky tests under control?

    A suite people ignore is worse than no suite. These are the causes we design out.

    Challenge

    A test fails, then passes on retry.

    Agnotic approach

    It is marked flaky, moved to quarantine with a named owner and a fix date, and does not block releases while it is there.

    Challenge

    Tests wait with fixed sleeps.

    Agnotic approach

    Tests wait for the condition they need, using the framework's own waiting, never a fixed delay.

    Challenge

    Tests share data and break each other.

    Agnotic approach

    Each test creates its own data through the API and cleans up after itself.

    Challenge

    Third-party services are slow or down.

    Agnotic approach

    Integration tests use stubs for outside services; a small separate suite checks the real connections.

    Challenge

    Tests pass only in a certain order.

    Agnotic approach

    Tests run in parallel and in random order in CI, so hidden dependencies show up early.

    Challenge

    UI tests break on every design change.

    Agnotic approach

    Locators use roles, labels and test IDs instead of layout, so a restyle does not break the suite.

    Hand-over

    Who owns the test suite after hand-over?

    Your team does. The hand-over includes everything they need to keep it healthy.

    • The suite lives in your repositories and runs in your CI system, not in a tool only we can access
    • A short guide to writing new tests, naming, test data and where each type of test belongs
    • An ownership map that links each area of the suite to the team that owns that part of the product
    • A flaky test report and the quarantine process, so problems stay visible
    • Synthetic test data scripts, so no production data is ever needed in tests
    • Pairing sessions with your engineers, and the option of ongoing support from us if you want it

    Who it is for

    Which teams ask us for test automation?

    Teams where release confidence is the bottleneck.

    QA automation for startups

    Products past their MVP, where a small team can no longer test every release by hand.

    Teams with long manual regression cycles

    Releases that wait days for manual regression testing to finish.

    Teams with an old Selenium suite

    Suites that nobody trusts any more, migrated or rebuilt with the tests worth keeping.

    Mobile teams shipping to app stores

    Device tests that run before every store submission.

    Regulated product teams

    Teams that need traceable, repeatable test evidence for each release.

    Standards

    Standards our test automation follows

    Open standards behind the tools and checks we use.

    W3C WebDriverWCAG 2.2ISO/IEC/IEEE 29119OpenAPI
    Facts and sources

    Facts behind test automation

    Tool documentation and standards that shape a test suite, each linked to its source.

    Scope and cost

    What drives the cost of QA automation?

    There is no price list. The size of test automation work comes from the factors below, and the quote follows the scoping call.

    Platforms

    Web, iOS, Android and APIs, and how many browsers and devices each needs.

    Flows in scope

    How many journeys and business rules the first gates must cover.

    Test data

    How hard it is to create realistic test data through your APIs.

    Environments

    Whether a stable test environment exists or has to be built first.

    CI system

    Which CI system you use and how much pipeline work the gates need.

    Existing suite

    Whether we start fresh or migrate and repair an existing suite.

    Engagement

    How does a test automation engagement work?

    From the first gates to a suite your team runs alone.

    1.

    Review and plan

    We review your current testing, rank flows by risk and agree the first gates.

    Plan
    2.

    Build the first gates

    API and smoke tests in CI, test data scripts and the flaky test process, running on every change.

    Build
    3.

    Grow and hand over

    Coverage grows where manual work repeats, and your team takes over with our support if needed.

    Hand over

    Frequently Asked Questions

    It is quoted per engagement. The main factors are the platforms, the flows the first gates cover, how test data is created, your environments and CI system, and whether an old suite is migrated.