Building Test Platforms for Payment Reliability at PayPay Card

In PayPay Card’s payment platform, where a wide range of systems are tightly interconnected, what kinds of technical challenges does the QA team take on? In this article, Amit shares how the QA Platform Team goes beyond the traditional scope of QA engineering to support the entire development process—from the design philosophy behind the backend automation framework CSAE to the team’s work on ATLAS, an AI-powered test generation platform.

Introduction

In fintech, reliability is not a nice-to-have. It is the baseline. At PayPay Card, millions of transactions move through systems that span legacy core platforms, modern services, batch jobs, and customer-facing applications. If we want to release with confidence, we need more than individual automated tests. We need platforms that let engineering teams define, execute, and maintain testing in a consistent way across very different systems.

That is the focus of our QA Platform team. We are not a team that only runs regression suites after development is complete. We build the internal automation platforms that other engineers use to validate payment flows, investigate failures, and scale testing across the organization.

What the QA Platform Team Builds

The simplest way to describe our work is this: we build shared automation infrastructure for engineers working on payment systems.

That includes backend automation for core payment flows, browser automation for customer and internal web journeys, and internal tooling that reduces the manual work required to design and operate tests. The users of these platforms are not only QA engineers. They are also developers and teams who need reliable feedback in CI/CD and in day-to-day feature delivery.

In practice, our platform work supports several different needs:

• End-to-end automation across critical payment flows
• Cross-protocol validation for systems that do not live in a single technical stack
• Self-service automation that can be executed repeatedly through GitHub Actions and connected delivery flows
• AI-assisted workflows that reduce the time required to convert requirements into usable test assets

We sit at the intersection of platform engineering and quality engineering. The goal is not only to increase automation coverage, but to make reliable testing easier to use across an engineering organization.

Core System Automation: Why CSAE Exists

One of the central pieces of this work is our backend automation framework, the Core System Automation Engine (CSAE). It is a Spring Boot-based framework built for the parts of payment testing that are usually the hardest to standardize: systems with mixed protocols, long-running jobs, and shared state.

This is the kind of problem where a simple API test framework is not enough. A realistic end-to-end scenario may need to trigger a batch job, send an internal protocol message, wait for downstream processing, validate database state, and then confirm an API or back-office result. In payment systems, those steps often cross technical boundaries that were built at different times and for different purposes.

That is why we designed CSAE as a platform rather than as a collection of isolated test scripts. We needed one framework that could orchestrate multiple interfaces inside a single scenario while still remaining maintainable as the system evolved.

The Core Engineering Challenges

Three engineering constraints shaped the framework from the beginning.

The first is asynchronicity. Many payment flows are not completed in a single request-response cycle. Batch scheduling, delayed jobs, and downstream processing mean that the framework has to coordinate waiting, polling, and validation in a controlled way.

The second is protocol diversity. A single business flow can touch proprietary service interfaces, files transferred through secured paths, Kafka events, REST or gRPC services, and ISO 8583 financial messages. We needed a test engine that could treat those very different interfaces as part of one executable business flow.

Note: Some integration interfaces are inherited from earlier system generations and are being progressively modernized as part of ongoing platform evolution.

The third is data integrity. Parallel execution is useful for speed, but shared QA environments can create collisions in Oracle-backed data or other persistent state. If the framework cannot control this well, automation becomes noisy and teams lose trust in it.

Why We Chose an Executor-Based Architecture

To address those constraints, we built CSAE around a Strategy Pattern and an executor-based architecture. Instead of embedding every protocol concern inside one large test engine, we separated each type of interaction behind its own executor. At runtime, the framework dispatches the appropriate executor for each step in a scenario.

This decision was important for maintainability. When a new integration is introduced, we can extend the framework by adding or evolving a focused executor instead of rewriting the orchestration model. That keeps business scenarios more stable even when technical integrations change underneath them.

We also chose a YAML-driven scenario design. The reason was not only readability. We wanted test authors to describe business flows declaratively, without forcing every change through framework code. That makes the platform easier to contribute to and reduces the cost of expanding coverage as new payment scenarios appear.

How CSAE Works in Practice

CSAE includes specialized executors for the kinds of interfaces we regularly need to automate:

The point of this design is not to make the framework look modular on paper. It is to keep scenario logic focused on business intent while encapsulating protocol-specific behavior inside reusable building blocks.

Scale and Developer Impact

Today, the framework supports more than hundreds end-to-end automated scenarios for core batch processing alone, with thousands additional API and UI scenarios across the broader platform. These cover the full card lifecycle along with customer-facing journeys and internal service integration tests.

Those scenarios run daily through GitHub Actions and provide continuous regression feedback to the teams that build and operate these systems. The practical benefit for developers is straightforward: they get an earlier signal on whether a change breaks a real payment flow, not just a narrow unit of code. In a domain where failures can span multiple systems, that kind of regression confidence matters.

Engineering Decisions That Improved Reliability

Several design choices became especially important as the platform matured.

Idempotent concurrency controls were necessary because parallel execution in shared environments can easily create state collisions. We introduced locking and execution controls so that faster automation would not come at the cost of unstable results.

Observability mattered because the framework itself behaves like an internal platform. Once automation becomes part of daily delivery, teams need to understand where a failure happened: in the application, in the environment, or in the framework orchestration layer. Instrumentation and monitoring help us debug the platform as a system, not just as a test suite.

The YAML-driven model was also an engineering choice about scale. It lowers the barrier for contribution and helps teams extend scenario coverage without treating every new case as a framework development task.

Multi-profile environment support was needed because the same business flows are exercised across different connected environments. A reusable platform has to adapt to environment differences without duplicating large amounts of scenario logic.

Web Automation in Payment-autoqa-Playwright

Backend automation is only one side of the problem. We also need to validate the web journeys that connect users and operators to those backend systems.

That is where our Playwright-based automation work comes in. In Payment-autoqa-Playwright, we use Node.js, Playwright, and Cucumber to automate browser flows and connect them to our broader test-management and CI/CD process. This matters because many payment journeys are only complete when the UI, backend behavior, and supporting systems all align.

The value of the Playwright repository is not just that it automates screens. It gives us a structured way to describe user-facing behavior in plain-language scenarios, run those scenarios repeatedly in delivery pipelines, and trace them back to defined test cases. In other words, it helps us make web automation part of the same platform thinking as backend automation, rather than treating it as a separate island.

ATLAS and AI-Assisted Test Engineering

Another area we are actively building is ATLAS (AI Testing Lifecycle as a Service), our AI-assisted test authoring platform. The core value of ATLAS is that it reduces the manual effort required to transform product and technical requirements into executable test assets.

That problem is worth solving because test authoring often breaks down into repetitive handoffs: reading PRDs, translating them into plans, breaking them into cases, composing data, and finally converting that work into automation artifacts. ATLAS is meant to compress that path.

Its role is not to replace engineering judgment. Its role is to reduce the mechanical work around test creation so engineers can spend more time reviewing intent, coverage, and edge cases.

ATLAS operates as a multi-agent pipeline where specialized AI agents handle distinct lifecycle stages sequentially:

• Requirements analysis — ingesting PRDs, legacy specifications, and business references to produce structured bilingual (English/Japanese) business summaries
• System design generation — mapping requirements to interfaces, database schemas, and step-by-step flow analysis with branch-level execution paths
• Test coverage/Plan design — proposing factor-based coverage strategies with a human approval gate before the pipeline proceeds
• Automation asset generation — converting approved test cases into executable scripts compatible with our CSAE framework (on-request, one case at a time)

A key architectural decision is the approval gate: ATLAS halts after proposing a coverage strategy and resumes only after human sign-off, ensuring AI output stays anchored to engineering judgment rather than running unchecked. The pipeline also includes a feedback processor that accepts structured review input and applies targeted corrections to generated artifacts. 

Supporting the Broader Developer Experience

The QA Platform team also supports the developer experience around automation, not only the frameworks themselves.

That includes CI/CD orchestration, automated scheduling and reporting flows, internal documentation support, and platform integrations that make automation easier to discover and operate. This work is less visible than a test scenario or an AI feature, but it is part of what turns a framework into a usable internal platform.

A Brief Note on Mobile Automation

We have also recently started building our mobile automation capability, following the same platform philosophy as our backend and web work: a shared core framework with project-level scaffolding. The stack is Java-based using Appium (with UIAutomator2 for Android and XCUITest for iOS) as the automation engine, Cucumber for BDD-style scenario authoring, TestNG for execution management, and BrowserStack for cross-device cloud execution. The architecture separates concerns across the following repositories:

• A core library providing reusable capabilities — the Screen Object pattern, gesture and touch abstractions (MobileActions, Direction), platform-aware driver management (AndroidDriverFactory / IosDriverFactory), capability resolution for both local Appium and BrowserStack environments, database utilities, Allure reporting integration, and a flow orchestration layer.
• Project-specific test repositories where teams write Cucumber feature files organized by domain — covering core app flows, financial services, bill pay, mini-app journeys, and online channels

This work is still maturing compared to our backend and web platforms, so I describe it honestly as an actively growing capability rather than a fully established platform today.

What Comes Next

The direction ahead is an extension of the same idea: move testing infrastructure earlier into the engineering lifecycle and make it easier to use across different kinds of systems.

That means deeper shift-left collaboration during PRD and technical design stages, continued expansion of backend and browser automation capabilities, and tighter integration between AI-assisted authoring and executable automation assets. The long-term goal is not to accumulate more isolated tools. It is to create a connected platform that helps teams deliver reliable payment experiences with less manual friction.

Closing

What I find most meaningful about this work is that it changes the role of automation inside engineering. Instead of treating tests as something added at the end, we treat automation as shared infrastructure that supports how teams design, build, and release software.

By building platforms instead of isolated tests, we help engineering teams deliver reliable payment experiences at scale.

If you are interested in this kind of platform engineering, I would be glad to connect.