Tech Talks vol.41 – Automation QA Engineers

About Tech Talks

In the Tech Talks series, we’re excited to bring you the unique culture and perspective of the PayPay Product Team with members representing diverse backgrounds from approximately 50 countries around the world.
In this issue, we spoke with three Automation Engineers.

What is the significance of automation QA engineers at PayPay and what is their mission?

Anand:
PayPay follows a weekly rollout cycle for its products. To ensure bug-free releases, regression tests are necessary, which requires a significant amount of effort.

Until about two years ago, these regression tests were done manually without any automation. This meant that even with a large team of manual QA engineers, it would take up to a week to complete the tests.

By automating a portion of the regression tests, the time required for testing can be reduced. This allows developers to focus on fixing bugs and conducting other tests. This is why automation QA engineers play a vital and indispensable role at PayPay.

Currently, we have automated approximately 70% of the regression test cases. While this is a significant achievement, it still means that around 30% of the tests are handled by manual QA engineers, taking up three days every week. One of the missions of the QA team is to minimize the need for manual efforts. Our goal is to reduce the testing time to one or two days by the end of the year.

Another mission of the QA team is to detect bugs at an early stage. As a product progresses through its software development life cycle (SDLC), the cost of addressing bugs increases exponentially. Therefore, we strive to identify bugs earlier through trial and error, in order to reduce associated costs.

Please share specific details about your role and responsibilities.

Anand:
Currently, I am the Engineering Manager for the Payments & Mini App and QA Tech teams. My specific roles include assigning tasks within the team, managing the progress of roadmap projects, improving automation processes, establishing and tracking overall KPIs for PayPay’s automation team, participating in hiring, conducting 1 on 1s with team members, and providing feedback and evaluations.

Raymond:
I focus on testing merchant facing products. While our testing is done within PayPay’s domain, our role is slightly different from teams responsible for consumer facing features. However, our fundamental mission remains the same, which is to deliver high-quality products.

The O2O (online to offline) team follows an agile approach, allowing for rapid development. Manual testing alone is not sufficient, so we prioritize automating as many test cases as possible. My mission is to ensure the reliability of our automation testing.

Hsin:
I became the team leader in January 2024. My primary role is to support both automation and manual QA engineers in their tasks.

Previously, as the QA lead for automation, my focus was primarily on automation-related tasks. However, as a team leader, I now need to delve deeper into both automation and manual testing aspects. This presents a new challenge that I am currently navigating.

Which project left the biggest impression on you?

Anand:
All projects at PayPay have posed a challenge, but there are two in particular that have left a lasting mark.

The first project was an improvement initiative that took place when I first joined PayPay. At that time, the manual and automation QA teams had overlapping tasks and lacked effective communication.

As the lead of the improvement project, I focused on fostering understanding between the teams, as they had different processes and ways of thinking. We worked towards merging our approaches into a unified system, which took some time but ultimately resulted in the successful integration of important functionalities. I remember this project vividly because of the sense of accomplishment it brought.

The second project was aimed at reducing the instability of our automation processes. Previously, all test cases started with the login process that would take a few minutes to reach the screen to be tested, which caused instability and required painstaking maintenance. To address this, we collaborated with the developer team to modify the automation framework and introduced the concept of deep linking. Thanks to this enhancement, we were able to skip the login part and start the tests directly from the specific screen being tested. I am proud to have contributed to the automation team’s success through this project.

Raymond:
Last year, there was a project that took three months to complete, even though it was initially expected to be finished within one month. Initially, I thought it was a simple project, but during the testing process, it became apparent that collaboration with various other teams was necessary.

Some of these teams only had Japanese-speaking members, which made me anxious about overcoming the language barrier. However, once I reached out to them, I was pleasantly surprised by their willingness to communicate and provide support. Whenever I had questions, I received prompt responses over Slack. Thanks to their help, we successfully completed the project. This experience highlighted to me that at PayPay, there is a strong culture of people who are happy to lend a hand.

Hsin:
One of the most challenging projects I’ve recently completed was an agile project. The Core App team failed to deliver tasks on time, resulting in multiple extensions to the schedule. As the person responsible for automation, I had to constantly update scripts and review test designs whenever there were changes to the specifications or schedule. Unfortunately, this caused the project to take a few weeks longer than originally planned. Despite this setback, we couldn’t extend the rollout deadline. As a result, we had to overcome the challenge of deploying the changes to the production environment immediately after the regression test. Overall, it was one of the most difficult projects I’ve worked on in the recent past.

How do you approach difficult projects?

Hsin:
Working in QA, I make certain that I continuously share information with the PM and all team members, and whenever there are any changes, remind people about it so nothing is missed. That way, even if something irregular happens, I can coordinate with the PM to change its scope to Day 2, for example, so the team can continue what they’re doing without any hiccups. Active communication with stakeholders is very important.

Anand:
As Engineering Manager, I’m currently taking on a new challenge of ensuring that all projects are completed on schedule. The Payments & Mini App and QA Tech teams that I look after work by differing roadmaps and strategies, so ensuring that each team completes projects as planned is a huge challenge.

To accomplish this, I make sure I communicate closely with team members as well as share information with the entire QA team whenever we’re doing something new. Having a Plan B and C for every project is also a big challenge, but through engaging in various different projects I’ve come to realize the importance of always having an alternative plan.

What do you enjoy about being an automation QA engineer at PayPay?

Hsin:
As an automation QA engineer at PayPay, I find great joy in seeing the results of the updates I have made and witnessing the increase in test coverage through the creation of new scenarios. It has been especially thrilling for me, as this is the first time working as an automation QA engineer at PayPay.

Once I became the automation QA lead, my perspective shifted. I began thinking about how I could improve the performance of the entire automation QA team, not just my own team. I strongly believe that the QA team can make a significant contribution to the company when both the automation and manual teams work together. We are currently in the process of working out the details, which is quite exciting, and I see it as an opportunity for personal growth as well.

Raymond:
Normally, in many companies, the manual QA team solely focuses on manual testing, while the automation QA team is responsible for automation testing. However, at PayPay, we have the opportunity to do both manual and corresponding automation tests. This aspect of my role is incredibly rewarding because it allows me to contribute not just to the automation QA team, but also to the manual QA team.

Since becoming the automation QA lead, I have discovered the satisfaction of making improvements at the team level. When I witness the growth and recognition of my team members, it brings me happiness, I find greater fulfillment in celebrating the achievements of my team members more than my own.

Moving forward, my goal is to help the team reach a higher level of autonomy, where things can run smoothly without constant instructions from me. While my current role is more managerial, I also enjoy getting involved in hands-on technical tasks. Therefore, I strive to maintain a balance between these responsibilities.

At PayPay, there are no rigidly defined roles and positions. This means that you are not confined to a specific role or task. Instead, you have the freedom to explore new opportunities and pursue different areas according to your aspirations. PayPay provides a diverse and inclusive workplace that allows individuals to choose their desired path.

What future do you envision for QA at PayPay?

Anand:
QA plays a critical role in every software engineering project and SDLC. By enabling the automation QA engineer to identify bugs at an early stage, we can greatly enhance the overall quality of our app. My aim is to contribute to early bug detection, thereby improving quality, increasing revenue, and promote efficient workflows.

What are your goals or aspirations for the future?

Hsin:
urrently, I am responsible for automations at PayPay Securities. I joined the newly formed QA team at PayPay Securities and am currently focused on building the automation framework, which will serve as the foundation for running automated tests. Although this is a significant challenge, I am committed to achieving success.

On the PayPay side, my mission is to support both the manual and automation QA teams. Currently, there is a gap in information between these two sides. I will work towards bridging this gap and eventually eradicating the distinction between manual and automated testing. We are one team with different roles, and I aim to establish this mindset.

Can you share what makes being an automation QA engineer great, along with messages for our readers?

Hsin:
As Raymond mentioned, PayPay offers a wide range of options and an environment where you can take on various challenges if you wish. Additionally, PayPay is a company that is constantly evolving, so you must always be open to learning new things. If you have any ideas, you can pitch them internally and contribute in that way as well. So, if you have many ideas and want to try something new, PayPay is a great choice!

Raymond:
I agree with Hsin, and I would like to add that at PayPay, anyone can propose new ideas regardless of their position or tenure in the company. There are also opportunities to get involved in large projects. Your contributions are evaluated, which is highly satisfying.

Another positive aspect is that you’d have the opportunity to work with people from all over the world. I’m from Indonesia, and since joining PayPay, I’ve had the chance to collaborate with colleagues from Taiwan, Russia, Nepal, and other countries. This kind of experience is rare to come by. PayPay is a company with a global mindset, combined with the best aspects of Japanese culture, such as maintaining courtesy; it makes for a very pleasant working environment!

Lastly, a message for our readers, please!

Anand:
PayPay has a culture that places a high value on teamwork and mutual respect. If you are seeking an environment that fosters continuous learning and personal growth, then this is the right place for you. Here, everyone is encouraged to share their ideas, and they can count on their colleagues for support. We welcome individuals who are eager to embrace new challenges. If this sounds like you, we look forward to you joining our team!

*Job openings and employee affiliations are current as of the time of the interview.