Event Report! Trial & Error in Balancing Speed and Quality in the PayPay App

On June 7, 2022, WWDC Extended was held jointly by LINE Corporation, PayPay Corporation, ZOZO, Inc., and Yahoo Japan Corporation.
WWDC Extended is an event held online to further enjoy the keynote sessions of WWDC, and this year, iOS engineers from LINE, PayPay, and ZOZO took the stage for the LT.

Naoya (PONTA) from PayPay participated and spoke on the topic of “Trial & Error in Balancing Speed and Quality in the PayPay App.”

This article introduces the content of the day’s LT.
The event was held for a Japanese audience, so the materials are provided in Japanese only.

PayPay’s Development Cycle

PayPay has been churning out new releases once a week since the service was launched. As a latecomer (relatively speaking) to the cashless service market, PayPay constantly work with a sense of speed.

Having said that, the emphasis on speed should not result in frequent incidents and glitches, so PayPay has gone through numerous trial-and-error processes to achieve both speed and quality.

See the image below for the development, testing, and release cycle.

For example, if blue is version 1 and yellow is version 2, we start beta testing version 1 on Monday, finish on Friday, and if there are no problems, submit for public release the following Monday. At the same time, we start beta testing version 2 on the same day.

This cycle is repeated every week. Within this cycle, here are some of the measures we have implemented to ensure speed and quality.

Beta Testing With Hundreds of Employees

First, let me introduce to you our large-scale internal beta testing.

In the past, once QA testing was completed, we used TestFlight’s internal tester function to have development team members and PMs check the version before submitting.

However, this was a challenge because the number of test subjects was quite small. For our part, we wanted to reduce the risk of release as much as possible, so we created a mechanism to have several hundred employees beta test using TestFlight’s external tester function.

For internal testing, it is not suitable for large groups because you have to register an account with App Store Connect and thereby grant extra privileges. In the case of external testers, it is possible to grant them only minimal privileges – that is, they can only do beta testing.

We also have an internal Slack channel for feedback so that if members do a beta test and have any issues or comments, they can feel free to post it. If something is posted on that channel, the developer or PM will give a response. We also try to reply quickly, because if we don’t get a response or it is slow to come, we can’t keep the wheels turning.

In the internal development process, there is a certain bias, and sometimes we overlook areas that we would normally be able to notice. By having people who are not involved in the project use the software, we are able to get feedback that is closer to that of actual users.

If there is a relatively major new feature, we also use Slack to announce the content and ask for careful testing.

Monitoring Negative Feedback Posted on Social Media Using IFTTT

Next, I would like to discuss the monitoring of negative feedback.

Until now, we’ve been slow to detect problems that are not caught via monitoring after release. It’s difficult to discover front-end specific problems.

Crashes, API errors, etc. can be detected with the help of various standard monitoring services that make it relatively easy to find problems when they occur. But it is difficult to detect problems such as broken screens or incorrect screen transitions.

That’s why PayPay operates a monitoring mechanism for social media using IFTTT, which is a service that allows web services to be combined like blocks, and PayPay is using it to link Twitter and Slack. For example, we post a list of negative feedback tweets caught by negative search queries such as “bug,” “glitch,” “strange,” and “can’t” to a Slack channel.

Depending on the characteristics of the service being operated, it’s often the case that irrelevant tweets are picked up, so adjustments, such as filtering the search query, are necessary. In actual operation, there have been several cases where we were able to detect problems quickly because certain people were tweeting when relatively critical problems occurred.

Introducing FutureFlag, Allowing Us to Roll Back Features

Now let’s talk about adopting FutureFlag. Until now, if a major bug was found, it was necessary to submit an application and wait for review before a fix or rollback could be made. In some cases, the nature of financial services does not allow for this, so we had to make changes dynamically from the outside.

We then introduced a future flag mechanism that can be turned on or off for each feature so that it can be rolled back. The list of flags is retrieved at the time the application is launched. This mechanism allows us to roll back features as soon as problems are detected. Not only can the flags be simply turned on or off, but they also can be configured as to which OS (iOS or Android) and which version of the OS to enable.

In terms of implementation, it can be replaced at the VC (View Controller) level, allowing for rollback with low risk. Basically, we try to use this flag for major new features or when screens are redesigned.

Message
PayPay is currently actively recruiting iOS engineers. We currently have dozens of iOS engineers and a diverse team with members from over 10 countries, and we all look forward to working with you to build the PayPay app, which was ranked #1 in the App Store Award 2021 Top Apps!

*The recruitment status is current at the time of the interview.

Author: Naoya Yamamoto / Editor: Misaki
*Employees’ affiliations are as of the time of the interview.