Developer Sandbox

At Button, we practice a philosophy called “omotenashi.” It’s most commonly used in Japan’s travel industry to refer to extreme hospitality. However, when applied to building software it’s stretched to include anticipating your users’ desires and needs. Omotenashi is the basis of our Contextual Commerce product — to provide consumers with what they want and need at the exact right moment.

In the Summer of 2017, our product and design teams took stock of areas where we could provide more omotenashi to our customers. After conversations with Sales, Partner Success, and Partner Engineering it was clear that we needed to make it easier for customers to test their integration.


Any action on Button today is “live” which isn’t ideal when Merchants and Publishers are integrating and setting up. It also impacts Merchants who perform ongoing testing.

  • In many cases, Publishers are forced to spend real money to test integrations.

  • When Merchants test, their transactions incur real commissions. This pollutes the data and makes distinguishing real payments challenging.

This challenge impacts almost every piece of our technology stack. Dashboard, SDK, API, developer documentation, guides, Merchant and Publisher test apps.


Together, our product team sat down and outlined the goals we wanted to accomplish with the new developer sandbox. They included:

  • Enable Merchants to place test orders that are visible in the Dashboard for confirmation, but do not incur commission cost.

  • Separate test and live orders in the Dashboard, API, and Webhooks.

  • Enable Publishers to test orders without spending real money.

  • Testing doesn’t impact more than one partner.

  • Reduce the integration time for partner developers.


Using the feature to onboard all users

Being a pain point in the integration process, we wanted to get this out quickly. Instead of using Sandbox to onboard all partners into the Dashboard, we’ll only turn the feature on for a few partners. Those users will see “new feature” onboarding.

Ability to test with real partners

It’d be ideal to allow Publishers and Merchants to test with real partners on each side of the marketplace. However, this would require more engineering work on our side and additional permissions from each organization.


Design isn’t normally involved in the integration process so the main focus is to learn from those folks who are on the front lines. I put together a list of questions to ask our Partner Integrations team.

  • Walk me through the last integration with which you assisted.

  • What’s the biggest hurdle for a developer in our current integration?

  • How did the developer learn about Button?

  • At what point in the sales process was the developed engaged?

  • Tell me about the last time a developer tried to integrate and failed. What was your response.

From there, I reached out to several contacts at both Publisher and Merchant companies to understand their needs.

  • What did the process look like from sales to integration?

  • What was your first experience using the dashboard?

  • Who was involved in the integration process?

  • How would you describe your integration process?

  • What would’ve been most helpful during integration?

  • Does your engineering team use separate environmental builds like “staging?”

I took my findings and presented it to our product team as well as a larger group of observers and team members.

Surprised by:

I was most surprised by the number of requests we’ve had for this feature. Sid, who leads our Partner Integrations team, said every technical kickoff call he’s had has seen developers ask about testing.

More confident about:

Every partner organization uses a similar development structure (staging, production) and thought of Sandbox as a “pre-launch” process. That really helped us make the decision to implement Sandbox as mode rather than a feature.


I learned a ton through this research and gained a lot of appreciation and empathy for our Partner Integrations team. However the most interesting thing was seeing how existing partners were solving for not being able to test. One Publisher was even placing cheap hotel reservations in Cambodia!

Three solutions informed by past integrations


Present the Sandbox as a separate feature

Pros: Create a space just for developers, easily scalable patterns

Cons: No marketing means no links testing, navigation doesn’t make it easy to discover

Button Sandbox separate feature

The screen a user might see after they select Sandbox from the Publisher dropdown.

Present Sandbox information inline with existing features

Pros: Create a space just for developers, easily scalable patterns

Cons: No marketing means no links testing, navigation doesn’t make it easy to discover

Button Sandbox inline feature

The inline introduced duplicate information that made the interface confusing and crowded.

Present the Sandbox as a separate feature

Pros: Create a space just for developers, easily scalable patterns

Cons: No marketing means no links testing, navigation doesn’t make it easy to discover

Steered by our research, we made the decision to make Sandbox a mode of the Dashboard. It builds on the user’s familiarity with the existing interface in a way the other two don’t and made the path for self-serve approvals really clear. The negatives identified turned out to not matter a great deal to the users we interviewed. All features would still be accessible when not in Sandbox mode. Plus, this portion of the integration is done by developers so it makes sense to use a paradigm with which they’re familiar.

Button Sandbox mode



In order to create the sense of a global testing “mode” in the Dashboard, we needed display that choice at the highest level of the interface, the navigation. Looking at other dashboards in the affiliate and payments space, we narrowed in on a switch or toggle component. We mocked up different variations of this, specifically focusing on the language used.

The components below were tested first with our internal engineering team and then with engineers from Merchant and Publisher partners who’d be using this feature.

Button Sandbox toggles

Iterations on top of iterations. The UI outlined in blue tested best with both our internal and external developers.

Button Sandbox navigation

Different states of the Sandbox toggle in various screen widths.


Developers were really familiar with Sandbox environments in 3rd-party dashboards. In fact, our Partner Engineering lead said that most developers specifically asked about it. With that in mind, we felt it was best to keep feature-level onboarding simple and obvious.

We went back and forth on what the call to action should be in the onboarding. Knowing that most unsuccessful integrations were due to developers not reading the docs we launched with “View docs.”

Button Sandbox onboarding

Dialogue triggered when hovering over the Sandbox mode toggle.

Button Sandbox modal

Onboarding dialogue that’s triggered when a user switches Sandbox mode on for the first 3 times.

Button Sandbox icons


One of the key benefits of using a global Sandbox mode versus a separate feature or inline solution was the ability to only need minimal changes. These included a persistent way of knowing you were in Sandbox, badging labels, and the occasional copy edit.

Button Sandbox Sandbox on and off differences

The difference in interface when a user toggles Sandbox mode on.

In-App Testing for Publishers

Publishers follow these steps to test their integration in combination with our Developer Sandbox.

  1. Publishers insert the auto-generated code from the Dashboard into their app.

  2. A Button for the Test Merchant appears once the code is implemented. The Button takes the user to a mobile website where they can purchase from a list of items.

  3. Once the fake order is placed, users are encouraged to check the Dashboard in Sandbox mode to see the transaction and verify the price, currency, and list item.

Button Publisher app

Example of a Publisher rendering the Test Merchant Button in their app and the flow that follows.

In-App Testing for Merchants

We took the Sandbox project as an opportunity to update the visual style of our Partner Test App to better follow Material Design and iOS 11 guidelines.

While Publishers don’t need the app to test, we created a new feature called Integration Test just for Merchants. It works like this:

  1. Tapping the “Place Test Order” takes the Merchant to the staging or developer version of their app where they can place an order.

  2. Upon checking out, a notification is triggered by Button when we received the purchase in our system. The Merchant’s developers can confirm the amount and currency of their test purchase in the Button Test App.

Button Sandbox Test App

Integration Test feature in the newly designed Button Test App.

Button Sandbox Test App

Test flow for a Merchant placing an order through Button.


Most of our Dashboard changes at Button are rolled out slowly to users. Sandbox was no different, especially as part of the onboarding relied on Partner Engineering educating developers during the first technical call. We officially launched Sandbox and the updates to the Button Test App on October 3, 2017.


The introduction of a dedicated Sandbox environment in the Dashboard has had a positive impact on the integration experience for both Publishers and Merchants.

The time between the technical kickoff call and code submission to Button decreased from 12 days to 5 days.

The feature has been widely adopted by our partners and 100% of integrations since the launch has utilized Sandbox.


The Developer Sandbox was truly a team effort. I want to thank the following folks for their invaluable contributions. Without them, this would not have been possible.

  • Chris Maddern, Exec Stakeholder

  • Eden Hammond, Product Manager

  • Daniel McGrath, Engineer

  • Pavel Pantus, iOS

  • Wes Smith, iOS

  • Chuck Greb, Android

  • Liwei Mao, Data Scientist

  • Wilson Tsao, Data Scientist

Explore Other Projects



Creating the best tool to help optimize fantasy football rosters for daily and season-long contests.

BuzzFeed News Midterm Elections

Midterm Elections

In November 2018, I had the opportunity to design BuzzFeed's coverage of the Midterm Election through tickers, live shows, and of course, plenty of emojis.