Principle #4: Learn by making, not talking.

How might your team ship to learn?

Quick note: for new subscribers, welcome! Today’s post is the fourth principle in a series of posts on my 7 Principles of Great Product Teams. If you’re interested, you can read principles #1, #2, #3.

---

Great product teams learn by making. Let me explain with a story.

Shortly after Google’s acquisition of YouTube, I moved over to help with the go-to-market efforts.

YouTube felt like the wild west. We were losing money, getting sued by Viacom for $1B (back when that was a lot of money), and the market perception was that YouTube was just a bunch of videos of dogs on skateboards. Eric Schmidt, who was the Google CEO at the time, regularly questioned why we made the acquisition in the first place. Headlines like the one below were very common.

We knew that we had to change the situation quickly. Consumer usage of the product was skyrocketing, storage, server and delivery costs were ballooning, but we hadn’t figured out a viable monetization strategy. While advertising seemed like a natural choice for Google’s newest asset, no one could agree on how to monetize the property without damaging the user experience or building custom ad formats for every advertiser that demanded it.

The company was caught between a historical reliance on a splashy homepage ad and TV advertisers that were scared of advertising on the risky-feeling content. The team was faced with a few existential questions like: How do we ensure viewers have a great experience? How do we ensure creators get paid to continuously create high quality content? How do we incentivize advertisers to invest in producing high quality video ads?

In an email to YouTube founder Chad Hurley, the soon-to-be-leader of YouTube, Shishir Mehrotra posed a question: ”everyone loves watching the innovative ads during the Super Bowl — why don’t TV and video ads feel like the Superbowl everyday?” From that Eigenquestion, several new ideas were born.

One of these was to create a skippable video ad format. The theory was that viewers would be more likely to stay if they had control over the ads they watched, and advertisers would be incentivized to create more interesting and compelling ads.

Everyone in the company had an opinion on the concept of skip buttons on video ads. The sales team hated it. The YouTube Player Experience team was intrigued. The executive team thought we should focus on selling our marquee asset, the home page, instead of developing ads on each video.

At the same time, Shishir and Salar (YouTube CEO at the time) had given me a shot at product management despite not fulfilling the Google requirement of graduating college with a computer science degree. I was anxious to prove myself and make a big impact quickly. This was ‘PM influence’ trial by fire.

In the early days of the project, I toiled over how to deal with the competing opinions while designing a range of solutions with the team. I’ll never forget the nudge I got from my manager, Phil Farhi, at the time. His advice was un-intuitive but masterful, “Just test the extremes. Get the experiment going tomorrow. Then you’ll know how good or bad this idea can be before you have to convince more people.”

So we ran experiments with an enormous ‘skip ad’ button over the YouTube player experience that people couldn’t avoid. And another with a tiny skip button that was difficult to read. The two extremes gave us the data points we needed to understand the best and worse cases, and enabled us to speak intelligently to what we thought would happen.

Phil’s advice was a perfect example of the principle. Stop talking about the idea, just make something and learn something. Then come back and we’ll figure it out.

We could have debated the idea for months, but instead we shipped something quickly and learned.

As a new PM, I would have never taken the approach that Phil recommended without a nudge, especially in what felt like a high-stakes situation. It was an incredibly valuable lesson in building product. Eventually, we gave skippable ads a distinct brand called TrueView and emphasized the idea that ‘you only pay when people watch your video ad.’ The product became extremely compelling to advertisers and went on to become a multi-billion dollar business with an amazing team working on it.

Learning by making

I’ve learned this lesson the hard way several times throughout my career. One source of inspiration has been outside of technology in other creative pursuits, like art. Painters don’t deliberate on ideas for months or years, they paint. Photographers don’t imagine the perfect shot, they shoot lots of photos. The craft of making product is similar, you have to default toward making.

By contrast, ‘talking’ in the context of ‘learn by making, not talking’ can come in many forms. Sometimes it’s over debating how to handle a complex problem space, sometimes it’s mocking up too many options to a problem, and sometimes it’s analyzing data for too long to gain confidence in a potential path. There are many ways to get in your own way when it comes to getting something in front of customers. Often in these cases — making, prototyping, shipping enables a team to learn significantly faster.

Of course, this doesn’t mean build the first solution you think of, and it doesn’t mean build things that don’t solve the customer’s problem. It’s simply an acknowledgement that at some point, you have enough intuition or enough data to begin making and seeing how your solutions feel to customers. And in the end, the more (and faster) your team is able to learn directly with customers, the better the solution becomes.

Putting it into practice

Here are three ideas for how you can put this principle into practice:

(a) Amazon’s decision framework: one-way & two-way doors

One thing that keeps teams in a cycle of talking instead of making is misunderstanding whether the key choices in the experience are one-way doors or two-way doors. Popularized by Jeff Bezos, the question is whether a decision or experience is reversible or not. If it’s not reversible, then it’s a one-way door and the team needs to proceed with caution. If it’s a two-way door, and the experience or decision can be reversed, then the team should consider biasing toward making and shipping.

I’ve found that having this shared language on a team is imperative because often choices get framed as one-way doors by default when they are indeed two-way doors. You can read more about this concept and see an exercise in this template.

(b) DoorDash’s cupcake: build the smallest version to learn

I heard one great example of learning by making at a dinner with Tony Xu, DoorDash’s CEO. I believe the metaphor originally comes from Peter Merholz. To paraphrase Tony, most people aim to describe the biggest version of how a plan can be executed when the plan is proposed. But at DoorDash, they strive to start with the absolute smallest version of what could be executed, then go try it in a single geography, or with a small set of customers. In other words, they bake a cupcake without any expectation that it will turn into a full-sized cake.

(c) Intercom’s mantra: shipping is the beginning of the process

In discussing the principle of ‘learn by making’ with a few people, I learned of Paul Adam’s blog post about how shipping is the start of the learning process at Intercom. Paul’s ideas resonate and my favorite line from the post is:

We ship to learn. We know that we will be wrong more often than we will be right. Because we care most about learning, we prioritize speed to execution.

So to ask the question, how might your team ship to learn and test their assumptions?

---

👋 This little newsletter experiment is just getting started. So if you enjoyed this post, I’d love it if you subscribed and shared with a friend. Have a wonderful day! ✌️

---

Note: also published on the Coda blog.