Episode 6 - Tiny Boxes, Huge Consequences
How checklists save lives, hours, and experiments. And how to make checklists that your team will actually use.
When something is as simple as a few boxes and words on a small piece of paper, it's easy to underestimate its power. Yet the humble checklist has transformed industries, saved countless lives, and rescued many projects from chaos. Checklists are why it’s much safer to fly than to drive. Checklists helped us get astronauts to the moon and bring them home safely (and were responsible for parts of the moon landing remaining classified for over 40 years!). And checklists reduce surgical complications by 36% - more than just about any other intervention.
Checklists aren't flashy or complicated, but their impact can be profound. But why is something as straightforward as "checking a box" so valuable?
The answer is because checklists prevent failures of execution, not failures of knowledge. Failures of knowledge can be remedied by training and education. But failures of execution reflect a fundamental limitation of the human brain. Most people can only keep 7 things in working memory, but most processes are longer than that! Checklists are the lightest-weight tool available to fill that gap.
So how can you and your team best put these incredibly simple project management tools to work to prevent errors and keep everyone on track?
How a Disaster Gave Birth to the Checklist
Checklists first emerged from tragedy. In 1935, Boeing’s innovative new airplane, the B-17, crashed during a demonstration flight. The cause? Pilot error. The crew forgot a basic step: disengaging a lock. Realizing humans can only hold so many tasks in memory, Boeing developed the first pre-flight checklist. This simple 19-step list quickly became aviation standard practice, ensuring pilots didn't overlook critical steps.
NASA adopted this powerful tool for the Apollo missions. Astronauts walking on the moon relied heavily on cuff checklists to manage their complex tasks in the unfamiliar lunar environment. Even highly-trained people like the astronauts need the assistance of documentation when operating in the high-stress environment of outer space.
One humorous and secret addition even appeared during Apollo 12—a playful prank by ground crew who included NSFW Playboy centerfold images tucked discreetly into the astronaut's cuff lists.
Checklists Save Lives in Operating Rooms
The aviation industry's success with checklists inspired Dr. Atul Gawande, who wondered if they could also help in operating rooms. He observed highly trained surgeons making preventable mistakes, like leaving sponges inside patients or even operating on the wrong limb.
Gawande's groundbreaking research revealed that a simple surgical checklist—just 19 items—reduced surgery-related complications by 36% and surgical deaths by a staggering 47%. The World Health Organization now officially recommends surgical checklists worldwide, validating how critical a structured, simple reminder can be, even among highly trained experts. Gawande wrote a book about the experience, The Checklist Manifesto that has become a favorite read of many on the Boomerang team.
Bringing the Checklist to the World of Knowledge Work
While not life-or-death, checklists also offer enormous benefits to teams doing complex knowledge work—like launching software, onboarding new hires, or running marketing experiments.
At Boomerang, we learned firsthand how powerful checklists could be after repeatedly encountering preventable mistakes during product launches. After launching (and, sadly, discontinuing) Boomerang for Yahoo! Mail, we realized we needed a reliable, repeatable process to prevent oversights.
Our product launch checklist was developed after missteps like forgetting to do something that we had identified as important from a previous launch. This has become a vital part of our workflow, refined over more than a decade. It includes everything from analytics setups to pricing grid checks—avoiding costly errors and saving countless hours.
Our marketing and in-product experiment checklist has also been a lifesaver for our team. In 2024, our “year of experiments,” we conducted 52 different experiments across product and marketing. At first, we’d inevitably forget something important - like making sure we could reassemble who was in which cohort for analytics, or that we needed to make all the variants accessible for QA purposes. But by the end, our experiment checklist had solved that problem, and as long as we remembered to use it, our experiments usually ran without a hitch.
Another place checklists are critical for Boomerang's team is employee onboarding. At Google, they discovered that simply giving new hires a short onboarding checklist increased productivity by 25%, and was equivalent to onboarding employees a full month faster. Now, things like administrative to-dos, key training, and more is documented in a checklist that we plan ahead of time and share with new team members on their first day.
Creating Checklists That Actually Work
Not all checklists are created equal. To be effective, they need to follow three key rules:
Make Items Decidable and Deterministic: Each step must clearly define what “done” looks like. Avoid vague instructions like "Check analytics." Instead, say something concrete: "Confirm Google Analytics is capturing event X."
Balance Specificity and Generalization: Checklists should be specific enough to be actionable, yet general enough to apply broadly. They become most powerful when refined over multiple uses, building on collective learning.
Ensure Team Ownership and Buy-In: Checklists can't simply be imposed from above. Teams must be involved in their creation and improvement, understanding why each item matters, removing ones that become irrelevant, and adding new ones when helpful additions are identified. Without buy-in, checklists risk becoming mere paperwork rather than useful tools.
These principles are laid out nicely in academic research on secure software development done at Vanderbilt University. But we’ve independently learned these lessons through trial and error as well. We’ve found a dramatic uptick in launch checklist adherence since we started updating it as a team before a project starts and after a postmortem meeting. That also gives us a focused, dedicated time to set up instance-specific items before the process begins.
Our experiment checklist has become more deterministic and more decideable as we’ve improved it. Originally, the checklist contained items like “set up analytics,” which were not specific enough to confirm. Now, it’s expected that an engineer setting up an experiment will trigger the event behavior, then go into the analytics platform and confirm that the event shows up from testing. Reliability has increased significantly.
Another important distinction is between "do-confirm" checklists, for workflows that can be verified after all the items are done, and "read-do" checklists, for instructions that need to be followed in a specific order. Our product launch checklist is a series of “do-confirm” lists, where different people can work on different items in parallel, making sure they’re all done before launch day. Our payroll checklist is a “read-do” checklist, because it’s very important that we change deductions before we try to pay taxes!
Tip of the Week: Identify Your Own Checklist Opportunity
Here’s your tip for this week: Look at your workflow or team operations. Identify at least one task or event you regularly handle that has at least 5-7 steps. Could a checklist reduce mistakes, save time, or help onboard new team members more smoothly? If so, build your first checklist.
Whether you're running payroll, preparing a product launch, or even organizing your next team offsite, investing the time to create and refine a checklist is an investment in future efficiency and sanity.
And remember, checklists aren't about compensating for ignorance or lack of skill—they protect you from human error when your mind is busy juggling too many details.
To learn more, listen to the full podcast episode.