logo
episode-header-image
Feb 2023
50m 53s

#126 Make Your A/B Testing More Effectiv...

DATACAMP
About this episode

One of the toughest parts of any data project is experimentation, not just because you need to choose the right testing method to confirm the project’s effectiveness, but because you also need to make sure you are testing the right hypothesis and measuring the right KPIs to ensure you receive accurate results.

One of the most effective methods for data experimentation is A/B testing, and Anjali Mehra, Senior Director of Product Analytics, Data Science, Experimentation, and Instrumentation at DocuSign, is no stranger to how A/B testing can impact multiple parts of any organization. Throughout her career, she has also worked in marketing analytics and customer analytics at companies like Shutterfly, Wayfair, and Constant Contact.

Throughout the episode, we discuss DocuSign’s analytics goals, how A/B testing works, how to gamify data experimentation, how A/B testing helps with new initiative validation, examples of A/B testing with data projects, how organizations can get started with data experimentation, and much more.

Up next
Aug 25
#317 How to Reengineer Your Business Processes with Nelson Repenning, Distinguished Professor at MIT Sloan & Don Kieffer, Senior Lecturer in Operations Management at MIT Sloan
Every day, knowledge workers face the challenge of managing competing priorities and constant interruptions. When systems are managing us rather than us managing them, productivity suffers and morale plummets. But what if the key to improvement isn't complex reorganization but ra ... Show More
1h 6m
Aug 18
#316 Enterprise AI Agents with Jun Qian, VP of Generative AI Services at Oracle
Combining LLMs with enterprise knowledge bases is creating powerful new agents that can transform business operations. These systems are dramatically improving on traditional chatbots by understanding context, following conversations naturally, and accessing up-to-date informatio ... Show More
56m 36s
Aug 13
#315 DataFramed x Alter Everything: Future-Proofing Your Career in AI and Data Analytics | Richie & Megan Bowers
The relationship between AI and data professionals is evolving rapidly, creating both opportunities and challenges. As companies embrace AI-first strategies and experiment with AI agents, the skills needed to thrive in data roles are fundamentally changing. Is coding knowledge st ... Show More
41m 5s
Recommended Episodes
Nov 2021
A/B Testing in Plain English by eBay Product Manager
Just about anything can be A/B tested, and it can be done by just about anyone. Nicole Lau, Product Manager at eBay, is here with us to explain A/B testing in a simple way so that you can get to using it and running insightful experiments. This episode is brought to you by Though ... Show More
28m 24s
Feb 2020
How to Set Up — and Learn — from Experiments
Stefan Thomke, professor at Harvard Business School, says running experiments can give companies tremendous value, but too often business leaders make decisions based on intuition. While A/B testing on large transaction volumes is common practice at Google, Booking.com, and Netfl ... Show More
23m 51s
Jun 2016
Conjoint Analysis: like AB testing, but on steroids
Conjoint analysis is like AB tester, but more bigger more better: instead of testing one or two things, you can test potentially dozens of options. Where might you use something like this? Well, if you wanted to design an entire hotel chain completely from scratch, and to do it i ... Show More
18m 27s
Sep 2023
Using AI To Land Better Cases? Why Not?
Let’s talk about production and winning cases. It’s not only about acquiring new clients, but also acquiring new clients that can deliver results. A potential breakthrough using Artificial Intelligence, AI.How do we frame, understand, and implement the best of AI’s capabilities? ... Show More
40m 32s
Jun 2023
Throwing away the script on testing
Sofy is a no-code test automation platform for mobile apps. SofySense is their OpenAI-powered AI assistant. See what they’re up to on their blog or check out their open roles.One of the biggest challenges in testing is deciding whether to use mock or live data. Interested in read ... Show More
21m 6s
Jul 2023
ELC 080: Use Data and Analytics to Level Up Your Instructional Design
Using data and analytics for instructional design may seem difficult. But Megan Torrance explains it in simple terms. Listen in to level up your ID skills. 
33m 28s
May 2022
Analytics Masterclass Part 2: Outcomes, Empathy Action & Profit
In Part II of our Analytics and Data Masterclass, shares his tips for working effectively with your analytics and analysts. Learn how to ask outcome questions rather than questions about marketing activity. These questions will  ensure your focus on the company's most important m ... Show More
17m 20s
Mar 2017
The Thing about A/B tests — To Learn or To Ship: That is the question.
Not all A/B tests are created equal. The most fundamental distinction is between “To Ship" tests and “To Learn" tests. It may feel like this distinction doesn’t matter, or that you can decide at any point in the project timeline. But if you believe that you’re fooling yourself an ... Show More
27m 45s