Every year I watch a company launch a "better" website and lose money doing it. New design, new copy, new brand direction, and a conversion rate that quietly drops the month after launch. Nobody notices until the traffic report comes in.
That's the exact problem Brian Massey has spent the last eighteen years solving. Brian is the founder of Conversion Sciences, and he joined me on the Marketing by Design podcast to explain why most website redesigns are really just expensive guesses, and what it actually takes to know instead of guess. His team has run more than 35,000 tests since 2007 and helped 300+ companies generate over $1 billion in new revenue. I wanted to break down what I learned from him here, because the lesson applies whether you're running a B2B SaaS site or a healthcare website where every word on the page carries compliance weight.
Who Brian Massey Is, and Why the Lab Coat Matters
Brian shows up to conferences and client calls in a white lab coat. It's not a gimmick. He built Conversion Sciences around the idea that a website is not a finished product, it's a live experiment. He calls his team's job title "Conversion Scientist," and the role blends project management, data analysis, and user research into one person who runs your site the way a lab runs a study: hypothesis, test, result, repeat.
You can read more about his background on his LinkedIn profile, and we've written up the full breakdown of what his agency does, who it's built for, and what it costs on our Conversion Sciences partner page. I don't refer clients to agencies I haven't vetted, and Brian's track record is one of the clearest examples I've seen of a CRO shop that backs up its claims with actual numbers instead of vibes.
Why "Redesign and Hope" Still Runs 90% of Websites
Here's the pattern Brian described that stuck with me. A company decides it's time for a refresh. A small team of smart people spend six to twelve months making thousands of small decisions about layout, copy, and design. Then they launch it all at once. The result is often a website that looks objectively better and performs objectively worse.
The problem isn't taste. It's that a full redesign pushes out hundreds of untested assumptions in a single release, and a few of those assumptions are wrong. You don't find out which ones until traffic has already dropped. This is exactly why we push clients toward a build process that treats design decisions as testable, not just aesthetic, especially in regulated industries like healthcare where a redesign that hurts conversion also means fewer patients getting the information they came for.
Your Website Is the Biggest Focus Group You'll Never Pay For
Most marketers think about user research as something you do with ten people in a conference room and a gift card. Brian's point is simpler and more useful: you already have a focus group, and it's bigger, more qualified, and completely free. It's everyone who's already showing up on your website looking to solve the exact problem your business solves.
The difference between a focus group and your actual traffic is that your traffic doesn't need to be recruited or incentivized. They're already there, already motivated, and already telling you what they want through what they click, what they ignore, and where they leave. An A/B test just gives you a structured way to listen.
Personalization Only Works When Data Leads It
We got into personalization during the conversation, since it's one of the most common things B2B marketers get wrong. Most personalization efforts start with a guess about who's on the page and what they want, then design toward that guess. Brian's approach flips it. Run the test first. Let the data tell you where the actual split is.
His example was simple: a test comes back inconclusive, until you segment by device and find Android users responded one way and iOS users responded the opposite way. That's real personalization, built from evidence instead of assumption. It's a small example, but it's the same principle behind everything we build into how we help clients understand who's actually showing up on their site before we start making changes based on who we assume is showing up.
AI Gave Everyone Ten Good Ideas. It Didn't Give Anyone the Answer.
This is the part of the conversation I think every marketer needs to hear right now. Brian pointed out that you can ask an AI model for ten ideas to improve your website, and it will hand you ten genuinely good ideas. The problem is that roughly half of them won't actually move your conversion rate, and there's no way to know which half without testing them.
AI has sped up the parts of his business that used to take hours. Analysis that took two hours now takes ten minutes, with several caveats included. But the speed increase hasn't replaced the need for proof. If anything, it's raised the bar, because now there are more ideas competing for the same limited testing capacity. The businesses winning right now aren't the ones generating the most ideas. They're the ones with a real process for knowing which ideas are worth building.
Brian used the bank teller and the ATM as his analogy, and it's a good one. When ATMs showed up, everyone assumed the teller job was over. Instead, the number of tellers grew, because ATMs changed what banks could afford to build. AI is doing something similar to CRO. It's not replacing the experts, it's changing what they spend their time on.
What This Means If You Run a B2B or Healthcare Website
If your website is the primary way people learn about your business, evaluate you against competitors, or decide whether to trust you with something serious, this isn't optional information. It's the difference between a site that quietly loses ground every quarter and one that compounds.
A few things worth taking from Brian's approach directly:
Stop treating your next redesign as a single, all-at-once event. Break it into pieces you can test independently, so you know which changes actually helped before you commit to all of them.
Use your existing traffic before you pay for anything else. It's already telling you what's working and what isn't, you just have to look at it as data instead of noise.
Let AI generate ideas, not decisions. Ten good ideas from an AI model are a starting point for testing, not a finished strategy.
And if you're in a space like healthcare, where trust and clarity carry more weight than almost any other industry, none of this is optional polish. It's the mechanism that turns visitors into patients, leads, or referrals instead of bounces.
Listen to the Full Conversation
Brian and I get into a lot more in the full episode, including how his team is using AI to code and QA their own test variations, and what he thinks happens to CRO agencies over the next few years as the tools keep getting faster. You can catch the full episode on the Marketing by Design podcast, and if you want a shortcut to what a high-converting website actually looks like before you start your own redesign, grab our Visual Swipe File. It's the same kind of evidence-first thinking Brian and I both believe in, just applied to design instead of code.
And that, is marketing by design.
.webp)
.webp)
