How to Tell If the Agency You Hired Did the Work or Just Used ChatGPT
The strategy they gave you looks polished and complete. Here's how to tell if it actually took them five minutes to make.
You paid an agency to give you a brand strategy, website copy, or a performance marketing strategy. They turned it around in two days. It had all the right sections: a mission statement, a positioning statement, messaging pillars, and your target audience/ICP. Looks good, right?
What many business owners don't realize is AI generated output is an average of everything that's ever been written on the internet. That means the mission statement, positioning, messaging, and strategy could apply to half of the companies in your category. And when you try to use it to write your website copy or brief a designer, you find yourself rewriting everything from scratch anyway.
This isn't rare. As AI tools have made it faster and cheaper to produce everything, a growing number of agencies and freelancers are using them to fulfill client deliverables without double-checking its work. While the output looks professional and checks every box on the deliverable list, it's completely generic and does nothing to move your business forward.
Here are some ways to test whether the agency you hired did any of the actual work you hired them for:
How do I know if my agency actually developed my positioning strategy?
A positioning statement built by a human strategist comes with a specific point of view based on real research. Ask your agency why they landed on that specific positioning over the alternatives they considered. If they can walk you through the trade-offs (why they chose to emphasize one thing over another, what they ruled out and why) the work is probably real.
If their answer is vague, circular, or defaults to "that's what the research showed," push harder. Ask to see their research, which should show specific findings with real citations. AI tools may hallucinate and cite sources that are fake or out of date.
Did my agency gather any real inputs before they started writing?
An effective brand strategy needs to be built based on the challenges you're are facing today. That information can really only come from stakeholder interviews, a review of your past marketing tactics, and an audit of what your competitors are doing right now.
If your agency gave you the deliverable without speaking to anyone else in your company or asking questions specifically about your business, it's time to raise the red flag.
The strategy document they delivered won't be tailored to the specific needs of your business and the competitive analysis will be a generic overview of your industry, which tells you things you already know. If nobody gathered any real inputs before they started their work, then none of it will be written about and for your business.
Ask what research informed their work. If their answer is thin, so is the strategy.
Why does my brand messaging sound the same as my competitors?
Pull up three competitors' websites. Read their positioning statements, their hero copy, and their about pages. Then compare it to what your agency gave you.
If the language is interchangeable, meaning if you could swap your copy onto your competitor's site without it feeling out of place, then it's very likely AI produced it and it wasn't double-checked by a human.
These tools are trained on the same data, and you will see specific terms that AI almost always generates scattered throughout your agency's deliverables: trusted partner, customer-first approach, innovative solutions, built for businesses like yours, moves the needle, and strategic moat.
If you notice these patterns, it's time for a second opinion.
How fast should a real brand strategy take to deliver?
Brand strategy done properly takes time. Not because strategists are slow, but because the process requires real research that can't be rushed: stakeholder interviews, competitive mapping, and one to two rounds of client reviews. If a meaningful strategy document is created in less than a week after your kickoff call, ask what research informed it and when that research was conducted.
Delivering something quickly doesn't mean it's automatically bad, but when it was delivered after not asking any questions, then it's worth questioning its validity.
Is using AI to build a brand strategy always a problem?
No. The issue isn't whether an agency uses AI, and in fact most good ones do. AI can accelerate research synthesis, pressure-test messaging options, and speed up document production without compromising quality.
The question is what comes before and after using AI.
A strategist who has spent time in discovery with you making sure they understand your competitive landscape, specific business problems, and what has and hasn't worked can use AI to produce better work faster. The inputs are real, which means the output reflects your actual situation. And they will always double-check its work.
The thing to flag isn't the use of AI. It's whether the agency did the necessary work before and after to ensure that the output AI delivered will actually help grow your business.
What should I do if I found out my agency just gave me a ChatGPT answer?
Don't scrap everything immediately, because some of what's in the document may still be usable. The problem is you probably won't know which parts can be repurposed.
Make sure you get that second opinion before spending any money executing against that strategy, whether it's on your website, on ads, or on content. It needs to be pressure tested against what your specific customers are actually searching for and what your competitors are doing right now that is taking business away from you.
That's where we come in. Propel Collective can give you the guidance to see how much you can salvage and what gaps will need to be filled to make your marketing strategy work for you.