Gareth Hodgson reflects on a recent experience during a depth interview when a participant’s responses suddenly shifted, clearly suggesting AI assistance. The blog explores the implications for qualitative research, vigilance, and maintaining data integrity in the age of AI.
I experienced a new one this week. The human AI respondent.
I’m not talking about an AI bot filling in a survey. I’m not talking about AI bot responses to a qual platform.
I’m talking about, sitting down to do an online depth interview on Teams with somebody, a real actual living and breathing human being, and the interview becoming absolutely unusable because of AI.
The project was in the construction industry with a contractor from London. Without wishing to stereotype too much, for the sake of personification, let’s call him “Gary”.
His camera was off. Not a big deal, not part of the recruitment criteria to be seen on camera.
The interview started great. The first five to ten minutes were a breeze. As you’re probably familiar with, early on it was mostly background and context, simple stuff for Gary to answer. I learned more about him to shape my later questions, and cheeky cockney wide-boy Gary got more comfortable with what the interview entailed.
And then I asked the first “difficult” question.
And that’s when I got the first pause from Gary.
He clarified the question.
Oh good I thought, he’s giving this serious thought.
I could hear what I thought was the tapping of a keyboard in the background, but not an immediate red flag, maybe he’s just multi-tasking. An e-mail he needs to quickly respond to or something.
And then he answered. Very articulately. His sentence structure changed. The language he was using changed. Very detailed. Very jargon heavy.
The flag still wasn’t at full red… but it was flashing amber.
And then he threw in an example to support the point he’d made. Back into Gary-patter-mode as he laughed along.
Hmmmm…. Maybe that was just him in professional mode and he speaks more “on it” when he’s talking the detail of “shop”?
But this continued. The pauses and stalling. Repeating the question. Typing. The abrupt shift from conversational banter-ous contractor language into highly structured, report-style answers. A headline statement. Multiple supporting bullet-like points, in more depth and detail than I’ve ever heard a human formalise information in my life. A summary conclusion. An occasional counter argument. I’ve spent enough time on ChatGPT for this not to feel familiar.
But he was still savvy enough to dress it up around the edges with real/relatable examples, humour or World Cup small-talk. But all in Gary language with a seismic shift of tone and vocabulary to his detailed “answers”. He had a format and was genuinely trying his best to pull off the ultimate ruse.
I was 99.9% sure that I was right. He’s just using AI. But with this being virgin territory I wanted to get a 2nd, 3rd and 4th opinion. From humans and AI.
Long story short, nobody disagreed. Man, woman or machine. A black mark for Gary with the recruiters. No incentive for all his efforts. And a replacement interview was booked.
Lesson learned, I’ll be stopping the interview short next time this invariably happens again. Be vigilant out there people!
Such a shame. In terms of the quality, depth and breadth of content, it was literally the greatest depth interview I’ve ever done!
Virtual qualitative research is evolving in the age of AI. Researchers must be vigilant for AI-assisted responses, carefully monitoring language, structure, and detail. Early detection allows adjustments without compromising data integrity. Gareth’s experiences demonstrate how understanding the signs and having robust protocols protects the quality of qualitative insight while maintaining participant engagement.