We introduced Frank—known in Italy as Pino—a few months ago. Now we're going to show you what he's really capable of!
In our previous article about our AI co-moderator, we told you who Pino is, where the idea came from, and the three levels you can set him at: minimally invasive, balanced, super-active.
But then, talking to some of you, we realised there's one question that keeps coming up: "OK, I get it, but in practice… how does he decide whether to probe or not?" 🤔
So let's show Pino at work.
Every time a participant sends a response, Pino reads it, evaluates it, and assigns it a score: the AI Score.
The logic is fairly intuitive: the more complete, articulate, and nuanced the response, the higher the score. The more vague, brief, or generic, the lower the score.
Take these two responses to the question "If you were a dessert, which one would you be and why?"
The first comes from Mario: "If I were a dessert I'd definitely be chocolate, because chocolate is very sweet, but at the same time it can sometimes be bitter too. So it has sides that are a bit harder, a bit more intense, or it can be very sweet, or have a thousand different flavours — it's something that doesn't always taste the same, it has so many variations."
A long response with plenty of nuance. Pino gives it a high enough score, nods, and stays quiet.
The second comes from Luigi, answering "Why did you choose those adjectives exactly? Give me a brief explanation." His response: "because it's sweet, there's definitely sugar so it's sugary, and it tastes good."
Low AI Score. This time Pino steps in: "Could you be a bit more specific? For example, what is it that makes this dessert really good to you?"
And this is where the threshold you set at the start of the research comes in:
🟢 Minimally invasive Pino → steps in only on truly unsatisfying or non-existent responses.
🟡 Balanced Pino → steps in on medium-to-low score responses. The preset we recommend in most cases.
🔴 Super-active Pino → steps in on slightly higher scores too, so even on mediocre but not disastrous responses.
For instance, Luigi's response might have triggered a probe with balanced or super active Pino, but slipped through unnoticed with low-key Pino. It's a simple but very powerful system, because it lets you calibrate participants' experience based on the type of research, the target audience, the length of the community or diary.
Now that you know how Pino decides whether to step in, let's look at when he does, meaning the specific situations where he joins the conversation. (The examples below come from our countless internal tests to make Pino work well, so if some responses look a bit odd, that's fine, it was us stress-testing poor Pinuzzo. 😎)
The most classic case you've already seen above: a vague, self-referential, low-score response. Pino notices and offers a more specific direction to restart from.
But across months of internal testing, other recurring patterns have emerged: subtler ones, worth walking through one by one.
Pino can analyse, understand, probe and… he can count.
Pino reads the question and works out how many elements a response needs to be considered complete.
Concrete example. The question is: "To wrap up, give me three adjectives to describe this drink…". The participant answers: "I'd say appealing and quite refreshing."
Two adjectives out of three. Technically a response, and not a bad one. But something's missing. Pino notices and probes: "Would you like to add a third adjective to describe it?"
It sounds like a small thing, but anyone who has ever moderated knows: these are exactly the small, tedious follow-ups that researchers often skip because they have a hundred other things to manage. And every missed probe is one less data point in the final report.
Multimedia responses are one of the best things about online qualitative research: photos, videos, audio. But they're also the responses where the most gets lost, because participants often focus on showing and forget to explain.
And this is where Pino knows how to speak up, even when faced with a video.
Real case. The question was: "What kind of glass or container would you use for this drink? Show us an example… and tell us why that one!" The request was twofold: show me the glass AND tell me why.
Participant's response (video + transcribed audio): "I'd drink it in this green glass. Because it's green."
The glass is there. The reason is… slightly tautological. Pino reads, watches, and probes: "Could you tell me a bit more about why this particular glass suits this drink — beyond the colour?"
Notice the detail: "beyond the colour." Pino doesn't pretend not to have seen the video — he acknowledges the attribute the participant mentioned and uses it as a starting point to ask for something more.
This is where things get more sophisticated. Because spotting a vague response is one thing; spotting a response that's completely disconnected from the question is another.
Real case. In one of our tests, the activity was organised into three sub-questions, all under one umbrella: how did your Easter Monday go? The three sub-questions were:
The responses that came in were: "It was the communication exam", "I was with the assistant, gee", "The exam went well, thank goodness!"
Consistent with each other, sure. But completely disconnected from Easter Monday. It's a fairly extreme example, but it shows how Pino pays close attention not just to the richness of a response but also to its relevance.
Pino reads the whole thing and probes: "Thanks, but here we were asking about your Easter Monday — how you spent the day, who you were with, and whether you had a good time."
In one sentence, he thanks the participant, acknowledges what they said, and gently brings them back on track. It's not a small thing, it's exactly what a good human moderator would do.
To summarise: every response gets an AI Score, you set the sensitivity threshold, and Pino steps in only where it makes sense. On vague content, on incomplete responses, on videos that show without explaining, on participants who go off topic, and, when needed, on situations that call for immediate human attention.
It's, at the end of the day, what we expect from a good colleague: speak when it matters, stay quiet when it doesn't, and flag when something important is happening.
That's exactly what Pino does, while you're free to focus on the probes that really count: the strategic ones, the ones where an insight is at stake, the ones where your experience as a researcher makes all the difference.
