Video: Becoming AI-Native in Research Series w/ Great Question — Session 2 | Duration: 2376s | Summary: Becoming AI-Native in Research Series w/ Great Question — Session 2 | Chapters: Welcome and Introductions (4s), Workshop Introduction (42.54s), Agentic AI Moderator (126.695s), Shifting Prototype Tools (189.265s), AI Quality Control (267.795s), AI Moderation Uses (375.46s), Visual Intelligence Layer (501.18s), Synthesis and Analysis (633.74s), Prototype Test Setup (775.925s), Study Configuration & Setup (1021.07s), Expense Submission Workflow (1192.07s), AI Research Limitations (1506.905s), Prototype Testing Insights (1761.265s), Resources and Q&A (1869.525s), Cognitive Offloading Concerns (1951.045s), Customization and Tuning (2066.195s), Q&A and Wrap-Up (2155.315s), Closing Remarks (2289.555s)
Transcript for "Becoming AI-Native in Research Series w/ Great Question — Session 2": Hello. Hello. Hey, everyone. I'm running a little bit late this morning. I don't like that we're here. Hey, Nick. Good to see you. Awesome. I see a few folks are sort of saying trickling in and saying where they're coming from, which is great. Keep that coming. And looks like, yeah, we've got a bunch of cool people there. Oh, Sarah Stum. Good to see you. A bunch of great people. Alyssa. Oh, this is awesome. Love to see it. Daniel Blufferfry. That's a familiar name. Great. Hello from myself. Great. Cool. So today is, I'm gonna be talking about, AI native research and particularly about our AI moderator and becoming AI native in research. I including gonna do some demo of some product, which I'm a little bit nervous about, but mostly excited. Let me throw it to some slides so that I can keep, I can keep myself, on track. And share my screen. Right. I hope you can all see that now. Boom. So this is actually session two, of a monthly workshop that we've been doing, on being AI native in research. If for those who don't know, my name's Ned. I'm cofounder and CEO of Great Question. We are the UX research platform built for, AI native teams. The idea is that it's intended to be more like a workshop than a webinar. Might be a subtle distinction there, but I'm actually gonna be in the product live and showing some stuff. It's live, so things inevitably something will go wrong. I guarantee it. Something actually already went wrong right before this, but we're we're figuring it out. But if you wanna hear in August, that's fine. I'll send you the link afterwards and so you can get that, and and you're gonna be caught up anyway. Bit more about me. Already spoken about that. That's fine. So in August, we showed, our agentic AI moderator, and it's for the first time, it's developed a bunch since then. But the idea is that it behaves more like a researcher than than a script or a survey. It's able to do things like following up. It adapts based on what gets thrown at them from the participant, and it can even see their participant's screen. It's got visual intelligence. So it went live, and people are using it. Usage is growing, which has been great, and we've been particularly improving its agentic moderation capability, not only by adding new functionality to that prototype testing, which is a bit I'm personally mostly excited about, but also just improving things like latency, barge in detection, multiple language support, which is we haven't yet released yet. Don't wanna get ahead of myself. And, and so agenda prototype testing is sort of what I'll talk about today and how and where you can use this. And so this is the sort of things we're gonna be covering. I'm gonna skip through some of these things just for the sake of time. But here's what we're hearing from research team. One is people aren't using Figma so much for a lot of their prototype testing. I think a year ago, it was the thing. If you had some sort of way to do prototype testing and usability testing, you had to be using Figma. Like, there was just no other choice. Now it's like less and less teams are using it, particularly at this kind of mock up usability testing stage. Instead, people are using, obviously, Claude and, ChatGPT, but they're also using Lovable, v zero a lot. When they are using Figma, it's more likely to be using Figma make. And the I think the reason is obvious. It's so much faster to go and generate prototypes in these tools, versus going and needing pixel perfection. You don't really need that, for a lot of these sort of, like, high level first pass iterations. Our team is still using Figma, but it's right at the very end, or for other things like, brainstorming sessions and, like, pulling together mood boards and things like that. This is reflects also when I went to we we hosted a an event with a bunch of product design research leaders at Config, and I would say 90% of them aren't using Figma in the way that they did even even six months ago. The second that we're hearing is, AI is actually able to go and hold the the quality bar. So I think because it it this is kind of a evergreen problem around when people are trying to democratize research. How do I go and hold that bar? Especially when you can't oversee all the research that's happening or all the customer conversation that's that are happening. And so we've seen AI being able to do this for you, one, because you can retroactively go and review everything that your team is doing and go and grade them on it and see, actually, the interviews they're running are good, bad, or ugly, and sometimes all three, and go and give people coaching points. But also for us, we'll be building in tools that are using our skills. We have a skills library, which I'll share access to, which you can use, in two places. We have our called skills library that you can use to go and do things like review build screeners or review existing screeners, build your discussion guides, audit your panel for good hygiene. But equally, we now have skills inside a great question, which allow you to enforce anytime someone's calling the MCP to say, build a screener. Let's make sure they're building a screener in the way that we wanna be building it, where we wanna maybe disqualify first, where we wanna have red herring answers in there, where we don't wanna tell them or be obvious about which the right answer is so we can fill we can, get the right participants coming through. And so, this becomes really important. We're investing more and more into this, both from a governance perspective, but also from a quality control, both in setup but then in post study launch and running review. And some of that's coming out, but there's more to come. But I know that's top of mind for researchers at the moment. The third one that we're hearing a lot about and getting a lot of questions about is, will participants even talk to an AI? There's so much talk about AI moderation and and, like, I hear this pushback a lot. Folks like, well, my I get AI mod. That's cool, but my users won't talk to, you know, an AI moderator. I'm like, well, actually, I I think I think you'll be surprised, definitely from the data that we're seeing. But it really depends on where you point it at, and it's not useful for all use cases. I personally am not using it. I'm doing a ton of, like, early stage discovery for some new products right now. I'm not really using the AI moderator for that use case because I personally wanna hear it. I wanna be able to guide the conversation. I wanna double click and I wanna consume it, and I want it to be firmly embedded in my head. And so I'm not gonna do that. It feels very wasteful for that. However, if there's someone I'm trying to interview and I can't get their time, then it's a good fallback in that scenario. And I think it's also respectful for me to be able to say to them, hey. I'd love to get you time to go and understand, you know, this VP of product. I'm trying to get this, I'm trying to get time with you to understand how you're building product in this particular way. They're like, you know, I can talk to you in December maybe. Alright. Well, maybe you could go and give your answers to this. I'm moderated. Yeah. Absolutely. I'll do that tonight, after the kids go down. I'm like, awesome. And so there is this, like I think we're still learning about where and how this actually makes sense for folks. So, where to use it? I think, you know, think there are opportunities in in places like, you know, drop off in a sign up flow. You're not gonna do that as a researcher. People that downgrade, where you've got really time poor segments, high net worth individuals, really senior people, people that are hard to find that don't their shit colors just don't line up to yours. And so, this is where the AI moderation we're finding is being useful. There's a ton more, but these are some of the example use cases. On our end, we shipped a bunch of stuff. Obviously, Genpix prototyping and website testing is the release we're most excited about. It is moving away from Figma prototype, so the we do come plan to come back and support them again in this new build up. But, basically, you can paste basically anything. A Figma prototype, a Lovable, Visa, a Replit, whatever you're testing, it's been built to be agnostic to that, which also means it works in your live site. You go and add the tasks. You can do that by talking to it, either via our, UI interface, which I'll show you in a second, or via MCP. It it sets everything up for you. And then the moderator watches, you know, asks the questions of the participant and can also watch them as they go through their screen, track and see what's going on. They can see when they hesitate in a form field and ask why. They can see whether someone takes the wrong turn and went down a different path and maybe asks why or then suggests a correction. Hey. If you come back, you can actually go you know, here's where you might go and try next. And everything session that comes back comes back as video where you can watch it and see the participant going through that experience, with everything labeled and transcribed into a report, which I'm, I'm excited to show you what that looks like. And so this visual intelligence layer is really what what what is important here. And And so it's able to see what the participant did. And so here you can see as an example you might not be able to read this. Forgive me. I'll make this bigger next time. But, you know, it sees that they clicked to submit the to re submit the reimbursement. You can see that they're looking at the dashboard, and then then we can see when they're doing each of these actions around what the question was asked and then what the respondents typed, or or spoken, or communicated in in whatever way. And so that that level of granularity is gonna become really useful when you think about your, analysis and synthesis on the back end. What were they actually looking at when they gave this bit of feedback? Had they even seen this screen or this part of it when it at that time? Then after they saw that, was that the grand reveal which helped them understand this moment, or is there more that we can need to do to go and iterate on on this thing that we're trying to test? So it's helping to aggregate this behavior, when you're doing it at a certain amount of volume. The next piece is our synthesis agent. When the sessions are done, you don't necessarily wanna go and sit through it. You can if you want, but, you know, you don't need to go and read all these different sessions. We're breaking it down for you into each of the questions. You can see, you know, all the answers under each question type. You can see the aggregate across that. You can see it broken down into individual themes. We're just doing a lot of that heavy lifting. Where do people hesitate in the reimbursement flow? Well, seven of nine paused before choosing a category. Five said and we're gonna tell you the time because we've got that. We've got all the time stamps. Five said afterwards they expected it to be prefilled. That's probably a reasonable, request. Did they still submit? Four did. One gave up at the receipt step. They couldn't be biffed. They just moved on. Okay. Great. Now we've got that intelligence. We can go and figure out, is this worth it based on these conversion rates through this funnel, for us to change this or iterate on this, or is this actually just expected? Of course, people are gonna question before they go and indicate a category. It's important for them to do that. We actually it's actually not something we need to solve. But every moment of one of these points back to that moment. So we can then go and create a clip or highlight reel to understand help me understand the hesitation moments that everyone had in the reimbursement flow. Great. There's a highlight reel. We also got updated a bunch to our MCP. I'll send some more out on this afterwards, but, MCP usage has been absolutely insane. Hundreds of teams now using it to generate research, recruit, analyze, and it is read and write. So you can do it to, you know, understand all your customers and your customer segments as well as pushing users back in to do recruitment, setting up new studies, editing studies, reading past studies. It's all there. I'm really proud of the work that the team's done, and there's weekly improvements to that. You should go and check out our change log if you haven't already. Some minor things, transcript search now returns the actual quote so so that you can cite a line without pulling down the whole transcript. So this is just gonna reduce your token consumption, which is important for more and more teams. And we've also, increased, some of the deep search volume, about, basically increase the performance, minor stuff. Enough gas bagging. Let's go and actually go and see something cool. And so what I'm gonna do is I'm gonna go and set up a, a study here, a prototype test. By the way, if you've got questions, throw them in the chat. There is, like, a q and a section, but feel free to put them on the chat. By the way, I'm not actually I haven't looked at it. Okay. Great. No questions. I'm sure someone will Alana or someone will nudge me if I've missed some. My notes are taking up my screen. Let me fix that. Okay. So, I'm here. This is the new study builder on great question. This will replace the traditional study builder for those who haven't seen that. That that is the wrong URL. That's our new by the way, a little bit of a foreshadowing. Moving to, I was just redirecting. Okay. Great. I'll come back to that. This is just a little foreshadowing for a new website, which has been soft launch. And so here I I come, and I've got a few things I can do here when I wanna go and set up this prototype test. One, I could but everything I do in the interface, I can do via MCP. So it could just be within Claude or ChatGPT or Copilot and say, hey. I'm looking to better understand this prototype I've done. Or you could generate it from Claude and say, alright. Now test this on great question, and it will push it in and set up the bones of the study. If I'm a researcher talking to researchers, the thing that we hear a lot is that, look. I've already gone and created this discussion guide, and it's somewhere else. And maybe that's in Claude. Great. Push it in. But it could also be that it's in Google Docs because you've collab been collaborating with your team on, on on on on this discussion guide. Are we asking the right questions? What really what decision are we actually gonna make here? What metrics are we planning to change based on what we learned here? And then there's comments from your stakeholders, and there's all these other important people saying, well, I think it should be this. I think it should be that. Who who should we talk to? Here's a suggestion. Here's another suggestion. There's a ton of information in there, and having to go and retype it up in here sucks. And so we let you just upload that, if you want as a document. But I'm gonna keep things relatively simple right here just for the sake of this. I'm gonna say, you know, set up a prototype test, to help us understand, what people, think about our new expense submission workflow. Keep it simple. If you like, bring in any additional context from past studies. And then let me grab a link. I'm gonna grab, in this case, a magic patterns link across your everything. Okay. So it's gonna think now. If I want, I can go and see the thinking on it. I need to build a prototype test. Great. It's gonna pull up our author study playbook, to guide the setup, which is a playbook is effectively is is a skill. We work on playbooks, now we're calling them skills, which I can show you in a minute, which help guide them. In fact, I can give you a quick quick nudge on that while we're making this go through. So, effectively, you've got this list of skills. This is in the product live today. You can go and edit the skill, give it some instructions like how to write a screener. You can then decide whether it's enforced or not, and then which tool calls it's gonna be enforced on, and it is awesome. And the reason we built this is it makes it easy to distribute skills so that everyone has the skills available when they're running research, and it's easier to enforce them. They can't be routed around them. If you're gonna make that tool call, the skill is gonna be enforced. Anyway, coming back here. So it's gone and thought for a bunch, and then it's gone and built this, study for us, which is super cool. And so here I can see the study that it's built. It's also gone and written an explanation. It's decided how it flows. There's a couple of warm up questions. There's a prototype test, with some tasks that it's gone and created and some wrap up questions. And it's even pulled some context from past work around other studies we've run that relate to this. And it's asking me, wanna me to line up a recruit for this for this next. Like, let's go and actually recruit people. Awesome. I can do all this within the chat interface. Again, all from within Claude or whatever you use. This is a little bit cozy, so I'm just gonna expand out. And on the left, I can go and see my chat that I've got here, and I can continue to chat to go and edit steps here. You can say, put in a step that asks people for their favorite, beetle, and it will go and, update that. It might question why I'm asking that, but I I, hopefully, it'll just put it in. And then I can see all my different steps on the study. Not all of these are AI moderated or need to be AI moderated. You might decide that you want this just to be a regular question. We always think about it like a dial. It's effectively a you know, think about it as a survey. We dial it up a little bit. We can ask a follow-up question. Why is it Ringo? And then we decide how many probes we make. And, you know, what is it that you like about Ringo's drumming? And, but but it and we could dial a bit more, and they can talk back to it. And then we talk dial a bit more. We can see their video, see their camera, which allows us to measure body language. We can see their screen, which then allows us to measure visual intelligence. So these skills are just effectively, building on top of each other. And we can see that it's gone and added this question, who's your favorite beetle? And this is not an AI moderator. It's actually not gonna ask a follow-up. I could tell it too, but that seems a little silly. So it's gone and written this plan, and it's also gone and here is our prototype here that it's gone and pulled up here, and it's got these little subtasks against this prototype. What are the different actions we wanna go ask with an indicator that lets the user know where they're progressing? And then, it goes through, ask a question, and then a wrap up question to thank them for their time, and we're off to the races. I wanna see the high level plan. I can see that here. And here, I've got the link to the study to start recruiting, or I'm just gonna open this one up here. And this is the fun part, because this is where I have to figure out how to talk to you and talk to it, and then and it, let's see how well this goes. So I connect my camera, my video. I enable my screen. You will get to decide how much of this you actually need to even get from the user because you may not if you if you just it's a basic survey. We don't need the camera and the mic and the screen. Like, why are you asking me for this? Equally, you might not need them. You would like them, and but you wanna optimize for the end user. You know, if they're on a bus, they don't wanna be talking to this thing. But, equally, if they're doing the dishes or something else with their hands or they're, you know, they're not unable to talk, the idea of to type the idea of talking to it may actually be, be to their favor. Allow that. Let me share my screen. So today, I submit work expenses by logging into a tool. I can't remember what it's called. And I will upload my receipt, and then put it in a category, tell them what it's being used for, put the amount, and then indicate how I wanna get reimbursed. I think maybe I'm doing it through Brex, I think, is the answer. I'm realizing that you can't hear can anyone hear the audio here that's coming through? I may need to figure out how to share my computer audio. No audio from the butt. Okay. Thank you, Nick. Let me see if I can do that. It's not gonna let me do that in this tool that I'm using. That's okay. I'll fix it. Hey, Johnny. How you doing? Kevin Pinkley. So, the thing I was gonna say there was I'll, we do have the ability to have a glossary. And so the glossary is useful for, you call it bricks, whereas the company I was referring to was bricks. If we were a real expense management tool, we would have a list of in fact, I'll go after this call, a list of all the common tools and phrases people would use, which then makes it easier for it to pick it up, particularly if I've got a weird accent like a kiwi, something like that. And now let's come back to this. The most frustrating part of submitting expense is just all the manual, like, reentry. I'm, like, all this information's on the receipt. Can you just get it out of there? Like, why are you asking me for this? And now it's taking me through to the prototype. So you can see it's rendering the prototype here on the right hand side, and then I can, it's telling me, hey. Take a moment to look around the screen. It's sort of verbalizing this. I can mute it as well. And now when I'm ready, I can, start, and now I'm in progress. What is this, and what can you do here? So this looks like a a expense management tool. I can see my different corporate cards. I can see some past transactions, I guess. I can see how I can request a card and, submit a new reimbursement. For some reason, there's two request a card buttons. Yeah. That's what I'm seeing. So I've got a new task here, another subtask. You can see that it's gone and burned burned along a little bit. And so now I need to, submit an expense. Let me figure this out. I go to new new reimbursement. I choose something to upload. Oh, that's cool. It's scanning it. And it's putting that in there. That category looks about right. Office supplies, describe it, stuff for the office. Cool. How confident are the expenses actually submitted? I'm pretty confident. Probably four. It's giving me this feedback in real time, which the user research indicates has been positive for people to understand that they're actually being heard correctly or being able to make corrections without us having been having to go and, you know, come back later on and find out something was wrong. I was pretty confident. It said that it's re reimbursement submitted, and, you know, it's pending approval. So we need to wait till that's done, but it seems pretty good. And so you can see how we sort of step through this process. And then on the back end of this I'm not gonna keep keep going through this because, it's like watching paint dry. But on the back end of this, we will then produce a new report. Oh, great. We've got some questions. Awesome. Let me just show you this real quick. We produce a report, that is automatically being generated from these sessions. And I can go and see the basic responses. If I wanna click into each of these, I can go and look at each of them, watch them back. Clicking on this takes opens up this video. This will be a summary of their answers. I can then go and look at the analysis, which has broken it down session, question by question. It's got an ad generated summary. It's got a highlight reel where I can view each individual answer within this. There's some rapid iteration happening on the UI here. And then we generate a report, which you can edit, both the prompt and the report, but has gone and, created it, in this case, for a prototype test, has gone and broken it down to our initial rendition of what we think is the best way to do a report on this, where I can see the answers from each participant, and it's got citations littered throughout it. So, for this particular question, overall, the order felt intuitive for 10 out of 13 responses. I can then go down, and I can go and see the actual answers and be like, is that actually accurate? If I wanna go and validate it, I can then click into it, and it's gonna take me back to that moment in that video and that transcript so that I can validate it, which is then the precursor to which building, highlight reels that we can go and share with the team to, again, make it more visceral. I can then continue to go down through the report, emerging learnings and open questions, all the detailed evidence that I want, and, ultimately, a, video asset I can go and share with some, you know, caveats on, you know, the population this was done with. So that is the product side of things. And now I'm gonna go to some of the questions. Johnny Farr. If we're using AI, how do we ensure that we're still getting the depth that moderated research with experienced researcher typically provides? I'm hesitant using AI research because we'll end up with a bunch of what's and not enough whys. Yep. I can see it beneficial for concept test, but for continuous research or general research, it may not be as effective as insights. We'd love to hear perspective. Yeah. My perspective is I I'm not in a rush to replace the moderator research that me and my team do with an AI moderator, because some to some extent, it's that, like, too much of the what's and none of the why's. Like, I think there's, like, there's some risk of that, but it's and it's the it which is really about depth. But I just don't think that the technology is really that replacement for that yet. And I say yet with I caveat yet with, like, everything to do with AI technology and its limitations today because it is all developing so fast. Even things like synthetic users, which I was always very skeptical of, I'm becoming less than skeptical over time. But, again, it's only for really specific use cases that I that I'm not that I'm optimistic about it. Things like, you know, pretesting a study would be a good example there. And so, yeah, I think to your point, I I I I, I don't think it's it's an effective replacement for continuous research or generative in research. I've already explained generative, but I think on the continuous side, my view is the continuous is actually become more important than ever for every team because if you're you can't overly rely on, AI generated insights, stacked into AI generated insights to go and make a decision. You need to still stay close to the customer to really truly understand what's going on to build your own internal context as your own large language model so that which is effectively instinct, otherwise known as instinct or, what's the other word for instinct here, intuition, that has to be rooted on customer insight, not on the output of an LVM, in my in my my experience, in my belief. And so I think that's not gonna go away. It's gonna become more important that everyone does that more often so that we don't lead ourselves astray. Yep. Thanks, Johnny. Cool. So that is our new perfect. That is, the new, prototype testing, functionality. I went and, you know, see it, set up the URL, set the flexibility, test it, and ask the synthesis agent, and I can now go and do this as well for via, via chat as well, which is awesome. Do it via via my asking additional questions. So I could say, what was the most surprising thing we learned here? I'm sorry. I'll reflect the chat back against this, against this asset. And I could bring in more context here. I could say, what's the most surprising thing we learned versus everything else we've learned to date? Like, what's what is new here from versus previous studies? There's a bunch of other flexibility you can do there. Business head flow is easy while the recording show they never reached a submission confirmation. Oh my gosh. That's not good. Cool. But that's good to know. That should stand out in this report. Only 10 of 13 reached visible post submission state, while three of 13 had at least one observed exit without confirmation. Right. That's the just the counter to that. White Matter's self report is masking a real completion confirmation gap. It's interesting. We should maybe add this type of question. What was the most surprising thing we learned here? We could add this to the prompt that goes into this report generation so that these, like, disparities, are improved in this report, but also in all future subsequent reports as the templated, prompt that goes in here. But that's valuable. Anyway, I'm getting off track. And so let me bring my notes back up. And so that's effectively the the prototype testing. You can go and use this. I believe this is on for all accounts today. If it's not, you can email me at great question dot co, and I'll get it I'll get you, into it. Before we let folks go, I'm happy to open up the q and a. If anyone has any other questions about the product or about AI research or wants to share a learning about what they've done or they've seen or been experiencing, I would totally welcome that. But we have a brand new UX guides and a brand new called UX skills library, which you can access. The new UX research guide, this guide to synthetic users, guides on unmoderated testing, b two b research, and much more. It is awesome. You should go and check it out. And two, this, new Claude UX skills library, it's got our own synthetic use of skill. Again, there's tons of limitations. You can go and get the guide to understand more about that. There's discussion guide builders. There's a whole heap of stuff. These are being created by us, and our research team, but also from customers. If you've got a cool UX skill that you've gone and built, we would love to feature you, and go and share that with the community if you're down for it. So, let me know. Again, there at greatquestion.co. New comment. Daniel, some powerful stuff in terms of being able to test at pace. How have you thought about designing the great question platform to ensure people spend more time watching respondents, developing their own POV about themes before enriching with AI analysis? I mean, I think these summaries are awesome, but worry a little about cognitive offloading, yeah, as soon as you've got your completes. Yeah. It's an interesting question. I haven't thought too much about that specific quick great. Thanks, Alyssa. I haven't thought too much about that specific question, Daniel, but I think that that is interesting. I I guess I've been thinking about how do we make this stuff more consumable, and I think the report is one way to consume it. The other is through highlight reels and clips. It's gonna be tough to say, should you go and sit through this entire video to go and get the get the answer versus do the synthesis? I think you need a combination of both. Can we make it easier to con to, create and consume those highlight reels and maybe make that front and center so that you are consuming first party data? Maybe. Maybe that's part of the answer. Could we also do something in terms of the report where we have something about consumption that's attached to it? You know, like, x many clips have been actually consumed or watched, and and how far they've been watched and how far they've been engaged with, and where what has been edited as well. That's something that I always wanna know is, hey. I I don't need to just read a Claude output. Like, tell me what please tell me you edited it. And and you at least read the thing before you send it to me. Like, I can otherwise, I'll just chat the call. Like, this is insane. That's That's an issue one. I'd love to hear what you you're thinking about this. And, obviously, we've got more work to do there. Not whether it's an hour or offline. And Alyssa. Oh, yeah. Great. On Addison. Hey. And Kevin as well. Jeez. The you guys are saving up the gold for, for the, for the backroom. Kevin Pickley, are we able to leverage skills and great questions to change the format of the generated report? Yes. Not today, but in the coming days. I'll need to check-in with Jack Jolley on that, and, Kevin, I'll come back to you. But, yes, we wanna make it so that you can edit, edit the, the work that we're doing is to make it easy to edit the prop that generates this, which I believe is living in skills. It may live somewhere else, and, the actual report itself, so that you can say, actually, I don't believe in that. Listen to how we talk or whatever. Yep. Totally. Because as an example, some people might like, I don't want you to say 10 or 13 participants. I want you to say whatever that percentage is. I would not do that. Some people want to. We're gonna give you that ability to go and to go and do that. The Addison, have you considered indicators of confidence in finding interpretations? Yes. We are, and, this is very light. This is very light. That's another layer. And can you tune it? Yes. When we roll it out, you would be able to tune it. Addison, I actually I owe you an email. I got some stuff I wanna shoot you on that. Yeah. That's still thinking about it versus reality right now. Alyssa, do you have any sense of how our participants are liking the experience of, talking to an AM moderator? Are they treating it seriously? You know, I think we could probably do some more research here. The early indications are, yes, people are treating it seriously. We are getting significantly more themes than we are out of other methods, and there is academic research which indicates a 125% increase in themes produced via, talking to an AI moderator versus traditional survey. But that's not really saying much versus a traditional survey, you know, just, for various reasons. But we could probably actually go and do, like, a, like, an, maybe a a system usability score or, like, maybe a CSAT or something. I need to think about what the right method would be. But let me come back to you on that list, and that feels like a good one to, to push on. Let's get more objective with it. Cool. Here we go. There's a stonker in here from Johnny Farr. Exciting stuff. I'm building a new SOPs and governance for our org on research with QA at GQ. And Claude, great. Can I use Claude to automatically highlight transcripts? Oh, you're killing me. Or build a script that creates highlights, reels, or is that still mostly manual? Might have missed the documentation. Yes. This is currently a restriction. I don't know the current road map functionality on it. Although I was emailing with the team, last night of this morning about that. I don't know when you're gonna be able to do those things. But, I will follow-up. I'm keeping track of all this stuff. So, Johnny, I'll follow-up on on the the road map, Tom, for that. It's, clearly, it's very hard, for a variety of reasons to do some of that stuff, and we are completely overhauling our our, like, HotReel tagging system, to make it more intuitive and to make it easier for the MCP and agents to go and create these things and edit them. So it's non trivial, but, yeah, I'll I'll get back to you on that. Cool. And I think that's almost it. And our next session is, with Constantine Pappas, who I'm super excited about, because, Constantine has spent a ton of time thinking about all these kinds of problems, is very deep in the literature behind it, and, has challenged my, perceptions, and attitudes more than once. And so it's gonna be it's gonna be I think it's gonna be I don't wanna say it's gonna be spicy. It's gonna be fun. It's gonna be fun. I'm looking forward to that one. And so that is Wednesday, September 30 at 10AM. Looking forward to everyone coming there, and, it's gonna yeah. I'm sure I know that I'm gonna get scored at least once in that one. So if you wanna watch me with, egg on my face, this is your opportunity. But I'll send a link in the follow-up so you can all, you can all sign up and find out about it. And that's kinda us. Anyone got anything else I wanna share? Questions they wanna ask? We can wrap it up. Awesome. Great. I think that's covered it. Thank you so much, everyone. Appreciate you all. I'll see you all soon.