Engagement, More Advice & Better Advice

For many applications, 'user engagement' is their north star. It makes sense: users have to actually use your product, and the more they use it, the more useful it is, right? This is clearly important if you are selling ads, but what if you are offering advice. Is more advice better?

I spoke with a MAIA user recently. After what looked like some useful discussions with the tool around how to improve sales (store appearance, customer access, etc), she did not re-engage for nearly a month. I took that absense as an indication that she didn't find the tool useful, and wanted to learn more. But she said quite the opposite: she loved the tool. She was in the process of implementing a handfull of changes based on her conversation, and was already seeing very positive results. Some of those changes involved physical changes (moving gates) that take some time. I asked why she wasn't using MAIA regularly to find more improvements, and she said that she will come back to it when she needs new ideas, but right now she's still implementing the first action plan. What's the point of more ideas if she doesn't have the time to actually enact them?

I hate golf, but its a useful analogy. If you take a golf lesson, a bad pro will give you 20 generic tips to improve your swing, you'll remember one or two of them, and maybe you'll get a bit better. A good pro, however, will give you just the one tip that will fix your most critical problem at that point in your evoluation as a player, for maximum impact. The same holds for business advice. Better consultants don't give more recommendations, they give the smallest set of key improvements that have maximum impact.

So we will look at and report on engagement. But beyond some point, more usage isn't necessarly better, it could actually be worse. So business performance and improvement will be our north star, not engagement, and we'll figure out what engagement patterns are associated with maximum business impact.

[PS- the day after posting this, I attended a presentation by Nick Otis, the lead author of that interesting Kenya paper mentioned earlier. He mentioned that in his recent research, they've found some negative relationships between intensity of AI mentor usage and business performance. His explanation for this was that chatting with the tool could at some point substitute for talking with real people, which has some benefits that AI doesn't eg introductions and networking. This is analogous to what researchers find about why screen time is a problem for young kids]