We are not alone!

We've been looking for other projects similar to ours, and hadn't found much. There are private-sector products delivering AI chatbots via WhatsApp. And we've been reading a lot and studing the excelent work being done in the education area, in particular by the Khan Academy and their Khanmigo AI tutor (they have put together fantastic resources for educators about AI, benefits and risks, etc, and we are using them as a guide, particularly when it comes to openness and safety, check out more about Khanmigo here). But I hadn't seen anything specifically for developing country entrepreneurs, until this great paper preprint was released yesterday. From all my searching, these are the first people to do it, which is impressive in its own right, but they also have done it in as very clean and rigorous impact evaluation. It is great work!

The findings are really interesting, and show that its not as simple as just hooking up deleloping country entrepreneurs to ChatGPT and then watch all the good things happen. The advice from their AI mentor seems to have had a notable positive impact on businesses that were already doing well, but actually may have hurt businesses that were already struggling. That may just be a temporary result, as it could take longer to turn around a struggling business than to improve an already successful business, with an initial dip in earnings as changes are made (compared to control firms who were struggling but may not be taking corrective action spured on by the mentor) - we'll have to see as they (hopefully) continue to collect outcomes data over time.

It is also important to note that their AI Mentor has a particular approach, focus, and structure designed into it, and as a result the logs in the paper do show a dialogue that sounds different from ours. And 'ours' also act and sound very different from one another, depending on which flavor is being tested: structured or unstructured diagnostic, hard-skills or PI-focused, expert consutlant or collaborative coach, generalist or sector (eg bodega)-specific, etc. There are lots of approaches to instruct the LLM, materials to use for context, and ways to structure the user interaction. This paper shows that not only may some be more effective than others generally, but some could be better than others depending on the situation facing the business.

More work to be done, but a fantastic first study!