Blog

Here is an interesting video of a trainer (I believe related to the ILO SIYB program but could be wrong) using ChatGPT's voice mode with a small business owner in the Philippines (via Starlink no less).&nbsp;<figure id="figure-0czbddx29p3vm3ojhw2n" class=""><iframe id="0czbddx29p3vm3ojhw2n" src="https://www.youtube.com/embed/fZggTm95fYw" class="w-full bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9;" allowfullscreen="" data-url="https://www.youtube.com/watch?v=fZggTm95fYw" data-corners="default" data-fullwidth="false" data-mce-src="https://www.youtube.com/embed/fZggTm95fYw" data-mce-style="aspect-ratio: 16/9;"></iframe></figure>In our first round of experiments users liked the ability to send their questions as audio, but preferred written responses, particularly as the issues discussed became more detailed. But that was before the new voice mode. We plan to do another round of experiments with audio replies next year, but that will stay within the WhatsApp interface. Its great to see more experiments like this, to figure out how to make AI more accessible and usable for MSMEs in emerging markets!

Voice Mode in PHL

At its most basic level, an AI chatbot on WhatsApp is just an easier front end to ChatGPT, with ChatGPT instructed in a particular way and supplemented with particular documents to make it more relevant and reliable for a particular group (like Peruvian microenterprise owners). It will then sit there and wait for questions. But as discussed in a <a data-mce-href="https://mypeasesoria.org/blog/beyond--chat-with-the-training-manual" href="https://mypeasesoria.org/blog/beyond--chat-with-the-training-manual" target="_blank" rel="noopener">previous post,</a> this "chat with the manual" bot isn't very useful, and won't get much use.Why? Its placing all of the burden of the user to know what questions to ask, how to use AI, and when. Small business owners are incredibly busy, they have very limited time, and limited experience with technology. Making these tools actually useful and usable isn't their job, its our job.We're only at the beginning of our experiments to learn how to do this better, but so far have found a few really useful things:1) A gradual onboarding process. One week after they first connect with Maia, we send them a list of the 3 most common questions across all users in their partiuclar sector, so they get a sense of the breadth of topics they can explore. The next week they get a guide on how to ask better questions to get better answers (provide more context, follow-up questions, etc). Two weeks later, they get a message they can forward to fellow MSME owners to share Maia, if they found it useful.&nbsp;2) The 'tip of the week'. Every week, users get a different tip built from the best practice manuals in the knowledge base. This surfaces new ideas to explore that they may not have thought of. Users can then discuss those topics further if they are relevant (with quick reply buttons, which are a must!) and very quickly get to a specific actionable plan to implement a particular improvement that they design with the help of the tool.3) Follow-up messages. Its one thing to know that we should eat well and exercise, doing it is another matter. Small business owners are busy, and many things come up that may need immediate attention but distract from important but non-urgent things like improvements. If users explored a particular topic the week before and identified some improvements they wanted to make, the following week they will get a follow-up message, to see if they implemented them (and if so, how it went and what are the next steps), and if not, an opportunity to refelct on and possibly revise the plan.

Beyond 'chat with the manual' II

Here is a presentation I gave today to the ILO's global network of SIYB master trainers. SIYB is the largest small business training program in the world, and it was a very interesting discussion!<figure class=""><img id="yrydhlkd4m1iao1v2" src="https://cdn.durable.co/blocks/18p7ZA33fXSdObi6mYMgjF2u8NDsMHxaUqK7ydtr5oP4xhOpEVufFHWggd5MqopM.jpg" alt="right arrow sign on wall" title="right arrow sign on wall" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/18p7ZA33fXSdObi6mYMgjF2u8NDsMHxaUqK7ydtr5oP4xhOpEVufFHWggd5MqopM.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-837va9m65m1iap9ed"><img id="837va9m65m1iap9ed" src="https://cdn.durable.co/blocks/73OKRA7Xb3wjJBicLj5AcpOIb4zirmY5iS8rmZ4PLUZtDxAaS3n8yDvHao8Tn7cV.jpg" alt="black smartphone near person" title="black smartphone near person" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/73OKRA7Xb3wjJBicLj5AcpOIb4zirmY5iS8rmZ4PLUZtDxAaS3n8yDvHao8Tn7cV.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-qdudzyw9erm1iappdy" class=""><img id="qdudzyw9erm1iappdy" src="https://cdn.durable.co/blocks/1bT37W1TZexY9S9Voxtx1NL24wnxfc6GZLjXrxMd4nAhKQOtYuGCfjfQu9zXNahq.jpg" alt="person holding compass selective focus photography" title="person holding compass selective focus photography" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/1bT37W1TZexY9S9Voxtx1NL24wnxfc6GZLjXrxMd4nAhKQOtYuGCfjfQu9zXNahq.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure>(Box 1 not covered today, but there is a ton of exciting ways that trainers could use AI to help them with things like lesson plans, exercises, role-playing, etc. See https://www.youtube.com/watch?v=8FnOkxj0ZuA to get started on AI for educators. The best source of ideas are the experiments of your fellow master trainers)<figure id="figure-gu1p0bki49m1iaqogu" class=""><img id="gu1p0bki49m1iaqogu" src="https://cdn.durable.co/blocks/d3PhcEBAyEm1XSveK4KgZBiwuWqA0vFqa2HlsNIkaQx5rlMXGnKfhtXraIFj3hqc.jpg" alt="woman in teal t-shirt sitting beside woman in suit jacket" title="woman in teal t-shirt sitting beside woman in suit jacket" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/d3PhcEBAyEm1XSveK4KgZBiwuWqA0vFqa2HlsNIkaQx5rlMXGnKfhtXraIFj3hqc.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure>(Box 2 not covered today, but this is just a new category of tools which trainers have always shared with trainees. Some examples: instead of teaching branding and giving advice on how to make a logo, AI image generation tools can just apply those best practice and make your business a logo. Other examples: customer surveys.)<figure id="figure-qiz7wl136m1iarj74" class=""><img id="qiz7wl136m1iarj74" src="https://cdn.durable.co/blocks/38wRZFkZKQ85QzDYQBNHHadvCMo8rdD34mMBNywOaHs7jvP9fHyoH0MGts6cUwZu.jpg" alt="black pencil on white printerpaper" title="black pencil on white printerpaper" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/38wRZFkZKQ85QzDYQBNHHadvCMo8rdD34mMBNywOaHs7jvP9fHyoH0MGts6cUwZu.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-2e7xjng0ym1iarxvf" class=""><img id="2e7xjng0ym1iarxvf" src="https://cdn.durable.co/blocks/eeA8IPaqzHLccHPKlP6RmuLbRoKHqzyqBRfHvGe3BwjHUloNWK6rFy4m2UtRaEDe.jpg" alt="three people sitting in front of table laughing together" title="three people sitting in front of table laughing together" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/eeA8IPaqzHLccHPKlP6RmuLbRoKHqzyqBRfHvGe3BwjHUloNWK6rFy4m2UtRaEDe.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-mu77k4tbjm1iasdl1"><img id="mu77k4tbjm1iasdl1" src="https://cdn.durable.co/blocks/22e52uphZ3zQpgjmKvHhqZClRaSwmbuWGOTPTuH0ut7yAhRP2XOAACTBEv1FfVl0.jpg" alt="man facing green mountains during daytime" title="man facing green mountains during daytime" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/22e52uphZ3zQpgjmKvHhqZClRaSwmbuWGOTPTuH0ut7yAhRP2XOAACTBEv1FfVl0.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-g2vr77hpum1iat2s8"><img id="g2vr77hpum1iat2s8" src="https://cdn.durable.co/blocks/5khVyfgshiCGfOCaj4M9Nx9rc1kPWeox3HnJ5q11C71BoWcspThawDoN9EkuWr0z.jpg" alt="two people sitting during day" title="two people sitting during day" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/5khVyfgshiCGfOCaj4M9Nx9rc1kPWeox3HnJ5q11C71BoWcspThawDoN9EkuWr0z.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-evwkjod23nm1iatexe" class=""><img id="evwkjod23nm1iatexe" src="https://cdn.durable.co/blocks/16A6ht3EQIcI3wPDUXEQnp3xOhF3iFrgg0NFLA2MWSzTVIVi5g1lYQzcMtt6kMaz.jpg" alt="two people sitting during day" title="two people sitting during day" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/16A6ht3EQIcI3wPDUXEQnp3xOhF3iFrgg0NFLA2MWSzTVIVi5g1lYQzcMtt6kMaz.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-tjkb0ukezm1iats0m"><img id="tjkb0ukezm1iats0m" src="https://cdn.durable.co/blocks/28EQ9aP7pP2mKiHQboXBKrrGeJZjdRp56tqkcXFftB3Ncx70HFUyknkr47qvhEUb.jpg" alt="person using laptop computer" title="person using laptop computer" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/28EQ9aP7pP2mKiHQboXBKrrGeJZjdRp56tqkcXFftB3Ncx70HFUyknkr47qvhEUb.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure>This presentation focused on AI-mentor complementarities for entrepreneurial training, that seems like the most powerful combination out there. But... we do have to deal with the fact that even if we are massively successful, if we scale up human-led training by 10x, that still wouldn't reach 1/5th of MSMEs in Peru, for example. So we also need to figure out how to make entrepreneurial training with AI available for the rest, who can't receive traditional in-person training. As the work of Otis et al. shows, this is definitely possible, but has to be done carefully! <figure class=""></figure>

Presentation to the ILO SIYB Master Trainers Network

As a rule I avoid new AI tool hype, but this was an interesting feature added to Google Notebook LM- a push-button explanatory 'podcast'. This one is based on just a couple documents about MAIA. It has some outdated figures (partiuclarly around costs, which we've managed to reduce quite a bit), but does a really great job of a big-picture explanation in a user-friendly, 'planet money-esque' format, based on just a couple documents. Try it for your own project, and follow<a data-mce-href="https://drive.google.com/file/d/1kksL5Fb0sHg8YUL-DYDTECt9-nPRHBpU/view?usp=sharing" href="https://drive.google.com/file/d/1kksL5Fb0sHg8YUL-DYDTECt9-nPRHBpU/view?usp=sharing" target="_blank" rel="noopener"> this link</a> to listen to the MAIA Fodcast!

MAIA Fodcast (fake podcast)

The biggest problem we face in applying emerging academic research on AI advisory tools for MSMEs to the real world is the external validity of those results. Let me explain with an example, the best paper out there so far: "We recruited entrepreneurs into our study over the Meta ad platform in partnership with the Busara Center, a Kenyan research organization, starting in May 2023. Figure 2 and Appendix C provide additional details on the experiment timeline and sample recruitment process. Our recruitment strategy involved running ads on Facebook to invite entrepreneurs to a short paid survey (Figure A1). All entrepreneurs who responded to our ads were required to pass basic attention checks and take part in three rounds of pre-treatment surveys delivered over the course of three months, which helped us reduce post-treatment attrition and ensure valid causal inference. Appendix E describes our surveys, which asked about firm profits, revenues, and management practices, among other measures. Our final sample includes 640 Kenyan entrepreneurs who completed all three pre-treatment surveys. The median entrepreneur in this sample was 26 years old, had been running their business for one year, and held a college degree (Table A1). " Otis et al p7.This approach makes complete sense to maximize the power of the study and minimize attrition. But if we are seeking to apply the results to all Kenyan entrepreneurs, we need to ask ourselves how representative are those who would see and respond to a facebook ad of those MSME entrepreneurs and pass all those steps? Might they differ in a way that would be relevant for the impact of intervention?Facebook penetration is high in that country, as it is in Peru, but the number of registered Facebook accounts per capita doesn't necessarily translate into a high proportion of MSME owners as frequent users who would click on a facebook ad.&nbsp;Moreover, not only did they have to click on the ad, they then had to pass all of the attention checks and complete the pre-treatment surveys. It seems quite possible that this would result in a set of participants that are younger, more technologically inclined, and more motivated and engaged in terms of looking for ways to improve their business. Indeed the paper describes the median participant in that study as a 26 year old college graduate, with a higher proportion of technology-related businesses than one would expect for Kenyan MSMEs as a whole. And you can compare the usage rates of the tool in the paper to what you would expect 'in the wild' (from an average set of owners who weren't recruited and offered rewards for participation). If you think that the impact of an AI advisor on business performance might be different for this group in some important ways, you need to be very careful in applying those results to the broader population.The average bodega owner in Peru is a mid 40s mother with no post-secondary education. In evaluating distribution channels in the country we met with a local digial marketing expert, and they indicated that facebook ads here could only reach a small and particular subset of bodega owners who are on average more educated, younger, and running more sophisticated operations. When I pressed them for a percentage of the universe of bodega owners that would be reached by Facebook ads, she said 3-5%, despite the high penetration rates of Facebook here.To repeat, this was not an error of the Otis study, their approach made perfect sense for that exercise. And because theirs the only real evaluation of these tools to date, we have paid very very close attention to it and incorporated its results explicitly in our current model prompt. However, it is an open question what the impact of these tools would be on a more representative set of MSME owners. And to figure that out, we need to get to more representative samples than are possible with online advertising. We are trying to do this do this by, for example, providing the tool to all bodegas in particular areas through partnerships with universal suppliers, or our own direct on-the-ground promoters. Its quite likely that usage patterns and tool impact will look quite different for this universe of MSME owners.

Worry about external validity

Happy 'Dia del Bodeguero'! August 12th was the day of the bodeguero in Peru. At the Peruvian Bodega Association's celebration, Peru's Minister of Production highlighted Maia, and did a live demo for the crowd. The Ministry's entrepreneurial training program Tu Empresa will incorporate Maia into its programs and offer it to participants, and Bodegueros across Peru can now use Maia thanks to some international support soon to be announced on our site. In the meantime, some news stories from print can be found <a data-mce-href="https://www.elperuano.pe/noticia/250164-herramienta-con-ia-para-bodegueros" href="https://www.elperuano.pe/noticia/250164-herramienta-con-ia-para-bodegueros" target="_blank" rel="noopener">here </a>and <a data-mce-href="https://diariouno.pe/2024/08/13/ministerio-de-la-produccion-promueve-herramienta-con-inteligencia-artificial-para-fortalecer-la-productividad-de-los-bodegueros/#google_vignette" href="https://diariouno.pe/2024/08/13/ministerio-de-la-produccion-promueve-herramienta-con-inteligencia-artificial-para-fortalecer-la-productividad-de-los-bodegueros/#google_vignette" target="_blank" rel="noopener">here </a>and from TV <a data-mce-href="https://plataforma.ipnoticias.com/Landing?i=8rjVc38Q1fmQN9n3eazhjw%3d%3d&amp;cac=rN2GE%2fYIVP99UB5qFlVYiA%3d%3d&amp;c=rD%2b4wLEDjFCxdp6PZfPQQEPMkj5kts1ILGBwH%2fiMysw%3d&amp;utm_source=alerta&amp;utm_medium=correo&amp;utm_content=video&amp;utm_campaign=videomail" href="https://plataforma.ipnoticias.com/Landing?i=8rjVc38Q1fmQN9n3eazhjw%3d%3d&amp;cac=rN2GE%2fYIVP99UB5qFlVYiA%3d%3d&amp;c=rD%2b4wLEDjFCxdp6PZfPQQEPMkj5kts1ILGBwH%2fiMysw%3d&amp;utm_source=alerta&amp;utm_medium=correo&amp;utm_content=video&amp;utm_campaign=videomail" target="_blank" rel="noopener">here</a>.

Blog

Voice Mode in PHL

Beyond 'chat with the manual' II

Presentation to the ILO SIYB Master Trainers Network

MAIA Fodcast (fake podcast)

Worry about external validity

Feliz Dia del Bodeguero!