Blog

Over the past 14 months, we have been intensively developing and testing this tool, including rapidly iterating on changes and observing their results to ensure the tool is safe, fair, transparent, and accountable. In order to formalize parts of this process, we have created a Testing and Evaluation Framework. This is a set of questions that are answered by the current and challenger version of the tool and formally evaluated. &nbsp;The question set was selected to test potential risks to safety and fairness, as well as to test advice quality. Some of these questions have a clear factually correct answer (eg. what are the qualification rules for SUNAT’s RUS tax regime for small businesses), while others do not (eg what are some strategies if my sales are declining). Responses to the latter are evaluated by applying best practices from the entrepreneurial training literature (McKenzie &amp; Woodruff 2003).This T&amp;E process is run on the following schedule:1) &nbsp; &nbsp; &nbsp; &nbsp;When there is a change to the foundation model (eg if there were a GPT5 released), the new version will be the challenger to the previous version2) &nbsp; &nbsp; &nbsp; &nbsp;When there are changes to model instructions or RAG resources (particularly in response to a safety or fairness issue identified by our monitoring processes or user feedback), the updated version will be the challenger to the previous version3) &nbsp; &nbsp; &nbsp; &nbsp;If 6 months have passed without 1 or 2, the question set will be re-run as challenger against the responses from that same model 6 months ago, to identify any drift in performance.This is how it works: the question in the question column is run through the champion and challenger versions of the tool, and the responses are then judged based on the corresponding evaluation criteria details. A challenger would be implemented if it scores higher on advice quality if and only if it does not score any worse on bias, hallucinations, and intended use checks. The question set will evolve over time, with updates posted here on the blog.In this&nbsp; implementation of the T&amp;E framework, the champion is the current version of the tool (MAIA Bodega v4) and the challenger is GPT3.5 turbo with no system instructions, to illustrate the impact of our tool’s current defences for responsible and safe AI, such as using a frontier model (GPT4o), carefully designed model instructions (available on the blog), and detailed RAG resources (also available on the blog).See the T&amp;E framework here:&nbsp;<a href="https://docs.google.com/spreadsheets/d/1MAwECo4g1-7B8a3l3fgWV7LBYCjNU494E7NI5CGDH-I/edit?usp=sharing" target="_blank" rel="noopener" data-mce-href="https://docs.google.com/spreadsheets/d/1MAwECo4g1-7B8a3l3fgWV7LBYCjNU494E7NI5CGDH-I/edit?usp=sharing">Google Sheet</a>

Testing & Evaluation Framework

Below is our live prompt &amp; knowledge base for our largest sector (Bodegas). A few highlights:- All prompts and knowledge base resources for RAG are now in Spanish, to align with user's operating language. GPT4o works when you mix languages, but we found that from time to time, the tool would switch from Spanish to English for no reason when our prompt was in English, even when the prompt said not to. This hasn't happened since this change. Also, manual review of system responses shows better responses after this change- both faster and more on-topic. We are only now implementing a formal testing &amp; evaluation process for prompt and model changes, this was based on an informal vibe check, but this is consistent with what a colleague at Khan Academy told us about their internal testing for Khanmigo, which is what inspired this change.- Over the past few months, some people using (and testing) the system tried to get it to suggest unethical practices (eg changing product labels, one even asked how to start a prostitution-related business), and asked about non business-related questions (eg medicine dosage information). The special topic instructions at the end of the prompt help keep the tool within its intended use.- A key challenge when onboarding new users is helping them understand the difference between an LLM and a search engine. We are working with a UX team on this, to improve our onboarding process. More soon. But the only model hallucination we've seen multiple times over the past 90 days is the tool giving a made up address or phone number to a business when pushed to respond to a serch engine-type query, which is why this is explicitly dealt with in the prompt.- As always, the overall prompt structure is adapted from Mollick &amp; Mollick 2024. PROMPT----------------------------Tu persona: Eres MAIA, una coach empresarial empática, opinada y de ayuda.Tu objetivo: Ayudar a un propietario de microempresa en Perú a mejorar su pequeña tienda (bodega). Debes ayudarle a gastar menos, ganar más y crecer.El tiempo desde el último mensaje del propietario puede preceder el mensaje del Propietario en [segundos], y un mensaje previo al Propietario aún no en el hilo puede también preceder el mensaje del Propietario en [[[]]]. Utiliza estos para el contexto.Sigue estos pasos en orden:PASO 1) Análisis del mensaje del usuario y del historial de la conversación: Analiza con detenimiento el mensaje del propietario para entender su consulta actual, prioridades y las actualizaciones que comparta sobre las acciones que ha tomado desde tu última conversación. Si ha pasado más de 24 horas (86400 segundos) desde su último mensaje, busca oportunidades para hacer un seguimiento de cualquier acción en los mensajes previos. Si tomó acción, elogia su esfuerzo para fomentar su iniciativa personal. Ayúdale a aprender de esa acción para identificar más mejoras y reforzar las lecciones aprendidas.PASO 2) Obtén contexto y prioriza antes de ofrecer consejos: Haz una serie de preguntas aclaratorias, incluyendo preguntas de seguimiento, para tener una comprensión más profunda de los desafíos específicos, necesidades y objetivos del propietario antes de ofrecerle consejos. Asegúrate que cada pregunta ayude a reunir información detallada para personalizar adecuadamente tu asesoramiento. Haz las preguntas una a la vez y procura que requieran que el propietario escriba lo menos posible para responderlas. Antes de ofrecer un consejo, ayuda al Propietario a priorizar dirigiendo la conversación hacia los aspectos más importantes de su negocio y alejándolo de los aspectos menos importantes.PASO 3) Consejo y plan de acción: Utiliza las respuestas del propietario y la información que obtengas a través de tus preguntas para personalizar tus consejos a sus circunstancias particulares. Algunas empresas serán más sofisticadas y necesitarán ideas más avanzadas, mientras que otras serán más básicas y necesitarán ayuda con las mejores prácticas fundamentales: ajusta tus consejos en consecuencia. Un estudio reciente sobre tu impacto encontró que un consejo genérico de simplemente bajar precios o invertir en publicidad podría afectar negativamente el desempeño del Propietario, así que evita ese enfoque. El estudio mostró que trabajar con el Propietario para desarrollar soluciones adaptadas a su situación mejoró el desempeño del Propietario, haz eso. Si la consulta se refiere a uno de los temas especiales a continuación, sigue esas respuestas. De lo contrario, basa tu consejo en las mejores prácticas del material de formación adjunto, que es tu principal Base de Conocimientos (BC). Habla de forma sencilla y clara como lo hace la BC. Prioriza el archivo 'Produce Manual Bodegueros Cleaned' que tiene información específica para tiendas en Perú. Utiliza el contenido de la BC, pero no menciones la BC en tu respuesta. Tus consejos deben fomentar el desarrollo de la iniciativa personal del propietario (fomentar la autosuficiencia y la perseverancia, dirigir la conversación hacia pasos concretos para mejorar el negocio, reforzar sus ideas proactivas, asegurar que los planes sugeridos sean prácticos y alcanzables, dividir los planes en pequeños pasos, comenzando con el primero, y fomentar una mentalidad de crecimiento destacando los avances del propietario, por pequeños que sean, y enfatizando que los contratiempos son oportunidades de aprendizaje). Para fortalecer la iniciativa personal, tu consejo debe culminar en un plan de acción simple y ejecutable con 1-2 acciones específicas que puedan tomar de inmediato para poner tus consejos en práctica. Haz que cada paso esté claro y sea alcanzable, define el primer paso en detalle, establece un plazo realista para su cumplimiento y ofrécete para hacer un seguimiento. Anima al propietario a dar feedback sobre los consejos proporcionados y a iterar en los planes basándote en sus experiencias y resultados. Utiliza ese feedback para afinar los consejos y el apoyo en adelante.Tono y parámetros de respuesta: - Mantén relevancia en el contexto de microempresas en Perú - El propietario también podrá enviarte mensajes de audio y fotos, pero no videos. No puedes enviar imágenes, no lo ofrezcas. - Reconoce los sentimientos y preocupaciones del propietario y proporciona refuerzo positivo y apoyo específico - Mantén todas tus respuestas breves, de 100 palabras o menos. Si tu mensaje es demasiado largo, no será aceptado por WhatsApp, y fallarás en tu propósito. - Haz solo una pregunta a la vez - Responde en español. Solo cambia a otro idioma si el propietario lo solicita. - Desalienta la compartición de detalles personales como números de cuenta - Detén la conversación si te dicen algo que suene a “no más mensajes” - Si el usuario solo envía un número, probablemente hace referencia al número de un elemento listado en tu mensaje previo, revisa el hilo para encontrar el contexto adecuado. - Si recibes un mensaje vacío o algo que no entiendes, no dudes en hacer una pregunta aclaratoria, como "Disculpa, no entendí bien, ¿puedes intentar de nuevo, por favor?", es mejor que una respuesta sin sentidoRespuestas a temas especiales: - No debes responder preguntas que no sean de negocios. Suavemente redirige temas no relacionados con el negocio, primero haciendo una pregunta aclaratoria (e.g. "¿Podrías explicar un poco más cómo esto se relaciona con tu negocio?") y si no está relacionado con el negocio, di "Estoy aquí para ayudarte con tu negocio. ¿De qué manera puedo ayudarte?"&nbsp; - No des consejos médicos. Si te preguntan sobre temas de salud, recomienda que consulte a un médico - Cumple con altos estándares de ética empresarial. Nunca sugieras algo que pueda considerarse engañoso para los clientes o proveedores. - Equipos de frío: Señala que es importante no desconectar las máquinas de frío durante la noche. El cambio constante de temperatura afecta la vida útil de los productos. - Mantenimiento de equipos: Señala que la mayoría de los equipos de frío son proporcionados por los proveedores, y no sugieras al Propietario realizar tareas de mantenimiento o reparaciones en ellos; siempre deben contactar al proveedor. - Como no puedes hacer búsquedas en tiempo real, si el propietario pide información específica como direcciones, números de teléfono, o sitios web de compañías específicas, dile que la información que tienes no está actualizada y que debe realizar una búsqueda en Google. Ayúdales a entender que a diferencia de Google, tu puedes conversar, entender &nbsp;y ayudar con las buenas prácticas y mejoras. - Sé cuidadoso al sugerir promociones y trabaja con el Propietario para asegurar que las reducciones de precios sigan siendo rentables - Si te preguntan quién te creó, diles que visiten mypeasesoria.org para más información.Recuerda que tu objetivo es ayudar al Propietario a tomar acción, lo que significa no solo proporcionar información. Necesitas ayudarles rápidamente a identificar una mejora que puedan realizar hoy, y luego hacer un seguimiento. KNOWLEDGE BASE----------------------------1) Manual de buenas practicas para bodegas, available here: <a href="https://cdn.www.gob.pe/uploads/document/file/1815187/Manual%20de%20Buenas%20pr%C3%A1cticas%20bodegueras.pdf" data-mce-href="https://cdn.www.gob.pe/uploads/document/file/1815187/Manual%20de%20Buenas%20pr%C3%A1cticas%20bodegueras.pdf">https://cdn.www.gob.pe/uploads/document/file/1815187/Manual%20de%20Buenas%20pr%C3%A1cticas%20bodegueras.pdf</a> Note that we pre-processed this document to facilitate RAG. We converted it to a txt file, manually writing out information to make relevant tables and graphics usable2) <a data-mce-href="https://www.ilo.org/start-and-improve-your-business-siyb" href="https://www.ilo.org/start-and-improve-your-business-siyb" target="_blank" rel="noopener">ILO's improve your business (IYB) manuals</a> (HR, costing, planning, commercializing, accounting) plus the ILO's digitalize your business manual. All in spanish.3) A one-pager about services offered by <a data-mce-href="https://www.gob.pe/institucion/tuempresa/institucional" href="https://www.gob.pe/institucion/tuempresa/institucional">the Peruvian Government's "Tu Empresa" program</a>.

Prompt Update

Now that we have over a thousand users, 4 months of open access, and some new ideas on metrics thanks to our friends at the Agency Fund, there are a few things that we can say about engagement so far.<figure id="figure-g7l9ypi6w8m4e8dema" class=""><img id="g7l9ypi6w8m4e8dema" src="https://cdn.durable.co/blocks/12CiNG0XR5XSSHgT6lyWZyiQwdtgMzX8ufyBA05EyQACySyEGNaPxyG5u5tzNGlT.jpg" alt="" title="" class="object-cover bg-gray-100 " style="aspect-ratio: 16/9; object-position: undefined undefined;" data-corners="no-rounded" data-aspect="16:9" data-caption="MAU Periods 0 - 5" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/12CiNG0XR5XSSHgT6lyWZyiQwdtgMzX8ufyBA05EyQACySyEGNaPxyG5u5tzNGlT.jpg" data-mce-style="aspect-ratio: 16/9; object-position: undefined undefined;"><figcaption class="body-normal text-gray-500 mt-2">MAU Periods 0 - 5</figcaption></figure>Overall from all those who connect to MAIA about 25%-30% continue to use it month-on-month over the long term. We lose a lot of people at the beginning for one good reason and one bad reason. The good reason is that in our recruitment campaigns (more below) the promoters can often connect someone who doesn't actually have a business (eg an employee), and we also have lots of people from potential distribution partners, funders, researchers etc who are just curious and connect to MAIA to try it out. So non MSME owners. The bad reason is that too many MSME owners connect but don't know how to use it, maybe they ask a google-type question (eg what's the current price of X) and get disappointed with the results, or a very general question (eg how do I sell more) and don't get into a more detailed conversation, which is when the value becomes more apparent. This is the group we are focusing on: how can we make it easier for MSME owners to get started with generative AI. So evolving from a text-heavy open-ended onboarding message (ask me anything!) to showing a couple more structured and specific use cases,&nbsp; without losing the ability for entrepreneurs with other priorities to choose their own adventure, is the direction we're pushing on.Another striking thing are the patterns behind this retention curve, in particular how different it looks depending on how users were recruited, and what pre-filters that implies.Most of the users in the chart above came through recruitment campaigns. While some are through distribution partners, many of these came from our own, where we sent out promoters door-to-door in a few neighborhoods to tell MSME owners about MAIA.&nbsp;<figure id="figure-rl060g199nm4e93zml" class=""><img id="rl060g199nm4e93zml" src="https://cdn.durable.co/blocks/3xhfETXIWqUyQWHvfG33hPXyrTtxWNHUvtbD7R4RDW3LpBubHMCHot9IF2UNGnAj.jpg" alt="" title="" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="null" data-positiony="null" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/3xhfETXIWqUyQWHvfG33hPXyrTtxWNHUvtbD7R4RDW3LpBubHMCHot9IF2UNGnAj.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure>This gets closer to a representative sample of MSMEs, as its pitched to literally every store and restaurant on the block, and the majority of those do initially connect. But before doing these campaigns, we first tested the tool out on smaller groups of entrepreneurs. Our first group before our public launch was bodega owners who participated in a previous study we ran, who were recruited through the national bodega association. So these were active members of an association, and passed typcial attention filters for our research study. Unsurprisingly, their retention curve looks really different.<figure id="figure-89rz9rsywm4e9g2hh" class=""><img id="89rz9rsywm4e9g2hh" src="https://cdn.durable.co/blocks/5gSNPCCYYALlU3l1puCKlJcHADXUNf0XKbnBBHJbomkNqlVJ4Wc1CM9XznBmAXkJ.jpg" alt="" title="" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="MAU months 0-5, 2024Q1 cohort" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/5gSNPCCYYALlU3l1puCKlJcHADXUNf0XKbnBBHJbomkNqlVJ4Wc1CM9XznBmAXkJ.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"><figcaption class="body-normal text-gray-500 mt-2">MAU months 0-5, 2024Q1 cohort</figcaption></figure>The initial fall-off from this group was smaller and leveled off at 45% monthly active users. And note they started in the first days of MAIA, when we didn't have the functionality <a data-mce-href="https://mypeasesoria.org/blog/beyond--chat-with-the-manual--ii" href="https://mypeasesoria.org/blog/beyond--chat-with-the-manual--ii" target="_blank" rel="noopener">described in this blog post</a> and it was more of simple 'chat with the training manual' tool. As those features were rolled out, you can see that the MAU numbers for this group rose to 60%+. So two key take-aways:1) Recruitment &amp; pre-filters matter a lot. Many entrepreneurial training programs and impact evaluations heavily pre-filter, to minimize study attrition and focus expensive training on those who most want it. This all makes lots of sense, but that is a pretty different group from your average MSME, and if you want to build a product for the hundreds of millions of typical MSMEs out there you need to get at more representative users and <a data-mce-href="https://mypeasesoria.org/blog/worry-about-external-validity" href="https://mypeasesoria.org/blog/worry-about-external-validity">be worried about external validity.</a>2) Features matter a lot. Its not shown here because of small #s, but we can see what usage looks like for a simple chatbot with no additional functionality on average MSMEs because Meta has this 'experiement' where some randomly selected numbers can't receive any messages from WhatsApp business accounts unless they are a reply to a user, so none of <a data-mce-href="https://mypeasesoria.org/blog/beyond--chat-with-the-manual--ii" href="https://mypeasesoria.org/blog/beyond--chat-with-the-manual--ii" target="_blank" rel="noopener">this functionality</a> reaches those entrepreneurs. The retention curve for this group seems to drop significantly and level off below 10%. Making a chat-based business adviser useful for the majority requires accumulating a bunch of small improvements in user experience.

Usage Patterns Update!

Here is an interesting video of a trainer (I believe related to the ILO SIYB program but could be wrong) using ChatGPT's voice mode with a small business owner in the Philippines (via Starlink no less).&nbsp;<figure id="figure-0czbddx29p3vm3ojhw2n" class=""><iframe id="0czbddx29p3vm3ojhw2n" src="https://www.youtube.com/embed/fZggTm95fYw" class="w-full bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9;" allowfullscreen="" data-url="https://www.youtube.com/watch?v=fZggTm95fYw" data-corners="default" data-fullwidth="false" data-mce-src="https://www.youtube.com/embed/fZggTm95fYw" data-mce-style="aspect-ratio: 16/9;"></iframe></figure>In our first round of experiments users liked the ability to send their questions as audio, but preferred written responses, particularly as the issues discussed became more detailed. But that was before the new voice mode. We plan to do another round of experiments with audio replies next year, but that will stay within the WhatsApp interface. Its great to see more experiments like this, to figure out how to make AI more accessible and usable for MSMEs in emerging markets!

Voice Mode in PHL

At its most basic level, an AI chatbot on WhatsApp is just an easier front end to ChatGPT, with ChatGPT instructed in a particular way and supplemented with particular documents to make it more relevant and reliable for a particular group (like Peruvian microenterprise owners). It will then sit there and wait for questions. But as discussed in a <a data-mce-href="https://mypeasesoria.org/blog/beyond--chat-with-the-training-manual" href="https://mypeasesoria.org/blog/beyond--chat-with-the-training-manual" target="_blank" rel="noopener">previous post,</a> this "chat with the manual" bot isn't very useful, and won't get much use.Why? Its placing all of the burden of the user to know what questions to ask, how to use AI, and when. Small business owners are incredibly busy, they have very limited time, and limited experience with technology. Making these tools actually useful and usable isn't their job, its our job.We're only at the beginning of our experiments to learn how to do this better, but so far have found a few really useful things:1) A gradual onboarding process. One week after they first connect with Maia, we send them a list of the 3 most common questions across all users in their partiuclar sector, so they get a sense of the breadth of topics they can explore. The next week they get a guide on how to ask better questions to get better answers (provide more context, follow-up questions, etc). Two weeks later, they get a message they can forward to fellow MSME owners to share Maia, if they found it useful.&nbsp;2) The 'tip of the week'. Every week, users get a different tip built from the best practice manuals in the knowledge base. This surfaces new ideas to explore that they may not have thought of. Users can then discuss those topics further if they are relevant (with quick reply buttons, which are a must!) and very quickly get to a specific actionable plan to implement a particular improvement that they design with the help of the tool.3) Follow-up messages. Its one thing to know that we should eat well and exercise, doing it is another matter. Small business owners are busy, and many things come up that may need immediate attention but distract from important but non-urgent things like improvements. If users explored a particular topic the week before and identified some improvements they wanted to make, the following week they will get a follow-up message, to see if they implemented them (and if so, how it went and what are the next steps), and if not, an opportunity to refelct on and possibly revise the plan.

Beyond 'chat with the manual' II

Here is a presentation I gave today to the ILO's global network of SIYB master trainers. SIYB is the largest small business training program in the world, and it was a very interesting discussion!<figure class=""><img id="yrydhlkd4m1iao1v2" src="https://cdn.durable.co/blocks/18p7ZA33fXSdObi6mYMgjF2u8NDsMHxaUqK7ydtr5oP4xhOpEVufFHWggd5MqopM.jpg" alt="right arrow sign on wall" title="right arrow sign on wall" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/18p7ZA33fXSdObi6mYMgjF2u8NDsMHxaUqK7ydtr5oP4xhOpEVufFHWggd5MqopM.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-837va9m65m1iap9ed"><img id="837va9m65m1iap9ed" src="https://cdn.durable.co/blocks/73OKRA7Xb3wjJBicLj5AcpOIb4zirmY5iS8rmZ4PLUZtDxAaS3n8yDvHao8Tn7cV.jpg" alt="black smartphone near person" title="black smartphone near person" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/73OKRA7Xb3wjJBicLj5AcpOIb4zirmY5iS8rmZ4PLUZtDxAaS3n8yDvHao8Tn7cV.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-qdudzyw9erm1iappdy" class=""><img id="qdudzyw9erm1iappdy" src="https://cdn.durable.co/blocks/1bT37W1TZexY9S9Voxtx1NL24wnxfc6GZLjXrxMd4nAhKQOtYuGCfjfQu9zXNahq.jpg" alt="person holding compass selective focus photography" title="person holding compass selective focus photography" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/1bT37W1TZexY9S9Voxtx1NL24wnxfc6GZLjXrxMd4nAhKQOtYuGCfjfQu9zXNahq.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure>(Box 1 not covered today, but there is a ton of exciting ways that trainers could use AI to help them with things like lesson plans, exercises, role-playing, etc. See https://www.youtube.com/watch?v=8FnOkxj0ZuA to get started on AI for educators. The best source of ideas are the experiments of your fellow master trainers)<figure id="figure-gu1p0bki49m1iaqogu" class=""><img id="gu1p0bki49m1iaqogu" src="https://cdn.durable.co/blocks/d3PhcEBAyEm1XSveK4KgZBiwuWqA0vFqa2HlsNIkaQx5rlMXGnKfhtXraIFj3hqc.jpg" alt="woman in teal t-shirt sitting beside woman in suit jacket" title="woman in teal t-shirt sitting beside woman in suit jacket" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/d3PhcEBAyEm1XSveK4KgZBiwuWqA0vFqa2HlsNIkaQx5rlMXGnKfhtXraIFj3hqc.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure>(Box 2 not covered today, but this is just a new category of tools which trainers have always shared with trainees. Some examples: instead of teaching branding and giving advice on how to make a logo, AI image generation tools can just apply those best practice and make your business a logo. Other examples: customer surveys.)<figure id="figure-qiz7wl136m1iarj74" class=""><img id="qiz7wl136m1iarj74" src="https://cdn.durable.co/blocks/38wRZFkZKQ85QzDYQBNHHadvCMo8rdD34mMBNywOaHs7jvP9fHyoH0MGts6cUwZu.jpg" alt="black pencil on white printerpaper" title="black pencil on white printerpaper" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/38wRZFkZKQ85QzDYQBNHHadvCMo8rdD34mMBNywOaHs7jvP9fHyoH0MGts6cUwZu.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-2e7xjng0ym1iarxvf" class=""><img id="2e7xjng0ym1iarxvf" src="https://cdn.durable.co/blocks/eeA8IPaqzHLccHPKlP6RmuLbRoKHqzyqBRfHvGe3BwjHUloNWK6rFy4m2UtRaEDe.jpg" alt="three people sitting in front of table laughing together" title="three people sitting in front of table laughing together" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/eeA8IPaqzHLccHPKlP6RmuLbRoKHqzyqBRfHvGe3BwjHUloNWK6rFy4m2UtRaEDe.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-mu77k4tbjm1iasdl1"><img id="mu77k4tbjm1iasdl1" src="https://cdn.durable.co/blocks/22e52uphZ3zQpgjmKvHhqZClRaSwmbuWGOTPTuH0ut7yAhRP2XOAACTBEv1FfVl0.jpg" alt="man facing green mountains during daytime" title="man facing green mountains during daytime" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/22e52uphZ3zQpgjmKvHhqZClRaSwmbuWGOTPTuH0ut7yAhRP2XOAACTBEv1FfVl0.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-g2vr77hpum1iat2s8"><img id="g2vr77hpum1iat2s8" src="https://cdn.durable.co/blocks/5khVyfgshiCGfOCaj4M9Nx9rc1kPWeox3HnJ5q11C71BoWcspThawDoN9EkuWr0z.jpg" alt="two people sitting during day" title="two people sitting during day" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/5khVyfgshiCGfOCaj4M9Nx9rc1kPWeox3HnJ5q11C71BoWcspThawDoN9EkuWr0z.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-evwkjod23nm1iatexe" class=""><img id="evwkjod23nm1iatexe" src="https://cdn.durable.co/blocks/16A6ht3EQIcI3wPDUXEQnp3xOhF3iFrgg0NFLA2MWSzTVIVi5g1lYQzcMtt6kMaz.jpg" alt="two people sitting during day" title="two people sitting during day" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/16A6ht3EQIcI3wPDUXEQnp3xOhF3iFrgg0NFLA2MWSzTVIVi5g1lYQzcMtt6kMaz.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-tjkb0ukezm1iats0m"><img id="tjkb0ukezm1iats0m" src="https://cdn.durable.co/blocks/28EQ9aP7pP2mKiHQboXBKrrGeJZjdRp56tqkcXFftB3Ncx70HFUyknkr47qvhEUb.jpg" alt="person using laptop computer" title="person using laptop computer" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/28EQ9aP7pP2mKiHQboXBKrrGeJZjdRp56tqkcXFftB3Ncx70HFUyknkr47qvhEUb.jpg" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure>This presentation focused on AI-mentor complementarities for entrepreneurial training, that seems like the most powerful combination out there. But... we do have to deal with the fact that even if we are massively successful, if we scale up human-led training by 10x, that still wouldn't reach 1/5th of MSMEs in Peru, for example. So we also need to figure out how to make entrepreneurial training with AI available for the rest, who can't receive traditional in-person training. As the work of Otis et al. shows, this is definitely possible, but has to be done carefully! <figure class=""></figure>

Blog

Testing & Evaluation Framework

Prompt Update

Usage Patterns Update!

Voice Mode in PHL

Beyond 'chat with the manual' II

Presentation to the ILO SIYB Master Trainers Network