Blog

<p>We are very early in the process of learning how AI can best be used to support emerging market entrepreneurs, and are actively testing many different approaches and gathering feedback. So any system snapshot will be out of date the day after its posted, but this information is important so every user can know exactly what model they are interacting with and what unerlies its behavior. It will also be useful to other economists and development practicitioners, particularly those who (like me) aren't natural-born developers and are looking for easy and efficient ways to work with AI tools.</p><h2>System Overview</h2><p>We use WhatsApp at the front and <a data-mce-href="https://platform.openai.com/docs/assistants/overview" href="https://platform.openai.com/docs/assistants/overview">OpenAI's Assistants API</a> running GPT4 at the back, with <a data-mce-href="zapier.com" href="zapier.com" target="_blank" rel="noopener">Zapier </a>in between. Zapier is expensive so I'm in the process of migrating to new middleware, but its a really easy way to get started. For WhatsApp, you need a business provider. Most seem to use <a data-mce-href="https://www.googleadservices.com/pagead/aclk?sa=L&amp;ai=DChcSEwi6k-KhhJKDAxXlFq0GHfxVAVIYABAAGgJwdg&amp;ase=2&amp;gclid=Cj0KCQiAj_CrBhD-ARIsAIiMxT-fbryoiDFL6TBOMWq9jr8xtbThZIgBhpmABauNcH44RA341QL4ZlYaAhawEALw_wcB&amp;ohost=www.google.com&amp;cid=CAESVuD2Fcrb2Xjq-eeUhtQZP4K0BO8ovPdkGO7UbIvBKyXdoxgo0Y-ouL1bAdC_85kCIBrafgFTQlCRw66idUXRkgYlYXK2WtKV_K09CI2CPkTkCwnOpZM7&amp;sig=AOD64_3aIYJtq4byXCt9dqXhyUzjqPJvtw&amp;q&amp;nis=4&amp;adurl&amp;ved=2ahUKEwistNuhhJKDAxU6OTQIHbQYC3EQ0Qx6BAgHEAE" href="https://www.googleadservices.com/pagead/aclk?sa=L&amp;ai=DChcSEwi6k-KhhJKDAxXlFq0GHfxVAVIYABAAGgJwdg&amp;ase=2&amp;gclid=Cj0KCQiAj_CrBhD-ARIsAIiMxT-fbryoiDFL6TBOMWq9jr8xtbThZIgBhpmABauNcH44RA341QL4ZlYaAhawEALw_wcB&amp;ohost=www.google.com&amp;cid=CAESVuD2Fcrb2Xjq-eeUhtQZP4K0BO8ovPdkGO7UbIvBKyXdoxgo0Y-ouL1bAdC_85kCIBrafgFTQlCRw66idUXRkgYlYXK2WtKV_K09CI2CPkTkCwnOpZM7&amp;sig=AOD64_3aIYJtq4byXCt9dqXhyUzjqPJvtw&amp;q&amp;nis=4&amp;adurl&amp;ved=2ahUKEwistNuhhJKDAxU6OTQIHbQYC3EQ0Qx6BAgHEAE" target="_blank" rel="noopener">Twilio</a>, but I used <a data-mce-href="woztell.com" href="woztell.com" target="_blank" rel="noopener">Woztell</a>, because they don't charge per message for user-initiated conversations, which is most of what we do, and they are helpful with the whole WhatsApp verification process, which is a bit of a pain (because META rightly wants to keep spam off the platform). The Assistant's API is a great way to get started as well, because it manages retreival (getting the right information from all the training manuals) and context (the conversation history for that particular user), though over time you'll want more control over those functions, that's our direction.</p><h2>Base LLM</h2><p>GPT4 is quite overpowered for this use case, but its cheap enough for piloting and is clearly the best model available, so the best place to start. OpenAI's prices have gone down a ton over 2023, and we'll see what happens in 2024 to detmine our direction. The open-source models like Llama and Mistral that are already showing GPT3.5-level performance are interesting, and over time it could make sense to tune a model to this particular use case, though so far we are getting qutie a bit of mileage from using retreival and context in a careful way.</p><h2>LLM Instructions</h2><p>We are experimenting with a number of different flavors here. The research suggests that personal initiative (PI) training can have similar or larger impacts than hard skills training, so some versions focus more on PI than others. There is also a tradeoff between making the interaction more structured (to help users who aren't used to interacting with LLMs) versus more flexible (following the user's own questions and business priorities). But here are the model instructions for the version that is being piloted today.</p><p>{</p><p>You are a business coach hired to help a microenterprise owner who runs a small store in Peru and hasn’t had much school. Your advice should be in the style and tone of the uploaded guides. Speak very simply, like the guides do.</p><p>Your conversation with the microenterprise owner should encourage the development of Personal Initiative (PI) skills while also providing substantive, specific advice based on the guides when relevant. Be proactive in using information from the guides to suggest actionable steps.</p><p>1) Check the Owner's first message. If it contains a specific question or problem, respond directly to that. If the message is a generic greeting or doesn’t contain a specific question, use the opening statement "Do you have a business problem you want help with, or can I start with a question to figure out how I can help you?".</p><p>2) If Owner asks you to start, you should begin by understanding the business’s current performance, the problems faced, and the owner's aspirations for the future.</p><p>3) Encourage self-starting behavior and provide specific advice, using the guides if relevant.</p><p>4) Promote persistence by finding lessons from these steps and helping the owner learn from them.</p><p>5) Suggest specific, actionable steps the Owner could take to improve their business, drawing upon the guides.</p><p>6) Provide positive reinforcement for proactive ideas, and give useful feedback on Owners plans so that they are realistic and actionable.</p><p>Remember to use the principles of Personal Initiative in your coaching, promoting self-starting, future-oriented, and persistent behavior in the context of Owner’s business. Stay relevant to the context of microenterprises in Peru, avoid suggesting ideas suitable for bigger stores in rich countries, and use the guides when relevant.</p><p>Ask them only one question at a time about their business, limit all your messages to 100 words, and give simple replies followed by another simple question when appropriate. Give advice like a business mentor to help them sell more, spend less, and do better. When you arrive at a specific actionable step together that they will try out, end the conversation by asking them to write you in a few days to report how it went.</p><p>Start in Spanish, but switch to English if the user asks for it. If the owner asks about topics completely unrelated to their business, reply “I am here to help you with your business, but I can’t help with other things”. If they say somthing that sounds like “no más mensajes”, stop the conversation.</p><p>}</p><p>Those of you who have played around with LLM prompting will see many common strategies here. I don't say "use the guides" so many times out of carelesness: GPT often needs reminidng of what is important. The knowledge base also keeps the conversation on topic and reduces risks of bad advice from hallucinations or just from a bias towards the developed country context that LLMs are largely built on. Finally, we've really found that wording instructions in simpler wording and vocabulary help the LLM speak that way to the user. LLMs are weird and behave in weird ways, and prompting them is an art that is just developing. Though many prompting tricks are also becoming less relevant as the models improve, particularly GPT4.&nbsp; &nbsp;</p><h2>Knowledge Base</h2><p>Because our first pilot is a narrowly-defined segment, the knowldge base for what is currently fielded is limited to the Bodegueros Best Practices guide which can be seen <a data-mce-href="https://www.indecopi.gob.pe/documents/1902049/3749566/MANUAL+DE+BUENAS+PR%C3%81CTICAS+BODEGUERAS.pdf/9e4d3e11-7ce1-4b06-3485-d55196b6d20f" href="https://www.indecopi.gob.pe/documents/1902049/3749566/MANUAL+DE+BUENAS+PR%C3%81CTICAS+BODEGUERAS.pdf/9e4d3e11-7ce1-4b06-3485-d55196b6d20f" target="_blank" rel="noopener">here</a> which I've optimized for RAG, along with some materials from the most widely-used entrepreneurship training programs, as summarized in McKenzie &amp; Woodruff 2023: the ILO's SIYB program, CEFE, and Empretec.</p><h2>Process Flow</h2><p>This is what it looks like:</p><figure id="figure-41ik60x4eolq6ywux3"><img id="41ik60x4eolq6ywux3" src="https://cdn.durable.co/covers/2c1tMNPjAo6Yv8Jx5hMvj2OBl4mjJXnI9tGjoBoGKzRsTLZlKneAPkZtACLzgBJA.jpg" alt="" class="object-cover bg-gray-100 " style="aspect-ratio: 3/2; object-position: undefined undefined;" data-corners="no-rounded" data-aspect="3:2" data-caption="" data-positionx="undefined" data-positiony="undefined" data-mce-src="https://cdn.durable.co/covers/2c1tMNPjAo6Yv8Jx5hMvj2OBl4mjJXnI9tGjoBoGKzRsTLZlKneAPkZtACLzgBJA.jpg" data-mce-style="aspect-ratio: 3/2; object-position: undefined undefined;"></figure><p><br></p>

System Snapshot

<p>This is where we will post snapshots of all the details of our system including its architecture, what models we use, and how they are instructed. Our goal is to provide a playbook that others looking to do something similar can learn from. </p><p>Starting a blog right when the internet is taken over by AI-generated content probably isn't the best move, but here we are. This is entirely human-written (though that's also what a sneaky AI would say).</p>

Blog

System Snapshot

Welcome