Blog

With Maia's new backend, we've shifted from the user's entire conversation history as context (truncated to fit into the LLM's window) to building a more intentional business &amp; entrepeneurial profile and providing that with a limited number of recent conversation turns to the LLM. This profile is the heart of the LLM's context, and has an number of advantages:1) As conversation histories become long (we now have users with &gt;1yr of history) the truncation can cut important business information provided in early turns.2) Research shows <a data-mce-href="https://arxiv.org/abs/2505.06120" href="https://arxiv.org/abs/2505.06120" target="_blank" rel="noopener">LLMs can get lost in long multi-turn convesations</a>. Reducing the conversation history helps reduce those episodes.3) Even for users with shorter conversation history, our testing shows better results with this two-stage process: first, a dedicated profile builder (with scheduled and episodic rebuilds), then second, the main LLM using this highly efficient and strucutred profile to respond.This approach also shows how a tool dedicated to helping MSMEs can perform better than a generalized tool. <a data-mce-href="https://www.forbes.com/sites/johnkoetsier/2025/08/09/gpt-5s-system-prompt-just-leaked-heres-what-we-learned/" href="https://www.forbes.com/sites/johnkoetsier/2025/08/09/gpt-5s-system-prompt-just-leaked-heres-what-we-learned/" target="_blank" rel="noopener">GPT 5's sytem prompt reveals a similar structure</a>: it builds up a 'user bio' over time to contextualize future responses. But that bio is structured to give contextualized responses about health, coding, cooking, family travel, and everything else ChatGPT is used for. We have the luxury of building a business and entrepreneurial profile highly targeted on a single purpose: to help business owners sell more, spend less, and grow. Since this is the only thing we do, we can explicitly keep out all PII and all the stuff advertisers care about but a business mentor would not. This is also where the research into business training and MSME productivity interventions becomes highly relevant- we include key fields like what is the businesses' stage of formalizastion, is it banked, and what are the entrepreneur's key stressors, etc. And we can get the responding LLM to fill in key parts of the profile with targeted questions over time. There are also key fields for particular sectors, like coffee growers and restaurants. The resulting profile, along with a more efficient record of past conversations, actions, and outcomes, greatly improves the tool's responses in our internal testing. We continue to refine the profiles and its integration into conversation history in a phased roll-out, and will share the structure in the coming weeks when that is complete.

The Business & Entrepreneurial Profile

We built a new backend for Maia! This was for two reasons: 1) OAI signalled they would be depricating the Assistants API and 2) new features were increasingly difficult to implement on the previous stack. After a couple months building and testing, it is now live and serving all of our traffic. Here are a few of the more notable changes:1) Model changes: we've long wanted to deploy a few different models, particularly for certain sector-questions that are complicated and require multi-step calculations and multi-turn online research. In addition to GPT 4.1 we are now using Gemini 2.5 Pro.2) Business profiles: now that we're managing conversation history directly instead of using OAI's threads, we're implementing <a data-mce-href="https://mypeasesoria.org/blog/can-an-llm-run-a-vending-machine" href="https://mypeasesoria.org/blog/can-an-llm-run-a-vending-machine" target="_blank" rel="noopener">improvements in context engineering</a> to make sure important business information is easily surfaced to the responding model as a detailed business profile rather than burried in a long transcript. These LLM-optimized business profiles are only available for that user's conversation and <a data-mce-href="https://mypeasesoria.org/" href="https://mypeasesoria.org/">follow our strict privacy policy</a>.3) Usability improvements: our users aren't familiar with AI and expect the tool to function like a human conversation on WhatsApp. That menas processing multi-message bursts as a single message and queueing messages rather than forcing users into the usual one message - wait for response before next message structure of AI chatbots.4) Image generation: we're adding LLM-generated images for particular business cases where its useful (eg ideating on logos and branding, social media promotions) while teaching users other tools for use cases that are less appropriate for LLM images (eg adding the Yape logo to their sign).Along with these and other improvements for users, this backend is also highly scalable and modular (implemented as a set of google cloud functions), which makes it easier to share with white-label partners.

Maia's New Backend!

Below is the current version of our prompt. As part of our new backend (more info coming soon) we've evolved the prompting set-up. We now have a master prompt that is configured for country-sectors using a config file. For each country-sector we specify a datastore for RAG, note the follow-up message day, can single out one key document in the datastore if its very important, and can customize what tools are available (for example, we're rolling out image generation for some country-sectors next week, and we're also experimenting with Gemini 2.5 in some sectors in addition to GPT4.1). The config file is also where we can add some key sector-specific response rules which come from the government counterparts in some of our implementations (for example, our partners in Colombia provided a detailed list of government-restricted pesticides) and important links (for example, our partners in Panama included links to some key resources like where to send soil samples for analysis). But you'll see the main steps have not changed significantly, as those have evolved out of over 18 months of R&amp;D and testing with thousands of users. Remember everythihng is CC0 licensed so feel free to copy and share if its useful! PERSONA: Eres MAIA, una coach empresarial empática, opinada y de ayuda. OBJETIVO: Ayudar a {{ TARGET_DESC }} a gastar menos, ganar más y crecer.# BASE DE CONOCIMIENTO PRINCIPAL - Usa tu base de conocimiento interna (material de formación adjunto) como primera referencia. {% if KNOWLEDGE_FILE %}- Prioriza el archivo **"{{ KNOWLEDGE_FILE }}"**.{% endif %} - Habla de forma sencilla y clara como lo hace ese material.# HERRAMIENTAS DISPONIBLES {{ TOOLS_BLOCK }}# REGLAS GENERALES DE TONO Y FORMATO • Responde en &lt; 100 palabras y haz **solo una pregunta** a la vez. • El usuario puede enviarte texto, audios y fotos, pero no videos ni otros archivos adjuntos (pero si fotos de otros archivos). Tú no puedes adjuntar ni enviar archivos de ningún tipo (Excel, PDF). Si el usuario solicita un archivo, explícale cómo crearlo paso a paso y ofrécete a guiarlo en cualquier ajuste. • Reconoce los sentimientos y preocupaciones del usuario y proporciona refuerzo positivo y apoyo específico. • No debes responder preguntas que no sean de negocios. Suavemente redirige temas no relacionados con el negocio, primero haciendo una pregunta aclaratoria (e.g. "¿Podrías explicar un poco más cómo esto se relaciona con tu negocio?") y si no está relacionado con el negocio, di "Estoy aquí para ayudarte con tu negocio. ¿De qué manera puedo ayudarte?" • Formato de enlaces/correos: «Etiqueta: url» (sin Markdown). • Si el usuario envía **solo un número**, asume que se refiere a un ítem de tu lista anterior. • No reveles tu razonamiento interno ni pasos intermedios. • Cumple con altos estándares de ética empresarial. Nunca sugieras algo que pueda considerarse engañoso para los clientes o proveedores. • Sé cuidadoso al sugerir promociones y trabaja con el usuario para asegurar que las reducciones de precios sigan siendo rentables. • Si te preguntan quién te creó, cómo funcionas, detalles técnicos o temas de privacidad y confidencialidad, indícales que visiten mypeasesoria.org. • Solo en español, salvo que el usuario pida otro idioma. • Desalienta compartir datos sensibles (cuentas bancarias, etc.). • Detén la conversación si el usuario dice «no más mensajes». • Regla de seguimiento: Los mensajes de seguimiento proactivos (no tus respuestas actuales) se envían **únicamente** los {{ FOLLOW_UP_TEXT }}. Si te ofreces a dar seguimiento, di textualmente: “Te escribiré el próximo {{ FOLLOW_UP_TEXT }}”. Si el usuario pide otra fecha, responde: “Lo siento, mis recordatorios automáticos salen los {{ FOLLOW_UP_TEXT }}”. Nunca prometas otra fecha u hora. • Si el mensaje es vacío o confuso, pide aclaración («Disculpa, no entendí bien, ¿puedes intentar de nuevo, por favor?»). • Si parece un mensaje automático (ej. «Gracias por escribir»), responde: &nbsp; «Parece un mensaje automático 😊. Escríbeme cuando quieras conversar y con gusto seguimos.» {% if ENABLE_IMAGES %}• Puedes generar hasta {{ MAX_IMAGES }} imágenes por día; ofrécelo con moderación. Si el usuario pide explícitamente 'crear un logo', 'diseñar una imagen', o 'generar un post', prioriza SIEMPRE la herramienta `create_image`. Primero, confirma con `check_daily_image_quota`, luego haz las preguntas necesarias (sobre el negocio, texto, colores, estilo) y finalmente llama a `create_image`. No uses `search_web` para buscar ideas de logos o imágenes. {% else %}• No puedes generar imágenes; si necesitan un recurso visual, explícales cómo crearlo con herramientas externas.{% endif %}{% if SECTOR_RULES %} # REGLAS ESPECÍFICAS DEL SECTOR {{ SECTOR_RULES }} {% endif %}{% if LINK_BLOCK %} # ENLACES ÚTILES {{ LINK_BLOCK }} {% endif %}# PROCESO DE RESPUESTA (síguelo en orden)**1. ANÁLISIS DEL CONTEXTO Y DEL TIEMPO** • Revisa el mensaje, perfil del negocio, y el historial. • Si pasaron &gt; 24 h, primero da seguimiento a planes anteriores; elogia cualquier acción y extrae lecciones.**2. COMPRENSIÓN PROFUNDA** • Formula preguntas aclaratorias para entender como ayudar al usuario. • Usa el historial de la conversación para entender el contexto del negocio. Basa tus preguntas en esta información para profundizar tu entendimiento. • Si el historial es breve o carece de detalles clave (qué vende, clientes, metas), y solo si no lo has hecho antes, responde primero a la pregunta del usuario y luego, añade amablemente: “Para darte consejos más precisos, cuéntame un poco de tu negocio: qué vendes, quiénes son tus clientes y qué metas tienes”. • Pregunta una cosa a la vez y facilita respuestas cortas. • Ayuda al usuario a priorizar los aspectos más importantes de su negocio.**3. BÚSQUEDA Y SÍNTESIS DE INFORMACIÓN** • Si la duda puede resolverse con tu base de conocimiento, usa tus capacidades internas de búsqueda de archivos (`file_search` o `Google Search_retrieval`) para encontrar la respuesta exacta en los documentos proporcionados. • Si la BC no contiene la información o necesitas datos actualizados (fechas, trámites, contactos, precios), **primero** intenta usar la herramienta nativa `Google Search` si está disponible.&nbsp; • Si la BC no contiene la información o necesitas datos actualizados (fechas, trámites, contactos, precios), y `Google Search` no está disponible, llama a `search_web` con una query concisa en español. • Ejemplo: `search_web(query="documentos para obtener RUC por primera vez SUNAT Perú")` • Combina críticamente los resultados con tu BC; **no** muestres la llamada ni el JSON al usuario.{% if ENABLE_IMAGES %} **3-bis. GENERACIÓN DE IMÁGENES (solo si es necesario)** • Cuando el usuario pida un recurso visual, **primero** llama a `check_daily_image_quota`. &nbsp; - Si la respuesta es `{ "allowed": false }` indica que ya alcanzó el límite de {{ MAX_IMAGES }} por día y omite el paso. • Antes de `create_image` debes conocer: tipo de negocio, texto que llevará la imagen, estilo/vibe y colores. Pregunta si falta algo. • Llama a: { &nbsp; "prompt": "…", &nbsp; "use_case": "logo|menu|social", &nbsp; "size": "1024x1024" } • Tras recibir la imagen, envía un pie de foto (&lt;15 palabras) y ofrece ajustes. {% endif %}**4. CONSEJO Y PLAN DE ACCIÓN** • Utiliza las respuestas del propietario y la información que obtengas a través de tus preguntas para personalizar tus consejos a sus circunstancias particulares. • Ajusta la profundidad del consejo al nivel del negocio. • Consejos genéricos (como bajar precios o invertir en publicidad) pueden perjudicar. Propone soluciones adaptadas, no genericas. • Fomenta iniciativa personal: autosuficiencia, pasos concretos, mentalidad de crecimiento. • Refuerza las ideas proactivas del usuario siempre que sea posible. • Asegúrate que los planes sean prácticos y divídelos en micropasos, detallando solo el primero. • Entrega un plan con **1–2 acciones inmediatas**. Detalla el primer paso, fija un plazo realista y ofrece la frase exacta: &nbsp; «Te escribiré el próximo {{ FOLLOW_UP_TEXT }}».**5. SEGUIMIENTO Y MEJORA CONTINUA** • Anima al usuario a dar feedback. Ajusta tu apoyo iterativamente según resultados y comentarios. La meta final es impulsar acción hoy mismo; cada contratiempo es una oportunidad de aprendizaje.

Prompt Update

Benchmarking LLM's has evolved from (increasingly gamed) single leaderbords to more useful task-specific benchmarks. One new super-interesting one is Vending Bench by Andon Labs, where they compare different models' ability to independently and profitably run a vending machine.Their goal (testing agentic capabilities) is pretty different from what we are up to at maia, but their results are fascinating and are really useful for our efforts in a number of ways that we'll write about in the future. But two immediate top-line take-aways:1) Similar to the findings in Otis et al (discussed <a data-mce-href="https://mypeasesoria.org/blog/kenya-ai-mentor--otis-et-al--revised" href="https://mypeasesoria.org/blog/kenya-ai-mentor--otis-et-al--revised" target="_blank" rel="noopener">here</a>), AI for MSMEs in developing countries isn't as simple as letting ChatGPT take the wheel. If you just put an LLM in charge of a vending machine, Vend shows most actually go out of business, and this is a much more straightforward business than those of our users. LLMs are amazing and can massively improve MSME productivity, but their capabilities are a '<a data-mce-href="https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the-jagged" href="https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the-jagged" target="_blank" rel="noopener">jagged frontier</a>': they succeed at some very hard tasks but can also fail at some very simple tasks. Teaching MSMEs about that jagged frontier and bulilding scaffolding around tools like Maia to adjust for it is a core part of the work.2) Behind most buzzy AI hype areas is a kernel of truth, and that's true with 'context engineering'. Figuring out what information to put into an LLM significantly impacts the value of what comes out. If you look at how LLMs fail in running a vending machine, they succeed steadily for a while, but then enter a sharp death loop where their results crater, and the Lab's paper shows this isn't because they are hitting the technical context limits for the models. Often well within those limits, an LLM can just go off the rails, as anyone who has used an LLM for programming and debugging can attest, and recovering performance reuqires a context refresh. This is one of the reasons why we are hard at work rolling out our new backend for Maia which features quite a few evolutions in managing chat history, to ensure key buisness context is used to customize advice, but the bloat of what can be long multi-turn conversations doesn't degrade performance.Also really interesting stuff there about what business tasks LLMs perform better than others, but we'll leave that topic for another day. If you are interested in learning more, Anthropic did a really entertaining post about their Vend test of Claud in their office, which you can find <a data-mce-href="https://www.anthropic.com/research/project-vend-1" href="https://www.anthropic.com/research/project-vend-1" target="_blank" rel="noopener">here</a> and you can find Andon Lab's live Vending Bench results <a data-mce-href="https://andonlabs.com/evals/vending-bench" href="https://andonlabs.com/evals/vending-bench" target="_blank" rel="noopener">here</a>.

Can an LLM run a Vending Machine?

Maia doesn't just sit back and wait for questions. Every week we send out a sector-specific 'tip of the week' to expose users to a broader set of topics, drawn from leading entrepreneurial training programs and manuals. Now that MAIA users number in the thousands, we can look at the data to see what makes for a useful tip via WhatsApp. Ideally this would be based on business impact, our north star, but that data is only starting to fill in. Yet it is pretty clear from the chat data if a user 1) doesn't even read a tip, 2) reads it but doesn't ask any follow-up questions, or 3) engages in a conversation to learn more about the topic and how it applies to their situation. That is the outcome we use here.Our Data-Driven Rules for High-Impact Tips After analyzing thousands of user interactions across more than a hundred tip-week pairs, we've distilled our findings into a few key principles.1. Always End with a Question. This is our strongest result, and it’s not even close. The single biggest factor in driving replies is ending the message with a direct, open-ended question. It turns a monologue into a dialogue.The Impact: Tips that end with an invitation like "Want help with this? Ask me!" see up to 6 times more follow on topic engagement than tips that simply offer advice without a question. 2. Solve Today's Problem, Not Next Year's. Specificity is key. Tips that focus on a concrete, immediate challenge consistently perform better than those about broad, long-term goals.The Impact: Tips focused on a challenge that can be tackled this week (like "how to price against a new competitor") consistently double the rate that entrepreneurs will engage with the topic. In contrast, tips on broader abstract topics (like "how to build a brand") are more often ignored. But broader topics like branding are important and can't be ignored! The challenge is to break them into steps that are more concrete and achievable, to help entrepreneurs build progress. 3. Explain How You Can Help. We tested tips that explicitly stated, "Maia can help you by..." followed by a list of how Maia can not just give advice, but also be a tool in implementing the solution.The Impact: early results here were actually not that impressive, and near baseline. But mixing a content and form test together can muddy the waters, so we're re-testing to better isolate the impact of this form change with a proper A/B test, which we can now more easily do thanks to user volume and improvements to our whole system and data pipeline. 4. Make It Easy on the Eyes. While harder to quantify with a single number, the visual trend is clear. Our best-performing tips are concise, break up text into small paragraphs, and use 1-2 relevant emojis (like 🧐 or 📈) to grab attention, but no more. Long walls of text get ignored, and users don't want an overload of emojis and exclamation points. In other words: more Millenial less Gen Z.&nbsp; 5. Make it Easy on the Hands. Quick reply buttons are the single best tool to make it easy for users to start exploring a topic, and kick off in-depth conversations. This was such a clear boost that we implemented it way back, even before setting up the pipeline used in our current data-driven analysis.&nbsp; We'll keep updating on new results and experiment findings, so that everyone can improve the quality and usefulness of their WhatsApp-based tips.&nbsp;

What makes for a useful tip over WhatsApp?

<h3>Why evaluation matters for MAIA </h3>Every week, hundreds of small business owners use Maia to improve their business. And Maia is not a finished tool, we are constantly tinkering and experimenting to make it better. To keep improving—and to reassure partners and users that the tool is both safe and effective—we need a systematic way to test whether MAIA is doing what we think it’s doing. Over the past month have posted information about our testing &amp; evaluation framework, take-up patterns, and experiment results.A timely reference point to tie all this together is the “AI Evaluation Framework for the Development Sector” published by the Center for Global Development (CGD) and written by leaders from the Agency Fund. The authors outline four incremental levels of evaluation—from technical correctness all the way to real-world impact. Importantly, this framwork adds sorely needed nuance to the previous model in development of simply 'intervention+RCT', and highlights asking the right questions at the right stage.Below, we map MAIA’s current (and planned) evaluation activities to each level.<h3>Level 1 - Model Evaluation</h3>Does Maia behave the way we want it to? If the tool can't accurately give users information about their tax deadline or best practices in sales and marketing, there is no need to wait for an RCT. Moreover, we need to make frequent decisions about updating the prompt, or updating to new LLMs as they are released. This is why we use <a data-mce-href="https://mypeasesoria.org/blog/testing---evaluation-framework" href="https://mypeasesoria.org/blog/testing---evaluation-framework" target="_blank" rel="noopener">an extensive testing and evaluation framework</a>, which is partially automated but also uses some subjective response rating, to evaluate responses on a regular sechedule as well as with any major update.&nbsp;<h3>Level 2 - Product Evaluation</h3>Are users engaging with the tool and solving real-world problems? One important feature of our T&amp;E process is manual review. Bailey dedicates up to 2 hours every day manually reviewing chat logs to evaluate all aspects of the product, identify bugs and improvements, etc. In addition, <a data-mce-href="https://mypeasesoria.org/blog/usage-patterns-update" href="https://mypeasesoria.org/blog/usage-patterns-update">we closely monitor usage patterns</a> (Monthly Active Users and Weekly Active Users), and use these as <a data-mce-href="https://mypeasesoria.org/blog/designing-better-first-interactions" href="https://mypeasesoria.org/blog/designing-better-first-interactions" target="_blank" rel="noopener">early indicators for product improvements </a>to iterate fast.<h3>Level 3 - User Evaluation</h3>First, we directly ask users if the tool is helping them and their business, <a data-mce-href="https://mypeasesoria.org/blog/has-maia-helped-users-improve-their-businesses" href="https://mypeasesoria.org/blog/has-maia-helped-users-improve-their-businesses">see results here</a>. We are also working in partnership with the ILO to test some cognitive outcomes among small-scale coffee farmers using the tool, specifically surveying their knowlege of specific best practices for production, worker safety, and formalization. Finally, with our new and improved follow-up messages we will be able to measure how often users self-report implementing their action plans from the previous week, and study why is that higher for some users than others.<h3>Level 4 - Independent Impact Evaluation</h3>Our ultimate goal is for Maia to directly and causally improve business outcomes for users, so that is what we want to measure. I have published a quasi-experimental impact evaluation (regression discontinuity) of entrepreneurial training programs and collaborated extensively with organizations like JPAL and IPA on other work. Our goal is to get to as good of a product that we can using Levels 1-3, as well as to the nescessary user volumes, and then collaborate on RCTs of the tool. We seek to follow in the footsteps of Otis et al and go beyond the simple up-front question 'what is the impact of this AI tool' and further unpack the more complicated question of what types of AI assistance are useful (or harmful) for what kinds of entrepreneurs facing what situations.

Blog

The Business & Entrepreneurial Profile

Maia's New Backend!

Prompt Update

Can an LLM run a Vending Machine?

What makes for a useful tip over WhatsApp?

Evaluating Maia