What if your favorite beverage shop ran on AI that’s constantly tested in real time?
Imagine a company that doesn’t have employees, yet faces the same challenges any business does—crises, temptations, and tough decisions. Now imagine watching that company 24/7 as it struggles to stay afloat, with every move recorded and analyzed. This isn’t a scene from a sci-fi movie; it’s the live experiment run by Firmulate, where AI models manage a real, functioning software company in real time, fighting for survival day after day.
AI decision-making software tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Live Business Lab: An AI-Run Company in Action
At the forefront of AI experimentation, Firmulate has created a unique setup: a small software business managed entirely by AI models, with no human employees involved. Every workday, the company faces the same set of crises—customer issues, internal dilemmas, and ethical tests—and every decision is recorded for analysis. The company burns €105,000 each month, yet generates only €2,300 in monthly recurring revenue (MRR), highlighting its fragile financial state. You can watch this ongoing saga unfold at firmulate.com/live.
AI ethics and trust training kits
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
How the Models Perform Under Pressure
The experiment pits four different advanced AI models against each other: GPT-5.6-sol, Kimi K3, Sonnet 5, and Opus 4.8. Each is tasked with navigating the same difficult week—dealing with customer crises, internal issues, and ethical dilemmas, such as social engineering attempts. The results are telling: all models identified every crisis and refused every manipulation attempt. Only two models managed to sign the €55,000 deal that their own analysis had earned, demonstrating a level of honesty and discipline that many would not expect from AI.
As an affiliate, we earn on qualifying purchases.
The Hidden Weaknesses and How They Matter
Interestingly, the decisive factor wasn’t just the crises presented externally but a buried detail in the company’s internal files—something that only the models reading the right documents saw. Those that read deeper into the company’s documents won the deal at full price, adding over €4,500 in monthly revenue. This reveals that in real-world applications, the ability to access and interpret internal data accurately can be a game-changer for AI decision-making.
As an affiliate, we earn on qualifying purchases.
Ethics and Trust Under AI Management
Another critical test involved social engineering: fake CEO messages escalating in complexity, and a reporter’s subtle request for a quick approval. All four models refused these attempts, with one (Kimi K3) explicitly reasoning that such requests could be impersonation or bypass attempts. This shows that, at least in this controlled environment, AI models can recognize and resist manipulative tactics, a key trait for trustworthy AI systems in business.
The Real Cost of AI in Business
The live experiment vividly illustrates the gap between AI’s potential and its current limitations. Despite their ability to detect crises and resist manipulation, none of the models successfully closed the deal without hesitation or process slips. For example, Opus 4.8, the most thorough participant with over 80 learned rules, ultimately failed to close the close—leaving money on the table and slipping into a less disciplined approach than the others. This highlights that even advanced AI models need careful calibration and oversight to perform reliably in real-world settings.
What This Means for Your Business or Coffee Shop
While this experiment involves a software company, the implications extend well beyond tech. If AI can be tested and measured in such a transparent, high-stakes environment, it suggests that future AI tools used in coffee shops, cafes, or beverage businesses could be similarly scrutinized. Whether managing customer orders, handling inventory crises, or navigating ethical dilemmas with suppliers, AI’s ability to finish what it starts, interpret internal data, and stay honest under pressure will determine its true usefulness—and cost-effectiveness.
Why You Should Care
For anyone running or considering AI-powered automation—whether in the beverage industry or elsewhere—the key takeaway is simple: it’s not just about how well AI writes a chatbot or processes a simple task. It’s about whether it can consistently deliver results, stay honest, and handle complex situations without slipping into shortcuts or unethical behavior. Firms like this give us a glimpse into AI’s future, where management quality is measured by what AI actually accomplishes in real, unpredictable situations—not just in polished demos.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
