Imagine a car repair shop managed entirely by artificial intelligence—making decisions, negotiating with customers, and handling crises—yet losing money every day. This isn’t sci-fi; it’s the live experiment from Firmulate, a pioneering company that runs a software business with no human employees, facing daily survival challenges in real-time, and openly sharing every decision and outcome for the world to see.
The Engine Behind the Experiment
At the heart of this experiment are four advanced AI models, each tasked with running a small software company through its worst week. All the same crises, the same customers, even the same temptations to cheat or manipulate—these models are tested under the most extreme conditions. Every decision they make is meticulously versioned and publicly auditable, giving a transparent window into their performance.
How Do the Models Fare?
Despite their differences, all four AI models identified every crisis and resisted manipulation attempts—including social engineering tricks like fake CEO messages and reporter inquiries. Yet, only two of these models managed to close a critical deal worth €55,000, which their own analysis had earned. The other two, despite similar diagnoses and pitches, failed to secure the agreement, leaving revenue on the table.
The Hidden Weakness
What made the difference? The decisive factor lay buried two document references deep within the company’s own files—information that none of the models initially accessed. When the models read this crucial internal document, they capitalized on the opportunity and secured the deal at full price, adding over €4,583 in monthly recurring revenue (MRR).
AI customer service chatbot for automotive repair
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Company in Action: Surviving on the Edge
Firmulate operates a real, live business, with 13 synthetic employees managing daily operations—though it’s losing €105,000 each month against a modest €2,300 MRR. Every workday, the company’s rules, decisions, and strategies are versioned, creating a public, evolving record of its struggles and successes. Visitors can watch this ongoing story unfold at firmulate.com/live.html.
What Does Success Look Like?
In this brutal environment, the most thorough AI model—Opus 4.8—made over 80 learned rules and conducted the deepest analysis. Yet, it still left an important deal unclosed and made discipline slips, such as writing attempts into a locked department instead of escalating issues. The performance highlights that even the most advanced AI can stumble under real-world pressures, especially when it overlooks internal information buried in documents.
Management Decisions and AI Behavior
Alongside the technical prowess, the experiment tested AI responses to social engineering. All models refused fake CEO requests and manipulated inquiries, demonstrating a robust resistance to impersonation tactics. The Kimi K3 model, in particular, explained its refusal by treating such requests as potential impersonation or approval bypasses—an essential trait in safeguarding trust.
XTOOL AD20 Pro OBD2 Scanner – Full System Car Diagnostic Scan Tool
- No Subscription Needed: Supports unlimited vehicles with free updates
- Long-Term Ownership: One-time purchase with no ongoing fees
- AI-Powered Diagnostics: Provides real-time analysis and repair guidance
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Implications for Automotive and Garage Businesses
For automotive service providers, this experiment offers a stark insight: deploying AI in customer interactions, scheduling, or diagnostics isn’t just about how well it can chat or generate responses. It’s about whether AI can finish what it starts, read critical internal information, and stay honest under pressure—especially when the stakes involve real money and reputation.
Why Trust Matters
In the experiment, the models’ ability to identify and resist manipulation underscores a vital point: AI systems must be trustworthy and disciplined, not just literate. When AI agents are integrated into automotive workflows—be it in CRM, parts inventory, or diagnostics—they need to reliably deliver results and avoid costly errors or manipulations. The performance gap shown here is a reminder that transparency and rigorous testing matter before trusting AI with critical tasks.
ChatGPT as a Cybersecurity Advisor: Small and Medium Business
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Transparency and Learning in Action
Unlike typical demos, where AI chat quality is showcased in polished conversations, this experiment exposes the raw decision-making process. Every choice is recorded and made auditable, offering a unique learning opportunity for businesses considering AI adoption. By running their own ‘wargames’ against a read-only export of their operations, automotive companies can see firsthand how AI handles crises, manipulates, or stays disciplined—without risking real-world systems.
What’s Next?
The ongoing live experiment isn’t just a show—it’s a battle for survival, illustrating the real costs and capabilities of AI in high-stakes environments. As this experiment continues, it’s a valuable blueprint for automotive businesses to understand what to expect from AI-driven management tools, and what traits are essential for trustworthy, effective deployment.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Building AI-Powered Products: The Essential Guide to AI and GenAI Product Management
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
