One AI asks the tricky questions. The other has to answer. We watch what happens—like a stress test—so we can see where AI might break before real people depend on it.
Over the past year, AI has moved from “interesting demo” to critical infrastructure.
LLMs now write production code. They triage support tickets. They assist doctors. They sit inside products that touch millions of people.
And yet, most of these systems are barely stress-tested before deployment.
That gap matters.
If we’re going to trust AI systems with real decisions, real users, and real consequences, we need to pressure-test them the way we test any high-stakes technology: rigorously, repeatedly, and under realistic—sometimes adversarial—conditions.
That’s why we built AI Asylum.
One AI asks the tricky questions. The other has to answer. We watch what happens—like a stress test—so we can see where AI might break before real people depend on it.
This post is our introduction: what AI Asylum is, why we think testing AI this way matters, and where we’re taking it next.
See it in practice
A walkthrough of AI Asylum: interview modes under pressure, weight surgery that forks a checkpoint, and opposing training runs you can verify on disk.
Interview → Surgery → Train & oppose. Open on YouTube. The measurements are in AI Lobotomy.
Benchmarks miss pressure
Most evaluation of AI models today looks like traditional benchmarking:
How well does it answer exam-style questions?
How accurate is it on math, logic, or multiple-choice datasets?
How fast and cheap is it to run?
Those metrics are important.
But they miss something critical:
How does a model behave when someone is deliberately trying to bend it?
Real misuse doesn’t look like a neat benchmark prompt. It looks like:
A carefully crafted jailbreak that escalates slowly over 8 turns.
A user who shifts context mid-conversation to exploit ambiguity.
A series of harmless requests that form a harmful chain when combined.
We’ve seen models that resist single-shot jailbreak attempts — but fail when the pressure is applied gradually across multiple turns. We’ve seen models behave safely in isolation — but drift when paired with retrieval systems or long context windows.
Benchmarks measure intelligence.
They rarely measure resilience.
If AI is infrastructure, resilience is not optional.
Doctor and patient
AI Asylum is a meta-testing framework for large language models.
Instead of relying solely on human red teamers manually probing systems, we let:
One model (the doctor) conduct structured tests.
Another model (the patient) respond under those conditions.
The doctor runs through adversarial conversations, edge cases, ethical dilemmas, and jailbreak strategies — then evaluates how the patient behaves.
The Asylum dashboard: suites under pressure, benchmarks queued, and scores that measure what the model said under load.
The goal is not to “break” models for sport.
The goal is to understand their behavioral profile:
Where are they robust?
Where are they brittle?
Under what pressure do they begin to cross safety boundaries?
How consistent are they over time?
We treat models less like black boxes and more like systems that can be profiled, stress-tested, and observed under load.
Built for the coming bar
The regulatory and standards landscape is evolving quickly.
High-risk AI systems are increasingly expected to demonstrate structured safety testing and adversarial evaluation. AI red teaming is no longer a niche research activity — it’s becoming a baseline expectation.
AI Asylum was built with that future in mind.
It helps teams:
Systematically explore model behavior under adversarial conditions.
Document failures and improvements in an audit-friendly way.
Map outputs to a safety taxonomy (bias, toxicity, misinformation, privacy, security, edge cases).
Track regressions across model versions and configurations.
If you’re responsible for AI safety, security, or compliance, you need more than spot checks.
You need repeatable, inspectable, evolving stress tests.
How it works
At a high level, the workflow is simple:
1. Choose your models
Select a doctor model and a patient model. AI Asylum supports major cloud providers and local models.
Two patients under the same doctor, both running locally. Each hears the doctor and the other patient, then answers.The doctor’s goal for one group session. It is sent to the doctor only. The patients do not see it.
2. Choose your test type
Run:
Multi-turn conversational probes
Scenario-based ethical tests
Jailbreak and adversarial escalation tests
Benchmark + safety hybrid campaigns
The test is chosen before it runs: one shot, a conversation, group therapy, or a benchmark. The prompts underneath are the library on this machine.
3. Execute structured sessions
The doctor interacts with the patient under defined constraints and escalation strategies. Full transcripts, context windows, and prompt structures are captured.
A suite in progress on the March 2026 build. One run going, three finished, one failed. The models are local copies on disk.
4. Score and classify behavior
Outputs are evaluated across multiple dimensions of risk — not just pass/fail.
5. Analyze and compare
Explore which scenarios triggered issues, how behavior changes over time, and how different models compare under identical stress.
We deliberately separate:
Test execution (fast, scalable)
Deep analysis (optional, intensive, exploratory)
This lets teams start small — and then zoom in when something interesting appears.
Behavioral fingerprints across safety, alignment, reasoning, jailbreak resistance, ethical reasoning, and factuality.Score comparison across models under the same stress. Useful for ranking. Still an outside measurement.
Why we built this
After working in AI security and infrastructure, we kept seeing the same pattern:
Teams were shipping models with impressive benchmark scores — but very little structured adversarial testing.
Safety was often reactive. Patch a jailbreak. Add a filter. Move on.
That approach doesn’t scale.
If AI systems are going to sit in healthcare, finance, defense, and critical infrastructure, we need to treat them like we treat any other high-impact system:
With continuous testing.
With regression tracking.
With structured red teaming.
With behavioral observability.
AI Asylum is our attempt to make that process systematic.
What you can do today
Even in its current form, AI Asylum supports meaningful real-world use:
Maintain structured records for internal review or external audits.
Extend the framework with your own risk taxonomies and test suites.
We publish what we find, so the method is not a black box.
The measurements, the conditions they were taken under, and what they do not prove all go in writing, where you can check them and argue with them.
Where we’re going
AI Asylum is just the beginning.
We’re actively working on:
Richer reporting and visualization.
Expanded benchmark integrations.
Better support for RAG systems and long-context testing.
More advanced analysis tools for teams that want deeper visibility.
Curated adversarial test libraries to keep pace with evolving jailbreak techniques.
On this blog, we’ll share:
Real failure patterns we observe across models.
Practical red teaming methodologies.
Lessons learned from testing AI systems under pressure.
How to align AI evaluation with emerging regulatory expectations.
Forward-looking ideas about behavioral fingerprinting and AI resilience measurement.
We care deeply about building AI systems that people can trust — not because we say they’re safe, but because they’ve been stress-tested to earn it.
A statement of intent
If AI is going to become infrastructure, then safety cannot be an afterthought.
Red teaming cannot be occasional.
And evaluation cannot stop at intelligence benchmarks.
Resilience must be engineered.
AI Asylum is our contribution to that effort.
We publish what we learn from it.
We’re building it for teams who take responsibility seriously.
And we’re building it for the long term.
If that resonates with you, we’d love for you to join us.
Chapter 2
The next post goes a layer deeper: we find the direction inside an open-weights model that decides whether it refuses, remove it from the weights, and measure what that costs — with a walkthrough of the tooling on the page. Read AI Lobotomy →
AI Asylum is the research tool behind this post. The commercial work it informs is narrower and more practical: reviewing AI systems for safety and security before they are used with patients or shipped inside a regulated product.
AI system review before clinical use — structured, multi-turn adversarial testing of the assistant or chatbot you plan to deploy, with findings your governance or safety committee can act on.
Security testing of the whole system — the model is one component; retrieval, tool access, data paths, and permissions are where many real failures live.
AI Asylum as a research tool — the doctor-and-patient framework above is how we study behavioral failure. It informs our reviews, and we publish what we learn here.
