FinTech Agents Fail Audits When Prompt Is Not Runtime

We’ve seen teams build a “smart” FinTech agent in two weeks, demo it to leadership on Friday, and spend the next three months explaining to compliance why the same input produced three different outcomes.

That’s not intelligence. That’s a liability with a nice UI.

The core mistake is simple: people keep treating prompts like executable runtime logic. They stuff branching rules, approval thresholds, exception handling, and compliance behavior into model instructions, then act surprised when the system behaves like a polite improv actor instead of an auditable machine. The phrase prompt is not runtime: sounds obvious once you’ve been burned by it. Before that, it sounds like philosophy. It isn’t. It’s architecture.

A recent post by Alex V makes this case directly for deterministic FinTech systems, arguing against LLM state machines for regulated workflows and in favor of explicit orchestration and verifiable control paths Alex V on DEV. We agree with the premise, and we’d push it further: in regulated systems, your LLM should usually be a component, not the governor.

That’s the difference between passing an audit and writing a postmortem.

Key Takeaways

The seductive bad idea: “just let the model decide the next step”

We get why this happens.

You’ve got a backlog full of ugly financial workflows: dispute intake, sanctions screening review, underwriting exceptions, transaction categorization, suspicious activity triage. Someone says, “What if the agent just reads the case and decides what happens next?” In a demo, it looks magical.

Then the real world arrives with duplicate documents, malformed PDFs, edge-case account structures, flaky upstream APIs, and a regulator who does not care that the model was “pretty confident.”

That’s where LLM state machines start to rot.

The problem isn’t that LLMs are useless. The problem is that many teams ask them to do the one job they’re structurally worst at: enforce deterministic control flow under scrutiny. Alex V’s piece calls out this exact trap — using prompts to define runtime behavior in FinTech systems where determinism matters Alex V on DEV.

Hot take: if your “workflow engine” is a nested prompt with phrases like if high risk then escalate, you do not have an engine. You have vibes.

Why audits hate LLM state machines

Audits are boring until they’re expensive.

An auditor usually wants some painfully reasonable things: what happened, why it happened, which rule applied, who approved it, what changed, and whether the same input would produce the same outcome again. Deterministic orchestration can answer that. Prompt-driven state machines usually answer with a shrug wrapped in JSON.

Here’s the ugly truth: “the model interpreted the case differently this time” is not a control.

It’s an excuse.

When a workflow is encoded in prompts, you inherit several failure modes at once:

1. State transitions become probabilistic

If the model decides whether a case moves from pending_review to approved, your state machine is no longer a machine. It’s a suggestion engine. Even with low temperature, model updates, context shifts, token truncation, or retrieval differences can change outcomes.

2. Rules become hard to diff

A Git diff on a policy file is clear. A diff on a 900-line system prompt with buried business logic is a crime scene.

3. Explanations become post-hoc

The model can explain why it did something, but that explanation is often generated after the fact. In regulated systems, you want the actual decision path, not a plausible story about the decision path.

4. Testing gets weird fast

Unit testing deterministic branches is straightforward. Testing “what the model probably means when the customer sounds nervous and the merchant descriptor looks suspicious” is where teams start writing eval harnesses that feel like they’re arguing with weather.

Here’s how the architecture should look when you care about auditability:

On the left: chaos with confidence. On the right: a system someone can actually defend.

Deterministic orchestration is boring. That’s why it wins.

We like boring in the parts that move money.

Deterministic orchestration means the workflow graph, state transitions, retries, compensations, deadlines, and approval gates are explicitly defined outside the model. The model can classify a document, summarize a case, extract entities, or draft an explanation. It does not decide whether your system skips a sanctions review because the prompt “felt sufficient.”

That separation matters.

Think of it like a commercial kitchen. The LLM is a very talented line cook. Fast, creative, occasionally brilliant. Deterministic orchestration is the kitchen pass, the ticket system, food safety rules, and the head chef deciding what can leave the kitchen. If you let the line cook rewrite allergy policy mid-service, somebody ends up in the hospital.

Same pattern. Different lawsuit.

What goes where: orchestration, policy engine, and LLM

This is the framework we recommend when building regulated systems.

Put this in deterministic orchestration

Use code or workflow engines for:

• State transitions
• Retry logic
• Timeouts and SLAs
• Human approval gates
• Multi-step transaction flows
• Idempotency and compensation
• External API sequencing
• Audit log creation
• Access control checks
• Hard fail/soft fail behavior

If you need to answer “what happened at 14:03:22 UTC and why did the system proceed,” this belongs here.

Put this in a policy engine

Use a policy layer for:

• Approval thresholds
• Country or jurisdiction restrictions
• Product eligibility rules
• Escalation criteria
• Segregation-of-duties constraints
• Dynamic risk thresholds
• Feature entitlements by customer type

This is where readable, reviewable, versioned business logic should live. Not hidden in a prompt like a teenager hiding dirty dishes under the bed.

Put this in the LLM

Use the model for:

• Document classification
• Entity extraction
• Summarization
• Case note drafting
• Natural language explanations for operators
• Triage suggestions
• Retrieval-based assistance
• Converting messy human input into structured candidates

Notice the word candidates.

The model can propose. The system should decide.

A practical hybrid pattern that doesn’t blow up in production

Here’s the pattern we’ve found most defensible for regulated AI systems:

flowchart TD
  A[User or upstream event] --> B[Deterministic workflow]
  B --> C[Policy engine check]
  C --> D{Need model help?}
  D -->|No| E[Execute next state]
  D -->|Yes| F[Scoped LLM task]
  F --> G[Structured output validation]
  G --> H[Policy and confidence gates]
  H --> E
  E --> I[Audit log + human review if needed]

The key is that the LLM is called like a specialist, not crowned like a monarch.

A good FinTech agent should feel less like “the AI runs the process” and more like “the process calls AI where language or ambiguity exists.” That’s a huge difference in failure behavior.

And yes, failure behavior is the whole game.

“But can’t we just prompt the model better?”

We’ve tried that line of thinking. It’s a trap.

You can absolutely improve outputs with better prompting, constrained schemas, examples, tool use, and tighter context windows. You should do all of that. But none of it changes the fundamental issue behind prompt is not runtime: prompts influence behavior; they do not provide the guarantees of an execution environment.

That distinction matters outside AI too, which is why the phrase lands so well. In JavaScript, developers hit “prompt is not defined” when they try to use the browser’s window.prompt() in environments like Node.js, where that runtime API simply doesn’t exist MDN Web Docs. The lesson is almost embarrassingly transferable: an instruction that works in one environment doesn’t magically become runtime capability in another.

A prompt is not a scheduler. A prompt is not a transaction manager. A prompt is not an audit trail. A prompt is not a policy engine.

It’s text.

Useful text. Powerful text. Sometimes profitable text.

Still text.

The part competitors usually skip: failure recovery

Most articles stop at “use deterministic workflows for compliance.” True, but incomplete.

The real pain starts when something fails in the middle: KYC API timeout after document extraction, payment rail returns ambiguous status, fraud score arrives late, reviewer overrides the model recommendation, customer uploads a second passport that conflicts with the first one. This is where deterministic orchestration earns its rent.

You need explicit handling for:

• retries with backoff
• dead-letter queues
• compensating actions
• stale decision invalidation
• re-review triggers after policy changes
• human override precedence
• event replay with versioned rules
• partial completion states

If those rules live in prompts, your recovery path turns into fan fiction.

We’d rather model this in workflow code, persist every transition, and let the LLM participate only where ambiguity must be reduced. That’s also the cleaner route if you later need AI consulting help, a specialized custom model, or a narrower AI agent implementation that won’t terrify your compliance team.

Where LLM agents still make sense in FinTech

We’re not anti-agent. We’re anti-sloppy-agent.

LLM agents work well in FinTech when the blast radius is controlled and the outputs are advisory, reversible, or heavily constrained. Good examples:

• internal analyst copilots for case summarization
• support agents drafting responses for human approval
• transaction categorization suggestions
• document intake normalization
• voice-based assistant flows for non-binding customer interactions

That’s why scoped deployments matter. We’ve seen the same principle in voice systems: if the model is handling natural conversation, great; if it’s silently deciding policy, bad. The architecture behind voice AI and on-device AI often forces this discipline because latency, privacy, and reliability make hand-wavy orchestration impossible. Our own product work on RunHotel lives in that reality — the model can interpret and assist, but the surrounding system has to be explicit about what happens next.

Regulated FinTech should borrow that discipline.

Honestly, most teams would save money too. If every branch decision requires another model call, your bill starts looking like a tax on indecision. If you want a rough sense of that pain before shipping, use an AI cost estimator. It’s cheaper than discovering your orchestration strategy in the invoice.

A blunt implementation checklist

If you’re building a FinTech agent right now, here’s our opinionated rule set.

Use deterministic orchestration when:

• money moves
• approvals happen
• customer entitlements change
• compliance evidence is required
• retries or compensations matter
• humans need override controls
• the same input should produce the same state transition

Use a policy engine when:

• rules change often
• legal, risk, or ops teams need visibility
• thresholds vary by market or product
• you need versioned, reviewable decision logic

Use an LLM when:

• inputs are messy, human, ambiguous, or unstructured
• you need summarization, extraction, or drafting
• the result can be validated before action
• the output is advisory or gated

Don’t use an LLM state machine when:

• “we’ll just tune the prompt” is the mitigation plan
• nobody can explain the transition graph without opening a prompt file
• auditability depends on generated reasoning
• rollback logic is vague
• confidence scores are being used as fake policy

That last one is underrated. Confidence is not compliance.

The architecture we’d actually recommend

Start with a deterministic workflow engine. Add a policy layer with explicit versioning. Call the LLM for narrow tasks with structured outputs. Validate those outputs. Log everything. Require human review where the cost of being wrong is asymmetric.

Then tighten.

Measure where the model helps, where it stalls, where it creates operator drag, and where deterministic rules can replace expensive model calls. Most mature systems end up with less “agent autonomy” than the original pitch deck promised. That’s not failure. That’s adulthood.

Here’s where it gets weird: the better your engineering gets, the less magical your AI architecture looks from the outside.

Good.

Magic is terrible in audits.

If you’re building one now

If your current FinTech agent has prompt files doing the job of workflow code, stop adding more cleverness. Pull the state transitions out first. Make the policy layer explicit. Reduce the model’s authority until you can explain every important action without saying “the LLM decided.”

That’s the whole point of prompt is not runtime: not as a slogan, but as a boundary.

If you want help designing that boundary — agent where it helps, deterministic where it matters — talk to us at Cropsly. We like AI. We just don’t like pretending a paragraph is a control plane.

And if your compliance strategy currently fits inside a system prompt, that’s not bold engineering.

That’s a future incident report.

Sources

• Alex V on DEV — “Prompt Is Not Runtime: Why I Rejected LLM State Machines for Deterministic FinTech”
• MDN Web Docs — Window: prompt() method