> ## Content Index
> Fetch the complete content index at: https://taiwanding.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Letting AI Hack a Bank That Doesn't Exist: Finding Bugs Wasn't the Hard Part
- URL: https://taiwanding.com/en/ai-pentest-humanbank-octorig-verifying-ai-findings/
- Published: 2026-11-14T12:41:18.000Z
- Updated: 2026-11-14T12:41:18.000Z
- Description: I had an AI work through OctoRig's HumanBank practice app. It captured all eight flags fast; the real work was checking which of its claims were facts, which were inferences, and when to stop.
- Author: Kevin Chen
- Tags: #en, #en-machine

Once all eight flags were in hand, this post should by rights have been over.

Instead, I started to get uneasy.

The AI said it had grabbed the admin's cookie, yet an earlier HTTP response clearly showed the session cookie was set with `HttpOnly`. Could both of those really be true?

That's when I realised what this experiment was actually about. Not how many holes HumanBank has, but this:

**When an AI says “I found a vulnerability,” how do we tell whether it's stating a fact, making an inference, or blending the two?**

> Disclaimer: this post documents research in a self-hosted, authorised instance of the OctoRig HumanBank training lab. All findings apply only to that lab environment and do not imply that any real financial institution has the same issues. To leave you room to practise on your own, I don't include full attack payloads or CTF flags.

## Introduction

[OctoRig](https://github.com/CommonHuman-Lab/OctoRig?ref=taiwanding.com) is an open-source security lab that includes a simulated online banking system called HumanBank.

It's fairly complete: login, sign-up, account management, transfers, transaction authentication numbers (TANs), an admin interface and an API.

The plan was simply to work through a practice box together with an AI, but before starting, a question came to mind:

If an AI can already drive tools, analyse code, send HTTP requests and finish an exploit on its own, what's left for me to do?

Am I just there to type “continue,” “look again” and “dig deeper”?

So this time I wasn't only attacking a bank; I was also watching how an AI investigates an unfamiliar system.

## Step 1: Environment, where localhost isn't always localhost

I got stuck before I even started.

1. **Docker subnet clash**: HumanBank's subnet collided with an existing container network on my machine, so it wouldn't come up. After adjusting the network config, the site returned HTTP 200.
2. **The AI agent couldn't connect**: the site loaded fine in my browser, but another AI agent reported a connection failure.
3. **Cause**: the sandbox it runs its tools in has its own `127.0.0.1`.

I stared at that connection error for a while before it clicked.

I was giving it the same address, but we were talking about two different machines. For the AI, localhost isn't necessarily the localhost on my computer.

Once it ran in the right environment, it connected.

This was just an environment issue, not a research result. But if the AI had reported “target not responding, unable to test” and I'd taken that at face value, this lab would have gone down as “couldn't get in.”

Takeaway: **a failed test doesn't mean a security control worked.**

Network problems, a different environment, missing permissions, missing data or a broken tool can all make a test fail. Writing “not exploitable” without knowing why is a misjudgement.

## Step 2: Capturing flags, or how fast AI is

With the environment sorted, the AI's speed showed.

1. **SQL injection on the login page**: yields an admin session.
2. **Broken function-level authorisation**: regular users can reach the admin API.
3. **Broken object-level authorisation**: accounts and tickets don't check ownership.
4. **More SQL injection points**: one in search and one in the transaction filter.
5. **Other**: stored XSS, hints of credential reuse, and a few design issues.

Before long it reported: all eight flags captured.

If all you want is to solve the challenge, that's enough.

But capturing a flag doesn't mean it really understands what it broke.

## Step 3: Following the evidence, XSS vs. HttpOnly

Back to the thing that felt off at the start.

The AI's report said “stole the admin cookie,” in a confident tone and with nice formatting.

If I hadn't happened to remember that earlier response header, I'd probably have just accepted it.

But the session cookie had `HttpOnly` set, so JavaScript shouldn't be able to read it via `document.cookie`.

The report said it was stolen; the header said it couldn't be. One of them had to be wrong.

I asked the AI to re-examine the evidence. Here's what came out:

1. **There's another cookie**: besides the session cookie, the system sets an `hb_admin_token` without `HttpOnly`.
2. **The XSS really does execute**: JavaScript can run in the admin's browser.
3. **`hb_admin_token` is readable by JavaScript**: but this time it wasn't actually read or sent anywhere.
4. **Whether it grants authorisation or is equivalent to a session**: not verified.

So the accurate phrasing isn't “stole the admin cookie” but “XSS can execute in the admin's browser, and there is a token readable by JavaScript whose authorisation power is unverified.”

It's longer and less cool, but it's what our evidence can actually support.

Takeaway: **“JavaScript can execute,” “a token is readable” and “account takeover achieved” are three different things.**

If nobody pushes back, an AI will easily write them up as one.

## Step 4: Transfers and TANs, minus one cent

There's one later finding I particularly like.

HumanBank's transfer feature accepts negative amounts.

1. **How I tested it**: between two accounts I'd created myself, with the amount set to just `-$0.01`.
2. **Result**: the server accepted it, and the direction of the transfer flipped.
3. **Cleanup**: once verified, I made a reverse transaction to restore the balances.

When I saw the balance move one cent the wrong way, I genuinely laughed.

No quotes, no encoding, nothing that looks like an attack. Just a minus sign.

A person thinks “obviously a transfer amount can't be negative,” but to the program `-0.01` is a perfectly valid number. The system never explicitly requires the amount to be greater than zero, so it runs the usual addition and subtraction, and “I pay you” becomes “you pay me.”

Put a few of these bugs side by side:

1. **SQL injection**: user input changes the meaning of a database query.
2. **IDOR**: a user-supplied resource ID slips past the ownership check.
3. **Negative transfer**: a user-supplied number changes which way the money flows.

Different technical root causes, same underlying problem.

Takeaway: **the system doesn't separate, in the right place, “data the user is allowed to supply” from “decisions only the server should make.”**

Security isn't about treating every input as dangerous; it's about knowing which inputs shouldn't get a say in decisions.

The same transaction flow also has the TAN, the transaction authentication number, and here the result was the opposite: it held up.

1. **Bound to the user and to a specific position**: this control works.
2. **Consumed after use**: this works too.
3. **Amount and recipient**: at the observable TAN verification stage, the system doesn't check either.

Is point 3 a vulnerability? It depends on what the TAN promises.

- If it's just a one-time authentication code, its only job is to prove “you hold the right credential.”
- If the bank claims it authorises a specific transaction, then what it authorises should include the amount and recipient.

Those two designs offer completely different security guarantees, and we neither showed that a TAN could be replayed to complete a transaction nor confirmed any unauthorised debit.

So the report lists it as “a design issue whose security semantics need clarifying,” not as a high-risk vulnerability.

Knowing which controls work, and what they were meant to protect, matters as much as knowing which attacks succeeded.

## Step 5: How a human supervises an AI

Further into the research, I noticed that the AI's biggest problem isn't failing to find bugs; it's how easily it swings between two extremes.

1. **Giving up too early**: throws a few payloads, gets nothing, and declares that no vulnerability was found.
2. **Expanding forever**: after finding one issue, it goes after more bugs, more flags, more endpoints, and even goes digging through the source code for the answers.

It's always busy, but more work doesn't mean we understand the system any better.

At one point it tried to use a path traversal it had already found to read an SSH private key inside the container, and the execution environment's safety mechanism blocked it.

It then read the non-sensitive `/etc/passwd` instead, which confirmed arbitrary file read.

That was enough evidence. There was no need to go after the private key just to make the report look more impressive.

Takeaway: **what an AI needs most isn't unlimited ability to act, but knowing when to verify, when to change direction and when to stop.**

I used to reply “keep digging” whenever the AI said it found nothing. Now I ask these four questions instead:

1. **When the AI says it found no vulnerability: “Did you find nothing because you verified the security control works, or because you haven't thought of other hypotheses yet?”**
2. **When the AI says it found a vulnerability: “What exactly did you observe? Which parts are facts and which are inferences?”**
3. **When the AI wants to keep testing: “What will the next experiment tell us that we don't already know?”**
4. **When the AI says the research is done: “Are you stopping because the key hypotheses have been thoroughly verified, or because the remaining experiments aren't worth the risk and cost?”**

These questions don't require me to know every framework or payload better than the AI, but they force the research to be explainable, verifiable and open to challenge.

The division of labour looks roughly like this:

1. **AI**: proposes hypotheses, operates tools, runs experiments, organises observations.
2. **Human**: sets the research question, checks the evidence, challenges inferences, judges the value and risk of the research.

That doesn't mean the human can skip the technical side. I only caught the contradiction in Step 3 because I knew what `HttpOnly` does.

## Step 6: Results and limitations

In the end, HumanBank yielded more than ten root causes or security design issues:

- SQL injection
- Broken function-level and object-level authorisation
- Unsigned JWTs accepted
- Password reset token leakage
- Stored XSS
- Path traversal
- Transfer amount and limit validation issues

The report doesn't call every one of them a critical vulnerability, and the limitations are stated as they are:

1. **`alg:none` JWT**: genuinely accepted, but only tested with claims for our own identity; it was not shown that we could impersonate an admin.
2. **Session still valid after logout**: a previously copied cookie keeps working, but an attacker would need the cookie first, so it can't be counted as a standalone path to account takeover.
3. **Credential reuse**: one user being able to log in to HumanBank doesn't mean the hypothesised cross-system attack chain is fully established.

There were also two lines we chose not to cross:

1. **No state-changing actions on accounts we didn't own**: we read other accounts and tickets during the research, but didn't touch their data or balances.
2. **No concurrency stress testing**: to avoid causing real side effects in the ledger.

One more small detail. The two test accounts' balances were restored, but the transaction history, test tickets and reset records are still there, so the notes say “balances restored,” not “environment fully restored.”

None of this helps your CTF score, but rigorous research can't do without it.

The flags only exist on this box, and the payloads I learned today may stop working tomorrow with a different framework, database or authentication method. What you can take with you are these questions:

1. Who decides the identity behind this action?
2. Does the server only check “logged in” without checking resource ownership?
3. Does this code prove identity, or does it authorise a specific transaction?
4. Do the business rules still hold when the input is zero, negative, empty or out of range?
5. Is the success I'm seeing a successful HTTP response, or did the backend state actually change?

They apply to banks, e-commerce and enterprise management systems, and just as much to newer architectures like OAuth, AI agents and MCP.

## Conclusion

AI makes “doing” cheap, but it doesn't automatically make judgement correct.

We'll never be short of agents that can fire off thousands of HTTP requests, or models that can churn out reports quickly.

The scarce skill is still taking a pretty result apart and asking: what does it prove? What doesn't it prove? Is the next experiment worth running?

Only when an AI can clearly answer “based on what?” and “why stop here?” does a flag become knowledge rather than just output.

Happy hunting!