Binary Reverse Engineering - 90s Movie Tie-In


This is part of a series of CTFs for an awesome security company.

Ok, our first binary that’s x64 for these CTFs, this is great. I’ve worked a lot with anti-debugging tricks and quite a bit with x64, let’s see if there’s anything hiding in here I haven’t seen.

Console with DEBUGGER DETECTED printed to it

Ok, a simple little console. Let’s open it in x64dbg, but let’s make sure I disable tools related to hiding a debugger (ScyllaHide). When I run the help command in the console it brings up “DEBUGGER DETECTED” and exits.

Assembly showing call to IsDebuggerPresent()

Best way to fix this one is to follow the call into IsDebuggerPresent(), the first instruction loads the PEB so follow it in memory, and change the byte at +2 to 0.

No strange modules are loaded or anything. If I type:

> access security

A sound clip plays (Gonna nop this one out real quick because just like from the movie, it’s obnoxious). Let’s find a reference to where that text is.

There’s a comparison checking if what we typed after ‘access’ is not ‘fl4g’

Let’s try typing that instead…

Console output giving the challenge gpcaynnb and waiting for a response

Ok, now have a challenge / response setup, interesting. Let’s check the challenge function.

Just glancing at it shows me, it calls rand(), writes “Challenge: “ to the console, waits for user input then checks if a debugger is present (which is already patched). It gets the thread we’re currently executing on, the thread context, allocates 0xC6 bytes and a bit more.

After we type our user input it checks to see if the length of our input is 0xA, but because of how it’s being read off the ReadConsole, our input includes \r\n, so the actual length of what we type must be 8, or it can’t be valid. Let’s see what happens if I give i ‘abcd1234’.

Assembly showing the anti-debugging technique that uses GetThreadContext()

There’s a bit more anti-debugging, this time checking the state of the DR registers, DR0-3 are hardware breakpoints. It’s a bit tough to tell exactly the fields here because the CONTEXT structure is so different for different architecture. ReactOS documentation to the rescue! It lists the correct breakdown for the locations of the DR registers in the CONTEXT.

Ok, the program is now going to:


The execute flag on this memory is unusual, and that isn’t a ton of bytes, let’s see what it does with it.

There’s a bunch of complicated math and calculation that goes on, but it eventually fills up the stub and jumps to it! So the execute flag wasn’t accidental.

Here’s the code at the stub it built:

Assembly of dynamically built stub

Ok, now we’re getting somewhere.

The function it build takes an argument which is the challenge text and an out buffer to write the response string to.

My challenge was “ainqcpce”.The response my stub generated was “qydgsfsu”.

If that wasn’t enough, it is now going to hash it. Damn.

CryptCreateHash(context, CALG_SHA, NULL, NULL, &hash);

mem1 = VirtualAlloc(NULL, 9, MEM_COMMIT, PAGE_READWRITE);

memcpy(mem1, response, 9);
CryptHashData(hash, mem1, 8, NULL);
CryptGetHashParam(hash, 4, &hashData, &hashLen, NULL);

mem2 = VirtualAlloc(NULL, 0x14, MEM_COMMIT, PAGE_READWRITE);

CryptGetHashParam(hash, HP_HASHVAL, mem2, &hashData, NULL);

My HASH_VAL was:
AB 29 5A 26 E2 8B 8A 75 16 AE EB 7E C1 CF FB 2D B1 DB 10 66

The data is then destroyed and the next function checks our original input against the hashed value. It was just too juicy to modify my original input at this point.

It would have been much more annoying if it had been hashed immediately when I entered it and forced me to go back, but I was able to modify my original input at this moment to give it the correct challenge response.

Stepping through the code knowing what to look for was key here. Once I saw the EXECUTE flag on the VirtualAlloc call I had a huge clue. I really appreciated the function that was built to generate the valid response. I might go back and reverse the challenge / response function for fun later though. It was too kind on the Anti-Debugging. All of the ideas employed are great here, it was executed very well, hashing the response and our input.

The only criticism I would say is this could be harder via some more anti-debugging tricks and moving the Hash(Input) to the beginning before the stub is executed.

Keygen to decode the challenge / response correctly:

Python code to decrypt the challenge/response