CTF based on Hackers (1995)

This is part of a series of CTFs for an awesome security company.
As these are still part of their hiring process I won’t be disclosing their name.

The file received was zerocool.dade

Ok, so this is not a normal extension. Pop this guy open in a hex editor and it’s magic header is PK, a reference to Phil Katz and PKWARE! Also, contains a file called ac1dburn.kate.

Hex editor showing the beginning on the file zerocool.dade

Side note, sadly I had to lookup this reference as it’s been a long time since I’ve seen the movie Hackers. I enjoyed the reference all the same.

ac1dburn.kate is an exe (thanks MZ!), when running it we get a nice little console asking us to enter our password.

Let’s check the disasm.

I caught some strings in the bss section that look a lot like hashes.


Checking the imports we see that advapi is calling cryptography functions.

Random side note I always find interesting is the compiler settings this exe was compiled with.

Stack Frames /RTCs (Seen below, 0xCCCCCCCC)

Assembly with stack frames enabled

Also: What compiler generates mov [esp], eax instead of push eax ?!? (Shown Below)

This code is riddled with it. It’s definitely not space efficient, and it should be the same exact cost in execution time unless I’m missing something… The only explanation I can come up with is rather than doing push 0x8004, push eax, it wanted them in that order because of the latency cost and was able to put in the lea’s between those instructions? No idea here.

Assembly with stack frames enabled

This brings us to the next part, the first interesting function we’ve found that uses our user input and calls the cryptography functions in advapi. I’ve named this function Crypt().

They appear in this order:

CryptAcquireContextA with CRYPT_VERIFYCONTENT and provider set to PROV_RSA_AES
CryptCreateHash with ALG_ID set to CALG_SHA1 / CALG_SHA256
CryptHashData with our input, and the data length
CryptGetHashParam with dwParam set to HP_HASHVAL

I wanted to make sure I understood the hashing algorithm used and the hash it generated so I threw the same input I put in the program into python.

Python source code showing the sha1 hex digest of 'abc'

And they were the same. Now we have an understanding of our input’s mutation.

Ok, so the final goal is to get our hash to be the same as the one I found previously:


Assembly of strncmp called with a hash of our input and a static one (above)

I want to dig a little deeper to see if there’s anything else.

Let’s just bypass this strncmp call and see what happens.

Console output of the garbage text we see if we try and bypass the strcmp

Ok, so the message is encoded as well. Let’s look into that.

Another hash of our input is created, this time as an sha256.

I spent a bit of time in this crypt function which I typedef’d as:

int __usercall Crypt(BYTE *pbData<ecx>, DWORD *pdwDataLen<edx>, BYTE *outBuf<eax>, DWORD dwDataLen, ALG_ID Algid)

The hidden argument in eax tripped me up a bit. This looks like __fastcall convention, and it’s close but not quite. IDA calls these __usercall so I just used that. Looking a bit deeper, Detect-It-Easy can’t identify the linker. Could this be delphi / pascal calling convention?

After this Crypt function a new function sub_401E10 is called twice with different arguments, which I’ve renamed to DecryptMessage.

I’ve typedef’d as this:

int __cdecl DecryptMessage(BYTE *outBuf, BYTE *encryptedData, BYTE *sha256, int encryptedLen)

It returns how many bytes written to outBuf.

DecryptMessage(&outBuf1, &0x406640, &inputSHA256, 33);
DecryptMessage(&outBuf2, &0x406620, &inputSHA256, 33);
outBuf1 = 92 8A 49 AB 81 50 92 89 68 BE 84 6D 49 4B BA 07 25 AB 87 54 AA 68 59 D7 B2 BC D7 EB A1 93 5F 1F
outBuf2 = D4 89 4E EC 8E 6C C9 98 7E F2 88 7D 1D 56 A0 1B 25 BD D9 42 E6 78 11 8A FD B8 8D AD E5 82 5B 06

These buffers are what ends up written to the console.

I wish I had a bit more experience with cryptography, but we’re not going to be able to generate a hash that decrypts the encoded message as easy as looking up the pre-computed SHA1 hash in a rainbow table. It would be trivial to set a python project to loop over a word list and compute them, but it seems out of scope and for only a third level challenge I’m betting this is an acceptable solution. I doubt we’ll get off as easy next time.