What is this file and how can it help you get the flag?

Attached was a file called challenge.flp.


Let's inspect this file:

This looks like an image of a floppy disk, 7Zip can usually extract these files:

We have three pretty old executable files (dated 1987 and 1992):

Two of them are NE (New Executable) files for Windows 3.x, and one looks like a standard MZ file for DOS.

If we try to rename MAIN.ENC to MAIN.EXE and run it via DOSBox (a cool emulator program which emulates an IBM PC compatible computer running a DOS operating system) - DOSBox itself immediately crashes. Perhaps we will have more luck by running the client and server?

Peeking into the assembly of the executables, we see that they are supposed to be run under Windows 3.1 only:

Some retro gaming sites offer packages which allow running old games on Windows 3.11. We can use one of those to run the executables and see what happens:

Running the server:

Running the client:

Looks like they are trying to communicate in some way (obviously, given they are a client and server) but nothing seems to happen.

Time to open a disassembler and try to understand was supposed to happen.

A few words about the NE (New Executable) format, from Wikipedia:

The New Executable (abbreviated NE or NewEXE) is a 16-bit .exe file format, a successor to the DOS MZ executable format. It was used in Windows 1.0–3.x, multitasking MS-DOS 4.0 [...].

The first product to be released using the New Executable format was Windows 1.0 in 1985, followed by the 1986 multitasking MS-DOS 4.0, which was a separate branch of MS-DOS development, released between mainstream MS-DOS versions 3.2 and 3.3, and sometimes referred to as "European MS-DOS 4.0".


The Portable Executable (PE) format replaced NE format in 32-bit and 64-bit versions of Windows [...].

Why is this important? Because support for NE files in modern disassemblers is usually partial at most. IDA Free doesn't support it (IDA Pro does, but is pretty expensive). Ghidra has partial support but basic functionality such as string referencing isn't supported. In this writeup, we'll use a tool called W32Dasm (version 10.0) which probably is much less convenient than IDA-Pro, but can be used freely.

We'll start from the client. After opening CLIENT.EXE in the disassembler, we click the "String Data References" button and get the following string references:

Since "Calculating Key" was the first string printed to the console, let's search for it.

The reference brings us to the following function:

Even without diving into all the details, we can make a few basic observations:

How is the data being sent? Here's the function at 0xFF2:

It is calling WRITECOMM, which is used for serial communication. And the strings window mentioned COM1, which is referenced by:

So it looks like COM1 is used for the communication.

Let's setup COM1 and allow the programs to communicate.

DOSBox supports serial communication, we just need to open the DOSBox configuration file (dosbox.conf) and change serial1=dummy to serial1=directserial realport:com1. Now the DOSBox COM1 port is connected to the physical COM1 port on our host.

Wait, what? This is 2019, we don't have a physical COM1 port on our host. We can download software to create a virtual COM1 port, but let's just use something we already have installed - VirtualBox. We just need to copy DOSBox into an existing virtual machine and configure the serial connection.

VirtualBox has several options to configure a COM port (see here).

We'll use the TCP Socket:

TCP Socket: Useful for forwarding serial traffic over TCP/IP, acting as a server, or it can act as a TCP client connecting to other servers. This option enables a remote machine to directly connect to the guest's serial port using TCP.

First, we'd like client to send us the key, so we'll setup a virtual COM port as a client and implement a TCP server.

The implementation:

The VirtualBox settings:

We start our TCP server, start our VirtualBox guest, boot Windows 3.11 via DOSBox in our guest and launch our client.

The result:

If we inspect the result with a HEX editor, we see that the first character is indeed 0x91 (our magic byte), following by the textual representation of a byte stream (512 characters, representing our 256-byte key):

Note: A different approach to extract the key, used by another team member, was to patch the client with an infinite loop, mark the memory location of the key with an easy-to-find pattern and inspect the emulator memory, searching for the pattern.

Now we have to send the data back, feeding the server. We setup the virtual COM device as a server and implement a TCP client.

The VirtualBox settings:

The implementation:

We start our VirtualBox guest, boot Windows 3.11 via DOSBox in our guest and launch our server. We then trigger our TCP client.

(Note the after some trial and error, we found out that there is need to send some NULL bytes after the key due to some OS buffering issue. Otherwise, only ~half of the buffer would be passed to the application. This was reproducible even by popping up a command line in the guest OS and running type COM1, so it's not related to DOSBox or the challenge executable).

The result:

Looks good! We now should have a DOS executable. However, when we try to run in in DOSBox, it crashes!

Time to take a look at the server disassembly.

We start from the strings:

We follow "Decrypted OK. Outputfile: MAIN.EXE" and arrive to:

Here we can see the check for the magic byte at address 0001.0C51, after which the program calls call 004C and declares that the decryptions was successful.

Inspecting that function, the main decryption loop seems to be:

It jumps over the first 0x100 bytes of MAIN.ENC and performs a simple XOR decryption on the remaining file.

After inspecting the decrypted MAIN.EXE and searching for any observations, it looks like the end of the file contains lots of meaningful text. This is a good sign. However, every now and then, some of the text was messed up. If we align the hex dump in a certain way, we can identify several columns which are corrupted, while the rest of the text seems Ok.

This can be seen in the following dump (scroll to the right to see the text):

This might mean that we got (almost) all of the key right, and something went wrong with several bytes. We can go back to the client disassembly and try to understand what went wrong, but another approach is to try and guess the key from the information we have.

Let's mark the columns that seem wrong:

This time, we marked the incorrect columns. For the first two bytes, we can guess the correct value according to the third line, that should be "strcat". For the next incorrect byte, the last line should contain "cs". And the forth byte can be guessed using "fmemcpy" in the first line. We continue to guess the correct values in a similar manner.

Since everything is aligned so nicely, we can easily use the last "hash line" we created in order to mark the errors as a base for a mask to fix the output file.

Now, we just need to XOR the current values of the file with the expected values to get the correct mask values.

After manually fixing the mask, we get:

Now we apply it to MAIN.EXE with the following Python script:

Finally, we inspect the fix:

Looks good!

Again, we try to run the program in DOSBox:

We are making progress.

In order to run the program in DOS 3.30, we will need two things:

We can create a floppy using:

Note: This didn't work on Kali, but worked on Ubuntu.

Now we launch DOSBox, mount a folder where the two floppies are located and execute boot PCDOS3~1.IMG FLOPPY.IMG to boot to DOS 3.30 with our main program available.

We run the file and get:

Again, progress, but still not quite there yet. We have a divide error. Why? Let's open the disassembly and investigate.

We open the executable in Ghidra and see the following:

Take a look at the overview bar on the right: We have a tiny (purple) code section, which in fact is fully captured in the screenshot. The code is followed by a huge (pink) allocation of data, which starts at 1000:0034 with the string UPX:

UPX (Ultimate Packer for Executables) is an open source executable packer supporting a number of file formats from different operating systems (source: Wikipedia).

In order to analyse the code, we'll have to unpack the executable:

Now, we can open the executable in Ghidra and start analysing.

The magic happens at FUN_1000_0551. Let's break it down.

First, we have logic which verifies that we are running on DOS 3.3, and exits otherwise:

Then, we have logic that checks something called a "memory signature" and a "file signature", prints them, and exits if the memory signature isn't correct:

If the signature is Ok, we move on to a loop that decrypts the flag. However, before arriving to it, there's an explicit division by zero at 1000:05ee. That must be what's crashing our program, we'll handle it in a moment.

Finally, the flag is printed:

Back to the division by zero. We can easily override the faulty instructions with NOPs and we should be good to go, right?

We copy the patched executable to the floppy, run again and get:

This looks like a partial flag, but there seems to be some corruption. Maybe it's related to the fact that we've unpacked and patched the executable?

Fortunately, we can bypass the faulty division using the DOSBox debugger, which is able to debug DOS programs in the CPU level.

In order to easily locate the program's code segment, we can use the following shortcut:

  1. Set a breakpoint on INT 21: BPINT 21 *
  2. Run the program in the DOS console: main_fix.exe
  3. We will hit the breakpoint several times. Continue execution until the program starts printing to the screen.
  4. The current code segment is probably what we're looking for.

We can see that CS = 0xD3E.

If we move our code view to 0D3E:05EE using the c cs:05EE command, we'll see our famous division:

Let's delete our global INT 21 breakpoint:

And set a breakpoint on div ax by using bp cs:05EE. We continue execution and shortly after, hit our breakpoint.

Note: The program had logic to disable INT 3 (software breakpoints). However, the DOSBox debugger does not use INT 3 to realize breakpoints, but rather uses something similar to HW breakpoints.

In the register overview, we'll see EAX=00000000, we can modify the value to 1 using sr eax 1. However, after continuing to execute, even though we've diverted the division by zero, we still get a corrupted flag!

Time to dive in deeper. Where should the flag be coming from?

Let's take another look at the logic that decrypts the flag:

The interesting locations to inspect are local_34 and local_a.

local_34 points in runtime to ds:ffc4:

If we look closely, we can see that the last four letters are ack}, which are also part of the partial flag we get, and make sense in the context. This probably means that our key ends with 00 00 00 00.

The string is XORed with es:[local_a], which points in runtime to es:64:

This points to the middle of a system error string ("Abnormal program termination") and continues with an indistinguishable byte array, following by the 4 NULL bytes we were expecting.

Let's align all three ingredients we have: The two strings which are being XORed, and the result:

Interestingly, we see that the beginning of the flag that makes sense ("BSidesTLV{DOS is") aligns perfectly with "am termination\r\n", and the corruption starts immediately after that. Moreover, the suffix ("ttack}") makes sense given the challenge name ("DoSaTTaCK"). So this must mean that for some reason, 0F52:0074-0F52:0081 are not getting the value they should. Where is their value decided?

Here are the locations in the code:

They are referenced by the following function:

This function is calling INT 21 35 (a.k.a. "Get Interrupt Vector") for four interrupts: INT 0, INT 4, INT 5 and INT 6. The return value (ES:BX = pointer to interrupt handler) is stored in the location used for our key. We must be getting different results compared to the challenge author.

For general knowledge, the interrupts are:

We tried booting DOS 3.3 on VirtualBox and got different results. However, also there, it looked like some interrupt addresses ended with 00 F0. Is it possible that only the higher address is different?

Let's take another look at the flag we already have:

Separate the corrupted part from the good part:

Assume that anything that is XORed with 00 F0 is good:

Add obvious spaces and complete the last word to "attack":

Couldn't find a single English word that fits, but if we assume these are two words, "only" jumps to mind:

It's pretty easy to complete "DOS is ??? only an attack" - DOS (which is used these days "Denial of Service") is not only an attack, it's also an operating system.

We were lucky that the authors didn't use l33t or capitals :-)

After a while, a clarification was posted by the authors:

Due to a bug in the DoSaTTaCK challenge, the flag was displayed correctly only on DOS 6.2 despite the fact that the challenge requested specifically to use DOS version 3.30. The bug has been fixed and now the flag will be displayed properly only on DOS 3.30. Those who were brave and thorough enough to try it on several environments and solved the challenge with the bug will get 300 bonus points

Looks like there's more than one way to get a flag.