Last week, our friends at Sucuri sent us a modified version of an Apache webserver redirecting some of its requests to the infamous Blackhole exploit packs. Sucuri has published a blog post on this attack. (You can find more about Blackhole here.)
Our analysis of this malware, dubbed Linux/Cdorked.A, reveals that it is a sophisticated and stealthy backdoor meant to drive traffic to malicious websites. We urge system administrators to check their servers and verify that they are not affected by this threat. Detailed instructions to perform this check are provided below. (Update 5/1/2013: An improved tool coded in C replaced the Python script we originally published.)
In fact, Linux/Cdorked.A is one of the most sophisticated Apache backdoors we have seen so far. Although we are still processing the data, our Livegrid system reports hundreds of compromised servers. The backdoor leaves no traces of compromised hosts on the hard drive other than its modified httpd binary, thereby complicating forensics analysis. All of the information related to the backdoor is stored in shared memory. The configuration is pushed by the attacker through obfuscated HTTP requests that aren't logged in normal Apache logs. This means that no command and control information is stored anywhere on the system.
Technical analysis of Linux/Cdorked
Here we provide the first technical analysis of Linux/Cdorked, which seems to be affecting hundreds of webservers right now. In the Linux/Cdorked binary all the important or suspicious strings are encrypted. As shown in the following image, a function is responsible for decrypting the strings on demand with a static XOR key.
The version of Linux/Cdorked that we have analyzed contains a total of 70 strings that are encoded this way. As shown in the following screenshot, the key used for encoding the data is 27A4E2DADAF183B51E3DA7F6C9E6239CDFC8A2E50A60E05F.
As mentioned before, Linux/Cdorked does not write any files on the disk. Instead, it allocates around six megabytes of shared memory to keep its state and configuration information. This memory block, a POSIX shared region of memory (shm), is used by all Apache subprocesses but can also be accessed by any other process since the malware authors didn't limit its permission. The following screenshot shows the (read, write for everyone) permission rights assigned to the shared memory region.
There are two ways the attacker can control the behavior of the backdoored server: through a reverse connect shell or through special commands, all of them are triggered via HTTP requests.
The Linux/Cdorked.A backdoor
The HTTP server is equipped with a reverse connect backdoor that can be triggered via a special HTTP GET request. It is invoked when a request to a special path is performed with a query string in a particular format, containing the hostname and port to connect. The client IP of the HTTP dialog is used as a key to decrypt the query string as a 4 byte XOR key. Additionally, IP specified in X-Real-IP or X-Forwarded-For headers will override the client IP as the XOR key. This means we can craft a X-Real-IP header that will in effect be a “\x00\x00\x00\x00” key. Query string also needs to be hex-encoded before sending.
While the shell is used by the attacker, the HTTP connection creating it is hung (the backdoor code does not implement forking). This implies that malicious shells can be found if one has access to the server and checks for long-running HTTP connections. On the other hand, the HTTP request does not appear in Apache's log file due to the way the malicious code is hooked into Apache.
Redirection in Linux/Cdorked.A
When redirecting a client, the malware adds base64 encoded string to the query containing information like the original visited URL and whether or not the request was originally to a javascript file so the server could provide the right payload.
An example redirection looks like:
Location: hxxp://dcb84fc82e1f7b01. xxxxxxgsm.be/index.php?j=anM9MSZudmNiaW11Zj1jY3
Zja3FqdSZ0aW1lPTEzMDQxNjE4MjctMzYwNDUzNjUwJnNyYz0yMzImc3VybD13d3cuaW5mZWN0ZWRzZXJ2
ZXIuY29tJnNwb3J0PTgwJmtleT0xM0Q5MDk1MCZzdXJpPS9mb3J1bS93Y2YvanMvM3JkUGFydHkvcHJvdG
9hY3Vsb3VzLjEuOC4yLm1pbi5qcw==After decoding, the following parameters appear:
js=1&nvcbimuf=ccvckqju&time=1304161827-360453650&src=232&surl=www.infectedserver
.com&sport=80&key=13D90950&suri=/forum/wcf/js/3rdParty/protoaculous.1.8.2.min.jsThe "surl" parameter shows the infected host and the "suri" indicates what the original requested resource was.
After the redirection, a web cookie is set on the client so it is not redirected again. This cookie is also set if a request is made to a page that looks like an administration page. The backdoor will check if the URL, the server name, or the referrer matches any of the following strings : '*adm*', '*webmaster*', '*submit*', '*stat*', '*mrtg*', '*webmin*', '*cpanel*', '*memb*', '*bucks*', '*bill*', '*host*', '*secur*', '*support*'. This is probably done to avoid sending malicious content to administrators of the website, making the infection harder to spot. The following screenshot shows part of the code responsible for handling the web cookie.
A few other conditions must be met before redirection happens; for example, a check is done for the presence of the Accept-Language, Accept-Encoding, and Referrer header.
Other Linux/Cdorked.A commands
We found 23 commands in Linux/Cdorked.A that can be sent to the server via a POST to a specially crafted URL. The request must also contain a cookie header starting with "SECID=". The query string value must hold 2 hex encoded bytes that are encrypted with the client IP, using the same technique as the shell. The SECID cookie data will be used as arguments to some of the commands. We believe that the URLs to redirect clients are sent to the backdoor using this method. The redirection information will be stored encrypted in the allocated shared memory region. We also believe that the conditions for redirection are set this way, for example, a white list of user agents to redirect can be preconfigured and a black list of IPs to avoid redirection.
This is the complete list of commands found in the binary we have analysed: 'DU', 'ST', 'T1', 'L1', 'D1', 'L2', 'D2', 'L3', 'D3', 'L4', 'D4', 'L5', 'D5', 'L6', 'D6', 'L7', 'D7', 'L8', 'D8', 'L9', 'D9', 'LA', 'DA'.
Finally, some information about the status of the backdoor is returned in the ETag HTTP header, as shown in the screenshot below. We are still investigating the purpose of each of the commands and will publish our results as soon as the analysis is completed. In short, they all either add content to, or remove it from, the configuration in the shared memory region.
Linux/Cdorked.A Remediation
As previously mentioned, the permissions on the shared memory allocation are loose. This allows other process to access to memory. We have made a free tool to allow systems administrators to verify the presence of the shared memory region and dump its content into a file (you can download dump_cdorked_config.c or copy and paste the code from the listing below).
We also recommend using debsums for Debian or Ubuntu systems and `rpm --verify` for RPM based systems, to verify the integrity of your Apache web server package installation. (However, remember to temper this advice with the reality that the package manifest could have been altered by an attacker.) Checking for the presence of the shared memory is the recommended way to make sure you are not infected. We would be interested in receiving any memory dumps for further analysis.
At the time of writing, the ESET Livegrid monitoring system is showing hundreds of webservers that seem to be affected by this backdoor with thousands of visitors being redirected to malicious content. We will publish more information on the scale and complexity of this operation in the days to come.
The following researchers contributed to this report: Olivier Bilodeau, François Chagnon, Alexis Dorais-Joncas, Sebastien Duquette and Marc-Étienne Léveillé.
Resources:
SHA1 of the analyzed binary: 24e3ebc0c5a28ba433dfa69c169a8dd90e05c429Code for the shared memory dump tool: dump_cdorked_config.c
// This program dumps the content of a shared memory block
// used by Linux/Cdorked.A into a file named httpd_cdorked_config.bin
// when the machine is infected.
//
// Some of the data is encrypted. If your server is infected and you
// would like to help, please send the httpd_cdorked_config.bin
// and your httpd executable to our lab for analysis. Thanks!
//
// Build with gcc -o dump_cdorked_config dump_cdorked_config.c
//
// Marc-Etienne M.Léveillé <leveille@eset.com>
//
#include <stdio.h>
#include <sys/shm.h>
#define CDORKED_SHM_SIZE (6118512)
#define CDORKED_OUTFILE "httpd_cdorked_config.bin"
int main (int argc, char *argv[]) {
int maxkey, id, shmid, infected = 0;
struct shm_info shm_info;
struct shmid_ds shmds;
void * cdorked_data;
FILE * outfile;
maxkey = shmctl(0, SHM_INFO, (void *) &shm_info);
for(id = 0; id <= maxkey; id++) {
shmid = shmctl(id, SHM_STAT, &shmds);
if (shmid < 0)
continue;
if(shmds.shm_segsz == CDORKED_SHM_SIZE) {
// We have a matching Cdorked memory segment
infected++;
printf("A shared memory matching Cdorked signature was found.\n");
printf("You should check your HTTP server's executable file integrity.\n");
cdorked_data = shmat(shmid, NULL, 0666);
if(cdorked_data != NULL) {
outfile = fopen(CDORKED_OUTFILE, "wb");
if(outfile == NULL) {
printf("Could not open file %s for writing.", CDORKED_OUTFILE);
}
else {
fwrite(cdorked_data, CDORKED_SHM_SIZE, 1, outfile);
fclose(outfile);
printf("The Cdorked configuration was dumped in the %s file.\n\n", CDORKED_OUTFILE);
}
}
}
}
if(infected == 0) {
printf("No shared memory matching Cdorked signature was found.\n");
printf("To further verify your server, run \"ipcs -m -p\" and look");
printf(" for a memory segments created by your http server.\n");
}
else {
printf("If you would like to help us in our research on Cdorked, ");
printf("please send the httpd_cdorked_config.bin and your httpd executable file ");
printf("to our lab for analysis at leveille@eset.com. Thanks!\n");
}
return infected;
}
