Web server technology detection by using known filetypes

Web servers use server-side scripting engines to support web applications. Using the web server configuration, dynamic pages are identified by filename extensions and processed by their relevant processors.

Below is an example of IIS 6.0 application configuration. As you can see, asp files are processed by asp.dll and php files are processed by php-cgi application. Other web servers also use similar methods.


In some situations, web servers respond differently to nonexistent files with known extensions. Sending requests to random filenames with known extensions and comparing the HTTP response results may reveal the server-side scripting technologies supported by the web server.

While scanning customer networks, I saw that various web servers can respond differently to many known extensions including asp, aspx, cfm, php, jsp, shtml. Also additional vulnerabilities can be found in these responses, including Internal IP address disclosure, application errors etc.


You can see some real life examples below. We will send a simple request to nonexistent asp, html, and php files. Then compare the responses. These tests will be made to the root directory, but you should also notice that some special subdirectories’ configuration might be different.

$ nc www.baidu.com 80
GET /CsARl9W0s9esF7Vl HTTP/1.1
Host: www.baidu.com

HTTP/1.1 302 Found
Date: Sat, 31 Oct 2009 15:19:34 GMT
Server: Apache/1.3.27
Location: http://www.baidu.com/search/error.html
Transfer-Encoding: chunked
Content-Type: text/html; charset=iso-8859-1
...

GET /CsARl9W0s9esF7Vl.html HTTP/1.1
Host: www.baidu.com

HTTP/1.1 302 Found
Date: Sat, 31 Oct 2009 15:19:50 GMT
Server: Apache/1.3.27
Location: http://www.baidu.com/search/error.html
Transfer-Encoding: chunked
Content-Type: text/html; charset=iso-8859-1
...

GET /CsARl9W0s9esF7Vl.asp HTTP/1.1
Host: www.baidu.com

HTTP/1.1 302 Found
Date: Sat, 31 Oct 2009 15:20:00 GMT
Server: Apache/1.3.27
Location: http://www.baidu.com/search/error.html
Transfer-Encoding: chunked
Content-Type: text/html; charset=iso-8859-1
...

GET /CsARl9W0s9esF7Vl.php HTTP/1.1
Host: www.baidu.com

HTTP/1.1 302 Found
Date: Sat, 31 Oct 2009 15:20:22 GMT
Server: Apache/1.3.27
Location: http://www.baidu.com/forbiddenip/forbidden.html
Transfer-Encoding: chunked
Content-Type: text/html; charset=iso-8859-1
...



As you can clearly see, the web server responds to nonexisting files as 302 found message and redirects us to /search/error.html. But when it comes to PHP files, we see a redirection to /forbiddenip/forbidden.html


Below is another example.

$ nc wiki.nginx.org 80
GET /CsARl9W0s9esF7Vl HTTP/1.1
Host: wiki.nginx.org

HTTP/1.1 404
Server: nginx/0.8.21
Date: Sat, 31 Oct 2009 15:26:10 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.1.6
Content-language: en
Vary: Accept-Encoding, Cookie
X-Vary-Options: Accept-Encoding;list-contains=gzip,Cookie;string-contains=wikidbToken;string-contains=wikidbLoggedOut;string-contains=wikidb_session
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: private, must-revalidate, max-age=0

2e8c
...


$ nc wiki.nginx.org 80
GET /CsARl9W0s9esF7Vl.asp HTTP/1.1
Host: wiki.nginx.org


HTTP/1.1 404
Server: nginx/0.8.21
Date: Sat, 31 Oct 2009 15:26:44 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.1.6
Content-language: en
Vary: Accept-Encoding, Cookie
X-Vary-Options: Accept-Encoding;list-contains=gzip,Cookie;string-contains=wikidbToken;string-contains=wikidbLoggedOut;string-contains=wikidb_session
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: private, must-revalidate, max-age=0

2eec
...


GET /CsARl9W0s9esF7Vl.php HTTP/1.1
Host: wiki.nginx.org


HTTP/1.1 404
Server: nginx/0.8.21
Date: Sat, 31 Oct 2009 15:27:15 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.1.6

19
No input file specified.

0


Website responds differently to nonexistent php files. So, X-Powered-By PHP header seems valid. Since URL rewriting is in place, we can clearly verify that the wiki application used by nginx is written in PHP (It is actually MediaWiki).

I call this method as technology detection by using known filetypes. Using this method in addition to other fingerprinting techniques, such as HTTP response banner grabbing, is useful to improve web security scanners.

I previously implemented this in Arachne, a simple web security scanner that I developed for my MSc thesis back in 2006. It sends requests to nonexistent files with known extensions, then compares the results to see if a technology is used in that web server.

With technology detection by using known filetypes;

• Web application scans can be optimized for detected technologies
• Web application scanners can reduce the number of tests performed
• Scanners can reduce false-positives for nonexistent files
• If URL rewriting is used, this method can be used to determine used technologies in the web application