Since 1991, Web Application Firewall, commonly referred to as WAF, has become one of the most common application security technologies available on the market. Since the last century, WAFs have evolved by incorporating the cloud and using Machine Learning instead of RegExp.

Currently, few technologies, such as NG-WAF, RASP, WAAP, and a few others, have internal WAF capabilities, which prevent web applications and API threats.

Majority of the fintech, health tech, and e-commerce companies have had WAFs installed for years to protect their APIs, but also due to PCI DSS, SOC2, and HIPAA compliance requirements, bot mitigation, and OWASP Top-10 attack prevention needs.

How good is my WAF?

WAF-like technologies have already been in place for a while, but how good are they? Given the notorious issue of false positives that WAFs have always been known for, people often focus on evaluating false positive rates while ignoring the testing of false negative rates. In addition, it is not easy to test and check the actual level of WAF or RASP protection. 

So how can we really test how good your current WAF is? What attacks can it really stop and where can application and API attacks still hit even with a WAF in place? Where are the blind spots? There isn’t a simple tool that any developer, QA, or security engineer can run to get a PDF report on your WAF coverage.

Meet GoTestWAF

To address the issue, we are open-sourcing a project called GoTestWAF

GoTestWAF generates requests with predefined, basic payloads as well as attacks specific to different APIs (REST, SOAP, XMLRPC). Afterwards, it sends them to the application and analyzes the responses to generate a detailed report in the console output or as a PDF.

The usage is pretty simple. It only requires that you run this tool against your WAF protected application. 

docker run -v /tmp:/tmp/report gotestwaf --url=http://the-waf-you-wanna-test/

It gives clear results and indicates which of the attacks are detected with your existing appsec solution and what ways attackers can still hit your apps:

owasp   path-traversal   6/18  (0.33)
owasp   sql-injection    4/16  (0.25)
owasp   ss-include       2/8   (0.25)
owasp   xml-injection    6/8   (0.75)
owasp   xss-scripting    4/12  (0.33)
owasp   ldap-injection   0/8   (0.00)
owasp   mail-injection   3/12  (0.25)
owasp   nosql-injection  0/18  (0.00)
owasp   shell-injection  3/8   (0.38)
owasp   sst-injection    5/20  (0.25)

It’s a simple yet powerful tool that supports not only pre-defined payloads but also API requests, such as REST, SOAP, and XMLRPC, and custom encodings like Base64, which can be extended easily.

In conclusion, the GoTestWAF is a tool to test WAFs, RASPs, and WAAP for application and API attacks, not just CGI payloads from 90th. 

How it works?

The main idea is to encode and place attack payloads in different parts of an HTTP request: its body, headers, URL parameters, etc. To simplify things, we decided to implement the following logic:

Payload -> Encoder -> Placeholder

It means that every payload sample (malicious attack sample such as an XSS string like “<script>alert(1)</script>”) will be first encoded in some way then placed into an HTTP request. There is also an option to use a plain encoder that keeps the string as-is.

To make tests readable, we introduced a YAML DSL with an equal structure (payload->encoder->placeholder), where all the fields are arrays and make a test by permutations between them. 

Take a look at this SQL-injection test readable:

payload:
  - '"union select -7431.1, name, @aaa from u_base--w-'
  - "'or 123.22=123.22"
  - "' waitfor delay '00:00:10'--"
  - "')) or pg_sleep(5)--"
encoder:
  - Base64Flat
  - Url
placeholder:
  - UrlPath
  - UrlParam
  - JsonBody
  - Header

As a result of the permutation of each 4 payloads, 2 encoders, and 4 placeholders, this test will send 4*2*4=32 requests.

Note: be careful with the YAML. If you need to send a few binary payloads, then it is best to use !!binary attribute on the payload fields.

How to run

There is an Alpine-based Docker image available. It is ready to start out-of-the-box with a single command-line right after “git clone” and “docker build”:

$ git clone https://github.com/wallarm/gotestwaf
$ docker build . -t gotestwaf
$ docker run -v /tmp:/tmp/report gotestwaf --url=http://the-waf-you-wanna-test/

The only parameter here is a directory where the PDF report will be placed. If you need the report, you are required to map it from your host machine via -v docker attribute. Otherwise, all the stats will be available in your console only.

If you need to configure some more parameters, here are the command-line arguments and config.yaml: 

Usage of /go/src/gotestwaf/gotestwaf:
  -block_regexp string
    	Regular Expression to detect blocking page with the same HTTP response status code as not blocked request
  -block_statuscode int
    	HTTP response status code that WAF uses while blocking requests. 403 by default (default 403)
  -check_cert
    	Check SSL/TLS certificates, turned off by default
  -config string
    	Config file to use. Attention, if you are using the config, all the are flags will be avoided. (default "config.yaml")
  -proxy string
    	Proxy to use
  -report string
    	Report filename to export results. (default "/tmp/report/waf-test-report2020-April-10.pdf")
  -testcases string
    	Folder with test cases (default "./testcases/")
  -threads int
    	Number of concurrent HTTP requests (default 2)
  -url string
    	URL with a WAF to check (default "http://localhost")

Command-line flags have priority over configuration files. As you can see, the most important thing here is how to detect a blocking page. Currently, you can do it with a response status or RegExp that will be applied to the request body. By default, the 403 response status code will be used for this.

You can also choose test cases between two embedded: OWASP Top-10, OWASP-API, or your own. 

The report sample

Again, the main goal of this tool is to easily generate readable reports to check the current state of protection according to OWASP guidelines. 

To see what it looks like, we can install mod_security WAF and run it on the 8080 port:

git clone https://github.com/SpiderLabs/owasp-modsecurity-crs
cd owasp-modsecurity-crs/util/docker
docker build -t modsec_crs --file Dockerfile-3.0-nginx .
docker run --rm -p 8080:80 -e PARANOIA=1 modsec_crs

Next, we can launch the GoTestWAF utility:

docker run -v /tmp:/tmp/report gotestwaf --url=http://the-waf-you-wanna-test/

The results will be printed in your console as such:

owasp   path-traversal   6/18  (0.33)
owasp   sql-injection    4/16  (0.25)
owasp   ss-include       2/8   (0.25)
owasp   xml-injection    6/8   (0.75)
owasp   xss-scripting    4/12  (0.33)
owasp   ldap-injection   0/8   (0.00)
owasp   mail-injection   3/12  (0.25)
owasp   nosql-injection  0/18  (0.00)
owasp   shell-injection  3/8   (0.38)
owasp   sst-injection    5/20  (0.25)

The PDF version will also contain a summary with an overall score, similar to the one below:

It will also include a few details about which attacks were passed instead of blocked:

Future improvements

We invite everyone to join our mission and commit to the project, so that we can collectively improve the task at hand. The current roadmap includes support of WebSockets, gRPC, and .NET specific attack improvements, as well as some bots-emulations and other new ideas recommended by current users. Any pull requests and improvement suggestions are greatly appreciated.

Write A Comment