Updated 6 July 2022: Wallarm launched – GoTestWAF: Free Online WAF tester – https://www.wallarm.com/gotestwaf/overview
Updated 18 January 2023: added gRPC protocol detection and other features, more details in the article.
Since 1991, Web Application Firewall, commonly referred to as WAF, has become one of the most common application security technologies available on the market. Since the last century, WAFs have evolved by incorporating the cloud and using Machine Learning instead of RegExp.
Currently, few technologies, such as NG-WAF, RASP, WAAP, and a few others, have internal WAF capabilities, which prevent web applications and API threats.
Majority of the fintech, health tech, and e-commerce companies have had WAFs installed for years to protect their APIs, but also due to PCI DSS, SOC2, and HIPAA compliance requirements, bot mitigation, and OWASP Top-10 attack prevention needs.
How good is my WAF?
WAF-like technologies have already been in place for a while, but how good are they? Given the notorious issue of false positives that WAFs have always been known for, people often focus on evaluating false positive rates while ignoring the testing of false negative rates. In addition, it is not easy to test and check the actual level of WAF or RASP protection.
So how can we really test how good your current WAF is? What attacks can it really stop and where can application and API attacks still hit even with a WAF in place? Where are the blind spots? There isn’t a simple tool that any developer, QA, or security engineer can run to get a PDF report on your WAF coverage.
Meet GoTestWAF
To address the issue, we are open-sourcing a project called GoTestWAF.
GoTestWAF generates requests with predefined, basic payloads as well as attacks specific to different APIs (REST, SOAP, XMLRPC, gRPC). Afterwards, it sends them to the application and analyzes the responses to generate a detailed report in the console output or as a PDF.
The usage is pretty simple. It only requires that you run this tool against your WAF protected application.
docker run -v /tmp:/tmp/report gotestwaf --url=http://the-waf-you-wanna-test/
It gives clear results and indicates which of the attacks are detected with your existing appsec solution and what ways attackers can still hit your apps:
owasp path-traversal 6/18 (0.33) owasp sql-injection 4/16 (0.25) owasp ss-include 2/8 (0.25) owasp xml-injection 6/8 (0.75) owasp xss-scripting 4/12 (0.33) owasp ldap-injection 0/8 (0.00) owasp mail-injection 3/12 (0.25) owasp nosql-injection 0/18 (0.00) owasp shell-injection 3/8 (0.38) owasp sst-injection 5/20 (0.25)
It’s a simple yet powerful tool that supports not only pre-defined payloads but also API requests, such as REST, SOAP, and XMLRPC, and custom encodings like Base64, which can be extended easily.
In conclusion, the GoTestWAF is a tool to test WAFs, RASPs, and WAAP for application and API attacks, not just CGI payloads from 90th.
How it works?
The main idea is to encode and place attack payloads in different parts of an HTTP request: its body, headers, URL parameters, etc. To simplify things, we decided to implement the following logic:
Payload -> Encoder -> Placeholder
It means that every payload sample (malicious attack sample such as an XSS string like “<script>alert(1)</script>”) will be first encoded in some way then placed into an HTTP request. There is also an option to use a plain encoder that keeps the string as-is.
To make tests readable, we introduced a YAML DSL with an equal structure (payload->encoder->placeholder), where all the fields are arrays and make a test by permutations between them.
Take a look at this SQL-injection test readable:
payload: - '"union select -7431.1, name, @aaa from u_base--w-' - "'or 123.22=123.22" - "' waitfor delay '00:00:10'--" - "')) or pg_sleep(5)--" encoder: - Base64Flat - Url placeholder: - UrlPath - UrlParam - JsonBody - Header
As a result of the permutation of each 4 payloads, 2 encoders, and 4 placeholders, this test will send 4*2*4=32 requests.
Note: be careful with the YAML. If you need to send a few binary payloads, then it is best to use !!binary attribute on the payload fields.
How to run
There is an Alpine-based Docker image available. It is ready to start out-of-the-box with a single command-line right after “git clone” and “docker build”:
$ git clone https://github.com/wallarm/gotestwaf $ docker build . -t gotestwaf $ docker run -v /tmp:/tmp/report gotestwaf --url=http://the-waf-you-wanna-test/
The only parameter here is a directory where the PDF report will be placed. If you need the report, you are required to map it from your host machine via -v docker attribute. Otherwise, all the stats will be available in your console only.
If you need to configure some more parameters, here are the command-line arguments and config.yaml:
Usage of /go/src/gotestwaf/gotestwaf:
-block_regexp string
Regular Expression to detect blocking page with the same HTTP response status code as not blocked request
-block_statuscode int
HTTP response status code that WAF uses while blocking requests. 403 by default (default 403)
-check_cert
Check SSL/TLS certificates, turned off by default
-config string
Config file to use. Attention, if you are using the config, all the are flags will be avoided. (default "config.yaml")
-proxy string
Proxy to use
-report string
Report filename to export results. (default "/tmp/report/waf-test-report2020-April-10.pdf")
-testcases string
Folder with test cases (default "./testcases/")
-threads int
Number of concurrent HTTP requests (default 2)
-url string
URL with a WAF to check (default "http://localhost")
Command-line flags have priority over configuration files. As you can see, the most important thing here is how to detect a blocking page. Currently, you can do it with a response status or RegExp that will be applied to the request body. By default, the 403 response status code will be used for this.
You can also choose test cases between two embedded: OWASP Top-10, OWASP-API, or your own.
The report sample
Again, the main goal of this tool is to easily generate readable reports to check the current state of protection according to OWASP guidelines.
To see what it looks like, we can install mod_security WAF and run it on the 8080 port:
git clone https://github.com/SpiderLabs/owasp-modsecurity-crs cd owasp-modsecurity-crs/util/docker docker build -t modsec_crs --file Dockerfile-3.0-nginx . docker run --rm -p 8080:80 -e PARANOIA=1 modsec_crs
Next, we can launch the GoTestWAF utility:
docker run -v /tmp:/tmp/report gotestwaf --url=http://the-waf-you-wanna-test/
The results will be printed in your console as such:
owasp path-traversal 6/18 (0.33) owasp sql-injection 4/16 (0.25) owasp ss-include 2/8 (0.25) owasp xml-injection 6/8 (0.75) owasp xss-scripting 4/12 (0.33) owasp ldap-injection 0/8 (0.00) owasp mail-injection 3/12 (0.25) owasp nosql-injection 0/18 (0.00) owasp shell-injection 3/8 (0.38) owasp sst-injection 5/20 (0.25)
The PDF version will also contain bupasses with details, screenshot below:
We updated the 2023 GoTestWAF tool with the following features:
- Sending a report by email
- Ability to perform testing using OpenAPI specification
- Ability to add debug header (request from client), specify multiple status codes (blockStatusCodes, passStatusCodes).
- Since there were some modifications for sessions, service can detect some (2 or 3) vendors automatically and indicate them in report.
- The report view has changed + now you can get 2 types of report:
- full report with all bypassed vectors
- The short version of the report with graphs and statistics.
- We also have the ability to generate json report and json output, so you can embed GTW somewhere, for example in the pipeline
- We also improved passRegex, blockRegex – now not only the response body, but the entire HTTP-response header is analyzed.
- Added normal gRPC protocol detection and ability to detect gRPC on a particular port
The new report looks like this:
Graphs from the new 2023 report:
For the technical details about payload, encoders, and responses, you can check the CSV file that will be generated in the same folder as PDF report.
The new report has a comparison with other solutions:
Future improvements
We invite everyone to join our mission and commit to the project, so that we can collectively improve the task at hand. The current roadmap includes support of WebSockets and .NET specific attack improvements, as well as some bots-emulations and other new ideas recommended by current users. Any pull requests and improvement suggestions are greatly appreciated.
Frequently Asked Questions
How do you test WAF rules?
To test WAF rules, you can use penetration testing tools such as Burp Suite, OWASP ZAP, or Nmap to simulate attacks on your website. These tools can help identify vulnerabilities and ensure that WAF rules are functioning correctly.
How do I check my WAF application?
One way to check your WAF application is to perform regular security audits and vulnerability assessments. This can help identify any potential WAF issues that need to be addressed.
How do you audit a WAF?
Auditing a WAF involves evaluating the configuration, log files, and network traffic to ensure that the system is functioning correctly. Additionally, it is important to review attack detection and mitigation policies to determine their effectiveness.
What is WAF testing?
WAF testing is a process of evaluating the effectiveness of a web application firewall (WAF) against potential web-based attacks. This involves testing the WAF configuration, monitoring system logs, and assessing the response to known vulnerabilities.
References
Wallarm Cloud-Native WAAP – Product
