Since 1991, Web Application Firewall, commonly referred to as WAF, has become one of the most common application security technologies available on the market. Since the last century, WAFs have evolved by incorporating the cloud and using Machine Learning instead of RegExp.
Currently, few technologies, such as NG-WAF, RASP, WAAP, and a few others, have internal WAF capabilities, which prevent web applications and API threats.
Majority of the fintech, health tech, and e-commerce companies have had WAFs installed for years to protect their APIs, but also due to PCI DSS, SOC2, and HIPAA compliance requirements, bot mitigation, and OWASP Top-10 attack prevention needs.
How good is my WAF?
WAF-like technologies have already been in place for a while, but how good are they? Given the notorious issue of false positives that WAFs have always been known for, people often focus on evaluating false positive rates while ignoring the testing of false negative rates. In addition, it is not easy to test and check the actual level of WAF or RASP protection.
So how can we really test how good your current WAF is? What attacks can it really stop and where can application and API attacks still hit even with a WAF in place? Where are the blind spots? There isn’t a simple tool that any developer, QA, or security engineer can run to get a PDF report on your WAF coverage.
To address the issue, we are open-sourcing a project called GoTestWAF.
GoTestWAF generates requests with predefined, basic payloads as well as attacks specific to different APIs (REST, SOAP, XMLRPC). Afterwards, it sends them to the application and analyzes the responses to generate a detailed report in the console output or as a PDF.
The usage is pretty simple. It only requires that you run this tool against your WAF protected application.
It gives clear results and indicates which of the attacks are detected with your existing appsec solution and what ways attackers can still hit your apps:
It’s a simple yet powerful tool that supports not only pre-defined payloads but also API requests, such as REST, SOAP, and XMLRPC, and custom encodings like Base64, which can be extended easily.
In conclusion, the GoTestWAF is a tool to test WAFs, RASPs, and WAAP for application and API attacks, not just CGI payloads from 90th.
How it works?
The main idea is to encode and place attack payloads in different parts of an HTTP request: its body, headers, URL parameters, etc. To simplify things, we decided to implement the following logic:
Payload -> Encoder -> Placeholder
It means that every payload sample (malicious attack sample such as an XSS string like “<script>alert(1)</script>”) will be first encoded in some way then placed into an HTTP request. There is also an option to use a plain encoder that keeps the string as-is.
To make tests readable, we introduced a YAML DSL with an equal structure (payload->encoder->placeholder), where all the fields are arrays and make a test by permutations between them.
Take a look at this SQL-injection test readable:
As a result of the permutation of each 4 payloads, 2 encoders, and 4 placeholders, this test will send 4*2*4=32 requests.
Note: be careful with the YAML. If you need to send a few binary payloads, then it is best to use !!binary attribute on the payload fields.
How to run
There is an Alpine-based Docker image available. It is ready to start out-of-the-box with a single command-line right after “git clone” and “docker build”:
The only parameter here is a directory where the PDF report will be placed. If you need the report, you are required to map it from your host machine via -v docker attribute. Otherwise, all the stats will be available in your console only.
If you need to configure some more parameters, here are the command-line arguments and config.yaml:
Command-line flags have priority over configuration files. As you can see, the most important thing here is how to detect a blocking page. Currently, you can do it with a response status or RegExp that will be applied to the request body. By default, the 403 response status code will be used for this.
You can also choose test cases between two embedded: OWASP Top-10, OWASP-API, or your own.
The report sample
Again, the main goal of this tool is to easily generate readable reports to check the current state of protection according to OWASP guidelines.
To see what it looks like, we can install mod_security WAF and run it on the 8080 port:
Next, we can launch the GoTestWAF utility:
The results will be printed in your console as such:
The PDF version will also contain a summary with an overall score, similar to the one below:
It will also include a few details about which attacks were passed instead of blocked:
We invite everyone to join our mission and commit to the project, so that we can collectively improve the task at hand. The current roadmap includes support of WebSockets, gRPC, and .NET specific attack improvements, as well as some bots-emulations and other new ideas recommended by current users. Any pull requests and improvement suggestions are greatly appreciated.