In the previous post, you learnt how to execute an automated penetration test by means of Zed Attack Proxy (ZAP). This time, you will learn how to execute the test via a Command Line Interface (CLI) which will make it possible to add the test to your CI/CD pipeline.
In the previous post, the different steps were explained how to execute an automated penetration test. The application under test being used was WebGoat, a vulnerable application developed by OWASP in order to learn security vulnerabilities. This application will be used in this post also. The steps to be executed for a penetration test with ZAP are:
- Setup the context and session (especially login credentials when a part of the application is behind a login);
- Manually explore all parts of the application;
- Spider the application;
- Execute the automated scan;
- Inspect the results.
Beware that an automated scan will not find all vulnerabilities! It is always necessary to execute a manual penetration test. The automated scan will however give you a good indication about the state of security of your application.
The files being used in this post are available at GitHub in directory
2. ZAP CLI
ZAP contains an API for controlling ZAP. The ZAP CLI tool is a tool which wraps the API in order that commands can be executed via the command line. In this section, you basically will perform the same or similar actions as in the previous post, except that you will not use the ZAP Desktop this time. A complete list of the commands of ZAP CLI can be found at the GitHub website.
2.1 Installation and Preparation
As a prerequisite, you must have ZAP already installed of course. Next to ZAP, you need to install ZAP CLI, which is quite simple. Just execute the following command:
$ pip install --upgrade zapcli
Before you continue, it is very important to set the following environment variables:
export ZAP_PORT=8090 export ZAP_PATH=/usr/local/bin/zap.sh export ZAP_API_KEY=<your API key>
The ZAP_API_KEY can be found in ZAP Desktop. Therefore, start ZAP Desktop and choose Tools – Options… in the menu. In the API section, the API key is shown and needs to be used for the environment variable (but do not yet set the environment variable until it is mentioned to do so in the next section). You also have the possibility to disable the usage of the key, but this is not advised.
First, close the ZAP Desktop tool before you continue. Let’s see what happens when you try to start ZAP:
$ zap-cli start [INFO] Starting ZAP daemon Traceback (most recent call last): File "/home/<user>/.local/bin/zap-cli", line 8, in <module> sys.exit(cli()) File "/home/<user>/.local/lib/python3.8/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/home/<user>/.local/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/home/<user>/.local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/<user>/.local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/<user>/.local/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/home/<user>/.local/lib/python3.8/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context().obj, *args, **kwargs) File "/home/<user>/.local/lib/python3.8/site-packages/zapcli/cli.py", line 58, in start_zap_daemon zap_helper.start(options=start_options) File "/home/<user>/.local/lib/python3.8/site-packages/zapcli/zap_helper.py", line 88, in start with open(log_path, 'w+') as log_file: PermissionError: [Errno 13] Permission denied: '/usr/local/bin/zap.log'
It seems that you do not have sufficient rights to write to the
zap.log. There are several solutions for this, but the easiest one is to set the log path to a directory where you have sufficient permissions. Navigate to this directory and execute the following command:
$ zap-cli --log-path . start [INFO] Starting ZAP daemon
This seems to have started the ZAP daemon. Let’s verify this:
$ zap-cli status [INFO] ZAP is running
Allright, ZAP is running. Let’s try to shutdown the ZAP daemon:
$ zap-cli shutdown [INFO] Shutting down ZAP daemon ... raise ProxyError(e, request=request) requests.exceptions.ProxyError: HTTPConnectionPool(host='127.0.0.1', port=8090): Max retries exceeded with url: http://zap/JSON/core/action/shutdown/?apikey= (Caused by ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response')))
Why is this not working? The reason for the exception is quite vague because it mentions ‘Cannot connect to proxy’ and ‘Remote end closed connection without response’. But why? The answer is shown in the preceding URL
http://zap/JSON/core/action/shutdown/?apikey=. As you can see, the parameter
apikey is empty. There are numerous questions on the internet about this exception, but very few will give you the answer. The solution is to set the ZAP_API_KEY environment variable.
$ export ZAP_API_KEY=<your API key> $ zap-cli . shutdown [INFO] Shutting down ZAP daemon
And now it is possible to shut down the ZAP daemon. It appears that you do not need the API key for every API call, but when you set the environment variables, you will not need to worry about this.
2.2 Context Preparation
Remember from the previous post, that it was important to set user credentials in the Context and the manual exploration of the website. In this post, you will reuse this information in order to feed it to ZAP CLI.
Open the previously stored session via File – Open Session… and export the context via File – Export Context… Select the context, the path where you want to save it and click the Save button.
The manually explored URLs of the application are also present in the Session. You will need those URLs for ZAP CLI. Right-click in the Sites section on http://localhost:8080/ and choose Export Selected URLs to File… and name the file
webgoat-exported-urls.txt. You will need this file in the next paragraph.
Close ZAP Desktop.
2.3 Scan With ZAP CLI
At this moment, you have done all the necessary preparation in order to get started to scan your application with ZAP CLI. Identical steps as during the ZAP Desktop scan will be performed, but this time via the ZAP CLI and you will make use of the Context and the exported URLs which have been created in the previous post.
Before you continue, ensure that the WebGoat application is running. Otherwise, it is a good moment to start the application now.
$ docker start goatandwolf
Start the ZAP daemon.
$ zap-cli --log-path . start [INFO] Starting ZAP daemon
Import the context. Ensure to include the complete path to the context file.
$ zap-cli -v context import /<path>/Webgoat.context [INFO] Imported context from /<path>/Webgoat.context
When you list the contexts, you will notice that the list seems to be empty, but it isn’t. It is probably a bug that the name is not correctly displayed.
$ zap-cli context list [INFO] Available contexts: 
Next step is to explore the website, you will use the exported URLs file you created in the previous paragraph. What you will do, is to open each URL which is present in this file with the
zap-cli open-url command. Create a bash script
open-urls.sh for this with the following contents.
#!/bin/bash input="webgoat-exported-urls.txt" while IFS= read -r line do zap-cli open-url "$line" done < "$input"
Give the bash script executable permissions.
$ chmod +x open-urls.sh
Execute the bash script, this can take some time dependent on how many URLs you have in the text file.
$ ./open-urls.sh [INFO] Accessing URL http://localhost:8080 [INFO] Accessing URL http://localhost:8080/WebGoat ...
Run the spider. Notice that the context is provided and also the user
mydeveloperplanet which will allow you to login to the application.
$ zap-cli -v spider -c Webgoat -u mydeveloperplanet "http://localhost:8080/WebGoat" [INFO] Running spider... [DEBUG] Spidering target http://localhost:8080/WebGoat... [DEBUG] Running spider in context 1 as user 2230 [DEBUG] Started spider with ID 0... [DEBUG] Spider progress %: 0 [DEBUG] Spider #0 completed
Do not run the AJAX spider. It is good to note that with the argument
--help, you can view the command arguments available for this command.
$ zap-cli ajax-spider --help Usage: zap-cli ajax-spider [OPTIONS] URL Run the AJAX Spider against a URL. Options: --help Show this message and exit.
As you can see, only the URL is available as a parameter. This is strange because in the Desktop version it is possible to provide a context and a user. It also seems that it negatively influences the end results (less alerts were reported). Therefore, do not run the AJAX spider.
Start the active scan. This will take some time (about 10 to 15 minutes).
$ zap-cli -v active-scan --recursive -c Webgoat -u mydeveloperplanet "http://localhost:8080/WebGoat" [INFO] Running an active scan... [DEBUG] Scanning target http://localhost:8080/WebGoat... [DEBUG] Scanning in context 1 as user 2230 [DEBUG] Started scan with ID 0... [DEBUG] Scan progress %: 0 [DEBUG] Scan progress %: 1 [DEBUG] Scan progress %: 3 ... [DEBUG] Scan progress %: 97 [DEBUG] Scan progress %: 99 [DEBUG] Scan #0 completed
Generate the report.
$ zap-cli -v report -o /<path>/report-zap-cli-first-scan.html -f html [DEBUG] Generating HTML report [INFO] Report saved to "/<path>/report-zap-cli-first-scan.html"
Save the session. This will allow you to open it in ZAP Desktop for example.
$ zap-cli -v session save /<path>/webgoat-20210410-active-scan.session [DEBUG] Saving the session to "/<path>/webgoat-20210410-active-scan.session"
And shutdown the ZAP daemon.
$ zap-cli -v shutdown [INFO] Shutting down ZAP daemon [DEBUG] Shutting down ZAP. [DEBUG] ZAP shutdown successfully.
ZAP CLI gives you the opportunity to execute an automated penetration test via the command line. This will make it also possible to include it into your CI/CD pipeline. Beware that it will give you only an indication about the security of your application and it does not replace a complete penetration test. Also note that the results are not identical to the results from the previous post. But when you execute the same steps as in this post starting from a clean session in ZAP Desktop, the results will be quite similar. Reason for the difference is probably the execution of the AJAX spider or the manually exploration of the website from within ZAP Desktop.