Originally from a two-part blog series on Harness.
Part One: https://harness.io/blog/featured/vulnerability-scanning-ci-cd-pipeline/
Part Two: Automated Vulnerability Scanning in your CI/CD Pipeline - Part Two
GitHub: GitHub - ravilach/sonatype-harness: Sonatype and Harness Integratitons
Sonatype Nexus IQ Setup
In this example, we will be running Sonatype Nexus IQ on an Amazon EC2 CentOS box. Going through Sonatype’s prerequisites, CentOS is fine running Open JDK 11 and Docker. Running these commands will install Open JDK 11, Docker Engine, download Nexus IQ, and install JQ.
#Install JAVA
sudo yum install java-11-openjdk
#Install Docker
#https://docs.docker.com/engine/install/centos/
sudo yum install -y yum-utils
sudo yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo
sudo yum install docker-ce docker-ce-cli containerd.io
sudo systemctl start docker
#Install WGET
sudo yum install wget
#Install JQ
sudo yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
sudo yum install jq
#Install Nexus IQ
mkdir nexus-iq-server
cd nexus-iq-server
wget -O nexus-iq-server.tar.gz https://download.sonatype.com/clm/server/latest.tar.gz
tar xfvz nexus-iq-server*.tar.gz
Once the tar.gz is expanded, can run the demo.sh to quickly boot Nexus IQ server.
Nexus IQ by default serves traffic over port 8070. To access the web UI for the first time, you can head to your public_ip:8070. The default user/password is admin/admin123.
Once logged in, Nexus IQ is driven by Organizations. You can navigate to Orgs & Policies to see that structure. By default, there is a Root and Sandbox Organization. You can rename the Sandbox Organization and underlying Application to something more relatable by selecting Actions → Edit App Name.
One additional step would be to rename the internal Application ID. Similar to renaming the Application Name, Actions → Change App ID.
Once you hit Change, the Application ID will be something more logical.
With the basic Nexus IQ configuration out of the way, the next step is to get started on the Harness Platform.
Harness Platform Setup
If this is your first time leveraging the Harness Platform, the first step would be to install a Harness Delegate. The first delegate we will install is to easily interact and deploy with the Kubernetes cluster.
Setup → Harness Delegates → Install Delegate with type of Kubernetes YAML.
Download the tar.gz and expand the contents.
Inside the expanded folder, there will be a readme file with the kubectl command needed to install which is kubectl apply -f harness-delegate.yaml. Run the kubectl apply command.
In a few moments, the Kubernetes Delegate will be available via the Harness UI.
With the Kubernetes Delegate up and running, we can also install a Harness SSH Delegate so we can run needed Sonatype Nexus CLI commands.
Back on your CentOS instance, make a new folder for the Harness Delegate and CD into the folder.
#Folder for SSH Delegate
mkdir harness-delegate
cd harness-delegate.
Back on the Harness Platform we can get the SSH download link.
Setup → Harness Delegates → Install Delegate with Download Type Shell Script.
Click Copy Download Link and paste into the terminal. The cURL command will download a tar.gz of the Delegate. Expand the tar.gz and run the start.sh command.
#Install SSH Delegate
tar xfvz harness*.tar.gz
cd harness-delegate
./start.sh
In a few moments, you will know have a pair of Harness Delegates in the UI. One for Shell and one for Kubernetes.
With both Delegates up and running we have a wide paintbrush on how to architect the scan. One approach is using Nexus IQ’s CLI [command-line-interface] for a container scan and we can directly interact with that.
Sonatype Nexus IQ CLI
There are several ways to run a container scan with Nexus IQ. One of the easiest ways is to leverage the Nexus IQ CLI. Depending on the distribution you get from Sonatype, the CLI could ship with the Nexus IQ download.
An easy way to interact with your CentOS instance with Harness and ensuring software is in place is with a Harness Delegate Profile.
Setup → Harness Delegates → Delegate Profiles → Add Delegate Profiles
Can create a new Profile to Install Nexus IQ CLI.
#Install Nexus CLI
mkdir ~/nexus-cli
cd ~/nexus-cli
wget -q -O nexus-iq-cli.jar https://download.sonatype.com/clm/scanner/latest.jar
Click Submit. Then can run the Delegate Profile by heading back to the Delegate UI and selecting the “Install Nexus IQ CLI” as the Profile.
In a few moments, the Delegate will install the Nexus IQ CLI. You can validate on the CentOS instance also.
With the Nexus CLI available, the next step will take us back to Harness.
Harness Pipeline Setup
To start creating a Harness Pipeline, Harness leverages an abstraction model called the CD Abstraction Model. The first piece would be to wire your Kubernetes cluster to Harness.
Setup → Cloud Providers + Add Cloud Provider. Type is Kubernetes Cluster.
Since you have a running Harness Delegate inside your Kubernetes cluster, the easiest way to wire the cluster is to just inherit the details from the Delegate. Select the Delegate Name which represents your Kubernetes Delegate, click Test, Next, then Submit.
Once the Cloud Provider is wired, The heart of your Pipeline is the Harness Application.
Setup → Applications + Add Application
Once you hit Submit, time to start filling out the abstraction pieces.
The first abstraction piece would be letting Harness know where to deploy with a Harness Environment.
Setup → Webgoat → Environments + Add Environment
Once inside the Environment, can add a target piece of infrastructure with an Infrastructure Definition.
Setup → Webgoat → Environments → WebGoat Farm + Add Infrastructure Defination.
Cloud Provider Type: Kubernetes Cluster
Deployment Type: Kubernetes
Cloud Provider: Webgoat K8s Cluster
Once you hit Submit you will be wired up to deploy to the Kubernetes cluster.
The next item would be to define a Harness Service or what you will be deploying.
Setup → Webgoat → Services + Add Service with Deployment Type Kubernetes.
Once you hit Submit, can add the Webgoat Docker Image as an Artifact Source from a Docker Registry.
Harness ships with a pre-wire to public Docker Hub [Source Server Harness Docker Hub].The Docker Pull command for Webgoat is “webgoat/webgoat-8.0” and that will be used for the Docker Image Name.
Once you hit Submit, all of the scaffolding for a Kubernetes deployment is generated for you by the Harness Platform.
With the artifact out of the way, there are a few ways to programmatically call Nexus IQ from Harness.
Calling Nexus IQ from Harness
The approach that the example will take have the Harness SSH Delegate invoke the Nexus IQ CLI on the CentOS instance.
To get started, the first step is to label the Harness Delegate with a Custom Selector.
Setup → Harness Delegates → Your SSH Delegate + Custom Selector. Can name the Selector “NexusIQScan”.
Once you hit submit, the Selector will be available.
The next step would be to prepare the CD Abstraction Model with another Cloud Provider representing where Nexus IQ is installed. The rationale behind this would allow the Nexus IQ steps to be part of a managed Workflow in Harness.
Setup → Cloud Providers → + Add Cloud Provider. The type is Physical Data Center. Can name the Provider “Nexus IQ DC”.
Once you hit Submit, the Nexus IQ DC will appear.
Back in the Webgoat Harness Application, you can now create a Harness Environment that maps back to the CentOS instance.
Setup → Webgoat → Environments + Add Environment. Can name the Environment “Nexus IQ Server”.
Next, you will need to create the Infrastructure Definition inside the Environment. Since we already have a Harness Delegate running in the CentOS instance and created a Custom Selector to select that particular Delegate, the Hostname can be “localhost”.
Name: Nexus IQ CentOS
HostName(s): localhost
Connection Attribute: Any
Once you hit Submit, the Infrastructure Definition will be out of the way.
The Harness Platform can orchestrate and manage multiple types of deployments as Harness Workflows. These are driven off of the Harness Service. Earlier you wired the Webgoat Docker Image as a Harness Service. We can create a generic Shell Service to call the CLI.
Setup → Webgoat → Services + Add Service.
Name: Nexus IQ Shell
Deployment Type: Secure Shell (SSH)
Artifact Type: Other
Once you hit Submit, you will see the scaffolding there for the SSH Deployment.
To have Harness manage the commands needed to execute a Nexus IQ CLI scan, can create a Harness Workflow representing that.
Setup → Webgoat → Workflows + Add Workflow
Name: Scan Artifact
Workflow Type: Rolling Deployment
Environment: Nexus IQ Server
Service: Nexus IQ Shell
Infrastructure Definition: Nexus IQ CentOS
Once you hit Submit, a templated Workflow will be generated. The Phases, names, and order can be changed, condensed, etc.
Can slot in the CLI Call under Step 3 “Deploy Service” by clicking + Add Step.
In the Add Step UI, can search for “Shell” and select Shell Script.
Click Next and can fill out a basic shell script around what a Nexus IQ scan requires. Make sure to select the Delegate Selector [NexusIQScan] that was assigned to the Shell [CentOS] Delegate.
Name: Shell Script
Type: BASH
Script:
#Download and Scan
mkdir -p ~/nexus-scans
cd ~/nexus-scans
sudo docker pull webgoat/webgoat-8.0
sudo docker save webgoat/webgoat-8.0:latest > webgoat-8.0.tar
java -jar ~/nexus-cli/nexus-iq-cli.jar -i webgoat-application -s http:/<public_ip>:8070 -a <user>:<password> -t build webgoat-8.0.tar
Timeout: 10m
Delegate Selector: NexusIQScan
Once you click Submit, your Shell Script will be under Deploy Service.
At this point, you can execute the call and see how a basic call from Harness looks. On the top right, click Deploy.
Hit Submit and watch your first scan be executed.
Can navigate back to Nexus IQ to view your report!
http://:8070 → Reports → Reporting → Webgoat → View Report
Operationalizing Vulnerability Scans with Harness
From the scan standpoint, will be masking the Nexus Credentials in the Harness Secrets Manager then prompting the user for a few details before submitting and interpreting the scan.
To add a pair of secrets for the Nexus IQ user and password, can leverage the Harness Secrets Manager.
Security → Secrets Management → Secrets → Encrypted Text + Add Encrypted Text
Name: nexusiq_user
Value: admin
Once you click Submit, can add another for the password.
Name: nexusiq_password
Value: admin123
With the secrets added, you can leverage Workflow Variables to prompt the user for certain parameters.
Setup → Webgoat → Workflows → Scan Artifact. On the right hand side, click on “> Workflow Variables”
Click on the Pencil next to Workflow Variables to + Add.
For example, we can prompt the user for the Nexus IQ Application ID so that the scan gets related to the correct rule-set, etc. For this example we are only adding one variable, the art of the possible to parameterize all the inputs.
Variable Name: nexusiqappid
Type: Text
Default Value: webgoat-application
Once you click Save, you will see the Workflow Variable populated.
With Harness, you can reference Secrets with the syntax “${secrets.getVaule(“secret_name”)}” and access Workflow Variables with “${workflow.variables.name}”.
Can update the Shell Script with the following updates.
Setup → Webgoat → Workflows → Scan Artifact → Shell Script
#Download and Scan Template
mkdir -p ~/nexus-scans
cd ~/nexus-scans
sudo docker pull webgoat/webgoat-8.0
sudo docker save webgoat/webgoat-8.0:latest > webgoat-8.0.tar
java -jar ~/nexus-cli/nexus-iq-cli.jar -i ${workflow.variables.nexusiqappid} -s http://<public_ip>:8070 -a ${secrets.getValue("nexusiq_user")}:${secrets.getValue("nexusiq_password")} -t build webgoat-8.0.tar
Click Submit and your changes have been saved. You can run through another Deploy to validate the changes worked. In the upper right of the Workflow, click Deploy.
Hit Submit and you will be running a scan in a more operationalized fashion.
Once we have the scan data, the next step is to interpret the results and then deploy. We can generate another pair of Harness Workflows [which we will later stitch together in a Harness Pipeline] to gather the results.
Create a new Workflow to interpret the scan results.
Setup → Webgoat → Workflows + Add Workflow.
Name: Interpret Scan
Workflow Type: Rolling Deployment
Environment: Nexus IQ Server
Service: Nexus IQ Shell
Infrastructure Definition: Nexus IQ CentOS
Once you hit Submit, create a Workflow to deploy Webgoat. You will come back and fill in the details for the scan interpretation soon.
Setup → Webgoat → Workflows + Add Workflow
Name: Deploy Webgoat
Workflow Type: Rolling Deployment
Environment: Webgoat Farm
Service: Webgoat
Infrastructure Definition: Goat K8s Cluster
Once you hit Submit and return to the Workflow list, you will see three Harness Workflows.
The next piece is to build in some sort of interpretation logic to parse through the scan results.
The next step will be to add a shell script to the Interpret Scan Workflow.
Programmatically Interpreting Scan Results
Part of an automated DevSecOps practice is to influence the deployment if there are violating vulnerabilities found in a scan. At this time there is not a native integration between Nexus IQ and Harness, though can get access to and parse the scan results with a few cURL commands.
The commands that are needed are getting the internal Nexus IQ App ID, the Report ID, and to count the violations in the report. If there are severe violations, stop the deployment.
To start building this in, head back to Interpret Scan and add a Workflow Variable then a shell script in the Deploy Step.
Setup → Webgoat → Workflows → Interpret Scan
The first item to add is a Workflow Variable to prompt what would be a severe enough policy violation in Nexus IQ to stop the deployment.
Click on the pencil icon under Workflow Variables.
Variable Name: minfailpoilcyviolation
Type: Number
Allowed Values: 5,6,7,8,9
Default Value: 7
Required: True
Once you click Save, you can add the shell script under the Deploy Service in the Interpret Scan Workflow.
In the Add Step UI, search for shell and add Shell Script. Click next.
Assuming your user/password are still in the Harness Secrets Manager, in the Configure Shell Script prompt, enter the following:
Name: Shell Script
Type: Bash
Timeout: 5m
Delegate Selector: NexusIQScan
Script:
#Interpret Scan Results - Template
echo "Interpreting Scan Results"
#Get AppID
export AppID=$(curl -u ${secrets.getValue("nexusiq_user")}:${secrets.getValue("nexusiq_password")} -X GET 'http:/<public_ip>:8070/api/v2/applications?publicId=webgoat-application' | jq '.applications[0] .id' | tr -d '"')
echo "AppID: " $AppID
#Get Report IDs
export ReportID=$(curl -u ${secrets.getValue("nexusiq_user")}:${secrets.getValue("nexusiq_password")} -X GET 'http://<public_ip>:8070/api/v2/reports/applications/'$AppID'' | jq '.[0] .reportDataUrl' | cut -d'/' -f 6)
echo "ReportID: " $ReportID
#Get Violation Count
export TotalViolations=$(curl -u ${secrets.getValue("nexusiq_user")}:${secrets.getValue("nexusiq_password")} -X GET "http://<public_ip>:8070/api/v2/applications/webgoat-application/reports/"$ReportID"/policy" | jq '.components[] .violations[] .policyThreatLevel' | grep -c '[${workflow.variables.minfailpoilcyviolation}-9]')
echo "Total Sev Violations: " $TotalViolations
#Exit if Violations Match
if [[ $TotalViolations -gt 0 ]] ; then
echo "Violations Preventing Deployment"
exit 1
else
echo "Moving Forward with Deployment"
exit 0
fi
Then hit Submit and your Shell and Workflow Variables should be present. Feel free to make any modifications to the script, was based on the Nexus IQ REST API.
The next step would be stitching all of the Workflows into a Harness Pipeline.
Automated CI/CD DevSecOps Pipeline - With Harness
So far our three separate Workflows are run independently of each other. By leveraging a Harness Pipeline, can stitch together the three workflows together.
Setup → Webgoat → Pipelines + Add Pipeline. Can name “DevSecOps”.
Once you hit Submit, can add Pipeline Stages to correspond to each Workflow.
Once you click + Add, the first step would be to scan the artifact.
Execute Workflow: Scan Artifact
Once you hit Submit, can go through and add the interpret and deploy steps in a similar fashion.
Clicking through the two additional Stages. First, add Interpret.
Then add Deploy.
Once you hit Submit with the Deploy Stage, your Pipeline will have three Stages.
You are now ready to run your Pipeline and see the fruits of your work.
Running Your First Automated DevSecOps Pipeline
There are a few ways to run a Pipeline in Harness. Can navigate to the left-hand navigation and select Continuous Deployment.
Continuous Deployment → Deployments → Start New Deployment
Pick a tag [latest] of Webgoat and hit Submit.
Once you hit Submit, you can watch your first automated DevSecOps Pipeline run. As expected, the Pipeline will fail because Webgoat has lots of violations.
Can take a look at the Nexus IQ report also and see that the threats are there.
http://<public_ip>:8070 → Reports → Webgoat → View Report
Harness, Your Partner in DevSecOps
Harness is here to partner with you no matter where you are in your DevSecOps journey. Application security testing methodologies are evolving with more items shifting left towards the developer and self-service. Having DevSecOps steps in your pipelines is a prudent move. Thanks for taking a look at the this post and make sure to sign up for a Sonatype Nexus IQ trial and a Harness account, today!
Cheers,
-Ravi