Rapid7 InsightVM Jenkins Integration for Docker Image Vulnerability Assessment

This document aims to show you how you can install a test environment to integrate InsightVM with Jenkins CI/CD for Docker image vulnerability assessment. The idea is to assess container’s vulnerability during software builds with InsightVM (Rapid7 also have container instance vulnerability assessment for about a year already)

The idea is outlined at https://blog.rapid7.com/2018/08/23/assess-containers-during-software-builds-with-insightvm/ that is to find out vulnerabilities of Docker images before you create instances on production.

Rapid7 InsightVM has recently announced integration with Jenkins CI/CD. If you go to https://help.rapid7.com/insightvm/en-us/release-notes/ you see on version 6.5.31 (Product Update 2018-08-22) you will see:

Mentioned help page in Release notes is at https://insightvm.help.rapid7.com/docs/containers-cicd-plugin . While we understand for most of us this may be a challenging task we wanted to create an easy step-by-step document that will help you this environment.

It is assumed that you have Centos7 64 bit installation (default installation with internet access) on your environment and we will install Docker, Jenkins, Github and integrate Jenkins with InsightVM

  1. First of all you need to change a few things on Centos. That is disable selinux and firewalld

Edit /etc/sysconfig/selinux file and change

SELINUX=enabled line to SELINUX=disabled and save.

Then issue following commands as root (# means that you need to be root on following commands)


# systemctl disable firewalld

# systemctl stop firewalld

And we need to reboot server

# reboot

 

  1. After having root access to our Centos7 we need to install docker. (Additional info can be found at https://docs.docker.com/install/linux/docker-ce/centos/ ) So here are the commands step-by-step required. As root:

# yum remove docker \

docker-client \

docker-client-latest \

docker-common \

docker-latest \

docker-latest-logrotate \

docker-logrotate \

docker-selinux \

docker-engine-selinux \

docker-engine

# yum install -y yum-utils \

device-mapper-persistent-data \

lvm2

# yum-config-manager \

–add-repo \

https://download.docker.com/linux/centos/docker-ce.repo

# yum install docker-ce

# systemctl start docker

# systemctl enable docker

 

Above commands simply uninstall if there are any OS default packages for docker, install docker community edition (which is the latest stable version) and starts/enables docker service so it survives reboots.

  1. After that we can as root user install a test docker image called hello-world. So for that following command will download docker image named hello-world and will run this docker image.

# docker run hello-world

  1. Now it is time to install Jenkins on your Centos7 Server.

     

    While you can use Jenkins Docker Image as we will be building Docker Images inside Jenkins I recommend (to avoid dokcer-in-docker complications) to install it directly on Centos7 not run Jenkins as a Docker Image. Issue following commands to install Jenkins: (https://www.vultr.com/docs/how-to-install-jenkins-on-centos-7)
     

Install and enable Epel Repository that brings several additional packages to your Centos7 machine
# yum install epel-release

Then install java ( Use 1.8)


# yum install java-1.8.0-openjdk.x86_64

Enable Jenkins Centos7 Package Repo and import its keys.
# wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo

# rpm –import https://pkg.jenkins.io/redhat-stable/jenkins.io.key

And finally install Jenkins (latest stable version)

# yum install jenkins

Verify you have installed Jenkins:

# rpm -qa |grep jenkins

jenkins-2.138.1-1.1.noarch

Enable and Start Jenkins:
# systemctl enable jenkins

jenkins.service is not a native service, redirecting to /sbin/chkconfig.

Executing /sbin/chkconfig jenkins on

# systemctl start jenkins

  1. Browse http://<REMOTE_CENTOS7_SERVER_IP>:8080 and you will see a screen first like:

Then like:

The password is in /var/lib/jenkins/secrets/initialAdminPassword file. Copy its contents and paste it here. There is no default password every installation has different password that is randomly generated.

  1. After you log in. You will see:
  2. Click on “Install Suggested Plugins” and you will see following screen

     

  3. Create a user:

     

  4. And Click on “Save and Finish” You will see Jenkins URL:
  5. After you login you will see following screen
  6. Before creating a new job we need to first create an API KEY on Insight Platform for InsightVM. More details creating API Key is at https://insightvm.help.rapid7.com/docs/containers-cicd-plugin#section-generate-the-rapid7-api-key This API key basically allow you to integrate InsightVM with Jenkins. We will be using this API Key on Jenkins
     
  7. Next step is to install a few Jenkins Plugins.
  1. Plain Credentials plugins comes with Default Jenkins Installation. To install Jenkins plugins on http://<Jenkins_IP>:8080 go to “Manage Jenkins” and after that “Manage Plugins”

  1. Choose “Docker API” Plugin which also contains Docker-Java plugin and click on “Download now and install after restart”

  1. Click on “Restart Jenkins when installation is complete and no jobs are running”

  1. After Jenkins restarts itself check the plugins are installed at “Manage Jenkins” “Manage Plugins” Installed . You wil also see Plain Credentials plugins are installed.

  1. Now as we have setup Jenkins we also need to add Rapid7 InsightVM Jenkins plugin. The latest version can be obtained directly from http://download2.rapid7.com/download/InsightVM/rapid7-insightvm-container-assessment-1.0.0.hpi but please be sure to check latest version at https://insightvm.help.rapid7.com/docs/containers-cicd-plugin (if there is any new version) Go to “Manage Jenkins” “Manage Plugins” Advanced Upload Plugin and upload it accordingly

And restart Jenkins

  1. After Jenkins restarts, return to the “Manage Jenkins” page. Click Configure System.

Scroll to the “Rapid7 InsightVM Container Assessment” section. Select location for your Platform and click on “Add”

  1. In the window that appears, complete these fields:
    a. In the “Domain” field, select "Global credentials (unrestricted)."
    b. In the “Kind” field, select "Secret text."
    c. In the "Scope" field, select “Global (Jenkins, nodes, items, all child items, etc).”
    d. In the “Secret” field, enter your API key.
    e. Leave the “ID” field blank.
    f. Enter a description for your reference.

  1. Then Select API Key you have just added and click on “Test Connection”

And “Save” it.

  1. Before going in to creating a new job on Jenkins there is one more step that we need to do. On Devops environment the images we build be located at Github or a Source Code Versioning environment. As Github is very popular we will use Github. Following steps are to create github account. Go to www.github.com
  2. Click on “Sign Up” page on the right hand side.

     

  3. Fill in the required information on Step 1

  1. Choose a plan. As this will be a test environment we will use public free repositories
  2. Step 3 is for user experience
  3. Before creating a repository we need to verify our email. Email is like below. Click on the link to verify your email

     

  4. After clicking email link above we can start a Repository. Click on “Start a project”

     

    and then fill in the details and click on “Create Repository”

    after that our repo created

  5. To access your repo install Github Desktop from https://desktop.github.com/ and install it. After installation Github Desktop Application will run. Login to your project with user and password you created.

     


    and then fill in the user and pass

  6. For our project details to be seen on Github Desktop on freshly created Repo click on “Set up on Desktop”

    and you will see following screen on Github Desktop. Click on “Clone” button
     
  7. The repo is cloned at (by default) C:\Users\<USERNAME>\Documents\GitHub\<RepoName> Open this directory with Windows Explorer. For our project we need to create a file named “Dockerfile” . We basically use Centos 7.0 Docker Image and will only run a simple command so we can create a new layer on Centos 7.0. Dockerfile should contain:

FROM centos:centos7.0.1406

MAINTAINER The Ikaruslab Project admin@ikaruslab.com

RUN mkdir /var/run/bogusdir

  1. On Github Desktop commit the Dockerfile to our github repo
    To push this change to github we need to push this:

     


So our repo link is right now is: https://github.com/githubikarus/ivmjenkinsint . This URL is like https://github.com/<username>/<reponame>

  1. Now it is time to create our first Jenkins Job. Login to Jenkins and Create Job using “create new job” link.

     
  2. We will be creating Freestyle project
  3. On new windows on Source Code Management click on “Git”

     

    and enter username and password for your project. In our case it will be https://github.com/githubikarus/ivmjenkinsint and enter credentials

  4. Fill in Username and Password (Kind will be Username and Password which is default Kind)

     


    and click on “Add” and at following window select your recently added credentials

  5. After this step we need a trigger so if source code changes (basically we only have Dockerfile in git repo) on Github we will trigger a rebuild and re-assessment of same image to achieve that we add a “Build Trigger” that is to check if anything changed on repo. Following image is showing how to do that. Following trigger uses “Poll SCM” and will check github repo for any changes every 3 minutes (if there is no change no re-build and re-assessment will be done)
  6. After this we add a build step. So we have detected a change in SCM and now it is time to Build a new image (after that Rapid7 can re-assess image). At “Build” step at this stage we compile docker image with “docker build –no-cache -t jenkins-ivm-int:${BUILD_ID} -f Dockerfile . ” command this actually creates a docker image named jenkins-ivm-int with the tag of ${BUILD_ID}. This BUILD_ID is Job Number of specified Job in Jenkins.

     

    if build fails with following error (You can see outputs at Console Output of specific build)

Encountered an exception.

Io.netty.channel.AbstractChannel$AnnotatedConnectException: connect(..) failed: Permission denied: /var/run/docker.sock

On Linux machine issue following command to add Jenkins user to Docker group and restart Jenkins service
 

# usermod -G docker jenkins
# systemctl restart Jenkins

 

  1. Next at “Build” Step select click Add build step. Select Assess Container Image with Rapid7 InsightVM. This will add a build step with a blank configuration.
  2. Configure the items under “Options” as desired.
  3. Click Add under the respective “Rules” section to configure the conditions that will trigger a build action. Two rule types are available:

     

    • “Threshold Rules” – Sets a numeric limit. Values that exceed this limit will trigger the build action.
    • “Name Rules” – Matches text using the “contains” operator. A match will trigger the build action.

       

      An example is like following “Imae Name or ID” is a compulsory field.We give a name where we stated at “Execute Shell” step that is jenkins-ivm-int:${BUILD_ID}

  4. After saving your Job. Run it manually to test using “Build Now” button on Jenkins job page. Of course for further steps you can change Dockerfile in your Repo then commit and push it so new commits are auto-detected and rebuild and reassessment of image done automatically.

     

    if build fails with following error (You can see outputs at Console Output of specific build)

Encountered an exception.

Io.netty.channel.AbstractChannel$AnnotatedConnectException: connect(..) failed: Permission denied: /var/run/docker.sock

On Linux machine issue following command to add Jenkins user to Docker group and restart Jenkins service
 

# usermod -G docker jenkins
# systemctl restart Jenkins

  1. After a successful build you will see on specified job id’s Console Output
  2. On Linux machine Jenkins installed you can see built images for assessment. As you can see Tag contains BUILD_ID that is Jenkins Job Number. You can use more variables for your images. The detailes can be found at https://wiki.jenkins.io/display/JENKINS/Building+a+software+project

     


     

  3. On InsightVM interface you will see “Builds” Section of Container tab like (You can access to this screen from Containers – Builds of InsightVM Web UI.)
  4. Appendix 1

On Build step we have used Freestyle Job. You can also use Pipeline scripts to build your and deploy (if you like) image. On production environments you will see more Pipeline script than Freestyle project.

Following picture shows Pipeline and Freestyle project.

Pipeline script can be like below. We first define Build step which includes checkout (which uses GitHub checkout) from a repo and then build from Dockerfile. After that we go into test state which uses assessContainerImage function on some certain criteria:

node {

// Define a variable to use across stages

def my_image

stage('Build') {

// Get Dockerfile and code from Git (or other source control)

checkout([$class: ‘GitSCM’, branches: [[name: ‘*/master’]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: ‘0579e8f0-e8a8-4d21-8004-d7a39cfe013b’, url: ‘https://github.com/githubikarus/ivmjenkinsint’]]])

// Build Docker image and set image reference

my_image = docker.build("insightvm-jenkins-integration-pipeline:${env.BUILD_ID}")

echo "Built image ${my_image.id}"

}

stage('Test') {

// Assess the image

assessContainerImage failOnPluginError: true,

imageId: "${my_image.id}",

thresholdRules: [

exploitableVulnerabilities(action: 'Mark Unstable', threshold: '1')

],

nameRules: [

vulnerablePackageName(action: 'Fail', contains: 'bash')

]

}

stage('Deploy') {

echo "Deploying image ${my_image.id} to somewhere…"

// Push image and deploy a container

}

}


You can download this pipeline script directly from our Github Repo https://raw.githubusercontent.com/githubikarus/ivmjenkinsint/master/Jenkinsfile

There are 3 sections following which needs to be change for your Jenkins environment. Checkout function uses Github. To generate checkout syntax you can use “Pipeline Syntax” link on which you can add your Github repo, credentials and generate Pipeline code snippet for checkout() function.

Fill in the fields required and on same screen below click on “Generate Pipeline Script” and you can copy following snippet to use in your Pipeline Script.

After that at second step, you need to modify the name of your docker image.
 

// Build Docker image and set image reference

my_image = docker.build("insightvm-jenkins-integration-pipeline:${env.BUILD_ID}")

And last one is the criteria on the Test stage.

imageId: "${my_image.id}",

thresholdRules: [

exploitableVulnerabilities(action: 'Mark Unstable', threshold: '1')

],

nameRules: [

vulnerablePackageName(action: 'Fail', contains: 'bash')

]

You can also create code snippet at Snippet Generator for assesContainerImage function like

And finally when the Build triggers you can see on related Job main screen