jenkins.plone.org Set Up Howto

This document describes how to set up the entire Jenkins infrastructure for jenkins.plone.org. Those are the main steps:

  • Set up Jenkins server (jenkins.plone.org, with Ansible)
  • Set up Jenkins nodes (node[1-x].jenkins.plone.org, with Ansible)
  • Set up the Jenkins jobs on the Jenkins server (with Jenkins Job Builder)

Prerequisites

Checkout this repository:

$ git clone git@github.com:plone/jenkins.plone.org.git
$ cd jenkins.plone.org

Create and activate a virtualenv:

$ virtualenv -p python2.7 .
$ source ./bin/activate

Install all the tools needed (ansible, ansible roles and jenkins-job-builder):

$ pip install -r requirements.txt
$ ansible-galaxy install -r ansible/roles.yml
$ git submodule update --init

Note

For the roles that are downloaded from checkouts, plone.jenkins_server and plone.jenkins_node, you will need to remove them and clone them manually if you want to make changes on them.

$ cd ansible/roles
$ rm -rf plone.jenkins_server
$ rm -rf plone.jenkins_node
$ git clone git@github.com:plone/plone.jenkins_server
$ git clone git@github.com:plone/plone.jenkins_node

Check ansible/inventory.txt and make sure that you can connect to the machines listed there.

Copy your public ssh key to all servers:

$ ssh-copy-id -i ~/.ssh/<SSH-KEY>.pub root@<SERVER_IP>

Set Up Jenkins Server

$ ./update_master.sh

Set Up Jenkins Nodes

$ ./update_nodes.sh

Set Up Jenkins Jobs

Do the steps described above to clone, activate virtualenv and fetch submodules.

Put jenkins-job-builder in development mode:

$ cd src/jenkins-job-builder
$ pip install -r requirements.txt
$ python setup.py develop

Test the jobs are properly setup:

$ jenkins-jobs --conf jobs/config.ini.in test jobs/jobs.yml -o output

Note

A folder named output should contain one file per each jenkins job configured on jobs.yml

Create your own jobs/config.ini by copying it from jobs/config.ini.in:

$ cp jobs/config.ini.in jobs/config.ini

Add your own credentials to jobs/config.ini. You can find them when you log into Jenkins and copy your API token (e.g. http://jenkins.plone.org/user/tisto/configure).

Create your own ansible/secrets.yml by copying it from ansible/secrets.yml.in:

$ cp ansible/secrets.yml.in ansible/secrets.yml

Add github API secrets that are needed for the github login functionality on jenkins.plone.org. You can find those settings on plone organization in github: https://github.com/organizations/plone/settings/applications

Look for the Plone Jenkins CI application name.

For the github_api_key you need a personal token (from https://github.com/jenkins-plone-org github user).

Now finally install the jobs on the server:

$ ./update_jobs.sh

Manual Configuration

There are currently a few steps that we need to carry out manually. We will automate them later.

  1. Github post-commit hook for buildout.coredev:
  1. Manage Jenkins -> Configure System:
  • E-mail Notification:
  1. Manage Jenkins -> Manage Credentials -> Add Credentials: SSH Username with private key:
  • Scope: System
  • Username: jenkins
  • Description: jenkins.plone.org private ssh key
  • Private Key: From a file on Jenkins master: File: /var/lib/jenkins/jenkins.plone.org

=> Upload jenkins.plone.org private ssh key manually to /var/lib/jenkins => chown jenkins:jenkins jenkins.plone.org