by
manage workflow continuous integration

Software automation can help make your IT staff’s life easier

Tuleap Trackers let you track virtually any kind of item, or “artifact.” However, sometimes you need to aggregate data from different sources and tools. The powerful Tuleap REST API gives you the interoperability you need to do this.

Let’s look at how this might work for a support team, for example. Support staff would use a “Support Request” tracker to monitor customer requests entered directly by customers into the tracker. Once the support ticket is resolved, a support team member would share a patch file. Before releasing the patch, however, it has to be checked to ensure that it will be usable by everyone. The support team leader can set up a tracker workflow to cross-check the patch to ensure that it does not cause any regression in the customer environment. One way to do this is to use Jenkins.

Automated Jenkins build trigger

When a support team member resolves an issue, he or she attaches a patch file and changes the issue status from “Under Resolution” to “Fixed.” When the issue status is changed, Tuleap can automatically trigger a build of a predefined Jenkins job. The job will retrieve artifact and tracker information in order to build the software and download the attached patch accordingly. The patch will be applied to the customer software tag and tested.

Example of workflow for support team

Once the test has been completed, the build will automatically update the issue by adding test results and changing the artifact status. If the test is successful the patch will be shared with the customer. If not, the patch will be rejected.

This level of workflow automation has the potential to substantially increase the efficiency of your processes and make your support team’s life much easier! Below, you will learn how to retrieve information in Jenkins from your Tuleap platform and see how to update a Tuleap issue from a Jenkins job.

Workflow automation made simple

To state it simply, the goal here is to automatically launch a Jenkins build when issue status is updated and, at the end of the build, have Jenkins update the issue and publish the build result.

The first part of this tutorial focuses on the Tuleap side: getting the tracker to send information to a Jenkins job and triggering a new build. The second part focuses on continuous integration tool: getting the Jenkins job to send information to the Tuleap tracker after the build.

Part 1 – Automatically trigger a Jenkins job build on Tuleap issue update

Step 1: Configuring the Tracker Workflow

In this example we will use a simple bug tracker to trigger the Jenkins build. So let’s start by creating a bug tracker using the bug tracker template.

open source Bug tracker

Next, administrate this new bug tracker and configure the workflow:

  • Go to the tracker Administration tab and click the “Manage Workflow” icon.

In this example, we want to start the build when the artifact status is updated to “resolved.”

Therefore, the transition will concern the bug tracker “status” field.

open source issue tracker

  • Go to the “Transitions” tab and select the “status” field and submit.
  • Next, configure a new transition and apply an action to this transition, the change in status from “verified” to “resolved.”

open source workflow tracker

  • Then click on the bullet corresponding to the transition and submit. You will see a “details” link to the right of your transition.
  • Click on the “details” link and configure the transition.

The last step is to configure the action that will be performed for this specific transition. Here, we want to launch a Jenkins job build.

  • Select “launch a continuous integration build” from the drop-down menu and submit.
  • Then copy and paste the URL of your Jenkins job, and add “/buildWithParameters” at the end (for example: https://example.com/jenkins/job/myjob/buildWithParameters).

workflow trigger jenkins job

Your Tuleap tracker is now configured! The next step is to configure the Jenkins job you want to connect to the tracker.

Step 2:  Configuring Jenkins Job

This part of the tutorial assumes that you already have a Jenkins server. Therefore, details on how to install and set up your CI server will not be covered here. So, we will start with the new Jenkins job created in Step 1 of this tutorial and go through the steps necessary to retrieve the artifact information.

  • Go to the “security” tab of the job configuration interface. Allow “anonymous” users to build the job.

 

jenkins-job-configuration

  • Then add a new parameter, “json,” that will be sent by Tuleap. This parameter is a string and will contain the whole JSON struct with all of the information relative to the artifact that triggered the build.
  • The parameter passed by Tuleap will be formatted like this :
    {"parameter":[{"name":"userId","value":"VVV"},{"name":"projectId","value":"WWW"},{"name":"artifactId","value":"XXX"},{"name":"trackerId","value":"YYY"},{"name":"triggerFieldValue","value":"ZZZ"}]}
    
  • Next, we will look at how to parse the data to get the most relevant information.

Step 3: Parsing parameters

You can parse and save the “json” parameter any way you choose using the information relative to the issue that Tuleap will provide for your application or scripts:

  • userId: Tuleap User ID that edited the issue (ex: Dorian Gray)
  • projectId: Tuleap Project ID hosting tracker x
  • trackerId: Tuleap Tracker ID (bug tracker in this example)
  • artifactId: Tuleap Artifact ID
  • triggerFieldValue: The field update that triggered the build

The information is easy to retrieve by parsing with a “jq” command. However, if you do not have JQ installed on your server, you can use “sed” as seen below:

jjenkins-parsing-parameters

Here, we created a build step (Script Shell) and 5 new variables to store the issue information.

USERID=$(echo "$json" | sed 's/.*{\"name\":\"userId\",\"value\":\"//' | sed 's/\"}.*//')
PROJECTID=$(echo "$json" | sed 's/.*{\"name\":\"projectId\",\"value\":\"//' | sed 's/\"}.*//')
TRACKERID=$(echo "$json" | sed 's/.*{\"name\":\"trackerId\",\"value\":\"//' | sed 's/\"}.*//')
ARTEFACTID=$(echo "$json" | sed 's/.*{\"name\":\"artifactId\",\"value\":\"//' | sed 's/\"}.*//')
TRIGGERFIELDVALUE=$(echo "$json" | sed 's/.*{\"name\":\"triggerFieldValue\",\"value\":\"//' | sed 's/\"}.*//')

With JQ you should be able to retrieve the issue information like this:

USERID=$(echo "${json}" | jq  '.parameter[0].value')
PROJECTID=$(echo "${json}" | jq  '.parameter[1].value')
ARTIFACTID=$(echo "${json}" | jq  '.parameter[2].value')
TRACKERID=$(echo "${json}" | jq  '.parameter[3].value')
TRIGGERFIELDVALUE=$(echo "${json}" | jq  '.parameter[4].value')

The job has been configured and the issue information has been retrieved. Now it is time to test!

Step 4: Testing

To complete the test, submit a new bug to the tracker and change the status.

  • Complete the required fields
  • Set the status field to “verified”
  • Submit
  • Edit your new issue
  • Change the status field to “resolved”
  • Submit again

You will then see a message in blue at the top of the page. This means that your workflow configuration has triggered a new build on Jenkins.

If there are any errors in your configuration, you will see an error message instead (see below).

success jenkins job build

The job has been launched. Now you can go to the Jenkins side to check that your script has retrieved the Tuleap values.

In this example the output returns:

Started by user anonymous
[EnvInject] - Loading node environment variables.
Building on master in workspace /var/lib/jenkins/jobs/test_job_temporary/workspace
[workspace] $ /bin/sh -xe /tmp/hudson4233075936227157871.sh
+ USERID=159
+ PROJECTID=111
+ TRACKERID=61
+ ARTEFACTID=431
+ TRIGGERFIELDVALUE=Resolved

In this case, the information has been transmitted successfully from Tuleap to Jenkins. Now it is time to configure Jenkins to send information to Tuleap.

Part 2 – Configure Jenkins to send information to Tuleap

Step 1: Create a bot user

  • Go to the Tuleap site administration interface to create a new restricted user.
  • Log on your Tuleap platform as site admin and click the “new user” button to create the Jenkins user.

Complete the form. Setting the user as “restricted” is recommended. A “restricted” user won’t be able to access the project if the user is not a member, and you will be able to fine-tune the user’s permissions for your project by adding the user to a dedicated group.

Step 2: Add Jenkins user to your project

In this step, you will add the new user to your project.

add jenkins user group

By default, this user will have access to the project as a normal user. If you want to fine-tune the permissions for this special user you can add the user to a new group and add restrictions.

Step 3: Configure Jenkins

In your job configuration, add the username and password.
Then select “inject passwords to the build as environment variables.” Use the password you set for the bot user you created on the Tuleap platform.

Set the username directly in the script. USERNAME=”jenkins_bot”

Another way to do this is to use “credentials” feature of Jenkins. It is much more cleaner to pass the password and username this way but you need admin privileges to add credentials on your CI platform.

Next, you will see how to update an artifact during a build execution in a Jenkins Job.

Step 4: Update the issue and more

As described above, you now have all of the information you need to update your Tuleap Tracker. You were able to retrieve information about the issue (artifactId), the tracker (trackerId), and the related Tuleap project (projectId). You also know who (userId) changed the issue status that triggered the build and why it was triggered (triggerFieldValue).

To update the artifact summary, you must:

  • Get a token for the jenkins_bot user
  • Update the artifact using the token
  • Delete the token

You can use cURL to call the REST API if you know it is available on the Jenkins platform. If you need to parse JSON structs returned by cURL, there are applications, like JQ for example, that facilitate this task.

Here is the complete script to complete this task:

USERNAME="jenkins_bot"
ARTEFACTID=$(echo "$json" | sed 's/.*{\"name\":\"artifactId\",\"value\":\"//' | sed 's/\"}.*//')  

TULEAP_ADDR=https://TULEAPHOSTNAME
# Get the token
getToken=`curl -XPOST --header 'Content-type: application/json' -d '{"username":"'$USERNAME'", "password":"'$PASSWORD'"}' $TULEAP_ADDR/api/tokens`

# Save token and userId
USER_ID=$(echo "$getToken" | sed 's/.*user_id\"://' | sed 's/,.*//')
TOKEN=$(echo "$getToken" | sed 's/.*token\":"//' | sed 's/",.*//')

# Update Artifact (field_id is corresponding to the ID of the « summary » field in my tracker on my plateform)
curl -XPUT --header 'Content-type: application/json' --header "X-Auth-Token: $TOKEN" --header "X-Auth-UserId: $USER_ID" $TULEAP_ADDR/api/artifacts/$ARTEFACTID \
-d '{"values":[{"field_id": 1479, "type": "string", "label": "Summary", "value": "Test Bug (Tested on Jenkins)"}]}’

# Delete the token
curl -XDELETE  --header 'Content-type: application/json' --header "X-Auth-Token: $TOKEN" --header "X-Auth-UserId: $USER_ID" $TULEAP_ADDR/api/tokens

Now that the Jenkins job has been updated, it is time to test the whole use case.

Step 5: Testing

  • Select your issue.
  • Change the status value from “verified” to “resolved.”
  • Check the results on both tools.

The image below shows the follow-up information:

In the example above, the user Dorian Gray changed the status; a few seconds later, Jenkins Bot updated the summary.

In this tutorial you learned how to retrieve issue information from a Jenkins job and how to update the artifact data at the end of your job. You can use a shell script to retrieve Tuleap information. Installing JQ on your Jenkins platform is helpful in parsing what is returned by cURL. Python scripts using JSON and PycURL libraries can also make artifact updates easier.

Taking it further: continuous integration and interoperability between Tuleap and Jenkins

This use case is just one of many possibilities. You can leverage continuous integration and interoperability between Tuleap and Jenkins to automate a number of tasks in your workflow:

  • Update issue when build status is updated.
  • Retrieve user information to send an email notification at the end of the build.
  • Upload a file to an issue.
  • Create a new artifact in another tracker.

Related Documentation


About

Manon Midy

Manon has been working in the software engineering world from 2008. She enjoys the challenge to create economic value for a company where innovation and open source are the DNA. She is convinced it’s possible to provide professional services embracing FLOSS values (open-mind, transparency, co-elaboration) with business objectives. She believes the real strength of Enalean comes from the valuable men and women in their teams, as well as the powerful Tuleap techno.