Software automation can help make your IT staff’s life easier
Tuleap Trackers let you track virtually any kind of item, or “artifact.” However, sometimes you need to aggregate data from different sources and tools. The powerful Tuleap REST API gives you the interoperability you need to do this.
Let’s look at how this might work for a support team, for example. Support staff would use a “Support Request” tracker to monitor customer requests entered directly by customers into the tracker. Once the support ticket is resolved, a support team member would share a patch file. Before releasing the patch, however, it has to be checked to ensure that it will be usable by everyone. The support team leader can set up a tracker workflow to cross-check the patch to ensure that it does not cause any regression in the customer environment. One way to do this is to use Jenkins.
Automated Jenkins build trigger
When a support team member resolves an issue, he or she attaches a patch file and changes the issue status from “Under Resolution” to “Fixed.” When the issue status is changed, Tuleap can automatically trigger a build of a predefined Jenkins job. The job will retrieve artifact and tracker information in order to build the software and download the attached patch accordingly. The patch will be applied to the customer software tag and tested.
Once the test has been completed, the build will automatically update the issue by adding test results and changing the artifact status. If the test is successful the patch will be shared with the customer. If not, the patch will be rejected.
This level of workflow automation has the potential to substantially increase the efficiency of your processes and make your support team’s life much easier! Below, you will learn how to retrieve information in Jenkins from your Tuleap platform and see how to update a Tuleap issue from a Jenkins job.
Workflow automation made simple
To state it simply, the goal here is to automatically launch a Jenkins build when issue status is updated and, at the end of the build, have Jenkins update the issue and publish the build result.
The first part of this tutorial focuses on the Tuleap side: getting the tracker to send information to a Jenkins job and triggering a new build. The second part focuses on continuous integration tool: getting the Jenkins job to send information to the Tuleap tracker after the build.
Part 1 – Automatically trigger a Jenkins job build on Tuleap issue update
Step 1: Configuring the Tracker Workflow
In this example we will use a simple bug tracker to trigger the Jenkins build. So let’s start by creating a bug tracker using the bug tracker template.
Next, administrate this new bug tracker and configure the workflow:
- Go to the tracker Administration tab and click the “Manage Workflow” icon.
In this example, we want to start the build when the artifact status is updated to “resolved.”
Therefore, the transition will concern the bug tracker “status” field.
- Go to the “Transitions” tab and select the “status” field and submit.
- Next, configure a new transition and apply an action to this transition, the change in status from “verified” to “resolved.”
- Then click on the bullet corresponding to the transition and submit. You will see a “details” link to the right of your transition.
- Click on the “details” link and configure the transition.
The last step is to configure the action that will be performed for this specific transition. Here, we want to launch a Jenkins job build.
- Select “launch a continuous integration build” from the drop-down menu and submit.
- Then copy and paste the URL of your Jenkins job, and add “/buildWithParameters” at the end (for example: https://example.com/jenkins/job/myjob/buildWithParameters).
Your Tuleap tracker is now configured! The next step is to configure the Jenkins job you want to connect to the tracker.
Step 2: Configuring Jenkins Job
This part of the tutorial assumes that you already have a Jenkins server. Therefore, details on how to install and set up your CI server will not be covered here. So, we will start with the new Jenkins job created in Step 1 of this tutorial and go through the steps necessary to retrieve the artifact information.
- Go to the “security” tab of the job configuration interface. Allow “anonymous” users to build the job.
- Then add a new parameter, “json,” that will be sent by Tuleap. This parameter is a string and will contain the whole JSON struct with all of the information relative to the artifact that triggered the build.
- The parameter passed by Tuleap will be formatted like this :
{"parameter":[{"name":"userId","value":"VVV"},{"name":"projectId","value":"WWW"},{"name":"artifactId","value":"XXX"},{"name":"trackerId","value":"YYY"},{"name":"triggerFieldValue","value":"ZZZ"}]} - Next, we will look at how to parse the data to get the most relevant information.
Step 3: Parsing parameters
You can parse and save the “json” parameter any way you choose using the information relative to the issue that Tuleap will provide for your application or scripts:
- userId: Tuleap User ID that edited the issue (ex: Dorian Gray)
- projectId: Tuleap Project ID hosting tracker x
- trackerId: Tuleap Tracker ID (bug tracker in this example)
- artifactId: Tuleap Artifact ID
- triggerFieldValue: The field update that triggered the build
The information is easy to retrieve by parsing with a “jq” command. However, if you do not have JQ installed on your server, you can use “sed” as seen below:
Here, we created a build step (Script Shell) and 5 new variables to store the issue information.
USERID=$(echo "$json" | sed 's/.*{\"name\":\"userId\",\"value\":\"//' | sed 's/\"}.*//')
PROJECTID=$(echo "$json" | sed 's/.*{\"name\":\"projectId\",\"value\":\"//' | sed 's/\"}.*//')
TRACKERID=$(echo "$json" | sed 's/.*{\"name\":\"trackerId\",\"value\":\"//' | sed 's/\"}.*//')
ARTEFACTID=$(echo "$json" | sed 's/.*{\"name\":\"artifactId\",\"value\":\"//' | sed 's/\"}.*//')
TRIGGERFIELDVALUE=$(echo "$json" | sed 's/.*{\"name\":\"triggerFieldValue\",\"value\":\"//' | sed 's/\"}.*//')
With JQ you should be able to retrieve the issue information like this:
USERID=$(echo "${json}" | jq '.parameter[0].value')
PROJECTID=$(echo "${json}" | jq '.parameter[1].value')
ARTIFACTID=$(echo "${json}" | jq '.parameter[2].value')
TRACKERID=$(echo "${json}" | jq '.parameter[3].value')
TRIGGERFIELDVALUE=$(echo "${json}" | jq '.parameter[4].value')
The job has been configured and the issue information has been retrieved. Now it is time to test!
Step 4: Testing
To complete the test, submit a new bug to the tracker and change the status.
- Complete the required fields
- Set the status field to “verified”
- Submit
- Edit your new issue
- Change the status field to “resolved”
- Submit again
You will then see a message in blue at the top of the page. This means that your workflow configuration has triggered a new build on Jenkins.
If there are any errors in your configuration, you will see an error message instead (see below).
The job has been launched. Now you can go to the Jenkins side to check that your script has retrieved the Tuleap values.
In this example the output returns:
Started by user anonymous [EnvInject] - Loading node environment variables. Building on master in workspace /var/lib/jenkins/jobs/test_job_temporary/workspace [workspace] $ /bin/sh -xe /tmp/hudson4233075936227157871.sh + USERID=159 + PROJECTID=111 + TRACKERID=61 + ARTEFACTID=431 + TRIGGERFIELDVALUE=Resolved
In this case, the information has been transmitted successfully from Tuleap to Jenkins. Now it is time to configure Jenkins to send information to Tuleap.
Part 2 – Configure Jenkins to send information to Tuleap
Step 1: Create a bot user
- Go to the Tuleap site administration interface to create a new restricted user.
- Log on your Tuleap platform as site admin and click the “new user” button to create the Jenkins user.
Complete the form. Setting the user as “restricted” is recommended. A “restricted” user won’t be able to access the project if the user is not a member, and you will be able to fine-tune the user’s permissions for your project by adding the user to a dedicated group.
Step 2: Add Jenkins user to your project
In this step, you will add the new user to your project.
By default, this user will have access to the project as a normal user. If you want to fine-tune the permissions for this special user you can add the user to a new group and add restrictions.
Step 3: Configure Jenkins
In your job configuration, add the username and password.
Then select “inject passwords to the build as environment variables.” Use the password you set for the bot user you created on the Tuleap platform.
Set the username directly in the script. USERNAME=”jenkins_bot”
Another way to do this is to use “credentials” feature of Jenkins. It is much more cleaner to pass the password and username this way but you need admin privileges to add credentials on your CI platform.
Next, you will see how to update an artifact during a build execution in a Jenkins Job.
Step 4: Update the issue and more
As described above, you now have all of the information you need to update your Tuleap Tracker. You were able to retrieve information about the issue (artifactId), the tracker (trackerId), and the related Tuleap project (projectId). You also know who (userId) changed the issue status that triggered the build and why it was triggered (triggerFieldValue).
To update the artifact summary, you must:
- Get a token for the jenkins_bot user
- Update the artifact using the token
- Delete the token
You can use cURL to call the REST API if you know it is available on the Jenkins platform. If you need to parse JSON structs returned by cURL, there are applications, like JQ for example, that facilitate this task.
Here is the complete script to complete this task:
USERNAME="jenkins_bot"
ARTEFACTID=$(echo "$json" | sed 's/.*{\"name\":\"artifactId\",\"value\":\"//' | sed 's/\"}.*//')
TULEAP_ADDR=https://TULEAPHOSTNAME
# Get the token
getToken=`curl -XPOST --header 'Content-type: application/json' -d '{"username":"'$USERNAME'", "password":"'$PASSWORD'"}' $TULEAP_ADDR/api/tokens`
# Save token and userId
USER_ID=$(echo "$getToken" | sed 's/.*user_id\"://' | sed 's/,.*//')
TOKEN=$(echo "$getToken" | sed 's/.*token\":"//' | sed 's/",.*//')
# Update Artifact (field_id is corresponding to the ID of the « summary » field in my tracker on my plateform)
curl -XPUT --header 'Content-type: application/json' --header "X-Auth-Token: $TOKEN" --header "X-Auth-UserId: $USER_ID" $TULEAP_ADDR/api/artifacts/$ARTEFACTID \
-d '{"values":[{"field_id": 1479, "type": "string", "label": "Summary", "value": "Test Bug (Tested on Jenkins)"}]}’
# Delete the token
curl -XDELETE --header 'Content-type: application/json' --header "X-Auth-Token: $TOKEN" --header "X-Auth-UserId: $USER_ID" $TULEAP_ADDR/api/tokens
Now that the Jenkins job has been updated, it is time to test the whole use case.
Step 5: Testing
- Select your issue.
- Change the status value from “verified” to “resolved.”
- Check the results on both tools.
The image below shows the follow-up information:
In the example above, the user Dorian Gray changed the status; a few seconds later, Jenkins Bot updated the summary.
In this tutorial you learned how to retrieve issue information from a Jenkins job and how to update the artifact data at the end of your job. You can use a shell script to retrieve Tuleap information. Installing JQ on your Jenkins platform is helpful in parsing what is returned by cURL. Python scripts using JSON and PycURL libraries can also make artifact updates easier.
Taking it further: continuous integration and interoperability between Tuleap and Jenkins
This use case is just one of many possibilities. You can leverage continuous integration and interoperability between Tuleap and Jenkins to automate a number of tasks in your workflow:
- Update issue when build status is updated.
- Retrieve user information to send an email notification at the end of the build.
- Upload a file to an issue.
- Create a new artifact in another tracker.