Automatically or manually. Choose how remaining effort is calculated.

 

In the Tuleap issue tracker tool, computed fields are special fields. They allow to do the sum of all values of a field in artifacts sharing the same field name. For example, we use a computed field to calculate the remaining effort of a release. This post aims to explain how the computed field is working, both on the functional and technical sides.

Agile use case

Imagine, your agile work is organized in releases, sprints, user stories and tasks, with the below hierarchy.
 

The Sprint remaining effort = the sum of all user stories remaining effort planned in the sprint.
The remaining effort of a Release = the sum of all sprints release remaining effort

The Scrum burndown displays visually the remaining effort

How does it work?

To understand how the fields computation is made, it is interesting to know how data is stored into Tuleap.

  • Each tracker is composed of several fields.
  • Fields can get a simple or complex type (int/float/list/computed/artifact_link...).
  • For history purpose, every change on a field is stored as a changeset.

Here is the structure of the Database in Tuleap:

click to see the image bigger

To compute value of a computed field for an artifact, it is necessary to check if it has nodes artifacts until we reach a leaf or a node without any leave. Each leave should have a value defined. It is necessary to compute all these values.

Let's start with a simple calculation

 

A simplistic approach to get the result is to make a classic recursive tree walk: a first query for Release R1, then a second one for Sprint A, then a new one for Userstory #1, then a query for Userstory #2 and so on. As you might imagine, this will have a major impact in term of performances.

To avoid this issue, we have developed a query that retrieves:

  • the list of all nodes of an artifact
  • the list of all leaves of an artifact

This has 2 main advantages:

  • the number of queries is constant regardless the number of nodes in the tree (one query per node depth)
  • this constant doesn't tend to increase over the time

It's way more common to have a small tree with a large number of leaves (think about Epics, Stories and Tasks with a very large number of tasks) rather than a tree with very deep node and a few leaves.

For release R1, this means that the query has to browse all the attached sprints.

The query will return:


+----------------------+-------+-----------+
+          artifact_id + value + type      +
+----------------------+-------+-----------+
+                    A +  null +  computed +
+                    B +  null +  computed +
+----------------------+-------+-----------+

Then we want to play the query again to see the sprints' leaves.

+----------------------+-------+---------+
+          artifact_id + value +  type   +
+----------------------+-------+---------+
+                    1 +    10 + integer +
+                    2 +     8 + integer +
+                    3 +     5 + integer +
+                    4 +     8 + integer +
+----------------------+-------+---------+

Now, it is necessary to add the values to get the sum of the release. At this stage, it a standard tree traversal algorithm. Here is the simplified code:

$artifact_ids_to_fetch = array($release_id);
do {
    // retrieve nodes and leaves with the SQL query
    $list = $this->query(...);

    $artifact_ids_to_fetch = array();
    foreach ($list as $row) {
        if ($row['type'] === 'computed') {
            // it's a node, we want to retrieve its own node and leaves
            $artifact_ids_to_fetch[] = $row['artifact_id'];
        } elseif (isset($row['value'])) {
            // add the value to remainings effort
            $sum += $row['value'];
        }
    }
} while (count($artifact_ids_to_fetch) > 0);

A user story linked to multiple sprints

As illustrated in the above schema, your team has started working on the User story #2 during the Sprint #A, but at the end of the sprint, the story is not yet finished. The remaining_effort is up to 5.
In this case, your team can keep the story with its current stage, and link it to the sprint B.

 

When a user story is linked to several sprints, we do not want to compute twice the remaining effort value. Otherwise, the release remaining effort would be wrong.

Here, the release remaining effort equals to 20. But it is not equal to sprint #A + sprint #B because the story #2 is linked to two sprints. When your tree has more than one node level, it is possible for a leave to be linked to several nodes. In order to not compute twice the same value, the algorithm will browse all the tree and store only the values it has not encountered yet. It will not take into account the remaining effort values it has already seen.

$artifact_ids_to_fetch = array($release_id);
$already_seen          = array();
do {
    // retrieve nodes and leaves with the SQL query
    $list = $this->query(...);
    
    $artifact_ids_to_fetch  = array();
        foreach ($list as $row) {
            if (! isset($already_seen[$row['id']])) {
                if ($row['type'] === 'computed') {
                     // it's a node, we want to retrieve its own nodes and leaves
                    $artifact_ids_to_fetch[] = $row['artifact_id'];
                } elseif (isset($row['value'])) {
                    // add the value to remainings effort
                    $already_seen[$row['id']] = true;
                    $sum += $row['value'];
                }
            }
        }
} while (count($artifact_ids_to_fetch) > 0);

Introducing manual values

For any reasons, you can change the automated calculation of the remaining effort. You can increase or decrease it manually.

 

Here the sprint #B has been manually changed from 15 to 40. It does not equal anymore to the sum of story #3 and #4. This time, to calculate a correct release remaining effort, the algorithm has to stop on the manual value of the sprint remaining effort. It is not needed to continue to browse the linked user stories.

do {
 
   $list = $this->query(...);
    
    $artifact_ids_to_fetch  = array();
    foreach ($list as $row) {
        if (! isset($already_seen[$row['id']])) {
            if (isset($row['value']) && $row['value'] !== null) {
                // it s a manual value on a computed field
                $sum += $row['value'];
            } elseif ($row['type'] === 'computed') {
                 // it's a node, we want to retrieve its own nodes and leaves
                $artifact_ids_to_fetch[] = $row['artifact_id'];
            } elseif (isset($row['value'])) {
                  // add the value to remainings effort
                  $already_seen[$row['id']] = true;
                  $sum += $row['value'];
            }
        }
    }
} while (count($artifact_ids_to_fetch) > 0);

Conclusion

One big advantage of Tuleap, is that the software doesn't impose you a way of working. You control your workflow!

To give you this freedom we had to plan many use cases. So when doing computation, it might have:

  • tree with a complex artifact structure
  • nodes with a huge number of leaves.

The best way we've found to solve this is the implementation of breadth-first search traversing tree algorithm, which enables us to:

  • read every nodes/leaves once
  • stop when a node has a manual value

If you have already encountered and solved the same problematic, we are interested by any suggestions you can make to improve our algorithm or to share return of experience.


Join Online meetings More tutos

Share this post

Leave a comment

To prevent automated submissions please leave this field empty.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Websites located at tuleap.org and other tuleap.org subdomains need to store and access cookies on your device. We need your acceptance. Get more information.

Yes, I agree No, I disagree