This communication covers a change of computation shipped in Echoes in February 2024.

We are rolling out of change of computation for effort allocation which will result in slightly more accurate numbers for most team, with no downsides.

Tuesday, 20 February 2024 (all times are CET)

Echoes is committed to give you the best insights and dashboards with a seamless experience. We continuously assess how users integrate Echoes in their workflow and how they interact with Echoes features. We are changing the default-weight given to unlabelled GitHub Pull Requests (PRs), or Merge Requests in GitLab.

No action is required from you. However, given that we depart from an historical rule, we deem important to notify you and motivate this change.

Recall that the Effort label is a value expressed in T-shirt size (XS, S, M, L, XL) to weigh whether a PR and its associated work took a lot of effort or not. Then, on a per-Team basis, Echoes normalizes total weight with the number of active contributors to the team, so that different teams can have different meaning for the weights. Today, Echoes interprets the absence of an Effort label to be equivalent to an Effort-M-label. We are changing this default value to interpret the absence of an Effort label to be equivalent in weight to an Effort-XS label. The default value plays an important role in teams where members do not uniformly tag effort: the default-weighted PRs will bias dashboards. As a result, Echoes recommends to tag an Effort value. However, Echoes is committed to adapt to your process rather than the other way around.

We pondered the advantages and disadvantages of various default choices and came to the conclusion that using the Effort-XS weight as default will benefit many Echoes users and have no downsides. Indeed, today the main usage of Effort-tags is pairwise: to surface important and effort-heavy work and to reduce the bias from numerous “small PRs”. Defaulting to an Effort-M weight as we do today has a downside: this default does not match the statistical reality. Indeed, from a statistics point of view, small PRs outnumber the large ones. The change we will roll out will reduce this gap. Indeed, without an Effort label, assuming the PR to have a small weight is the best guess. Therefore, after this change, we can expect dashboards to change slightly: dashboards will get closer to the reality. More accurate dashboards! This is a win for Echoes users.

This change will also benefit developers, product people, and managers who label PRs. Switching the default weight will not only — as explained above — reduce the amount of tags users have to place on PRs. This change will also simplify the decision users make when picking an Effort tag: users’ choices can only have effects in one direction and towards increasing the PR’s importance. Therefore, the mindset becomes to surface the associated work rather than having to think in both directions and spending time for tuning down the importance of (numerous) small PRs. Fewer and better focused tags! This improved experience is another win for Echoes users.

This change only affects teams that inconsistently use Effort labels. Hence, the change will not have any effect for teams who label all PRs with an Effort label (recommended) as labels take precedence over default values. The change will also have no effect for teams who never label PRs with Effort labels as the change does not modify the relative weight between unlabelled PRs.

The rollout takes two steps. Echoes will perform this rollout on a per-account basis as follows:

The rollout will take place on Tuesday, 20 February 2024 along the day. Updates will be posted on this page.

Thus if you have screenshots of dashboards from before the rollout and compare them to the same dashboard (e.g., same team, time-period) you may have a slightly modified pictures.

You need not take any action, however if you have any question our Support team ([email protected]) is always ready to answer.