By Zahn Alimbay in Program Effectiveness — May 25, 2024

Let’s Talk About Performance Metrics

Following the first significant bribery and corruption scandals of the early 2000s, companies started devoting more attention to the growing US FCPA exposure and investing in their prevention and detection capabilities—from introducing codes of conduct to employee training and communication to monitoring dealings with public officials to establishing ethics hotlines and investigation management. Tracking those activities and trying to understand what works and what doesn’t become an essential element of every Compliance Program design.

What do we measure?

Almost all Compliance Departments do some kind of performance tracking nowadays, whether it is the number of employees trained, C-suite communications issued, third parties screened, Speak Up reporting volumes, or number of remedial actions triggered by the audits or those resulting from internal investigations. Depending on the industry, size and maturity of Program, performance metrics can go from basic XLS trackers to hundreds, if not thousands, of data points.

This is generally driven by several concurrent factors, including Enforcement agencies’ continuing focus on Compliance Program effectiveness (i.e., the requirement to demonstrate that the program works in practice), oversight, reporting and public disclosure obligations, as well as the desire to analyse whether invested efforts bring intended results.

As the famous saying goes, “You can’t manage what you can’t measure.” (This quote is often attributed to management guru Peter Drucker, but according to the Drucker Institute, he never actually said that.)

The problem with Metrics

The truth is that not all that can be measured can be managed, and vice versa. Not all which can be managed and improved can be measured. Think of corporate culture or measuring perception.

Sometimes, we become obsessed with metrics. We look at numbers as if they hold all the solutions, like a mystical crystal ball. The compliance field is no exception to this trend. But it’s not the data itself; it’s what we do with it matters. And that’s where the problem lies.

Quantity vs. Quality

Let’s take the example of training. Organisations, at minimum, track the number of attendees and sessions conducted, engagement and completion rates, subjects on which employees are trained, employee feedback and knowledge retained.

But not all engagements are the same, and not all means of delivery are equal. Content, as well as who delivered the training (personal experience, charisma, ability to engage, and audience), make a big difference.

Two sessions on the same subject delivered by different people to different audiences in different contextual situations (say, geographical location) will never have the same effect. Quantitative metrics will never reveal this, however.

While metrics are essential for focusing attention and tracking purposes, they do not tell the whole story. Importantly, it will not tell you whether the compliance process or program as a whole operates effectively. For that, we need to talk about quality, not quantity.

(Re-) Focusing on what matters

So far, so good, except that no quantitative metrics will tell us anything about what causes the data to show what it shows. It's just the numbers at the end of the day. At best, we can only see the results or consequences of actions.

Understanding the “Why”. The key to achieving a real change is understanding what drives the behaviour in the first place. Organisations and people do not exist in isolation, and every action has its reason and purpose (intended or unintended, even if not obvious). The question to ask: what are the settings, triggers, actions, and results that such behaviour aims to achieve?

As we learned from the health and safety field many years ago, something that more and more companies finally bring into the field of ethics and compliance is root-cause analysis.

Indeed, once we understand what actually happened and, more importantly, why it happened, we can (i) remediate the issue, (ii) devise targeted controls to close the gap and improve detection, and (iii) design fit-for-purpose prevention strategy that can help to ensure this will not re-occur in the future.

Qualitative Engagements. Nothing helps to understand what is happening in the organisation better than a face-to-face engagement. This is why compliance professionals need to spend as much time as possible with the business leaders and front-line teams on the ground. It’s not only about conversations but insights.

Working out the Patterns. Correlation is not a causation. However, data can help us understand whether behavioural patterns warrant compliance professionals’ attention. Once you see the emerging patterns, look at those through the prism of root-cause analysis, cause-to-effect links and lessons learned from past compliance incidents and misconduct. Does it all make sense? Are there any anomalies in the data? Something unusual? What do trends show? Dive in.

Focusing on Outcomes and Impact. The most critical element of all of this, and often the hardest, is to work out whether compliance actions or controls, if and when operated as intended, achieve the intended results and lead to the impact we want them to have.

Does a better understanding of the regulatory or procedural requirements (delivered through the training), in fact, lead to a reduction in misconduct or a decrease in organisational risk levels? Or is there anything else at play here? What are the levers (enablers) and barriers? Does the company need to invest in better training resources or re-emphasise the role of middle management in driving the ethical message?

This often requires us to examine those often “hard-to-measure” components such as trust, moral attentiveness and engagement, and corporate culture that are key to unlocking a deeper understanding of behaviours and their underlying causes. Speak Up is a great example here, too.

Insights, oversight and foresight

Based on my experience, I can confidently say that few Boards and senior leaders are actually interested in compliance metrics alone. What they are trying to understand is what those numbers tell them about the issues the organisation faces, the risks, and how effective the compliance program is in preventing, detecting, and responding to those.

It's the compliance leaders' role to help them achieve the right level of comfort in understanding how the company deals with compliance risks. But there’s more to this; it’s about anticipating the risks—looking around the corner and understanding what to worry about.

The insights path the way to foresight.

See you next Saturday!

If you are embarking on your own compliance transformational journey and need help designing and enhancing your compliance program, get in touch with us. We are here to help!

Comhla Intelligent Compliance

At Comhla, we are driven by a mission to revolutionise the way organisations approach compliance and misconduct prevention. By leveraging our in-depth governance, compliance and internal control expertise, actionable data insights and cutting-edge applied research in organisational science, we help our customers build effective regulatory and compliance management to safeguard their license to operate, protect the bottom line and enhance reputation as responsible businesses.

Learn More https://comhla.co

We aim to publish weekly. The information provided in this newsletter is not intended to and does not render legal, accounting, tax, or other professional advice or services.

Subscribe to Breaking the Mould