Quantify yourself

Having spent quite a bit of this summer thinking about machine learning and artificial intelligence, it seems to me that there’s a very important transformation ongoing from a focus on the qualitative to a focus on the quantitative. The moment we start with A/B testing, deploying multi-armed bandits or training machine learning models, the very first action we need to take is to define, in precise, quantitative terms, what the factors are that we are optimizing for and what the relative priority of these factors is. And, of course, what factors aren’t allowed to change outside a certain set of boundaries.

In many ways, this isn’t the first time we’re moving into this direction. Earlier, the notion of key performance indicators was widely used to control teams, departments and companies. Google and, before that, Intel have made extensive use of the OKR mechanism (Objective & Key Results), which combines a qualitative objective with quantitative key results.

Still, it seems to me that there’s at least one real change between earlier initiatives and today’s trend and this is the way we interact with software-intensive systems. Until now – and this still is the primary way of working – we’ve developed systems by defining how these should conduct their operations. Using requirement specifications and similar techniques, we’d describe what the system was expected to do in qualitative terms and then design it and define how it should accomplish the intended goal.