Output 3.2: Controlled experiment (A/B test) capabilities
(see 2018-2019 Audiences Annual Plan section on the Better Use of Data Program)

Initiate and proceed with a cross-departmental working group that makes concerted improvements to our ability to make scientific product decisions through controlled experiments. The group will iteratively standardize the technology tools, scientific methods, and guidelines by which we can run experiments, leading to experiments becoming increasingly more common in our decision making.

This group may evolve from or overlap deliberately with the instrumentation group, because controlled experiments rely on instrumentation capabilities.

Part of Outcome 3: Data collection
(see 2018-2019 Audiences Annual Plan section on the Better Use of Data Program)

Reduced cost of collecting data on program metrics and on the feature usage that supports those metrics. New features and products will have proper instrumentation from their initiation, and the data we use will be more trustworthy and have fewer caveats when analyzed and communicated.