A Short Guide to Metrics

Simple rules to follow when creating metrics

1. Over time: You must see how metrics change over time. Ideally you can view them by day, week, and month. No pie charts!
2. In groups: Metrics require balance. Think of the extremes. If you over-optimize for one metric, what problem will it create?
3. Weighted appropriately: If different times of the week or geographic areas are more important, your metrics should reflect this.
4. Robust: Chances are your data is not pristine. Be careful of outliers. Median or percentile can be better than the mean.
5. Keep it simple: People should be able to understand the metric.

Nice to have

1. Responsive: Prefer metrics that change quickly as a result of your actions.