What do we need to know? Utility.
Ensure that evaluation Statements provide useful and actionable information, in the context in which the Report will be read. For example, simply accepting or rejecting hypotheses is not as useful as a best guess, with an indication of how confident we are in the Statement, about what to do next.
Say something about Bayesian updating.
Davidson, J: Certainty About Causation
Academic training Teaches us to be terribly cautious about our conclusions: “The evidence appears to suggest”. We are trained to push for at least 95% certainty (p < .05) – and even then we don’t call it “proof”.
- This language is often incredibly frustrating for clients
- In many contexts, decisions are made based on much less certainty (≈ 60-70% or less?)
- Need to match methods – and the way we talk about the certainty of our conclusions – with decision-maker needs (not with academic conventions)