This task involves the work of analyzing how people are engaging with/using the first iteration of Edit Check (references).
Decisions to be made
- 1. Will we make any fundamental changes to where and how the Reference Check is presented before investing more energy into refining and scaling the initial approach?
Learning objectives
Learn whether the concept(s) we've designed and built show clear signs that lead us to be confident they will be effective in causing
- Experienced volunteers at the English and French Wikipedias to:
- Trust the prompts Edit Check will be presenting to newcomers and Junior Contributors will encourage them to make changes that experienced volunteers consider to be productive (read: aligned with project policies and conventions).
- Newcomers from Sub-Saharan Africa to:
- Add sources when they add new content or explicitly decide not to do so
- Feel supported and empowered by the feedback and calls to action Edit Check presents them with
Evaluation
| Facet | Research Question | Metric(s) for evaluation |
|---|---|---|
| Awareness | Are newcomers and Junior Contributors encountering the guidance Edit Check is designed to offer them? | Proportion of newcomers and Junior Contributors that make an edit attempt and are presented Edit Check feedback |
| Engagement | Are newcomers and Junior Contributors considering and acting on the calls to action Edit Check is presenting them with? | Proportion of newcomers and Junior Contributors that are presented Edit Check feedback and engage with it in some way (for example, (1) select “yes” to adding a citation, (2) explicitly choose not to add a reference with providing a reason, (3) explicitly choose not to add a reference without providing a reason) |
| Relevancy | Is Edit Check presenting newcomers and Juniors Contributors from SSA with feedback that is relevant to the changes they are attempting to make? | 1) Proportion of newcomers and Junior Contributors that are presented with Edit Check feedback and indicate that the check was shown in error and 2) Proportion of newcomers and Junior contributors that add a reference after being presented edit check |