We already saw that we can use Features as a way of highlighting parts of Variable labels which are common between different Variables, e.g.
Student-teacher interaction is more solution-oriented Teachers attend training, Stakeholder=Teachers Teachers attend workshops, Stakeholder=Teachers Students attend workshops, Stakeholder=Students Students take part in discussion on social media, Stakeholder=Students
A more elegant solution is often to use “grouping boxes” like this:
Student-teacher interaction is more solution-oriented -Activities, stakeholder=teachers Teachers attend training Teachers attend workshops -Activities, stakeholder=students Students attend workshops Students take part in discussion on social media
We can also use grouping boxes simply to highlight or separate parts of a Theory. Used like this, they mean virtually nothing. They just make the Theory easier to understand and, crucially, help us resist the temptation to use causal arrows for this purpose
Some outcome -First part Some activity Some input activity -Second part A further activity One more input
Theorymaker native speakers claim they can say just about anything which needs to be said in the world of monitoring and evaluation just using these basic ingredients: Variables joined together by Rules. We will now look at an example of Theorymaker native speakers doing just that: highlighting the overall structure of a Theory by using a new organisational element (“grouping boxes”) which can however be defined in terms of Variables and Rules.
Variables grouped by stakeholder
In Theories of Change it is usually a good idea to show what or who can vary, and to mark this clearly with text or graphics. So rather than just “Level of approval of policies” we prefer to know exactly who is doing the approving - the general population? Politicians?
Student-teacher interaction is more solution-oriented Teachers attend training Teachers attend workshops Students attend workshops Students take part in discussion on social media
We have already seen that we can use Features in Theorymaker to highlight things like type and activity:
Student-teacher interaction is more solution-oriented Teachers attend training, Type=Activity, Stakeholder=Teachers Teachers attend workshops, Type=Activity, Stakeholder=Teachers Students attend workshops, Type=Activity, Stakeholder=Students Students take part in discussion on social media , Type=Activity, Stakeholder=Students
This is especially important when more than one Variable belongs to the same agent or stakeholder (and/or time-point), in which case it is useful to highlight this graphically e.g. by using common colours. So we might use one colour for Variables belonging to “Teachers” and another for Variables belonging to “Students”. And we often want to group together those Variables we consider to be activities, or outcomes, etc.
Ordinary logframe formats don’t make it easy to group together Variables which belong together. In Theorymaker, this can be done by coding with colours and shapes. For example we could make activities oval, teachers orange and students green, like this:
Student-teacher interaction is more solution-oriented Teachers attend training; shape=oval; colour=orange3 Teachers attend workshops; shape=oval; colour=orange3 Students attend workshops; shape=oval; colour=green3 Students take part in discussion on social media ; shape=oval; colour=green3
… the disadvantage with this approach is that we need a key to explain that green means “students”, ovals are for activities, etc.
Grouping boxes used like this don’t add information. The diagram above says no more than the one before it. They just help to organise the Theory optically. The reason they are so important is that, without them, people using traditional frameworks like logframes often end up abusing supposedly causal connections just in order to introduce some structure into the plan; see xx. So we sometimes see monstrosities like this:
Student-teacher interaction is more solution-oriented Activities Activities for teachers Teachers attend training Teachers attend workshops Activities for students Students attend workshops Students take part in discussion on social media title=This is a really poor Theory of Change
Variables grouped by narrative
Often we see elements in Logframes which include a description of an intervention and its consequences, e.g. of the form “Getting X by doing Y”. Sometimes they are intended as some kind of title for the elements upstream, for example as the name or description of a project phase or focus.
Improving student outcomes through training and improved motivation Improved student outcomes through training teachers Improving teaching by training teachers Improved teaching Training teachers Improving student motivation through after-school activities Motivated students After-school activities title=Another terrible Theory of Change wrap=7
Now we certainly don’t want to deal with Variables of that kind of form (“Improving student outcomes through training and improved motivation”). We should always prefer logically simple Variables. See xx. But often these phrases aren’t really meant as Variables but are inserted into a logframe just as narrative titles of some part of the Logframe - or even of all of it. Grouping boxes provide a much better way to include these kinds of descriptions.
-Improving student outcomes through training and improved motivation Improved student outcomes --Improving teaching by training teachers Improved teaching Training teachers --Improving student motivation through after-school activities Motivated students After-school activities
Variables grouped by organisational unit
Variables grouped by “Pillars”
Another common way of organising projects into parts is “pillars”. There is no formal definition of this idea - pillars can group project Variables according to narrative and/or organisational unit, and so on.
see xx. Note the use of Scenario Analysis in economics.
Which is the superprocedure, the final Rule?
Limitations of grouping boxes
It isn’t always possible to use boxes everywhere we want, for example it is usually difficult to use them both for separate time-points, e.g. baseline and endline, if we also want to use them for actors, e.g. students and teachers; or in this case, where the grouping boxes overlap because Variables can belong to more than one category of the same type.
So we have to resort to a different method to group the other attribute, and vice-versa:
-Endline Student achievement (endline), type=student -Baseline Student achievement (baseline), type=student Student ability, type=student Teacher skills, type=teacher
-Student Student achievement (endline), time=endline Student achievement (baseline), time=baseline Student ability, time=baseline -Teacher Teacher skills, time=baseline
King, Julian. 2016. “Using Economic Methods Evaluatively.” American Journal of Evaluation, 1–13. doi:10.1177/1098214016641211.