The book is disguised, in a light-hearted way, as an introduction to “Theorymaker”, the fictional dialect of a fictional island, Theorymaker Island. Learning Theorymaker will help you express evaluation ideas and concepts precisely. Written Theorymaker looks, and is, a lot like English, though it is structured in a special way. But mostly inhabitants of Theorymaker Island, who call themselves Theorymaker native speakers, communicate using a special kind of diagram. This is especially popular with the younger ones who, of course, have an app for that. The diagrams correspond exactly to the written language but they are more fun and easier to understand.
Theorymaker.info is a free website which converts written Theorymaker to diagrams.
Theorymaker seems to have originated when a ship full of pirates from an unknown country were shipwrecked on a little-known island near England, hundreds of years ago. The indigenous people welcomed the newcomers, the pirates behaved themselves very well and a new dialect developed which is based very much on English. However, the natives were delighted to find that the newcomers’ grammar was far superior to English for expressing concepts connected to the pirates’ favourite pastime - making plans, executing them, and assessing the results, which we would now call “M&E”. So Theorymaker is a dialect very close to English but in which M&E concepts can be particularly easily expressed. In particular, Theorymaker native speakers are more precise in the way they express “Variables” and their possible (factual and counterfactual) “Levels”, and the way in which these can be linked up into more or less deterministic “Mechanisms” which they describe with “Theories” about those Mechanisms. So concepts like “Effectiveness” and “Impact” are quite easy to define and use in Theorymaker. What’s more, Theorymaker makes it easy to express and compare many of the key ideas of a wide range of evaluation approaches from Outcome Mapping to RCTs.
Another remarkable thing about Theorymaker is that many Theorymaker native speakers are very hard of hearing; so although Theorymaker can by all means be spoken out loud, many Theorymaker native speakers written Theorymaker, which is highly structured, or a pictorial language of diagrams which is equivalent to the written and spoken dialect and is favoured by younger Theorymaker native speakers.
So, this book will teach you to read and write Theorymaker, a dialect for M&E, and you will learn to express those tricky causal concepts as well as those ancient Theorymaker native speakers learned to, hundreds of years ago.
Theorymaker speech, writing and diagrams
One of the most convenient things about Theorymaker is that it is mainly just English, with a few tweaks. When Theorymaker native speakers are speaking about evaluation matters - Variables, efficiency, outcomes, etc - they use a special kind of intonation which they capture in writing by using a special grey background like this:
This sentence is in Theorymaker.
For everything else, Theorymaker native speakers just use English. They do use quite a lot of Theorymaker though, because they like to pass time in the long evenings talking about things like attribution and contribution, bless them.
Written Theorymaker should be pretty easy for non-native speakers to read and write. However, it can be difficult for non-native speakers to speak.
For example, this piece of Theorymaker:
Teacher skills Teacher presence on training course
says something like “Teacher presence on training course contributes to Teacher skills”. Or you could read it as “Teacher skills are influenced by presence on training course”.
If you hover your mouse over the word “TEXT” to the top right of the diagram, you will see the equivalent written Theorymaker.
The second line is indented by one space, and Theorymaker native speakers have a special kind of intonation for this which is difficult for non-native speakers to say or even hear accurately.
English speakers can just use bits of Theorymaker without learning the whole dialect.
You can pick and choose which Theorymaker phrases you want to use.
Hopefully this will lead, at least piecemeal, to better understanding between English speakers when speaking about things that matter in evaluation. So just as you can sparkle at a party (sometimes) by throwing in a bit of French, you can make a mark with your M&E colleagues with even just a few phrases of Theorymaker.
Is Theorymaker a “framework” for evaluation?
You might be excited by how the various Theorymaker concepts fit together and you might wonder if this means you are about to read about an overarching framework for evaluation. It is true that these concepts fit together in the sense that they are compatible one another and are defined in terms of one another but I prefer to think of Theorymaker as a bunch of interconnected concepts, a language, a bazaar rather than a cathedral.
Does Theorymaker give us a template for any programme theory or evaluation?
I don’t suggest for the moment that you need to use all of these pieces in a particular order or all of the time. Evaluation and monitoring in practice do not face a fixed set of narrow problems, so learning Theorymaker does not mean you have to adopt a rigid “template”.
Theorymaker gives us a set of concepts we can use piecemeal to creatively to address the unpredictable mess of problems we have to deal with as evaluators and does not tie us to using all of it, all at once or all of the time.
It quite often happens that I have to review a project which has been using some kind of logical framework with non-standard or substandard features. There is no point me saying “oh if only you had used my special patented logframe template” or “I will just redesign your five-year programme for the sake of this terminal evaluation even though you have been using some other framework”. It is more useful to have a set of tools from which we can pick as necessary to address one or two issues which are particularly thorny.
Theorymaker native speakers are not trying to sell you new evaluation “methods” or templates or tools. We really have enough of that already. It’s much more about understanding and discussing and getting clear about some of the basic concepts we use when we do evaluation.
Personally, I’d much rather do an evaluation based on good old-fashioned Logframe done by somebody who was thinking while they wrote it, than to conduct one based on a new and complicated but poorly thought-through framework.
It is also worth noting that Theorymaker, like most dialects, provides various different ways to say the same thing. Theorymaker native speakers use which ever is most convenient and mellifluous. So while we will learn first of all just one way, there are other styles too which you will sometimes more convenient as your proficiency in Theorymaker improves.
Why bother learning a fictional dialect?!
Because the Theorymaker native speakers manage to communicate with one another about tricky topics like evaluation without the misunderstandings which plague English and (I assume) the other languages which M&E staff use today. Theorymaker native speakers can recite a Theory of Change or a project proposal as if it was an amusing story or an inspiring poem, and no-one argues about what it means or the words it contains. They often conduct Randomised Controlled Trials, for fun and especially on holidays, but they also love Outcome Mapping and many other approaches to evaluation; and they see no contradiction in this.
So, “learning Theorymaker” helps us to introduce some words, concepts and conventions which we can use to think and write about projects and programmes from an evaluation perspective. Using these conventions, we can have a fairly standard way to communicate how we think projects work and what effects can be attributed to them.
So we will define some slightly new way of making causal statements, as well as some key words like Variable, Level of a Variable, and so on, for talking about these statements. We will do it in a way which is (I hope) compatible with most approaches in social science textbooks, but which is a bit different from them too.
Sure, you can say that Theorymaker is a bit weird. But at least it’s pretty much like English and yet it can express some complicated causal ideas as well as the kind of much more scary mathematical notation you’d find in, say, an economics textbook or the works of Judea Pearl. What’s more, Theorymaker can be expressed in the form of diagrams too and everyone loves a diagram.
Theorymaker, a website which understands Theorymaker
That’s all very well, but I can’t spend my life messing about with PowerPoint to make those kinds of diagrams.
That’s why I made TheoryMaker.
theorymaker.info speaks Theorymaker!
theorymaker.info is a free website which produces Theorymaker diagrams from written Theorymaker. Above most of the diagrams in this book you will see a small “Clone” link. If you click on that link, you will be taken to theorymaker.info, where you will be given your own version of the diagram which you can play with and edit, and, if you wish, save.
At the Theory Maker website, you will see a text window on the left which you can type into. If you type something in Theorymaker into that window, for example the text you see above the diagrams in this book, a corresponding diagram appears.
What else you can do at Theorymaker.info
To find out more about the app click the menu inside the app: “What is Theory Maker”.
To learn how to use Theory Maker, click on an Example which interests you to see the text which created it together with some explanatory notes.
About Theorymaker diagrams
Theorymaker diagrams, the kind Theory Maker produces, are a kind which graph theorists call directed graphs. They are pictures of what is being said in Theorymaker: a Theory about causal links between Variables. And the reverse way of thinking is, if you have created a Theory Maker diagram by typing text, you have created not just a picture but an actual Theory, namely the text you typed.
Theorymaker also includes some features called “decoration” whose only purpose is to decorate the diagrams - for example to use different colours or switch from top-down to left-to-right orientation. You can find out much more about these things at Theorymaker.info.
So this is in Theorymaker:
A goal An outcome
and this is essentially the same diagram but provides also some decoration:
A goal; colour=red An outcome
Theorymaker slang and meta-Theorymaker
Going through the book you will encounter Theorymaker slang.
Rules for using Theorymaker slang are printed in boxes like this. Theorymaker Slang allows us to introduce new, alternative phrases into Theorymaker, defined in terms of existing phrases.
Slang is powerful. When you have understood the basic idea of how a Rule and how it predicts (or defines) the Levels of a child Variable on the basis of the Levels of a parent Variable, you have all you really need. Everything else Theorymaker native speakers need to capture in Theories how projects and programmes work, and to conduct evaluations on the basis of such Theories, can be defined in terms of this basic building block. That is what Slang is for.
Slang is a way of re-writing more complex Theorymaker Statements to make them simpler or to make a specific point.
In a more formalistic, mathematical style we would talk about “additional notation” rather than “Slang”.
There are two kinds of slang:
- new symbols to replace old
- ways of simplifying existing blocks of text
Sometimes we also need to talk about Theorymaker, e.g. “The next line is a Statement in Theorymaker”. Mostly we talk about Theorymaker using “words in capitals” like “Theory” and “Statement”5.
Pedants and nerds like me will be happy to see that the Law of valid Theorymaker, plus Theorymaker slang and meta-Theorymaker, approximate to a “formal grammar” (in the sense of xx) for Theorymaker, complete with syntax and semantics. For more details, see xx.
We will also encounter Theorymaker proverbs.
Some Theorymaker proverbs will be shown in boxes like this. They encapsulate ancient Theorymaker wisdom in the form of hopefully easy-to-remember messages about the Theorymaker approach.
(Actually they are pretty prosaic at the moment but hopefully some more literary inspiration will come to me later.)
Finally, we will encounter “Claims” - results of reasoning about Theorymaker.
Nerds and philosophers should note that Theorymaker does not provide any way of doing formal reasoning within Theorymaker. We just reason in ordinary English. That is why claims are called “claims” and not, say, “theorems”.
Claims are presented like this:
This is what a Claim about Theorymaker looks like.
So these definitions and the things we can say with them are strictly just English, not Theorymaker.↩