Reporting : Variables and language-Variables
Here we look again at what we mean by one Variable “reporting” another. In Theorymaker, a Report can have various different formal properties: sub-types are Judgements, Descriptions, Measurements etc. We are particularly interested in reporting on the quality or value of something. Ultimately we will see that evaluation involves Reporting on the Value of Projects.
Theorymaker native speakers use the word “Variable” to talk about both language-Variables and the Variables they refer to. So you can say “number of visitors to the new centre” and mean either the language Variable which is part of your plan, your shared model of how things are going to work; you can think of this as a social phenomenon or as the paper-and-pencil (or more likely electronic) manifestation of this shared model. Alternatively you can mean the actual number of visitors, a Variable which you can observe or measure e.g. by going to the centre, without any knowledge of any plan or Theory.
If you are busy developing a Theory of Change, you won’t have time to worry about the difference. But when we come to look at how our evaluation Report is supposed to reflect various aspects of the project being evaluated, we will need to look at the way language-Variables refer to or “report” Variables in the rest of the world beyond the plan, and that is what we will do in this chapter. In this Chapter we will also explore how language-Variables are themselves Variables.
The report says 'The height of child X is ...' ((positive continuous numbers)) !Rule: some reporting or measurement Rule ensuring the report of the height reflects the actual height The height of child X is ((positive continuous numbers)) direction=TB
So this diagram shows a causal link between the actual height of the child and the report of its height, say on a school health record. This causal link will normally be a relatively involved one which we could probably “zoom in” on to identify other intervening Variables: we could call the whole thing “the reporting Mechanism”, following a “reporting Rule”.
So the Levels of a simple Variable reporting on the state of a traffic light are something like these:
'The traffic light is red' 'The traffic light is orange' 'The traffic light is green'
… where the traffic light Variable is something like this:
The traffic light is red The traffic light is orange The traffic light is green
Language-Variables are Variables too
We already saw that language-Variables report or refer to Variables, Statements report Facts, and that Theories (consisting of language-Variables and Statements) report Mechanisms. So in this sense Variables and language-Variables are two very different kinds of things.
But on the other hand, non-Theorymaker native speakers might be surprised to hear the following claim.
A language-Variable not only reports a Variable but is also itself a perfectly ordinary Variable.
When you think about it, of course it is.
Imagine I am measuring a child’s height and fill the number in on the child’s health form in the box marked “height”.
Each possible response is a Statement, one Level of a language-Variable. Loosely, we can call this language-Variable “the height as reported on the form” and say it reports the actual Variable, the height of the child. We call it a linguistic Variable simply because it actually uses language - as always, in a particular context; in this case an important part of the context is that the other people around me speak the same language and understand the health form, etc.
When we talk about the language-Variable, rather than using it, we can recognise it as a Variable just like the actual height is. So when the Level of the Variable
child's height is 1m, the Level of the language-Variable
report of child's height should be
The critical thing to understand is that under the right conditions, language-Variables can influence and be influenced by other Variables.
Reports are about Variables, not things
In English, we use words like “appraisal”, “judgement” etc. in different ways. We can say “we are measuring the child” interchangeably with “we are measuring the child’s height”, which names the specific reported Variable which the Report will announce. Theorymaker only uses the second form. So in English we can say “go and appraise that restaurant” - but if the actual Appraisal is about the campaign’s overall cuisine quality, on a 1-5 scale, then what we have reported is just that single Variable, which has a 1-5 scale, and not the myriad other aspects of the restaurant. The appraisal Mechanism in this case has to home in on just the one aspect of the restaurant which interests us most.
'Overall quality of the restaurant ((1-5))' -Various aspects of the restaurant (some convoluted causal link)Overall quality of the restaurant ((1-5)) Weight of the restaurant Length of the restaurant Myriad other aspects ...
Theorymaker and communication theory
In order to write this chapter, we need to first have a theory of meaning in a Variable-based world. This ties in closely with Shannon’s communications theory, which is Variable-based.
This reminds us of Shannon’s transmitter-channel-receiver paradigm (Shannon 2001) and also of the stimulus-response paradigm in psychology: we already saw that causation and explanation are essentially contrastive; also information is essentially contrastive, and the objective measure of information content depends on the number of alternative messages.
The Appraisal has the same information content as the Appraised Variable.
A Theorymaker theory of meaning?
Our job as evaluators (or as anyone doing any kind of Reporting) is to make sure that the Report Variable is caused by the reported Variable so that the Report Variable always takes the Level “Variable B has the Level b” just when Variable B just does in fact take the Level b, for all Levels of B; in this case the Report Variable will always be correct (and there we have a very quick and dirty Theorymaker theory of meaning, truth and reference …).
You can see this results in what may seem like a primitively realist analysis of meaning: A Statement X is true if and only if there is an actual state of affairs which corresponds to it and which has the same logical shape as it; it is also in a sense a counterfactual or contrastive theory of meaning.
You could say this makes Theorymaker constructivist, because the Variables which our language refers to are defined via language Variables; and yet Theorymaker is realist because the Variables we are communicating about are “really out there”; they actually cause the truth and falsehood of our Statements.
Reporting that intervenes
The meta-Theorymaker word “Reporting” does sound a bit like a purely passive, receptive procedure. But of course the reporting Rule often involves to a greater or lesser extent intervening.
We should note there is nothing stopping Variables in the reporting Mechanism intervening on other Variables (although not actually on the reported Variable itself or on any Variable which affects it).
Everything we have said (and will still say, under Wicked Systems) about different kinds of Mechanism applies to reporting Mechanisms too. So for example, a reporting Mechanisms might adapt its intervention according xxxx
Reporting on more than one Variable
We already saw that Statements may be composite, not just simple. So Reports can be composite too.
Remember the isomorphic principle: xx - when we say “Rule” this is essentially unimportant whether we are thinking of a single Rule with multiple stages or a composite Rule. Remember also the principles of collapsing and factorising, which essentially mean we don’t have to keep worrying whether the reported Variable is several Variables or one.
Any Variable, or set of Variables, or a Mechanism, can be the subject of a Report.
Reporting on Mechanisms
(Need to say more about this)
You might have two reporting Variables, reporting on the upstream and downstream Variable in a simple Mechanism. But these two Variables won’t normally constitute a Mechanism. The Level of the bottom-right Variable in this diagram is controlled by the Variable it reports on, not by the “Report on classroom noise”.
-Reported Mechanism Teacher irritation Classroom noise -Reporting Variables Report on teacher irritation Teacher irritation Report on classroom noise Classroom noise
Nevertheless it is quite common to report on some aspect of a larger Mechanism, but the reporting Variable is still a Variable, not somehow a whole Mechanism which does reporting.
For example, we might want to report on a project’s cost-effectiveness (more precisely, on its outcomes/inputs ratio, see xx).
-Reported Mechanism, e.g. a project to be evaluated Project outcomes Project inputs -Reporting Variable Project outcomes/inputs ratio !Rule: divide outcomes by inputs using Soft Arithmetic Project outcomes Project inputs
This diagram is a possible fragment of an Evaluation Theory.
-Reported Mechanism e.g. a Project a b -Reporting e.g. on aspects of the Project Report on a a Report on b b Report on relationship between a and b !Rule: e.g. soft division a;b Report which summarises Reports on a and b Report on a;Report on b title=Reporting understood as a Mechanism. Note all links are causal diagram;fontsize=10
some consequence used as a proxy ('indicator') -Reported Mechanism e.g. a Project a - -Reporting e.g. on aspects of the Project Report on a (via proxy) - some consequence used as a proxy ('indicator') title=Reporting understood as a Mechanism: use of proxy Variable. Note all links are causal diagram;fontsize=10 rank=same;a;Report on a (via proxy)
The reported and reporting Variables: same type?
As we already saw, we call a Report “numerical” if its Levels include the actual symbols for numbers; and by extension we call the Variable it refers to “numerical” too.
We’d like to say, “the reporting Variable is always of the same type as the Variable it reports”. But this can be problematic when we turn later to xx emergent reporting Variables.
Shannon, Claude Elwood. 2001. “A mathematical theory of communication.” ACM SIGMOBILE Mobile Computing and Communications Review 5 (1). ACM: 3–55.