Rules - different types

Theorymaker is useful in exposing “how things work” in our Theories and Theories of Change because it provides us with some rough templates for the most common kinds of Rule, the ways in which Variables can influence one another.

In the model below, rather than making individual and independent positive contributions, motivation and ability multiply their effect on performance. (According to soft arithmetic for multiplication, which is defined even for non-numeric Variables.)

Student exam performance !Rule:multiply

 Motivation

 Ability

In the next model, the effect of external disturbances on performance is marked as negative.

Student exam performance

 Motivation

 Ability

 (!-)External disturbances

In the case below, the influence of motivation is marked as very weak, whereas the wide arrow shows the substantial influence of ability.

Student exam performance

 (Influence=very weak;width=2)Motivation

 (width=9)Ability

 (!-)External disturbances

In this next example, one rich Variable influences another via a rich Rule (marked with a ?).

Student mood ((Rich))  !Rule:Rich

 Type of activities ((Rich))  

Next, the performance is hypothesised to be optimal under a medium level of stress, falling off again as stress increases too far.

Student exam performance !Rule:updown

 Level of stress 

Below, a person’s feeling of satisfaction with some task is highest when the level of challenge is balanced by motivation and resources.

Satisfaction !Rule:balanced

 Challenge;Motivation and resources

Here, a person’s feeling of satisfaction with some task is conceived of as a ((no,yes)) Variable which is yes if and only if the person’s motivation and resources are adequate for the task. So only if the motivation and resources are greater than or equal to some Level determined by the challenge, the person feels satisfied. This kind of configuration is quite common in project design.

Satisfaction ((no,yes)) !Rule:adequate<U+2507><U+2507>

 Challenge;Motivation and resources

In this final example, more details about the Rule are given in the text.

Student mood ((Rich))  !Rule:Rich effect of activity type lasts a long time 

 Type of activities ((Rich))  

The nature of the various links between parent and consequence Variables in typical logic models or Theories of Change is rarely explicitly specified. Sometimes this is not a problem, but in other cases serious confusion can arise.

Nearly all theoretical discussions on the kinds of Rules which are of relevance to evaluators focus (implicitly) on one of two kinds of Rules or functions: propositional (Boolean; very common in project frameworks) or probabilistic / linear (most common in social science).

What is the point of this kind of categorisation?

First, note that Rules are the heart of Definitions as well as Mechanisms. So while understanding different kinds of Rule is important for understanding Mechanisms, which are key to programme theory, yet the same ideas will help us understand Definitions, which are particularly important in evaluation theory.

Second, although in this chapter we look at unproblematic kinds of Rules, the most interesting part of this classification is in a later Section xx, in which we look at the different ways in which Rules can be adaptive, rich etc. And in chapter xx we look at what happens in open systems, when a Mechanism is changed in unpredicted ways.

So this approach gives a single way to get some systematic understanding of many of the vary various different claims we hear about “chaos” and “complexity”, “non-linear systems”, etc both in programme theory and evaluation theory.

In This chapter, we will look at relatively unproblematic Rules.

Wouldn’t it be convenient if there were just two or three different kinds of Variable, and maybe just three or four different possible kinds of Rule to link them up? Then we could easily mark the appropriate Rule on each simple Theory and everyone would be clearer about the Variables and how they are inter-related, about how the whole Mechanism is supposed to work.

Unfortunately, life isn’t that simple. Any attempt to do that would encompass whole tracts of the physical, social and mathematical sciences.

For example,

  • In the narrow case in which the influence and consequence Variables are all binary, yes/no Variables, the ancient Greeks discussed circumstances under which we can say that an influence Variable is necessary and/or sufficient for the consequence Variable to happen.
  • And as we have already seen, there are also numerical Variables; the interrelationships between them have mainly been the domain of the physical sciences. Numerical Variables can combine to influence other Variables in many different ways.
  • But there are also what we call here (chapter xx) lo-hi Variables, like well-being and satisfaction and attraction and approval, which have degrees of intensity but which can not be easily mapped onto numbers. There are also no agreed formal ways to express Rules between these kinds of Variables (though there are lots of informal ways - our lives are full of these kinds of Variables and our conversation is full of our Theories about the links between them). So informally we can say “the more people come, the better the meeting will be” but a social scientist would have difficulty to express this idea formally without using numbers. Theorymaker provides us with ways of expressing Rules involving lo-hi Variables.
  • And some Variables are vague, and Rich, and in various senses qualitative, and it has been left mostly to some branches of the social sciences to discuss the different kinds of connection they may have with one another.
  • Just as on Earth, most Theorymaker projects and programmes adapt to circumstances as they develop. Theorymaker has some great ways of expressing adaptive and emergent Rules, see chapter xx. Theorymaker native speakers look at most of our project designs, in which there are no adaptive feedback loops, rather the way an Earthling pianist would listen to a child just learning to play the piano.

All of these different kinds of Variables and Rules connecting them, and plenty of others too, are taken seriously in Theorymaker and may be included in different kinds of Theories of Change.

I don’t believe there is any way to definitively categorise all possible kinds of Rules. But we can make a start.

What I present here is only a beginning.

Saying whether an influence is positive or negative

Simply saying that family spending power is influenced by these three Variables doesn’t say very much:

Family spending power

 Income

 Debts

 Savings 

Is the overall influence maybe negative or positive? Perhaps the influence is just about detectable, just statistically significant; that might be very important for a social scientist but it isn’t going to interest a donor for my project. What if we have to invest a million Euros into the project in order to slightly improve the spending power of just one family?

Ideally, a simple Theory should also include a Rule to say how and how much different combinations of those influence Variables affect the consequence Variable. Theorymaker can help us do this.

Individual influences marked on arrows

If we want to express the likely relationship between family spending power, income and debts, the least we can do is specify the direction of the influences.

Family spending power

 (!-)Income

 (!-)Debts

 ()Savings 

Although we still haven’t been able specify in any detail how income, savings and debt influence a family’s spending power, but at least we have been able to say whether their influences are overall positive or negative as indicated by the little minus and plus signs.

Suppose you want to say that Variable2 has, for example, a strong influence on Variable1.

This diagram:

Variable1 !Rule Variable2 has strong influence on Variable1 

 Variable2

… can be rewritten like this; and in the corresponding diagram, the individual influence of the Variable is marked at the head of the arrow.

Variable1

 (Influence=strong)Variable2

Individual influences: - and + signs

… and in particular …

 A !Rule B has an overall positive influence on A
  B

can be rewritten like this:

A
 (Influence=+)B

and similarly with negative influences.

In Theorymaker, marking a + at the head of an arrow suggests that its Influence is “overall positive”, and marking a - suggests the opposite.

“Overall positive”

A Rule between two ordered Variables is overall positive if the Effect of any positive Difference on the upstream Variable is also a positive Difference.

Reading skill of particular child 

 Special reading intervention

Slang: influences are “overall positive”

By convention, when confronted with a simple Theory in which there is no information about the Rule or even the direction of the influences, but in which the Variables are ordered Variables see above, Theorymaker native speakers make the following assumption:

Assume the influence of each Influence Variable on its Consequence(s) is roughly positive, unless otherwise stated.

This rule isn’t a hypothesis. There is no reason to suppose that, in the real world, the majority of Mechanisms actually do work like this. Perhaps most actual Mechanisms are devilishly complicated. On the other hand, as Pearl points out, (Pearl 2000) xx, as Theories are human constructions, it is pretty likely that we choose the simplest we can find to work with, ones that depend on as few other Variables as possible17.

If the Variables involved were all “proper” numerical Variables, we could use mathematics to calculate whether the overall effect of the Influence Variable was positive. But in the more general case of ordered Variables which are not necessarily numeric, we can only resort to Soft Arithmetic, which can often but not always tell us the answer to this kind of question.

(Multiply) overall positive

In the above model, rather than making individual and independent positive contributions, performance increases multiply overall positive with motivation and ability. This means that the influence of every parent Variable is overall positive on the child Variable when the other parent Variables are held constant.

Binary Variables (“sufficient”)

In this example, either or both of the Influence Variables is enough to make the Consequence happen. This kind of configuration is famous and was of interest in Aristotle’s time; we say that each Variable is sufficient for the Consequence.

Dad annoyed !Rule:or ((no,yes)) 

 Kids shout loudly ((no,yes))

 Kids are rude ((no,yes))
Enough publicity !Rule:or ((no,yes)) 

 Successful TV campaign ((no,yes))

 Successful internet campaign ((no,yes))

Binary Variables (“necessary”)

There is another equally famous configuration for two or more binary Variables in which we say that each Variable is necessary for the consequence.

Applicant gets a passport ((no,yes)) !Rule:and

 Applicant has submitted all the documents ((no,yes))

 Applicant has paid the fee ((no,yes))

Count-to-nominal: Choosing the winning candidate

In this example, three candidates are standing for election:

Candidate elected  ((Candidate 1, Candidate 2, Candidate 3))  !Rule:? select the candidate with the most votes 

 Number of votes for Candidate 1 ((0-max))

 Number of votes for Candidate 2 ((0-max))

 Number of votes for Candidate 3 ((0-max))

A Variable like “Candidate elected” is usually called a nominal Variable. Note that the Consequence is certainly not any kind of sum of the Influences, so we spell out the Rule involved (“select the candidate with the most votes”).

Sometimes it is important to spell out what kinds of Variables are involved and sometimes it is important to spell out the Rule which says how the Influences affect the Consequence.

Moderators

More binary examples

Parallel contribution; overdetermination; sufficient not necessary conditions

The concept of necessary and sufficient conditions has been important historically in attempts to analyse the concept of causation. For these chapters we will consider only the effect of Boolean Variables, but with a twist: the other influence Variables in a given context do not have to be Boolean. There might be lots of them and many or all of them might be integer or any other kind. The trick is just to focus on the worst and best possible combinations of these Variables: can they foil the effect of the Variable of interest (represented with orange boxes in the diagrams) on the child (outcome) Variable?

In the context of this Mechanism, Soldier 1’s firing the gun is a sufficient but not necessary condition. (In the context of the same Mechanism, Soldier 2’s firing the gun is also a sufficient but not necessary condition). We can best understand sufficiency in the way we have just introduced, not as an absolute property of a Variable but as a contribution in the context of some given Mechanism with at least one other influence Variable.

Victim dies !Rule:or

 Soldier 1 fires gun ((no,yes))

 Soldier 2 fires gun ((no,yes))
Other conditions = most unfavourable Other conditions = most favourable
Variable of interest = No Outcome = No Outcome = Yes
Variable of interest = Yes Outcome = Yes Outcome = Yes

Necessary but not sufficient

Fire !Rule:and

 Fuel & oxygen ((no,yes))

 Spark ((no,yes))
Other conditions = most unfavourable Other conditions = most favourable
Variable of interest = No Outcome = No Outcome = No
Variable of interest = Yes Outcome = No Outcome = Yes

Necessary and sufficient

Victim dies !Rule equals soldier 1

 Soldier 1 fires gun ((no,yes))

 Rain in Tasmania ((no,yes))
Other conditions = most unfavourable Other conditions = most favourable
Variable of interest = No Undetermined Undetermined
Variable of interest = Yes Undetermined Undetermined

We can see here that rain in Tasmania plays no part: looking across the rows, it makes no difference to the Difference attributable to the Variable of interest.

No influence

Rain in Tasmania plays no part: looking down the columns, the Difference in the Variable of interest makes no difference.

Victim dies !Rule no Rule

 Rain in Antarctica ((no,yes))

 Rain in Tasmania ((no,yes))
Other conditions = most unfavourable Other conditions = most favourable
Variable of interest = No Outcome = No Outcome = No
Variable of interest = Yes Outcome = Yes Outcome = Yes

Also in this case, rain in Tasmania plays no part: nothing makes any Difference to the outcome.

Multiple Variables - Q&Q

Student exam performance

 Quantity and quality of private study

  Parental support

… we can think of that as an abbreviated version of this:

Student exam performance

 Quantity of private study;Quality of private study

  Parental support

(If one were really going to measure this, one would need at least:

-Student



Student exam performance

--Unit of private study

 Number of minutes of private study;Quality of each unit of private study

--

  Parental support

)

… but the former is in some ways conceptually simpler. Note that the pressure to (too quickly) convert a Theory of Change into easily-measurable “indicators” will often lead, in this kind of case, to the measurement of only one or the other. Whereas only a Theory which captures both quality and quantity can best explain how Parental Support is converted into Student Exam Performance.

Rich Rules

We will look more at rich Rules in section xx.

Life is full of impressive examples in which we discuss claims of causation or at least influence between Variables, even though it would be a real challenge to list their Levels.

Looking back at an earlier example:

Writing the connection like this exposes the two Variables involved.

the Board launched a major revision of strategy ((no, yes)) !Rule: something really complicated

 the report contents were: ((rich))

So although the report Variable could have trillions of different Levels (all the different possible contents the report could have), only a large subset of them would lead to the major revision of strategy.

Another possible configuration:

the Board launched a major revision of strategy *yes--* ((no, yes)) !Rule: equals

 the report contents were *damning--* ((no, yes))

where the Rule equals would just mean true if true, false if false. However, this isn’t the same thing as the previous rendering - it says something like “because the report contents were damning, the Board launched a major revision of strategy”, which isn’t quite the same thing.

Painting competition

Another example - children’s paintings sent in to a painting competition. Each picture could be, say, given a score by judges. So the judges’ score and the painting form a Mechanism. Each painting can be seen as a Level of the Variable covering “all the different possible paintings” - which of course one could never precisely delineate. Nevertheless, here is a simple Rule:

Expert score ((1,2,3,4,5)) 

 Painting

This is a perfectly good causal model in Theorymaker, and I’d argue, in English too:

The response of the management committee !Rule:incomplete 

 The contents of the financial report

… even though there is a world of things not precisely defined, for example, are we thinking of vaguely linear Variables on some negative - positive axis? Or are they free of even that restriction?

Sub-symbolic Rules?

Maybe some Rules are simply not capable of being expressed symbolically - what in AI are called “sub-symbolic” algorithms,

Structure for narratives

Suppose all we have is pieces of narrative-like theory, with no numbers or measurement at all, all with rich Variables and rich Rules. Even so, Theories of Change can provide a way to organise them and join them up. It’s not of course the only way - in particular one could join them up narratively - but it might be a good compromise between flavour and structure, emic and etic.

Much talk about increasing distrust of statistics (Davies 2017) - people having negative reactions even to positive stories because they were told in terms of numbers.

References

Pearl, Judea. 2000. Causality: Models, reasoning and inference. Cambridge Univ Press. http://journals.cambridge.org/production/action/cjoGetFulltext?fulltextid=153246.

Davies, William. 2017. “How statistics lost their power – and why we should fear what comes next | William Davies | Politics | The Guardian.” The Guardian. https://www.theguardian.com/politics/2017/jan/19/crisis-of-statistics-big-data-democracy.


  1. and which avoid spooky “action-at-a-distance”, see xx