Reasonable preferences

Jared Moore and David Gottlieb

Roadmap

  1. Introduction
  2. The negative argument: why morals are not derived from reason.
  3. The positive argument: what sentiments are morals derived from?

Introduction

Reason, slave of the passions

Hume: “Reason is, and ought only to be, the slave of the passions.”

’Tis not contrary to reason to prefer the destruction of the whole world to the scratching of my finger. (Treatise, 2.3.3.6 / 267)

Mel Brooks: “Tragedy is when I cut my finger. Comedy is when you fall into an open sewer and die.”

What do you think?

Is this deal irrational?

The AI Mel Brooks

  • We try to train AI to reason correctly (among other things).
  • Suppose we have an AI that always reasons correctly.
  • Could it decide it would be best if all humans fall into a sewer and die so that it doesn’t cut its finger?

The European Enlightenment

  • The Enlightenment was a period when European intellectuals felt that prevailing beliefs and ways of living were suddenly in need of justification

    • The usual spin (Kant): people (Europeans) are beginning to leave their “self-incurred minority” and enter rational adulthood.
    • Kant is wrong to think that people are thinking for themselves for the first time ever
  • But something was changing in Europe:

    1. Social arrangements were changing: industrialization, science, successful rebellions against monarchies, and the spread of republican governance.
    2. Colonialism was bringing Europeans into contact with very different ways of life.

Choosing ways of life

American founder Ben Franklin wrote:

When an Indian Child has been brought up among us, taught our language, and habituated to our Customs, yet if he goes to see his relations and make one Indian Ramble with them there is no persuading him ever to return.

And … when white persons of either sex have been taken prisoner young by the Indians, and lived awhile among them, …

… tho’ [brought back to European society] by their friends, … in a short time they become disgusted with our manner of life, and the care and pains that are necessary to support it, and take the first opportunity of escaping again into the Woods, from whence there is no reclaiming them. (Letter to Peter Collinson, 9 May 1753)

Questions about difference

  • Can we justify our way of life? Or should we change it?
  • What explains the commonalities, or differences, between our way of life and other ways?
  • As Hume notes, if morals are derived from reason, they must be necessary: true for all times and places.
  • Does sentimentalism better account for differences in moral judgment?

The Negative Argument

Hume’s case in a nutshell

Morals can’t be derived from reason, because morals motivate us to action, and only passions can motivate us to action. Reason involves evaluating things as true or false, but no such conclusion by itself moves us to action.

Hume’s psychology

Everything in our experience is a perception. There are two kinds of perception.

Impressions.

Impressions are “stronger perceptions.” They are feelings given to us either from outside (perceptions of the world) or inside (emotions).

Ideas.

Ideas are “fainter perceptions, or the copies of [impressions] in the memory and imagination.” For example, the idea of a triangle is a copy formed from impressions of triangles.

Passions.

… are impressions. To act, we must have a passion that moves us to act.

Reason.

… operates on ideas. It can judge whether they are factual, or how they relate to each other. (Does a triangle have three sides? Is the Earth round?)

Two kinds of reasoned judgments

Since there are two kinds of judgments of reason, if moral judgments are judgments of reason, they must be either

  1. Factual judgments
  2. Judgments about the relations between ideas

Therefore, moral wrongs must be errors of one of these kinds of judgment.

Are moral mistakes factual mistakes?

Factual mistakes can affect our actions:

A fruit, for instance, that is really disagreeable, appears to me at a distance, and thro’ mistake I fancy it to be pleasant and delicious. Here is one error.

I choose certain means of reaching this fruit, which are not proper for my end. Here is a second error.

But we don’t morally blame someone for a factual mistake:

I ask if a man guilty of these two errors is to be regarded as vicious and criminal? Or if it be possible to imagine, that such errors are the sources of all immorality?

Are morals relations between ideas? The parricidal tree

Relations between ideas are not enough to trigger moral judgment:

Of all crimes that human creatures are capable of committing, the most horrid and unnatural is ingratitude, especially when it is committed against parents. …

To put the affair, therefore, to this trial, let us choose any inanimate object, such as an oak or elm; and let us suppose, that by the dropping of its seed, it produces a sapling below it, which springing up by degrees, at last overtops and destroys the parent tree….

I ask, if in this instance there be wanting any relation, which is discoverable in parricide or ingratitude? Is not the one tree the cause of the other’s existence; and the latter the cause of the destruction of the former, in the same manner as when a child murders his parent?

The argument against morals being relations between ideas

  1. Ingratitude is a paradigm moral wrong. (Exemplified by parricide.)
  2. A sapling that outgrows and kills its parent tree has, in relevant part, the same relation to its parent that a human parricide does.
  3. But there is no moral wrong in what the sapling does.
  4. Therefore, the moral wrong of ingratitude does not derive merely from the relation of ideas.

How would you oppose this argument?

Two ways to deny that the sapling is like the parricide – and why they’re both circular

’Tis not sufficient to reply, that a choice or will is wanting. For in the case of parricide, a will does not give rise to any different relations, but is only the cause from which the action is deriv’d; and conse- quently produces the same relations, that in the oak or elm arise from some other principles.

If it be answer’d, that this action is innocent in animals, because they have not reason sufficient to discover its turpitude; but that man, being endow’d with that faculty, which ought to restrain him to his duty, the same action instantly becomes criminal to him; shou’d this be said, I wou’d reply, that this is evidently arguing in a circle. For before reason can perceive this turpitude, the turpitude must exist; and consequently is independent of the decisions of our reason, and is their object more properly than their effect.

Julie and Mark

Julie and Mark, who are sister and brother, are traveling together in France. They are both on summer vacation from college. One night they are staying alone in a cabin near the beach. They decide that it would be interesting and fun if they tried making love. At the very least it would be a new experience for each of them. Julie is already taking birth control pills, but Mark uses a condom too, just to be safe. They both enjoy it, but they decide not to do it again. They keep that night as a special secret between them, which makes them feel even closer to each other. So what do you think about this? Was it wrong for them to have sex? (Haidt 2001)

The positive argument: the sentimental basis of morals

Preview: moral sentiments

  • Hume: moral approval is a disinterested feeling of approval
  • Smith: moral approval is when, in assessing an act, we imaginatively feel the same emotional reaction that produced it (i.e., we sympathize with it).

Hume’s high-level answer

We make moral judgments when we like or dislike someone’s actions or character without thinking about whether it’s good or bad for us in particular.

The good qualities of an enemy are hurtful to us; but may still command our esteem and respect. ’Tis only when a character is consider’d in general, without reference to our particular interest, that it causes such a feeling or sentiment, as denominates it morally good or evil.

Morality as too general to be innate

Hume:

For as the number of our duties is, in a manner, infinite, ’tis impossible that our original instincts shou’d extend to each of them….

Lessons for AI?

Smith: morality and sympathy

  • Moral judgments are feelings of sympathetic agreement – feeling the same way others do.
  • We develop our moral sense because we take pleasure in sympathizing with each other, …
  • and in sympathizing with the pleasure we each take in sympathizing with each other, …
  • and so on.

Sympathy is how we feel each others’ feelings

  • We can’t feel each other’s feelings directly.*
  • Instead, we do it by perspective-taking. We imagine ourselves in like circumstances and imagine how we would feel.
    • Argument 1: we don’t sympathetically feel others’ emotions if we don’t know anything about what provokes them.
    • Argument 2: we often sympathetically feel emotions that the target of sympathy definitely doesn’t have. We only feel them because we imagine we would have them.

Approval as perfect sympathy

A spectator judges an actor as morally appropriate when the actor’s actual emotions perfectly agree with the sympathetic emotions of the spectator:

When the original passions of the person principally concerned are in perfect concord with the sympathetic emotions of the spectator, they necessarily appear to this last just and proper, and suitable to their objects.

I.e., we judge someone to be morally right when they exhibit just the feelings we imagine we would feel in their circumstances.

An experiment in the failure of sympathy

What does it feel like when someone seems to have a very different feeling about something than you would or do?

Do you have sensations in your body? Pleasant, unpleasant? What are they like?

What do you think about the person in question?

Striving towards concord

  • Not only do we enjoy sympathizing with the feelings of another, …
  • and enjoy another sympathizing with our feelings, …
  • we also enjoy sympathizing with another’s enjoyment of us sympathizing with their feelings.
  • Because of this, we tend towards concord:
    • As actors, we learn to feel the sorts of feelings that others sympathize with more easily
    • As spectators, we learn to sympathize with the sorts of feelings people have as actors

Sophie DeGrouchy: starting from a richly social picture

Each person finds herself, for all necessities— her well- being and life’s comforts— in a particular dependence on many others [ … ] This particular dependence on a few individuals begins in the crib; it is the first tie binding us to our fellow creatures. (de Grouchy, letters, quoted in (buckner-2023-rational-machines?), 305)

We learn to share each others’ feelings because this is how we survive as babies. Sharing each others’ feelings is more basic than having our own feelings.

Is sentiment enough for morality?

  • Sympathy helps us live together, because we cooperate better when we care about each other’s interests.
  • But is sentiment enough for morality?
  • If we built sentimental machines, would we fall short of building moral machines?

Possible objections:

  • Our sympathetic imagination can be wrong (Anushka)
  • If our sympathetic habits are learned, how can we learn to challenge the biases of our environment? (Jolie)
  • Can we sympathize with agents different from us? (Eli)

Sentimentalist AI

Two approaches

  1. Can AI systems figure out other people’s beliefs and desires?

  2. Can AI systems be said to have motivations?

Can AI systems figure out other people’s beliefs and desires?

  • Yes, largely under the heading of “theory of mind”

  • There’s some quibbiling about the kind of architecture needed: whether a system has to be “born with it” (a symbolic architecture in some form) or whether this can be learned (an empiricist or “learned” approach).

  • We’ll talk about the evolutionary and psychological evidence for these things more in week five

Can AI systems be said to have motivations (emotions, an affective response, a basic passion)?

  • What counts as having a motivation?

    • (What would it mean for AI systems to have the “right” motivations?)
  • What kind of (AI) architectures might we use here?

  • We’ll talk about this more in week 6 but it also relates to identity and selfhood which we’ll discuss in week 4

References