Reflective deliberation

Jared Moore and David Gottlieb

Review of Kant

  • The moral good is unconditionally good, and the only unconditional good is the good will.
  • The goodness of the will lies in its conformity to reason.
  • In particular, a good will makes a law for itself.
  • This has moral bite because, when we act wrongly, we contradict our own will.
    • The law of, Take a loan out even though I do not intend to repay, is either:
      • a self-contradictory law (and thus not really a law), or
      • a law that I do not really will as a law (rather, I only want it as a special exception for myself).
    • Notice that the moral error is also a rational error: I do wrong (moral error) because my practical reasoning is contradictory (rational error).

Kant as exemplar of rationalism

  • Sentimentalists say that a moral agent requires a sentimental motivation at the heart of their morality.
    • A rational agent could lack that sentimental motivation, …
    • … So a rational agent is not necessarily a moral agent.
  • Rationalists say that the motivation at the heart of morality is something a rational agent cannot lack.
    • Whatever it is, it comes along with being a rational agent.
  • Kant: a rational agent cannot lack (1) making laws for itself and (2) avoiding contradiction.
    • Thus, a rational agent is necessarily a moral agent.

Korsgaard on reflective self-consciousness

Korsgaard as interpreter of Kant

  • Korsgaard’s argument is mostly an interpretation of Kant.
  • It’s not necessarily the only correct interpretation of Kant.
  • But it’s very valuable because it shows that a reasonably straightforward and convincing interpretation of Kant is possible.

Our self-conscious nature

  • Morality is “grounded in human nature” (Korsgaard 1996, 91).
    • Korsgaard emphasizes a continuity with Hume and Williams here. For Hume, it is grounded in our sentimental nature. For Kant and Korsgaard, it is grounded in our rational nature. But all agree that moral motivation must be “internal” to us in the sense of Williams.
  • Our nature is “self-conscious in the sense that it is essentially reflective” (92).
    • When internal states like perceptions and desires are given to us, we are able to reflect on them and decide whether they give us reasons.

Standing apart from perceptions and desires

I perceive, and I find myself with a powerful impulse to believe. But I back up and bring that impulse into view and then I have a certain distance. Now the impulse doesn’t dominate me and now I have a problem. Shall I believe? Is this perception really a reason to believe? I desire and I find myself with a powerful impulse to act. But I back up and bring that impulse into view and then I have a certain distance. Now the impulse doesn’t dominate me and now I have a problem. Shall I act? Is this desire really a reason to act? The reflective mind cannot settle for perception and desire, not just as such. It needs a reason. (Korsgaard 1996, 93)

Discussion question: in your own life, have you had a desire or perception and then reflected on whether it was “really a reason”? What was that like? Have you had a desire or perception and immediately acted on it without ever reflecting on whether it was really a reason?

Callback: Kant on inclination

  • Korsgaard’s discussion of our reflection on perceptions and desires is an interpretation of Kant’s discussion of our “inclinations,” including sympathy.
  • Kant argues that any inclination can be wrong. As a result, only the good will is unconditionally good.
  • Korsgaard calls this “the normative problem”:
    • When we have an inclination, we are always faced with the problem of deciding whether it is really a reason.
    • This is always in principle a “problem” exactly because no inclination is unconditionally good.
    • Recall that, according to Korsgaard, the word “good” “refers to reflective success.” It names what we reflectively endorse.
  • Hamlet: “There is nothing either good or bad, but thinking makes it so.” There is nothing outside us to tell us what is good or bad, only our judgment.

The remaining problem: why does our rational self-reflection lead us to morality?

Practical identity

  • “The reflective structure of the mind is a source of ‘self-consciousness’ because it forces us to have a conception of ourselves” (Korsgaard 1996, 100).

    When you deliberate, it is as if there were something over and above all of your desires, something which is you, and which chooses which desire to act on.

  • This self-conception has normative content.

    • Normative: we take it to give us reasons for action.

Examples of practical identities

  • Examples:
    • As a teacher, …
    • As a Stanford SYMSYS major, …
    • As an American, …
    • As a Christian, …
  • The force of the reason is, if I don’t do what a teacher does (e.g.), I can’t rationally think of myself as a teacher.

Temptation and cheating

  • If I never show up for my lectures, I can’t think of myself as a teacher. But what if I miss just one?

    You can stop being yourself for a bit and still get back home, and in cases where a small violation combines with a large temptation, this has a destabilizing effect on the obligation. You may know that if you always did this sort of thing your identity would disintegrate, like that of Plato’s tyrant in Republic ix, but you also know that you can do it just this once without any such result. (Korsgaard 1996, 102)

Discussion question: have you ever faced this kind of temptation with regard to any of your practical identities? Example (Jared): how many times can you insult a friend until you are a bully and not a friend?

Is there an identity that can’t be given up?

  • If I do something that a teacher (friend, American, etc.) can’t do, I can give up on being a teacher (friend, American, etc.).
  • In this way, the reasons given us by our contingent practical identities are not unconditional.
  • Kant says morality is unconditional. Is there an unconditional practical identity?

The practical identity of the reflective deliberator

What is not contingent is that you must be governed by some conception of your practical identity. For unless you are committed to some conception of your practical identity, you will lose your grip on yourself as having any reason to do one thing rather than another - and with it, your grip on yourself as having any reason to live and act at all. But this reason for conforming to your particular practical identities is not a reason that springs from one of those particular practical identities. It is a reason that springs from your humanity itself, from your identity simply as a human being, a reflective animal who needs reasons to act and to live. (Korsgaard 1996, 120–21)

Discussion question: can you give this up?

The value of humanity

  • Recall Hamlet: “There is nothing either good or bad but thinking makes it so.”
  • Korsgaard argues that all of our valuing in all of our practical identities depends on our valuing of humanity.
    • We value particular ends through our particular practical identities. I value getting my grades in on time as a teacher.
    • We value humanity through our commitment to having reasons for how we act and live.
  • Thus: valuing anything at all rationally requires me to value my humanity.

Discussion question: Does valuing my humanity rationally require me to value humanity in general?

Rationalism and AI

Rationalists think there is something a rational agent cannot lack that makes it a moral agent. I.e.:

  • there is some form of reasoning, which …
  • if done correctly, …
  • necessarily leads to moral conclusions.

The research program this suggests for AI moral reasoning might include:

  • can we specify this form of reasoning precisely enough to think about whether particular systems are capable of it or not?
  • can we make a moral reasoner by making a system that incorporates this form of reasoning?
  • if, without thinking about morality at all, we build a system that incorporates this form of reasoning, could it emergently produce moral reasoning without being designed to?
  • what might that look like? would it be desirable or undesirable?

Rationalisms we have seen so far

  • The formulaic framework for rationalism is, some form of reasoning which if done correctly necessarily leads to moral conclusions.
  • What do we fill in for “some form of reasoning”?
    • Kant:
      • Argument in smallest nutshell so far: practical reason involves giving ourselves laws, and if we give laws that we wouldn’t want to be laws, we contradict ourselves.
      • “Some form of reasoning”: reasoning about what to do based on the relevant features of a situation, putting aside any inclinations.
    • Korsgaard:
      • Argument: when we reflectively deliberate, we take ourselves to be bound by norms of the roles that we take on. But we cannot “take off” the role of reflective deliberator, so we are always bound by it.
      • “Some form of reasoning”: reasoning about what to do relative to self-assumed norms.

Rationalisms we have seen so far

Small group discussion: do you think LLM chatbots have the kinds of abilities Kant and / or Korsgaard regard as the basis of moral reasoning? Are there any tests you could give a chatbot to explore this question? How else might you investigate it? If LLM chatbots don’t have the abilities that underlie moral thinking, can we think of AI systems that might?

  • These both involve the ability to act: practical reasoning is reasoning that causes us to act.
    • Can LLMs act?
    • What AI might count as acting?

Claude’s opinion

Rationalism, sentimentalism, and being scared of superintelligence

If rationalism is true and:

  • we meet intelligent aliens, …
    • … would they be persuadable by moral argument?
  • we construct a human-level artificial intelligence, …
    • … would it share our moral commitments?

If moral reasoning might emerge from more general reasoning abilities, maybe we could be pleasantly surprised by the moral commitments of intelligent aliens – or of our own super-intelligent AI.

How does intelligence relate to an agent’s ends?

The Orthogonality Thesis.

Intelligence and final goals [i.e., ends] are orthogonal axes along which possible agents can freely vary. In other words, more or less any level of intelligence could in principle be combined with more or less any final goal. (bostrom-2012-the-superintelligent-will?)

An agent that is very intelligent but doesn’t share our ends could be dangerous to us. (Consider the paper clip maximizer.) Accordingly, Bostrom and others have thought it is important for us to think about whether it is likely that other intelligences will share any ends with us. Other moral reasoners might be more likely to share ends with us. (E.g., if Kant is right that the moral law requires treating all rational beings as ends in themselves.)

How does intelligence relate to an agent’s ends?

So it is very relevant to ask whether there are (1) general capacities that (2) intelligent agents are likely to have which (3) could emergently produce moral reasoning. Sentimentalism and rationalism give different kinds of answers about what we should expect those capacities to be.

References

Korsgaard, Christine M. 1996. The Sources of Normativity. Cambridge: Cambridge University Press. https://research.ebsco.com/linkprocessor/plink?id=a757352a-d090-3ea7-b65d-62390e3907b5.