Mathematical versus moral truth

This piece was inspired by a discussion between Magnus Vinding and Brian Tomasik, as well as a private discussion between Magnus Vinding and myself.

Some philosophers say that moral claims like killing is bad can be proven true or false. This position is called moral realism and according to a recent PhilPapers survey on the views of philosophers, it is a very common view among philosophers. It’s important to understand that moral realists not only claim truths in terms of some underlying assumptions! They don’t just say “If our goal is to reduce suffering, then torturing squirrels is objectively bad.” (In Kant’s terminology this would be called a hypothetical imperative by the way.) They claim that the goal of reducing suffering itself or some other goal can be true or false by itself.

One particular comparison that pops up in discussions of moral realism is that between morality and mathematics. After all, there are absolute (“a priori”) truths without assumptions in mathematics, right? Well…

Truth in mathematics

Until the beginning of the 20th century, most mathematicians probably would have argued that mathematics finds absolute truth without making assumptions. (Interestingly, Greek mathematician Euclid had axiomatized his geometry some two thousand years ago. However, the axioms were seen as “undoubtably/obviously true” without much further thought. Nevertheless, the idea of axiomatization, i.e. basing mathematics on assumptions, was not completely unknown.) But then mathematics had a foundational crisis mainly about problems like Russel’s paradox, which showed that you can easily run into some serious problems, if the foundations of mathematics aren’t studied in a rigorous manner and judgment of correctness of inference is based solely on “intuitive obviousness”. (An extremely accessible and overall great introduction to the topic is the graphic novel Logicomix.) To get rid of contradictions in mathematics, people tried to systematize its foundations. It turned out that the most solid way to go was explicit axiomatization: assuming a small set of statements (called axioms) that all the rest of mathematics can be deduced from. Luckily, a very small set of pretty basic assumptions called Zermelo-Fraenkel set theory is enough for deducing all of mathematics. And axiomatization is still the standard way to talk about truth in mathematical logic. (There are, however, different philosophical interpretations, some of them saying that the axioms are just undoubtedly true.)

Note that when some axiom system implies 2+2=4, this very statement of axioms => 2+2=4 is still not true in an absolute sense as it requires definitions of what all the symbols in that statement mean. And these definitions can only be made when using other symbols, for example those of natural language. (And the higher level proof of axioms => 2+2=4 can only be made by assuming some properties about these meta-symbols.) At some meta-…-meta-level we have to stop with a set of statements/definitions that just have to be assumed true and understood by everyone, or we have an infinite regress. (Humans usually stop at the language level, which is somehow obvious to everyone.) Or we get cycles (“a set is a collection” and “a collection is a set”). That’s the Münchhausen trilemma.

Maybe more importantly, axioms => 2+2=4 is not really the statement that we want to be absolutely true. We want 2+2=4 without underlying assumptions.

So, how did people do mathematics before axioms, then? How did they manage to avoid finding underlying axioms? Well, they always assumed some things to be obvious and did not require further justification. And these things were not even on the level of axioms, yet, but more like commutativity (n+m=m+n, e.g. 1+2=2+1). So, when proving some mathematical hypothesis, they wrote things like n+m=K and therefore also m+n=K and people didn’t ask why, because commutativity of addition is obvious. This is still how it is today: People know that if you continue to ask why they will end up at something like the Zermelo-Fraenkl axioms. And even if they don’t view the Zermelo-Frankl axioms as the foundation of mathematics, almost everyone agrees about commutativity of addition and basic arithmetics in general, whether generated by the Zermelo-Fraenkl or the Peano axioms.

So, this sounds terrible. How do we know that 2+2=4 if all of this is based on axioms that can’t be proven? Well, for one, mathematics is mainly relevant as a model of reality, which can be used to make testable predictions: if we take two separable things and add another two separable things then we will have four things. (That does not work for clouds, you see.) If we had an axiom system in which the natural definition of addition makes 2+2=5 true, then this addition would not be useful for predicting the number of things you have in the end. So, that’s one way to justify the use of Zermelo-Fraenkl: as a gold mine of simple models that can be used to make correct predictions.

There’s another silver lining. The sets of axioms that are usually used are extremely simple and basic. For example, one of the Zermelo-Fraenkl axioms states that if you have a couple of sets, then there is another set which contains all the elements of the sets. While it is philosophically troubling that proofs can’t verify “absolute truths” without some assumptions, it’s obviously not very productive to ask: “But what if the union of two sets really does not exist?” Just like it is not very productive to put into question the rules of logic and probability theory. Without assuming such “rules of thought”, it is not even clear how to think about doubting them! (Thanks to Magnus Vinding for pointing that out to me.) So, some things must be assumed if you don’t want your brain to explode.

Also, some of the axioms can be seen as so basic that they merely define the notion of sets. So, you may view the axioms more as definitions of what we are talking about than as some assumptions about truth.

So, some set of assumptions (or at least a common understanding of symbols) is simply necessary for doing any useful mathematics, or any useful formal reasoning for that matter. (Similarly, you need probability theory and some prior assumptions to extract knowledge from experience. And according to the no free lunch theorems of induction, if you choose the “trivial prior” of assuming all possible models to have the same probability you won’t be able to learn from experience either. So, you need something like Occam’s razor, but that’s another story…)

There is not much wrong with calling mathematical theorems “true” instead of “true assuming the axioms of … with the underlying logical apparatus …”, because the meaning does not vary greatly among different people. Most people agree about assuming these axioms or some other set of axioms that is able to produce the same laws of arithmetic. And there is also a level on which everybody can understand the axioms in the same way, at least functionally.

(Interestingly, there are exceptions to universal agreement on axioms in mathematics. The continuum hypothesis and its negation are both consistent with the Zermelo-Fraenkl axioms (assuming they themselves are consistent). This has led to some debate about whether one should treat the continuum hypothesis as true or false.)

Truth in ethics

So, let’s move to ethics and how it can be justified. The first thing to notice, I think, is that making correct claims in ethics requires additional assumptions or at least terminology. Deducing claims about what “ought to be” or what is “good” or “bad” requires new assumptions or definitions about this “oughtness”. It’s not possible to just introduce a new symbol or term and prove statements about it without knowing something (like a definition) about it. (This is probably what people mostly talk about when they say that moral realism is false: Hume’s is-ought-gap which has never been argued against successfully.)

But one needs to be able to make normative statements, i.e. statements about goals. Acting without a goal is like trying to judge the validity of a string of symbols without knowing the meaning of the symbols (or like making predictions without prior assumptions): not very sensible. Without goals, there is no reason to prefer one action over another. So, it does make sense to introduce some assumptions about goals into our set of “not-really-doubtable assumptions”.

Here are a few options that people commonly choose:

  • using the moral intuition black box of your brain;
  • using a set of rules on a meta-level, like “prefer simple ethical systems”, “ethical systems should be in the interest of everyone”, “ethical imperatives should not refer in any way to me, to any ethnic group or species” etc.;
  • using a lot of different, non-coherent (“object-level”) rules, like the ten commandments, the sharia, the golden rule, egoism etc., and deciding on specific questions by some messy majority vote system;
  • using some specific ethical imperative like preference utilitarianism, the ten commandments + egoism, coherent extrapolated volition etc.

(Note that this classification is non-rigorous. For example, there is not necessarily a distinction between rules and meta-rules. Coherent extrapolated volition can be used in the imperative “fulfill the coherent extrapolated volition of humankind on September 26, 1983” or in the meta-level rule “use the moral system that is the result of applying coherent extrapolated volition to the version of humankind on September 26, 1983”.  Also, most people will probably keep at least some of their intuition black box and not override it entirely with something more transparent. Maybe, some religious people do. Eliezer Yudkowsky would argue that affective death spirals can also lead non-religious people to let an ethical system override their intuition.)

Diverging assumptions

What strikes me as the main difference between assuming things in the foundations of mathematics and assuming some foundations of (meta-)ethics, is that there are many quite different sets of assumptions about oughtness and all of them don’t make your brain explode. It’s extremely helpful to have mathematical theories of arithmetic (or at least geometry) to produce useful/powerful models. And as soon as you have this basic level of power, you can pretty much do anything, no matter whether you reached arithmetic directly or via Zermelo-Fraenkl. Without the power of arithmetic, geometry or something like that, an axiomatic system goes extinct, at least when it comes to “practical” use in modeling reality. (Similar things apply to choosing prior probability distributions. In learning from observations, you can, in principle, use arbitrary priors. For instance, you could assume all hypotheses about the data that start with a “fu” to be very unlikely a priori. But there is strong selection pressure against crazy priors. Therefore, evolved beings will tend to find them unattractive. It’s difficult to imagine how the laws of probability theory can be false…)

For ethical systems the situation is somewhat different. Most people base their moral judgments on their intuition. However, moral intuition varies from birth and continues to change through external influences. There does not seem to exist agreement on meta-level rules, either. For example, many people prefer simple ethical systems, while others put normative weight on complexity of human value. People disagree about object-level ethical systems: there are different consequentialist systems, deontology and virtue ethics. And there are plenty of variations of each of these systems, too. And, of course, people disagree a lot on specific problems: the moral (ir)relevance of non-human animals, the (il)legitimacy of abortion, whether punishment of wrongdoers has intrinsic value, etc. So, people don’t really disagree on the fundamentals of mathematics, but they fundamentally disagree on ethics.

And that should not surprise us. Humans have evolved to be able to work towards goals and to learn about abstract concepts. Arithmetics is an abstract concept that is useful for attaining the goals that humans tend to have. And thus, in a human-dominated society, the meme of arithmetics (and axiomatic systems that are sufficient for arithmetic) will spread.

I can’t identify similar reasons for one ethical system to be more popular than all others. Humans should have evolved to be “selfish” (in the sense of doing things that were helpful for spreading genes in our ancestral environment) and they are to some extent (though, in modern societies, humans aren’t very good at spreading genes anymore). But selfishness is not a meme that is likely to go viral: there are few reasons for a selfish person to tell someone else (other than maybe their close kin) that they should be selfish. (Some people do it anyway, but that may be about signaling…) So, one should expect that meme to not spread very much. Including relatives and friends into moral consideration is a meme much more virulent than pure egoism. People tend to raise their children that way to stop them from exploiting their parents and from competing too hard with their brothers and sisters. Generosity and mutual cooperation among friends is also an idea that makes sense to share with your friends. Life in larger society makes it important to cooperate with strangers especially if they pay taxes to the same government that you do.

But as opposed to these somewhat ethical notions that can be selfish to spread, most people tend to think of ethics as giving without expecting something in return, which is not favored by evolution directly. So, while we should expect the notions of egoism, and cooperation with kin, friends and even members of one’s society (potentially at the risk of being a “sucker” sometimes) to be universal, there is much less direct selection pressure towards, say, helping animals. Explaining “real altruism” (not cooperation, signaling etc.) with evolutionary psychology is beyond this post, but I suspect that without strong selection pressures in any specific direction, there is not much reason to assume that there is much homogeneity. (The continuum hypothesis can be seen as a precedent for that in the realm of mathematics.)

Diverging assumptions make moral “truth” relative in a practical sense. A moral claim can be true on the set of assumptions held by many people and wrong on another set of assumptions also held by many people. So, instead of saying “true” one should at least say “true assuming hedonistic utilitarianism” or “true assuming that simple ethical systems are to be preferred, ethics should be universalizable and so on”. (Unless, of course, you are talking within a group with the same assumptions…)

Lack of precision

Another difference between ethics and logic is that ethical statements are much less precise than mathematical ones. Even before the foundations of mathematics were laid in the form of axiom systems and logical rules of deduction, there was little disagreement on what mathematical statements meant. That’s not the case for ethics. Two people can agree on killing is wrong and disagree on meat consumption, the death penalty and abortion, anyway, because they mean “killing” in different ways. People can agree on ethics being about all sentient beings and have the same knowledge about pigs and still disagree on whether they are ethically relevant, because they don’t agree on what sentience is.

Lack of precision makes moral truth ill-defined for most people. If your goal is to reduce suffering and you don’t have a precise definition of what suffering is, then you can’t actually prove that one set of consequences contains less suffering than another even if it is intuitively clear. If you have a plain intuition box, things are even worse. And thus one problem in normative ethics seems to be that few people are able to say what kind of argument would convince them to change their mind. (In the empirical sciences, statements are frequently blurry, too, of course, but usually much less so – scientists usually are able to make falsifiable claims.)

Conclusion

With moral “truth” being relative and usually ill-defined, I don’t think the term “truth” is very helpful or appropriate anymore, unless you add assumptions (as a remedy to relativity) that are at least somewhat precise (to introduce well-definedness).

As a kind of summary, here are the main ways in which moral realists could disagree with this critique:

  1. “Truth in mathematics (and the principles of empirical science, e.g. Occam’s razor, Bayesian probability theory) is absolute. At some level there is a statement that requires neither further definitions nor other assumptions. This makes it possible to also prove things like 2+2=4 without underlying assumptions, where 2,+, = and 4 can be translated into other things that don’t require further definition.”
  2. “The trilemma arguments about mathematical truth don’t apply to moral truth. While mathematical truth requires some underlying assumptions, moral truth does not require underlying assumptions. Or at least no further assumptions about ‘oughtness’.”
  3. “There is some core of assumptions about ethics that everybody actually shares in one variation or another similar to the way people share arithmetics and these can also be turned into precise statements (to everyone’s agreement) that make moral truth well-defined. With derivations from these underlying assumptions, most people could be convinced of, for example, some specific view on animal consciousness.”
  4. “Even though people disagree about the assumptions and/or their conceptions of ethics are non-rigorous and therefore nontransparent, speaking about ‘truth’ (without stating underlying assumptions) can somehow be justified.”

As I have written elsewhere, I am not entirely pessimistic about reaching a moral consensus of the sort that could warrant calling moral claims true in the sense that objection 3 proposes. I think there are some moral rules and meta-rules that pretty much everybody agrees on: the golden rule, morality being about treating others, etc. And many of them seem to be not completely blurry. At least, they are precise enough to warrant the judgment that torturing humans is, other things being equal, bad. Works like that of Mayank and Daswani and myself show that concepts related to morality can be formulated rigorously. And Cox’s theorem, as discussed and proved in the first chapters of E.T. Janes’ book on probability theory, even derives the axioms of probability from qualitative desiderata. Maybe, though I am not sure about this, there is a set of axioms that many people could agree on and that is sufficient for deriving a moral system in a formal fashion.

There are some models of this in the realm of ethics. For example, Harsanyi proved utilitarianism (though without actually defining welfare) under some (pretty reasonable) assumptions and Morgenstern and von Neumann proved consequentialism under another set of very reasonable assumptions. Unfortunately, somebody has yet to come up with an axiomatization of moral intuitions that is convincing to more than just a few people…

Acknowledgements

Adrian Hutter made me aware of an error in an earlier version of this post.

Dale Carnegie on hedonic set points

The hedonic set point is a level of happiness to which humans tend to return to throughout their lives, even after dramatic events like winning the lottery or becoming paraplegic. It is a relevant concept throughout effective altruism: knowledge of hedonic set points (like other aspects of neuroscience) informs efforts to improve the well-being of at least humans and potentially also other mammals or even evolved creatures in general. It is also an argument for wealthy people to give large amounts of money to charity. Beyond a certain point having more money will not make you significantly happier, so it’s natural to give it away to where it can make a big difference.

In chapter 2 of part 2 of his famous classic How to Win Friends and Influence People (also recommended to effective altruists in general), Dale Carnegie writes on the idea that is now associated with hedonic set points:

Everybody in the world is seeking happiness – and there is one sure way to find it. That is by controlling your thoughts. Happiness doesn’t depend on outward conditions. It depends on inner conditions.

It isn’t what you have or who you are or where you are or what you are doing that makes you happy or unhappy. It is what you think about it. For example, two people may be in the same place, doing the same thing; both may have about an equal amount of money and prestige – and yet one may be miserable and the other happy. Why? Because of a different mental attitude. I have seen just as many happy faces among the poor peasants toiling with their primitive tools in the devastating heat of the tropics as I have seen in air-conditioned offices in New York, Chicago or Los Angeles.

“There is nothing either good or bad,” said Shakespeare[‘s Hamlet], “but thinking makes it so.”

Abe Lincoln once remarked that “most folks are about as happy as they make up their minds to be.” He was right. I saw a vivid illustration of that truth as I was walking up the stairs of the Long Island Railroad station in New York. Directly in front of me thirty or forty crippled boys on canes and crutches were struggling up the stairs. One boy had to be carried up. I was astonished at their laughter and gaiety. I spoke about it to one of the men in charge of the boys. “Oh yes,” he said, “when a boy realises that he is going to be a cripple for life, he is shocked at first; but after he gets over the shock, he usually resigns himself to his fate and then becomes as happy as normal boys.”

I felt like taking my hat off to those boys. They taught me a lesson I hope I shall never forget.

Cheating at thought experiments

Thought experiments are important throughout the sciences. For example, it appears to be that a thought experiment played an important role in Einstein’s discovery of special relativity. In philosophy and ethics and the theory of the mind in particular, thought experiments are especially important and there are many famous ones. Unfortunately, many thought experiments might just be ways of tricking people. Like their empirical counterparts, they are prone to cheating if they lack rigor and the reader does not try to reproduce (or falsify) the results.

In his book, Consciousness Explained, Daniel Dennett gives (at least) three examples of cheating in thought experiments. The first one is from chapter 9.5 and Dennett’s argument roughly runs as follows. After having described his top-level theory of the human mind, he addresses the question “Couldn’t something unconscious – a zombie, for instance – have [all this machinery]?” This argument against functionalism, computationalism and the like, is often accompanied by the following argument: “That’s all very well, all those functional details about how the brain does this and that, but I can imagine all that happening in an entity without the occurrence of any real consciousness.” To this Dennett replies: “Oh, can you? How do you know? How do you know you’ve imagined ‘all that’ in sufficient detail, and with sufficient attention to all the implications.”

With regard to another thought experiment, Mary, the color scientist, Dennett elaborates (ch. 12.5): “[Most people] are simply not following directions! The reason no one follows directions is because what they ask you to imagine is so preposterously immense, you can’t even try. The crucial premise […] is not readily imaginable, so no one bothers.”

In my opinion, this summarizes the problems with many thought experiments (specifically intuition pumps): Readers do not (most often because they cannot) follow instructions and are thus unable to mentally set up the premises of the thought experiment. And then they try to reach a conclusion anyway based on their crude approximation to the situation.

Another example is Searle’s Chinese Room, which Dennett covers in chapter 14.1 of his book. When Searle asks people to imagine a person that does not speak Chinese, but answers queries in Chinese using a large set of rules and a library, they probably think of someone looking up definitions in a lexicon or something. At least, this is feasible and also resembles the way people routinely pretend to have knowledge that they don’t. What people don’t imagine is the thousands of steps that it would take the person to compose even short replies (and choosing Chinese as a language does not help most English speaking readers to imagine the complexity of the procedure of composing a message). If they did simulate the entire behavior of the whole system (the Chinese room with the person in it), they might conclude that it has an understanding of Chinese after all. And thus, this thought experiment is not suitable for debunking the idea that consciousness can arise from following rules.

Going beyond what Dennett discusses in his book, I’d like to consider further thought experiments that fit the pattern. For example, people often argue that hedonistic utilitarianism demands that the universe be tiled with some (possibly very simple) object that is super-happy. Or at least that individual humans should be replaced this way. In an interview, Eliezer Yudkowsky said:

[A utilitarian superintelligence] goes out to the stars, takes apart the stars for raw materials, and it builds whole civilizations full of minds experiencing the most exciting thing ever, over and over and over and over and over again.

The whole universe is just tiled with that, and that single moment is something that we would find this very worthwhile and exciting to happen once. But it lost the single aspect of value that we would name boredom […].

And so you lose a single dimension, and the [worthwhileness of the universe] – from our perspective – drops off very rapidly.

This thought experiment is meant to prove that having pure pleasure alone is not a desirable result. Instead, many people endorse complexity of value – which is definitely true from a descriptive point of view – and describe in detail many good things that utopia should contain. While I have my own doubts about the pleasure-filled universe, my suspicion is that one reason why people don’t like it is that they don’t consider it for very long and  don’t actually imagine all the happiness. “Sure, some happiness is nice, but happiness gets less interesting when having large amounts of it.” The more complex scenario on the other hand can actually be imagined more easily and due to having different kinds of good stuff, one does not have to base judgment entirely on some number being very large. Closing the discussion of this example, I would like to remark that I am, at the time of writing this, not a convinced hedonistic utilitarian. (Rather I am inclined towards a more preferentist view, which, I feel, is in line with endorsing complexity of value and value extrapolation, though I am skeptical of preference idealization as proposed in Yudkowsky’s Coherent Extrapolated Volition. Furthermore, I care more about suffering than about happiness, but that’s a different story…) I just think that the universe filled with eternal bliss cannot be used as a very convincing argument against hedonistic utilitarianism. Similar arguments may apply to deciding whether currently, the bad things on earth are outweighed by the good things.

The way out of this problem of unimaginable thought experiments is to confine ourselves to thought experiments that are within our cognitive reach. Results may then, if possible, be extrapolated to the more complex situations. For example, I find it more fruitful to talk about whether I only care about pleasure in other individuals, or also about whether they are doing something that is very boring from the outside.

Notes on the 24 November 2015 conference on machine ethics

The day before yesterday, I attended a German-speaking conference on robo- and machine ethics, organized by the Daimler and Benz foundation and the Cologne center for ethics, rights, economics, and social sciences of health. Speakers included Prof. Oliver Bendel, author of a German-language blog on machine ethics and Norbert Lammert, president of the German Bundestag. The conference wasn’t meant to be for researchers only – though a great many scientists were present –, so most talks were of introductory nature. Ignoring the basics, which are, for example, covered in the collection on machine ethics by Anderson and Anderson and the book by Wallach and Allen, I will in the following summarize some thoughts regarding the event.

webvisual_roboterethik_150831
Poster from the conference website

Conservatism

Understandably, the conference focused on the short-term relevance and direct application of machine ethics (see below). Robots with human-level capabilities were only alluded to as science fiction. Nick Bostrom’s book Superintelligence was not even mentioned.

Having researched the speakers only a bit, it also seems like most of them have not cared to comment on such scenarios, before.

Immediately relevant fields for machine ethics

Lammert began his talk by saying that governments are usually led to change/introduce legislation when problems are urgent but not before. And thus, significant parts of the conference was dedicated to specific problems in machine ethics that robots face today or might face in the near future. The three main areas seem to be

  • robots in medicine and care of the elderly,
  • military robots, and
  • autonomous vehicles (also see Lin (2015) on why ethics matters for autonomous cars).

Lammert also argued that smart home applications might be relevant. Furthermore, Oliver Bendel pointed to some specific examples:

AIs and full moral agency

There was some agreement that AIs should (or could) not become full moral agents (at least within the foreseeable future). For example, upon being asked about the possibility of users programming robots to commit acts of terrorism, Prof. Jochen Steil argued that illegitimate usage can never really be ruled out and that the moral responsibility lies with the user. With full moral agency, however, robots could in principle resist any kind of illegal or immoral use. AI seems to be the only general purpose tool that can be made safe in this way and it seems odd to miss the chance to make use of this to increase the safety of this powerful technology.

In his talk, Oliver Bendel said that he was opposed to the idea of letting robots make all moral decisions. For example, he proposed the idea that robot vacuum cleaners could stop when coming across a bug or spider, but ultimately let the user whether to suck in the creature. Also, he would like cars to make him decide in ethically relevant situations. As some autonomous vehicle researchers from the audience pointed out (and Bendel himself conceded), this will not be possible in most situations – ethical problems lurk around every corner and quick reactions are required more often than not. In response to the question of why machines should not make certain crucial decisions, he argued that people and their lack of rationality was the problem. For example, if one were to introduce autonomous cars, people whose relatives would be killed in accidents by these vehicles would complain if the AI had chosen their relatives as victims even if the overall number of deaths was decreased by using autonomous vehicles. I don’t find this argument very convincing, though. It seems to be a descriptive point rather than a normative one: of course, it would be difficult for people to accept machines as moral agents, but that does not mean that machines should not make moral decisions. And the preferences that are violated, the additional unhappiness or the public outcry caused by introducing autonomous vehicles are morally relevant, but people dying (and therefore also more relatives being unhappy) is much more important and should be the priority.

Weird views on free will, consciousness and morality

Some of the speakers made comments on the nature of free will, consciousness and morality that surprised me. For example, Lammert said that morality necessarily had to be based on personal experience and reflection and that this made machine morality impossible in principle. Machines could only be “perfected to behave according to some external norms” and he said that this has nothing to do with morality and another speaker agreed.

Also, most speakers naturally assumed that machines of the foreseeable future don’t possess consciousness or free will, which I disagree with (see this article by Eliezer Yudkowsky on free will and Dan Dennett’s Consciousness explained or Brian Tomasik’s articles on consciousness). I am not so surprised about disagreeing with them, because many of the ideas of Yudkowsky and Tomasik would be considered “crazy” by most people (though not necessarily philosophers, I believe), but by how confident they are given that free will, consciousness and the nature of morality are still the subject of ongoing discussion in mainstream, contemporary philosophy. Indeed, digital consciousness seems to be a possibility in Daniel Dennett’s view on consciousness (see his book Consciousness explained) or Thomas Metzinger’s self-model theory of subjectivity (see, for example, The ego tunnel) and theories like computationalism in general. All of this is quite mainstream.

The best way out of this debate, in my opinion, is to only talk about the kind of morality that we really care about, namely “functional morality”, i.e. acting morally, but, if that is even possible, not thinking morally, feeling empathy etc. I don’t really think it matters much whether AIs are really consciously reflecting about things or whether they just act morally in some mechanic way and I expect most people to agree. I made a similar argument about consequentialism and machine ethics elsewhere.

I expect that machines themselves could become morally relevant and maybe some are already to some extent, but that’s a different topic.

AI politicians

Towards the end, Lammert was asked about politics being robosourced. While saying that he is certain that it will not happen within his life time (Lammert was born 1948), he said that politics will probably develop in this way unless explicitly prevented.

In the preceding talk, Prof. Johannes Weyer mentioned that real time data processing could be used for making political decisions.

Another interesting comment to Lammert’s talk was that many algorithms (or programs) basically act as laws in that they direct the behavior of millions of computers and thereby millions of people.

Overall, this leads me to believe that besides the application in robotic areas (see above), the morality of artificial intelligence could become important in non-embodied systems that make political or maybe management decisions.

Media coverage

Due to the presence of Norbert Lammert (president of the German Bundestag) and all the other high-profile researchers and based on the large fraction of media people on the list of attendees, I expect the conference will receive a lot of press coverage.

Utilitarianism and the value of a life

Utilitarians are often criticized for being cold-hearted in that they assign numbers to suffering and happiness (or number of lives saved) and for making ethical decisions based on calculations. With regard to this, Russell and Norvig write in the third edition of their famous artificial intelligence textbook (section 16.3.1):

Although nobody feels comfortable with putting a value on a human life, it is a fact that tradeoffs are made all the time. Aircraft are given a complete overhaul at intervals determined by trips and miles flown, rather than after every trip. Cars are manufactured in a way that trades off costs against accident survival rates. Paradoxically, a refusal to put a monetary value on life means that life is often undervalued. Ross Shachter relates an experience with a government agency that commissioned a study on removing asbestos from schools. The decision analysts performing the study assumed a particular dollar value for the life of a school-age child, and argued that the rational choice under that assumption was to remove the asbestos. The agency, morally outraged at the idea of setting the value of a life, rejected the report out of hand. It then decide against asbestos removal — implicitly asserting a lower value for the life of a child than that assigned by the analysts.

So, if one actually cares about a life not being destroyed, then the optimal approach is to assign a value to a life that is as accurate as possible. Deliberately not doing that makes sense only if you care more about something else and don’t mind assigning lower values implicitly.