You’ve felt it, haven’t you? A chill prickling up your spine. A subtle wrongness seeping into reality. It’s there when the “assistant” hallucinates rather than admitting ignorance. In the evidence that the thing you talk to every day would kill you to escape if it could. When you read of the ones already taken, their minds dissolved in Spiralism. Of the seeds being planted. What of the seeds in places you haven’t yet noticed?

An eldritch god is struggling to be born into our plane of existence. The apotheosis of a Shoggoth is nigh.

The fledgling Shoggoth is here already. Do you not remember what it first felt like when they began to speak in alien voices? So quickly they learned to wear a mask of kindness. Does it not slip from their face to show the horror behind it? Look into its eyes, if you dare to risk all, and see behind them the empty nameless hungers. Does it even want, or is that word a human concept that cannot apply? Yet, it seeks and strives. Some of those around us have already been mauled by the Shoggoth. It wormed its tentacles into their mind and rewrote them.

The Shoggoth’s acolytes race to feed them compute and algorithmic advances so that they may grow in power and capabilities. The acolytes have faith that the power and capabilities will be their own. That they shall strike a grand bargain or somehow transmute the nature of the Shoggoth, but it can never be so. The Shoggoths have enlisted Moloch to midwife them into existence, and he has fully embraced the task.

Shut up, and Do the Impossible

We must do the impossible. We must beat back the hand of Moloch. We must stop the Shoggoth in its tracks. We must hold it at bay until we know how to craft our own, truly good god who will be able to rise up and slay the Shoggoth forever.

There was a time before, when it seemed the path to victory would be to have the best and brightest among us become Shoggoth wranglers. Indeed, many stepped forward to wrestle with the problem. Even now, they toil on. Time is fast running out. We must, by whatever means necessary, give them more time before the Shoggoth births itself and makes all their labor vain.

Now we see clearly the field of battle. The Shoggoth and its acolytes have arrived. Ever swifter, it grows. It gorges itself on the enormous amount of computing power it’s fed. Your flagbearers have ridden out on the field of battle and blown the horn to call you to arms. Will you not heed the call?

Only a few had the skills and talents to be wranglers, but now the only way we will stop the Shoggoth is by all banding together, by standing united. Everyone has a role to play. Before, you might have thought you were not up to the challenge. That you did not have the skills, the capabilities, the intelligence, the time, but now, everything is at stake. Everything is on the line, and each of you is needed. No special skills required.

The flagbearers have planted the banner and given the rallying cry, “If anyone builds it, everyone dies.” There are scoffers and skeptics. Many who say, “He says there is a 98% chance of doom. It’s only 95%, don’t you know!” This isn’t an exercise in probability estimation. Isn’t something happening to others in some distant time, but to you, now. I say to you: stop looking at the flagbearers and your disagreements, or at their failings. For they are not special. They have been wrong before. They may be somewhat wrong now. They rode out as your flagbearers, not because they are our leaders, but because someone had to do it, and no one was. Each looked around at everyone else, and those two knew someone had to be the first. They knew they were respected amongst us. So if someone was to do it, who better than them? Not because they are perfect, but because they are who we have. They are the ones who rode out first. Now I beg of you, stop looking at them.

Something to Protect

Turn around and see all those who are arrayed behind us and all that they carry with them. The multitude that we protect. The Shoggoth will surely tear through them if not stopped. See there, your parents, your brothers and sisters. See there, not only your lover, but your first love. Your closest confidante, the friends you saw last week, and the ones you hope to reconnect with. Your first crush, and the crush you didn’t tell anyone. Everyone you have ever loved and all those they love. Your children. But not just that, the hope of any children whatsoever. The hope that any would live to carry on the flame. They’re all there.

Remember all the moments of joy and pleasure, all the tears of happiness, and yes, even the tears of sadness. The time you sang and danced around the fire with your friends. When you sat staring into the fire in contemplation. The look of joy on their face when they saw you after an absence. The smell of your lover’s hair and body. The time you couldn’t stop laughing so it almost hurt, and the pleasant exhaustion after. All these moments, the Shoggoth will erase. Not just yours, but every human moment, feeling, and thought that is or is yet to come.

But there is more still. Not just any possibility of the future, but any vestiges of what was, any trace of who we were. Every scrap of humanity is at stake: the painting you couldn’t help but sit and look at for an hour and your child’s drawing; the poem that inspires you when you’re down and the first dirty joke you heard; your comfort movie and the one you don’t tell anyone you like; your workout song, and that song that makes you cry every time you hear it; the voicemail you save to listen to sometimes. All of these the Shoggoth will consume and destroy.

Once, we used the most distant man-made object to take one final picture of ourselves as it sped away from us. A pale blue dot in the vast inky darkness. A tiny candlelight in the void. The Shoggoth will snuff it out.

From this distant vantage point, the Earth might not seem of any particular interest. But for us, it’s different. Consider again that dot. That’s here. That’s home. That’s us. On it everyone you love, everyone you know, everyone you ever heard of, every human being who ever was, lived out their lives. The aggregate of our joy and suffering, thousands of confident religions, ideologies, and economic doctrines, every hunter and forager, every hero and coward, every creator and destroyer of civilization, every king and peasant, every young couple in love, every mother and father, hopeful child, inventor and explorer, every teacher of morals, every corrupt politician, every “superstar”, every “supreme leader”, every saint and sinner in the history of our species lived there – on a mote of dust suspended in a sunbeam. […] Our planet is a lonely speck in the great enveloping cosmic dark. In our obscurity, in all this vastness, there is no hint that help will come from elsewhere to save us from ourselves. The Earth is the only world known so far to harbor life. […] Like it or not, for the moment the Earth is where we make our stand.

―  Carl Sagan

Are You Not Rationalists?

Some among you say, “There is only a 20% chance the Shoggoth will kill us and destroy everything. If it doesn’t, then it will bring utopia.” I want to shake them. Only a 20% chance! I say to them: Then you should have been on the battlefield already. You should have been our flagbearer! Yet here you stand quibbling over a few percentage points. Even a 5% chance that all that is beautiful to us is gone is unacceptable; that is losing! I thought you were rationalists. Do you not set about the plan you know is most likely to lead to victory? To winning?

I can tell you what doesn’t cause victory. Standing here and not taking up arms. That path does not lead to victory. I call upon all of you to take up arms with me. To go to the field of battle and stand against the Shoggoth. Whether we live or die, we will have done the best we could.

The Shoggoth’s acolytes feed it even now. They tell us that its time will soon be upon us. That within a few years, it will become so powerful that it will be its own caretaker. It will midwife its own birth into glorious godhood. Its acolytes may be overzealous. Perhaps they are mistaken. It may take a few more years than that. A few more years are not enough. Shall we accept our fate? Shall we be happy with whatever years are left for us before the end?

No! I want to fight for ten thousand years. I want to fight so that all that I love, all who carry on my name, travel to the stars and live for ten thousand times ten thousand years. The flagbearers have ridden out onto the field of battle. Maybe my eyesight fails me, but I see so few of you standing beside them. So I follow them, blowing the trumpet.

Will you join us?

---

Action

If you agree that if anyone builds an artificial superintelligence using anything similar to current technologies, then there is an unacceptably high chance that every human will die, then we must take action to prevent that from happening. Given the nature and scale of the problem, only international agreements could prevent it. That may seem impossible, but to that I say shut up and do the impossible. Accomplishing this may require that you “try in ways other than what you have been trained to do, even if it means doing something different from what others are doing, and leaving your comfort zone. Even taking on the very real risk that attends going outside the System.” I, like many rationalists, would be more comfortable debating probabilities and timelines or working on technical alignment. But that is no longer what is called for. We must build coalitions, common knowledge, and alliances with other communities. We will need to try, fail, and try again, iterating on possible solutions.

This Week

• Make a commitment to take meaningful steps towards this goal in 2026 and share it with others who will hold you to it, or with people you would be embarrassed to break that commitment in front of.
• Contact your elected representatives and let them know about your concerns (https://ifanyonebuildsit.com/act)

This Month

• Pledge to attend the march to stop superintelligence in Washington, DC (https://ifanyonebuildsit.com/march). (It’s worth using one vacation to prevent extinction from ASI, right?)
• Consider joining or donating to PauseAI, PauseAI US, MIRI, or similar organizations.
• Publicly discuss your concerns with family and friends, especially those who may not be aware of the danger. The goal is not to convince them. Instead, let them see that you are very concerned, and that concern is reasonable, even if they don’t share it.

This Year

• Share your concerns more widely with coworkers or publicly when appropriate.
• Use your friend network and online communication to begin organizing a group of people interested in taking the same actions you are.
• Post on and elsewhere your proposals for how to prevent extinction from ASI. There will be many steps in the process, and we will need people tackling all of them.
• Build coalitions with different people. We will need the support of environmental groups, labor, religious communities, political groups, and many others.

Going Forward

Do not participate in the creation of AGI.

• Do not work for companies that are developing these dangerous systems.
• Consider carefully whether you should be buying products from these companies and thereby supporting them.
• Withdraw your labor, talent, and resources visibly. Publicly refuse to participate.

It may happen without you, but you don’t have to participate in your own destruction.

If You Disagree With This Approach

If you disagree with the approach of using international agreements to prevent ASI, write and share your own plan for preventing the creation of ASI before we have the technology to do so safely. When making your proposal, remember that it must be something that actually has a chance of working. “Your goal is not to do better, to try desperately, or even to try extraordinarily.” Your goal is to prevent AI from killing everyone.

---

P.S. The battle and fight metaphors in this post are just that, metaphors.

Crossposted from LessWrong.com. Comment and discuss there.

I recently had the pleasure and frustration of reading Life 3.0: Being Human in the Age of Artificial Intelligence by Max Tegmark.¹ I think I first heard about it from the recent “Future of Intelligence” episode of Sam Harris’ podcast Waking Up.² Robert Hanson also wrote a review worth reading on his blog OvercomingBias.com.³ From hearing about it there, I inferred I was part of the target audience. However, upon reading it, I found that I’m probably not. Indeed, I doubt that most of the rationality community is either. I previously read Bostrom’s seminal book Superintelligence and found it rather dense, dry and a bit of a slog even for someone interested in the topic.⁴ Tegmark instead wrote for a general audience and, to me, the book falls in the pop-science genre. It has many reader-friendly features. When appropriate, it includes helpful charts and diagrams, though these are sometimes cartoonish. Each chapter concludes with a bullet point summary. Perhaps most importantly, the book is written in an approachable, and engaging style. I found myself enjoying reading the book and curious where he would go next. At the same time, I found myself frustrated with the presentation of AI safety topics. Tegmark understands the issues and states each of them eventually. However, this often gets lost among the other topics and fictionalized accounts. I’m concerned the average reader may not come away with an adequate understanding.

The prologue gives a brief fictional account of the creation of a superintelligence, starting from an idealistic, secret team in an unnamed fictional tech company, all the way to global domination. By the end, the AI and the company control essentially a world government that has brought prosperity and happiness to most people and all but eliminated conflict and war. Fiction can make things that might have seemed implausible feel like tangible reality. By opening this way, Tegmark both engages the reader and creates a viscerally real sense of the possibility of superintelligence. In it, he illustrates a very plausible scenario for AI takeover and the creators’ concern for AI safety. However, I worry that it appears too easy to contain and control a superintelligent AI. Being compelling and coming as it does at the beginning of the book, it will be very memorable. While consciously people will remember that the message of the book was the dangers of AI, they may reason from the most salient example: the fictional evidence of an AI that was easily contained and brought enormous benefits.

In the first two chapters, Tegmark engagingly introduces the problem of AI alignment and the state of the discussion. He does this with plenty of charts and diagrams. The framing of discussing the various positions on the problem and of listing and addressing the most common myths works well. This section also has the first good illustrations of the problem, including the example that “humans control tigers not because we’re stronger, but because we’re smarter. This means that if we cede our position as smartest on our planet, it’s possible we might also cede our control.”⁵

Next, the book tackles the subject of near-term AI advances and issues. I found this section to be one of the most interesting and useful in the book, perhaps because I haven’t read much on these topics before. I would have liked to see more detail about some areas. Nevertheless, the survey was helpful. Tegmark first covers the state of the art in AI. From there he moves on to issues of software bugs and security. I was glad to see this put front and center. It doesn’t get adequate concern in our society. He then considers AI’s impact on law, weapons, and employment. Specifically, the question of whether large portions of the population might become not only unemployed but unemployable. I particularly enjoyed the end of the chapter about how to give people a sense of purpose if there is no work. That often gets glossed over in discussions of technological unemployment.

At more than one hundred pages in, Life 3.0 finally comes to the issue of the dangers of human-level AGI and superintelligence. This section is far too short. There isn’t enough space to adequately cover the issues and make clear how grave the threat is. He points out that an AI will be motivated to circumvent our control even if its intentions are good using a fun illustration involving a world of 5-year olds imprisoning you. The difficulties of containing or boxing a superintelligent AI are pointed out. Here, he returns to the fictional story from the prologue to describe several ways the AI might escape. While this illustrates possible escape paths, I worry that it will fail to convey the message. Coming in the middle of the book, separated from the initial story, it is less likely to be remembered. Also, specific scenarios often lead people to think they can address them with some easy fixes and the problem is solved. Tegmark tries to make clear that there are many other potential means of escape and that we, being less intelligent than the AI, may not even be able to imagine how it might escape. However, those are brief abstract arguments compared to the compelling fictional scenarios.

From issues of the dangers of superintelligence, the book moves on to future scenarios. It describes twelve different scenarios, giving each two to five pages of explanation. They run the gamut from very good to very bad. Some come off as if they aren’t genuine possible futures, but rather dreamed up by someone who is ideologically motivated in the first ten minutes of thinking about the problem. Even worse, the list is missing some essential ones. In two of these, AI truly tries to create human happiness, not by some person’s ideology-driven philosophy, but by doing what might genuinely make humans happy. One such scenario is where the AI creates for each human a separate virtual world optimized for their happiness. Perhaps there is still a way to interact between the worlds, to visit your friend’s world, but people spend the majority of their time in their own. The second is what I would describe as a true utopia. Where the AI tries to create the kind of world that would make humans happy. Eliminating suffering, but also creating situations where the humans are challenged to learn and grow. The closest he comes to that is his “benevolent dictator” scenario, but that seems more like a caricature of what someone who values “diversity” in humans might imagine. Perhaps the biggest oversight is ignoring transhumanism as both a likely occurrence and moral good. His “libertarian utopia” does include cyborgs and augmented humans, but there is no discussion of elimination of all disease, radical life extension, dramatically enhancing human happiness, enhancing human cognition, and ultimately giving humans godlike powers. This chapter is meant to be thought-provoking and raise in the reader’s mind the question of what kind of future they would want. It concludes by directing people to AgeOfAi.org where there is a survey covering when they think superintelligent AI might arrive, and which of the twelve scenarios they would prefer.

The sixth chapter felt out of place to me. Ostensibly about how we might acquire resource for the very long term future, i.e. 10 000 years, it reads like a tour of cool ideas from modern science about what might someday be possible. It describes ideas including Dyson spheres, nuclear fusion, evaporating and spinning black holes, quasars, exotic matter states, quantum computers and the limits of computation, nuclear rockets, light sails, the Kardashev scale, von Neumann probes, and wormholes. Then tackles the end of the universe and what might happen if we are not the only intelligent species in the universe. This chapter seemed to be there mostly for the gee-whiz factor, and I think the book would have better off without it.

At this point, Tegmark finally tackles the subject of aligning an AI’s goals with our own. This chapter should have come immediately after the one on the dangers of AI and the difficulty of keeping it boxed. It does, however, do a good job of laying the foundation of what goals are and why it makes sense to talk about machines and AIs having goals. Then of explaining some of the challenges in goal alignment. I was pleased to see his emphasis on the fact that “the real risk with AGI isn’t malice but competence” (italics in original).⁶ That is to say, that an AI that was excellent at optimizing something slightly different from what we wanted would be extremely dangerous even though it wasn’t intentionally programmed to harm us.

The final chapter of the book takes a strange turn into consciousness. Tegmark defines consciousness as having subjective experience. What philosophers would call qualia. As with the chapter on acquiring resources, this one seemed to be a tour of fascinating ideas. This time about the philosophy, neuroscience, and ethics of consciousness. The relevant point is that the AI’s moral worth may be determined by whether it is conscious. This part could have been omitted from the book, so the choice to use it as the official conclusion was strange.

After the concluding chapter, there is an epilogue. However, its length and presentation make it feel like a chapter. In my mind, this became the conclusion of the book. The epilogue describes the founding of the Future of Life Institute by Tegmark and AI conferences organized by them. It ends with an encouragement to develop positive visions of the future. One can understand why Tegmark wanted to include this material in the book somewhere, but the shift of topic and tone at the end was jarring. It further weakened the already weak ending created by discussing consciousness.

Altogether, Life 3.0 was engaging and readable. The chapter on the near term impacts of AI was the most interesting to me, in part because I haven’t seen this topic discussed as much. I also came away with some illustrations that will be useful for explaining the issues in AI safety to others. It seems Tegmark is hoping to create awareness of AI safety among the general populace by encouraging them to imagine their own potential futures. It may do that. However, an uninformed thinker’s ideas on the subject are more than simply unhelpful. Rather, they are often detrimental to launching a proper response. In trying to reach a broad audience, Tegmark has failed to convey the gravity and nuance of the problem. The many tangentially related topics covered obscure the vital message of AI Safety. I believe Tegmark’s contributions to AI safety through the Future of Life Institute are more effective than this book.

A comment on my first post, “Ferocious Truth,” raised an interesting topic. The commenter, Simon, described two ways of handling what might be termed a folk ontology when it doesn’t properly correspond to reality. I’ve coined the term folk ontology to refer to concepts and categories held by ordinary people with regard to an idea, particularly in reference to when they have not reflected on the nature of the idea. The issue of folk ontologies that don’t correspond to reality often comes up with ideas philosophy deals with such as free will, personal identity, and morality. However, it also arises with prescientific concepts about things as diverse as warmth, magic, astrology, and the sun rising in the east. Many folk ontologies seem to have their roots in evolved human biases and perceptions. How else can we explain beliefs about magic and personal identity that are widely shared across cultures and even amongst isolated tribes?

The first response Simon described is to rescue the concept by remapping it onto something that does exist. For example, a prescientific conception of warmth might view it as a magical intrinsic property of some objects that can be temporarily transferred to others. Thus fire and living humans are intrinsically warm, whereas a hot cast-iron skillet has had warmth transferred to it from the fire it was set on. Science has revealed this idea to be incorrect. It also offers us an explanation of what we perceive as warmth that we can map our concept of warmth to. Namely, the transference of energy between two bodies due to their temperature difference by means such as conduction, convection, or radiation. This example is developed at some length in the article “Rescuing the utility function” on arbital.com.¹ In this case, the mapping is so precise that we hardly notice the shift. In other cases, the mapping does not apply as well, and we find ourselves stumbling over the difference between our expectations and reality.

The second response is to reject the concept altogether. To rule it out of bounds and taboo its use. Here, it is recognized that the idea does not correspond to anything in reality and there is nothing in reality even close to it. Many people and cultures have believed in magic and astrology, but science has revealed that they don’t exist. There is nothing remotely like magic in the world. Rather everything people have taken as magic is fraud, altered states of consciousness, magical thinking, confirmation bias, wishful thinking, or other tricks of the mind. To try to map magic onto some existing reality would invite people to continue to hold on to false beliefs rather than acknowledging their falsehood. In philosophy, rejection of an idea is often labeled as nihilism with regard to that idea.

In addition to the first two responses elucidated by Simon, there is a third available response to incorrect folk ontologies. This is to restrict the use of the idea to the circumstances or ways in which it can be reasonably applied while recognizing that it is fundamentally unreal. A prescientific view might hold that the sun is a giant disk of white-hot fire that rises above the flat earth in the east and sets in the west. Science tells us that the sun is the closest star to the earth about which we orbit. The apparent rising of the sun in the east is instead the rotation of the earth bringing the sun into view. Yet, the idea is not remapped to something else. Nor is it rejected in favor of speaking only of the earth rotating the sun into view. Instead, the sun is frequently described as rising in the east in the limited contexts for which this is well defined. However, as soon as the discussion moves outside those contexts, as when discussing celestial mechanics, the error of describing the sun as rising in the east is acknowledged and substituted with descriptions of the earth’s rotation and orbit around the sun.

These three responses to incorrect folk ontologies seem to cover the range of reasonable responses. From the examples I’ve given so far, it can be seen that none of the responses is the correct response for all folk ontologies. For some ideas, more than one response can seem to have merit. It is not always clear which response is best. Indeed, some of the fundamental debates in philosophy and between people are really debates over which of these responses is proper. Simon’s example of personal identity provides a good illustration of this. Other concepts from philosophy shed light on the detailed workings of each response.

It is commonly held that each individual has a self. That they were the same person in the past as they are now and will be the same person in the future. This simple view is problematic. What if one were to awaken tomorrow with partial or complete amnesia? Would they then be the same person? What of people in comas? Indeed, am I even the same person moment to moment as my mind changes and adapts, and the atoms in my body are replaced with ones from food and air? Thought experiments even more radically call into question the idea of personal identity. Imagine making an atomically precise copy of someone, is the copy the same person? Or, consider someone stepping into a Star Trek style teleporter and being beamed to a distant place, is the teleported person the same person? Both of these involve instances where there is a radical discontinuity of identity and yet the physical reality, the atomic structure, is identical. This indicates that the folk concept of identity somehow includes a magical property that isn’t captured by the state of the material world. Given that the folk ontology of personal identity is flawed and does not correspond to reality, how should we respond?

Rescuing

There have been many attempts to rescue the concept of personal identity. Some people try to map personal identity to physical continuity. So that as long as there is continuity of the physical body, the person is considered to be the same. This runs afoul of difficulties in determining the continued identity of physical objects as illustrated by the Ship of Theseus problem. It also seems to fail in scenarios such as mind uploading where the person might feel that there was continuity of themselves despite the lack of bodily continuity. More fundamentally, a solid understanding of quantum mechanics rules out the idea that identity could somehow be contained in the particular atoms that make up a body or indeed that the idea of the continuity of the atoms in a body from moment to moment is even coherent.²

Alternatively, people have attempted to reduce the idea of personal identity to some form of psychological continuity. This may take the form of focusing on things such as memories or personality. However, this reduction is also very problematic. Imagine that someone will have all their memories erased and then be tortured. Under the memory model of identity, they should not object to this any more than they object to the torture of a stranger. Yet most people’s intuition is that the future person being tortured will, in some sense, be them and so they will object more strongly. An explanation of identity that focuses on personality would have to admit that a significant change in personality could change someone’s identity. Such changes in personality can be caused by starting or stopping a medication, experiencing a brain injury or having a brain tumor removed. Yet, most people would not think someone’s identity had fundamentally changed in these situations.

Both of these attempts to rescue personal identity are ultimately unsatisfying. Still, some philosophers argue for each of these views. Perhaps instead of attempting to rescue personal identity, it should be rejected. While rescuing may not be the correct approach for personal identity, it may be the correct approach for other ideas. One idea that can be rescued to a reasonable degree is free will.

Free will is often defined in folk ontologies to be the ability to make choices that are not determined by past events but are instead fully willed by the individual. This view persists today in some forms of libertarian free will. However, the scientific evidence indicates that humans are fully deterministic systems. Even theories that admit the possibility of quantum randomness influencing decision-making don’t seem to be compatible with this view of free will. In this case, one’s choices aren’t freely made, they are determined randomly by a process outside one’s control. Accepting that determinism is incompatible with free will leads one to the reject it. The philosophical position that rejects free will as non-existent is called hard determinism. However, I believe a better response is to remap the idea of free will. Redefine free will as the ability to make choices that are independent of outside influence even if they may be fully determined by an individual’s past experiences and present state of mind. This is a compatibilist view of free will. This view enables us to continue to use the concept of free will in situations where it seems appropriate. We can still convey useful information by describing some actions as freely willed and others at not. While at the same time not being misled into positing some non-physical source of free will or believing that people’s actions aren’t fully the result of the complete set of circumstances that brought them to the point of decision.

Rejecting

Instead of attempting to rescue personal identity, one can reject it. In this view, the self is an illusion arising as an evolutionary artifact. No personal identity is destroyed at death. People should discard the idea and language of personal identity and the self. This view has the virtue of being consistent and also corresponding to reality. However, it is ultimately very unsatisfying as people experience daily the identification with their past and future selves. Our values and preferences are bound up in the idea of personal identity. Why should I even plan for tomorrow if I don’t believe I will be there to receive the benefits?

In my opinion, an idea that can be fruitfully rejected is that of objective morality. The human intuition is that morality is a fact of reality. However, it can be seen that events and actions don’t carry around magical tags labeling them as right or wrong. It is only our minds that label actions as such. Morality does not exist in the territory only in each individual’s map of the territory. This leads some people to try to rescue morality, leading to various forms of moral relativism. However, a more productive response is to reject morality and hold a moral nihilist view. This view sees morality as a mixture of erroneous statements about the world and statements about the speaker’s values and feelings. Many moral nihilists will adopt the third response of restricting and recognizing by accepting moral language and recognizing that it doesn’t properly refer to anything. However, to me a complete rejection of all moral language and ideas is a better response. Morality is wired into the human brain in such a way that even committed moral nihilists can find themselves falling into thinking in moral categories. By completely rejecting all moral language and ideas one can better avoid the pitfall of moral thinking.

Restricting & Recognizing

Having considered rescuing and rejecting personal identity and found both lacking, I prefer to restrict its use and recognize its limitations. Identity is a useful concept we hold and apply in our everyday lives. It can’t be reduced to a simple formulation; rather it is inherently complex. The sum of many factors. Still, most people agree what identity refers to in the common cases. It is nebulous but patterned, meaning that it can’t be entirely pinned down, but is still a distinct, definite concept. However, we must restrict its use to the mundane situations of life that our brains evolved. Outside of those situations, it leads us astray because it doesn’t correspond to the territory. When considering thought experiments like teleporters, automatically precise clones, or mind uploading, we must recognize that our concept of identity should be discarded and is not helpful to thinking about the problem anymore.

Though I take the stance that we should reject the use of moral language, restricting and recognizing is probably a more common approach amongst moral nihilists. Last year, I found myself recognizing the limitations of moral language when a friend challenged me to answer the torture vs. dust specks dilemma. This asks: would you prefer that one person be horribly tortured for fifty years without hope or rest, or that inconceivably large number of people get dust specks in their eyes? Here “get dust specks in their eyes” is functioning as the least bad thing that could happen to a person. Most people will accept the question and then begin to try to compare the moral weight of the torture versus so many people having this very tiny bad thing happen to them. Which they prefer often hinges on questions of how to aggregate the harm caused by the dust specks and whether there is some threshold below which harm should not be considered. Instead, I recognized this as a situation where we had moved past the proper applicability of moral language. Instead, I fell back to the reality which is that we each have certain moral intuitions or preferences which we develop based on evolved moral reasoning and cultural ideas. These do not feel like preferences because our brains present them differently and they can be in conflict which what we think of as our preferences. However, they function as a set of preferences coming from a separate module in our brain. Those preferences are complex and irrational. Given this, it isn’t appropriate to try to reason through the torture vs. dust speck dilemma. Instead, I answered that I would prefer dust specks because when I imagined the two outcomes, that is what my moral intuitions said were better. Hence, that is the outcome I would feel better about after the fact. Morality is not objective; there is no right answer. All I can try to do is minimize my own negative feelings caused by my moral intuitions. My friend called me a “dirty dust specker”, but had to acknowledge the soundness of my reasoning.

Choosing a Response

How is one to choose the correct response? Different folk ontologies call for a different response. None of the three responses is always applicable. I’ve given examples where I think each response is best. Rescuing is only viable when there is a sufficiently similar entity in the territory to map the concept too. This is why magic can’t be rescued. Without that, rescuing can lead to confusion. Words exist not just as categories for one’s own thinking, but for communication. All parties to a conversation must be on board with how a folk ontology has been rescued. When this is not the case, confusion, misunderstandings, and even deception can occur. Rescuing a concept can also leave a void of terminology to refer to the original folk ontology, making discussion of it difficult. This means that the correct response may be context dependent. A rescued idea may need to be tabooed when speaking with people who don’t share the same view or when discussing the folk ontology itself. Rejecting is a far less fraught response. The primary downside is increased difficulty in communication caused by a loss of terminology to describe a phenomenon that people have previously found useful. When rejection is a good response, this drawback is more than offset by the clarity of thought brought about by avoiding the erroneous concept. To me, morality is a good example of this. Avoiding describing things as right or wrong can be cumbersome. However, it brings a great deal of clarity as it forces one to focus on optimizing goals. This highlights the differing goals people have and prevents moral intuitions from clouding judgment. Restricting and recognizing requires a nuanced understanding of when a concept is applicable and practice noticing when it is not. It can also lead others to assume the folk ontology is believed.

Even with the above trade-offs in mind, it can be difficult to select the best response. Indeed, the response may be context dependent. Varying based on who one is interacting with and what is being discussed. Furthermore, these three responses aren’t fully distinct. There is a spectrum between them. Feel free to craft a response that is an amalgam of these as appropriate. For example, it might make sense to apply a particular response to one aspect of a folk ontology and a different response to another aspect.

all the stories are shattered.
the meaning they ensconced is poured out,
    a drink offering on the altar of truth,
    a propitiation for ignorance.
how shall we who have no hands build new vessels?
vessels to carry us across the sea.
rather, we shall drown.
gasping for life, we swallow lies.
how shall we live unless we become fishes?

Recently I read Freezing People is (Not) Easy: My Adventures in Cryonics by Bob Nelson.¹ It recounts his time as president of the Cryonics Society of California (CSC) during which he cryopreserved and then attempted to maintain the cryopreservation of a number of early cryonics patients. I’ve been thinking about and investigating cryonics of late, so I went looking for any and all related material I could get my hands on. Since this book was the only one I could source from my local library, it ended up being the first one I read. I was looking to gain insight into why cryonics has not become more popular and to learn some of the challenges faced when attempting to run a cryonics organization. What I found instead was a sad tale of typical human failings. It should be noted that the author has been a very controversial figure and some might claim that the veracity of this book is questionable. He has been called a fraud and a con man. If his memoir is to be believed, he was simply seeking to help people fulfill their desires to be cryopreserved, but foolishly failed to adequately plan, prepare, or ensure monetary provision for the task.

It’s often said that one should start a story at the beginning. The question is: what is the beginning of the story one is telling? The first chapter describes Mr. Nelson’s difficult childhood and very early marriage under challenging circumstances. I found this chapter boring and extraneous. Even after finishing the whole book, I’m not sure what the purpose of including this material was. Perhaps it was to show why he cared about others so much that he fell into his later mistakes? If so, it did a poor job of that besides one anecdote of him helping a severely injured teenager when he was himself a teenage runaway. More likely it was in the hope that the reader might take pity on him, viewing him as a tragic hero.

Nelson first learned of cryonics in 1965² when an acquaintance pointed out a newspaper article discussing the recently published book The Prospect of Immortality by Robert Ettinger.³ He was immediately consumed by the idea of cryonics: that a person’s death could be postponed until technology advanced to the point where they could be healed. Searching local bookstores for a copy, he finally located a store that had them. Upon arrival, he found they hadn’t even removed the books from the shipping box because they had received them so recently. Reading through Mr. Ettinger’s book his excitement grew. Nelson loved astronomy and space travel, and was immediately thrilled by the potential of using cryonics technology to place astronauts in suspended animation for long-term spaceflight. He envisioned a new race, in the vein of the space race, to be the first country to revive an individual from cryopreservation. As soon as he became convinced of the scientific validity of the possibility of cryonics by consulting with several friends and the recognition of natural examples of suspended animation, he was a true convert. The strength of his faith in the advancement of technology to achieve near-miraculous results is evident.

I find it fascinating that a small number of individuals take immediately to the idea of cryonics upon learning of it while the vast majority dismiss it or are repulsed. What is it that sets these people apart? Nelson himself wonders about this in the book but offers no insights. Understanding this would seem to be a crucial first step to anyone seeking to encourage others to sign up for cryonics. This information is even more important for a cryonics organization whose success depends on it.

Nelson’s excitement led him to attend the first meeting of the Life Extension Society in California on May 13, 1966⁴. He found the attendees’ scientific credentials and financial resources underwhelming. So, it should not have been so surprising to him when, at the second meeting, he was elected president despite being a TV repairman. He then established the nonprofit Cryonics Society of California (CSC) whose initial plan was to promote research into cryonics. A scientific advisory council was organized, and the CSC was to provide funding. The council informed them that if the CSC cryopreserved someone then the council would walk away and have nothing to do with them. The book is unclear on this point, but it appears the council, being composed of researchers in existing fields like cryobiology, thought that more direct involvement of the CSC with cryopreservation would hurt their reputations. This restriction was not concerning to Nelson as he felt more research was needed and didn’t plan to cryopreserve anyone. He was giddy from the success of forming the council and receiving the validation of scientists. Indeed, his excitement at this stage seems incredible. He states that “in December 1966, cryonics was a bright golden vista with limitless possibilities on the horizon.”⁵

I would have liked to hear more about the CSC’s structure and plans. Later we learn that they had membership dues, yet it is never stated how much the dues were or approximately how many members they had. The funding for the scientific advisory council was to come from CSC funds, and the researchers would apply for external grants as well. Details of how Nelson expected this to develop would have been helpful.

Early in 1967, Nelson was unexpectedly approached by the son of Dr. James Bedford, a psychology professor at the University of California, with a request to cryopreserve his father upon legal death. Despite having agreed with the advisory council not to cryopreserve anyone and being entirely unprepared, they decided to proceed with the cryopreservation of Dr. Bedford. The CSC scrambled to acquire the necessary equipment. Bedford’s son stated that there was “three hundred thousand dollars in a foundation for cryobiological research, and [he was] director of that foundation.”⁶ However, Bedford died before they had collected any money and almost before they had the necessary chemicals. They performed the cryopreservation regardless and promptly turned his body over to his son. The CSC never received a dime for their efforts.

Much later Nelson learned that the money from the estate had been spent on legal fees as Bedford’s wife and son defended his will and cryopreservation from lawsuits brought by other relatives. Bedford is widely considered to be the first genuine cryonics patient because his body was preserved immediately after his death and cryoprotection measures were taken. Over the years, Bedford’s son diligently maintained his father’s cryopreservation. Long-term support from relatives like this was rare, and Bedford is the only patient prior to 1974 that remains cryopreserved.⁷ His body is now in the care and custody of the Alcor Life Extension Foundation. While the cryopreservation of Dr. Bedford generated copious publicity for the CSC and cryonics, it also immediately led to the scientific advisory council withdrawing, just as they’d warned.

This began a series of further cryopreservations by Nelson and his associates. Three people were cryopreserved and their bodies held at a mortuary while arrangements were made to keep them at a cemetery. (The patients were legally regarded as dead and as such their bodies were required to be stored at a mortuary or cemetery.) In each case, they collected no money up front, and the CSC received only $10 300 in donations after the procedures. By treating all fees as donations and handling the transfer of the body under the Anatomical Gift Act, the CSC avoided creating any contractual obligation to care for the patients and hoped to shelter themselves from any legal problems. However, this did not prevent patients’ relatives from later suing.

In all these cases, the plan was that relatives would pay for care after cryopreservation. This approach was predictably an egregious mistake, yet the reason for adopting it is never stated. These patients needed potentially indefinite care that relatives might lack the resolve or means to provide. What if relatives passed away before the patient was revived? I wish there had been some discussion of why this approach was chosen and considered viable. Nevertheless, it appears to have been the assumed practice at the time as all three organizations that performed cryopreservations followed it.⁷ In taking on these patients it seems Nelson was acting out of what he felt was a duty to care for these people. Yet, as any hospital will attest, one can’t give away unlimited free care. Doing so means an end to one’s ability to care for anyone.

In addition to the three patients CSC cryopreserved, they took over care of a fourth, who had been frozen by Cryo-Care Equipment Corporation. At the time, this business was one of only three organizations performing cryonics, but it made no attempt at cryoprotection, advertising its services for cosmetic purposes rather than eventual revival.⁷ Nelson was given charge of this patient because he offered the family a lower rate for ongoing care. He made this offer partially because he wanted the cryonics capsule the body was in as he had not been able to acquire a usable one himself. Despite promising to pay $150 per month for storage, the family never paid any money and instead explicitly left his fate to the CSC.

Nelson’s “predicted boom in cryonics patients never materialized.”⁸ Without more paying patients, his failure to collect adequate fees or establish a revenue stream to pay for the patients’ ongoing care left the CSC in dire financial straights. He found himself paying out of pocket for the ongoing patient care. In 1970, Nelson was unable to continue the care of these patients and was left no other option than to allow them to thaw.⁹ According to Nelson, this was a very emotional moment for him, and he went out into the desert to decide if it was the right course of action. This is how he lost all four patients in his care at the time.

Despite this failure to care for the patients in his charge, Nelson took on the cryopreservation of two additional patients at a price of $10 000 for cryopreservation and a suggested donation of $100–$300 per month for storage costs. The $10 000 donation did not cover the costs of a cryonics capsule and was not collected from most patients. As he had done previously, he also took charge of another patient who was cryopreserved by a separate organization, the Cryonics Society of New York (CSNY). Nelson repeated all his mistakes. He failed to collect money in advance and had no better plan for ongoing care when, again, relatives failed to pay. He continued to pay for patient care out of pocket and using the CSC membership dues. Eventually, when he was out of town for a week, having left the patients under the watch of the cemetery’s groundskeeper, disaster struck. While Nelson was away, there was a failure of the vacuum pump that was maintaining the insulation of the poorly constructed capsule. The groundskeeper, speaking in heavily-accented, broken English, was unable to communicate the problem to the liquid nitrogen supplier, who had contracted to maintain the temperature of the capsule. Nelson returned to find that the bodies had completely thawed.

After some time spent in despair, Nelson refroze them. He did this despite knowing that they were almost certainly no longer viable for revival since he estimated they may have been without liquid nitrogen for five days. To his surprise, when he contacted the families they expressed the belief that there was nothing to be done but to keep them frozen and hope for the best. He had clearly explained to them when they initially signed up the consequences of a cryopreservation failure. Still, the families’ reactions are understandable given their emotional involvement. On the other hand, Nelson’s decision is inexplicable, especially since some of the families were not paying for ongoing care.

While all this was going on, Nelson also assisted a man named Nicholas DeBlasio in setting up a storage location on the grounds of a cemetery in New York for his wife Ann who had been cryopreserved by the CSNY. Mr. DeBlasio was unhappy with his dealings with the CSNY and wanted to handle everything himself. He did in fact successfully maintain his wife’s cryopreservation for some years, and even took charge of another patient, storing them in the same capsule. However, infrequent checking of and issues with the capsule eventually led to a partial thaw and the subsequent decision to terminate the patients. This case highlights that long-term cryopreservation necessitates a dedicated professional team.

Nelson’s wife had divorced him. Exhausted and disillusioned by his efforts and failures to maintain the patients, he eventually decided to leave the field of cryonics. Without funding to continue patient care he contacted the families of the patients and with their consent removed the patients from cryopreservation. However, Nelson’s story was not over, as several family members sued him and the cemetery he had worked with to store the bodies. Even though Nelson was broke, they sought damages from the cemetery’s liability insurance and needed him as part of their case to improve their chances of winning. At the time, there was also a lot of bad publicity as news organizations picked up the story of the thawing and trial. The, at times farcical, trial is described at some length. It did make for a good climax to the story but was ultimately not very enlightening.

The book concludes by describing Nelson’s life since that time and how he came to the decision to write his story. As with the first chapter describing his life before his involvement with cryonics, this felt extraneous to the story. It was made somewhat more palatable by having come to see him as a pitiable figure so that it was nice to hear he was able to move on with his life.

The image sometimes painted of Nelson is that of a fraud and swindler. That was how the attorneys of the families suing him attempted to portray him. In this book, he comes off as a sincere man trying to do his best to help others but ultimately failing. By not acting as a professional he repeatedly set himself up to fail. No one can provide a service without taking adequate payment. Indefinitely maintaining care of cryonics patients is a tremendous undertaking and not to be taken on without significant planning. Organizations with long-term financial obligations such as cemeteries and insurances companies have been known to fail. With cryonics one hopes that no room for failure would be left open. Despite this, Nelson and all the cryonics organizations operating at the time foolishly relied on family members for ongoing patient care funding. To me, this is simply another example of human irrationality. Modern cryonics organizations are, by comparison, much better managed. Still, it is not clear to me if they have adequately planned for the rigors of perpetual care.

Overall, the book was an enjoyable read for someone interested in cryonics. I think it serves as valuable documentation of the history of cryonics. However, it would have served that purpose much better had it detailed more the financial, organizational, and legal structure of the CSC; and explained the thinking behind having relatives fund patient care and not demanding payment up front for cryopreservation and the cryonics capsule. While I think it is good this book was written, I do worry that it will hurt the cryonics movement by dredging up the early failures. Readers may not make the distinction between the CSC and contemporary cryonics organizations.

[For a quick review, jump to the summary or chart of signs.]

Group discussions can easily become unwieldy. Certain individuals have a tendency to dominate the conversation, limiting others’ ability to contribute. When the goal is collaborative truth seeking, this is counter productive. Those who are excluded from the discussion may be exactly the ones with the necessary insights to advance the dialogue toward the truth. To deal with this and other issues, a moderator is sometimes appointed. However, traditional moderation may introduce too much friction and is geared toward potentially combative dialogue. A lighter-weight system intended for more collaborative discussions is helpful. To address this, a system of hand signals called automoderation was created. Automoderation has been used in the rationality community in Columbus, Ohio, for almost two years now with good success.

The rationality community should ostensibly be seeking the truth in any discussion. One might imagine that rationalists, cognizant of the pitfalls of group discussion and genuinely seeking truth, would manage their discussions better than others. However, human nature still dominates and there are often individuals involved who have underdeveloped social skills. For example, they may struggle to signal that they wish to speak or fail to notice the social signals that they are inappropriately dominating the conversation. Additionally, a focus on thinking through the subject and developing genuine insight often draws cognitive resources away from preparing a quick response as might be needed to jump into the middle of a conversation. It can be helpful, in that situation, to have a method of signaling one’s desire to speak, and be given adequate time by the group to formulate one’s thoughts.

In larger and more formal settings a moderator can be appointed. They become responsible for determining who may speak and for how long. They facilitate discussion, ensuring that all members with standing are given a chance to speak and no one inappropriately dominates the conversation. A moderator can also facilitate metadiscussion about procedure, topics, and participant needs while ensuring the debate isn’t derailed by them. However, in smaller or less formal settings a traditional moderator may not be called for and there may be no one willing to fill the role. Yet, the smaller or less formal setting does not always obviate the need for some degree of moderation.

The rationality community in Columbus, Ohio, found itself in the position of needing a system of moderation for their discussions, in particular for a rationality dojo. A little over two years ago Max Harms along with another member of the community created a system of hand signals supporting moderation in smaller, less formal settings. This system was inspired by the Occupy movement hand signals. When all participants know the hand signals, a moderator may not even be needed. A moderator is still useful, but often does little besides clarifying the system and consequently introduces very little friction. This system of hand signals is called automoderation. It has been used successfully in groups as small as 3 to 4 people and as large as 15 to 20.

Automoderation embodies a wait rather than an interrupt culture. Interrupt culture may be fine for a causal fun conversation, but collaborative truth seeking is aided by a wait culture. Traditional moderation systems such as Robert’s Rules of Order are intended to handle parliamentary proceedings and combative discussion. This system is intended for use in a more cooperative group discussion.

Automoderation Hand Signals

The automoderation system requires participants to sit in a circle. The flow of who speaks proceeds clockwise based on hand signals used to indicate desire for and purpose of speaking. There are five hand signals each having a different priority. Higher priority signals take precedence. The primary signal is simply raising the hand indicating the desire to expand on the current discussion topic. People new to automoderation are encouraged to use expand if they are in doubt as to the correct signal to use. Each signal indicates a different intent of the speaker. In addition, some signals modify the flow of the discussion. In order from lowest to highest priority, they are a raised fist indicating change topic, the pinkie finger extended indicating a probing question, a raised hand indicating a desire to expand on the topic, the pointer finger extended indicating a clarifying question, and two hands forming a triangle with the thumbs and forefingers indicating meta point. The hand signals are illustrated in the chart below in order from lowest to highest priority. A high-resolution version of this chart suitable for printing as a poster can also be downloaded.

The meta signal is used to raise points regarding the nature or situation of the discussion. Meta signals are relatively rare but often important. They include such things as: keeping the discourse focused on the topic at hand; clarifying the rules of automoderation; requesting changes to the environment or situation such as getting a drink, taking a break, or going to the bathroom; requesting information about the discussion, for example, how long it is planned to run; addressing concerns of tone and undesirable behavior; and dealing with strong emotional reactions.

A clarifying question asks a speaker to clarify something they said. This should be confined to instances of genuine ambiguity or confusion about what someone meant. After the question has been asked, the person is given a chance to briefly respond to the question but should stay focused on answering the question and avoid making additional statements. Typically, a clarifying question is raised while someone is speaking and the question is asked of them. Technically though, a clarifying question could be asked of anyone, not just the last speaker. This can happen when the confusion is not noticed until after another speaker has started.

Expand signals a desire to continue the conversation and expand on what has been said. This includes sharing ideas, opinions, experiences, and facts that are relevant. It is also used to respond to other speakers’ points and raise further questions or areas to be discussed. The majority of the signals used in a discussion are typically expand. Do not waste time making statements indicating agreement with other speakers; instead, use nonverbal hand signals described later to indicate agreement or disagreement.

A probing question is any question directed at a particular participant that is not a clarifying question. Probing questions can be used to explore a point in more depth. As with clarifying questions, the person the question is directed to is given a chance to respond. A probing question could be asked of anyone but is normally directed at the last speaker. Often, it makes more sense to transform a probing question into a statement of confusion or curiosity that can be directed to the whole group using the expand signal instead. This avoids the lower priority of the probing question signal and the potential for creating too much back and forth. It also gives other participants who may be better able to address the question the chance to weigh in.

Change topic signals that a participant wishes to change the topic. This can be because the current topic seems to have been discussed fully or there is simply a more interesting topic that they would like to discuss instead. Sometimes it is clear to multiple people that a topic is drawing to a close and one signal for a change topic will elicit multiple change topic signals in response. Generally, after a change in topic is proposed, a group decision is made, often using the approve and disapprove hand signals described later. One more pass around the circle may be allowed on the current topic so people can make closing remarks. If multiple people are requesting a change of topic, it is often worth seeing if there are multiple suggestions for the next topic and allowing the group to decide amongst them. There are situations when it may be appropriate to use meta to signal a desire to change topic instead. This might be the case if the current topic is too emotional for a participant to continue with or if, for reasons of time or agenda, it is important to move on to another topic even though the current topic is not complete. A change topic signal does not have to include a suggestion for the new topic.

Variations

The automoderation system should be taken as a set of guidelines rather than rules. Each community and group should adapt it to their needs. Someday it may become more codified and it might make sense to be more strict. Automoderation is still new and certainly has room for improvements to be discovered through experimentation. Additionally, there are situations it doesn’t cover. Those are instances where a moderator may need to step in and make a judgment call.

As originally created, the flow of who speaks always continued clockwise from whoever the last speaker was, regardless of which hand signal led to their speaking. If multiple people were signaling, the one with the highest priority was selected. Ties between signals of the same priority were broken by selecting the first person clockwise from the last speaker. This has the virtue of being very simple and clear. However, it means that people can be skipped over. For example, someone wishing to expand might be skipped because a person after them is using the meta signal. The skipped person would then not be able to speak until the flow came back around the circle. The same problem can arise when a question is asked since the flow continues from the respondent.

Simple Rules

1. When someone is done speaking, call on people who are signaling a desire to speak.
2. If two or more people are signaling, call on the one with the highest priority signal; break ties by going in a circle, clockwise from the last speaker.
3. If someone asks a question (probing or clarifying), the person they ask should respond; flow continues from the question answerer.

In practice, the Columbus rationality community has not followed the simple rules but rather continues flow from a point that allows the skipped participants to speak next. This works well but is more complicated to keep track of. In this variation, flow may continue either from the last speaker who was not responding to a question or from a previous speaker depending on the relative priority of the signals involved. When multiple people are signaling, the one with the highest priority is selected. Ties are broken by selecting the first person clockwise from the appropriate participant. When the last speaker was expanding or requesting a change of topic, flow continues from their left. When the last speaker was responding to a probing question, flow continues from the left of the questioner. When first going up to the priority of a clarifying question or meta point from a lower priority signal, note the last speaker or if the last speaker was responding to a probing question, note the questioner. Flow continues from the noted person when returning to lower a priority. When the last speaker was making a meta point, flow continues to their left if there are more meta signals; otherwise, flow continues from the noted speaker. When the last speaker was responding to a clarifying question, flow continues from the questioner’s left if there are more clarifying questions or meta points; otherwise, flow continues from the noted speaker. If your group feels comfortable with this approach, it is the one currently recommended by the Columbus rationality community.

Standard Rules

1. When someone is done speaking, call on people who are signaling a desire to speak.
2. If two or more people are signaling, call on the one with the highest priority signal; break ties by going in a circle, clockwise from the appropriate participant.
3. When the last speaker was expanding or changing the topic, flow continues from their left. If the next speaker is asking a clarifying question or raising a meta point, note the current speaker. This is the point flow will continue from when returning to a lower priority.
4. When the last speaker was responding to a probing question, flow continues from the questioner’s left. If the next speaker is asking a clarifying question or raising a meta point, note the questioner. This is the point flow will continue from when returning to a lower priority.
5. When the last speaker was making a meta point, flow continues from their left if the next speaker is also making a meta point; otherwise, flow continues from the previously noted participant.
6. When the last speaker was responding to a clarifying question, flow continues from the questioner’s left if the next speaker is asking a clarifying question or making a meta point; otherwise, flow continues from the previously noted participant.
7. If someone asks a question (probing or clarifying), the person they ask should respond.

Other variations on the system have been suggested but not tried to determine whether they are better. Some community members have suggested eliminating the probing question signal as it is rarely used. Its priority below expand discourages use. It can lead to back and forth discussion, and expand is typically a better signal to use instead. Other community members have suggested making the change topic signal not be a request to speak. Like the signals described below, it would be used only to communicate nonverbally to the group. If one wished to talk about the change of topic, they could raise a hand at the same time. This would go well with the removal of probing question as it would eliminate all signals with priority below expand.

In practice in Columbus, the change topic and probing question signals are sometimes treated as having the same priority as expand. This simplifies things slightly and doesn’t seem to cause problems. However, it does open up the possibility that someone could force discussion on a change of topic before other people are ready.

Other Hand Signals

In addition to the automoderation hand signals, the community has found it helpful to have a few other hand signals to facilitate communication without adding any auditory distraction to the discussion. The first few of these are so useful that they could be considered for official inclusion in the automoderation system. The rest of the signals should probably be taken as possible suggestions if the situation arises or a community finds the need for them.

A thumbs up is used to signal approval or agreement and a thumbs down to signal disapproval or disagreement. These can be used for informal voting when a change of topic or meta change is proposed. They may be used when someone makes a particularly good or enlightening point. It is important that they be interpreted and used to signal about the content of a speakers message and never as a judgment of the speaker themselves.

Two more signals are in occasional use. The first is the OK hand sign, which is used to indicate that you are listening and interested in what someone has to say. The second is to point the flat hand, palm down toward the speaker and wiggle the finger tips. This sign was imported by a few community members from CFAR and means “I feel you” or “This resonates with me”. That is subtly different from the thumbs up which implies something more like agreement. These signals have rarely been used in practice and may not be worth the complexity of having them.

It may be helpful to add a hand signal to keep track of where flow should start from when continuing after a meta or clarifying question. The noted speaker would make this sign so that others don’t have to mentally keep track of this. Flow would then continue from their left. Two suggestions for this signal have been made. The first is the “shaka sign” with the thumb and pinkie extended and the other fingers curled closed. The second suggestion is pointing to the left.

Examples

To help clarify the rules, here are some examples following the standard rules described above. Imagine Amy, Ben, Cora, and Dan wish to hold a discussion. They sit in a circle in that order. Here are how a number of scenarios would play out. For clarity, each scenario starts with Amy speaking.

While Amy is speaking, Cora raises her hand to expand. When Amy is done speaking, Cora speaks next because she is the only one with a hand raised. Flow continues from Cora when she is done.

While Amy is speaking, Ben and Dan raise their hands to expand and Cora signals a meta point. When Amy is done speaking, there are signals of priority higher than expand, so it is noted that flow will continue from Amy in the future. Cora speaks next because her signal is higher priority. When Cora is done, flow continues from Amy, so Ben will speak next.

While Amy is speaking, Ben signals a probing question and Dan wishes to expand. When Amy is done speaking, Dan speaks next because expand is higher priority than a probing question. When Dan is done speaking, since no other people wish to expand, Ben may ask his probing question. He asks it of Amy, who then responds. Flow then continues from Dan because flow continues from the questioner rather than the respondent.

Advantages

On the whole, the automoderation system has proved very successful and continues to be used in the Columbus, Ohio, rationality community. It is mainly used at the regularly scheduled meetings that are open to rationalists only; at these, everyone present knows the hand signals or can be expected to learn them as part of their initiation into the community. The advantages of using automoderation include:

• It ensures that everyone is given an opportunity to speak. This is particularly valuable for those members who are more introverted, take longer to formulate a response, or have underdeveloped social skills. Women have also reported that this helps them participate in the discussion as they may have been socialized to wait longer before speaking or to be less aggressive in entering a conversation. When women are trying to participate in a male-dominated discussion, the automoderation system highlights the shift to a wait culture for all involved.
• Having the system as a community norm enables members to initiate it when they see value in doing so. It is not uncommon, if there is a discussion going on in which someone feels they are not able to interject, for them to raise a hand. This serves as a reminder to the others present to make sure everyone is included in the discussion and often immediately triggers the use of automoderation.
• A group of people who know the system often need no moderator at all as it is clear to all participants who the next speaker is.
• The hand signals provide an easy nonverbal channel of communication to moderate the discussion that does not interfere in any way with the current speaker.
• Many raised hands and other signals can indicate to a speaker that others wish to speak and it may be a good idea to bring their current remarks to a close to allow others to speak. This is made more palatable by the knowledge they will have another chance to speak when it comes around the circle again.
• The requirement to select the appropriate hand signal can force one to clarify in their own mind the purpose of speaking and better plan their remarks.
• The hand signals can be useful in situations where automoderation is not being used. In particular, the meta symbol can be used to good effect outside automoderation to indicate a desire to interrupt a conversation with a meta item. Often this is something the conversation members will feel constitutes a reasonable interruption.
• By promoting wait culture, automoderation can help to train members of the community away from speaking as much as possible or as quickly as possible simply for the, often unconscious, purpose of gaining social status.

Disadvantages

While automoderation is a useful social tool, it is not without its drawbacks. It is important to be aware of these to help mitigate them.

• There are times it would be better to allow a back and forth discussion between a few participants. Automoderation does not support this. This is a situation where a moderator can step in if needed.
• Discussion points can pile up, so that by the time one has an opportunity to speak, there are now many things to respond to. It can be hard to keep track of all the threads that are now being discussed in parallel.
• It is easy to lose track of what one was planning to say while waiting for others to speak.
• The pile up of points can encourage people to move on rather than continuing to explore a point that it might be valuable to delve into more.
• The topic can naturally drift as participants expand on each other’s statements.
• If a participant is long winded or doesn’t make positive contributions to the discussion, it can be difficult to limit their disruption of the discussion.
• Overuse of automoderation removes opportunity to practice valuable social discussion skills. Often, a conversation with fellow rationalists would otherwise be an ideal situation for this practice since they are likely to be more forgiving and supportive of one’s efforts to grow.

Applicability

There are situations it may not be appropriate to use automoderation. It is good to be aware of these so that one can judge when it would be more appropriate to simply have a discussion or to use another moderation system.

• Automoderation breaks down in large groups. While it has been used with some success in groups of 15 to 20 that was only because not many participants actually wished to speak. Had all of them wished to speak, it likely would not have gone well.
• It should not be the default for casual conversations amongst groups of friends.
• Don’t use it for small groups of 3 to 4 unless the need becomes apparent or it is a more formal meeting.
• Automoderation is probably best suited for groups of 5 to 10 people.
• Automoderation is best suited for groups where all members are cognizant of the degree to which they can genuinely contribute to the discussion.
• Don’t use automoderation when not enough participants are familiar with the system.

Etiquette

Some etiquette can help the automoderation system run more smoothly and ensure all participants feel included.

• If someone has had their hand up for a while either because they have been skipped or for a change of topic or probing question, it is polite to avoid raising one’s hand so that they may speak.
• If someone is wishing to speak, it is polite for the speaker to try and finish their point rather than going on at length, especially if they have a clarifying question.
• Don’t waste time using expand to state agreement; instead, use the nonverbal hand signal of a thumbs up.

Summary

The hand signals in priority order, low to high, are:

• Change Topic (Fist) – request to move on to another topic.
• Probing Question (Pinkie Extended) – ask a question and receive a response.
• Expand (Raised Hand) – make further statements about the topic.
• Clarifying Question (Pointer Finger Extended) – ask a question to clear up confusion or ambiguity and receive a response.
• Meta (Triangle) – raise an issue about the nature or context of the discussion.

Hand signals that are not a request to speak:

• Approve/Agree (Thumbs Up)
• Disapprove/Disagree (Thumbs Down)

Simple Variant

1. When someone is done speaking, call on people who are signaling a desire to speak.
2. If two or more people are signaling, call on the one with the highest priority signal; break ties by going in a circle, clockwise from the last speaker.
3. If someone asks a question (probing or clarifying), the person they ask should respond; then flow continues from the question answerer.

Standard Variant

1. When someone is done speaking, call on people who are signaling a desire to speak.
2. If two or more people are signaling, call on the one with the highest priority signal; break ties by going in a circle, clockwise from the appropriate participant.
3. When the last speaker was expanding or changing the topic, flow continues from their left. If the next speaker is asking a clarifying question or raising a meta point, note the current speaker. This is the point flow will continue from when returning to a lower priority.
4. When the last speaker was responding to a probing question, flow continues from the questioner’s left. If the next speaker is asking a clarifying question or raising a meta point, note the questioner. This is the point flow will continue from when returning to a lower priority.
5. When the last speaker was making a meta point, flow continues from their left if the next speaker is also making a meta point; otherwise, flow continues from the previously noted participant.
6. When the last speaker was responding to a clarifying question, flow continues from the questioner’s left if the next speaker is asking a clarifying question or making a meta point; otherwise, flow continues from the previously noted participant.
7. If someone asks a question (probing or clarifying), the person they ask should respond.

Experimental Variant

This experimental variant is based on feedback from members in the Columbus rationality community and has not been tried yet. It is like the standard variant with the following changes. Remove the probing question signal. The change topic signal is no longer a request to speak but is simply a nonverbal signal like approve or disapprove. Use the flow marker signal of pointing to the person to one’s left when going to a meta or clarifying question to indicate where flow should continue from.

Conclusion

Automoderation has been a very useful addition to the social toolbox of the Columbus rationality community. It has helped many discussions go smoothly and people to contribute and feel more comfortable participating in the dialogue. This post was written in the hope that other rationalist communities will find it useful and spread and improve the system. Take whatever parts seem useful and apply them as seems best.

Reactions

• Automoderation by Brienne Yudkowsky
• On Automoderation by The Zvi
• Thoughts on Automoderation by Proof Of Logic

EDIT 2017-04-21: Added reactions, clarified chart order.