Abstract:
One of the more promising empirical theories of phenomenal consciousness is the Global Workspace Theory (GWT), which says that a mental representation becomes conscious when it enters a global workspace from which it can be broadcast to various specialized psychological modules [1-3]. There has been some discussion of GWT in connection with recent advancements in AI, often with the idea being that if GWT is true, AI systems of the near future could be conscious [4-6]. This has potential moral implications for how we should treat such systems, given plausible assumptions about the connection between consciousness and matters like wellbeing and moral patienthood [7-8]. We contribute to this discussion by viewing things through the framework of resource-rational analysis [9-11], exploring a version of GWT on which consciousness and intelligence can come apart in surprising ways. On the view in question, an AI system is likelier to be a zombie (lack consciousness) the more intelligent it is.
The guiding idea of research-rational analysis is that human cognition should be understood as accomplishing various tasks in the rationally optimal way given limitations on cognitive resources that come with minds like ours, including limitations on computational power and memory. Rational analysis approaches have been developed across a wide range of cognitive domains, resulting in rational or Bayesian models of perception, causal learning, memory, categorization, and more [12-14]. The approach has not been pursued at length for phenomenal consciousness, which is understandable if it is assumed that consciousness has no intimate link to rationality.
However, according to GWT, there is a consciousness-rationality link [15]: the global workspace is an optimal or rational solution to a communication problem across modules. On the view we explore, the development of a global workspace should be understood not as an optimal solution for all intelligent systems, but as an optimal solution for human cognition given our particular constraints.
Thomas Griffiths has previously argued that many AI systems will avoid the resource constraints that human cognition runs up against [16]. To illustrate with a now familiar example, it would take a human being thousands of years to read all the text that GPT-4 was trained upon, something impossible for us but not AI. Taking this as our inspiration, we develop a view saying that AI systems not facing our constraints could make do without a global workspace without thereby sacrificing their intelligence or ability to accomplish various tasks. These systems would be intelligent but not conscious, given GWT.
To flesh this out, consider the GWT claim that consciousness is associated with a narrow informational bottleneck: the mind is divided into different modules engaged in parallel processing of rich representational contents, but only a tiny fraction of this information makes it onto the stage of consciousness, which operates in serial fashion. In a discussion of the GWT bottleneck, Stanislas Dehaene [3] observes that while there is extensive empirical evidence that unconscious neural circuits involved in perception engage in sophisticated Bayesian inference, this probabilistic character is not part of our visual experience—perception does not seem probabilistic [19]. Dehaene proposes that this is because consciousness relies on a form of statistical sampling that involves “collapsing all unconscious probabilities into a single conscious sample, so that we can move on to further decisions” (p. 93).
We highlight the informational bottleneck because, at least prima facie, it sounds like a matter of resource constraints. If only our heads were bigger, the highway of our minds could open up a new lane, avoiding the bottleneck. Less metaphorically, maybe if we were not so resource-constrained, we could just plug all those unconscious probabilities into our decision-theoretic calculation of how to act, rather than relying on the simplified executive summary. If this is right, it would open the door to AI systems to avoid such bottlenecks without thereby sacrificing intelligence or performance. If the bottleneck is understood as an essential part of GWT, such systems would lack phenomenal consciousness. Here are three morally important consequences that would follow if such a view were right.
First, it should be possible to construct intelligent machines that accomplish complex tasks at human or superhuman levels without being conscious, and so without being owed whatever moral consideration is attached to consciousness. For example, we could shoot a highly intelligent AI probe into the sun without destroying a conscious being and so (perhaps) without morally wronging anyone or anything [17].
Second, AI developers are unlikely to stumble into accidentally creating conscious AI just by virtue of making machines more and more intelligent. Some researchers have argued that this happens for other capacities—for example, they claim that large language models have acquired an emergent capacity for Theory of Mind [20]. But if having a human-like global workspace is an optimal solution only for those systems with human-like resource limitations, it becomes much less likely that AI models will develop such a workspace and so develop consciousness without intentional engineering. It may even be that the more intelligent the AI system, the less likely it is that it will be conscious because the less likely it will face the limitations that make a global workspace an optimal solution.
Third, in connection with his instrumental convergence thesis, Nick Bostrom has argued that sufficiently intelligent AI systems are likely to pursue their own cognitive enhancement as a means to achieving their final goals, whatever those goals happen to be [18]. If consciousness is the result of a resource limitation, then sufficiently intelligent and conscious AI systems should make efforts to annihilate their own consciousness, so as to enhance themselves away from those limitations. This is not merely a scenario where consciousness is one thing and intelligence is another—a familiar thought. Rather, it is a scenario where consciousness and intelligence pull in opposite directions (outside our human resource constraint niche), so that the elimination of their own consciousness is something we should expect intelligent AI systems to converge upon. Sufficiently smart AI will seek its own zombification.
