Abstract:
To avoid predicting that certain kinds of compulsions contribute to welfare, many desire satisfaction theorists draw a distinction between genuine desires and other desires. Such theorists hold that it is only the satisfaction of genuine desires that contributes to welfare; other desires do not count. From the perspective of AI welfare, then, we must ask not only whether it is possible for artificial systems to have desires but also whether it is possible for them to have genuine desires. If it is essential to genuine desires but not desires in general that they are connected to phenomenal experience, for example, this fact could rule out welfare in non-conscious artificial systems. We discuss several existing proposals about how to characterize genuine desires and show that they do not do well at capturing our intuitive judgments about a range of cases. We then introduce two theories that do better. Both theories suggest that AI systems can have genuine desires if they can have desires.
