The prime futurist fear is that humanity will create some advanced technology with an ostensibly positive purpose, but it will buck our control and undo the world as it pursues some twisted version of the ends it was programmed to achieve. Quiet Babylon writes that this artificially-sentient oppressor has already arrived:
One of my favorite recurring tropes of AI speculation/singulatarian deep time thinking is meditations on how an evil AI might destroy us.
Here’s an example: The scenario imagined is where there is a button that humans push if the AI gets an answer right and the AI wants to get a lot of button presses, and eventually it realizes that the best way to get button presses is to kill all the humans and institute a rapid fire button-pressing regime.
You would have this thing that behaves really well, until it has enough power to create a technology that gives it a decisive advantage — and then it would take that advantage and start doing what it wants to in the world. Read more…
Using Technology to Deepen Democracy, Using Democracy to Ensure Technology Benefits Us All
An Open Letter To The Robot Cultists — From amor mundi
Any child of two can indulge in wish fulfillment fantasizing. It’s not a philosophy. It’s not a movement. And the way you Robot Cultists do it makes you a kind of techno-transcendental New Age cult too hype-notized to notice you are functioning as a crowdsourced cheerleading squad for celebrity CEOs and ramped up gizmo consumerism at a time when the world is literally perishing from extractive- industrial- petrochemical- consumer- indebted- corporate- militarism.
The digital revolution is a lie. Cyberspace isn’t a spirit realm. It belches coal smoke. It is accessed on landfill-destined toxic devices made by wretched wage slaves. It abetted financial fraud and theft at every level of society around the world. Its “openness” and its “freedom” turned out to be targeted marketing harassment, panoptic surveillance, and zero comments.
Rather than grasp this catastrophic fraud, you embrace it more ferociously, you hyperbolize cyberspatial deceptions into a more delusive fantasy still, fancying it will be home to a history shattering perfectly parental God-AI delivering you into the digital garden where your “spirit self” can live forever and “be” anything and “have” everything and “know” it all forever.
Your Robot Cult — whether in its eugenicist transhumanoid sects, or in its dead-ender AI (artificial imbecillence) Singularitarian nerd-rapture sects, or in its vitamin supplement replacement parts shiny robot body soul-migration techno-immortalist sects, or in its nano-santa nano-genies-in-a-bottle nano-cornucopiast sects, or in its greenwashing hyper-denialist “geo-engineering” sects — your Robot Cult, I say, takes all the lies of crass commercialism — it takes all its infomercial boner pills and anti-aging kremes and endless promises of consumer ecstasy — and then sets the volume dial on eleven, turning what was just ugly stupid embarrassing commonplace circus-barker deception and crack-pottery into full on fulminating faith. More…