Does a “Lovely” Have a Slave Mentality? – OR – Why a Super-Intelligent God *WON’T* “Crush Us Like A Bug”

Video (starts 27 minutes in) * Powerpoint (unfortunately not visible in the video) presented March 8, 2010 at The Third Conference on Artificial General Intelligence (Lugano, Switzerland)

Inspired by a question after the previous day’s Designing a Safe Motivational System for Intelligent Machines presentation. Immediately followed by a rebuttal by and debate/Q&A with SIAI’s Roko Mijic.

Designing a Safe Motivational System for Intelligent Machines

Video * Presentation Powerpoint * Proceedings Paper presented March 7, 2010 at The Third Conference on Artificial General Intelligence (Lugano, Switzerland)

As machines become more intelligent, more flexible, more autonomous and more powerful, the questions of how they should choose their actions and what goals they should pursue become critically important. Drawing upon the examples of and lessons learned from humans and lesser creatures, we propose a hierarchical motivational system flowing from an abstract invariant super-goal that is optimal for all (including the machines themselves) to low-level reflexive “sensations, emotions, and attentional effects” and other enforcing biases to ensure reasonably “correct” behavior even under conditions of uncertainty, immaturity,error, malfunction, and even sabotage.