Backward Induction: Rationality or Inappropriate Reductionism? – Part 2

by Mark R. Waser (originally appeared Dec. 23, 2012 at Transhumanity.Net)

Contains three hints (one given previously) before the puzzle solution from Part 1 is revealed.

Before I lose my mathematical/logical audience, let me clearly state that backward induction is a logical operation that *always* functions correctly.  Unfortunately, however, like so many similar operations, correct functioning upon garbage input most often includes outputting garbage (GIGO).  This is clearly the case when using backward induction on the centipede game.

Continue reading

Got Risk? Debating Eliezer Yudkowsky About “AIs that prove changes correct”

by Mark R. Waser (originally appeared Dec. 21, 2012 at Transhumanity.Net)

Rice’s Theorem (in a nutshell):

Unless everything is specified, anything non-trivial (not directly provable from the partial specification you have) can’t be proved

AI Implications (in a nutshell):

You can have either unbounded learning (Turing-completeness) or provability – but never both at the same time!


I have been semi-publicly called out by Eliezer Yudkowsky.  He posted the following on IEET* Director James Hughes’ Facebook Wall in response to a post that referenced my last article Continue reading

Coherent Extrapolated Volition: The Next Generation

by Mark R. Waser (originally appeared Dec. 16, 2012 at Transhumanity.Net)

Those unable to solve the Centipede Game puzzle from my last article (Backward Induction:  Rationality or Inappropriate Reductionism? – Part 1) can find a blinder-removing hint here.  People who believe that they’ve found the best solution should check the hint to verify that they have indeed found the best solution (i.e. since a “good enough” solution and a “better” solution both exist).  Part 2 of the article, with the solutions, will be along shortly.

I admire Eliezer Yudkowsky when he is at his most poetic:

our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted.

Continue reading