J. Schmidhuber. Advances in Evolutionary Computing, Springer-Verlag, Berlin Heidelberg, (2002)
Zusammenfassung
Details of complex event sequences are often not predictable, but
their reduced abstract representations are. I study an embedded active
learner that can limit its predictions to almost arbitrary computable
aspects of spatio-temporal events. It constructs probabilistic algorithms
that (1) control interaction with the world, (2) map event sequences
to abstract internal representations (IRs), (3) predict IRs from
IRs computed earlier. Its goal is to create novel algorithms generating
IRs useful for correct IR predictions, without wasting time on those
learned before. This requires an adaptive novelty measure which is
implemented by a co-evolutionary scheme involving two competing modules
collectively designing (initially random) algorithms representing
experiments. Using special instructions, the modules can bet on the
outcome of IR predictions computed by algorithms they have agreed
upon. If their opinions differ then the system checks who's right,
punishes the loser (the surprised one), and rewards the winner. An
evolutionary or reinforcement learning algorithm forces each module
to maximize reward. This motivates both modules to lure each other
into agreeing upon experiments involving predictions that surprise
it. Since each module essentially can veto experiments it does not
consider profitable, the system is motivated to focus on those computable
aspects of the environment where both modules still have confident
but different opinions. Once both share the same opinion on a particular
issue (via the loser's learning process, e.g., the winner is simply
copied onto the loser), the winner loses a source of reward -- an
incentive to shift the focus of interest onto novel experiments.
My simulations include an example where surprise-generation of this
kind helps to speed up external reward.
%0 Book Section
%1 Schmidhuber:2002
%A Schmidhuber, J.
%B Advances in Evolutionary Computing
%C Berlin Heidelberg
%D 2002
%E Ghosh, S.
%E Tsutsui, S.
%I Springer-Verlag
%K imported
%P 579-612
%T Exploring the Predictable
%X Details of complex event sequences are often not predictable, but
their reduced abstract representations are. I study an embedded active
learner that can limit its predictions to almost arbitrary computable
aspects of spatio-temporal events. It constructs probabilistic algorithms
that (1) control interaction with the world, (2) map event sequences
to abstract internal representations (IRs), (3) predict IRs from
IRs computed earlier. Its goal is to create novel algorithms generating
IRs useful for correct IR predictions, without wasting time on those
learned before. This requires an adaptive novelty measure which is
implemented by a co-evolutionary scheme involving two competing modules
collectively designing (initially random) algorithms representing
experiments. Using special instructions, the modules can bet on the
outcome of IR predictions computed by algorithms they have agreed
upon. If their opinions differ then the system checks who's right,
punishes the loser (the surprised one), and rewards the winner. An
evolutionary or reinforcement learning algorithm forces each module
to maximize reward. This motivates both modules to lure each other
into agreeing upon experiments involving predictions that surprise
it. Since each module essentially can veto experiments it does not
consider profitable, the system is motivated to focus on those computable
aspects of the environment where both modules still have confident
but different opinions. Once both share the same opinion on a particular
issue (via the loser's learning process, e.g., the winner is simply
copied onto the loser), the winner loses a source of reward -- an
incentive to shift the focus of interest onto novel experiments.
My simulations include an example where surprise-generation of this
kind helps to speed up external reward.
@incollection{Schmidhuber:2002,
abstract = {Details of complex event sequences are often not predictable, but
their reduced abstract representations are. I study an embedded active
learner that can limit its predictions to almost arbitrary computable
aspects of spatio-temporal events. It constructs probabilistic algorithms
that (1) control interaction with the world, (2) map event sequences
to abstract internal representations (IRs), (3) predict IRs from
IRs computed earlier. Its goal is to create novel algorithms generating
IRs useful for correct IR predictions, without wasting time on those
learned before. This requires an adaptive novelty measure which is
implemented by a co-evolutionary scheme involving two competing modules
collectively designing (initially random) algorithms representing
experiments. Using special instructions, the modules can bet on the
outcome of IR predictions computed by algorithms they have agreed
upon. If their opinions differ then the system checks who's right,
punishes the loser (the surprised one), and rewards the winner. An
evolutionary or reinforcement learning algorithm forces each module
to maximize reward. This motivates both modules to lure each other
into agreeing upon experiments involving predictions that surprise
it. Since each module essentially can veto experiments it does not
consider profitable, the system is motivated to focus on those computable
aspects of the environment where both modules still have confident
but different opinions. Once both share the same opinion on a particular
issue (via the loser's learning process, e.g., the winner is simply
copied onto the loser), the winner loses a source of reward -- an
incentive to shift the focus of interest onto novel experiments.
My simulations include an example where surprise-generation of this
kind helps to speed up external reward.},
added-at = {2009-06-26T15:25:19.000+0200},
address = {Berlin Heidelberg},
author = {Schmidhuber, J.},
biburl = {https://www.bibsonomy.org/bibtex/2da08a6cc214983b969b5a435275a0a38/butz},
booktitle = {Advances in Evolutionary Computing},
description = {diverse cognitive systems bib},
editor = {Ghosh, S. and Tsutsui, S.},
interhash = {cca739519882432cdc1a9658b4249db5},
intrahash = {da08a6cc214983b969b5a435275a0a38},
keywords = {imported},
owner = {butz},
pages = {579-612},
publisher = {Springer-{V}erlag},
timestamp = {2009-06-26T15:25:54.000+0200},
title = {Exploring the Predictable},
year = 2002
}