Abstract
PANIC (Parallelism And Neural networks In Classifier systems), an Evolutionary Rule Based System (ERBS) to evolve behavioral strategies codified by sets of rules, is presented. PANIC assigns credit to the rules through a new mechanism, Q-Credit Assignment (QCA), based on Q-learning. By taking into account the context where a rule is applied, QCA is more accurate than classical methods when a single rule can fire in different situations. QCA is implemented through a multi-layer feed-forward neural network.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
P. J. Angeline, G. M. Saunders, and J. B. Pollack. An evolutionary algorithm that constructs recurrent neural networks. IEEE Transaction on Neural Networks, 1994.
A. G. Barto. R. S. Sutton, and C. J. C. H. Watkins. Learning and sequential decision making. Technical Report COINS 89-95, Comp. and Inform. Sci., Univ. Mass., 1989.
R. K. Belew, J. McInerney, and N. N. Schraudolph. Evolving networks: using the genetic algorithm with connectionist learning. Technical Report CSE CS90-174, Comp. Sci. and Engr., Univ. California, 1990.
D. J. Chalmers. The evolution of learning: An experiment in genetic connectionism. In Proceedings of the 1990 Connectionist Models Summer School, 1990.
A. Giani. Un nuovo approccio alla definizione e all'implementazione di siatemi a classificatori. Master's thesis. Dip. di Informatica. University of Pisa, Italy, 1992.
J. J. Grefenstette. Credit assignment in rule discovery systems based on genetic algorithms. Machine Learning, 3(2–3), 1988.
J. J. Grefenstette. A system for learning control strategies with genetic algorithms. In Proceedings of the Third International Conference on Genetic Algorithms and Their Applications. 1989.
J. J. Grefenstette. Lamarckian learning in multi-agent environments. In Proceedings of the Fourth International Conference on Genetic Algorithms and Their Applications, 1991.
J. H. Holland. Escaping brittleness: The possibilities of general-purpose learning algorithm applied to parallel rule-based systems, volume 2 of Machine learning: An artificial inteligencc approach. Morgan Kaufmann, 1986.
J. Koza. Genetic programming: On the programming of computers by the means of natural selection. MIT Press, 1992.
L. Lin. Self-improving reactive agents: Case studies of reinforcement learning frameworks. In From Animals to Animals: Proceedings of the First International Conference on Simulation of Adaptive Behaviour, 1990.
R. L. Riolo. Empirical studies of default hierarchies and sequences of rules in learning classifier systems. PhD thesis, University of Michigan, 1988.
G. G. Robertson and R. L. Riolo. A tale of two classifier systems. Machine Learning, 3, 1988.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representation by error propagation, volume 1 of Parallel Distributed Processing. MIT Press, 1986.
S. F. Smith. A learning system based on genetic adaptive algorithms. PhD thesis, University of Pittsburgh, 1980.
R. S. Sutton. Learning to predict by the methods of temporal differences. Machine Learning, 3, 1988.
R. S. Sutton. Reinforcement learning architectures for animats. In From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behaviour, 1990.
G. J. Tesauro. Practical issues in temporal difference learning. Technical Report RC 17223, IMB T. J. Watson Research Center, Yorktown Heights, NY, 1991.
C. J. C. H. Watkins. Learning with delayed rewards. PhD thesis, University of Cambridge, England, 1989.
T. H. Westerdale. A defence of the bucket brigade. In Proceedings of the Third International Conference on Genetic Algorithms and Their Applications, 1989.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1994 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Giani, A., Baiardi, F., Starita, A. (1994). Q-learning in Evolutionary Rule Based Systems. In: Davidor, Y., Schwefel, HP., Männer, R. (eds) Parallel Problem Solving from Nature — PPSN III. PPSN 1994. Lecture Notes in Computer Science, vol 866. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-58484-6_271
Download citation
DOI: https://doi.org/10.1007/3-540-58484-6_271
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-58484-1
Online ISBN: 978-3-540-49001-2
eBook Packages: Springer Book Archive