Evolving intelligent system

Last updated

In computer science, an evolving intelligent system is a fuzzy logic system which improves the own performance by evolving rules. [1] The technique is known from machine learning, in which external patterns are learned by an algorithm. Fuzzy logic based machine learning works with neuro-fuzzy systems. [2]

Intelligent systems have to be able to evolve, self-develop, and self-learn continuously in order to reflect a dynamically evolving environment. The concept of Evolving Intelligent Systems (EISs) was conceived around the turn of the century [3] [4] [5] [6] [7] [8] [9] with the phrase EIS itself coined for the first time by Angelov and Kasabov in a 2006 IEEE newsletter [8] and expanded in a 2010 text. [9] EISs develop their structure, functionality and internal knowledge representation through autonomous learning from data streams generated by the possibly unknown environment and from the system self-monitoring. [10] EISs consider a gradual development of the underlying (fuzzy or neuro-fuzzy) system structure and differ from evolutionary and genetic algorithms which consider such phenomena as chromosomes crossover, mutation, selection and reproduction, parents and off-springs. The evolutionary fuzzy and neuro systems are sometimes also called "evolving" [11] [12] [13] which leads to some confusion. This was more typical for the first works on this topic in the late 1990s.

Implementations

EISs can be implemented, for example, using neural networks or fuzzy rule-based models. The first neural networks which consider an evolving structure were published in. [14] [15] [16] [17] [18] These were later expanded by N. Kasabov [5] and P. Angelov [3] [4] [6] [19] for the neuro-fuzzy models. P. Angelov [3] [4] [6] [7] introduced the evolving fuzzy rule-based systems (EFSs) as the first mathematical self-learning model that can dynamically evolve its internal structure and is human interpretable and coined the phrase EFS. Contemporarily, the offline incremental approach for learning an EIS, namely, EFuNN, was proposed by N. Kasabov. [20] [21] P. Angelov, D. Filev, N. Kasabov and O. Cordon organised the first IEEE Symposium on EFSs in 2006 (the proceedings of the conference can be found in [22] ). EFSs include a formal (and mathematically sound) learning mechanism to extract it from streaming data. One of the earliest and the most widely cited comprehensive survey on EFSs was done in 2008. [23] Later comprehensive surveys on EFS methods with real applications were done in 2011 [24] and 2016 [25] by E. Lughofer.

Other works that contributed further to this area in the following years expanded it to evolving participatory learning, [26] evolving grammar, [27] evolving decision trees, [28] evolving human behaviour modelling, [29] self-calibrating (evolving) sensors (eSensors), [30] evolving fuzzy rule-based classifiers, [31] [32] [33] [34] [35] evolving fuzzy controllers, [36] [37] autonomous fault detectors. [38] More recently, the stability of the evolving fuzzy rule-based systems that consist of the structure learning and the fuzzily weighted recursive least square [7] parameter update method has been proven by Rong. [39] Generalized EFS, which allow rules to be arbitrarily rotated in the feature space and thus to improve their data representability, have been proposed in [40] with significant extensions in [41] towards 'smartness' of the rule bases (thus, termed as "Generalized Smart EFS"), allowing more interpretability and reducing curse of dimensionality. The generalized rule structure was also successfully used in the context of evolving neuro-fuzzy systems. Several facets and challenges for achieving more transparent and understandable rule bases in EFS have been discussed by E. Lughofer in. [42]

EISs form the theoretical and methodological basis for the Autonomous Learning Machines (ALMA) [43] and autonomous multi-model systems (ALMMo) [44] as well as of the Autonomous Learning Systems. [10] Evolving Fuzzy Rule-based classifiers, [31] [32] [33] [34] [35] in particular, is a very powerful new concept that offers much more than simply incremental or online classifiers – it can cope with new classes being added or existing classes being merged. This is much more than just adapting to new data samples being added or classification surfaces being evolved. Fuzzy rule-based classifiers [34] are the methodological basis of a new approach to deep learning that was until now considered as a form of multi-layered neural networks. [45] Deep Learning offers high precision levels surpassing the level of human ability and grabbed the imagination of the researchers, industry and the wider public. However, it has a number of intrinsic constraints and limitations. These include:

  1. The "black box", opaque internal structure which has millions of parameters and involves ad hoc decisions on the number of layers and algorithm parameters.
  2. The requirement for a huge amount of training data samples, computational resources (usually requiring GPUs and/or HPC) and time (usually requiring many hours of training).
  3. Iterative search.
  4. Requires retraining for new situations (is not evolving).
  5. Does not have proven convergence and stability.

Most, if not all, of the above limitations can be avoided with the use of the Deep (Fuzzy) Rule-based Classifiers, [46] [47] which were recently introduced based on ALMMo, while achieving similar or even better performance. The resulting prototype-based IF...THEN...models are fully interpretable and dynamically evolving (they can adapt quickly and automatically to new data patterns or even new classes). They are non-parametric and, therefore, their training is non-iterative and fast (it can take few milliseconds per data sample/image on a normal laptop which contrasts with the multiple hours the current deep learning methods require for training even when they use GPUs and HPC). Moreover, they can be trained incrementally, online, or in real-time. Another aspect of Evolving Fuzzy Rule-based classifiers has been proposed in, [48] which, in case of multi-class classification problems, achieves the reduction of class imbalance by cascadability into class sub-spaces and an increased flexibility and performance for adding new classes on the fly from streaming samples. [49]

Related Research Articles

The expression computational intelligence (CI) usually refers to the ability of a computer to learn a specific task from data or experimental observation. Even though it is commonly considered a synonym of soft computing, there is still no commonly accepted definition of computational intelligence.

Bart Andrew Kosko is a writer and professor of electrical engineering and law at the University of Southern California (USC). He is a researcher and popularizer of fuzzy logic, neural networks, and noise, and the author of several trade books and textbooks on these and related subjects of machine intelligence. He was awarded the 2022 Donald O. Hebb Award for neural learning by the International Neural Network Society.

<span class="mw-page-title-main">Intelligent agent</span> Software agent which acts autonomously

In artificial intelligence, an intelligent agent (IA) is an agent acting in an intelligent manner; It perceives its environment, takes actions autonomously in order to achieve goals, and may improve its performance with learning or acquiring knowledge. An intelligent agent may be simple or complex: A thermostat or other control system is considered an example of an intelligent agent, as is a human being, as is any system that meets the definition, such as a firm, a state, or a biome.

<span class="mw-page-title-main">Neuro-fuzzy</span>

In the field of artificial intelligence, the designation neuro-fuzzy refers to combinations of artificial neural networks and fuzzy logic.

Evolving classification functions (ECF), evolving classifier functions or evolving classifiers are used for classifying and clustering in the field of machine learning and artificial intelligence, typically employed for data stream mining tasks in dynamic and changing environments.

In predictive analytics, data science, machine learning and related fields, concept drift or drift is an evolution of data that invalidates the data model. It happens when the statistical properties of the target variable, which the model is trying to predict, change over time in unforeseen ways. This causes problems because the predictions become less accurate as time passes. Drift detection and drift adaptation are of paramount importance in the fields that involve dynamically changing data and data models.

Computational neurogenetic modeling (CNGM) is concerned with the study and development of dynamic neuronal models for modeling brain functions with respect to genes and dynamic interactions between genes. These include neural network models and their integration with gene network models. This area brings together knowledge from various scientific disciplines, such as computer and information science, neuroscience and cognitive science, genetics and molecular biology, as well as engineering.

In recent years, the use of biologically inspired methods such as the evolutionary algorithm have been increasingly employed to solve and analyze complex computational problems. BELBIC is one such controller which is proposed by Caro Lucas, Danial Shahmirzadi and Nima Sheikholeslami and adopts the network model developed by Moren and Balkenius to mimic those parts of the brain which are known to produce emotion.

The IEEE Systems, Man, and Cybernetics Society is a professional society of the IEEE. It aims "to serve the interests of its members and the community at large by promoting the theory, practice, and interdisciplinary aspects of systems science and engineering, human-machine systems, and cybernetics".

Morphogenetic robotics generally refers to the methodologies that address challenges in robotics inspired by biological morphogenesis.

An adaptive neuro-fuzzy inference system or adaptive network-based fuzzy inference system (ANFIS) is a kind of artificial neural network that is based on Takagi–Sugeno fuzzy inference system. The technique was developed in the early 1990s. Since it integrates both neural networks and fuzzy logic principles, it has potential to capture the benefits of both in a single framework.

Fusion adaptive resonance theory (fusion ART) is a generalization of self-organizing neural networks known as the original Adaptive Resonance Theory models for learning recognition categories (or cognitive codes) across multiple pattern channels. There is a separate stream of work on fusion ARTMAP, that extends fuzzy ARTMAP consisting of two fuzzy ART modules connected by an inter-ART map field to an extended architecture consisting of multiple ART modules.

This glossary of artificial intelligence is a list of definitions of terms and concepts relevant to the study of artificial intelligence, its sub-disciplines, and related fields. Related glossaries include Glossary of computer science, Glossary of robotics, and Glossary of machine vision.

In computer science, incremental learning is a method of machine learning in which input data is continuously used to extend the existing model's knowledge i.e. to further train the model. It represents a dynamic technique of supervised learning and unsupervised learning that can be applied when training data becomes available gradually over time or its size is out of system memory limits. Algorithms that can facilitate incremental learning are known as incremental machine learning algorithms.

Plamen P. Angelov is a computer scientist. He is a chair professor in Intelligent Systems and Director of Research at the School of Computing and Communications of Lancaster University, Lancaster, United Kingdom. He is founding Director of the Lancaster Intelligent, Robotic and Autonomous systems (LIRA) research centre. Angelov was Vice President of the International Neural Networks Society of which he is now Governor-at-large. He is the founder of the Intelligent Systems Research group and the Data Science group at the School of Computing and Communications. He is member of the Board of Governors also of the Systems, Man and Cybernetics Society of the IEEE for two terms (2015-2017) and (2022-2024). Prof. Angelov was named Fellow of the Institute of Electrical and Electronics Engineers (IEEE) in 2016 for contributions to neuro-fuzzy and autonomous learning systems. He is also a Fellow of ELLIS and the IET. Dr. Angelov is a founding co-Editor-in-chief of the Evolving Systems journal since 2009 as well as associate editor of the IEEE Transactions on Cybernetics, IEEE Transactions on Fuzzy Systems, IEEE Transactions on AI, Complex and Intelligent Systems and other scientific journals. He is recipient of the 2020 Dennis Gabor Award as well as IEEE and INNS awards for Outstanding Contributions, The Engineer 2008 special award and others. Author of over 400 publications including 3 research monographs, 3 granted US patents, over 120 articles in peer reviewed scientific journals, over 160 papers in peer reviewed conference proceedings, etc. These publications were cited over 15000 times, h=index 63. His research contributions are centred around autonomous learning systems, Wiley, 2012, dynamically self-evolving systems and the empirical approach to machine learning, Springer Nature, 2012. Most recently, his research is addressing the problems of interpretability and explainability, xDNN, 2020, catastrophic forgetting, continual learning, ability to adapt, computational and energy costs of deep foundation models and their whole life cycle.

<span class="mw-page-title-main">Amir Hussain (cognitive scientist)</span>

Amir Hussain is a cognitive scientist, the director of Cognitive Big Data and Cybersecurity (CogBID) Research Lab at Edinburgh Napier University He is a professor of computing science. He is founding Editor-in-Chief of Springer Nature's internationally leading Cognitive Computation journal and the new Big Data Analytics journal. He is founding Editor-in-Chief for two Springer Book Series: Socio-Affective Computing and Cognitive Computation Trends, and also serves on the Editorial Board of a number of other world-leading journals including, as Associate Editor for the IEEE Transactions on Neural Networks and Learning Systems, IEEE Transactions on Systems, Man, and Cybernetics (Systems) and the IEEE Computational Intelligence Magazine.

Javier Andreu-Perez is a British computer scientist and a Senior Lecturer and Chair in Smart Health Technologies at the University of Essex. He is also associate editor-in-chief of Neurocomputing for the area of Deep Learning and Machine Learning. Andreu-Perez research is mainly focused on Human-Centered Artificial Intelligence (HCAI). He also chairs a interdisciplinary lab in this area, HCAI-Essex.

<span class="mw-page-title-main">Michael Berthold</span> German computer scientist

Michael R. Berthold is a German computer scientist, entrepreneur, academic and author. He is a professor, and chair for bioinformatics and information mining at Konstanz University, and an honorary professor at Óbuda University. He is also the co-founder of KNIME, and is serving as a president and CEO of KNIME AG since 2017.

References

  1. ↑ Nikola K. Kasabov (23 August 2007). Evolving Connectionist Systems: The Knowledge Engineering Approach. Springer Science & Business Media. p. 9. ISBN   978-1-84628-347-5.
  2. ↑ Plamen P. Angelov; Xiaowei Gu (17 October 2018). Empirical Approach to Machine Learning. Springer. pp. 51–. ISBN   978-3-030-02384-3.
  3. 1 2 3 P. Angelov and R. Buswell, "Evolving Rule-based Models: A Tool for Intelligent Adaption," in IFSA world congress and 20th NAFIPS international conference, 2001, pp. 1062–1067.
  4. 1 2 3 P. P. Angelov,Evolving rule-based models: a tool for design of flexible adaptive systems. Springer Berlin Heidelberg, 2002.
  5. 1 2 N. K. Kasabov and Q. Song, "DENFIS : Dynamic Evolving Neural-Fuzzy Inference System and Its Application for Time-Series Prediction," IEEE Trans. Fuzzy Syst., vol. 10, no. 2, pp. 144–154, 2002.
  6. 1 2 3 P. Angelov, D. Filev, "On-line Design of Takagi-Sugeno Models." In: Bilgiç T., De Baets B., Kaynak O. (eds) Fuzzy Sets and Systems — IFSA 2003. IFSA 2003. Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence), vol 2715. Springer, Berlin, Heidelberg
  7. 1 2 3 P. P. Angelov and D. P. Filev, "An approach to online identification of Takagi-Sugeno fuzzy models," IEEE Trans. Syst. Man, Cybern. - Part B Cybern., vol. 34, no. 1, pp. 484–498, 2004.
  8. 1 2 P. Angelov, N. Kasabov, eIS: Evolving Intelligent Systems, 2006, In: IEEE SMC eNewsLetter, Vol. 15, 2006, p. 1-13.
  9. 1 2 P. Angelov, D. Filev, N. Kasabov, Evolving Intelligent Systems: Methodology and Applications, Wiley-IEEE Press, 2010.
  10. 1 2 P. Angelov, Autonomous learning systems: from data streams to knowledge in real time. John Wiley & Sons, Ltd., 2012.
  11. ↑ B. Carse, T. C. Fogarty and A. Munro, "Evolving fuzzy rule based controllers using genetic algorithms." Fuzzy sets and systems, 80(3), pp. 273-293, 1996.
  12. ↑ N. Kasabov, "Evolving Fuzzy Neural Networks—Algorithms, Applications and Biological Motivation", in Methodologies for the conception, design and application of soft computing, World Scientific, pp. 271– 274, 1998.
  13. ↑ P. P. Angelov, "Evolving Fuzzy Rule-Based Models", Proc. 8th IFSA World Congress, Taiwan, vol.1, pp.19-23, 1999.
  14. ↑ T. Martinetz and K. Schulten. "A "neural gas" network learns topologies" Artificial Neural Networks. Elsevier. pp. 397–402, 1991.
  15. ↑ B. Fritzke, "A growing neural gas network learns topologies." Advances in neural information processing systems. 1995.
  16. ↑ C. F. Juang and C. T. Lin, "An online self-constructing neural fuzzy inference network and its applications." in IEEE transactions on Fuzzy Systems, vol.6 no. 1, pp.12-32, 1998.
  17. ↑ S. Wu and M. J. Er, "Dynamic fuzzy neural networks-a novel approach to function approximation". in IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 30, no. 2, pp. 358-364, 2000.
  18. ↑ S. Wu, M. J. Er and Y. Gao, "A fast approach for automatic generation of fuzzy rules by generalized dynamic fuzzy neural networks". in IEEE Transactions on Fuzzy Systems, vol. 9, no. 4, pp. 578-594, 2001.
  19. ↑ P. P. Angelov, and D. P. Filev, "Flexible models with evolving structure." in: Proc. First International IEEE Symposium "Intelligent Systems", v. II, pp.28-33, IEEE Press, ISBN   0-7803-7134-8/02, 2002.
  20. ↑ N. Kasabov, "Evolving fuzzy neural networks-algorithms, applications and biological motivation." in Methodologies for the conception, design and application of soft computing, World Scientific, pp.271-274, 1998.
  21. ↑ N. Kasabov, "Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning." in IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 31, no.6, pp. 902-918, 2001.
  22. ↑ Proceedings of 2006 International Symposium on Evolving Fuzzy Systems, ISBN   0-7803-9718-5, IEEE Catalog number 06EX1440.
  23. ↑ P. P. Angelov, "Evolving fuzzy systems," http://www.scholarpedia.org/article/Evolving_fuzzy_systems, 2008.
  24. ↑ E. Lughofer, Evolving Fuzzy Systems: Methodologies, Advanced Concepts and Applications. Studies in Fuzzy and Soft Computing, Springer, 2011.
  25. ↑ E. Lughofer, Evolving Fuzzy Systems --- Fundamentals, Reliability, Interpretability and Useability. Handbook of Computational Intelligence, editor. P. P. Angelov, World Scientific, pp. 67-135, 2016.
  26. ↑ E. Lima, F. Gomide and R. Ballini, "Participatory Evolving Fuzzy Modeling," in IEEE International Symposium on Evolving Fuzzy Systems, 2006, pp. 36-41.
  27. ↑ N. M. Sharef, T. Martin, and Y. Shen, "Order independent incremental evolving fuzzy grammar fragment learner," in IEEE International Conference on Intelligent Systems Design and Applications, 2009, pp. 1221–1226.
  28. ↑ A. Shaker, R. Senge, and E. Hüllermeier, "Evolving fuzzy pattern trees for binary classification on data streams," Inf. Sci. (Ny)., vol. 220, pp. 34–45, 2013.
  29. ↑ J. A. Iglesias, P. Angelov, A. Ledezma and A. Sanchis, "Creating Evolving User Behavior Profiles Automatically," in IEEE Transactions on Knowledge and Data Engineering, vol. 24, no. 5, pp. 854-867, 2012.
  30. ↑ P. Angelov and A. Kordon, "Adaptive Inferential Sensors Based on Evolving Fuzzy Models," in IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 40, no. 2, pp. 529-539, 2010.
  31. 1 2 C. Xydeas, P. Angelov, S. Chiao, and M. Reoullas, "Advances in classification of EEG signals via evolving fuzzy classifiers and dependant multiple HMMs." in Computers in Biology and Medicine, vol. 3, no.10, pp. 1064-1083, 2006.
  32. 1 2 P. Angelov, X. Zhou and F. Klawonn, "Evolving Fuzzy Rule-based Classifiers," in IEEE Symposium on Computational Intelligence in Image and Signal Processing, 2007, pp. 220-225.
  33. 1 2 P. Angelov, E. Lughofer and X. Zhou "Evolving fuzzy classifiers using different model architectures," in Fuzzy Sets and Systems, vol. 159, no. 23, pp. 3160-3182, 2008.
  34. 1 2 3 P. P. Angelov and X. Zhou, "Evolving Fuzzy-Rule-Based Classifiers From Data Streams," in IEEE Transactions on Fuzzy Systems, vol. 16, no. 6, pp. 1462-1475, 2008.
  35. 1 2 P. Angelov, X. Zhou, D. Filev and E. Lughofer, "Architectures for evolving fuzzy rule-based classifiers," in IEEE International Conference on Systems, Man and Cybernetics, 2007, pp. 2050-2055.
  36. ↑ D. Dovžan, V. Logar and I. Škrjanc, "Implementation of an Evolving Fuzzy Model (eFuMo) in a Monitoring System for a Waste-Water Treatment Process," in IEEE Transactions on Fuzzy Systems, vol. 23, no. 5, pp. 1761-1776, 2015.
  37. ↑ P. Angelov, I. Škrjanc and S. Blažič, "Robust evolving cloud-based controller for a hydraulic plant," in IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS), Singapore, 2013, pp. 1-8.
  38. ↑ B.S.J. Costa, P.P. Angelov and L.A. Guedes, "Real-Time Fault Detection Using Recursive Density Estimation" J Control Autom Electr Syst, vol. 25, no. 4, pp. 428–437, 2014.
  39. ↑ H. Rong, P. Angelov, X. Gu and J. Bai, "Stability of Evolving Fuzzy Systems based on Data Clouds, IEEE Transactions on Fuzzy Systems" DOI: 10.1109/TFUZZ.2018.2793258, 2018.
  40. ↑ A. Lemos, W. Caminhas and F. Gomide, "Multivariable Gaussian Evolving Fuzzy Modeling System, IEEE Transactions on Fuzzy Systems", vol. 19 (1), pp. 91-104, 2011.
  41. ↑ E. Lughofer, C. Cernuda, S. Kindermann and M. Pratama, "Generalized Smart Evolving Fuzzy Systems, Evolving Systems", vol. 6 (4), pp. 269-292, 2015.
  42. ↑ E. Lughofer, "On-line Assurance of Interpretability Criteria in Evolving Fuzzy Systems --- Achievements, New Concepts and Open Issues, Information Sciences, vol. 251, pp. 22-46, 2013.
  43. ↑ P.P. Angelov, "Autonomous Machine Learning (ALMA): generating rules from data streams" in Special International Conference on Complex Systems, 2011, pp. 249-256.
  44. ↑ P.P. Angelov, X Gu, J Príncipe, "Autonomous learning multi-model systems from data streams", in IEEE Transactions on Fuzzy Systems, DOI:10.1109/TFUZZ.2017.2769039, 2017.
  45. ↑ I. Goodfellow, Y. Bengio, and A. Courville, Deep learning. Crambridge, MA: MIT Press, 2016.
  46. ↑ P. Angelov, X. Gu, "MICE: Multi-layer multi-model images classifier ensemble," in IEEE International Conference on Cybernetics (CYBCONF), 2017, pp. 1-8.
  47. ↑ P. Angelov, X. Gu, "A cascade of deep learning fuzzy rule-based image classifier and SVM," in IEEE International Conference on Systems, Man, and Cybernetics (SMC2017), 2017, pp.746-751
  48. ↑ E. Lughofer, O. Buchtala, "Reliable All-Pairs Evolving Fuzzy Classifiers," in IEEE Transactions on Fuzzy Systems, vol. 21 (4), pp. 625-641, 2013.
  49. ↑ E. Lughofer and E. Weigl and W. Heidl and C. Eitzinger and T. Radauer, "Integrating New Classes On the Fly in Evolving Fuzzy Classifier Designs and its Application in Visual Inspection," in Applied Soft Computing, vol. 35, pp. 558-582, 2015
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.