CAPSULE NEURAL NETWORKS
DOI:
https://doi.org/10.26906/SUNZ.2018.5.132Keywords:
deep learning, capsule neural networks, convolutional neural networksAbstract
The subject of study is the history of the formation and development of the theory of neural networks, modern approaches to the problems of recognition and classification of images. Particular attention is paid to the qualitative review of capsular and convolutional neural networks, the principles of their work and the identification of the main differences. The aim of the work is to analyze the current state of neural network research and possible prospects for the development of this industry. Objective: to analyze the historical development of the theory of neural networks. Conduct a comparison between types of neural networks based on the concept of deep learning: convolutional and capsule. The method of conducting the research is an analysis of modern literature and the main trends of the development of deep learning. The results of the study are the discovery of significant openings that have influenced the development of neural networks. Functioning of neural networks is based on the work of the nervous system of biological organisms. In particular, this is the principle of the activity of the biological neuron, ensembles of neurons, the discovery of "simple cells" in the visual cortex of the brain. Currently, neural networks based on the concept of deep learning, which allows multilayer computing models to study data with several levels of abstraction, are the most developed. Convolutional networks that use this concept have achieved significant success in recognizing images, videos and audio. Recurrent networks have appeared in the analysis of text and language. Convolutional Neural Networks have a number of shortcomings that are highlighted in the work. Capsule neural networks are an improvement in the concept of convolutional neural networks. They are based on "capsules", which are intended to detect the characteristics of the object. Capsules as a group of neurons are characterized by an activation vector. The vector approach proposed by researchers allows taking into account the rotation and translation of objects. Capsule neural networks require a much smaller training sample than convolutional. The conclusions of the work determine the main prospects for the development of the theory of neural networks, as well as the possible rapid development of uncontrolled training of neural networks. It is emphasized the importance of critical analysis of the problems of neural networks as a decisive factor for their future development.Downloads
References
Rashid T. Make your own neural network. – CreateSpace Independent Publishing Platform, 2016.
McCulloch W. S., Pitts W. A logical calculus of the ideas immanent in nervous activity/ W. S. McCulloch//The bulletin of mathematical biophysics. – 1943. – V. 5. – No. 4. – P. 115-133.
Rochester N., J.R. Holland, L.R. Haibt and W.L. Duda. Tests оn а сеll assembly theory of the action of the brain, using а large digital computer/ N. Rochester//IRE Transactions оn Information Theory. – 1956. - V. IТ-2. - P. 80-93.
Caianiello E.R. Outline of а theory of thought-processes and thinking machines //J.Theor.Biology. – 1961. – V. l. - P. 204-235.
Anderson J.A., Pellionisz А. eds. Neurocomputing 2: Directions for Research, Cambridge, МА: MIТ Press, 1990.
Minsky M.L. and S.A. Papert. Perceptrons, expanded edition, Cambridge, МА: MIТ Press, 1988.
Bryson А.Е., Jr. and У.С. Но. Aplied Optimal Control, Blaisdell, 1969. (Second Edition, 1975, Hemisphere pиblishing, Washington, ОС).
von Neumann J., Shannon C.E., McCarthy J. eds. Probabilistic logics and the synthesis of reliable organisms from unreliable components, in Automata Studies/ J. von Neumann//Princeton, NJ: Princeton University Press. -1956. - P. 43-98.
Rosenblatt F. The Perceptron: А probabilistic model for information storage and organization in the brain/ F. Rosenblatt//Psychological Review. - 1958, V. 65. - P. – 386-08.
Rosenblatt F. Оn the convergence of reinforcement procedures in simple perceptions/ F. Rosenblatt//Соmеll Aeronautical Laboratory Report. – 1960. – VG - 1196 - GA.
Hopfield J.J. Neural networks and physical systems with emergent collective computational abilities/ J. J. Hopfield//Proceedings of the National Academy of Sciences, USA. – 1982. - V. 79. - P. – 2554-2558.
Cowan J.D. А Mathematical Theory of Central Nervous Activity. Ph.D. Thesis, University of London, 1967.
Rumelhart О.Е., G.E. Hinton and R.J. Williams. Leaning representations of back-propagation еrrors/ O.E. Rumelhart//Nature (London). – 1986. - V. 323. - P. – 533-536.
LeCun Y. Backpropagation applied to handwritten zip code recognition // Neural computation. – 1989. – V. 1, No. 4. – P. 541-551.
Sabour S., Frosst N., Hinton G. E. Dynamic routing between capsules/ S. Sabour, N. Frosst, G. E. Hinton//Advances in Neural Information Processing Systems. – 2017. – P. 3856-3866.
LeCun Y., Bengio Y., Hinton G. Deep learning // Nature. – 2015. – V. 521, No 7553. – P. 436 – 444.
Taigman, Y., Yang, M., Ranzato, M. Deepface: closing the gap to human-level performance in face verification // Proc. Conference on Computer Vision and Pattern Recognition 1701–1708 (2014).
