Modern Approaches to Data Storage on a Distributed Infrastructure

Abstract

The purpose of this article is to study the problems of optimal big data storage and consider the latest approaches to solving these problems to ensure the required level of performance of computing systems. During the scientific research, methods of analysis and generalization, synthesis and system approach were used to study modern methods of data storage on a distributed infrastructure, which are used in the process of information processing. The paper considers the key aspects that arise with the development of technology and a significant increase in the amount of data used. It is established that today there are two fundamental approaches to ensuring the required level of system performance: vertically (approach with shared-memory architecture) and horizontally (shared-nothing architectures) scalable approaches, each of which has both positive and negative sides. According to the latest developments and research in the field of working with big data, the most suitable for solving problems is an architecture without separation of resources, which is analyzed in detail in the work, namely, what are the main conceptual and architectural solutions of distributed systems and the requirements for this architecture are indicated. With this architecture, each node (the machine or virtual machine running the database software) uses its own CPU, memory and disks separately and any coordination between nodes is carried out at the software level using a conventional network. In this approach to data storage on several nodes, depending on the volume of this data, two most common methods are used: replication (in the case of a small data set and the possibility of placing it on one computer) or partitioning (sharding), which includes dividing data into parties based on a specific range of key values in the data set. The need to achieve an appropriate balance between such principles as consistency, accessibility and resistance to distribution in the design of distributed systems is indicated. The statement about the possibility of having at most two of these properties for any distributed data system is substantiated.