Imagine that 3 billion smartphones, servers and IoT devices are all functioning in their environment without any clear understanding of the overall process. This is the day-to-day experience of distributed computing, the invisible backbone of Google’s search results, Netflix’s streaming service and your bank’s transaction system.
Whenever you send a message or check your account balance, dozens of machines are working harmoniously behind the scenes to make possible an instant response. In this blog, we’ll explore exactly what distributed computing is, how distributed systems operate and why it is so important in today’s cloud first world. Daily Techify is where we take the difficult things and make them easy. Let’s dive in.
What is Distributed Computing?
Distributed Computing is a computing paradigm where a set of independent computers (also called nodes) are used for doing a single computation or executing an application The system uses many, rather than one, powerful machines, connected over a network that share their results.
It is like a team project, everyone will take care of one part and then put everyone’s work together and create one final project. This is the essence of distributed computing: breaking up a large computation into smaller chunks, sending them to multiple machines and combining the results.
According to IDC, more than 75% of enterprise workloads are now deployed on distributed cloud infrastructure, compared with 40% in 2019. This is an indication of why this idea will not seem like it’s going anywhere soon.
What is a Distributed System?
Distributed systems is a collection of independent computers that seem to the users to be a unified system. In each node, its own operating system and its own memory are used and no computer shares RAM directly with another computer.
In these systems, communication takes place via messages rather than shared memory. The nodes share information via the network, such as TCP/IP, RPC or HTTP protocols and cooperate to ensure consistency even in the event of failure in parts of the network.
There are three defining characteristics of nearly all distributed systems:
- No shared memory: every node has its own local memory (local database)
- Communication: based on messages (nodes communicate through a network)
- Independent failure: if one node fails, the system will continue to run.
How Distributed Systems Work (Distributed Computing Diagram)
Imagine a restaurant kitchen with five cooks, each preparing one dish, each communicating with the others by the head chef’s orders and directions. That coordination pattern is the same as the nodes behave in a distributed diagram.
The architectural design used in distributed computing can be either of the following: the client-server architecture and the peer-to-peer architecture. The client-server architecture is a type of architecture where the client sends a request to the central server that serves the requests and provides responses. For instance, your web browser requests a webpage. P2P stands for peer-to-peer architecture and it is a type of architecture whereby each node is a client and server and exchanges resources directly with one another.
Simple distributed computing diagram:
This is the design for a client-based load-balanced application running on one or more server nodes with its shared data and storage layer.
Requests are sent via a load balancer through to available nodes, reaching and moving data to a common/replicated storage layer, ensuring the system remains responsive even during heavy traffic.
Distributed Computing Environment
A distributed computing environment combines three key components: hardware, software and network infrastructure. Hardware: the physical or virtual machines on which the workload is executed. Software consists of middleware, OS and Application logic to facilitate node coordination. The network layer links everything and moves the messages that synchronize nodes.
Middleware is crucial within that context. It sits between the operating system and applications and performs functions such as authentication, load balancing, and data serialization and thus can be used by developers without having to do these tasks at the lower level of the networking implementation.
Communication is not the only thing that matters coordination is also important. To keep nodes synchronized in a distributed environment, techniques such as clock synchronization (NTP), consensus algorithms (Raft, Paxos) and distributed locks are used.
System Models for Distributed and Cloud Computing
Each distributed system has a model, which dictates interaction, failure and security of components. These can be categorized into a few general types, collectively known as system models for distributed and cloud computing. Different architectures have different bells and whistles: client-server, peer-to-peer and layered.
In synchronous systems, the timing of the interaction is assumed, whereas in asynchronous systems, this is not assumed more accurately reflecting real-life internet conditions.
Failure models describe what happens when a system fails, from a simple crash failure to the more dangerous “Byzantine” failure in which the node fails in unpredictable ways or in a malicious manner. Security models make data more secure when it is being transferred between nodes by including authentication, encryption and access control mechanisms. The cloud computing system models are based on the same principles as distributed and cloud computing with elastic scaling and pay-as-you-go resource allocation on top.
Distributed Systems: Principles and Paradigms
Nearly every well-designed distributed system is based on 4 principles and these four principles continue to be taught in computer science courses in the classic Distributed Systems Principles and Paradigms book by Tanenbaum and van Steen.
Scalability allows for the system to scale up by adding more nodes instead of just upgrading one node. A fault-tolerance feature ensures that the system remains functional despite the failure of one node or more usually through the use of replication and redundancy.
Concurrency: Execute multiple processes simultaneously without damaging the shared data.
Transparency hides the intricacy of the distributed system so that the users can work with the system as though it is a unitary system.
From the paradigm perspective, the programmers use the following techniques to ensure that communication between the nodes takes place: Remote Procedure Call (RPC), Distributed Object (for example, CORBA), Message-Oriented Middleware (for example, Kafka and RabbitMQ).
Types of Distributed Applications
With a distributed computing application, each component of the application is executed on several interconnected computers rather than on a single computer, with the components coordinated to operate as a single application.
Cassandra and MongoDB are examples of distributed databases that replicate data across multiple servers, allowing for data to survive even if one server fails.
Distributed file systems, like Hadoop HDFS, partition large files into blocks and distribute them across a cluster, allowing organizations to process petabytes of data in parallel.
Microservices architectures involve dividing a single distributed application into smaller and smaller services that can be deployed separately and communicate with each other via an API (Netflix uses more than 1,000 microservices to power its streaming platform).
Distributed Computing Examples
This concept clicks quicker than any textbook definition can ever convey with real-world distributed computing examples.
Google Search performs more than 8.5 billion searches per day and uses thousands of distributed servers indexing and ranking web pages concurrently in data centers all over the world.
Blockchains such as Bitcoin share one single ledger among a network of over 15,000 nodes active worldwide with no single party controlling the data.
One of the earliest volunteer computing initiatives was SETI@home, where the processing power of more than five million home computers was leveraged to find signs of alien life in radio signals.
The Content Delivery Network (CDN), for example, Cloudflare, hosts the content at its data centers located in 300+ countries and delivers the data through the server nearest to the visitor, thereby cutting down the page load time.
Netflix and Uber use distributed database systems, such as Cassandra and MongoDB clusters that can perform millions of read/write transactions per second.
Benefits and Challenges of Distributed Computing
All the technologies have some benefits and challenges of distributed computing is no exception.
The upside is that they’re scalable, they can add nodes as needed rather than purchasing larger hardware. Moreover, they are also fault-tolerant, which means that the system still operates in case there is any hardware malfunction. The feature of resource sharing means that more than one application can share resources, and the performance is higher in the case where operations are executed in parallel rather than in sequence.
The challenge side becomes much more complex if more nodes are added to the system. Security is more difficult to manage in multiple entry points and latency delays can impact communications between geographically remote nodes. Multi-node consistency problems also can occur due to multiple nodes updating the same data and require special conflict-resolution strategies.
Conclusion
Distributed computing is no longer some abstract computer-science theory it’s the technology that makes just about every app, website and service that people use every day possible. Distributed system are at work in the digital world we use without thinking from Google’s search engine processing billions of queries to blockchain networks operating without any central authority.
Whether you’re developing, a student or just a curious reader, you’ll get a real boost from understanding these fundamentals from the distributed computing in environment to the system model for distributed computing and cloud computing. Continue the conversations about such topics with Daily Techify making technology into easy-to-understand, do-able knowledge.
FAQs
Difference between distributed computing and cloud computing?
Distributed computing refers to the division of work into several machines and cloud computing provides computing services over the Internet. Distributed computing is used inside cloud platforms to provide a scalable service.
Briefly explain the difference between parallel and distributed computing.
Multiprocessor computing is the execution of several processes at the same time on a single machine having a shared memory. Distributed computing is a method that performs processes on multiple machines that share no memory but communicate over the network.
What is the difference between distributed computing and distributed systems?
Distributed computing is used by various systems to achieve global scale, such as Google Search, blockchain networks, CDNs such as Cloudflare and distributed databases, such as Cassandra.
What are the reasons for using distributed computing instead of a single big machine?
A single point of failure means that a single server would cause a problem in the system and would not scale up. Distributed computing distributes the load, provides redundancy, performance and space for expansion without interruption.
