Introduction
Distributed system are spread out over a network. Without communication between nodes, distributed system are simply
disconnected standalone system. The critical decision made in communication protocol in system design influences the
distributed system's performance, availability and scalability.
Pro tip: Network protocols are build on stacks. Each layer depends on the abstraction of the layer below it.
Every time you make an API call, send a message, or load a webpage, data travels through the stack from the top layer to
the bottom layer and vice versa on the other end of the network as shown in Figure 1.

Application Layer (Software Layer)
- High level protocols that provide services to applications.
- Here decisions related to user-facing protocols are made. like HTTP/HTTPS, DNS, gRPC, APIs, JSON-RPC, etc.
- HTTP/HTTPS: foundation of the web.
- DNS: Domain Name System: maps human-readable domain names to IP addresses.
- Your software code makes API calls to other services or databases.
Transport Layer
- The Core of the OSI Layered stack.
- End-to-end communication between nodes are established using port numbers and IP addresses.
- Here decisions related to transport protocols are made. like TCP, UDP, SCTP, etc.
- TCP: Transmission Control Protocol: provides reliable, ordered, and error-checked delivery of data. MOST
Important
- UDP: User Datagram Protocol: provides connectionless, unreliable, and fast delivery of data.
Internet Layer
- The Internet Layer is responsible for routing data packets between networks.
- It uses IP addresses to identify nodes and routes data packets to their destination.
- IP: Internet Protocol: delivers packets on a best effort basis. MOST Important
- Router operates at the Internet Layer and forwards packets based on routing tables.
Data Link Layer
- Physical layer of the OSI Layered stack.
- It consists of other network protocols that operate on services like Ethernet, Wi-Fi, etc.
- Switches operate at the Data Link Layer and forward ethernet frames based on destination MAC addresses.
- MAC: Media Access Control: assigns unique addresses to devices on a network.
- Frame: A unit of data that is transmitted over a network. It contains a header, a payload, and a footer.

Network Layers role in debugging distributed system
when a distributed system is not working, it is important to understand the network protocols used by the system. The
troubleshooting engineer might ask for the following information:
- what is the issue?
- Is it the Application Error (Application Layer)?
- Is the port available or blocked (Transport Layer)?
- Or a routing issue (Internet Layer)?
- Is network connectivity issue (Data Link Layer)?
Transmission Control Protocol (TCP)
TCP is a protocol to ensure reliable data exchange between two process on top of IP.
It guarantees that data is delivered in the correct order, without any gaps, duplicates and loss.

The process of sending data over TCP is as follows:
- Connection Establishment: The sender and receiver establish a stable connection using a three-way handshake.
- Data Transfer: The sender sends data to the receiver in segments (series of numbered packets).
- Data Confirmation: The receiver must send back an acknowledgment for every individual piece of data it successfully
gets. If a confirmation is not received, the system knows that the information was lost and must be resent.
- Connection Closure: The sender and receiver close the connection using a four-way handshake.
TCP Features
- Flow Control: TCP ensures that the sender does not send data faster than the receiver can process it.
- Congestion Control: TCP controls the amount of data sent to prevent network congestion.
Pro tip: TCP's reliability and stability come at the cost of lower bandwidth and higher latencies.
User Datagram Protocol (UDP)
If there is no need for reliability and stability in data exchange, UDP can be used to send data over the network.
For example, treaming and gaming services rely on UDP because real-time data has a short shelf life. If a packet is lost during transmission, retransmitting it serves no purpose—the game or stream has already moved on. By the time the retransmitted data arrives, it's outdated. This is where UDP excels: unlike TCP, which would insist on redelivering lost packets and introduce latency, UDP prioritizes speed over reliability, keeping the user experience smooth.
Security protocols in Transport Layer
TLS (Transport Layer Security) runs on top of TCP and provides secure communication channel so that application layer protocols like HTTP can be securely transmitted over the network.
TLS provide encryption, authentication and integrity to the data transmitted over the network.
Asymmetric encryption:
Client and server exchange public and private keys to encrypt and decrypt data.
The shared secret key is never transmitted over the network. Although asymmetric encryption is slow and expensive, it's only used to create the shared encryption key.
symmetric encryption:
Client and server priodically renegotiated to minimize amount of the data that
can be deciphered if shared secret key is broken. The shared secret key is transmitted over the network.
TLS implements authentication using digital signatures based on asymmetric cryptography.
The client has no idea whether the public key shared by the server is authentic, so we have certificates to prove the ownership of a public key for a specific entity.
A certificate authority (CA) is a trusted third party that issues digital certificates to verify the identity of a server. we have certificates to prove the ownership of a public key.
When it rides on top of TLS, it’s also referred to as HTTPS. You should use HTTPS by default.