K.Boopathi
BlogNotesSecond BrainProjectsHire MeYour reading file0 pts

Microservice Guide

Nov 02, 202510 min read

Communication

Types:

  • Synchronous: HTTP, gRPC, API Gateway
  • Asynchronous: Message Queues, Event Bus, Change Data Capture

How netflix using Envoy and service mesh

REST/HTTP

  • Protocol: HTTP/HTTPS
  • Format: JSON (typically)
  • Paradigm: Stateless, resource-based
  • Best For: Simple CRUD operations, external APIs
  • Pros: Widely understood, easy to debug, browser-friendly
  • Cons: Higher latency due to HTTP/1.1, text-based overhead

Tools: Postman, REST-assured, WireMock

gRPC

  • Protocol: HTTP/2 with binary serialization
  • Format: Protocol Buffers (Protobuf)
  • Paradigm: RPC-based, strongly typed
  • Best For: High-performance internal service communication
  • Pros:
    • Lower latency and bandwidth usage
    • Built-in support for streaming (unary, client, server, bidirectional)
    • Automatic code generation in multiple languages
    • Multiplexing and header compression via HTTP/2
  • Cons: Learning curve, less human-readable, limited browser support

Tools: gRPC official libraries, Envoy proxy for load balancing

GraphQL

  • Protocol: HTTP/HTTPS
  • Format: Strongly typed schema
  • Paradigm: Query language for APIs
  • Best For: Reducing over-fetching/under-fetching, complex queries
  • Pros: Clients request exactly what they need
  • Cons: More complex on backend, potential for performance issues if not optimized

Tools: Apollo Server, AWS AppSync

Asynchronous Communication

Services communicate through message brokers without waiting for responses.

Message Queues

  • RabbitMQ:

    • Architecture: Broker-based, AMQP protocol
    • Throughput: 4K-10K messages/sec
    • Latency: Low (optimized for real-time messaging)
    • Model: Push-based (broker sends to consumers)
    • Best For: Task scheduling, job queues, request-response patterns
    • Features: Flexible routing (exchanges, bindings), message durability, clustering

  • Apache Kafka:

    • Architecture: Distributed log-based streaming platform
    • Throughput: 1M+ messages/sec
    • Model: Pull-based (consumers fetch from brokers)
    • Best For: Event-driven architectures, real-time analytics, event sourcing
    • Features: Message replay, partitioning for parallelism, high scalability, geo-replication
    • Companies Using: LinkedIn, Twitter, Netflix, Uber

API Gateway

Acts as a single entry point for all client requests, routing them to appropriate microservices while handling:

  • Authentication & authorization
  • Rate limiting & throttling
  • Request/response transformation
  • Load balancing
  • API versioning
  • Monitoring & logging

Popular API Gateways: Kong, AWS API Gateway, NGINX, Envoy, Traefik References:

  1. Microservices Communication Overview (YT)
  2. Microservices Communication using gRPC
  3. Change Data Capture for Data Sharing
  4. How Fiverr Shares Data Between Microservices
  5. API Gateway at Tinder
  6. Netflix: Zero Configuration Service Mesh with Envoy
  7. Service Mesh Tools — Consul.io
  8. Granularity Challenges in Microservices

Authentication & Authorization

Key Points

  • Use JWT with refresh tokens
  • Avoid “none” algorithm in JWT
  • Sign message payloads for message queues
  • Consider Open Policy Agent (OPA) for policy-based access control

References

Patterns & Architecture Styles

Event-Driven Architecture

Event Sourcing

Event sourcing is a Microservice design pattern that involves capturing all changes to an application’s state as a sequence of events, rather than simply updating the state itself. Each event represents a discrete change to the system and is stored in an event log, which can be used to reconstruct the system’s state at any point in time. it will be usefull when we want the history of updation record

  1. Event Generation: An event is generated whenever a change occurs in the system.

  2. Persistence in Event Store: The event is persisted to an event store, which is essentially a log of all events that have occurred in the system.

  3. Reconstruction of current state by replaying events: The current state of the system can be reconstructed at any time by replaying all of the events in the event store, in the order that they occurred.

  4. Service wise event store : Each service in the microservice architecture can have its own event store, which can be used to maintain its own state.

  5. Subscription to event store: Services can subscribe to events that are relevant to them and update their own state accordingly.

We can combine CQRS with event sourcing to handle update and read

CQRS (Command Query Responsibility Segregation)

  • You implemented the database-per-service pattern and want to join data from multiple microservices
  • Useful for database-per-service setups requiring cross-service joins

Saga Pattern

Saga is a design pattern used in distributed systems and microservices architecture to manage a sequence of related, independent transactions or operations. The primary goal of a Saga is to ensure that all these transactions are completed successfully or, if an error occurs, to provide a mechanism to compensate for the operations that have already occurred.

  • Two variants:
    1. Orchestration-based: Central controller directs flow
    2. Choreography-based: Each service emits events after local completion

References:

Orchestration vs Choreography

  • Orchestration: Centralized workflow control (example Node.js orchestrator)
  • Choreography: Decentralized event-based flow (services react to emitted events)

Outbox Pattern

The outbox pattern ensures data consistency in microservices by storing events locally before publishing them to a message broker, safeguarding against failures and maintaining atomicity between database changes and event notifications. Store the data in outbox table in db listen for changes in outbox table and publish the msg in queue.

Reliability

Reliability in microservices is achieved through fault isolation, redundancy, and resilience mechanisms like circuit breakers and retries. Each service’s failure should minimally impact others, and the overall system should degrade gracefully under load or partial outage. Distributed tracing and health checks are standard practices for monitoring. Reliability calculation considers the multiplicative effect of network calls, so design must minimize dependencies and implement fail-safe strategies

Resilience Patterns

Circuit Breaker

Prevents cascading failures by stopping requests to failing services and giving them time to recover.

States:

  • Closed: Normal operation, requests pass through
  • Open: Too many failures detected, requests fail immediately with fallback response
  • Half-Open: After timeout, allows test requests to check if service recovered

Implementation Tools: Netflix Hystrix, Resilience4j, Istio

To avoid spreading failures and saving resources

  1. Netflix’s Hystrix
  2. Opossum lib in nodejs

How netlify build resilence API using circuit breaker

  1. They have the circuit breaker for the API that depends on other service
  2. They create fallback if the other serive fail they generate response from the fallback method

When to Use: When accessing remote services prone to failure

Bulkhead Pattern

Isolates failures by partitioning system resources (processes, threads, connections) so one failure doesn’t consume all resources.

Techniques:

  • Process isolation: Each service in separate process
  • Thread pooling: Limit threads per service
  • Connection pooling: Limit connections to external services

Retry Pattern

Automatically retries failed requests with exponential backoff to handle transient failures.

Best Practices:

  • Use exponential backoff (2s, 4s, 8s, etc.)
  • Set maximum retry count
  • Only retry on idempotent operations or transient errors
  • Combine with circuit breaker to avoid thundering herd

Timeout Pattern

Sets maximum time to wait for a response, preventing indefinite hangs and resource waste.

Fallback Pattern

Provides default response or cached data when a service is unavailable.

Examples:

  • Return cached data
  • Use stale data
  • Provide degraded functionality
  • Default response

Consistency Models

  1. Two-phase commit(2PC)
  2. Eventual consistency (saga pattern)

Sidecar Pattern

  • Two processes in same host communicate via localhost
  • Used in service meshes for observability, proxying, etc.

Service Discovery & Configuration

Service Discovery

Locates service instances in a dynamic environment where services start, stop, and move frequently.

Three Implementation Approaches:

  1. DNS-Based:

    • Uses standard DNS for service lookup
    • Pros: Simple, language-agnostic
    • Cons: DNS caching issues, operational overhead
    • Tool: SkyDNS (for Kubernetes)

  2. Key-Value Store with Sidecar:

    • Services register with centralized store (Consul, Zookeeper, etcd)
    • Sidecar proxy handles local communication
    • Pros: Language-agnostic, transparent
    • Cons: Additional sidecar overhead, complex maintenance
    • Tool: Service Mesh (Istio, Linkerd)

  3. Library-Based:

    • Client library handles service discovery
    • Pros: Flexible, can discover any resource type
    • Cons: Language-specific, more code changes needed
    • Tool: Netflix Eureka, Spring Cloud Discovery

Eureka (Netflix)

  • Architecture: Client-side registry
  • Consistency: Eventual consistency
  • Best For: Spring Cloud environments, ease of use
  • Features:
    • Self-healing mechanism
    • Self-preservation mode during network partitions
    • Client-side load balancing

Consul (HashiCorp)

  • Architecture: Server-side registry, distributed
  • Consistency: Strong consistency
  • Best For: Complex microservices, multi-datacenter setups
  • Features:
    • Built-in health checks
    • Key-value store for configuration
    • Service mesh capabilities
    • DNS interface

Comparison Reference: Consul vs Eureka Analysis, LinkedIn Pulse

Service Mesh

Infrastructure layer that handles service-to-service communication, providing features like load balancing, traffic management, and security.

Architecture:

  • Data Plane: Sidecar proxies (one per pod) intercept all traffic
  • Control Plane: Manages proxy configuration and policies

Istio

Popular open-source service mesh built on Kubernetes.

Key Features:

  • Traffic Management: Route traffic, retries, circuit breakers
  • Security: mTLS, authorization policies
  • Observability: Automatic metrics and tracing
  • Virtual Services & Gateways: Define routing rules

Common Patterns with Istio:

  • Circuit breaker implementation at network level
  • Fine-grained traffic routing and load balancing
  • Automatic retries and timeouts

Other Service Mesh Options

  • Linkerd: Lightweight alternative to Istio
  • Consul Connect: Service mesh from HashiCorp
  • AWS App Mesh: AWS-native service mesh

Reference: Istio Circuit Breaker Guide, Red Hat

Data Management

Database per Microservice: Each service owns its independent database for autonomy and loose coupling

Schema per Microservice: Services define and manage their own data schemas independently

Polyglot Persistence: Using different database types (SQL, NoSQL, key-value) optimized for specific service needs

Data Replication: Async data synchronization across services maintaining eventual consistency

API Composition: Aggregating data from multiple services to fulfill client requests

Lambda architecture handles both batch and real-time data processing by using three layers: a batch layer for processing large historical datasets accurately, a speed layer for low-latency processing of real-time data, and a serving layer that combines these outputs for a unified view. This approach offers comprehensive analytics and fault-tolerance but is complex and involves maintaining separate codebases for batch and stream processing.

Kappa architecture simplifies this by treating all data as a real-time stream, using a single processing pipeline for both historical and live data. It eliminates the batch layer, reducing complexity and easing maintenance. It’s ideal for real-time analytics but may not be as accurate or efficient for large batch processing tasks.

Besides Lambda and Kappa, another emerging architecture is the Zeta architecture, which is designed to combine aspects of both Lambda and Kappa for enhanced flexibility and scalability in big data processing.

Data Sharing & Caching

Data Sharing

Caching

Doordash: Layered Caching Design

They implemented Layered caches (In a multi-layer cache, a key request progresses through the layers until the key is found or until it reaches the final source of truth (SoT) fallback function. If the value is retrieved from a later layer, it’s then stored in earlier layers for faster access on subsequent requests for the same key. This layered retrieval and storage mechanism optimizes performance by reducing the need to reach the SoT.)

Notes

  1. where you data from other boundary/service ask yourself is the data in the right spot is in the service that own

Monitoring & Observability

Tools & Frameworks

ToolPurposeLink
DaprAPIs for building portable & reliable microservicesdapr.io
Netflix ZuulAPI GatewayGitHub
Express GatewayNode.js API GatewayDocs
ConsulService discovery & configurationconsul.io

API Gateway Overview:

Messaging & Queues

Reference Videos & Conferences

Conceptual Reminders

  1. Always ask: “Is this data owned by the right service?” before sharing across boundaries.
  2. Prefer local caching and event propagation over direct synchronous dependencies.
  3. Favor async workflows (event-driven or saga-based) to maintain resilience.

Graph View