Settle | Shanel Perera | Shanel Perera

/settle/settle/settle

Project-01

(Settle)

Fault-Tolerant Distributed Payment Processing System

Overview

Settle is a prototype Distributed Payment Processing System developed as an academic project for the Distributed Systems (SE2062) module. Designed to emulate a resilient e-commerce payment backend, the system addresses core distributed computing challenges through a robust, fault-tolerant architecture spanning a multi-node cluster. Rather than just a theoretical implementation, the project brings distributed systems concepts to life by reliably processing concurrent client transactions even in the presence of node crashes or network delays.

The core architecture directly tackles the project's primary challenges through distinct, integrated subsystems:

Fault Tolerance & Failover: The system runs on a highly available 5-node cluster with automated failure detection. It features seamless failover mechanisms to redirect payments when a node goes down, alongside recovery protocols that safely integrate returning nodes back into the cluster.
Data Replication & Consistency: A distributed ledger ensures payment details are reliably replicated across servers. It utilizes deduplication (idempotency keys) to safely process retries and maintains strict data consistency across the network.
Time Synchronization: To combat clock skew and network latency, the system implements synchronization protocols to accurately timestamp, correlate, and reorder out-of-sequence distributed logs.
Consensus & Leader Election: A custom Raft consensus engine is implemented to manage leader election and ensure all distributed payment servers unequivocally agree on transaction states and log storage policies.

Live WebsiteUnavailable

GitHubshanelperera-exe/settle-distributed-backend

Details

TypeAcademic Prototype

ModuleDistributed Systems (SE2062)

Duration1 Month

Focus Areas

Fault Tolerance
Data Replication
Raft Consensus
Time Sync
System Design

Tech Stack

PythonFastAPIReactTypeScriptTailwind CSSPostgreSQLRedis

Apache ZooKeeperPrometheusGrafana

LokiJaegerDockerDocker ComposeNginxGitGithubGithub ActionsStripeVite

Features

Comprehensive Observability Stack: A fully integrated telemetry ecosystem featuring distributed tracing, metric dashboards, and centralized log aggregation provides deep, real-time visibility into the consensus state and cluster health.

External Gateway Integration: The system actively listens to and processes real-world simulated transactions by securely integrating with the Stripe API via webhooks.

Real-Time Interactive Dashboard: A dynamic, premium frontend interface was built from scratch to visually map the complex network state, track active leader elections, and monitor payment flows interactively.

Intelligent Load Balancing: An Nginx API gateway sits in front of the cluster to efficiently route incoming client requests and webhook events to available, healthy nodes.

Distributed Consensus: Custom Raft implementation ensuring quorum commits, randomized leader election, and resilient log replication across the cluster.

High Availability & Failover: ZooKeeper-based service registration with automatic failover, cluster health checks, and PostgreSQL database promotion.

Idempotency & Deduplication: Deterministic SHA-256 payload hashing and database-backed concurrency locking to prevent replay attacks during network delays.

Financial Ledger & State: Consistent double-entry ledger tracking deposits, withdrawals, and finalized transactions with robust state-machine replay protection.

Distributed Time Synchronization: Implements a Hybrid Logical Clock (HLC) and NTP synchronization for precise, causality-preserving event ordering across nodes.

Chaos Engineering: Admin-triggered endpoints for fault injection, simulating node crashes, network partitions, and resource spikes to test system resilience.