kache: kademlia-based distributed CDN

Kache: Kademlia-Cache written in Go

Kache is a distributed content delivery network (CDN) built on top of the Kademlia distributed hash table (DHT) protocol. It allows for efficient content delivery and caching across a decentralized network of nodes.

This project is designed to provide a scalable and resilient CDN solution that can be easily deployed and managed. It provides a series of features including:

• Auto-discovery of new nodes: Zero configuration needed. Nodes yell into the void using an mDNS background multicast engine (kache-private-mesh) and find local peers instantly.

• Authentication via x509 certificates: No uninvited guests allowed. Nodes must pass a strict cryptographic challenge-response protocol validated against a trusted Root CA certificate before joining the swarm.

• At-rest data encryption: Files saved on disk are secured using AES-GCM. If you omit a cache key path, an ephemeral, 32-byte key is safely generated in memory. If you provide a persistent path, a secure 0600 master key file is kept on disk to persist across daemon restarts.

• Latency-Aware routing: Kache maps network routing layout conditions using Exponentially Weighted Moving Averages (EWMA). When requesting files, it bypasses sluggish targets and pulls streams directly from the closest physical peer node.

• Proactive Swarm Replication: Why keep files on one node when you can spread them around? Kache calculates XOR distances using the Kademlia engine to proactively duplicate content chunks across nearby network neighbors on ingest.

• Full CLI to interact with the network: A dedicated CLI companion tool (kachectl) lets you upload, stream, track down swarm matrices, or verify overall network status seamlessly.

The Architecture (Or: How I Convinced My Laptops to Collaborate)

Under the hood, Kache splits operations across a few highly decoupled modules to make sure complex P2P protocols don’t devolve into spaghetti code:

• The Daemon (dcdnd): The system background wrapper. It parses parameters, resolves master cryptographic assets, mounts block filesystems, and gracefully tears down listening tunnels when hit with a termination interrupt.

• The API Routing Layer (api.go): Serves as the primary developer boundary. It exposes clear HTTP REST endpoints via Gin while orchestrating raw under-the-hood streams over multi-multiplexed libp2p network pipes.

• The DHT Engine (dht.go): Tracks layout maps, schedules asynchronous asset indexing providers via go-libp2p-kad-dht, and runs peer routing lookups.

• The Storage Core (cache.go): Implements disk-backed cache tracking with Last-Recently-Used (LRU) eviction and background automated sweeper custodians to constantly reclaim local machine bytes.

Fixing the “Open Swarm” Problem (The Security Overhaul)

Initially, Kache was a bit too friendly. Anyone running the daemon on the local network could seamlessly mount a peer node, access cache entries, and join the cluster data plane without checking in. To fix this, I implemented two major security mechanisms:

1. Zero-Knowledge Challenge Handshake (/kache/cluster-auth/1.0.0)

When an mDNS notice claims a new peer has arrived, the system triggers a strict verification protocol before passing down network streams:

   +───────────────────────────+                  +───────────────────────────+
   │      Challenger Node      │                  │      Responding Node      │
   +───────────────────────────+                  +───────────────────────────+
                 │                                              │
                 │     1. Stream Encrypted 32-byte Nonce        │
                 │─────────────────────────────────────────────>│
                 │                                              │ ──┐ 2. Hash nonce
                 │                                              │   │    (SHA-256)
                 │                                              │   │ 3. Sign hash with
                 │                                              │ <─┘    Private RSA Key
                 │     4. Send AuthResponse                     │
                 │        (Signature + x509 Certificate)        │
                 │<─────────────────────────────────────────────│
                 │                                              │
                 │ ──┐ 5. Verify Certificate chain              │
                 │   │    against shared Root CA Pool           │
                 │   │ 6. Verify RSA Signature against          │
                 │   │    expected nonce hash                   │
                 │ <─┘                                          │
                 │                                              │
                 ├─── [Validation Passes] ───────────────────────
                 │    Result: Secure Swarm Pipe Opened Successfully!
                 │
                 └─── [Validation Fails] ────────────────────────
                      Result: CRITICAL SECURITY VIOLATION -> Link Forcefully Killed (X)

1. Isolated At-Rest Key Lifecycle

To prevent node operators from snooping through cache contents directly on the file system, the cache system encrypts all files before writing them to disk using AES-256-GCM.

The best part? Nodes don’t share identical encryption keys. Data is transmitted over secure libp2p channels as plaintext, but when written locally, each node leverages a distinct master file-bound key hidden inside its private data repository.

The end(?)

This is a learning project, i wanted to delve into Kademlia as it really triggered my curiosity. I don’t expect this to be production ready or anything, but it was a fun experiment and I learned a lot about distributed systems, P2P networking, and secure authentication. If you’re interested in checking it out or contributing, the code is open source on Github.