Software Engineer. I Build Stuff.

github

Nilotpal Deka  • Peerlist

globalid is a simple Go package for generating globally unique, roughly time-ordered IDs, inspired by Snowflake-style ID generation.

globalid

pguard is a simple CLI tool for running commands with optional timeouts and graceful shutdown, similar to Unix timeout.

pguard

This project implements a high-performance, multi-threaded TCP echo server in Go, specifically designed for Linux environments. It utilizes the epoll I/O event notification facility for efficient handling of numerous concurrent connections. The server employs a multi-listener architecture with SO_REUSEPORT for kernel-level load balancing across multiple worker goroutines, providing a simple echo service.

A simple Multi-threaded Go TCP server using e

A simple OpenTelemetry library for Go applications that provides distributed tracing, metrics collection and structured logging with minimal setup complexity.

gotel: OpenTelemetry Toolkit for Go

maybe is a Go package providing type-safe optional values and functional programming utilities using Go generics. It helps eliminate nil pointer panics and provides a more expressive way to handle optional values in Go.

Maybe - About A Go Package for Optionals, Nul

A lightweight, in-memory Publish-Subscribe (PubSub) system written in Go leveraging Go's powerful concurrency model with channels.

PubSub Implementation Using Golang Channels

This project implements a Write-Ahead Log (WAL) in Go, using Protocol Buffers (default) as the encoding format for all log records. The goal of this WAL is to provide a reliable foundation for systems that require strong durability guarantees and predictable recovery behavior. The WAL records every change in a strictly sequential order and ensures that each entry is written to stable storage before it is considered committed. By using Protocol Buffers, each record is encoded in a compact and schema-safe format, which makes the log easy to extend and efficient to store.

Simple crash free write-ahead log in Golang.

I built a persistent key-value storage engine in Go based on Bitcask, the log-structured storage system used in Riak. I wanted to understand how databases actually work at a low level, the tradeoffs between performance and durability, how crash recovery works and why different storage engines exist for different workloads.Why I Did This:Bitcask has this beautifully simple design, just append everything to a log file and keep an in-memory index. No fancy B-trees, no complicated balancing algorithms. It's literally just "write goes brrr" and you maintain a HashMap to find stuff. Perfect for actually learning instead of drowning in theory.How I Built It:The basic idea:<ul><li>Writes just append to a file (super fast, sequential I/O)</li><li>In-memory HashMap tells you where each key lives</li><li>Reads are just HashMap lookup -&gt; seek to file offset -&gt; read value</li><li>Eventually you compact old files to reclaim space</li></ul>The interesting parts:1. Different write modes:I implemented three modes because I wanted to understand the tradeoffs everyone talks about:<ul><li>Async writes: Basically YOLO mode. Writes go to a buffer in memory and flush whenever. Got like 10k+ writes/sec which was honestly satisfying to watch. But yeah, crash and you lose recent writes. Not ideal for your bank account balance.</li><li>Batch sync: The "let's be reasonable" option. Buffer a bunch of writes, then call `fsync()` once. You get decent throughput (5-6k writes/sec) and only risk losing the current batch if things go sideways.</li><li>Direct sync: Full paranoia mode. Every single write calls `fsync()` immediately. Super safe but painfully slow. This is what you'd use for data you absolutely cannot lose.</li></ul>Implementing these really drove home why database docs are always like "configure your durability settings based on your use case" - the performance difference is wild.2. Checksums because data corruption is real:Added CRC32 checksums to every record because apparently disks lie to you sometimes? Each entry is [checksum][timestamp][key_size][value_size][key][value] and I verify the checksum on every read and during recovery.3. Crash recovery:When you start up, you rebuild the in-memory HashMap by scanning all the data files. But here's the thing - you have to validate checksums as you go because the last file might have partial writes from a crash. 4. Compaction (aka garbage collection for databases):Since you never update in place, old values just pile up. I wrote a background process that goes through old files, keeps only the latest value for each key and writes them to new files. Getting this right was lowkey the hardest part.What I Learned:The performance differences between write modes were eye-opening. Async mode was flying while direct sync was struggling at a few thousands. You really see why databases let you tune this stuff.Also, fsync() is expensive in ways I didn't fully appreciate. And checksums aren't just paranoia - they're necessary because hardware fails in weird ways.Plus it's just deeply satisfying to write db.Put("key", "value") and know exactly what's happening - the bytes being written to disk, the HashMap being updated, whether fsync() is being called.

ignite

This project implements a Write-Ahead Log (WAL) in Go, using Protocol Buffers (default) as the encoding format for all log records. The goal of this WAL is to provide a reliable foundation for systems that require strong durability guarantees and predictable recovery behavior. The WAL records every change in a strictly sequential order and ensures that each entry is written to stable storage before it is considered committed. By using Protocol Buffers, each record is encoded in a compact and schema-safe format, which makes the log easy to extend and efficient to store.

Crash free write-ahead log in Golang

A simple OpenTelemetry library for Go applications that provides distributed tracing, metrics collection and structured logging with minimal setup complexity.

gotel

maybe is a Go package providing type-safe optional values and functional programming utilities using Go generics. It helps eliminate nil pointer panics and provides a more expressive way to handle optional values in Go.

maybe

A lightweight, in-memory Publish-Subscribe (PubSub) system written in Go leveraging Go's powerful concurrency model with channels.