Projects
Chinchilla-Optimal Transformer Pre-training for Music
Fall 2025Independent ML Systems Project GitHub
- Independently designed and trained decoder-only Transformer models (NanoGPT) on the Lakh MIDI Dataset
- Optimized training throughput on NVIDIA H100 GPUs using BFloat16 mixed precision, Flash Attention, and torch.compile
- Achieved a test perplexity of 2.20 with 100% syntactically valid output
Stack: Python, PyTorch, CUDA, Flash Attention
Toy Load Balancer with Consistent Hashing
2024Systems Engineering Project GitHub
- Built a custom load balancer implementing consistent hashing to distribute traffic across dynamic server nodes
- Containerized the entire architecture (API Gateway, Nodes, Analytics) using Docker Compose for easy deployment
- Implemented a management API to dynamically add/remove servers and visualize request rebalancing in real-time
Stack: Docker, Python, Node.js, Shell