vLLM vs. llama.cpp: Which is the Real Production King?
A deep-dive architectural comparison between vLLM and llama.cpp. Compare throughput, memory footprints, and choose the right engine for your workload.
My Blog
Reflections on software engineering, pipeline automation, graphics, and system reliability.
Filter by Date:
Thinking Machines' Inkling: Under the Hood of the 975B Parameter Open Multimodal MoE
A deep dive into Mira Murati's new open-weights multimodal model, Inkling. Analyze the 975B MoE architecture, active parameter routing, and local serving.
Token Economics, LLM Gateways, and Router9
Understand the financial side of LLM serving. Calculate cost-per-token, setup open-source LLM gateways, and implement intelligent routers like Router9.
Inference Optimizations: Speeding up Prefill and Decode
Explore advanced LLM serving optimizations. Learn how FlashAttention, chunked prefill, speculative decoding, and KV cache eviction speed up TTFT and inter-token latency.
OpenClaw in Action: Connecting WhatsApp to Automated Workflows
A step-by-step developer guide to configuring OpenClaw's WhatsApp gateway, setting up secure session authorization, and building automated task sync skills.
The Self-Hosted AI Butler: Modular Assistance with OpenClaw
Learn how to build and host your own AI assistant with OpenClaw. Configure the ClawHub modular skill store, write SKILL.md files, and connect to chat platforms.
Nous Research's Hermes Agent: Under the Hood
Deep-dive into the configuration, CLI commands, and skill architecture of Nous Research's Hermes Agent. Learn to manage tool gateways and design custom skills.
The Landscape of Agentic AI: From Single-Agent Scripts to Multi-Agent Networks
Start the Autonomous AI Agents Series. Learn about the ReAct pattern, single-agent context decay, and multi-agent coordination graph topologies.
The Landscape of LLM Inference Engines: Open Source vs. Enterprise
A comprehensive developer guide comparing vLLM, TensorRT-LLM, TGI, SGLang, and llama.cpp. Learn about PagedAttention, RadixAttention, and GGUF.
Understanding the KV Cache: The VRAM Bottleneck of LLM Serving
Why do LLM serving nodes run out of VRAM? Explore the mechanics, calculations, and capacity bottlenecks of the Key-Value (KV) Cache.
The Two Pillars of LLM Inference: Prefill vs. Decode
Explore the two execution phases of LLM inference: compute-bound prefill vs. memory-bandwidth-bound decode. Understand arithmetic intensity and the role of the KV cache.
Basics of AI Inference: Demystifying Latency, Throughput, and Serving
Start the AI Inference Deep-Dive Series. Learn the fundamentals of inference vs training, key performance metrics, and the systems engineering behind serving.
The Model Taxonomy: LLMs, Vision Models, VLAs, and Diffusion
A developer's guide to the modern AI model taxonomy. Understand the architecture, modalities, and performance profiles of LLMs, ViTs, VLAs, and Diffusion.
Training vs. Inference Lifecycle: A Developer's Guide to Weights, Backpropagation, and Serving
A step-by-step developer's view of the AI model lifecycle—from raw weights and backpropagation during training to freezing weights and serving inference.
What is a Model Weight? Demystifying Tensors, Matrices, and File Formats
What actually happens when you download a model from Hugging Face? A developer's guide to tensors, weights, biases, and serialization formats like Safetensors and GGUF.
Claude Code Custom Plugins: Build and Host Your Own Marketplace
Learn how to build, structure, and host a custom, decentralized Claude Code plugin marketplace using a simple GitHub repository.
Running Local LLMs: Ollama vs. vLLM
A comprehensive guide to running Large Language Models locally on Windows, macOS, and Linux using Ollama and vLLM, including pros, cons, and performance tuning.
Anthropic's Claude Model Family: Specs, Pros, Cons, and Use Cases
Compare Anthropic's Claude models—Fable 5, Opus 4.8, Sonnet 5, Haiku 4.5, and Claude Science. Learn their pros, cons, and the best use cases.
Anthropic's Mid-2026 Wave: Claude Sonnet 5, Claude Science, and Fable 5 Redeployment
A summary of Anthropic's major announcements: the launch of Claude Sonnet 5, the new Claude Science AI workbench, and the global redeployment of Fable 5.
Claude Code Custom Skills: Design Methodology and Workspace Personas
Learn how to design, structure, and implement persistent, context-aware custom agent skills in Claude Code to automate code reviews and git workflows.
Claude Code Basics: Commands, Subagents, and Memory Layers
Learn the difference between slash commands and file context mentions, configure global vs local memory, and understand how custom skills and subagents operate in Claude Code.
Claude Code Customization: CLAUDE.md, AGENTS.md, and SKILLS.md
Master project-level customization for Anthropic's Claude Code by writing rules and guidelines in CLAUDE.md, AGENTS.md, and custom SKILLS.md.
Setting up Claude Code: The Ultimate Terminal AI Pair Programmer
A step-by-step setup guide for installing, configuring, and authenticating Anthropic's Claude Code CLI across macOS, Windows, and Linux.
LLMs, Agents, and Harnesses: Demystifying Claude Code
Confused about the difference between Claude Code and Claude Opus? A deep dive into the architecture of modern agentic coding tools—from Large Language Models (LLMs) to Autonomous Agents and Execution Harnesses.
Custom Model Endpoints: Hooking up Local & Enterprise Models with Antigravity CLI
Learn how to configure the Google Antigravity CLI (agy) to run with local LLMs (Ollama, vLLM) or private enterprise endpoints for maximum privacy and cost efficiency.
Going Async: Background Tasks, Timers, and Scheduling in Antigravity CLI
A deep dive into managing long-running tasks, using asynchronous tools, setting timers, and configuring recurring jobs with the Google Antigravity CLI.
Extending Antigravity CLI: Building Custom Skills and Project Rules
Learn how to supercharge the Google Antigravity CLI (agy) by building custom agent skills, project-scoped rules, and custom scripts to automate repository-specific workflows.
Setting up Google Antigravity CLI: The AI-First Terminal Assistant
A comprehensive guide on installing and configuring Google Antigravity CLI (agy) on Mac, Windows, and Linux, complete with TUI commands, settings.json customization, and real-world usage examples.
Python 3.15: A First Look at the Coolest New Features
A deep dive into the most exciting developer features in the Python 3.15 pre-release, including explicit lazy imports, frozendict, built-in sentinels, unpacking in comprehensions, and the Tachyon profiler.
SLURM Demo on AWS Ubuntu EC2 instance
Demo of slurm usage on a single instance of Ubuntu 24.04 EC2 instances on AWS
SLURM on WSL
Setting up SLURM on WSL
Introduction to SLURM
Simple Linux Utility for Resource Management (SLURM)
How to find a linux machine is a VM (Virtual Machine) or a Bare Metal
If you can SSH into a linux machine and want to find out if its baremetal or Virtual Machine
Storing Github access token in git credential store
Using git credentials store the github access token to avoid the re-prompting of username and pwd
Token generation for Registering Self Hosted Github Runner via REST API
Explains how to generate a token using github API to be used in turn with Github self hosted runner registration
Setting up a Self Hosted Github Runner
Explains how to setup a Github self hosted runner and register
Managing the NodeJS versions on Windows
Node Version Manager (nvm) helps in managing multiple NodeJS versions
Customizing the Powershell terminal with oh-my-posh
Instructions on customizing the terminal in powershell with oh-my-posh and winget
File permissions on Windows - chmod 400 in Powershell
Powershell equivalent of chmod 400 for Windows files
Github login using access token via command line
Logging in using github access token (no more passwords)
Adding Google Analytics to NuxtJS app
Adding Google Analytics GA4 property to NuxtJS App
Productive Taskbar Settings missing in Windows 11
Very useful Taskbar Settings goes missing in Windows 11.
Secureboot + Ubuntu + VirtualBox Signing kernel modules
Set of steps required for dealing with secureboot on Ubuntu where VirutalBox service has issues
Fixing the postfix error dpkg
Steps to fix the postfix error happening during apt upgrade ubuntu.
Running a react app on Local Kubernetes cluster on Windows 10
Process and steps for running react app on local k8s cluster using minikube on windows 10
Gatsby site hosted on AWS Amplify redirecting to homepage always
Using the rewrites and redirects on AWS Amplify for the depolyed personal website
Connecting AWS Amplify for deployment of website
Explains how to connect the gatsby website hosted on github to AWS Amplify for deployment
Evolution of this website
will be capturing the evolution of my website chronologically (latest first)
Making of "Tree Story" Animated Short
Behind the scenes of the Animated Short "Tree Story"
Profiling & Visualization Tools in Python - Part 1
Profiling & Visualization Tools in Python using cProfile
Missing permissions issue linking google analytics and adsense
Problem of linking google analytics and adsense
Adding Google Analytics to personal website
How I added google analytics to my personal website written in Gatsby
Chrome Remote Desktop on Fedora
How to install Chrome Remote Desktop on Fedora
Building grpc whl with Maya 2019 on Windows
My stuggle with building a grpc python library with Maya 2019 on Windows
Tech Stack of the website
Brief about the tech stack used for this website
Deleting linux from dual boot
Steps for safely removing linux from dual boot.