Tutorial

Table of Contents KV Cache Optimization via Tensor Product Attention Challenges with Grouped Query and Multi-Head Latent Attention Multi-Head Attention (MHA) Grouped Query Attention (GQA) Multi-Head Latent Attention (MLA) Tensor Product Attention (TPA) TPA: Tensor Decomposition of Q, K, V…

Table of Contents Pushing Docker App to AWS ECR for Lambda Deployment Using AWS CLI Setting Up AWS CLI and Pushing the FastAPI AI App to AWS ECR for Lambda Deployment What Is AWS CLI and Why Do We Need…

Table of Contents FastAPI Docker Deployment: Preparing ONNX AI Models for AWS Lambda Introduction Why Do We Need an API for AI Inference? Why Use Python FastAPI + Docker Before AWS Lambda Deployment? Configuring Your Development Environment Setting Up Docker…

Table of Contents Converting a PyTorch Model to ONNX for FastAPI (Docker) Deployment Introduction Recap of the Previous Lesson Why This Step Matters What You’ll Learn in This Lesson Technologies Used Configuring Your Development Environment Project Structure Converting ResNetV2-50 (TIMM)…

Table of Contents Introduction to Serverless Model Deployment with AWS Lambda and ONNX What Is Serverless Model Deployment? How Does Serverless Deployment Work? Why Use AWS Lambda for AI Model Inference? Limitations of AWS Lambda for AI Inference How Serverless…

Table of Contents Build a VLC Playlist Generator with SmolVLM for Video Highlight Tagging Configuring Your Development Environment Setup and Imports Helper Functions Main VideoHighlightDetector Class Creating the XSPF Playlist Gradio Interface Logic Launch the Gradio Application Output Summary Citation…

Table of Contents Running SmolVLM Locally in Your Browser with Transformers.js Introduction SmolVLM: A Small But Capable Vision-Language Model Transformers.js: Hugging Face Runtime for the Web Why This Pair Is Powerful (and Practical Caveats) Configuring Your Development Environment What Are…

Table of Contents KV Cache Optimization via Multi-Head Latent Attention Recap of KV Cache The Need for KV Cache Optimization Multi-Head Latent Attention (MLA) Low-Rank KV Projection Up-Projection Decoupled Rotary Position Embeddings (RoPE) RoPE in Standard MHA Challenges in MLA:…

Table of Contents Introduction to KV Cache Optimization Using Grouped Query Attention Understanding the KV Cache Grouped Query Attention What Is Grouped Query Attention? How Grouped Query Attention Reduces KV Cache? Implementing KV Caching via Grouped Query Attention Grouped Query…