Explore Insights by Sudipta Sarkar
FlashAttention represents a groundbreaking advancement in optimizing the attention mechanism within Transformer models by addressing inherent memory-bound bottlenecks through I/O-aware design principles. This survey traces the evolution from the original FlashAttention algorithm—which introduced tiling and recomputation to enable exact, memory-efficient attention—to subsequent iterations that leverage advanced GPU architectures for unprecedented performance gains. We examine the core innovations, performance benchmarks, and broader impacts on model training and deployment, including significant speedups in training foundational models such as LLaMA and Falcon. By minimizing high-bandwidth memory (HBM) accesses and maximizing hardware utilization, FlashAttention enables scaling to longer context lengths while reducing computational costs, profoundly influencing the development landscape of large language models.
Read the Full ArticleThis comprehensive paper presents a detailed exposition of Abstract Algebra and Number Theory, structured to provide both theoretical foundations and practical applications. We systematically explore algebraic structures from basic semigroups to advanced field theory, complemented by number-theoretic concepts including prime numbers, modular arithmetic, and classical theorems. The paper emphasizes the interplay between these mathematical domains and their crucial role in modern cryptography, coding theory, and computer science. With clear definitions, illustrative examples, and visual representations, this work serves as both an introductory guide and reference for students and researchers.
Read the Full ArticleThis comprehensive document presents a unified treatment of Real and Complex Analysis, exploring their theoretical foundations, interconnections, and modern applications. Beginning with the complete real number system and progressing through metric spaces, Lebesgue integra- tion, holomorphic functions, and complex integration, we establish the rigorous mathematical framework underlying both fields. Particular emphasis is placed on applications in cryptogra- phy, signal processing, quantum field theory, and engineering sciences. The exposition includes detailed proofs of fundamental theorems, illustrative examples, visual representations, and refer- ences to contemporary research, making it suitable for advanced undergraduate students, grad- uate researchers, and practitioners in mathematical sciences and engineering.
Read the Full ArticleThis comprehensive guide presents probability and statistics concepts in an intuitive, ac- cessible manner while maintaining mathematical rigor. Starting from fundamental principles, we progress through probability theory, statistical inference, and culminate with applications in modern generative artificial intelligence. Each concept is explained with real-world examples, intuitive explanations, and precise mathematical notation. The guide is designed for learners at various levels, providing clear understanding of how these mathematical foundations power today’s AI systems.
Read the Full ArticleThis comprehensive document presents an in-depth exploration of linear algebra, sys- tematically progressing from fundamental concepts to advanced applications in artificial intelligence and computer vision. The material is organized into three tiers: foundations (vectors, matrices, systems), intermediate topics (transformations, eigenvalues, rank), and advanced techniques (SVD, PCA, tensor algebra). Each concept is accompanied by rigor- ous mathematical definitions, intuitive geometric interpretations, practical computational algorithms, and real-world applications. The document features extensive visualizations us- ing TikZ, demonstrates connections between theoretical concepts and modern applications, and serves as both a textbook for students and a reference for researchers and practitioners in machine learning, computer vision, and scientific computing.
V1 (Old) Read the Full ArticleThis report presents a comprehensive graduate-level survey of complexity theory, a foundational branch of theoretical computer science concerned with the computational resources required to solve problems. The discussion begins with computability theory, covering fundamental notions such as decidability, the Chomsky hierarchy, and variants of Turing machines (TM, DTM, NTM, HTM). We then introduce formal measures of complexity and examine central complexity classes (L, NL, P, NP, co-NP, NP-Hard, NP- Complete, PSPACE, EXP, and beyond), providing illustrative examples and emphasizing their theoretical and practical importance. The report also catalogs major theorems (e.g., Cook–Levin, Savitch, PCP), foundational axioms and theses (e.g., Church–Turing Thesis), central hypotheses (e.g., P ≠ NP, ETH), and canonical decision problems (e.g., the Halting Problem, Circuit-SAT, TQBF). Where appropriate, proof sketches and visual aids are included to clarify concepts, and the interrelationships among classes are summarized. Designed for both seminar presentations and academic study, this document aims to deliver a rigorous yet accessible account of complexity theory as of August 2025.
Read the Full ArticleThis paper provides a comprehensive introduction to proof methodologies in discrete mathematics and computer science, emphasizing graph theory and algorithms. We define proofs, categorize their types, and highlight their importance in validating theoretical and applied results. Disproof strategies, particularly counterexamples, are also explored. Each proof method is illustrated with three detailed examples, primarily in graph theory and algorithms, with step-by-step explanations to ensure clarity. This work serves as an accessible resource for students and researchers.
Read the Full ArticleBackpropagation is the cornerstone of neural network training, enabling weight opti- mization to minimize prediction errors. This paper provides a rigorous yet intuitive intro- duction to backpropagation, covering weighted sums, activation functions, loss and cost functions, and gradient descent. Six numerical examples, each fully solved with forward and backward pass calculations and neural network diagrams, demonstrate its application across diverse architectures, activation functions, and loss functions. Visualizations of gradient descent and network structures, combined with real-world analogies, make the concepts accessible to researchers and practitioners.
Read the Full ArticleComprehensive handwritten notes covering core Computer Science subjects for competitive examinations including UGC NET Computer Science & Applications, GATE Computer Science & Engineering, ISI/CMI Data Science (DA), JEST Theoretical Computer Science, TIFR GS CS, and other CS-related competitive exams. These notes include detailed theory, solved examples, conceptual explanations, mathematical derivations, PYQ analysis, and important interview topics. Carefully structured for revision, conceptual clarity, and exam-oriented preparation.
View / Download Handwritten Notes