Omniverse#
Influential Ideas and Papers
- Generative Pre-trained Transformers
- Low-Rank Adaptation Of Large Language Models
- Chain-of-Verification Reduces Hallucination in Large Language Models
- Vector Semantics and Embeddings
- The Machine Learning Framework
- Empirical Risk Minimization
- Loss Functions
- Why Neural Network Initialization Matters
- Decision Boundary
- Is The Learning Problem Solvable?
- Lloyd’s K-Means Clustering Algorithm
- Naive Bayes
- K-Nearest Neighbours
- Mixture Models
- Principal Component Analysis
- Linear Regression
- Gradient Descent
- Metrics and Scoring Rules
Deep Learning
Playbook
- Training Dynamics And Tricks
- How to Calculate the Number of FLOPs in Transformer Based Models?
- Why Does Cosine Annealing With Warmup Stabilize Training?
- How To Fine-Tune Decoder-Only Models For Sequence Classification Using Last Token Pooling?
- How To Fine-Tune Decoder-Only Models For Sequence Classification With Cross-Attention?
- How To Do Teacher-Student Knowledge Distillation?
- Softmax Preserves Order, Is Translation Invariant But Not Invariant Under Scaling.
- How to Inspect Function and Class Signatures in Python?
Probability Theory
- Chapter 1. Mathematical Preliminaries
- Chapter 2. Probability
- Chapter 3. Discrete Random Variables
- Random Variables
- Discrete Random Variables
- Probability Mass Function
- Cumulative Distribution Function
- Expectation
- Moments and Variance
- Discrete Uniform Distribution
- Bernoulli Distribution
- Independent and Identically Distributed (IID)
- Binomial Distribution
- Geometric Distribution
- Poisson Distribution
- Important
- Exercises
- Chapter 4. Continuous Random Variables
- From Discrete to Continuous
- Continuous Random Variables
- Probability Density Function
- Expectation
- Moments and Variance
- Cumulative Distribution Function
- Mean, Median and Mode
- Continuous Uniform Distribution
- Exponential Distribution
- Gaussian Distribution
- Skewness and Kurtosis
- Convolution and Sum of Random Variables
- Functions of Random Variables
- Chapter 5. Joint Distributions
- Chapter 6. Sample Statistics
- Chapter 8. Estimation Theory
Operations
- Distributed Systems
- Profiling
- The Lifecycle of an AIOps System
- Stage 1. Problem Formulation
- Stage 2. Project Scoping And Framing The Problem
- Stage 3. Data Pipeline (Data Engineering and DataOps)
- Stage 4. Data Extraction (MLOps), Data Analysis (Data Science), Data Preparation (Data Science)
- Stage 5. Model Development and Training (MLOps)
- Stage 6. Model Evaluation (MLOps)
- Stage 7. Model Validation, Registry and Pushing Model to Production (MLOps)
- Stage 8. Model Serving (MLOps)
- Stage 9. Model Monitoring (MLOps)
- Stage 10. Continuous Integration, Deployment, Learning and Training (DevOps, DataOps, MLOps)
Software Engineering
- Config, State, Metadata Management
- Design Patterns
- Python
- Init vs New
- Global Interpreter Lock (GIL)
- The Iterator Protocol
- Decorator
- Generators Over Lists For Memory Efficiency
- Pydantic Is All You Need - Jason Liu
- Do Not Use Mutable Default Arguments
- Set Over List For Frequent Membership Tests
- Late Binding Closures
- Is vs Equality
- Introduction: Why Python Objects Are “Heavier”
- Concurrency, Parallelism and Asynchronous Programming
Computer Science
Data Structures and Algorithms
Linear Algebra
References, Resources and Roadmap