#L09 · Linear Algebra & Manifolds · Lv.5View in 3D Mind-Map

Singular Value Decomposition (SVD) & PCA

Beltrami, Jordan, Eckart & Young · 1936 CE

A = U\Sigma V^T = \sum_{i=1}^{r} \sigma_i \mathbf{u}_i \mathbf{v}_i^T

A^T A \mathbf{v}_i = \sigma_i^2 \mathbf{v}_i,\quad \text{Energy}(k) = \frac{\sum_{i=1}^k \sigma_i^2}{\sum_{j=1}^r \sigma_j^2}

Factorizes any m×n matrix into orthogonal rotation Vᵀ, diagonal singular values Σ, and rotation U—powering PCA dimensionality reduction, image compression, and LLM LoRA fine-tuning.

🧮 Live Formula Calculator & Plotter

1st Singular σ₁

2nd Singular σ₂

3rd Singular σ₃

\text{Rank-1 Energy} = \frac{10^2}{10^2+4^2+1^2} = 85.5\%

Rank-1 에너지 보존율

85.47%

Rank-2 누적 보존율

99.15%

프로베니우스 노름 ||A||_F

10.817

Prerequisite Path (19 STEP)

Full Path Analyzer

Click steps in order to follow the 3D neon path from basics to target.

STEP 1#G01Pythagorean Theorem

a^2 + b^2 = c^2 \implies c = \sqrt{a^2 + b^2}

STEP 2#G03Pi, Radian Measure & Circular Sector

l = r\theta

STEP 3#A02Polynomial Expansion & Factoring

(a+b)^2 = a^2 + 2ab + b^2

STEP 4#A03Laws of Exponents & Radicals

a^m \cdot a^n = a^{m+n}

STEP 5#L01Vector Operations, L1/L2 Norms & Normalization

\|\mathbf{v}\|_2 = \sqrt{\sum_{i=1}^{n} v_i^2}

STEP 6#A01Linear Equation & Balance Axiom

ax + b = 0 \implies x = -\frac{b}{a}

STEP 7#G022D/3D Euclidean Distance Formula

d(P

STEP 8#L033D Cross Product & Surface Normal Vector

\mathbf{a} \times \mathbf{b} = (a_y b_z - a_z b_y

STEP 9#G05Law of Sines & Law of Cosines

c^2 = a^2 + b^2 - 2ab\cos C

STEP 10#G04Unit Circle Trigonometry & Identities

\cos^2\theta + \sin^2\theta = 1

STEP 11#A04Quadratic Formula & Discriminant

x = \frac{-b \pm \sqrt{b^2 - 4ac}}{2a}

STEP 12#A05Vieta's Formulas (Roots & Coefficients)

\alpha + \beta = -\frac{b}{a}

STEP 13#L02Dot Product & Cosine Similarity

\mathbf{a} \cdot \mathbf{b} = \sum_{i=1}^{n} a_i b_i = \|\mathbf{a}\|\|\mathbf{b}\|\cos\theta

STEP 14#L04Matrix Multiplication & Linear Transformation

(AB)_{ij} = \sum_{k=1}^{n} A_{ik} B_{kj}

STEP 15#G06Trigonometric Addition & Double-Angle Formulas

\cos(\alpha\pm\beta) = \cos\alpha\cos\beta \mp \sin\alpha\sin\beta

STEP 16#L05Determinant & Matrix Inverse

\det(A) = ad - bc

STEP 17#L072D/3D Rotation Matrix & Orthogonal Group SO(3)

R(\theta) = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}

STEP 18#L08Eigenvalues & Eigenvectors (Av = λv)

A\mathbf{v} = \lambda\mathbf{v} \iff \det(A - \lambda I) = 0

STEP 19#L09Singular Value Decomposition (SVD) & PCA

A = U\Sigma V^T = \sum_{i=1}^{r} \sigma_i \mathbf{u}_i \mathbf{v}_i^T

Geometric & Intuitive Insight

수백만 화소의 사진이나 거대한 AI 가중치 행렬에서 가장 큰 특이값 σ₁, σ₂ 몇 개만 남기고 작은 특이값들을 버려도(Low-Rank Approximation), 원본 정보의 95% 이상이 그대로 보존되면서 용량은 1/10로 줄어듭니다!

Step-by-Step Derivation

Step 1.1단계: 임의의 m×n 행렬 A에 대해 대칭행렬 A^T A 의 고윳값 λ_i ≥ 0 와 직교 고유벡터 v_i 를 구합니다.
Step 2.2단계: 특이값을 σ_i = √λ_i 로 정의하고, u_i = (1/σ_i) A v_i 로 좌특이벡터를 구성합니다.
Step 3.3단계: 이를 행렬로 묶으면 AV = UΣ 가 되고, V가 직교행렬(V⁻¹=V^T)이므로 A = UΣV^T 가 완성됩니다.

AI & Coding Real-World Use

LLM Low-Rank Adaptation (LoRA), PCA dimensionality reduction, recommender systems, and image compression

#LLM LoRA Fine-Tuning#PCA Dimensionality#Recommender System

const explainedRatio = (s1:number, s2:number, s3:number) => (s1*s1) / (s1*s1 + s2*s2 + s3*s3);

Linear Algebra & Manifolds — Related Formulas in This Cluster

Vector Operations, L1/L2 Norms & Normalization

Lv.1 · L1/L2 Regularization

#L01

Dot Product & Cosine Similarity

Lv.2 · LLM Self-Attention Q·K

#L02

3D Cross Product & Surface Normal Vector

Lv.2 · 3D Surface Normal

#L03

Matrix Multiplication & Linear Transformation

Lv.2 · GPU Tensor Cores GEMM

#L04

Determinant & Matrix Inverse

Lv.3 · 3D Mouse Raycasting

#L05

Gaussian Elimination & Linear Systems (Ax = b)

Lv.2 · Linear Solver Ax=b

#L06