-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathlinear_algebra_ml.html
More file actions
1435 lines (1340 loc) · 109 KB
/
Copy pathlinear_algebra_ml.html
File metadata and controls
1435 lines (1340 loc) · 109 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Linear Algebra for ML</title>
<link rel="icon" type="image/svg+xml" href="favicon.svg" />
<link rel="stylesheet" href="site.css" />
</head>
<body>
<div class="mobile-nav">
<select onchange="showTopic(this.value)">
<option value="0">01 — Scalars, Vectors, Matrices, Tensors</option>
<option value="1">02 — Vector & Matrix Operations</option>
<option value="2">03 — Dot Product & Geometric Interpretation</option>
<option value="3">04 — Matrix Multiplication</option>
<option value="4">05 — Identity, Transpose, Inverse</option>
<option value="5">06 — Rank of a Matrix</option>
<option value="6">07 — Systems of Linear Equations</option>
<option value="7">08 — Linear Transformations</option>
<option value="8">09 — Column Space & Null Space</option>
<option value="9">10 — Eigenvalues & Eigenvectors</option>
<option value="10">11 — Orthogonality</option>
<option value="11">12 — Norms & Distances (L1, L2)</option>
</select>
</div>
<div class="app">
<nav class="sidebar">
<div class="sidebar-header">
<h1>🧮 Linear Algebra</h1>
<div class="bn-title">লিনিয়ার অ্যালজেব্রা</div>
<p>The Backbone of ML · AI Engineer's Guide</p>
<div class="progress-bar"><div class="progress-fill" id="progress" style="width:8%"></div></div>
</div>
<div id="nav-buttons"></div>
</nav>
<main class="main" id="main-content"></main>
</div>
<script>
const topics = [
{
title:"Scalars, Vectors, Matrices, Tensors",
bn_title:"স্কেলার, ভেক্টর, ম্যাট্রিক্স, টেন্সর",
tags:["Core Concept","Data Structures","PyTorch/NumPy"],
content:`
<div class="card law-card">
<div class="card-title"><span class="icon">⚡</span> LAW 1 — PREDICTION FIRST</div>
<p><strong>Before anything else, predict:</strong> If I give you a 100×200 image in RGB, what shape is its tensor?</p>
<p style="margin-top:8px;color:var(--teal)">✅ Answer: Shape = <code>(3, 100, 200)</code> — channels × height × width. <em>Or</em> <code>(100, 200, 3)</code> depending on framework. This is tensor thinking!</p>
</div>
<div class="card law-card law2">
<div class="card-title"><span class="icon">🔴</span> LAW 2 — FAILURE MODES</div>
<ul style="padding-left:16px;color:var(--muted)">
<li>❌ Confusing <strong>tensor rank</strong> (number of dimensions) with <strong>matrix rank</strong> (linear independence). They are DIFFERENT!</li>
<li>❌ Thinking a vector is always a column — in code, a 1D array is neither row nor column until you explicitly reshape</li>
<li>❌ Forgetting that scalars ARE tensors (0D tensor)</li>
</ul>
</div>
<div class="card">
<div class="card-title"><span class="icon">📖</span> CORE CONCEPT — English</div>
<p>Think of these as a <span class="hl-blue">hierarchy of containers</span>:</p>
<table>
<tr><th>Name</th><th>Dimensions</th><th>Example</th><th>Shape Notation</th></tr>
<tr><td><span class="hl-green">Scalar</span></td><td>0D — single number</td><td>Temperature = 37.5°C</td><td><code>()</code> or just a number</td></tr>
<tr><td><span class="hl-blue">Vector</span></td><td>1D — list of numbers</td><td>Price list = [100, 200, 350]</td><td><code>(3,)</code></td></tr>
<tr><td><span class="hl-purple">Matrix</span></td><td>2D — table of numbers</td><td>Spreadsheet: rows × cols</td><td><code>(3, 4)</code></td></tr>
<tr><td><span class="hl-red">Tensor</span></td><td>N-dimensional</td><td>RGB image, video batch</td><td><code>(batch, C, H, W)</code></td></tr>
</table>
<div class="formula-label">First-Principle Thinking: Scalar ⊂ Vector ⊂ Matrix ⊂ Tensor</div>
</div>
<div class="card">
<div class="card-title"><span class="icon">🇧🇩</span> বাংলা ব্যাখ্যা</div>
<p class="bn">চিন্তা করুন এটা একটা <strong>ধারকের শ্রেণিবিন্যাস</strong> (hierarchy of containers):</p>
<br>
<p class="bn"><span style="color:var(--green)">স্কেলার (Scalar)</span> → একটাই সংখ্যা। যেমন: তোমার বয়স = ২৫। কোনো দিক নেই, শুধু মাত্রা আছে।</p>
<p class="bn"><span style="color:var(--blue)">ভেক্টর (Vector)</span> → সংখ্যার একটা তালিকা। যেমন: তোমার [উচ্চতা, ওজন, বয়স] = [175, 70, 25]। এটা একটা দিক ও মাত্রা আছে।</p>
<p class="bn"><span style="color:var(--purple)">ম্যাট্রিক্স (Matrix)</span> → সংখ্যার একটা টেবিল (rows × columns)। ক্লাসে ছাত্রদের নম্বরের স্প্রেডশিট চিন্তা করো।</p>
<p class="bn"><span style="color:var(--red)">টেন্সর (Tensor)</span> → যেকোনো মাত্রার সংখ্যার কন্টেইনার। একটা রঙিন ছবি হলো ৩টা ম্যাট্রিক্স (R, G, B) — এটাই একটা ৩D টেন্সর!</p>
<br>
<p class="bn">🎯 <strong>মনে রাখার উপায়:</strong> স্কেলার = বিন্দু, ভেক্টর = রেখা, ম্যাট্রিক্স = সমতল, টেন্সর = ঘনক (বা তার বেশি!)</p>
</div>
<div class="card">
<div class="card-title"><span class="icon">📐</span> FORMULAS & NOTATION</div>
<div class="formula-label">Scalar</div>
<div class="formula-box">a = 5 (just a number, scalar ∈ ℝ)</div>
<div class="formula-label">Vector (column vector, bold lowercase)</div>
<div class="formula-box"><span style="color:var(--blue)">v</span> = [v₁, v₂, v₃]ᵀ = [1, 2, 3]ᵀ (shape: 3×1)</div>
<div class="formula-label">Matrix (uppercase bold)</div>
<div class="formula-box"><span style="color:var(--purple)">A</span> = ⎡1 2 3⎤ (shape: 2×3, meaning 2 rows, 3 cols)
⎣4 5 6⎦</div>
<div class="formula-label">Tensor (in code — PyTorch)</div>
<div class="formula-box">import torch
x = torch.zeros(32, 3, 224, 224) # batch × channels × H × W
print(x.shape) # torch.Size([32, 3, 224, 224])
print(x.ndim) # 4 (this is tensor RANK = number of dimensions)</div>
</div>
<div class="analogy-box">
<div class="card-title"><span class="icon">🎯</span> SIMPLE ANALOGY — বাস্তব উদাহরণ</div>
<p>Think of a <strong>Library</strong>:</p>
<ul style="padding-left:16px;margin-top:8px">
<li>📘 <span class="hl-green">Scalar</span> = The page number on one page (just: 42)</li>
<li>📚 <span class="hl-blue">Vector</span> = A single bookshelf — a list of books in order</li>
<li>🗂️ <span class="hl-purple">Matrix</span> = The entire library floor — rows of shelves, columns of positions</li>
<li>🏢 <span class="hl-red">Tensor</span> = The entire multi-floor library — floor (depth) × row × column</li>
</ul>
<p class="bn" style="margin-top:8px">বাংলায়: স্কেলার=পৃষ্ঠা নম্বর, ভেক্টর=বইয়ের তাক, ম্যাট্রিক্স=লাইব্রেরির একটা তলা, টেন্সর=পুরো বহুতল লাইব্রেরি!</p>
</div>
<div class="ml-app-box">
<h4>🤖 ML/AI Application — How This Is Used</h4>
<table>
<tr><th>ML Concept</th><th>Data Type</th><th>Shape Example</th></tr>
<tr><td>A single pixel value</td><td>Scalar</td><td><code>0.95</code></td></tr>
<tr><td>Word embedding</td><td>Vector</td><td><code>(768,)</code> — BERT word vector</td></tr>
<tr><td>Weight matrix in a layer</td><td>Matrix</td><td><code>(512, 256)</code></td></tr>
<tr><td>Batch of images</td><td>4D Tensor</td><td><code>(32, 3, 224, 224)</code></td></tr>
<tr><td>Video data</td><td>5D Tensor</td><td><code>(batch, frames, C, H, W)</code></td></tr>
</table>
</div>
<div class="card">
<div class="card-title"><span class="icon">💼</span> INTERVIEW Q&A</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q1: What is the difference between a matrix rank and a tensor rank (in ML context)? <span class="q-arrow">▶</span></div>
<div class="a"><span class="hl-blue">Matrix rank</span> = number of linearly independent rows/columns (related to information content). <span class="hl-red">Tensor rank</span> in ML = number of dimensions (axes) in the tensor. Example: A 3D tensor has rank 3. These are completely different concepts — don't confuse them in interviews! <br><br><span class="bn">বাংলায়: ম্যাট্রিক্স র্যাঙ্ক = তথ্যের স্বাধীন সারির সংখ্যা। টেন্সর র্যাঙ্ক = মাত্রার সংখ্যা।</span></div>
</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q2: Why do we use tensors in deep learning instead of just matrices? <span class="q-arrow">▶</span></div>
<div class="a">Real-world data is inherently multi-dimensional. Images have spatial (H,W) + color (C) dimensions. When training in batches, we add a batch dimension. For video, we add time. Tensors naturally represent all this. Also, GPU hardware is optimized for batched tensor operations — processing 32 images simultaneously is much faster than one at a time. <br><br><span class="bn">বাংলায়: বাস্তব ডেটা বহু-মাত্রিক। ছবিতে উচ্চতা, প্রস্থ, রঙ — এগুলো আলাদা মাত্রা। টেন্সর এটা সহজে ধরে রাখে।</span></div>
</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q3: In NumPy, what is the difference between shape (3,) and shape (3,1)? <span class="q-arrow">▶</span></div>
<div class="a"><code>(3,)</code> is a 1D array (vector) — it has no concept of row/column orientation. <code>(3,1)</code> is a 2D column vector — a matrix with 3 rows and 1 column. This distinction matters for broadcasting! <code>np.dot(A, v)</code> behaves differently depending on v's shape. Always use <code>.reshape(-1,1)</code> when you need an explicit column vector. <br><br><span class="bn">বাংলায়: (3,) হলো সরল তালিকা। (3,1) হলো স্পষ্টভাবে কলাম ভেক্টর। এই পার্থক্য broadcasting-এ গুরুত্বপূর্ণ।</span></div>
</div>
</div>
<div class="card">
<div class="card-title"><span class="icon">🏋️</span> EXERCISES — অনুশীলন</div>
<div class="exercise-box">
<h4>Exercise 1 (Basic)</h4>
<p>Identify the shape of each: (a) The number 42.5 → (b) [1, 2, 3, 4, 5] → (c) A 28×28 grayscale image → (d) A batch of 64 RGB images of size 32×32 →</p>
<p style="color:var(--muted);margin-top:6px;font-size:0.88em">Answer: (a) scalar (b) (5,) (c) (28,28) matrix (d) (64,3,32,32) tensor</p>
</div>
<div class="exercise-box">
<h4>Exercise 2 (Code)</h4>
<p>Write NumPy code to create: (a) A zero vector of size 5 (b) A 3×4 matrix of ones (c) A random tensor of shape (2, 3, 4)</p>
<div class="formula-box">import numpy as np
a = np.zeros(5) # (a) zero vector
b = np.ones((3, 4)) # (b) ones matrix
c = np.random.randn(2,3,4) # (c) random tensor
print(a.shape, b.shape, c.shape)</div>
</div>
</div>
<div class="card">
<div class="card-title"><span class="icon">🔗</span> RESOURCES — সম্পদ</div>
<a class="resource-link" href="https://numpy.org/doc/stable/user/quickstart.html" target="_blank">📘 NumPy Quickstart</a>
<a class="resource-link" href="https://pytorch.org/tutorials/beginner/blitz/tensor_tutorial.html" target="_blank">🔥 PyTorch Tensor Tutorial</a>
<a class="resource-link" href="https://www.3blue1brown.com/topics/linear-algebra" target="_blank">🎬 3Blue1Brown: Essence of LA</a>
</div>`
},
{
title:"Vector & Matrix Operations",
bn_title:"ভেক্টর ও ম্যাট্রিক্স অপারেশন",
tags:["Operations","Broadcasting","Element-wise"],
content:`
<div class="card law-card">
<div class="card-title"><span class="icon">⚡</span> LAW 1 — PREDICTION FIRST</div>
<p>Can you add a vector of shape <code>(3,)</code> to a matrix of shape <code>(4, 3)</code>? Think before reading!</p>
<p style="margin-top:8px;color:var(--teal)">✅ YES! This is called <strong>broadcasting</strong>. The vector gets "broadcast" across each of the 4 rows. Result shape: <code>(4, 3)</code>.</p>
</div>
<div class="card law-card law2">
<div class="card-title"><span class="icon">🔴</span> LAW 2 — FAILURE MODES</div>
<ul style="padding-left:16px;color:var(--muted)">
<li>❌ Adding matrices of incompatible shapes (e.g., 3×2 + 2×3) — shape mismatch error!</li>
<li>❌ Confusing element-wise multiply (<code>*</code>) with matrix multiply (<code>@</code> or <code>np.dot</code>)</li>
<li>❌ Forgetting that matrix addition requires <strong>exactly the same shape</strong></li>
</ul>
</div>
<div class="card">
<div class="card-title"><span class="icon">📖</span> CORE CONCEPT — English</div>
<p><strong>Vector Operations:</strong></p>
<ol class="step-list">
<li><strong>Addition/Subtraction:</strong> Add corresponding elements. Same shape required.</li>
<li><strong>Scalar Multiplication:</strong> Multiply every element by the scalar.</li>
<li><strong>Element-wise (Hadamard) Product:</strong> Multiply element by element (same shape).</li>
</ol>
<p style="margin-top:12px"><strong>Matrix Operations:</strong></p>
<ol class="step-list">
<li><strong>Matrix Addition:</strong> Same shape, add element by element.</li>
<li><strong>Scalar × Matrix:</strong> Scale every element.</li>
<li><strong>Broadcasting:</strong> NumPy/PyTorch auto-expand smaller arrays to match shape.</li>
</ol>
</div>
<div class="card">
<div class="card-title"><span class="icon">🇧🇩</span> বাংলা ব্যাখ্যা</div>
<p class="bn"><strong>ভেক্টর যোগ (Vector Addition):</strong> দুটো দলের খেলোয়াড়দের স্কোর যোগ করার মতো — প্রতিটা অবস্থানের সংখ্যা আলাদা আলাদাভাবে যোগ হয়।</p>
<br>
<p class="bn"><strong>স্কেলার গুণ:</strong> যদি প্রতিটা জিনিসের দাম ২ গুণ হয়, তাহলে পুরো মূল্য তালিকাকে ২ দিয়ে গুণ করো।</p>
<br>
<p class="bn"><strong>Hadamard Product (⊙):</strong> Element-wise গুণ। যেমন দুটো ছবির পিক্সেল পরস্পরের সাথে গুণ করা।</p>
<br>
<p class="bn"><strong>Broadcasting:</strong> একটা ছোট অ্যারেকে বড় অ্যারের সাথে মেলানোর জন্য স্বয়ংক্রিয়ভাবে প্রসারিত করা। যেমন: প্রতিটা সারিতে একই বায়াস যোগ করা।</p>
</div>
<div class="card">
<div class="card-title"><span class="icon">📐</span> FORMULAS</div>
<div class="formula-label">Vector Addition</div>
<div class="formula-box"><span style="color:var(--blue)">u</span> + <span style="color:var(--purple)">v</span> = [u₁+v₁, u₂+v₂, u₃+v₃]
Example: [1,2,3] + [4,5,6] = [5,7,9]</div>
<div class="formula-label">Scalar Multiplication</div>
<div class="formula-box">c·<span style="color:var(--blue)">v</span> = [c·v₁, c·v₂, c·v₃]
Example: 3·[1,2,3] = [3,6,9]</div>
<div class="formula-label">Element-wise (Hadamard) Product ⊙</div>
<div class="formula-box"><span style="color:var(--blue)">u</span> ⊙ <span style="color:var(--purple)">v</span> = [u₁·v₁, u₂·v₂, u₃·v₃] (NOT matrix multiply!)
Example: [1,2,3] ⊙ [4,5,6] = [4,10,18]</div>
<div class="formula-label">Broadcasting Example</div>
<div class="formula-box">A shape (4,3) + b shape (3,) → b is broadcast to (4,3)
⎡1 2 3⎤ ⎡10 20 30⎤ ⎡11 22 33⎤
⎢4 5 6⎥ + ⎢10 20 30⎥ = ⎢14 25 36⎥ (b replicated 4 times)
⎢7 8 9⎥ ⎢10 20 30⎥ ⎢17 28 39⎥
⎣2 3 4⎦ ⎣10 20 30⎦ ⎣12 23 34⎦</div>
</div>
<div class="analogy-box">
<div class="card-title"><span class="icon">🎯</span> ANALOGY</div>
<p><strong>Think of a restaurant bill:</strong></p>
<ul style="padding-left:16px">
<li>🍽️ <span class="hl-blue">Vector addition</span> = Adding two tables' orders item by item</li>
<li>💰 <span class="hl-green">Scalar multiply</span> = Applying 20% service charge to every item</li>
<li>🎁 <span class="hl-purple">Hadamard product</span> = Applying a different discount % to each item separately</li>
<li>📢 <span class="hl-red">Broadcasting</span> = Adding the same cover charge to every table</li>
</ul>
</div>
<div class="ml-app-box">
<h4>🤖 ML Application</h4>
<table>
<tr><th>ML Operation</th><th>Mathematical Operation</th></tr>
<tr><td>Add bias to all neurons</td><td>Broadcasting: <code>z = Wx + b</code> (b broadcasts over batch)</td></tr>
<tr><td>Batch normalization</td><td>Element-wise subtract mean, divide by std</td></tr>
<tr><td>Gradient updates</td><td>Vector subtraction: <code>θ ← θ - α·∇θ</code></td></tr>
<tr><td>Attention masking</td><td>Element-wise multiply by mask (0s and 1s)</td></tr>
<tr><td>Dropout</td><td>Hadamard product with random 0/1 mask</td></tr>
</table>
</div>
<div class="card">
<div class="card-title"><span class="icon">💼</span> INTERVIEW Q&A</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q1: Explain broadcasting with an example from neural networks. <span class="q-arrow">▶</span></div>
<div class="a">In a fully connected layer, we compute <code>Z = XW + b</code>. X has shape <code>(batch, features)</code>, W has shape <code>(features, neurons)</code>, so XW has shape <code>(batch, neurons)</code>. The bias <code>b</code> has shape <code>(neurons,)</code>. Broadcasting automatically adds b to every row (every sample in the batch) without needing to manually repeat it. <br><br><span class="bn">বাংলায়: প্রতিটা নমুনার (batch) সাথে একই bias যোগ করতে broadcasting ব্যবহার হয়।</span></div>
</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q2: What is the difference between * and @ (or np.dot) in Python/NumPy? <span class="q-arrow">▶</span></div>
<div class="a"><code>*</code> is element-wise (Hadamard) multiplication — shapes must be broadcastable. <code>@</code> or <code>np.dot</code> is matrix multiplication — inner dimensions must match. Example: for A(2,3) and B(3,4), <code>A@B</code> gives shape (2,4) but <code>A*B</code> would fail (shape mismatch). This is one of the most common bugs in ML code!<br><br><span class="bn">বাংলায়: * মানে প্রতিটা উপাদান আলাদাভাবে গুণ। @ মানে আসল ম্যাট্রিক্স গুণ।</span></div>
</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q3: What are the broadcasting rules in NumPy? <span class="q-arrow">▶</span></div>
<div class="a">NumPy compares shapes from the right (trailing dimensions). For two arrays to be broadcastable: each pair of dimensions must either be equal, or one of them must be 1. Example: (4,3,1) and (3,5) → pad left: (4,3,1) and (1,3,5) → result (4,3,5). Rule: dimension of 1 gets stretched. <br><br><span class="bn">বাংলায়: ডান দিক থেকে মাত্রা তুলনা হয়। যেখানে ১ আছে, সেটা প্রসারিত হয়।</span></div>
</div>
</div>
<div class="card">
<div class="card-title"><span class="icon">🏋️</span> EXERCISES</div>
<div class="exercise-box">
<h4>Exercise 1</h4>
<p>Given u = [2, 4, 6] and v = [1, 3, 5], compute: (a) u + v, (b) 3u - v, (c) u ⊙ v</p>
<p style="color:var(--muted);margin-top:6px;font-size:0.88em">Answer: (a) [3,7,11] (b) [5,9,13] (c) [2,12,30]</p>
</div>
<div class="exercise-box">
<h4>Exercise 2 (Broadcasting)</h4>
<p>Will these broadcast? (a) (3,4) + (4,) → ? (b) (3,4) + (3,) → ? (c) (3,1,4) + (1,2,4) → ?</p>
<p style="color:var(--muted);margin-top:6px;font-size:0.88em">Answer: (a) Yes → (3,4) (b) No — error! (c) Yes → (3,2,4)</p>
</div>
</div>
<div class="card"><div class="card-title"><span class="icon">🔗</span> RESOURCES</div>
<a class="resource-link" href="https://numpy.org/doc/stable/user/basics.broadcasting.html" target="_blank">📘 NumPy Broadcasting Guide</a>
<a class="resource-link" href="https://cs231n.github.io/python-numpy-tutorial/" target="_blank">🎓 CS231n NumPy Tutorial</a>
</div>`
},
{
title:"Dot Product & Geometric Interpretation",
bn_title:"ডট প্রোডাক্ট ও জ্যামিতিক ব্যাখ্যা",
tags:["Similarity","Projection","Cosine Similarity"],
content:`
<div class="card law-card">
<div class="card-title"><span class="icon">⚡</span> LAW 1 — PREDICTION FIRST</div>
<p>What does it mean when two word embeddings have a dot product of 0? Think geometrically!</p>
<p style="margin-top:8px;color:var(--teal)">✅ It means the words are <strong>completely unrelated</strong> — their vectors are perpendicular (orthogonal) in embedding space. Their cosine similarity = 0.</p>
</div>
<div class="card law-card law2">
<div class="card-title"><span class="icon">🔴</span> LAW 2 — FAILURE MODES</div>
<ul style="padding-left:16px;color:var(--muted)">
<li>❌ Confusing dot product (scalar result) with cross product (vector result)</li>
<li>❌ Using raw dot product for similarity without normalizing — MAGNITUDE matters!</li>
<li>❌ Forgetting that cosine similarity = dot product / (|a|·|b|) — raw dot product is NOT similarity</li>
</ul>
</div>
<div class="card">
<div class="card-title"><span class="icon">📖</span> CORE CONCEPT — English</div>
<p>The dot product has <span class="hl-blue">two definitions</span> that are mathematically equivalent:</p>
<div class="grid-2">
<div style="background:var(--bg3);padding:16px;border-radius:8px">
<p><span class="hl-green">Algebraic Definition:</span></p>
<p>Multiply corresponding elements and sum them up.</p>
<code>a·b = Σ aᵢbᵢ</code>
</div>
<div style="background:var(--bg3);padding:16px;border-radius:8px">
<p><span class="hl-blue">Geometric Definition:</span></p>
<p>Product of magnitudes × cosine of angle between them.</p>
<code>a·b = |a||b|cos(θ)</code>
</div>
</div>
<p style="margin-top:16px"><strong>What the sign tells you:</strong></p>
<table>
<tr><th>Dot Product</th><th>Angle θ</th><th>Meaning</th></tr>
<tr><td style="color:var(--green)">a·b > 0</td><td>θ < 90°</td><td>Vectors point in similar direction</td></tr>
<tr><td style="color:var(--muted)">a·b = 0</td><td>θ = 90°</td><td>Orthogonal — completely unrelated</td></tr>
<tr><td style="color:var(--red)">a·b < 0</td><td>θ > 90°</td><td>Vectors point in opposite direction</td></tr>
</table>
</div>
<div class="card">
<div class="card-title"><span class="icon">🇧🇩</span> বাংলা ব্যাখ্যা</div>
<p class="bn"><strong>ডট প্রোডাক্ট</strong> দুটো ভেক্টরের মধ্যে <strong>কতটা একই দিকে আছে</strong> সেটা মাপে।</p>
<br>
<p class="bn">💡 <strong>সহজ উদাহরণ:</strong> ধরো তুমি উত্তর দিকে ৫ কিমি হেঁটেছ (ভেক্টর a)। বাতাস উত্তর-পূর্ব দিকে বইছে (ভেক্টর b)। ডট প্রোডাক্ট বলবে বাতাস তোমার যাত্রাতে কতটা "সাহায্য" করছে।</p>
<br>
<p class="bn"><strong>কোসাইন সিমিলারিটি (Cosine Similarity):</strong> শুধু দিক দেখে, মাত্রা দেখে না। মেশিন লার্নিংয়ে দুটো শব্দ কতটা কাছের সেটা মাপতে ব্যবহার হয়।</p>
<br>
<p class="bn">cos(θ) = <strong>a·b / (|a|·|b|)</strong></p>
<p class="bn">→ মান ১ মানে একই দিক (identical), -১ মানে বিপরীত দিক, ০ মানে কোনো সম্পর্ক নেই।</p>
</div>
<div class="card">
<div class="card-title"><span class="icon">📐</span> FORMULAS</div>
<div class="formula-label">Dot Product (Algebraic)</div>
<div class="formula-box"><span style="color:var(--blue)">a</span>·<span style="color:var(--purple)">b</span> = a₁b₁ + a₂b₂ + ... + aₙbₙ = Σᵢ aᵢbᵢ
Example: [1,2,3]·[4,5,6] = 1×4 + 2×5 + 3×6 = 4+10+18 = <span style="color:var(--green)">32</span></div>
<div class="formula-label">Dot Product (Geometric)</div>
<div class="formula-box"><span style="color:var(--blue)">a</span>·<span style="color:var(--purple)">b</span> = |<span style="color:var(--blue)">a</span>|·|<span style="color:var(--purple)">b</span>|·cos(θ)
where |a| = √(a₁²+a₂²+...+aₙ²) [L2 norm = magnitude]</div>
<div class="formula-label">Cosine Similarity (used in NLP, recommendation systems)</div>
<div class="formula-box">cos_sim(<span style="color:var(--blue)">a</span>,<span style="color:var(--purple)">b</span>) = (<span style="color:var(--blue)">a</span>·<span style="color:var(--purple)">b</span>) / (|<span style="color:var(--blue)">a</span>| × |<span style="color:var(--purple)">b</span>|) ∈ [-1, 1]
Python: from sklearn.metrics.pairwise import cosine_similarity</div>
<div class="formula-label">Projection of a onto b</div>
<div class="formula-box">proj_b(a) = (a·b / |b|²) × b [vector projection]
scalar projection = a·b / |b| = |a|cos(θ)</div>
</div>
<div class="analogy-box">
<div class="card-title"><span class="icon">🎯</span> ANALOGY</div>
<p><strong>Think of pushing a door:</strong></p>
<ul style="padding-left:16px">
<li>💪 Pushing <strong>perpendicular to the door</strong> (same direction as it swings) → maximum dot product → door opens easily</li>
<li>😅 Pushing at <strong>45°</strong> → partial dot product → some of your force wasted</li>
<li>🙅 Pushing <strong>along the door's surface</strong> (parallel to door, 90° to motion) → dot product = 0 → door doesn't move!</li>
</ul>
<p class="bn" style="margin-top:8px">বাংলায়: ডট প্রোডাক্ট = আপনার শক্তির কতটা সত্যিকারের "কাজে" লাগছে সেটার পরিমাপ।</p>
</div>
<div class="ml-app-box">
<h4>🤖 ML Application</h4>
<table>
<tr><th>ML Use Case</th><th>How Dot Product Is Used</th></tr>
<tr><td>Neural network forward pass</td><td><code>z = w·x + b</code> — weighted sum of inputs</td></tr>
<tr><td>Attention mechanism (Transformer)</td><td><code>score = Q·Kᵀ / √d_k</code> — query-key similarity</td></tr>
<tr><td>Word2Vec similarity</td><td>Cosine sim between word vectors</td></tr>
<tr><td>Recommendation systems</td><td>User vector · Item vector = predicted preference</td></tr>
<tr><td>SVM (Support Vector Machine)</td><td>Decision boundary via dot products (kernel trick)</td></tr>
</table>
</div>
<div class="card">
<div class="card-title"><span class="icon">💼</span> INTERVIEW Q&A</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q1: In the Transformer attention mechanism, why is the dot product scaled by √d_k? <span class="q-arrow">▶</span></div>
<div class="a">When d_k (key dimension) is large, dot products grow large in magnitude, pushing softmax into regions with very small gradients (vanishing gradient). Dividing by √d_k keeps dot products in a reasonable range. Example: if d_k=64, scale by 1/8. This stabilizes training.<br><br><span class="bn">বাংলায়: বড় মাত্রায় ডট প্রোডাক্ট খুব বড় হয়ে যায়, যা softmax-কে অকার্যকর করে দেয়। √d_k দিয়ে ভাগ করলে মান নিয়ন্ত্রণে থাকে।</span></div>
</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q2: Why use cosine similarity instead of Euclidean distance for text similarity? <span class="q-arrow">▶</span></div>
<div class="a">Cosine similarity is <strong>magnitude-invariant</strong> — it only cares about direction. A document with the word "cat" appearing 100 times vs 10 times should still be similar to another document about cats. Euclidean distance would treat the 100-occurrence document as very "far away" just because of length. Cosine similarity normalizes for length, making it ideal for text comparison.<br><br><span class="bn">বাংলায়: কোসাইন সিমিলারিটি শুধু দিক দেখে, আকার দেখে না — তাই দীর্ঘ ও সংক্ষিপ্ত টেক্সট ন্যায্যভাবে তুলনা হয়।</span></div>
</div>
</div>
<div class="card"><div class="card-title"><span class="icon">🏋️</span> EXERCISES</div>
<div class="exercise-box">
<h4>Exercise 1</h4>
<p>Compute: (a) [1,0,0]·[0,1,0] = ? (b) [1,1]·[1,1] = ? What angle is between [1,0] and [0,1]?</p>
<p style="color:var(--muted);margin-top:6px;font-size:0.88em">Answers: (a) 0 (orthogonal!) (b) 2. Angle between [1,0] and [0,1] = 90°</p>
</div>
<div class="exercise-box">
<h4>Exercise 2 (Conceptual)</h4>
<p>Two word vectors: "king" and "queen" have cosine similarity 0.85, "king" and "car" have cosine similarity 0.12. What does this tell us about the embedding space?</p>
<p style="color:var(--muted);margin-top:6px;font-size:0.88em">The model learned that king and queen are semantically close (royalty concept), while king and car are almost unrelated. The embedding space captures meaning!</p>
</div>
</div>
<div class="card"><div class="card-title"><span class="icon">🔗</span> RESOURCES</div>
<a class="resource-link" href="https://www.3blue1brown.com/lessons/dot-products" target="_blank">🎬 3Blue1Brown: Dot Products</a>
<a class="resource-link" href="https://jalammar.github.io/illustrated-transformer/" target="_blank">🤖 Illustrated Transformer (attention)</a>
</div>`
},
{
title:"Matrix Multiplication",
bn_title:"ম্যাট্রিক্স গুণন",
tags:["Neural Networks","Transformations","Core Operation"],
content:`
<div class="card law-card">
<div class="card-title"><span class="icon">⚡</span> LAW 1 — PREDICTION FIRST</div>
<p>A weight matrix W has shape (512, 256) and an input x has shape (256,). What is the output shape of Wx?</p>
<p style="margin-top:8px;color:var(--teal)">✅ Output shape = <strong>(512,)</strong>. The inner dimension (256) cancels out. You're mapping a 256-dim input to a 512-dim output — this is what a neural network layer does!</p>
</div>
<div class="card law-card law2">
<div class="card-title"><span class="icon">🔴</span> LAW 2 — FAILURE MODES</div>
<ul style="padding-left:16px;color:var(--muted)">
<li>❌ Assuming AB = BA (matrix multiply is NOT commutative!)</li>
<li>❌ Getting the shape rule wrong: (m×<span style="color:var(--red)">n</span>) × (<span style="color:var(--red)">n</span>×p) = (m×p) — inner dims must match</li>
<li>❌ Using <code>*</code> instead of <code>@</code> in Python (element-wise vs matrix multiply)</li>
</ul>
</div>
<div class="card">
<div class="card-title"><span class="icon">📖</span> CORE CONCEPT — English</div>
<p><strong>The Golden Rule:</strong> (m×<span class="hl-red">n</span>) × (<span class="hl-red">n</span>×p) = (m×p)</p>
<p style="margin-top:8px">The <strong>inner dimensions must match</strong>. The result has the <strong>outer dimensions</strong>.</p>
<p style="margin-top:12px"><strong>How to compute element (i,j) of result C = AB:</strong></p>
<div style="background:var(--bg3);padding:14px;border-radius:8px;margin-top:8px">
<p>C[i,j] = <strong>dot product of row i of A with column j of B</strong></p>
<p style="color:var(--muted);font-size:0.9em;margin-top:4px">C[i,j] = Σₖ A[i,k] × B[k,j]</p>
</div>
<p style="margin-top:12px"><strong>Properties to remember:</strong></p>
<ul style="padding-left:16px">
<li>NOT commutative: AB ≠ BA (usually)</li>
<li>IS associative: (AB)C = A(BC)</li>
<li>IS distributive: A(B+C) = AB + AC</li>
<li>Transpose rule: (AB)ᵀ = BᵀAᵀ ← note the order reverses!</li>
</ul>
</div>
<div class="card">
<div class="card-title"><span class="icon">🇧🇩</span> বাংলা ব্যাখ্যা</div>
<p class="bn"><strong>মনে রাখার নিয়ম:</strong> (m×<span style="color:var(--red)">n</span>) × (<span style="color:var(--red)">n</span>×p) = (m×p)</p>
<p class="bn">ভেতরের সংখ্যা দুটো <span style="color:var(--red)">একসাথে মিলিয়ে যায়</span>, বাইরের সংখ্যা দুটো ফলাফলের আকার।</p>
<br>
<p class="bn"><strong>C[i,j] কীভাবে হিসাব করবো?</strong></p>
<p class="bn">A এর i-তম সারি নাও, B এর j-তম কলাম নাও, এদের ডট প্রোডাক্ট নাও → সেটাই C[i,j]</p>
<br>
<p class="bn">⚠️ <strong>গুরুত্বপূর্ণ:</strong> AB ≠ BA সাধারণত! ম্যাট্রিক্স গুণ ক্রমবিনিময়যোগ্য নয়।</p>
<p class="bn">(AB)ᵀ = BᵀAᵀ — ট্রান্সপোজ করলে ক্রম উল্টে যায়!</p>
</div>
<div class="card">
<div class="card-title"><span class="icon">📐</span> STEP-BY-STEP EXAMPLE</div>
<div class="formula-label">A (2×3) × B (3×2) = C (2×2)</div>
<div class="formula-box">A = ⎡1 2 3⎤ B = ⎡7 8 ⎤
⎣4 5 6⎦ ⎢9 10⎥
⎣11 12⎦
C[0,0] = row0_A · col0_B = 1×7 + 2×9 + 3×11 = 7+18+33 = <span style="color:var(--green)">58</span>
C[0,1] = row0_A · col1_B = 1×8 + 2×10 + 3×12 = 8+20+36 = <span style="color:var(--green)">64</span>
C[1,0] = row1_A · col0_B = 4×7 + 5×9 + 6×11 = 28+45+66 = <span style="color:var(--green)">139</span>
C[1,1] = row1_A · col1_B = 4×8 + 5×10 + 6×12 = 32+50+72 = <span style="color:var(--green)">154</span>
C = ⎡ 58 64⎤
⎣139 154⎦</div>
<div class="formula-label">In Python</div>
<div class="formula-box">import numpy as np
A = np.array([[1,2,3],[4,5,6]]) # shape (2,3)
B = np.array([[7,8],[9,10],[11,12]]) # shape (3,2)
C = A @ B # or np.matmul(A, B)
# C.shape = (2,2), C = [[58,64],[139,154]]</div>
</div>
<div class="analogy-box">
<div class="card-title"><span class="icon">🎯</span> ANALOGY — Currency Exchange</div>
<p>Imagine you have <strong>prices in 3 currencies</strong> (matrix A: items × currencies). You have <strong>exchange rates</strong> (matrix B: currencies × target_currencies). Matrix multiply A×B gives you prices in all target currencies at once — every element of the result combines a row of A (item prices) with a column of B (exchange rates).</p>
<p class="bn" style="margin-top:8px">বাংলায়: ৩টা মুদ্রায় পণ্যের দাম × বিনিময় হার = নতুন মুদ্রায় পণ্যের দাম। ম্যাট্রিক্স গুণ এই পুরো কাজটা একসাথে করে!</p>
</div>
<div class="ml-app-box">
<h4>🤖 ML Application — Every Neural Network Layer IS Matrix Multiplication</h4>
<table>
<tr><th>Layer Type</th><th>Operation</th><th>Shapes</th></tr>
<tr><td>Dense / Linear</td><td><code>Y = XW + b</code></td><td>X:(batch,in) × W:(in,out) → Y:(batch,out)</td></tr>
<tr><td>Conv (as matrix)</td><td>im2col + matmul</td><td>Unrolled patches × filters</td></tr>
<tr><td>Multi-head attention</td><td><code>Attention(Q,K,V) = softmax(QKᵀ/√d)V</code></td><td>Multiple matmuls</td></tr>
<tr><td>Backpropagation</td><td>Chain rule via matmul</td><td>δ = δₙₑₓₜ × Wᵀ</td></tr>
</table>
</div>
<div class="card">
<div class="card-title"><span class="icon">💼</span> INTERVIEW Q&A</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q1: Why is matrix multiplication so important in deep learning? <span class="q-arrow">▶</span></div>
<div class="a">Every layer in a neural network fundamentally performs matrix multiplication (plus non-linearity). The weight matrix W transforms input representations from one space to another. GPUs are specifically optimized for BLAS (Basic Linear Algebra Subprograms) operations, especially matrix multiplications, making them ideal for deep learning. The entire forward and backward pass is a chain of matrix multiplications.<br><br><span class="bn">বাংলায়: নিউরাল নেটওয়ার্কের প্রতিটি স্তর মূলত ম্যাট্রিক্স গুণ। GPU এই অপারেশনের জন্যই তৈরি।</span></div>
</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q2: What does it mean geometrically when you multiply a vector by a matrix? <span class="q-arrow">▶</span></div>
<div class="a">A matrix multiplication Mv transforms a vector v — it can rotate it, scale it, reflect it, or project it to a different dimension. Neural network layers learn to transform input data into more useful representations. A 3→2 layer projects 3D data onto a 2D plane; a 2→4 layer embeds 2D data into 4D space. The model learns the best transformation W for the task.<br><br><span class="bn">বাংলায়: ম্যাট্রিক্স দিয়ে গুণ মানে ভেক্টরকে ঘোরানো, প্রসারিত করা, বা ভিন্ন মাত্রায় নিয়ে যাওয়া।</span></div>
</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q3: Why is (AB)ᵀ = BᵀAᵀ and not AᵀBᵀ? <span class="q-arrow">▶</span></div>
<div class="a">Because matrix multiplication requires inner dimensions to match. If AB is valid (A is m×n, B is n×p), then Aᵀ is n×m and Bᵀ is p×n. AᵀBᵀ would need n×m times p×n — inner dims n≠p, which doesn't work! BᵀAᵀ is p×n times n×m = p×m, which matches (AB)ᵀ which is p×m. This is critical in backpropagation calculations!<br><br><span class="bn">বাংলায়: ট্রান্সপোজ করলে গুণের ক্রম উল্টে দিতে হয়, নইলে মাত্রা মেলে না।</span></div>
</div>
</div>
<div class="card"><div class="card-title"><span class="icon">🏋️</span> EXERCISES</div>
<div class="exercise-box">
<h4>Exercise 1</h4>
<p>Which of these matrix multiplications are valid? Give the output shape:<br>
(a) (3,4)×(4,5) → ? (b) (4,3)×(4,5) → ? (c) (3,)×(3,4) → ? (d) (5,3)×(3,) → ?</p>
<p style="color:var(--muted);font-size:0.88em;margin-top:6px">Answers: (a) Valid → (3,5) (b) Invalid! (c) Valid → (4,) [treated as 1×3 × 3×4] (d) Valid → (5,)</p>
</div>
<div class="exercise-box">
<h4>Exercise 2 (Neural Network)</h4>
<p>Input batch: X shape (32, 784) (batch of 32 MNIST images). Layer 1: W₁(784,256), b₁(256,). Layer 2: W₂(256,10), b₂(10,). What are the shapes at each step?</p>
<p style="color:var(--muted);font-size:0.88em;margin-top:6px">Z₁ = XW₁+b₁ → (32,256). Z₂ = Z₁W₂+b₂ → (32,10). Output: 32 samples, 10 class scores each.</p>
</div>
</div>
<div class="card"><div class="card-title"><span class="icon">🔗</span> RESOURCES</div>
<a class="resource-link" href="https://www.youtube.com/watch?v=XkY2DOUCWMU" target="_blank">🎬 3B1B: Matrix Multiplication</a>
<a class="resource-link" href="https://cs231n.github.io/linear-classify/" target="_blank">🎓 CS231n: Linear Classification</a>
</div>`
},
{
title:"Identity, Transpose, Inverse",
bn_title:"আইডেন্টিটি, ট্রান্সপোজ, ইনভার্স",
tags:["Matrix Properties","Inverse","Transpose"],
content:`
<div class="card law-card">
<div class="card-title"><span class="icon">⚡</span> LAW 1 — PREDICTION FIRST</div>
<p>For a weight matrix W, what does W·Wᵀ represent? And when does W⁻¹ not exist?</p>
<p style="margin-top:8px;color:var(--teal)">✅ W·Wᵀ gives a square matrix — it's related to correlations between rows of W. W⁻¹ doesn't exist when W is <strong>singular</strong> (determinant = 0) — meaning rows/columns are linearly dependent.</p>
</div>
<div class="card law-card law2">
<div class="card-title"><span class="icon">🔴</span> LAW 2 — FAILURE MODES</div>
<ul style="padding-left:16px;color:var(--muted)">
<li>❌ Trying to invert a non-square matrix (only square matrices have inverses)</li>
<li>❌ Using inverse when pseudo-inverse (Moore-Penrose) is needed for rectangular matrices</li>
<li>❌ Numerically computing inverses for large matrices — use LU decomposition or solve instead!</li>
<li>❌ Forgetting (Aᵀ)ᵀ = A and (A⁻¹)⁻¹ = A</li>
</ul>
</div>
<div class="card">
<div class="card-title"><span class="icon">📖</span> CORE CONCEPT — English</div>
<p><strong>1. Identity Matrix I</strong> — The "1" of matrices:</p>
<ul style="padding-left:16px;margin:8px 0">
<li>Square matrix with 1s on the diagonal, 0s everywhere else</li>
<li>For any matrix A: <strong>AI = IA = A</strong></li>
<li>Like multiplying by 1 in scalar arithmetic</li>
</ul>
<p style="margin-top:12px"><strong>2. Transpose Aᵀ</strong> — Flip along the diagonal:</p>
<ul style="padding-left:16px;margin:8px 0">
<li><strong>(Aᵀ)[i,j] = A[j,i]</strong> — rows become columns</li>
<li>Shape (m×n) → (n×m)</li>
<li>Key rule: (AB)ᵀ = BᵀAᵀ (order reverses!)</li>
</ul>
<p style="margin-top:12px"><strong>3. Inverse A⁻¹</strong> — The "reciprocal" of a matrix:</p>
<ul style="padding-left:16px;margin:8px 0">
<li><strong>A·A⁻¹ = A⁻¹·A = I</strong></li>
<li>Exists only if A is square AND non-singular (det(A) ≠ 0)</li>
<li>For solving Ax = b: x = A⁻¹b</li>
<li>Key rule: (AB)⁻¹ = B⁻¹A⁻¹ (order reverses!)</li>
</ul>
</div>
<div class="card">
<div class="card-title"><span class="icon">🇧🇩</span> বাংলা ব্যাখ্যা</div>
<p class="bn"><strong>আইডেন্টিটি ম্যাট্রিক্স (I):</strong> এটা ম্যাট্রিক্সের "১" এর মতো। যেকোনো ম্যাট্রিক্সকে I দিয়ে গুণ করলে সেই ম্যাট্রিক্সই পাওয়া যায়।</p>
<br>
<p class="bn"><strong>ট্রান্সপোজ (Aᵀ):</strong> ম্যাট্রিক্সকে কর্ণরেখা (diagonal) বরাবর উল্টে দাও। সারি হয়ে যাবে কলাম, কলাম হয়ে যাবে সারি।</p>
<br>
<p class="bn"><strong>ইনভার্স (A⁻¹):</strong> ম্যাট্রিক্সের "ভগ্নাংশ" বা "বিপরীত"। যেমন 3 × (1/3) = 1, তেমনি A × A⁻¹ = I।</p>
<br>
<p class="bn">⚠️ <strong>সাবধান:</strong> সব ম্যাট্রিক্সের ইনভার্স থাকে না! শুধু square ম্যাট্রিক্স এবং যার determinant ≠ 0, তারই ইনভার্স আছে।</p>
</div>
<div class="card">
<div class="card-title"><span class="icon">📐</span> FORMULAS</div>
<div class="formula-label">Identity Matrix (3×3)</div>
<div class="formula-box">I₃ = ⎡1 0 0⎤ AI = IA = A for any compatible A
⎢0 1 0⎥
⎣0 0 1⎦</div>
<div class="formula-label">Transpose</div>
<div class="formula-box">A = ⎡1 2 3⎤ → Aᵀ = ⎡1 4⎤
⎣4 5 6⎦ ⎢2 5⎥
⎣3 6⎦
(Aᵀ)ᵀ = A (AB)ᵀ = BᵀAᵀ (cA)ᵀ = cAᵀ</div>
<div class="formula-label">Inverse (2×2 formula)</div>
<div class="formula-box">A = ⎡a b⎤ → A⁻¹ = 1/det(A) × ⎡ d -b⎤
⎣c d⎦ ⎣-c a⎦
where det(A) = ad - bc (must be ≠ 0!)
Properties: AA⁻¹ = I (AB)⁻¹ = B⁻¹A⁻¹ (Aᵀ)⁻¹ = (A⁻¹)ᵀ</div>
<div class="formula-label">Orthogonal Matrix (special case: Qᵀ = Q⁻¹)</div>
<div class="formula-box">If Q is orthogonal: QᵀQ = QQᵀ = I → Q⁻¹ = Qᵀ (cheap to compute!)</div>
</div>
<div class="ml-app-box">
<h4>🤖 ML Application</h4>
<table>
<tr><th>ML Operation</th><th>Uses Which Property</th></tr>
<tr><td>Normal Equation (Linear Regression)</td><td>Inverse: <code>θ = (XᵀX)⁻¹Xᵀy</code></td></tr>
<tr><td>Whitening/decorrelating data</td><td>Inverse of covariance matrix</td></tr>
<tr><td>Attention in Transformers</td><td>Transpose: <code>QKᵀ</code></td></tr>
<tr><td>Weight initialization (orthogonal)</td><td>Orthogonal matrices preserve norms</td></tr>
<tr><td>Batch normalization</td><td>Identity initialization for scale/shift</td></tr>
</table>
</div>
<div class="card">
<div class="card-title"><span class="icon">💼</span> INTERVIEW Q&A</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q1: Why should you never explicitly compute the matrix inverse in ML code? <span class="q-arrow">▶</span></div>
<div class="a">Computing A⁻¹ is numerically unstable and O(n³) expensive. For Ax=b, instead of x=A⁻¹b, use <code>np.linalg.solve(A, b)</code> which uses LU decomposition — more stable and efficient. The normal equation <code>(XᵀX)⁻¹Xᵀy</code> also fails when XᵀX is nearly singular (multicollinearity). That's why gradient descent is preferred over the normal equation for large-scale ML.<br><br><span class="bn">বাংলায়: ইনভার্স সরাসরি হিসাব করা ধীর ও অস্থিতিশীল। <code>np.linalg.solve</code> ব্যবহার করুন।</span></div>
</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q2: What is a pseudo-inverse and when is it used? <span class="q-arrow">▶</span></div>
<div class="a">The Moore-Penrose pseudo-inverse (A⁺) generalizes the inverse for non-square or singular matrices. For an m×n matrix A, A⁺ is n×m. It gives the minimum-norm least-squares solution to Ax=b. In ML, it's used in linear regression with redundant features (rank-deficient XᵀX). <code>np.linalg.pinv(A)</code> computes it via SVD.<br><br><span class="bn">বাংলায়: যে ম্যাট্রিক্সের সাধারণ ইনভার্স নেই, তার জন্য pseudo-inverse ব্যবহার হয়।</span></div>
</div>
</div>
<div class="card"><div class="card-title"><span class="icon">🏋️</span> EXERCISES</div>
<div class="exercise-box">
<h4>Exercise 1</h4>
<p>For A = [[2,1],[5,3]], find A⁻¹ using the 2×2 formula. Verify that A·A⁻¹ = I.</p>
<p style="color:var(--muted);font-size:0.88em;margin-top:6px">det(A) = 2×3 - 1×5 = 1. A⁻¹ = [[3,-1],[-5,2]]. Check: [[2,1],[5,3]]×[[3,-1],[-5,2]] = [[1,0],[0,1]] ✓</p>
</div>
</div>
<div class="card"><div class="card-title"><span class="icon">🔗</span> RESOURCES</div>
<a class="resource-link" href="https://www.khanacademy.org/math/linear-algebra/matrix-transformations" target="_blank">📘 Khan Academy: Matrix Transforms</a>
<a class="resource-link" href="https://numpy.org/doc/stable/reference/routines.linalg.html" target="_blank">📘 NumPy Linear Algebra</a>
</div>`
},
{
title:"Rank of a Matrix",
bn_title:"ম্যাট্রিক্সের র্যাঙ্ক",
tags:["Linear Independence","Dimensionality","Feature Engineering"],
content:`
<div class="card law-card">
<div class="card-title"><span class="icon">⚡</span> LAW 1 — PREDICTION FIRST</div>
<p>You have a dataset with 10 features, but feature 3 = 2×feature 1. What is the rank of your feature matrix?</p>
<p style="margin-top:8px;color:var(--teal)">✅ Rank = <strong>9, not 10</strong>! Feature 3 is linearly dependent on feature 1 — it adds no new information. This is multicollinearity, and it makes the matrix rank-deficient.</p>
</div>
<div class="card law-card law2">
<div class="card-title"><span class="icon">🔴</span> LAW 2 — FAILURE MODES</div>
<ul style="padding-left:16px;color:var(--muted)">
<li>❌ Thinking rank = number of rows or columns (it's the number of <em>independent</em> ones)</li>
<li>❌ Ignoring rank deficiency in linear regression — causes degenerate solutions</li>
<li>❌ Not understanding that rank tells you the "true" dimensionality of data</li>
</ul>
</div>
<div class="card">
<div class="card-title"><span class="icon">📖</span> CORE CONCEPT — English</div>
<p>The rank of a matrix A is the <span class="hl-blue">number of linearly independent rows (or columns)</span>.</p>
<p style="margin-top:8px"><strong>What "linearly independent" means:</strong> A set of vectors is linearly independent if <em>no vector can be written as a combination of the others</em>.</p>
<div style="background:var(--bg3);padding:12px;border-radius:8px;margin:12px 0">
<p style="color:var(--muted)">Example: [1,0], [0,1] — INDEPENDENT (you can't get [0,1] from [1,0])</p>
<p style="color:var(--muted)">Example: [1,0], [0,1], [2,1] — DEPENDENT! [2,1] = 2×[1,0] + 1×[0,1]</p>
</div>
<p><strong>Key facts:</strong></p>
<ul style="padding-left:16px">
<li>rank(A) ≤ min(m, n) — can't exceed smaller dimension</li>
<li>Full rank: rank = min(m,n) — no redundancy</li>
<li>rank(A) = rank(Aᵀ) — row rank = column rank</li>
<li>rank(A) = 0 only if A is the zero matrix</li>
</ul>
</div>
<div class="card">
<div class="card-title"><span class="icon">🇧🇩</span> বাংলা ব্যাখ্যা</div>
<p class="bn">র্যাঙ্ক হলো ম্যাট্রিক্সের <strong>প্রকৃত তথ্যের পরিমাণ</strong>।</p>
<br>
<p class="bn">💡 <strong>সহজ উদাহরণ:</strong> তোমার কাছে ৩টা বন্ধু আছে এবং তাদের উচ্চতা, ওজন, এবং BMI আছে। কিন্তু BMI = ওজন/উচ্চতা² — তাই BMI আলাদা তথ্য দেয় না। এই ডেটাসেটের rank = ২, কারণ মাত্র ২টা feature স্বাধীন।</p>
<br>
<p class="bn"><strong>Full Rank:</strong> প্রতিটা feature সত্যিকারের নতুন তথ্য দেয়।</p>
<p class="bn"><strong>Rank Deficient:</strong> কিছু feature অন্যদের combination — অপ্রয়োজনীয় তথ্য আছে।</p>
<br>
<p class="bn">মনে রাখো: rank(A) ≤ min(সারির সংখ্যা, কলামের সংখ্যা)</p>
</div>
<div class="card">
<div class="card-title"><span class="icon">📐</span> FORMULAS & EXAMPLES</div>
<div class="formula-label">Finding rank via row reduction</div>
<div class="formula-box">A = ⎡1 2 3⎤ Row reduce → ⎡1 2 3⎤
⎢4 5 6⎥ ⎢0 -3 -6⎥
⎣7 8 9⎦ ⎣0 0 0⎦
→ rank(A) = <span style="color:var(--green)">2</span> (only 2 non-zero rows after reduction)
Note: row 3 = 2×row2 - row1 (linearly dependent!)</div>
<div class="formula-label">Rank in Python</div>
<div class="formula-box">import numpy as np
A = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(np.linalg.matrix_rank(A)) # Output: 2</div>
<div class="formula-label">Rank-Nullity Theorem</div>
<div class="formula-box">rank(A) + nullity(A) = n (n = number of columns)
where nullity = dimension of null space (# of free variables)</div>
</div>
<div class="ml-app-box">
<h4>🤖 ML Application</h4>
<table>
<tr><th>ML Context</th><th>Rank Matters Because</th></tr>
<tr><td>Linear regression (multicollinearity)</td><td>If X is rank-deficient, (XᵀX)⁻¹ doesn't exist</td></tr>
<tr><td>PCA / Dimensionality reduction</td><td>Finds the true rank (intrinsic dimensionality) of data</td></tr>
<tr><td>Low-rank factorization (LoRA, SVD)</td><td>Approximate high-rank matrices with low-rank ones</td></tr>
<tr><td>Recommendation systems (matrix factorization)</td><td>User-item matrix often has low rank</td></tr>
<tr><td>Neural network compression</td><td>Weight matrices often have low effective rank</td></tr>
</table>
</div>
<div class="card">
<div class="card-title"><span class="icon">💼</span> INTERVIEW Q&A</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q1: What is LoRA and how does rank relate to it? <span class="q-arrow">▶</span></div>
<div class="a">LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method. Instead of updating a full weight matrix W (potentially billions of parameters), it learns two small matrices: A (d×r) and B (r×d) where r ≪ d. The update ΔW = BA has rank at most r. This works because pre-trained model weight updates tend to have intrinsically low rank. With r=8, you update only 2×d×8 parameters instead of d² — massive reduction!<br><br><span class="bn">বাংলায়: LoRA বড় weight matrix আপডেটের বদলে দুটো ছোট low-rank matrix শেখে, যা অনেক কম parameter দিয়ে fine-tuning করতে দেয়।</span></div>
</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q2: What is multicollinearity and why is it a problem in ML? <span class="q-arrow">▶</span></div>
<div class="a">Multicollinearity occurs when features are linearly correlated (rank-deficient feature matrix). Problems: (1) The normal equation (XᵀX)⁻¹ becomes numerically unstable or undefined. (2) Coefficients become unreliable — tiny data changes cause huge coefficient swings. (3) Hard to interpret feature importance. Solutions: remove correlated features, use PCA, add L2 regularization (Ridge), or use gradient descent instead of normal equation.<br><br><span class="bn">বাংলায়: মাল্টিকোলিনিয়ারিটি মানে features পরস্পর correlated, যা regression অস্থির করে দেয়।</span></div>
</div>
</div>
<div class="card"><div class="card-title"><span class="icon">🏋️</span> EXERCISES</div>
<div class="exercise-box">
<h4>Exercise</h4>
<p>Find the rank of: A=[[1,2],[2,4]] and B=[[1,0,0],[0,1,0],[0,0,1],[1,1,0]]</p>
<p style="color:var(--muted);font-size:0.88em;margin-top:6px">A: row2 = 2×row1, so rank=1. B: first 3 rows are independent (identity), row4 = row1+row2, rank=3</p>
</div>
</div>
<div class="card"><div class="card-title"><span class="icon">🔗</span> RESOURCES</div>
<a class="resource-link" href="https://www.youtube.com/watch?v=uQhTuRlWMxw" target="_blank">🎬 3B1B: Linear Independence</a>
<a class="resource-link" href="https://arxiv.org/abs/2106.09685" target="_blank">📄 LoRA Paper</a>
</div>`
},
{
title:"Systems of Linear Equations",
bn_title:"রৈখিক সমীকরণ পদ্ধতি",
tags:["Ax=b","Least Squares","Normal Equation"],
content:`
<div class="card law-card">
<div class="card-title"><span class="icon">⚡</span> LAW 1 — PREDICTION FIRST</div>
<p>In linear regression, the "normal equation" is θ = (XᵀX)⁻¹Xᵀy. When should you use gradient descent instead?</p>
<p style="margin-top:8px;color:var(--teal)">✅ Use gradient descent when n (features) > ~10,000. Normal equation requires inverting XᵀX which is O(n³) — becomes too slow. Also use gradient descent when XᵀX is rank-deficient (multicollinearity).</p>
</div>
<div class="card law-card law2">
<div class="card-title"><span class="icon">🔴</span> LAW 2 — FAILURE MODES</div>
<ul style="padding-left:16px;color:var(--muted)">
<li>❌ Assuming every Ax=b system has a solution — might be overdetermined (no solution) or underdetermined (infinite solutions)</li>
<li>❌ Computing (XᵀX)⁻¹ directly — numerically unstable, use np.linalg.solve or lstsq instead</li>
<li>❌ Forgetting that gradient descent for least squares IS solving Ax=b iteratively</li>
</ul>
</div>
<div class="card">
<div class="card-title"><span class="icon">📖</span> CORE CONCEPT — English</div>
<p>A system of linear equations in matrix form: <strong>Ax = b</strong></p>
<p style="margin-top:8px">Three cases based on rank:</p>
<table>
<tr><th>Case</th><th>Condition</th><th>Solutions</th><th>ML Example</th></tr>
<tr><td><span class="hl-green">Exactly determined</span></td><td>A is square, full rank</td><td>Unique solution</td><td>Exact interpolation</td></tr>
<tr><td><span class="hl-red">Overdetermined</span></td><td>More equations than unknowns (m>n)</td><td>No exact solution → least squares</td><td>Linear regression (more data than params)</td></tr>
<tr><td><span class="hl-gold">Underdetermined</span></td><td>Fewer equations than unknowns (m<n)</td><td>Infinite solutions</td><td>Compressed sensing, sparse ML</td></tr>
</table>
<p style="margin-top:12px"><strong>Least Squares Solution</strong> (for overdetermined — most common in ML):</p>
<div style="background:var(--bg3);padding:12px;border-radius:8px">Minimize ||Ax - b||² → Solution: <strong>x = (AᵀA)⁻¹Aᵀb</strong></div>
</div>
<div class="card">
<div class="card-title"><span class="icon">🇧🇩</span> বাংলা ব্যাখ্যা</div>
<p class="bn"><strong>Ax = b</strong> মানে: A ম্যাট্রিক্স দিয়ে x ভেক্টর গুণ করলে b পাওয়া যাবে।</p>
<p class="bn">এটা মূলত বলছে: "x এমন একটা ভেক্টর খুঁজে বের করো যেটাকে A দিয়ে transform করলে b পাওয়া যায়।"</p>
<br>
<p class="bn"><strong>৩টা সম্ভাব্য অবস্থা:</strong></p>
<p class="bn">✅ সমীকরণ সংখ্যা = অজানা সংখ্যা → একটাই সমাধান</p>
<p class="bn">🤔 সমীকরণ > অজানা → সরাসরি সমাধান নেই, least squares দিয়ে সেরা সমাধান খুঁজি</p>
<p class="bn">♾️ সমীকরণ < অজানা → অসংখ্য সমাধান আছে</p>
<br>
<p class="bn">📊 <strong>Linear Regression একটা overdetermined system!</strong> n টা ডেটা পয়েন্ট থেকে মাত্র k টা parameter শিখতে হয়। সেরা fit = least squares।</p>
</div>
<div class="card">
<div class="card-title"><span class="icon">📐</span> FORMULAS</div>
<div class="formula-label">Linear System (Ax = b)</div>
<div class="formula-box">System: 2x + y = 5 Matrix form: ⎡2 1⎤⎡x⎤ ⎡5⎤
x - y = 1 ⎣1 -1⎦⎣y⎦ = ⎣1⎦
A x b
Solution: x = A⁻¹b = [[1/3, 1/3],[1/3,-2/3]] × [5,1]ᵀ = [2, 1]ᵀ</div>
<div class="formula-label">Normal Equation (Linear Regression)</div>
<div class="formula-box">Minimize: ||Xθ - y||² (sum of squared errors)
Setting gradient to zero → <span style="color:var(--green)">θ = (XᵀX)⁻¹Xᵀy</span>
In code (preferred over explicit inverse):
θ, _, _, _ = np.linalg.lstsq(X, y, rcond=None)</div>
<div class="formula-label">Gaussian Elimination (to solve Ax=b)</div>
<div class="formula-box">Augmented matrix [A|b] → Row operations → [I|x]
Steps: (1) Forward elimination to upper triangular
(2) Back substitution to get solution</div>
</div>
<div class="ml-app-box">
<h4>🤖 ML Application</h4>
<table>
<tr><th>ML Algorithm</th><th>Linear System Solved</th></tr>
<tr><td>Linear Regression</td><td><code>θ = (XᵀX)⁻¹Xᵀy</code> (overdetermined)</td></tr>
<tr><td>Ridge Regression</td><td><code>θ = (XᵀX + λI)⁻¹Xᵀy</code></td></tr>
<tr><td>Gaussian Process</td><td>Solve <code>(K + σ²I)α = y</code></td></tr>
<tr><td>Newton's method</td><td>Solve Hessian system: <code>H Δθ = ∇L</code></td></tr>
</table>
</div>
<div class="card">
<div class="card-title"><span class="icon">💼</span> INTERVIEW Q&A</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q1: Compare Normal Equation vs Gradient Descent for linear regression. <span class="q-arrow">▶</span></div>
<div class="a">Normal Equation: O(n³) for n features, no hyperparameter (learning rate), gives exact solution in one step, fails with singular/near-singular XᵀX. Gradient Descent: O(kn) per iteration for k iterations, needs learning rate tuning, iterative, works with regularization easily, scales to millions of features. Rule of thumb: Normal equation for n < 10,000 features; gradient descent otherwise.<br><br><span class="bn">বাংলায়: Normal equation একধাপে সমাধান দেয় কিন্তু বড় dataset-এ ধীর। Gradient descent ধীরে ধীরে শেখে কিন্তু scalable।</span></div>
</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q2: What does adding λI to XᵀX in Ridge Regression do mathematically? <span class="q-arrow">▶</span></div>
<div class="a">Adding λI (λ>0) to XᵀX ensures the matrix is always invertible (positive definite), even when XᵀX is rank-deficient. It "regularizes" the problem. Geometrically, it shrinks coefficients toward zero. It prevents overfitting by penalizing large weights. The condition number improves, making the solution numerically stable. This is called Tikhonov regularization.<br><br><span class="bn">বাংলায়: λI যোগ করলে ম্যাট্রিক্স সবসময় invertible হয় এবং বড় coefficient থেকে রক্ষা পাওয়া যায়।</span></div>
</div>
</div>
<div class="card"><div class="card-title"><span class="icon">🏋️</span> EXERCISES</div>
<div class="exercise-box">
<h4>Exercise</h4>
<p>Solve: 3x + 2y = 12 and x + y = 5 using matrix method. Then check: does the normal equation solution minimize sum of squared errors?</p>
<p style="color:var(--muted);font-size:0.88em;margin-top:6px">A=[[3,2],[1,1]], b=[12,5]. det(A)=1. A⁻¹=[[1,-2],[-1,3]]. x=[2,3]. Check: 3(2)+2(3)=12✓, 2+3=5✓</p>
</div>
</div>
<div class="card"><div class="card-title"><span class="icon">🔗</span> RESOURCES</div>
<a class="resource-link" href="https://www.khanacademy.org/math/linear-algebra/vectors-and-spaces/null-column-space" target="_blank">📘 Khan: Systems of Equations</a>
<a class="resource-link" href="https://cs229.stanford.edu/notes2022fall/main_notes.pdf" target="_blank">📄 Stanford CS229 Notes</a>
</div>`
},
{
title:"Linear Transformations",
bn_title:"রৈখিক রূপান্তর",
tags:["Transformations","Neural Networks","Geometry"],
content:`
<div class="card law-card">
<div class="card-title"><span class="icon">⚡</span> LAW 1 — PREDICTION FIRST</div>
<p>A neural network layer with a ReLU activation — is it a linear transformation? Predict before reading!</p>
<p style="margin-top:8px;color:var(--teal)">✅ NO! The matrix multiply part (Wx+b) IS linear (technically "affine" due to bias). But ReLU(x) = max(0,x) breaks linearity because ReLU(x+y) ≠ ReLU(x)+ReLU(y). Non-linearity is what makes deep networks powerful!</p>
</div>
<div class="card">
<div class="card-title"><span class="icon">📖</span> CORE CONCEPT — English</div>
<p>A function T is a <span class="hl-blue">linear transformation</span> if it satisfies two rules:</p>
<div class="grid-2">
<div style="background:var(--bg3);padding:14px;border-radius:8px">
<p><span class="hl-green">Rule 1: Additivity</span></p>
<p>T(u + v) = T(u) + T(v)</p>
<p style="color:var(--muted);font-size:0.88em">Transform then add = Add then transform</p>
</div>
<div style="background:var(--bg3);padding:14px;border-radius:8px">
<p><span class="hl-blue">Rule 2: Homogeneity</span></p>
<p>T(cu) = cT(u)</p>
<p style="color:var(--muted);font-size:0.88em">Scale then transform = Transform then scale</p>
</div>
</div>
<p style="margin-top:14px"><strong>Every linear transformation can be represented by a matrix!</strong></p>
<p style="margin-top:8px">Common linear transformations:</p>
<table>
<tr><th>Transformation</th><th>Matrix</th><th>Effect</th></tr>
<tr><td>Scale by s</td><td><code>[[s,0],[0,s]]</code></td><td>Stretch/shrink uniformly</td></tr>
<tr><td>Rotate by θ</td><td><code>[[cos θ,-sin θ],[sin θ, cos θ]]</code></td><td>Rotate vectors</td></tr>
<tr><td>Reflect x-axis</td><td><code>[[1,0],[0,-1]]</code></td><td>Flip vertically</td></tr>
<tr><td>Project onto x-axis</td><td><code>[[1,0],[0,0]]</code></td><td>Collapse y component</td></tr>
</table>
</div>
<div class="card">
<div class="card-title"><span class="icon">🇧🇩</span> বাংলা ব্যাখ্যা</div>
<p class="bn"><strong>রৈখিক রূপান্তর</strong> হলো এমন একটা ফাংশন যা স্থানকে "ভাঁজ ছাড়া" রূপান্তর করে।</p>
<br>
<p class="bn">💡 <strong>মনে রাখার উপায়:</strong> রাবারের একটা শিট কল্পনা করো। তুমি এটাকে টেনে লম্বা করতে পারো (scale), ঘোরাতে পারো (rotate), এবং ভাঁজ করতে পারো (shear) — এগুলো সব linear transformation। কিন্তু যদি শিটটাকে মুচড়ে দাও বা ভেঁজে ফেলো — সেটা non-linear!</p>
<br>
<p class="bn"><strong>নিউরাল নেটওয়ার্কে:</strong> প্রতিটা layer একটা linear transformation (matrix multiply) + non-linear activation function। Linear শুধু করলে স্তর যতই বাড়াও, একটা linear transformation-ই থাকে — non-linearity ছাড়া deep network কাজ করে না!</p>
</div>
<div class="card">
<div class="card-title"><span class="icon">📐</span> FORMULAS</div>
<div class="formula-label">Linear Transformation Definition</div>
<div class="formula-box">T: ℝⁿ → ℝᵐ is linear if:
1. T(u+v) = T(u) + T(v) [additivity]
2. T(cv) = cT(v) [scalar homogeneity]
Combined: T(αu + βv) = αT(u) + βT(v) [superposition]</div>
<div class="formula-label">Matrix Representation</div>
<div class="formula-box">Every linear T: ℝⁿ → ℝᵐ can be written as T(x) = Ax
where A is an m×n matrix.
Example — Rotation by 90°:
T([x,y]) = [-y, x] → A = ⎡0 -1⎤
⎣1 0⎦
Check: T([1,0]) = [0,1] ✓ T([0,1]) = [-1,0] ✓</div>
<div class="formula-label">Composition of Transformations</div>
<div class="formula-box">T₂(T₁(x)) = A₂(A₁x) = (A₂A₁)x
← Matrix multiplication IS composition of transformations!</div>
</div>
<div class="ml-app-box">
<h4>🤖 ML Application</h4>
<table>
<tr><th>ML Concept</th><th>Linear Transformation Role</th></tr>
<tr><td>Neural network layer (no activation)</td><td>y = Wx+b is affine (linear + translation)</td></tr>
<tr><td>Feature transformation/engineering</td><td>Map features to new space</td></tr>
<tr><td>PCA</td><td>Linear transformation to principal components</td></tr>
<tr><td>Word embeddings</td><td>Map one-hot → dense vector (linear transform)</td></tr>
<tr><td>Why deep nets need non-linearity</td><td>Stack of linear transforms = one linear transform</td></tr>
</table>
</div>
<div class="card">
<div class="card-title"><span class="icon">💼</span> INTERVIEW Q&A</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q1: Why does stacking multiple linear layers without activation functions not help? <span class="q-arrow">▶</span></div>
<div class="a">Because the composition of linear transformations is itself linear! W₃(W₂(W₁x)) = (W₃W₂W₁)x = Wx — just one matrix multiply. No matter how many layers you stack, you're still computing a linear function of the input. You could approximate it with a single layer. Non-linear activations (ReLU, tanh, sigmoid) break this — they create non-linear decision boundaries, enabling the network to learn complex patterns.<br><br><span class="bn">বাংলায়: linear × linear = linear। Activation function ছাড়া গভীর নেটওয়ার্ক শুধু একটা layer-ই।</span></div>
</div>
<div class="qa-item">
<div class="q" onclick="toggleQ(this)">Q2: Is y = 2x + 3 a linear transformation? <span class="q-arrow">▶</span></div>
<div class="a">Mathematically NO — it's an AFFINE transformation (linear + constant). T(0) = 3 ≠ 0, which violates the requirement that linear transformations map zero to zero. However, in ML, we often loosely call Wx+b "linear" informally. Technically: "linear" = Wx (no bias), "affine" = Wx+b. The bias term is the "+3" in y=2x+3. This matters for understanding what neural network layers can represent.<br><br><span class="bn">বাংলায়: y=2x+3 একটা affine transformation। Pure linear হলে x=0 → y=0, কিন্তু এখানে x=0 → y=3।</span></div>
</div>
</div>
<div class="card"><div class="card-title"><span class="icon">🏋️</span> EXERCISES</div>
<div class="exercise-box">
<h4>Exercise</h4>
<p>Check which are linear transformations: (a) T(x,y) = (2x, 3y) (b) T(x,y) = (x+1, y) (c) T(x,y) = (x², y)</p>
<p style="color:var(--muted);font-size:0.88em;margin-top:6px">(a) Linear ✓ (b) Not linear — T(0,0)=(1,0)≠(0,0) (c) Not linear — x² breaks homogeneity</p>
</div>
</div>
<div class="card"><div class="card-title"><span class="icon">🔗</span> RESOURCES</div>
<a class="resource-link" href="https://www.youtube.com/watch?v=kYB8IZa5AuE" target="_blank">🎬 3B1B: Linear Transformations</a>
</div>`
},
{
title:"Column Space & Null Space",
bn_title:"কলাম স্পেস ও নাল স্পেস",
tags:["Subspaces","Rank-Nullity","Solutions"],
content:`
<div class="card law-card">
<div class="card-title"><span class="icon">⚡</span> LAW 1 — PREDICTION FIRST</div>
<p>If Ax=b has no solution, what does that tell you about where b lives relative to the column space of A?</p>
<p style="margin-top:8px;color:var(--teal)">✅ b lies <strong>outside the column space</strong> of A. The column space is all possible outputs of Ax. If b isn't in there, there's no x that can produce it. Least squares finds the closest point IN the column space to b.</p>
</div>
<div class="card">
<div class="card-title"><span class="icon">📖</span> CORE CONCEPT — English</div>
<p><strong>Column Space C(A):</strong> The set of all possible outputs <code>Ax</code> for any input x.</p>
<ul style="padding-left:16px;margin:8px 0">
<li>= All linear combinations of columns of A</li>
<li>= "What can this matrix produce?"</li>
<li>dim(C(A)) = rank(A)</li>
</ul>
<p style="margin-top:12px"><strong>Null Space N(A):</strong> The set of all inputs x where <code>Ax = 0</code>.</p>
<ul style="padding-left:16px;margin:8px 0">
<li>= Vectors that get "annihilated" by A</li>
<li>= "What inputs produce zero output?"</li>
<li>dim(N(A)) = nullity(A) = n - rank(A)</li>
</ul>
<p style="margin-top:12px"><span class="hl-blue">Rank-Nullity Theorem:</span> rank(A) + nullity(A) = n (columns)</p>
<p style="color:var(--muted);font-size:0.9em">"The sum of what you can express plus what gets lost = total input dimensions"</p>
</div>
<div class="card">
<div class="card-title"><span class="icon">🇧🇩</span> বাংলা ব্যাখ্যা</div>
<p class="bn"><strong>কলাম স্পেস (Column Space):</strong> A দিয়ে যত ধরনের output সম্ভব, তার পুরো সংগ্রহ।</p>
<p class="bn">💡 উদাহরণ: ধরো A একটা projector। কলাম স্পেস হলো দেয়ালে যা যা দেখাতে পারে — projector-এর পরিসর।</p>
<br>
<p class="bn"><strong>নাল স্পেস (Null Space):</strong> এমন সব input x যা A দিয়ে transform করলে শূন্য হয়ে যায়।</p>
<p class="bn">💡 উদাহরণ: projector-এ যদি একটা lens-এর অক্ষ বরাবর আলো দাও, সেটা দেয়ালে কিছুই দেখায় না — এটাই null space।</p>
<br>
<p class="bn"><strong>Rank-Nullity Theorem:</strong> rank + nullity = columns</p>
<p class="bn">মানে: যত information A সংরক্ষণ করে + যত information হারিয়ে ফেলে = মোট input dimension</p>
</div>
<div class="card">
<div class="card-title"><span class="icon">📐</span> FORMULAS & EXAMPLE</div>
<div class="formula-label">Example — Finding Null Space</div>
<div class="formula-box">A = ⎡1 2⎤ Find null space: Ax = 0
⎣2 4⎦
Row reduce: ⎡1 2 | 0⎤ → ⎡1 2 | 0⎤
⎣2 4 | 0⎦ ⎣0 0 | 0⎦
Free variable: x₂ = t. Then x₁ = -2t.
Null space = span{[-2, 1]ᵀ} (all scalar multiples of [-2,1])
Note: rank(A)=1, nullity(A)=1, n=2. Rank-Nullity: 1+1=2 ✓</div>
<div class="formula-label">Column Space Example</div>
<div class="formula-box">A = ⎡1 2⎤ columns are [1,2]ᵀ and [2,4]ᵀ
⎣2 4⎦
But col2 = 2×col1! So column space = span{[1,2]ᵀ} — a 1D line!
(rank=1 → column space is 1-dimensional)</div>