Tensors

What is a Tensor?

• 0D: scalar, 1D: vector, 2D: matrix
• nD: generalization to any dimensions
• NumPy arrays, torch.Tensor, tf.Tensor

Common Shapes

• Samples: (batch, features)
• Images: (batch, H, W, channels)
• Sequences: (batch, length, features)

Element-wise Operations

• +, -, , /, *, sqrt, exp, log
• Vectorized, no loops
• Same shape required (or broadcasting)

Matrix Multiplication

• A @ B or np.dot(A, B)
• Shape: (m,n) @ (n,p) → (m,p)
• Core of neural networks

Broadcasting

• Align shapes from right
• Compatible: equal or one is 1
• Automatic expansion

Reshaping

• reshape(new_shape)
• -1 infers dimension
• flatten() for 1D

Axis Operations

• sum(axis=0), mean(axis=1)
• Reduce along dimension
• axis=0: columns, axis=1: rows