Autograd Value: Build the Graph

medium · autograd, computational-graph, gradients

Autograd Value: Build the Graph

Implement the scalar Value class that builds a computational graph during the forward pass.

What you are building

Each Value stores:

data: float
grad: float (initially 0.0)
_prev: set[Value] (parents)
_op: str (debug label)
_backward: Callable[[], None] (initially a no-op)

The forward pass should create new Value objects and wire their _prev sets.

Methods to implement

1) `init(self, data, _children=(), _op='')`

Store data as a float
Initialize grad = 0.0
_backward = lambda: None
_prev = set(_children)
_op = _op

2) Arithmetic ops

Implement these to return a new Value:

__add__, __mul__, __pow__, __neg__, __sub__, __truediv__

Rules:

Accept Value or Python numbers (int/float)
Convert numbers to Value
Set _op to a readable string (e.g., "+", "", f"*{n}")
Set _backward to propagate gradients (use +=)

3) Reverse ops

Implement:

__radd__, __rmul__, __rsub__, __rtruediv__

These are called when the left operand is a Python number.

Notes

This problem does not implement backward() yet.
Gradients remain 0.0 after forward-only graphs.
Use += in _backward because a node can be used multiple times.

Example

a = Value(2.0)
b = Value(-3.0)
c = Value(10.0)

# Forward pass builds the graph
f = (a * b + c) ** 2

assert f.data == 16.0
assert a in (a * b)._prev

Hints

Division can be implemented as self * other**-1.
Subtraction can be implemented as self + (-other).

from __future__ import annotations
from typing import Tuple, Set, Callable

class Value:
    """A scalar value with automatic differentiation support."""

def __init__(
        self,
        data: float,
        _children: Tuple["Value", ...] = (),
        _op: str = "",
    ) -> None:
        """TODO: Initialize the Value."""
        raise NotImplementedError()

def __repr__(self) -> str:
        return f"Value(data={self.data}, grad={self.grad})"

def __add__(self, other: "Value | float | int") -> "Value":
        """TODO: Implement addition."""
        raise NotImplementedError()

def __mul__(self, other: "Value | float | int") -> "Value":
        """TODO: Implement multiplication."""
        raise NotImplementedError()

def __pow__(self, n: float | int) -> "Value":
        """TODO: Implement power."""
        raise NotImplementedError()

def __neg__(self) -> "Value":
        """TODO: Implement negation."""
        raise NotImplementedError()

def __sub__(self, other: "Value | float | int") -> "Value":
        """TODO: Implement subtraction."""
        raise NotImplementedError()

def __truediv__(self, other: "Value | float | int") -> "Value":
        """TODO: Implement division."""
        raise NotImplementedError()

def __radd__(self, other: float | int) -> "Value":
        """TODO: Implement reverse addition (other + self)."""
        raise NotImplementedError()

def __rmul__(self, other: float | int) -> "Value":
        """TODO: Implement reverse multiplication (other * self)."""
        raise NotImplementedError()

def __rsub__(self, other: float | int) -> "Value":
        """TODO: Implement reverse subtraction (other - self)."""
        raise NotImplementedError()

def __rtruediv__(self, other: float | int) -> "Value":
        """TODO: Implement reverse division (other / self)."""
        raise NotImplementedError()