In 2.3 every arithmetic example used two arrays of exactly the same shape, or an array and a single scalar. Real data is rarely so tidy. You will want to subtract one number from every row, divide each column by its own standard deviation, or compare a 1000\times 3 table against a single 3-element vector.
Broadcasting is the rule NumPy uses to stretch a smaller array across a larger one so that an element-wise operation makes sense. It is worth learning properly, because almost every confusing NumPy error you will meet this semester is a broadcasting error in disguise.
The scalar case, revisited
You have already used broadcasting without naming it:
The scalar 10 does not have shape (2, 3). NumPy behaves as if it had been copied into every position. Nothing is actually copied in memory, which is why this is fast.
The rule
NumPy lines the two shapes up from the right and compares them dimension by dimension. Two dimensions are compatible when
they are equal, or
one of them is 1.
If every pair is compatible, the arrays broadcast; the dimension of size 1 is stretched to match. If any pair is neither equal nor 1, NumPy raises an error.
Missing dimensions on the left are treated as 1. So (3,) is padded to (1, 3) when compared against a 2D array.
A
(2, 3)
b
(3,)
padded to (1, 3)
result
(2, 3)
the single row is reused for both rows
Compare that with a column vector:
The same numbers, arranged differently, produce a completely different answer. Shape is not bookkeeping; it is meaning.
When it fails
Uncomment the last line and NumPy will tell you:
ValueError: operands could not be broadcast together with shapes (2,3) (2,)
Read that message literally. It is not saying the arrays are the wrong size; it is saying that after right-aligning (2,3) and (2,), the last dimensions are 3 and 2, which are neither equal nor 1. Your d has the right number of entries to be a column, but NumPy pads on the left, so it became a row.
Forcing the alignment you meant
To say “treat this as a column”, give it an explicit second axis. np.newaxis (or equivalently None) inserts a dimension of size 1:
reshape(-1, 1) does the same thing and reads more clearly to some people. Either is fine; be consistent.
Why this matters: centring a data matrix
Here is the operation you will perform constantly from week 5 onward, and again when we reach PCA in 3.2. Let X hold n observations in rows and p features in columns.
To centre each feature, subtract that feature’s mean. The means live along axis=0:
That single line is the whole reason broadcasting is worth your attention. PCA begins with exactly this step, and if you centre along the wrong axis you will get a plausible-looking answer that is wrong.
Now try to centre each observation instead, along axis=1:
(5, 3) - (5,) right-aligns to 3 against 5 and fails. The fix is keepdims, which retains the reduced axis with size 1:
A reduction removes an axis; keepdims=True keeps it as a 1. Whenever you reduce and then want to combine the result with the original array, you almost always want keepdims=True.
A caution
Broadcasting is silent when it succeeds. If two arrays happen to be compatible, NumPy will produce an answer whether or not that answer means anything. A 3\times 3 matrix and a 3-element vector always broadcast, so a bug in which you meant a column and supplied a row will not raise an error on square data. It will simply give you the wrong number.
This is precisely the class of mistake an AI assistant will confidently reproduce and not flag. Check shapes.
Note Exercise 1
Before running anything, write down the shape you expect. Then run it and see whether you were right.
Note Exercise 2
Which of these broadcast, and which raise an error? Decide for each before running it, and for the failures say which pair of dimensions is to blame.
Note Exercise 3
Z below is supposed to hold each column scaled to have standard deviation 1. It runs without error and it is wrong. Find the bug and fix it, then explain in one sentence why NumPy did not complain.
Note Exercise 4
Explain, in one sentence each, why (3, 1) and (1, 3) broadcast to (3, 3), while (3,) and (3, 1) broadcast to (3, 3) as well but (3,) and (2, 3, 4) do not.
Graded exercises
Note Predict the result shape
Two arrays have shapes (4, 1, 6) and (3, 1). Apply the rule by hand—right-align the shapes, pad the shorter one on the left with 1s, and take the larger of each pair—then assign the resulting shape tuple to answer.
Resist the urge to just build the arrays and look.
answer = (4, 3, 6)
Padding (3, 1) on the left gives (1, 3, 1). Comparing column by column against (4, 1, 6) gives \max(4,1)=4, \max(1,3)=3, \max(6,1)=6. You can confirm with np.broadcast_shapes((4, 1, 6), (3, 1)).
Note Scaling the right way round
X holds 6 observations in rows and 4 features in columns. Rescale it so that every feature (column) has standard deviation 1.
Getting this backwards produces numbers rather than an error, so check the result rather than trusting it.
Z = X / X.std(axis=0)
X.std(axis=0) has shape (4,), which pads to (1, 4) and stretches down the 6 rows—one divisor per feature. Had you wanted per-row scaling you would need X.std(axis=1, keepdims=True), because (6,) right-aligns 6 against 4 and fails.
Note A table from two vectors
Build the 10\times 10 multiplication table—entry (i, j) equal to (i+1)(j+1)—using broadcasting and no loop, starting from v = np.arange(1, 11).
v * v gives you 10 numbers, not 100. You need one copy of v to run down the rows and the other across the columns.
answer = v[:, np.newaxis] * v
Equivalently v.reshape(-1, 1) * v. A column (10, 1) against a row (10,) broadcasts to (10, 10). The same “column against row” pattern gives you pairwise differences, pairwise distances, and outer products generally.
Note Which Shapes Broadcast?
For which of the following pairs of shapes does NumPy broadcast successfully?
Line the shapes up from the right, padding the shorter one on the left with 1s. A pair of dimensions is compatible when they are equal or when one of them is 1.
(8, 1, 5) vs (7, 5) → (7, 5) pads to (1, 7, 5). Columns: 5\!=\!5, 1 vs 7 (fine, the 1 stretches), 8 vs 1 (fine). Result (8, 7, 5). Broadcasts.
(3, 4) vs (4,) → pads to (1, 4). Columns: 4\!=\!4, 3 vs 1. Result (3, 4). Broadcasts—this is the everyday “subtract a per-column vector from every row”.
(3, 4) vs (3,) → pads to (1, 3). Last dimensions are 4 and 3: neither equal nor 1. Fails. The three numbers you supplied look like one per row, but NumPy pads on the left, so they arrived as a row. keepdims=True, or [:, np.newaxis], is the fix.
(2, 3, 4) vs (2, 4) → pads to (1, 2, 4). Columns: 4\!=\!4, then 3 vs 2. Fails.
(5, 1) vs (1, 5) → both dimensions of each are stretched, giving (5, 5). Broadcasts. Note that both operands are stretched here, which is how a column times a row produces a full table.
Note The Silent Bug
X is a square4\times 4 data matrix with observations in rows and features in columns. A classmate writes
Z = X - X.mean(axis=1)
intending to centre each observation (each row) at zero. The code runs without any error. Which statement best describes what happened?
A reduction removes the axis it operates on. X.mean(axis=1) therefore has shape (4,)—one mean per row, but stored as a flat vector with no memory of having been a column.
Broadcasting then pads on the left, turning (4,) into (1, 4), and stretches it down the rows. So the four row means get subtracted across each row, position by position, which is meaningless. The result is not NaN; it is perfectly finite numbers that are simply wrong.
The fix is to keep the reduced axis as a length-1 dimension:
Z = X - X.mean(axis=1, keepdims=True) # shape (4, 1), stretches across columns
The final option is exactly backwards, and this is the important lesson: on (100, 4) data the buggy line would have raised a ValueError, because 4 and 100 are neither equal nor 1. It is the squareness of X that let the bug through silently. Square test data is a trap—whenever you reduce along an axis and then combine the result with the original array, reach for keepdims=True.