2.5 Universal Functions


: 20 minutes

A universal function, or ufunc, is a function that acts element by element on an array and returns a new array of the same shape. np.sqrt, np.exp, and np.maximum are all ufuncs; so, behind the scenes, are the ordinary operators +, *, and >, which are just readable spellings of np.add, np.multiply, and np.greater.

Three properties make ufuncs worth naming as a category rather than treating as a list of functions to memorise:

  1. They loop in compiled code. np.sqrt(A) is not a Python loop wearing a disguise; it is a single call into C. This is where the order-of-magnitude speed-up lives.
  2. They broadcast. Every binary ufunc follows the broadcasting rule from 2.4, so everything you learned there applies unchanged here.
  3. They preserve shape. A unary ufunc always returns the same shape it was given. If applying one changed your shape, you did not apply a ufunc.

That third point is the one to hold on to, because it separates ufuncs from the reductions of 2.6, which deliberately remove an axis. Confusing the two is the most common error in this section, and we return to it below.

Ufuncs come in two flavours: unary and binary. A unary ufunc takes one array and returns an array; a binary ufunc takes two.

TipYou are not expected to memorise these tables

The tables below are for reference, not recall. Labs and exams allow the documentation, and looking up whether the ceiling function is np.ceil or np.ceiling costs you five seconds. What the documentation will not tell you in the moment is whether you wanted an element-wise operation or a reduction. Spend your attention there.

Unary Functions

Some of the most commonly used unary ufuncs are listed in the table below.

Table of common unary functions (McKinney 2017)
Ufunc Description
sqrt Compute the element-wise square root (equivalent to arr ** 0.5)
square Compute the square of each element (equivalent to arr ** 2)
exp Compute the exponent e^x of each element
sign Compute the sign of each element: 1 (positive), 0 (zero), or –1 (negative)
ceil Compute the ceiling of each element (i.e., the smallest integer greater than or equal to that number)
floor Compute the floor of each element (i.e., the largest integer less than or equal to each element)
abs, fabs Compute the absolute value element-wise for integer, floating-point, or complex values
isnan Return Boolean array indicating whether each value is NaN (Not a Number)
isfinite, isinf Return Boolean array indicating whether each element is finite (non-inf, non-NaN) or infinite, respectively
log, log10, log2, log1p Natural logarithm (base e), log base 10, log base 2, and log(1 + x), respectively
rint Round elements to the nearest integer, preserving the dtype
cos, cosh, sin, sinh, tan, tanh Regular and hyperbolic trigonometric functions
arccos, arccosh, arcsin, arcsinh, arctan, arctanh Inverse trigonometric functions

The following example uses numpy.square to perform element-wise squaring of an array arr.

Binary Functions

Some of the most commonly used binary ufuncs are listed in the table below.

Table of common binary functions (McKinney 2017)
Ufunc Description
add Add corresponding elements in arrays
subtract Subtract elements in second array from first array
multiply Multiply array elements
divide, floor_divide Divide or floor divide (truncating the remainder)
power Raise elements in first array to powers indicated in second array
maximum Element-wise maximum; fmax ignores NaN
minimum Element-wise minimum; fmin ignores NaN
mod Element-wise modulus (remainder of division)
McKinney, Wes. 2017. Python for Data Analysis. 2nd ed. O’Reilly Media. https://www.oreilly.com/library/view/python-for-data/9781491957653/.

The following example uses numpy.maximum to select (element-wise) maximum from two arrays.

We can also use numpy.add to (element-wise) add two arrays. Here, the elements of matrices A and B are normally distributed with different means and standard deviations.

Note Sum of Normally Distributed Random Variables

In the code above, which statement is true about the distribution of the elements in np.add(A, B)?

Sum of two normally distributed random variables is known to be normally distributed as well. When the means and variances (not standard deviations) are summed as well. See more.

maximum is not max

This pair trips up nearly everyone, and it is a genuinely conceptual distinction rather than a naming quirk.

  • np.maximum(A, B) is a binary ufunc. It compares two arrays element by element and returns an array of the same shape.
  • np.max(A) is a reduction. It sweeps one array and returns a single number, or—given an axis—collapses that axis.

They do different jobs, and the second argument means something entirely different in each. np.max(A, B) is not “the element-wise maximum”; its second positional parameter is axis, so passing an array there produces a confusing error:

The same distinction runs through the whole library: np.minimum/np.min, np.add/np.sum, np.multiply/np.prod. The plural-looking, -imum names are element-wise; the short names are reductions. When you find yourself unsure, ask what shape you expect back—that settles it immediately.

Ufuncs broadcast

Because binary ufuncs obey the broadcasting rule, they are not restricted to equal shapes:

This is the same mechanism as in 2.4, not a separate feature. If a ufunc raises operands could not be broadcast together, reread the shapes; the ufunc itself is rarely the problem.

Invalid values are not errors

Ufuncs applied to a domain they cannot handle do not raise. They return nan or inf, emit a RuntimeWarning, and carry on:

This is deliberate: raising on element 0 of a million-element array would make vectorization useless. But it means a nan can travel a long way through your pipeline before it surfaces, and once it arrives, nan contaminates everything it touches—nan + 1 is nan, and np.mean of an array containing one nan is nan.

Detect them with np.isnan and np.isfinite, and guard the input rather than the output:

Warningnp.where evaluates both branches

The obvious-looking np.where(x > 0, np.log(x), 0.0) gives the right answer but still warns, because np.where is an ordinary function call: Python evaluates np.log(x) on the whole array—negatives included—before np.where ever gets to choose. It is a selection, not an if.

Hence the two-step form above: sanitise the argument first, then select.

Exercises

Note Element-wise or reduction?

For each pair of corresponding entries in A and B, keep the larger. The result should be a (3, 3) array—the same shape you started with, not a single number.

answer = np.maximum(A, B)

np.max reduces one array to a scalar; np.maximum compares two arrays position by position and preserves the shape. Equivalently, np.where(A > B, A, B).

Note Taming the warning

The line below computes the right numbers but prints a RuntimeWarning, because np.log is evaluated on the negative entries before np.where gets a chance to discard them.

Fix it by sanitising the argument to np.log: substitute 1 wherever x is not positive, so the logarithm is always defined. The positions you substituted are thrown away by the outer np.where anyway.

Fill the blank with the sanitised array that gets handed to np.log: it must agree with x wherever x is positive, and hold 1.0 everywhere else. The final line reports what np.log actually receives, which is the thing being marked.

safe_x = np.where(x > 0, x, 1.0)

so the whole guarded computation reads

np.where(x > 0, np.log(np.where(x > 0, x, 1.0)), 0.0)

The inner np.where replaces every non-positive entry with 1, so np.log only ever sees legal input; since \log 1 = 0 those entries would be harmless even if kept, and the outer np.where discards them regardless.

Note Element-wise versus Reduction

A and B both have shape (6, 4). What are the shapes of np.maximum(A, B) and of A.max(axis=0), respectively?

The two operations belong to different families.

np.maximum is a binary universal function: it walks the two arrays in step and compares corresponding entries, so it returns an array of the same shape, (6, 4). A ufunc never changes the shape of its input.

A.max(axis=0) is a reduction: it collapses the named axis. Axis 0 has length 6, so that axis disappears and what remains is (4,)—one maximum per column.

The mnemonic worth carrying: the axis you name is the axis that vanishes. And when you are unsure which of max/maximum you want, ask what shape you expect back. If it is the shape you started with, you want the ufunc; if an axis should disappear, you want the reduction.

Beware also that np.max(A, B) is not the element-wise maximum. The second positional parameter of np.max is axis, so passing an array there raises a TypeError whose message mentions integer scalar indices and gives no hint that you meant np.maximum.

Note Where np.where Surprises You

Let x = np.array([-2.0, 0.0, 3.0, 5.0]). The line

result = np.where(x > 0, np.log(x), 0.0)

produces the correct numbers [0., 0., 1.0986, 1.6094] but also prints a RuntimeWarning about an invalid value. Why?

np.where(cond, a, b) is a function, and Python evaluates a function’s arguments before calling it. So np.log(x) is computed over the whole of x, including -2 (giving nan) and 0 (giving -inf), and the RuntimeWarning is raised at that moment. Only afterwards does np.where pick entries and throw the offending ones away.

This is the crucial difference between np.where and an if: np.where selects, it does not skip. There is no short-circuiting anywhere in vectorized code.

The fix is to guard the input rather than filter the output:

result = np.where(x > 0, np.log(np.where(x > 0, x, 1.0)), 0.0)

The inner np.where substitutes a legal value at the positions where the logarithm is undefined, so np.log never sees bad input.

Why care, if the answer was right anyway? Because the same reasoning applied to division gives inf rather than nan, and inf does not always get discarded so tidily—and because a nan that survives into a later mean turns the whole result into nan with no warning at all.