Adopt new vector and matrix norm functions

[carlosgmartin]

Proposal:

Deprecate the following functions:

• norm
• vector_norm
• matrix_norm

Replace them with the following separate functions:

• p_norm(x, p): Computes the p-norm: $\|x\|_p = (\sum_{i < d} |x_i|^p)^{1/p}$.
• induced_norm(x, p, q): Computes the induced (p,q)-norm: $\|A\|_{p, q} = \sup_{\|x\|_p \leq 1} \|A x\|_q$.
  • $p = q = 2$ yields the spectral norm.
• entrywise_norm(x, p, q): Computes the entrywise (p,q)-norm: $\|A\|_{p, q} = \left( \sum_{j < n} \left( \sum_{i < m} |A_{ij}|^p \right)^{q/p} \right)^{1/q}$.
  • $p = q = 2$ yields the Frobenius norm.
• schatten_norm(x, p): Computes the Schatten p-norm: $\|A\|_p = \left( \sum_{i < \min\{m, n\}} \sigma_i(A)^p \right)^{1/p}$.
  • $p = 1$ yields the nuclear norm.
  • $p = 2$ yields the Frobenius norm.
  • $p = \infty$ yields the spectral norm.

Notice that the same value of p (or q) can yield different results, depending on which norm one is talking about.

Optionally, add the following convenience wrappers:

• nuclear_norm(x): Computes the nuclear norm.
• frobenius_norm(x): Computes the Frobenius norm.
• spectral_norm(x): Computes the spectral norm.

Rationale:

Currently, norm and matrix_norm conflate different norms into a single function. This is apparently a design mistake inherited from MATLAB. Such a design (1) creates unnecessary confusion and (2) prevents access to alternative values of p and q for each kind of norm. It would be nice if the standard rectified this situation.

Also, for vectors, the ord=0 "0-norm", which counts the number of nonzero elements, is a misnomer: It is not actually the limit of the p-norm as $p \to 0$. To count the number of nonzero elements, count_nonzero should be used instead.