Regularization penalties and constraints

Classes:

`ADMMPenalty`([aux_init, dual_init])	Base class for all regularizers and constraints.
`Box`(min_val, max_val[, aux_init, dual_init])	Set minimum and maximum value for the factor.
`GeneralizedL2Penalty`(norm_matrix[, svd, ...])	Penalty on the form \(\mathbf{x}^\mathsf{T} \mathbf{Mx}\), where \(\mathbf{M}\) is any symmetric positive semidefinite matrix.
`HardConstraintMixin`()	Mixin for hard constraints.
`L1Penalty`(reg_strength[, non_negativity, ...])	Add L1 (LASSO) regularization on the factor elements.
`L2Ball`(norm_bound[, non_negativity, ...])	Ensure that the L2-norm of component vectors are less than a given scalar.
`MatricesPenalty`([aux_init, dual_init])	Base class for penalties that are applied to a list of factor matrices simultaneously.
`MatrixPenalty`([aux_init, dual_init])	Base class for penalties that can be applied to a single factor matrix at a time.
`NonNegativity`([aux_init, dual_init])	Impose non-negative values for the factor.
`Parafac2`([svd, n_iter, ...])	Impose the PARAFAC2 constraint on the uncoupled factor matrices.
`RowVectorPenalty`([aux_init, dual_init])	Base class for penalties that can be applied to one row of a factor matrix at a time.
`TotalVariationPenalty`(reg_strength[, ...])	Impose piecewise constant components
`Unimodality`([non_negativity, aux_init, ...])	Constrain the component-vectors so they are unimodal.
`UnitSimplex`([aux_init, dual_init])	Constrain the component-vectors so they are non-negative and sum to 1.

class matcouply.penalties.ADMMPenalty(aux_init='random_uniform', dual_init='random_uniform')[source]

Base class for all regularizers and constraints.

Parameters:

aux_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables
dual_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables

Methods:

`aux_as_matrix`(aux)	Convert an auxiliary variable to a matrix (mode=0 and mode=2).
`auxes_as_matrices`(auxes)	Convert a list of auxiliary variables to a list of matrices (mode=1).
`init_aux`(matrices, rank, mode[, random_state])	Initialize the auxiliary variables
`init_dual`(matrices, rank, mode[, random_state])	Initialize the dual variables
`penalty`(x)	Compute the penalty for the given factor matrix or list of factor matrices.
`subtract_from_aux`(aux, dual)	Compute (aux - dual) for mode=0 and mode=2.
`subtract_from_auxes`(auxes, duals)	Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

aux_as_matrix(aux)[source]

Convert an auxiliary variable to a matrix (mode=0 and mode=2).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: aux (tl.tensor(ndim=2)) –
Return type:: tl.tensor(ndim=2)

auxes_as_matrices(auxes)[source]

Convert a list of auxiliary variables to a list of matrices (mode=1).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: auxes (list of tl.tensor(ndim=2)) –
Return type:: list of tl.tensor(ndim=2)

init_aux(matrices, rank, mode, random_state=None)[source]

Initialize the auxiliary variables

Initialization schemes

"random_uniform": The elements of the auxiliary variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the auxiliary
variables are drawn from a standard normal distribution.

"zeros": The elements of the auxiliary variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed auxiliary variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed auxiliary variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these auxiliary variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these auxiliary variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

init_dual(matrices, rank, mode, random_state=None)[source]

Initialize the dual variables

Initialization schemes

"random_uniform": The elements of the dual variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the dual
variables are drawn from a standard normal distribution.

"zeros": The elements of the dual variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed dual variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed dual variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these dual variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these dual variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

abstract penalty(x)[source]: Compute the penalty for the given factor matrix or list of factor matrices.

subtract_from_aux(aux, dual)[source]

Compute (aux - dual) for mode=0 and mode=2.

For some penalties, the aux is not a factor matrix but rather some other parametrization of a matrix. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

aux (tl.tensor(ndim=2)) – Auxiliary variables
dual (tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

tl.tensor(ndim=2)

subtract_from_auxes(auxes, duals)[source]

Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

For some penalties, the aux is not a list of factor matrices but rather some other parametrization of a list of factor matrices. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

auxes (list of tl.tensor(ndim=2)) – Auxiliary variables
duals (list of tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

list of tl.tensor(ndim=2)

class matcouply.penalties.Box(min_val, max_val, aux_init='random_uniform', dual_init='random_uniform')[source]

Set minimum and maximum value for the factor.

A box constraint works element-wise, constraining the elements of a factor to satisfy \(l \leq x \leq u\), where \(x\) represents the element of the factor and \(l\) and \(u\) are the lower and upper bound on the factor elements, respectively.

Parameters:

min_val (float or None) – Lower bound on the factor elements.
max_val (float or None) – Upper bound on the factor elements
aux_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables
dual_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables

Methods:

`aux_as_matrix`(aux)	Convert an auxiliary variable to a matrix (mode=0 and mode=2).
`auxes_as_matrices`(auxes)	Convert a list of auxiliary variables to a list of matrices (mode=1).
`factor_matrices_update`(factor_matrices, ...)	Update all factor matrices in given list according to this penalty.
`factor_matrix_row_update`(factor_matrix_row, ...)	Update a single row of a factor matrix according to this constraint.
`factor_matrix_update`(factor_matrix, ...)	Update a factor matrix according to this penalty.
`init_aux`(matrices, rank, mode[, random_state])	Initialize the auxiliary variables
`init_dual`(matrices, rank, mode[, random_state])	Initialize the dual variables
`penalty`(x)	Returns 0 as there is no penalty for hard constraints.
`subtract_from_aux`(aux, dual)	Compute (aux - dual) for mode=0 and mode=2.
`subtract_from_auxes`(auxes, duals)	Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

aux_as_matrix(aux)

Convert an auxiliary variable to a matrix (mode=0 and mode=2).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: aux (tl.tensor(ndim=2)) –
Return type:: tl.tensor(ndim=2)

auxes_as_matrices(auxes)

Convert a list of auxiliary variables to a list of matrices (mode=1).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: auxes (list of tl.tensor(ndim=2)) –
Return type:: list of tl.tensor(ndim=2)

factor_matrices_update(factor_matrices, feasibility_penalties, auxes)

Update all factor matrices in given list according to this penalty.

Parameters:

factor_matrices (list of tl.tensor(ndim=2)) – List of factor matrix to update.
feasibility_penalties (list of floats) – Penalty parameters for the feasibility gap of the different factor matrices.
auxes (list of tl.tensor(ndim=2)) – List of auxiliary matrices, each element corresponds to the auxiliary factor matrix for the same element in factor_matrices.

factor_matrix_row_update(factor_matrix_row, feasibility_penalty, aux_row)[source]

Update a single row of a factor matrix according to this constraint.

Parameters:

factor_matrix_row (tl.tensor(ndim=1)) – Vector (first order tensor) that corresponds to the single row in the factor matrix we wish to update.
feasibility_penalty (float) – Penalty parameter for infeasible solutions.
aux_row (tl.tensor(ndim=1)) – Vector (first order tensor) that corresponds to the row in the auxiliary matrix that correspond to the row supplied to factor_matrix_row.

factor_matrix_update(factor_matrix, feasibility_penalty, aux)[source]

Update a factor matrix according to this penalty.

Parameters:

factor_matrix (tl.tensor(ndim=2)) – Factor matrix to update.
feasibility_penalty (float) – Penalty parameter for infeasible solutions.
aux (tl.tensor(ndim=2)) – Auxiliary matrix that correspond to the factor matrix supplied to factor_matrix.

init_aux(matrices, rank, mode, random_state=None)

Initialize the auxiliary variables

Initialization schemes

"random_uniform": The elements of the auxiliary variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the auxiliary
variables are drawn from a standard normal distribution.

"zeros": The elements of the auxiliary variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed auxiliary variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed auxiliary variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these auxiliary variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these auxiliary variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

init_dual(matrices, rank, mode, random_state=None)

Initialize the dual variables

Initialization schemes

"random_uniform": The elements of the dual variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the dual
variables are drawn from a standard normal distribution.

"zeros": The elements of the dual variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed dual variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed dual variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these dual variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these dual variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

penalty(x)

Returns 0 as there is no penalty for hard constraints.

Hard constraints are always penalised with 0 even when the components are infeasible. Slightly infeasible components would otherwise result in an infinite penalty because the penalty function of hard constraints is 0 for feasible solutions and infinity for infeasible solutions. An infinite penalty would stop all convergence checking and not provide any information on the quality of the components. To ensure that the hard constraints are sufficiently imposed, it is recommended to examine the feasibility gap instead of the penalty and ensure that the feasibility gap is low.

Parameters:: x (tl.tensor(ndim=2) or list of tl.tensor(ndim=2)) – Factor matrix or list of factor matrices.

subtract_from_aux(aux, dual)

Compute (aux - dual) for mode=0 and mode=2.

For some penalties, the aux is not a factor matrix but rather some other parametrization of a matrix. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

aux (tl.tensor(ndim=2)) – Auxiliary variables
dual (tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

tl.tensor(ndim=2)

subtract_from_auxes(auxes, duals)

Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

For some penalties, the aux is not a list of factor matrices but rather some other parametrization of a list of factor matrices. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

auxes (list of tl.tensor(ndim=2)) – Auxiliary variables
duals (list of tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

list of tl.tensor(ndim=2)

class matcouply.penalties.GeneralizedL2Penalty(norm_matrix, svd='truncated_svd', aux_init='random_uniform', dual_init='random_uniform', validate=True)[source]

Penalty on the form \(\mathbf{x}^\mathsf{T} \mathbf{Mx}\), where \(\mathbf{M}\) is any symmetric positive semidefinite matrix.

The generalized L2 penalty adds a squared (semi-)norm penalty on the form

\[g(\mathbf{x}) = \mathbf{x}^\mathsf{T} \mathbf{Mx}\]

where \(\mathbf{M}\) is a symmetric positive semidefinite matrix and \(\mathbf{x}\) is a vector. This penalty is imposed for all columns of the factor matrix (or matrices for the \(\mathbf{B}^{(i)}\)-s). Note that the regular L2-penalty (or Ridge penalty) is a special case of the generalised L2 penalty, which we obtain by setting \(\mathbf{M}=\mathbf{I}\). Also, this penalty is a squared seminorm penalty since

\[g(\mathbf{x}) = \mathbf{x}^\mathsf{T} \mathbf{Mx} = \| \mathbf{Lx} \|_2^2,\]

where \(\mathbf{L}\) is a Cholesky factorization of \(\mathbf{M}\). However, the formulation with \(\mathbf{M}\) is more practical than the formulation with \(\mathbf{L}\), since

\[\mathbf{M} = \mathbf{L}^\mathsf{T}\mathbf{L}\]

is easy to compute with \(\mathbf{L}\) known, but not wise-versa (e.g. if \(\mathbf{M}\) is indefinite).

Graph Laplacian penalty

A special case of the generalized L2 penalty is the graph Laplacian penalty. This penalty is on the form

\[g(\mathbf{x}) = \sum_{m=1}^M \sum_{n=1}^N w_{mn} (x_m - x_n)^2.\]

That is, squared differences between the different component vector elements are penalised. Graph laplacian penalties are useful, for example, in image processing, where \(w_{mn}\) is a high number for vector elements that represent pixels that are close to each other and a low number for vector elements that represent pixels that are far apart.

To transform the graph Laplacian penalty into a generalised L2 penalty, we consider the component vector elements as nodes in a graph and \(w_{mn}\) as the edge weight between node \(m\) and node \(m\). Then, we set \(\mathbf{M}\) equal to the graph Laplacian of this graph. That is

\[\begin{split}m_{mn} = \begin{cases} -w_{mn} & m \neq n \\ \sum_m & m = n \end{cases}.\end{split}\]

The proximal operator:

The proximal operator of the generalised L2 penalty is obtained by solving

\[\text{prox}_{\mathbf{x}^\mathsf{T} \mathbf{Mx}}(\mathbf{x}) = \left(\mathbf{M} + 0.5\rho\mathbf{I}\right)^{-1}0.5\rho\mathbf{x},\]

where \(\rho\) is the scale parameter. There are several ways to solve this equation. One of which is via the SVD. Let \(\mathbf{M} = \mathbf{USU}^\mathsf{T}\), then, the proximal operator is given by

\[\text{prox}_{\mathbf{x}^\mathsf{T} \mathbf{Mx}}(\mathbf{x}) = (\mathbf{U}(\mathbf{S} + 0.5\rho\mathbf{I})^{-1}\mathbf{U}^\mathsf{T}) 0.5 \rho \mathbf{x}.\]

This operation is fast once \(\mathbf{U}\) and \(\mathbf{S}\) are known, since solving the diagonal system \((\mathbf{S} + 0.5\rho\mathbf{I})\) is fast.

Parameters:

norm_matrix (tl.tensor(ndim=2)) – The \(\mathbf{M}\)-matrix above
svd (str) – String that specifies which SVD algorithm to use. Valid strings are the keys of tensorly.SVD_FUNS.
aux_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables
dual_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables
validate (bool (default=True)) – Enable naive validation of the norm matrix.

Examples

This example creates a generalised L2 penalty for a simple Laplacian penalty on the form \(\sum_{n} (x_n - x_{n-1})^2\).

>>> import numpy as np
>>> import tensorly as tl
>>> from matcouply.penalties import GeneralizedL2Penalty
>>> num_elements = 30
>>> M = 2 * np.eye(num_elements) - np.eye(num_elements, k=1) - np.eye(num_elements, k=-1)
>>> M[0, 0] = 1
>>> M[-1, -1] = 1
>>> M = tl.tensor(M)
>>> penalty = GeneralizedL2Penalty(M)

This penalty can now be added to matcouply.decomposition.cmf_aoadmm via the regs-parameter. Alternatively, if the generalized_l2_penalty-argument of matcouply.decomposition.cmf_aoadmm is used, then a GeneralizedL2Penalty is added with method="svd".

Methods:

`aux_as_matrix`(aux)	Convert an auxiliary variable to a matrix (mode=0 and mode=2).
`auxes_as_matrices`(auxes)	Convert a list of auxiliary variables to a list of matrices (mode=1).
`factor_matrices_update`(factor_matrices, ...)	Update all factor matrices in given list according to this penalty.
`factor_matrix_update`(factor_matrix, ...)	Update a factor matrix according to this penalty.
`init_aux`(matrices, rank, mode[, random_state])	Initialize the auxiliary variables
`init_dual`(matrices, rank, mode[, random_state])	Initialize the dual variables
`penalty`(x)	Compute the penalty for the given factor matrix or list of factor matrices.
`subtract_from_aux`(aux, dual)	Compute (aux - dual) for mode=0 and mode=2.
`subtract_from_auxes`(auxes, duals)	Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

aux_as_matrix(aux)

Convert an auxiliary variable to a matrix (mode=0 and mode=2).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: aux (tl.tensor(ndim=2)) –
Return type:: tl.tensor(ndim=2)

auxes_as_matrices(auxes)

Convert a list of auxiliary variables to a list of matrices (mode=1).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: auxes (list of tl.tensor(ndim=2)) –
Return type:: list of tl.tensor(ndim=2)

factor_matrices_update(factor_matrices, feasibility_penalties, auxes)

Update all factor matrices in given list according to this penalty.

Parameters:

factor_matrices (list of tl.tensor(ndim=2)) – List of factor matrix to update.
feasibility_penalties (list of floats) – Penalty parameters for the feasibility gap of the different factor matrices.
auxes (list of tl.tensor(ndim=2)) – List of auxiliary matrices, each element corresponds to the auxiliary factor matrix for the same element in factor_matrices.

factor_matrix_update(factor_matrix, feasibility_penalty, aux)[source]

Update a factor matrix according to this penalty.

Parameters:

factor_matrix (tl.tensor(ndim=2)) – Factor matrix to update.
feasibility_penalty (float) – Penalty parameter for infeasible solutions.
aux (tl.tensor(ndim=2)) – Auxiliary matrix that correspond to the factor matrix supplied to factor_matrix.

init_aux(matrices, rank, mode, random_state=None)

Initialize the auxiliary variables

Initialization schemes

"random_uniform": The elements of the auxiliary variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the auxiliary
variables are drawn from a standard normal distribution.

"zeros": The elements of the auxiliary variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed auxiliary variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed auxiliary variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these auxiliary variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these auxiliary variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

init_dual(matrices, rank, mode, random_state=None)

Initialize the dual variables

Initialization schemes

"random_uniform": The elements of the dual variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the dual
variables are drawn from a standard normal distribution.

"zeros": The elements of the dual variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed dual variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed dual variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these dual variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these dual variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

penalty(x)[source]: Compute the penalty for the given factor matrix or list of factor matrices.

subtract_from_aux(aux, dual)

Compute (aux - dual) for mode=0 and mode=2.

For some penalties, the aux is not a factor matrix but rather some other parametrization of a matrix. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

aux (tl.tensor(ndim=2)) – Auxiliary variables
dual (tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

tl.tensor(ndim=2)

subtract_from_auxes(auxes, duals)

Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

For some penalties, the aux is not a list of factor matrices but rather some other parametrization of a list of factor matrices. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

auxes (list of tl.tensor(ndim=2)) – Auxiliary variables
duals (list of tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

list of tl.tensor(ndim=2)

class matcouply.penalties.HardConstraintMixin[source]

Mixin for hard constraints.

Methods:

penalty(x)

Returns 0 as there is no penalty for hard constraints.

penalty(x)[source]

Returns 0 as there is no penalty for hard constraints.

Hard constraints are always penalised with 0 even when the components are infeasible. Slightly infeasible components would otherwise result in an infinite penalty because the penalty function of hard constraints is 0 for feasible solutions and infinity for infeasible solutions. An infinite penalty would stop all convergence checking and not provide any information on the quality of the components. To ensure that the hard constraints are sufficiently imposed, it is recommended to examine the feasibility gap instead of the penalty and ensure that the feasibility gap is low.

Parameters:: x (tl.tensor(ndim=2) or list of tl.tensor(ndim=2)) – Factor matrix or list of factor matrices.

class matcouply.penalties.L1Penalty(reg_strength, non_negativity=False, aux_init='random_uniform', dual_init='random_uniform')[source]

Add L1 (LASSO) regularization on the factor elements.

The L1 penalty is frequently used to obtain sparse components. That is, components with many zero-valued elements. To accomplish this, the L1 penalty adds a penalty term on the form \(\sum_{i r} \gamma |a_{ir}|\), where \(a_{ir}\) is the \((i,r)\)-th element of the factor matrix.

Parameters:

reg_strength (float) – The regularization strength, \(\gamma\) in the equation above
non_negativity (bool) – If True, then non-negativity is also imposed on the factor elements.
aux_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables
dual_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables

Methods:

`aux_as_matrix`(aux)	Convert an auxiliary variable to a matrix (mode=0 and mode=2).
`auxes_as_matrices`(auxes)	Convert a list of auxiliary variables to a list of matrices (mode=1).
`factor_matrices_update`(factor_matrices, ...)	Update all factor matrices in given list according to this penalty.
`factor_matrix_row_update`(factor_matrix_row, ...)	Update a single row of a factor matrix according to this constraint.
`factor_matrix_update`(factor_matrix, ...)	Update a factor matrix according to this penalty.
`init_aux`(matrices, rank, mode[, random_state])	Initialize the auxiliary variables
`init_dual`(matrices, rank, mode[, random_state])	Initialize the dual variables
`penalty`(x)	Compute the penalty for the given factor matrix or list of factor matrices.
`subtract_from_aux`(aux, dual)	Compute (aux - dual) for mode=0 and mode=2.
`subtract_from_auxes`(auxes, duals)	Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

aux_as_matrix(aux)

Convert an auxiliary variable to a matrix (mode=0 and mode=2).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: aux (tl.tensor(ndim=2)) –
Return type:: tl.tensor(ndim=2)

auxes_as_matrices(auxes)

Convert a list of auxiliary variables to a list of matrices (mode=1).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: auxes (list of tl.tensor(ndim=2)) –
Return type:: list of tl.tensor(ndim=2)

factor_matrices_update(factor_matrices, feasibility_penalties, auxes)

Update all factor matrices in given list according to this penalty.

Parameters:

factor_matrices (list of tl.tensor(ndim=2)) – List of factor matrix to update.
feasibility_penalties (list of floats) – Penalty parameters for the feasibility gap of the different factor matrices.
auxes (list of tl.tensor(ndim=2)) – List of auxiliary matrices, each element corresponds to the auxiliary factor matrix for the same element in factor_matrices.

factor_matrix_row_update(factor_matrix_row, feasibility_penalty, aux_row)[source]

Update a single row of a factor matrix according to this constraint.

Parameters:

factor_matrix_row (tl.tensor(ndim=1)) – Vector (first order tensor) that corresponds to the single row in the factor matrix we wish to update.
feasibility_penalty (float) – Penalty parameter for infeasible solutions.
aux_row (tl.tensor(ndim=1)) – Vector (first order tensor) that corresponds to the row in the auxiliary matrix that correspond to the row supplied to factor_matrix_row.

factor_matrix_update(factor_matrix, feasibility_penalty, aux)[source]

Update a factor matrix according to this penalty.

Parameters:

factor_matrix (tl.tensor(ndim=2)) – Factor matrix to update.
feasibility_penalty (float) – Penalty parameter for infeasible solutions.
aux (tl.tensor(ndim=2)) – Auxiliary matrix that correspond to the factor matrix supplied to factor_matrix.

init_aux(matrices, rank, mode, random_state=None)

Initialize the auxiliary variables

Initialization schemes

"random_uniform": The elements of the auxiliary variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the auxiliary
variables are drawn from a standard normal distribution.

"zeros": The elements of the auxiliary variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed auxiliary variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed auxiliary variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these auxiliary variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these auxiliary variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

init_dual(matrices, rank, mode, random_state=None)

Initialize the dual variables

Initialization schemes

"random_uniform": The elements of the dual variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the dual
variables are drawn from a standard normal distribution.

"zeros": The elements of the dual variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed dual variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed dual variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these dual variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these dual variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

penalty(x)[source]: Compute the penalty for the given factor matrix or list of factor matrices.

subtract_from_aux(aux, dual)

Compute (aux - dual) for mode=0 and mode=2.

For some penalties, the aux is not a factor matrix but rather some other parametrization of a matrix. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

aux (tl.tensor(ndim=2)) – Auxiliary variables
dual (tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

tl.tensor(ndim=2)

subtract_from_auxes(auxes, duals)

Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

For some penalties, the aux is not a list of factor matrices but rather some other parametrization of a list of factor matrices. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

auxes (list of tl.tensor(ndim=2)) – Auxiliary variables
duals (list of tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

list of tl.tensor(ndim=2)

class matcouply.penalties.L2Ball(norm_bound, non_negativity=False, aux_init='random_uniform', dual_init='random_uniform')[source]

Ensure that the L2-norm of component vectors are less than a given scalar.

This is a hard constraint on the L2-norm of the component vectors given by

\[\|\mathbf{x}\|_2 \leq r,\]

where \(\mathbf{x}\) is a column vector for a factor matrix and \(r\) is a positive constant.

The L2-ball constraint is compatible with the non-negativity constraint.

Parameters:

norm_bound (float (> 0)) – Maximum L2-norm of the component vectors (\(r\) above)
non_negativity (float) – If true, then non-negativity is imposed too
aux_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables
dual_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables

Note

Proof of compatibility with non-negativity constraints

The compatibility with non-negativity constraints can be obtained with the standard projection onto convex sets (POCS) algorithm, which states that the projection onto a convex set that is the intersection of two other convex sets \(\mathcal{C} = \mathcal{C}_1 \cap \mathcal{C}_2\), \(\mathcal{P}_\mathcal{C}\) is given by

\[\mathcal{P}_\mathcal{C} = \prod_{n=1}^\infty \mathcal{P}_{\mathcal{C}_1} \mathcal{P}_{\mathcal{C}_2},\]

where \(\mathcal{P}_{\mathcal{C}_1}\) and \(\mathcal{P}_{\mathcal{C}_2}\) are the projections onto \(\mathcal{C}_1\) and \(\mathcal{C}_2\), respectively. In other words, to project only \(\mathcal{C}\), we can alternatingly project onto \(\mathcal{C}_1\) and \(\mathcal{C}_2\).

We can now use this relation to prove that the projection onto the intersection of the L2 ball and the non-negative orthant can be obtained by first projecting onto the L2 ball followed by a projection onto the non-negative orthant. Consider any point \(\mathbf{x} \in \mathbb{R}^N\). The projection onto the L2 ball of radius \(r\) is given by:

\[\mathbf{x}^{(0.5)} = \frac{\min(\|\mathbf{x}\|_2, r) \mathbf{x}}{\|\mathbf{x}\|_2}.\]

If any entries in \(\mathbf{x}\) are negative, then their sign will not change. Next, we project \(\mathbf{x}^{(0.5)}\) onto the non-negative orthant:

\[x_n^{(1)} = \max(x_n^{(0.5)}, 0).\]

This operation has the property that \(\|\mathbf{x}^{(1)}\|_2 \leq \|\mathbf{x}^{(0.5)}\|_2 \leq\). Thus, any subsequent projection either onto the L2-ball of radius \(r\) or the non-negative orthant will not change \(\mathbf{x}^{(1)}\), which means that \(\mathbf{x}^{(1)}\) is the projection of \(\mathbf{x}\) onto the intersection of the L2 ball of radius \(r\) and the non-negative orthant.

Methods:

`aux_as_matrix`(aux)	Convert an auxiliary variable to a matrix (mode=0 and mode=2).
`auxes_as_matrices`(auxes)	Convert a list of auxiliary variables to a list of matrices (mode=1).
`factor_matrices_update`(factor_matrices, ...)	Update all factor matrices in given list according to this penalty.
`factor_matrix_update`(factor_matrix, ...)	Update a factor matrix according to this penalty.
`init_aux`(matrices, rank, mode[, random_state])	Initialize the auxiliary variables
`init_dual`(matrices, rank, mode[, random_state])	Initialize the dual variables
`penalty`(x)	Returns 0 as there is no penalty for hard constraints.
`subtract_from_aux`(aux, dual)	Compute (aux - dual) for mode=0 and mode=2.
`subtract_from_auxes`(auxes, duals)	Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

aux_as_matrix(aux)

Convert an auxiliary variable to a matrix (mode=0 and mode=2).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: aux (tl.tensor(ndim=2)) –
Return type:: tl.tensor(ndim=2)

auxes_as_matrices(auxes)

Convert a list of auxiliary variables to a list of matrices (mode=1).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: auxes (list of tl.tensor(ndim=2)) –
Return type:: list of tl.tensor(ndim=2)

factor_matrices_update(factor_matrices, feasibility_penalties, auxes)

Update all factor matrices in given list according to this penalty.

Parameters:

factor_matrices (list of tl.tensor(ndim=2)) – List of factor matrix to update.
feasibility_penalties (list of floats) – Penalty parameters for the feasibility gap of the different factor matrices.
auxes (list of tl.tensor(ndim=2)) – List of auxiliary matrices, each element corresponds to the auxiliary factor matrix for the same element in factor_matrices.

factor_matrix_update(factor_matrix, feasibility_penalty, aux)[source]

Update a factor matrix according to this penalty.

Parameters:

factor_matrix (tl.tensor(ndim=2)) – Factor matrix to update.
feasibility_penalty (float) – Penalty parameter for infeasible solutions.
aux (tl.tensor(ndim=2)) – Auxiliary matrix that correspond to the factor matrix supplied to factor_matrix.

init_aux(matrices, rank, mode, random_state=None)

Initialize the auxiliary variables

Initialization schemes

"random_uniform": The elements of the auxiliary variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the auxiliary
variables are drawn from a standard normal distribution.

"zeros": The elements of the auxiliary variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed auxiliary variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed auxiliary variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these auxiliary variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these auxiliary variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

init_dual(matrices, rank, mode, random_state=None)

Initialize the dual variables

Initialization schemes

"random_uniform": The elements of the dual variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the dual
variables are drawn from a standard normal distribution.

"zeros": The elements of the dual variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed dual variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed dual variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these dual variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these dual variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

penalty(x)

Returns 0 as there is no penalty for hard constraints.

Hard constraints are always penalised with 0 even when the components are infeasible. Slightly infeasible components would otherwise result in an infinite penalty because the penalty function of hard constraints is 0 for feasible solutions and infinity for infeasible solutions. An infinite penalty would stop all convergence checking and not provide any information on the quality of the components. To ensure that the hard constraints are sufficiently imposed, it is recommended to examine the feasibility gap instead of the penalty and ensure that the feasibility gap is low.

Parameters:: x (tl.tensor(ndim=2) or list of tl.tensor(ndim=2)) – Factor matrix or list of factor matrices.

subtract_from_aux(aux, dual)

Compute (aux - dual) for mode=0 and mode=2.

For some penalties, the aux is not a factor matrix but rather some other parametrization of a matrix. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

aux (tl.tensor(ndim=2)) – Auxiliary variables
dual (tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

tl.tensor(ndim=2)

subtract_from_auxes(auxes, duals)

Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

For some penalties, the aux is not a list of factor matrices but rather some other parametrization of a list of factor matrices. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

auxes (list of tl.tensor(ndim=2)) – Auxiliary variables
duals (list of tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

list of tl.tensor(ndim=2)

class matcouply.penalties.MatricesPenalty(aux_init='random_uniform', dual_init='random_uniform')[source]

Base class for penalties that are applied to a list of factor matrices simultaneously.

Methods:

`aux_as_matrix`(aux)	Convert an auxiliary variable to a matrix (mode=0 and mode=2).
`auxes_as_matrices`(auxes)	Convert a list of auxiliary variables to a list of matrices (mode=1).
`factor_matrices_update`(factor_matrices, ...)	Update all factor matrices in given list according to this penalty.
`init_aux`(matrices, rank, mode[, random_state])	Initialize the auxiliary variables
`init_dual`(matrices, rank, mode[, random_state])	Initialize the dual variables
`penalty`(x)	Compute the penalty for the given factor matrix or list of factor matrices.
`subtract_from_aux`(aux, dual)	Compute (aux - dual) for mode=0 and mode=2.
`subtract_from_auxes`(auxes, duals)	Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

aux_as_matrix(aux)

Convert an auxiliary variable to a matrix (mode=0 and mode=2).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: aux (tl.tensor(ndim=2)) –
Return type:: tl.tensor(ndim=2)

auxes_as_matrices(auxes)

Convert a list of auxiliary variables to a list of matrices (mode=1).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: auxes (list of tl.tensor(ndim=2)) –
Return type:: list of tl.tensor(ndim=2)

abstract factor_matrices_update(factor_matrices, feasibility_penalties, auxes)[source]

Update all factor matrices in given list according to this penalty.

Parameters:

factor_matrices (list of tl.tensor(ndim=2)) – List of factor matrix to update.
feasibility_penalties (list of floats) – Penalty parameters for the feasibility gap of the different factor matrices.
auxes (list of tl.tensor(ndim=2)) – List of auxiliary matrices, each element corresponds to the auxiliary factor matrix for the same element in factor_matrices.

init_aux(matrices, rank, mode, random_state=None)

Initialize the auxiliary variables

Initialization schemes

"random_uniform": The elements of the auxiliary variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the auxiliary
variables are drawn from a standard normal distribution.

"zeros": The elements of the auxiliary variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed auxiliary variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed auxiliary variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these auxiliary variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these auxiliary variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

init_dual(matrices, rank, mode, random_state=None)

Initialize the dual variables

Initialization schemes

"random_uniform": The elements of the dual variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the dual
variables are drawn from a standard normal distribution.

"zeros": The elements of the dual variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed dual variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed dual variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these dual variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these dual variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

abstract penalty(x): Compute the penalty for the given factor matrix or list of factor matrices.

subtract_from_aux(aux, dual)

Compute (aux - dual) for mode=0 and mode=2.

For some penalties, the aux is not a factor matrix but rather some other parametrization of a matrix. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

aux (tl.tensor(ndim=2)) – Auxiliary variables
dual (tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

tl.tensor(ndim=2)

subtract_from_auxes(auxes, duals)

Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

For some penalties, the aux is not a list of factor matrices but rather some other parametrization of a list of factor matrices. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

auxes (list of tl.tensor(ndim=2)) – Auxiliary variables
duals (list of tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

list of tl.tensor(ndim=2)

class matcouply.penalties.MatrixPenalty(aux_init='random_uniform', dual_init='random_uniform')[source]

Base class for penalties that can be applied to a single factor matrix at a time.

Methods:

`aux_as_matrix`(aux)	Convert an auxiliary variable to a matrix (mode=0 and mode=2).
`auxes_as_matrices`(auxes)	Convert a list of auxiliary variables to a list of matrices (mode=1).
`factor_matrices_update`(factor_matrices, ...)	Update all factor matrices in given list according to this penalty.
`factor_matrix_update`(factor_matrix, ...)	Update a factor matrix according to this penalty.
`init_aux`(matrices, rank, mode[, random_state])	Initialize the auxiliary variables
`init_dual`(matrices, rank, mode[, random_state])	Initialize the dual variables
`penalty`(x)	Compute the penalty for the given factor matrix or list of factor matrices.
`subtract_from_aux`(aux, dual)	Compute (aux - dual) for mode=0 and mode=2.
`subtract_from_auxes`(auxes, duals)	Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

aux_as_matrix(aux)

Convert an auxiliary variable to a matrix (mode=0 and mode=2).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: aux (tl.tensor(ndim=2)) –
Return type:: tl.tensor(ndim=2)

auxes_as_matrices(auxes)

Convert a list of auxiliary variables to a list of matrices (mode=1).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: auxes (list of tl.tensor(ndim=2)) –
Return type:: list of tl.tensor(ndim=2)

factor_matrices_update(factor_matrices, feasibility_penalties, auxes)[source]

Update all factor matrices in given list according to this penalty.

Parameters:

factor_matrices (list of tl.tensor(ndim=2)) – List of factor matrix to update.
feasibility_penalties (list of floats) – Penalty parameters for the feasibility gap of the different factor matrices.
auxes (list of tl.tensor(ndim=2)) – List of auxiliary matrices, each element corresponds to the auxiliary factor matrix for the same element in factor_matrices.

abstract factor_matrix_update(factor_matrix, feasibility_penalty, aux)[source]

Update a factor matrix according to this penalty.

Parameters:

factor_matrix (tl.tensor(ndim=2)) – Factor matrix to update.
feasibility_penalty (float) – Penalty parameter for infeasible solutions.
aux (tl.tensor(ndim=2)) – Auxiliary matrix that correspond to the factor matrix supplied to factor_matrix.

init_aux(matrices, rank, mode, random_state=None)

Initialize the auxiliary variables

Initialization schemes

"random_uniform": The elements of the auxiliary variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the auxiliary
variables are drawn from a standard normal distribution.

"zeros": The elements of the auxiliary variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed auxiliary variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed auxiliary variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these auxiliary variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these auxiliary variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

init_dual(matrices, rank, mode, random_state=None)

Initialize the dual variables

Initialization schemes

"random_uniform": The elements of the dual variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the dual
variables are drawn from a standard normal distribution.

"zeros": The elements of the dual variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed dual variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed dual variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these dual variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these dual variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

abstract penalty(x): Compute the penalty for the given factor matrix or list of factor matrices.

subtract_from_aux(aux, dual)

Compute (aux - dual) for mode=0 and mode=2.

For some penalties, the aux is not a factor matrix but rather some other parametrization of a matrix. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

aux (tl.tensor(ndim=2)) – Auxiliary variables
dual (tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

tl.tensor(ndim=2)

subtract_from_auxes(auxes, duals)

Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

For some penalties, the aux is not a list of factor matrices but rather some other parametrization of a list of factor matrices. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

auxes (list of tl.tensor(ndim=2)) – Auxiliary variables
duals (list of tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

list of tl.tensor(ndim=2)

class matcouply.penalties.NonNegativity(aux_init='random_uniform', dual_init='random_uniform')[source]

Impose non-negative values for the factor.

The non-negativity constraint works element-wise, constraining the elements of a factor to satisfy \(0 \leq x\), where \(x\) represents a factor element

Parameters:

aux_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables
dual_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables

Methods:

`aux_as_matrix`(aux)	Convert an auxiliary variable to a matrix (mode=0 and mode=2).
`auxes_as_matrices`(auxes)	Convert a list of auxiliary variables to a list of matrices (mode=1).
`factor_matrices_update`(factor_matrices, ...)	Update all factor matrices in given list according to this penalty.
`factor_matrix_row_update`(factor_matrix_row, ...)	Update a single row of a factor matrix according to this constraint.
`factor_matrix_update`(factor_matrix, ...)	Update a factor matrix according to this penalty.
`init_aux`(matrices, rank, mode[, random_state])	Initialize the auxiliary variables
`init_dual`(matrices, rank, mode[, random_state])	Initialize the dual variables
`penalty`(x)	Returns 0 as there is no penalty for hard constraints.
`subtract_from_aux`(aux, dual)	Compute (aux - dual) for mode=0 and mode=2.
`subtract_from_auxes`(auxes, duals)	Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

aux_as_matrix(aux)

Convert an auxiliary variable to a matrix (mode=0 and mode=2).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: aux (tl.tensor(ndim=2)) –
Return type:: tl.tensor(ndim=2)

auxes_as_matrices(auxes)

Convert a list of auxiliary variables to a list of matrices (mode=1).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: auxes (list of tl.tensor(ndim=2)) –
Return type:: list of tl.tensor(ndim=2)

factor_matrices_update(factor_matrices, feasibility_penalties, auxes)

Update all factor matrices in given list according to this penalty.

Parameters:

factor_matrices (list of tl.tensor(ndim=2)) – List of factor matrix to update.
feasibility_penalties (list of floats) – Penalty parameters for the feasibility gap of the different factor matrices.
auxes (list of tl.tensor(ndim=2)) – List of auxiliary matrices, each element corresponds to the auxiliary factor matrix for the same element in factor_matrices.

factor_matrix_row_update(factor_matrix_row, feasibility_penalty, aux_row)[source]

Update a single row of a factor matrix according to this constraint.

Parameters:

factor_matrix_row (tl.tensor(ndim=1)) – Vector (first order tensor) that corresponds to the single row in the factor matrix we wish to update.
feasibility_penalty (float) – Penalty parameter for infeasible solutions.
aux_row (tl.tensor(ndim=1)) – Vector (first order tensor) that corresponds to the row in the auxiliary matrix that correspond to the row supplied to factor_matrix_row.

factor_matrix_update(factor_matrix, feasibility_penalty, aux)[source]

Update a factor matrix according to this penalty.

Parameters:

factor_matrix (tl.tensor(ndim=2)) – Factor matrix to update.
feasibility_penalty (float) – Penalty parameter for infeasible solutions.
aux (tl.tensor(ndim=2)) – Auxiliary matrix that correspond to the factor matrix supplied to factor_matrix.

init_aux(matrices, rank, mode, random_state=None)

Initialize the auxiliary variables

Initialization schemes

"random_uniform": The elements of the auxiliary variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the auxiliary
variables are drawn from a standard normal distribution.

"zeros": The elements of the auxiliary variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed auxiliary variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed auxiliary variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these auxiliary variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these auxiliary variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

init_dual(matrices, rank, mode, random_state=None)

Initialize the dual variables

Initialization schemes

"random_uniform": The elements of the dual variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the dual
variables are drawn from a standard normal distribution.

"zeros": The elements of the dual variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed dual variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed dual variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these dual variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these dual variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

penalty(x)

Returns 0 as there is no penalty for hard constraints.

Hard constraints are always penalised with 0 even when the components are infeasible. Slightly infeasible components would otherwise result in an infinite penalty because the penalty function of hard constraints is 0 for feasible solutions and infinity for infeasible solutions. An infinite penalty would stop all convergence checking and not provide any information on the quality of the components. To ensure that the hard constraints are sufficiently imposed, it is recommended to examine the feasibility gap instead of the penalty and ensure that the feasibility gap is low.

Parameters:: x (tl.tensor(ndim=2) or list of tl.tensor(ndim=2)) – Factor matrix or list of factor matrices.

subtract_from_aux(aux, dual)

Compute (aux - dual) for mode=0 and mode=2.

For some penalties, the aux is not a factor matrix but rather some other parametrization of a matrix. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

aux (tl.tensor(ndim=2)) – Auxiliary variables
dual (tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

tl.tensor(ndim=2)

subtract_from_auxes(auxes, duals)

Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

For some penalties, the aux is not a list of factor matrices but rather some other parametrization of a list of factor matrices. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

auxes (list of tl.tensor(ndim=2)) – Auxiliary variables
duals (list of tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

list of tl.tensor(ndim=2)

class matcouply.penalties.Parafac2(svd='truncated_svd', n_iter=1, update_basis_matrices=True, update_coordinate_matrix=True, aux_init='random_uniform', dual_init='random_uniform')[source]

Impose the PARAFAC2 constraint on the uncoupled factor matrices.

The PARAFAC2 constraint can only be imposed on the uncoupled \(\mathbf{B}^{(i)}\)-matrices, and states that

\[{\mathbf{B}^{(i_1)}}^{\mathsf{T}}\mathbf{B}^{(i_1)} = {\mathbf{B}^{(i_2)}}^{\mathsf{T}}\mathbf{B}^{(i_2)},\]

for any \(i_1\) and \(i_2\). This constraint ensures uniqueness on the components so long as the number of coupled matrices are sufficiently large. A sufficent condition is that there are \(R(R+1)(R+2)(R+3)/24\) matrices, where \(R\) is the rank of the decomposition [HL96]. However, the true number of datasets required for uniqueness is typically lower, and it is conjectured that uniquenes for any \(R\) holds in practice whenever there are four or more matrices [KTBB99].

Parametrization of matrix-collections that satisfy the PARAFAC2 constraint

The standard way of parametrizing collections of matrices that satisfy the PARAFAC2 constraint is due to Kiers et al. [KTBB99]. If \(\{\mathbf{B}^{(i)}\}_{i=1}^I\) satsifies the PARAFAC2 constraint, then there exists a matrix \(\mathbf{\Delta} \in \mathbb{R}^{R \times R}\) (the coordinate matrix) and a collection of orthonormal matrices \(\{\mathbf{P}^{(i)}\}_{i=1}^I\) (the orthogonal basis matrices) such that

\[\mathbf{B}^{(i)} = \mathbf{P}^{(i)} \mathbf{\Delta}.\]

For this implementation, we use the above parametrization of the auxiliary variables, which is a tuple whose first element is a list of orthogonal basis matrices and second element is the coordinate matrix.

The proximal operator

To evaluate the proximal operator, we use the projection scheme presented in [RSC+22, RSCA21]. Specifically, we project with a coordinate descent scheme, where we first update the basis matrices and then update the coordinate matrix. It has been observed that only one iteration of this coordinate descent scheme is sufficient for fitting PARAFAC2 models with AO-ADMM [RSC+22, RSCA21].

To project \(\{\mathbf{X}^{(i)}\}_{i=1}^I\) onto the set of collections of matrices that satisfy the PARAFAC2 constraint, we first update the orthogonal basis matrices by

\[\mathbf{P}^{(i)} = \mathbf{U}^{(i)} {\mathbf{V}^{(i)}}^\mathsf{T}\]

where \(\mathbf{U}^{(i)}\) and \(\mathbf{V}^{(i)}\) contain the left and right singular vectors of \(\mathbf{X}^{(i)} \mathbf{\Delta}^\mathsf{T}\).

Then we update the coordinate matrix by

\[\mathbf{\Delta} = \frac{1}{\sum_{i=1}^I \rho_i}\sum_{i=1}^I \rho_i {\mathbf{P}^{(i)}}^\mathsf{T}\mathbf{X}^{(i)},\]

where \(\rho_i\) is the feasibility penalty (which parameterizes the norm of the projection) for the \(i\)-th factor matrix.

Parameters:

svd (str) – String that specifies which SVD algorithm to use. Valid strings are the keys of tensorly.SVD_FUNS.
n_iter (int) – Number of iterations for the coordinate descent scheme
aux_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables
dual_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables

Methods:

`aux_as_matrix`(aux)	Raises TypeError since the PARAFAC2 constraint only works with mode=1.
`auxes_as_matrices`(auxes)	Convert a the auxiliary variables into a list of matrices (mode=1).
`factor_matrices_update`(factor_matrices, ...)	Update all factor matrices in given list according to this penalty.
`init_aux`(matrices, rank, mode[, random_state])	Initialize the auxiliary variables
`init_dual`(matrices, rank, mode[, random_state])	Initialize the dual variables
`penalty`(x)	Returns 0 as there is no penalty for hard constraints.
`subtract_from_aux`(aux, dual)	Raises TypeError since the PARAFAC2 constraint only works with mode=1.
`subtract_from_auxes`(auxes, duals)	Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

aux_as_matrix(aux)[source]: Raises TypeError since the PARAFAC2 constraint only works with mode=1.

auxes_as_matrices(auxes)[source]

Convert a the auxiliary variables into a list of matrices (mode=1).

This function computes the list of matrices parametrized by the coordinate matrix and orthogonal basis matrices by multiplying them together.

Parameters:: auxes (tuple) – Tuple whose first element is the \(R imes R\) coordinate matrix and second element is the list of the orthogonal basis matrices.
Return type:: list of tl.tensor(ndim=2)

factor_matrices_update(factor_matrices, feasibility_penalties, auxes)[source]

Update all factor matrices in given list according to this penalty.

Parameters:

factor_matrices (list of tl.tensor(ndim=2)) – List of factor matrix to update.
feasibility_penalties (list of floats) – Penalty parameters for the feasibility gap of the different factor matrices.
auxes (list of tl.tensor(ndim=2)) – List of auxiliary matrices, each element corresponds to the auxiliary factor matrix for the same element in factor_matrices.

init_aux(matrices, rank, mode, random_state=None)[source]

Initialize the auxiliary variables

For all initialization schemes, the orthogonal basis matrices are initialized using the first \(R\) rows of an identity matrix.

Coordinate matrix initialization schemes

"random_uniform": The elements of the coordinate matrix
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the coordinate matrix
are drawn from a standard normal distribution.

"zeros": The elements of the coordinate matrix are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed coordinate matrix (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed coordinate matrix (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these auxiliary variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these auxiliary variables correspond to.
random_state (RandomState) – TensorLy random state.

Returns:

Tuple whose first element is the \(R \times R\) coordinate matrix and second element is the list of the orthogonal basis matrices.

Return type:

tuple

init_dual(matrices, rank, mode, random_state=None)

Initialize the dual variables

Initialization schemes

"random_uniform": The elements of the dual variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the dual
variables are drawn from a standard normal distribution.

"zeros": The elements of the dual variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed dual variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed dual variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these dual variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these dual variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

penalty(x)[source]

Returns 0 as there is no penalty for hard constraints.

Hard constraints are always penalised with 0 even when the components are infeasible. Slightly infeasible components would otherwise result in an infinite penalty because the penalty function of hard constraints is 0 for feasible solutions and infinity for infeasible solutions. An infinite penalty would stop all convergence checking and not provide any information on the quality of the components. To ensure that the hard constraints are sufficiently imposed, it is recommended to examine the feasibility gap instead of the penalty and ensure that the feasibility gap is low.

Parameters:: x (list of tl.tensor(ndim=2)) – List of factor matrices.

subtract_from_aux(aux, dual)[source]: Raises TypeError since the PARAFAC2 constraint only works with mode=1.

subtract_from_auxes(auxes, duals)[source]

Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

Computing the difference between the auxiliary variables and the dual variables is an essential part of ADMM. However, the auxiliary variables is not a list of factor matrices but rather a coordinate matrix and a collection of orthogonal basis matrices, so this difference cannot be computed by simply subtracting one from the other. First auxiliary matrices must be computed by multiplying the basis matrices with the coordinate matrix and then the difference can be computed.

Parameters:

auxes (tuple) – Tuple whose first element is the \(R \times R\) coordinate matrix and second element is the list of the orthogonal basis matrices.
duals (list of tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

list of tl.tensor(ndim=2)

class matcouply.penalties.RowVectorPenalty(aux_init='random_uniform', dual_init='random_uniform')[source]

Base class for penalties that can be applied to one row of a factor matrix at a time.

Methods:

`aux_as_matrix`(aux)	Convert an auxiliary variable to a matrix (mode=0 and mode=2).
`auxes_as_matrices`(auxes)	Convert a list of auxiliary variables to a list of matrices (mode=1).
`factor_matrices_update`(factor_matrices, ...)	Update all factor matrices in given list according to this penalty.
`factor_matrix_row_update`(factor_matrix_row, ...)	Update a single row of a factor matrix according to this constraint.
`factor_matrix_update`(factor_matrix, ...)	Update a factor matrix according to this penalty.
`init_aux`(matrices, rank, mode[, random_state])	Initialize the auxiliary variables
`init_dual`(matrices, rank, mode[, random_state])	Initialize the dual variables
`penalty`(x)	Compute the penalty for the given factor matrix or list of factor matrices.
`subtract_from_aux`(aux, dual)	Compute (aux - dual) for mode=0 and mode=2.
`subtract_from_auxes`(auxes, duals)	Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

aux_as_matrix(aux)

Convert an auxiliary variable to a matrix (mode=0 and mode=2).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: aux (tl.tensor(ndim=2)) –
Return type:: tl.tensor(ndim=2)

auxes_as_matrices(auxes)

Convert a list of auxiliary variables to a list of matrices (mode=1).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: auxes (list of tl.tensor(ndim=2)) –
Return type:: list of tl.tensor(ndim=2)

factor_matrices_update(factor_matrices, feasibility_penalties, auxes)

Update all factor matrices in given list according to this penalty.

Parameters:

factor_matrices (list of tl.tensor(ndim=2)) – List of factor matrix to update.
feasibility_penalties (list of floats) – Penalty parameters for the feasibility gap of the different factor matrices.
auxes (list of tl.tensor(ndim=2)) – List of auxiliary matrices, each element corresponds to the auxiliary factor matrix for the same element in factor_matrices.

abstract factor_matrix_row_update(factor_matrix_row, feasibility_penalty, aux_row)[source]

Update a single row of a factor matrix according to this constraint.

Parameters:

factor_matrix_row (tl.tensor(ndim=1)) – Vector (first order tensor) that corresponds to the single row in the factor matrix we wish to update.
feasibility_penalty (float) – Penalty parameter for infeasible solutions.
aux_row (tl.tensor(ndim=1)) – Vector (first order tensor) that corresponds to the row in the auxiliary matrix that correspond to the row supplied to factor_matrix_row.

factor_matrix_update(factor_matrix, feasibility_penalty, aux)[source]

Update a factor matrix according to this penalty.

Parameters:

factor_matrix (tl.tensor(ndim=2)) – Factor matrix to update.
feasibility_penalty (float) – Penalty parameter for infeasible solutions.
aux (tl.tensor(ndim=2)) – Auxiliary matrix that correspond to the factor matrix supplied to factor_matrix.

init_aux(matrices, rank, mode, random_state=None)

Initialize the auxiliary variables

Initialization schemes

"random_uniform": The elements of the auxiliary variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the auxiliary
variables are drawn from a standard normal distribution.

"zeros": The elements of the auxiliary variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed auxiliary variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed auxiliary variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these auxiliary variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these auxiliary variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

init_dual(matrices, rank, mode, random_state=None)

Initialize the dual variables

Initialization schemes

"random_uniform": The elements of the dual variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the dual
variables are drawn from a standard normal distribution.

"zeros": The elements of the dual variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed dual variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed dual variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these dual variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these dual variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

abstract penalty(x): Compute the penalty for the given factor matrix or list of factor matrices.

subtract_from_aux(aux, dual)

Compute (aux - dual) for mode=0 and mode=2.

For some penalties, the aux is not a factor matrix but rather some other parametrization of a matrix. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

aux (tl.tensor(ndim=2)) – Auxiliary variables
dual (tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

tl.tensor(ndim=2)

subtract_from_auxes(auxes, duals)

Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

For some penalties, the aux is not a list of factor matrices but rather some other parametrization of a list of factor matrices. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

auxes (list of tl.tensor(ndim=2)) – Auxiliary variables
duals (list of tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

list of tl.tensor(ndim=2)

class matcouply.penalties.TotalVariationPenalty(reg_strength, l1_strength=0, aux_init='random_uniform', dual_init='random_uniform')[source]

Impose piecewise constant components

Total variation regularization imposes piecewise constant components by obtaining components whose derivative is sparse. This sparsity is obtained using an L1 penalty. That is

\[g(\mathbf{x}) = \alpha \|\nabla \mathbf{x}\|_1 = \alpha \sum_{n=1}^{N} |x_n - x_{n-1}|,\]

where \(\alpha\) is a regularization coefficient that controls the sparsity level of the gradient and \(\nabla\) is the finite difference operator. \(\mathbf{x}\) is a column vector of a factor matrix, and all column vectors are penalised equally.

The total variation penalty is compatible with the L1 penalty in the sense that it is easy to compute the proximal operator of

\[g(\mathbf{x}) = \alpha \sum_{n=2}^{N} |x_n - x_{n-1}| + \sum_{n=1}^N \| x_n \|_1.\]

Specifically, if we first evaluate the proximal operator for the total variation penalty, followed by the proximal operator of the L1 penalty, then that is equivalent to evaluating the proximal operator of the sum of a TV and an L1 penalty [FHHoflingT07].

To evaluate the proximal operator, we use the improved direct total variation algorithm by Laurent Condat [Con13] (C code of the improved version is available here: https://lcondat.github.io/publications.html).

Parameters:

reg_strength (float (> 0)) – The strength of the total variation regularization (\(\alpha\) above)
l1_strength (float (>= 0)) – The strength of the L1 penalty (\(\beta\) above)
aux_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables
dual_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables

Note

The C-code this penalty is based in has a CeCILL lisence, which is compatible with GPL, but not MIT. This penalty is therefore not available with the default installation of MatCoupLy. To use this penalty, you need to install the GPL-lisenced condat-tv.

Note

This penalty is only available with the numpy backend, since it is based on an external C-library.

Methods:

`aux_as_matrix`(aux)	Convert an auxiliary variable to a matrix (mode=0 and mode=2).
`auxes_as_matrices`(auxes)	Convert a list of auxiliary variables to a list of matrices (mode=1).
`factor_matrices_update`(factor_matrices, ...)	Update all factor matrices in given list according to this penalty.
`factor_matrix_update`(factor_matrix, ...)	Update a factor matrix according to this penalty.
`init_aux`(matrices, rank, mode[, random_state])	Initialize the auxiliary variables
`init_dual`(matrices, rank, mode[, random_state])	Initialize the dual variables
`penalty`(x)	Compute the penalty for the given factor matrix or list of factor matrices.
`subtract_from_aux`(aux, dual)	Compute (aux - dual) for mode=0 and mode=2.
`subtract_from_auxes`(auxes, duals)	Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

aux_as_matrix(aux)

Convert an auxiliary variable to a matrix (mode=0 and mode=2).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: aux (tl.tensor(ndim=2)) –
Return type:: tl.tensor(ndim=2)

auxes_as_matrices(auxes)

Convert a list of auxiliary variables to a list of matrices (mode=1).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: auxes (list of tl.tensor(ndim=2)) –
Return type:: list of tl.tensor(ndim=2)

factor_matrices_update(factor_matrices, feasibility_penalties, auxes)

Update all factor matrices in given list according to this penalty.

Parameters:

factor_matrices (list of tl.tensor(ndim=2)) – List of factor matrix to update.
feasibility_penalties (list of floats) – Penalty parameters for the feasibility gap of the different factor matrices.
auxes (list of tl.tensor(ndim=2)) – List of auxiliary matrices, each element corresponds to the auxiliary factor matrix for the same element in factor_matrices.

factor_matrix_update(factor_matrix, feasibility_penalty, aux)[source]

Update a factor matrix according to this penalty.

Parameters:

factor_matrix (tl.tensor(ndim=2)) – Factor matrix to update.
feasibility_penalty (float) – Penalty parameter for infeasible solutions.
aux (tl.tensor(ndim=2)) – Auxiliary matrix that correspond to the factor matrix supplied to factor_matrix.

init_aux(matrices, rank, mode, random_state=None)

Initialize the auxiliary variables

Initialization schemes

"random_uniform": The elements of the auxiliary variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the auxiliary
variables are drawn from a standard normal distribution.

"zeros": The elements of the auxiliary variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed auxiliary variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed auxiliary variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these auxiliary variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these auxiliary variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

init_dual(matrices, rank, mode, random_state=None)

Initialize the dual variables

Initialization schemes

"random_uniform": The elements of the dual variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the dual
variables are drawn from a standard normal distribution.

"zeros": The elements of the dual variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed dual variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed dual variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these dual variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these dual variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

penalty(x)[source]: Compute the penalty for the given factor matrix or list of factor matrices.

subtract_from_aux(aux, dual)

Compute (aux - dual) for mode=0 and mode=2.

For some penalties, the aux is not a factor matrix but rather some other parametrization of a matrix. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

aux (tl.tensor(ndim=2)) – Auxiliary variables
dual (tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

tl.tensor(ndim=2)

subtract_from_auxes(auxes, duals)

Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

For some penalties, the aux is not a list of factor matrices but rather some other parametrization of a list of factor matrices. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

auxes (list of tl.tensor(ndim=2)) – Auxiliary variables
duals (list of tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

list of tl.tensor(ndim=2)

class matcouply.penalties.Unimodality(non_negativity=False, aux_init='random_uniform', dual_init='random_uniform')[source]

Constrain the component-vectors so they are unimodal.

Unimodal vectors \(\mathbf{u} \in \mathbb{R}^n\) have the property that

\[u_1 \leq u_2 \leq ... \leq u_{t-1} \leq u_t \geq u_{t+1} \geq ... \geq u_{n-1} \geq u_n\]

Projecting a general vector into the set of unimodal vectors (called unimodal regression) requires solving a set of isotonic regression problems (i.e. projections onto monotincally increasing or decreasing vectors). Two isotonic regression problems for each element in the vector. However, there is an incremental algorithm for fitting isotonic regression problems called prefix isotonic regression [Sto08], which can be used to solve unimodal problems in linear time [Sto08].

Parameters:

non_negativity (bool) – If True, then the components will also be non-negative
aux_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables
dual_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables

Methods:

`aux_as_matrix`(aux)	Convert an auxiliary variable to a matrix (mode=0 and mode=2).
`auxes_as_matrices`(auxes)	Convert a list of auxiliary variables to a list of matrices (mode=1).
`factor_matrices_update`(factor_matrices, ...)	Update all factor matrices in given list according to this penalty.
`factor_matrix_update`(factor_matrix, ...)	Update a factor matrix according to this penalty.
`init_aux`(matrices, rank, mode[, random_state])	Initialize the auxiliary variables
`init_dual`(matrices, rank, mode[, random_state])	Initialize the dual variables
`penalty`(x)	Returns 0 as there is no penalty for hard constraints.
`subtract_from_aux`(aux, dual)	Compute (aux - dual) for mode=0 and mode=2.
`subtract_from_auxes`(auxes, duals)	Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

aux_as_matrix(aux)

Convert an auxiliary variable to a matrix (mode=0 and mode=2).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: aux (tl.tensor(ndim=2)) –
Return type:: tl.tensor(ndim=2)

auxes_as_matrices(auxes)

Convert a list of auxiliary variables to a list of matrices (mode=1).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: auxes (list of tl.tensor(ndim=2)) –
Return type:: list of tl.tensor(ndim=2)

factor_matrices_update(factor_matrices, feasibility_penalties, auxes)

Update all factor matrices in given list according to this penalty.

Parameters:

factor_matrices (list of tl.tensor(ndim=2)) – List of factor matrix to update.
feasibility_penalties (list of floats) – Penalty parameters for the feasibility gap of the different factor matrices.
auxes (list of tl.tensor(ndim=2)) – List of auxiliary matrices, each element corresponds to the auxiliary factor matrix for the same element in factor_matrices.

factor_matrix_update(factor_matrix, feasibility_penalty, aux)[source]

Update a factor matrix according to this penalty.

Parameters:

factor_matrix (tl.tensor(ndim=2)) – Factor matrix to update.
feasibility_penalty (float) – Penalty parameter for infeasible solutions.
aux (tl.tensor(ndim=2)) – Auxiliary matrix that correspond to the factor matrix supplied to factor_matrix.

init_aux(matrices, rank, mode, random_state=None)

Initialize the auxiliary variables

Initialization schemes

"random_uniform": The elements of the auxiliary variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the auxiliary
variables are drawn from a standard normal distribution.

"zeros": The elements of the auxiliary variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed auxiliary variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed auxiliary variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these auxiliary variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these auxiliary variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

init_dual(matrices, rank, mode, random_state=None)

Initialize the dual variables

Initialization schemes

"random_uniform": The elements of the dual variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the dual
variables are drawn from a standard normal distribution.

"zeros": The elements of the dual variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed dual variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed dual variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these dual variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these dual variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

penalty(x)

Returns 0 as there is no penalty for hard constraints.

Hard constraints are always penalised with 0 even when the components are infeasible. Slightly infeasible components would otherwise result in an infinite penalty because the penalty function of hard constraints is 0 for feasible solutions and infinity for infeasible solutions. An infinite penalty would stop all convergence checking and not provide any information on the quality of the components. To ensure that the hard constraints are sufficiently imposed, it is recommended to examine the feasibility gap instead of the penalty and ensure that the feasibility gap is low.

Parameters:: x (tl.tensor(ndim=2) or list of tl.tensor(ndim=2)) – Factor matrix or list of factor matrices.

subtract_from_aux(aux, dual)

Compute (aux - dual) for mode=0 and mode=2.

For some penalties, the aux is not a factor matrix but rather some other parametrization of a matrix. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

aux (tl.tensor(ndim=2)) – Auxiliary variables
dual (tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

tl.tensor(ndim=2)

subtract_from_auxes(auxes, duals)

Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

For some penalties, the aux is not a list of factor matrices but rather some other parametrization of a list of factor matrices. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

auxes (list of tl.tensor(ndim=2)) – Auxiliary variables
duals (list of tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

list of tl.tensor(ndim=2)

class matcouply.penalties.UnitSimplex(aux_init='random_uniform', dual_init='random_uniform')[source]

Constrain the component-vectors so they are non-negative and sum to 1.

This is a hard constraint which is useful when the component-vectors represent probabilities.

Parameters:

aux_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables
dual_init ({"random_uniform", "random_standard_normal", "zeros", tl.tensor(ndim=2), list of tl.tensor(ndim=2)}) – Initialisation method for the auxiliary variables

Methods:

`aux_as_matrix`(aux)	Convert an auxiliary variable to a matrix (mode=0 and mode=2).
`auxes_as_matrices`(auxes)	Convert a list of auxiliary variables to a list of matrices (mode=1).
`factor_matrices_update`(factor_matrices, ...)	Update all factor matrices in given list according to this penalty.
`factor_matrix_update`(factor_matrix, ...)	Update a factor matrix according to this penalty.
`init_aux`(matrices, rank, mode[, random_state])	Initialize the auxiliary variables
`init_dual`(matrices, rank, mode[, random_state])	Initialize the dual variables
`penalty`(x)	Returns 0 as there is no penalty for hard constraints.
`subtract_from_aux`(aux, dual)	Compute (aux - dual) for mode=0 and mode=2.
`subtract_from_auxes`(auxes, duals)	Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

aux_as_matrix(aux)

Convert an auxiliary variable to a matrix (mode=0 and mode=2).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: aux (tl.tensor(ndim=2)) –
Return type:: tl.tensor(ndim=2)

auxes_as_matrices(auxes)

Convert a list of auxiliary variables to a list of matrices (mode=1).

This is an identity function that just returns its input. However, it is required for AO-ADMM to seamlessly work when the auxiliary variable is a parametrization of a matrix.

Parameters:: auxes (list of tl.tensor(ndim=2)) –
Return type:: list of tl.tensor(ndim=2)

factor_matrices_update(factor_matrices, feasibility_penalties, auxes)

Update all factor matrices in given list according to this penalty.

Parameters:

factor_matrices (list of tl.tensor(ndim=2)) – List of factor matrix to update.
feasibility_penalties (list of floats) – Penalty parameters for the feasibility gap of the different factor matrices.
auxes (list of tl.tensor(ndim=2)) – List of auxiliary matrices, each element corresponds to the auxiliary factor matrix for the same element in factor_matrices.

factor_matrix_update(factor_matrix, feasibility_penalty, aux)[source]

Update a factor matrix according to this penalty.

Parameters:

factor_matrix (tl.tensor(ndim=2)) – Factor matrix to update.
feasibility_penalty (float) – Penalty parameter for infeasible solutions.
aux (tl.tensor(ndim=2)) – Auxiliary matrix that correspond to the factor matrix supplied to factor_matrix.

init_aux(matrices, rank, mode, random_state=None)

Initialize the auxiliary variables

Initialization schemes

"random_uniform": The elements of the auxiliary variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the auxiliary
variables are drawn from a standard normal distribution.

"zeros": The elements of the auxiliary variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed auxiliary variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed auxiliary variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these auxiliary variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these auxiliary variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

init_dual(matrices, rank, mode, random_state=None)

Initialize the dual variables

Initialization schemes

"random_uniform": The elements of the dual variables
are drawn from a uniform distribution between 0 and 1.

"random_standard_normal": The elements of the dual
variables are drawn from a standard normal distribution.

"zeros": The elements of the dual variables are
initialized as zero.

tl.tensor(ndim=2) : Pre-computed dual variables (mode=0 or mode=2)

list of tl.tensor(ndim=2): Pre-computed dual variables (mode=1)

Parameters:

matrices (list of tensor(ndim=2) or tensor(ndim=3)) – The data matrices represented by the coupled matrix factorization these dual variables correspond to.
rank (int) – Rank of the decomposition.
mode (int) – The mode represented by the factor matrices that these dual variables correspond to.
random_state (RandomState) – TensorLy random state.

Return type:

tl.tensor(ndim=2) or list of tl.tensor(ndim=2)

penalty(x)

Returns 0 as there is no penalty for hard constraints.

Hard constraints are always penalised with 0 even when the components are infeasible. Slightly infeasible components would otherwise result in an infinite penalty because the penalty function of hard constraints is 0 for feasible solutions and infinity for infeasible solutions. An infinite penalty would stop all convergence checking and not provide any information on the quality of the components. To ensure that the hard constraints are sufficiently imposed, it is recommended to examine the feasibility gap instead of the penalty and ensure that the feasibility gap is low.

Parameters:: x (tl.tensor(ndim=2) or list of tl.tensor(ndim=2)) – Factor matrix or list of factor matrices.

subtract_from_aux(aux, dual)

Compute (aux - dual) for mode=0 and mode=2.

For some penalties, the aux is not a factor matrix but rather some other parametrization of a matrix. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

aux (tl.tensor(ndim=2)) – Auxiliary variables
dual (tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

tl.tensor(ndim=2)

subtract_from_auxes(auxes, duals)

Compute (aux - dual) for each auxiliary- and dual-factor matrix for mode=1.

For some penalties, the aux is not a list of factor matrices but rather some other parametrization of a list of factor matrices. This function is used so the AO-ADMM procedure can work with any auxiliary-variable parametrization seamlessly.

Parameters:

auxes (list of tl.tensor(ndim=2)) – Auxiliary variables
duals (list of tl.tensor(ndim=2)) – Dual variables (or other variable to subtract from the auxes)

Returns:

The list of differences

Return type:

list of tl.tensor(ndim=2)