Func#

parametricmatrixmodels.modules.Func

class Func(f=None, fname=None, params=None, state=())[source]#

Bases: BaseModule

Module that implements a general element-wise function, optionally with trainable parameters and state.

Warning

Saving and loading a Model which contains this module can be very tricky, as the function is automatically serialized using dill. For loaded models to work correctly, the same import paths must be used as when the model was saved. This means that the function must be defined in the same module and with the same name as when the model was saved. If the function is defined in a different module or with a different name, the model will not be able to deserialize the function and esoteric errors will occur.

__init__(f=None, fname=None, params=None, state=())[source]#

Initialize a Func module.

Parameters:

f (Union[Callable[[Array], Array], Callable[[tuple[Array, ...], Array], Array], Callable[[tuple[Array, ...], Array, tuple[Array, ...]], tuple[Array, tuple[Array, ...]]], Callable[[tuple[Array, ...], Array, tuple[Array, ...], Any], tuple[Array, tuple[Array, ...]]]]) –
A function that performs the modules operation. It can take only the input features and return only the output features (if there are no trainable parameters), or the tuple of trainable parameters and the input features and return the output features, or the tuple of trainable parameters, the input features and module state and return the output features and new state, or the trainable parameters, input features, module state, and a JAX rng key and return the output features and new state. This function will be applied element-wise to the input data. The output shape need not match the input shape, but the function should return a constant shape for all inputs of a given shape.

Summary of allowed signatures:
1. f(input_NF: Array) -> Array
  used in the case of no trainable parameters, no state, and no rng.
2. f(params: tuple[Array, ...], input_NF: Array) -> Array
  used in the case of trainable parameters, no state, and no rng.
3. f(params: tuple[Array, ...], input_NF: Array, state: tuple[Array, ...]) -> tuple[Array, tuple[Array, ...]]
  used in the case of trainable parameters, state, but no rng.
4. f(params: tuple[Array, ...], input_NF: Array, state: tuple[Array, ...], rng: Any) -> tuple[Array, tuple[Array, ...]]
  used in the case of trainable parameters, state, and rng.
fname (str) – Name of the function. If not provided, the function’s Pythonic name will be used.
params (tuple[Array, ...]) – Initial trainable parameters of the module. This can be used to store any tuple of numpy arrays that the function might need to maintain state across calls. If not provided, an empty tuple will be used. If the function requires trainable parameters, their initial values must be provided here.
state (tuple[Array, ...]) – Initial state of the module. This can be used to store any tuple of numpy arrays that the function might need to maintain state across calls. If not provided, an empty tuple will be used.

`__call__`	Call the module with the current parameters and given input, state, and rng.
`_get_callable`	Returns a `jax.jit`-able and `jax.grad`-able callable that represents the module's forward pass.
`astype`	Convenience wrapper to set_precision using the dtype argument, returns self.
`compile`	Compile the module to be used with the given input shape.
`deserialize`	Deserialize the module from a dictionary.
`get_hyperparameters`	Get the hyperparameters of the module.
`get_num_trainable_floats`	Returns the number of trainable floats in the module.
`get_output_shape`	Get the output shape of the module given the input shape.
`get_params`	Get the current trainable parameters of the module.
`get_state`	Get the current state of the module.
`is_ready`	Return True if the module is initialized and ready for training or inference.
`name`	Returns the name of the module, unless overridden, this is the class name.
`serialize`	Serialize the module to a dictionary.
`set_hyperparameters`	Set the hyperparameters of the module.
`set_params`	Set the trainable parameters of the module.
`set_precision`	Set the precision of the module parameters and state.
`set_state`	Set the state of the module.

__call__(input_NF, training=False, state=(), rng=None)#

Call the module with the current parameters and given input, state, and rng.

Parameters:

input_NF (Array) – Input array of shape (num_samples, num_features).
training (bool) – Whether the module is in training mode, by default False.
state (tuple[Array, ...]) – State of the module, by default ().
rng (Any) – JAX random key, by default None.

Return type:

tuple[Array, tuple[Array, ...]]

Returns:

Output array of shape (num_samples, num_output_features) and new state.

Raises:

ValueError – If the module is not ready (i.e., compile() has not been called).

See also

_get_callable: Returns a callable that can be used to compute the output and new state given the parameters, input, training flag, state, and rng.

_get_callable()[source]#

Returns a jax.jit-able and jax.grad-able callable that represents the module’s forward pass.

This method must be implemented by all subclasses and must return a jax-jit-able and jax-grad-able callable in the form of

module_callable(
    params: tuple[np.ndarray, ...],
    input_NF: np.ndarray[num_samples, num_features],
    training: bool,
    state: tuple[np.ndarray, ...],
    rng: Any
) -> (
        output_NF: np.ndarray[num_samples, num_output_features],
        new_state: tuple[np.ndarray, ...]
    )

That is, all hyperparameters are traced out and the callable depends explicitly only on a tuple of parameter jax.numpy arrays, the input array, the training flag, a state tuple of jax.numpy arrays, and a JAX rng key.

The training flag will be traced out, so it doesn’t need to be jittable

Return type:: Callable
Returns:: A callable that takes the module’s parameters, input data, training flag, state, and rng key and returns the output data and new state.
Raises:: NotImplementedError – If the method is not implemented in the subclass.

See also

__call__: Calls the module with the current parameters and given input, state, and rng.

_handle_inputs(f, fname=None, params=None, state=(), input_shape=None, output_shape=None)[source]#

Handle the hyperparameters for the Func module.

This includes input validation and setting up underlying hyperparameters from the provided inputs

Return type:: None

astype(dtype)#

Convenience wrapper to set_precision using the dtype argument, returns self.

Parameters:

dtype (dtype | str) – Precision to set for the module parameters. Valid options are: For 32-bit precision (all options are equivalent) np.float32, np.complex64, "float32", "complex64", "single", "f32", "c64", 32 For 64-bit precision (all options are equivalent) np.float64, np.complex128, "float64", "complex128", "double", "f64", "c128", 64

Return type:

BaseModule

Returns:

BaseModule – The module itself, with updated precision.

Raises:

ValueError – If the precision is invalid or if 64-bit precision is requested but JAX_ENABLE_X64 is not set.
RuntimeError – If the module is not ready (i.e., compile() has not been called).

See also

set_precision: Sets the precision of the module parameters and state.

compile(rng, input_shape)[source]#

Compile the module to be used with the given input shape.

This method initializes the module’s parameters and state based on the input shape and random key.

This is needed since Model s are built before the input data is given, so before training or inference can be done, the module needs to be compiled and each module passes its output shape to the next module’s compile method.

The RNG key is used to initialize random parameters, if needed.

This is not used to trace or jit the module’s callable, that is done automatically later.

Parameters:

rng (Any) – JAX random key.
input_shape (tuple[int, ...]) – Shape of the input data, e.g. (num_features,).

Raises:

NotImplementedError – If the method is not implemented in the subclass.

Return type:

None

deserialize(serial)[source]#

Deserialize the module from a dictionary.

This method sets the module’s parameters and state based on the provided dictionary.

The default implementation expects the dictionary to contain the module’s name, trainable parameters, and state.

Parameters:: data – Dictionary containing the serialized module data.
Raises:: ValueError – If the serialized data does not contain the expected keys or if the version of the serialized data is not compatible with with the current package version.
Return type:: None

get_hyperparameters()[source]#

Get the hyperparameters of the module.

Hyperparameters are used to configure the module and are not trainable. They can be set via set_hyperparameters.

Return type:: dict[str, Any]
Returns:: Dictionary containing the hyperparameters of the module.

get_num_trainable_floats()#

Returns the number of trainable floats in the module. If the module does not have trainable parameters, returns 0. If the module is not ready, returns None.

Return type:: int | None
Returns:: Number of trainable floats in the module, or None if the module is not ready.

get_output_shape(input_shape)[source]#

Get the output shape of the module given the input shape.

Parameters:: input_shape (tuple[int, ...]) – Shape of the input data, e.g. (num_features,).
Return type:: tuple[int, ...]
Returns:: Shape of the output data, e.g. (num_output_features,).
Raises:: NotImplementedError – If the method is not implemented in the subclass.

get_params()[source]#

Get the current trainable parameters of the module. If the module has no trainable parameters, this method should return an empty tuple.

Return type:: tuple[Array, ...]
Returns:: Tuple of numpy arrays representing the module’s parameters.
Raises:: NotImplementedError – If the method is not implemented in the subclass.

get_state()#

Get the current state of the module.

States are used to store “memory” or other information that is not passed between modules, is not trainable, but may be updated during either training or inference. e.g. batch normalization state.

The state is optional, in which case this method should return the empty tuple.

Return type:: tuple[Array, ...]
Returns:: Tuple of numpy arrays representing the module’s state.

is_ready()[source]#

Return True if the module is initialized and ready for training or inference.

Return type:: bool
Returns:: True if the module is ready, False otherwise.
Raises:: NotImplementedError – If the method is not implemented in the subclass.

name()[source]#

Returns the name of the module, unless overridden, this is the class name.

Return type:: str
Returns:: Name of the module.

serialize()[source]#

Serialize the module to a dictionary.

This method returns a dictionary representation of the module, including its parameters and state.

The default implementation serializes the module’s name, hyperparameters, trainable parameters, and state via a simple dictionary.

This only works if the module’s hyperparameters are auto-serializable. This includes basic types as well as numpy arrays.

Return type:: dict[str, Any]
Returns:: Dictionary containing the serialized module data.

set_hyperparameters(hyperparams)[source]#

Set the hyperparameters of the module.

Hyperparameters are used to configure the module and are not trainable. They can be set via this method.

The default implementation uses setattr to set the hyperparameters as attributes of the class instance.

Parameters:: hyperparameters – Dictionary containing the hyperparameters to set.
Raises:: TypeError – If hyperparameters is not a dictionary.
Return type:: None

set_params(params)[source]#

Set the trainable parameters of the module.

Parameters:: params (tuple[Array, ...]) – Tuple of numpy arrays representing the new parameters.
Raises:: NotImplementedError – If the method is not implemented in the subclass.
Return type:: None

set_precision(prec)#

Set the precision of the module parameters and state.

Parameters:

prec (dtype | str | int) – Precision to set for the module parameters. Valid options are: For 32-bit precision (all options are equivalent) np.float32, np.complex64, "float32", "complex64", "single", "f32", "c64", 32. For 64-bit precision (all options are equivalent) np.float64, np.complex128, "float64", "complex128", "double", "f64", "c128", 64.

Raises:

ValueError – If the precision is invalid or if 64-bit precision is requested but JAX_ENABLE_X64 is not set.
RuntimeError – If the module is not ready (i.e., compile() has not been called).

Return type:

None

See also

astype: Convenience wrapper to set_precision using the dtype argument, returns self.

set_state(state)#

Set the state of the module.

This method is optional.

Parameters:: state (tuple[Array, ...]) – Tuple of numpy arrays representing the new state.
Return type:: None

Func#

This Page