API for standard architectures

We lists notable implemented bijection architectures. These all inherit from the Bijection class.

Autoregressive architectures

class torchflows.architectures.RealNVP(event_shape: Tuple[int, ...] | Size | int, **kwargs)

Real non-volume-preserving (Real NVP) architecture.

Reference: Dinh et al. “Density estimation using Real NVP” (2017); https://arxiv.org/abs/1605.08803.

__init__(event_shape: Tuple[int, ...] | Size | int, **kwargs)
Parameters:

  • event_shape – shape of the event tensor.

  • kwargs – keyword arguments to AffineCoupling.

class torchflows.architectures.InverseRealNVP(event_shape: Tuple[int, ...] | Size | int, **kwargs)

Inverse of the Real NVP architecture.

Reference: Dinh et al. “Density estimation using Real NVP” (2017); https://arxiv.org/abs/1605.08803.

__init__(event_shape: Tuple[int, ...] | Size | int, **kwargs)
Parameters:

  • event_shape – shape of the event tensor.

  • kwargs – keyword arguments to InverseAffineCoupling.

class torchflows.architectures.NICE(event_shape: Tuple[int, ...] | Size | int, **kwargs)

Nonlinear independent components estimation (NICE) architecture.

Reference: Dinh et al. “NICE: Non-linear Independent Components Estimation” (2015); https://arxiv.org/abs/1410.8516.

__init__(event_shape: Tuple[int, ...] | Size | int, **kwargs)
Parameters:

  • event_shape – shape of the event tensor.

  • kwargs – keyword arguments to ShiftCoupling.

class torchflows.architectures.MAF(event_shape: Tuple[int, ...] | Size | int, **kwargs)

Masked autoregressive flow (MAF) architecture.

Reference: Papamakarios et al. “Masked Autoregressive Flow for Density Estimation” (2018); https://arxiv.org/abs/1705.07057.

__init__(event_shape: Tuple[int, ...] | Size | int, **kwargs)
Parameters:

  • event_shape – shape of the event tensor.

  • kwargs – keyword arguments to AffineForwardMaskedAutoregressive.

class torchflows.architectures.IAF(event_shape: Tuple[int, ...] | Size | int, **kwargs)

Inverse autoregressive flow (IAF) architecture.

Reference: Kingma et al. “Improving Variational Inference with Inverse Autoregressive Flow” (2017); https://arxiv.org/abs/1606.04934.

__init__(event_shape: Tuple[int, ...] | Size | int, **kwargs)
Parameters:

  • event_shape – shape of the event tensor.

  • kwargs – keyword arguments to AffineInverseMaskedAutoregressive.

class torchflows.architectures.CouplingRQNSF(event_shape: Tuple[int, ...] | Size | int, **kwargs)

Coupling rational quadratic neural spline flow (C-RQNSF) architecture.

Reference: Durkan et al. “Neural Spline Flows” (2019); https://arxiv.org/abs/1906.04032.

__init__(event_shape: Tuple[int, ...] | Size | int, **kwargs)
Parameters:

  • event_shape – shape of the event tensor.

  • kwargs – keyword arguments to RQSCoupling.

class torchflows.architectures.MaskedAutoregressiveRQNSF(event_shape: Tuple[int, ...] | Size | int, **kwargs)

Masked autoregressive rational quadratic neural spline flow (MA-RQNSF) architecture.

Reference: Durkan et al. “Neural Spline Flows” (2019); https://arxiv.org/abs/1906.04032.

__init__(event_shape: Tuple[int, ...] | Size | int, **kwargs)
Parameters:

  • event_shape – shape of the event tensor.

  • kwargs – keyword arguments to RQSForwardMaskedAutoregressive.

class torchflows.architectures.InverseAutoregressiveRQNSF(event_shape: Tuple[int, ...] | Size | int, **kwargs)

Inverse autoregressive rational quadratic neural spline flow (IA-RQNSF) architecture.

Reference: Durkan et al. “Neural Spline Flows” (2019); https://arxiv.org/abs/1906.04032.

__init__(event_shape: Tuple[int, ...] | Size | int, **kwargs)
Parameters:

  • event_shape – shape of the event tensor.

  • kwargs – keyword arguments to RQSInverseMaskedAutoregressive.

class torchflows.architectures.CouplingLRS(event_shape: Tuple[int, ...] | Size | int, **kwargs)

Coupling linear rational spline (C-LRS) architecture.

Reference: Dolatabadi et al. “Invertible Generative Modeling using Linear Rational Splines” (2020); https://arxiv.org/abs/2001.05168.

__init__(event_shape: Tuple[int, ...] | Size | int, **kwargs)
Parameters:

  • event_shape – shape of the event tensor.

  • kwargs – keyword arguments to LRSCoupling.

class torchflows.architectures.MaskedAutoregressiveLRS(event_shape: Tuple[int, ...] | Size | int, **kwargs)

Masked autoregressive linear rational spline (MA-LRS) architecture.

Reference: Dolatabadi et al. “Invertible Generative Modeling using Linear Rational Splines” (2020); https://arxiv.org/abs/2001.05168.

__init__(event_shape: Tuple[int, ...] | Size | int, **kwargs)
Parameters:

  • event_shape – shape of the event tensor.

  • kwargs – keyword arguments to LRSForwardMaskedAutoregressive.

class torchflows.architectures.InverseAutoregressiveLRS(event_shape: Tuple[int, ...] | Size | int, **kwargs)

Inverse autoregressive linear rational spline (MA-LRS) architecture.

Reference: Dolatabadi et al. “Invertible Generative Modeling using Linear Rational Splines” (2020); https://arxiv.org/abs/2001.05168.

__init__(event_shape: Tuple[int, ...] | Size | int, **kwargs)
Parameters:

  • event_shape – shape of the event tensor.

  • kwargs – keyword arguments to LRSInverseMaskedAutoregressive.

class torchflows.architectures.UMNNMAF(event_shape: Tuple[int, ...] | Size | int, **kwargs)

Unconstrained monotonic neural network masked autoregressive flow (UMNN-MAF) architecture.

Reference: Wehenkel and Louppe “Unconstrained Monotonic Neural Networks” (2021); https://arxiv.org/abs/1908.05164.

__init__(event_shape: Tuple[int, ...] | Size | int, **kwargs)
Parameters:

  • event_shape – shape of the event tensor.

  • kwargs – keyword arguments to UMNNMaskedAutoregressive.

Continuous architectures

class torchflows.architectures.DeepDiffeomorphicBijection(event_shape: Size | Tuple[int, ...], n_steps: int = 150, solver='euler', nn_kwargs: dict = None, **kwargs)

Deep diffeomorphic normalizing flow (DDNF) architecture.

Notes:

  • this model is implemented without Geodesic regularization. This is because torchdiffeq ODE solvers do not output the predicted velocity, only the point.

  • while the paper presents DDNF as a continuous normalizing flow, it implemented as a residual normalizing flow in this library. There is no functional difference.

  • IMPORTANT: the Euler solver produces very inaccurate results. Switching to the DOPRI5 solver massively improves reconstruction quality. However, we leave the Euler solver as it is presented in the original method.

Reference: Salman et al. “Deep diffeomorphic normalizing flows” (2018); https://arxiv.org/abs/1810.03256.

__init__(event_shape: Size | Tuple[int, ...], n_steps: int = 150, solver='euler', nn_kwargs: dict = None, **kwargs)

Constructor.

Parameters:

  • event_shape – shape of the event tensor.

  • n_steps – parameter T in the paper, i.e. the number of ResNet cells.

class torchflows.architectures.RNODE(event_shape: Size | Tuple[int, ...], nn_kwargs: dict = None, **kwargs)

Regularized neural ordinary differential equation (RNODE) architecture.

Reference: Finlay et al. “How to train your neural ODE: the world of Jacobian and kinetic regularization” (2020); https://arxiv.org/abs/2002.02798.

__init__(event_shape: Size | Tuple[int, ...], nn_kwargs: dict = None, **kwargs)
Parameters:

  • event_shape –

  • f – function to be integrated.

  • end_time – integrate f from t=0 to t=time_upper_bound. Default: 1.

  • solver – which solver to use.

  • kwargs –

class torchflows.architectures.FFJORD(event_shape: Size | Tuple[int, ...], nn_kwargs: dict = None, **kwargs)

Free-form Jacobian of reversible dynamics (FFJORD) architecture.

Gratwohl et al. “FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models” (2018); https://arxiv.org/abs/1810.01367.

__init__(event_shape: Size | Tuple[int, ...], nn_kwargs: dict = None, **kwargs)
Parameters:

  • event_shape –

  • f – function to be integrated.

  • end_time – integrate f from t=0 to t=time_upper_bound. Default: 1.

  • solver – which solver to use.

  • kwargs –

class torchflows.architectures.OTFlow(event_shape: Size | Tuple[int, ...], ot_flow_kwargs: dict = None, solver='dopri8', **kwargs)

Optimal transport flow (OT-flow) architecture.

Reference: Onken et al. “OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport” (2021); https://arxiv.org/abs/2006.00104.

__init__(event_shape: Size | Tuple[int, ...], ot_flow_kwargs: dict = None, solver='dopri8', **kwargs)

ContinuousBijection constructor.

Parameters:

  • event_shape – shape of the event tensor.

  • f – function to be integrated.

  • context_shape – shape of the context tensor.

  • end_time – integrate f from time 0 to this time. Default: 1.

  • solver – which solver to use.

  • atol – absolute tolerance for numerical integration.

  • rtol – relative tolerance for numerical integration.

  • kwargs – unused.

Residual architectures

class torchflows.architectures.ResFlow(event_shape, **kwargs)

Residual flow (ResFlow) architecture.

Reference: Chen et al. “Residual Flows for Invertible Generative Modeling” (2020); https://arxiv.org/abs/1906.02735.

__init__(event_shape, **kwargs)

BijectiveComposition constructor.

Parameters:

  • event_shape – shape of the event tensor.

  • layers (List[Bijection]) – bijection layers.

  • context_shape – shape of the context tensor.

  • kwargs – unused.

class torchflows.architectures.ProximalResFlow(event_shape, **kwargs)

Proximal residual flow architecture.

Reference: Hertrich “Proximal Residual Flows for Bayesian Inverse Problems” (2022); https://arxiv.org/abs/2211.17158.

__init__(event_shape, **kwargs)

BijectiveComposition constructor.

Parameters:

  • event_shape – shape of the event tensor.

  • layers (List[Bijection]) – bijection layers.

  • context_shape – shape of the context tensor.

  • kwargs – unused.

class torchflows.architectures.InvertibleResNet(event_shape, **kwargs)

Invertible residual network (i-ResNet) architecture.

Reference: Behrmann et al. “Invertible Residual Networks” (2019); https://arxiv.org/abs/1811.00995.

__init__(event_shape, **kwargs)

BijectiveComposition constructor.

Parameters:

  • event_shape – shape of the event tensor.

  • layers (List[Bijection]) – bijection layers.

  • context_shape – shape of the context tensor.

  • kwargs – unused.

class torchflows.architectures.PlanarFlow(event_shape: Size | Tuple[int, ...], **kwargs)

Planar flow architecture.

Note: this model currently supports only one-way transformations.

Reference: Rezende and Mohamed “Variational Inference with Normalizing Flows” (2016); https://arxiv.org/abs/1505.05770.

__init__(event_shape: Size | Tuple[int, ...], **kwargs)

BijectiveComposition constructor.

Parameters:

  • event_shape – shape of the event tensor.

  • layers (List[Bijection]) – bijection layers.

  • context_shape – shape of the context tensor.

  • kwargs – unused.

class torchflows.architectures.RadialFlow(event_shape: Size | Tuple[int, ...], **kwargs)

Radial flow architecture.

Note: this model currently supports only one-way transformations.

Reference: Rezende and Mohamed “Variational Inference with Normalizing Flows” (2016); https://arxiv.org/abs/1505.05770.

__init__(event_shape: Size | Tuple[int, ...], **kwargs)

BijectiveComposition constructor.

Parameters:

  • event_shape – shape of the event tensor.

  • layers (List[Bijection]) – bijection layers.

  • context_shape – shape of the context tensor.

  • kwargs – unused.

class torchflows.architectures.SylvesterFlow(event_shape: Size | Tuple[int, ...], **kwargs)

Sylvester flow architecture.

Note: this model currently supports only one-way transformations.

Reference: Van den Berg et al. “Sylvester Normalizing Flows for Variational Inference” (2019); https://arxiv.org/abs/1803.05649.

__init__(event_shape: Size | Tuple[int, ...], **kwargs)

BijectiveComposition constructor.

Parameters:

  • event_shape – shape of the event tensor.

  • layers (List[Bijection]) – bijection layers.

  • context_shape – shape of the context tensor.

  • kwargs – unused.