ActInf ModelStream 020.1: Kenric Nelson: "Learning extreme models with the Coupled Free Energy" скачать в хорошем качестве

ActInf ModelStream 020.1: Kenric Nelson: "Learning extreme models with the Coupled Free Energy"

Variational Inference Optimized Using the Curved Geometry of Coupled Free Energy Kenric Nelson, Igor Oliveira, Megan Hess https://link.springer.com/chapter/10.... We introduce an optimization framework for variational inference based on the coupled free energy, extending variational inference techniques to account for the curved geometry of the coupled exponential family. This family includes important heavy-tailed distributions such as the generalized Pareto and the Student’s t. By leveraging the coupled free energy, which is equal to the coupled evidence lower bound (ELBO) of the inverted probabilities, we improve the accuracy and robustness of the learned model. The coupled generalization of Fisher Information metric and the affine connection. The method is applied to the design of a coupled variational autoencoder (CVAE). By using the coupling for both the distributions and cost functions, the reconstruction metric is derived to still be the mean-square average loss with modified constants. The novelty comes from sampling the heavy-tailed latent distribution with its associated coupled probability, which has faster decaying tails. The result is the ability to train a model robust against severe outliers, while assuring that the training process is stable. The Wasserstein-2 or Fréchet Inception Distance of the reconstructed CelebA images shows the CVAE has a 3% improvement over the VAE after 5 epochs of training. On the uniqueness of the coupled entropy Kenric P. Nelson https://arxiv.org/abs/2511.17684v2 The coupled entropy, Hκ, is proven to uniquely satisfy the requirement that a generalized entropy be a measure of the uncertainty at the scale, σ, for a class of non-exponential distributions. The coupled stretched exponential distributions, including the generalized Pareto and Student's t distributions, are uniquely parameterized to quantify linear uncertainty with the scale and nonlinear uncertainty with the tail shape for a broad class of complex systems. Thereby, the coupled entropy optimizes the representation of the uncertainty due to linear sources. Lemmas for the composability and extensivity of the coupled entropy are proven. The uniqueness of the coupled entropy is further supported by demonstrating consistent thermodynamic relationships, which correspond to a model used for the momentum of high-energy particle collisions. Applications of the coupled entropy in measuring statistical complexity, training variational inference algorithms, and designing communication channels are reviewed. Reduced Perplexity: A simplified perspective on assessing probabilistic forecasts Kenric P. Nelson https://arxiv.org/abs/1603.08830 A simple, intuitive approach to the assessment of probabilistic inferences is introduced. The Shannon information metrics are translated to the probability domain. The translation shows that the negative logarithmic score and the geometric mean are equivalent measures of the accuracy of a probabilistic inference. Thus there is both a quantitative reduction in perplexity, which is the inverse of the geometric mean of the probabilities, as good inference algorithms reduce the uncertainty and a qualitative reduction due to the increased clarity between the original set of probabilistic forecasts and their central tendency, the geometric mean. Further insight is provided by showing that the Rényi and Tsallis entropy functions translated to the probability domain are both the weighted generalized mean of the distribution. The generalized mean of probabilistic forecasts forms a spectrum of performance metrics referred to as a Risk Profile. The arithmetic mean is used to measure the decisiveness, while the -2/3 mean is used to measure the robustness. ---- Active Inference Institute information: Website: https://www.activeinference.institute/ Activities: https://activities.activeinference.in... Discord: https://discord.activeinference.insti... Donate: http://donate.activeinference.institute/ YouTube:    / activeinference   X: https://x.com/InferenceActive Active Inference Livestreams: https://video.activeinference.institute/