Main Page
under_construction.png

UNDER CONSTRUCTION

This section of the wiki (or the entire wiki) is currently being renovated, expect missing information, broken links, templates, etc.

Welcome to a free encyclopedia on computational mechanics that anyone can edit!

IJBOL THEORY WIKI

複雑科学研究所

ijbolwiki02.png

"Research is about friendship and having fun."

$$\sigma_\mu(A) = -\log\; \mu(A)$$

(🖊️ edit the sidebar)(🖊️ edit the header)(🖊️ edit the navbar)

SLT.png Singular Learning Theory
Sloppy.png Sloppy Models
TDA.png Topological Data Analysis
Neuro.png Neurodynamics
SpecSheaves.png Spectral Sheaf Theory
Equivariant.png Equivariant Learning Theory

Did you know?


(🖊️ edit this segment)

Featured Article:

Energy-based model

Energy-based probabilistic models are closely related to physics, specified as a Boltzmann distribution (with the Boltzmann factor $kT = 1$):

$$p(\mathbf{x}; \mathbf{w}) = \frac{1}{Z_\mathbf{w}}e^{E(\mathbf{x}; \mathbf{w})}$$

The earliest energy-based probabilistic models in machine learning were in fact called Boltzmann machines, and map directly onto Ising spin models with a learned coupling structure $\mathbf{w}$. Inserting the Boltzmann form into the log-likelihood learning objective $l(\mathbf{w}) = \int q(\mathbf{x}) \log(\mathbf{x}; \mathbf{w}) \mathrm{d}\mathbf{x}$ yields:

$$-l(\mathbf{w}) = \langle E(\mathbf{x}; \mathbf{w})\rangle_q - F_\mathbf{w}$$

Where $\langle \cdot \rangle_q$ denotes an average with respect to the data distribution $q(\mathbf{x})$ and $F_\mathbf{w} = -\log Z_\mathbf{w}$ is the Helmholtz free energy of the model distribution $p(\mathbf{x}; \mathbf{w})$. Thus, learning via maximizing the log-likelihood corresponds to minimizing the energy of observed data while increasing overall free energy of the model distribution. Maximizing $l(\mathbf{w})$ is also equivalent to minimizing the Kullback-Leibler divergence,

$$D_\mathrm{KL}(q\| p) = \int q(\mathbf{x}) \log \left( \frac{q(\mathbf{x})}{p(\mathbf{x}; \mathbf{w})}\right)\mathrm{d}\mathbf{x} = G_\mathbf{w}(q) - F_\mathbf{w}$$

Kullback-Leibler divergence $D_\mathrm{KL}(q \| p)$ is nonnegative measure of the divergence between two distributions $q$ and $p$ that is zero if and only if $q=p$. In the special case when $p$ takes the Boltzmann form, the KL divergence becomes the difference between the Gibbs free energy of $q$, defined as $G_\mathbf{w}(q) = \langle E(\mathbf{x}; \mathbf{w})\rangle_q - S(q)$ (where $S(q) = -\int q(\mathbf{x})\log q(\mathbf{x}) \mathrm{d}\mathbf{x}$ is the entropy of $q$) and the Helmholtz free energy $F_\mathbf{w}$ of $p$.

Continue reading...


(🖊️ edit this segment)

Recently Added Articles:

In the works:

Rewrite: Hopf Decomposition


(🖊️ edit this segment)

hopfield-kolmogorov.png

Kolmogorov-Anderson Program

We focus on algorithmic information theory and equilibrium statistical mechanics to understand strong generalization (internal model selection in favor of minimal description length) in statistical models. In particular we are interested in singular learning theory, equivariant feature learning, and homological nature of entropy (in the sense of Bennequin).

contemporary-memetics.png

Conceptual Dynamics

We seek to address a formidable challenge of modeling belief propagation across human networks, particularly in semasiological and heterogeneous settings. We integrate insights from social sciences and neurobiology into modern deep learning, to understand complex dynamics of belief systems and relationship between micro- and macroscale social behavior.

sheaf-theoretic-deep-learning.png

Spectral Selective State Spaces

Leveraging the recent successes of selective state space models such as S4/S6, we aim to integrate them into geometric deep learning framework, with particular focus on spectral sheaf theory and optimal transport on $O(d)$-bundles. We believe that this approach will be advantageous in analysis and control of complex dynamical and belief propagation systems.

unsupervised-invariant-detection.png

Unsupervised Invariant Distillation/Detection

Can unsupervised learning models and their latent space representations be used in statistical detection and classification of invariants by probing rich moduli spaces? In what cases can human intuition be assisted by unsupervised learning models?

Explore Further:

[[include {$StartPageMiscelany}]]


(edit)

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License