Generate and return the Kullback-Leibler Divergence (KLD) \(D_{KL}(P \parallel Q)\) of a given univariate Normal distribution \(Q\) from a reference Normal distribution \(P\).
The Kullback-Leibler Divergence, also known as the relative entropy, of a univariate Normal distribution \(Q\) from a reference univariate Normal distribution \(P\) is defined as,
\begin{equation}
\large
D_{KL}(P \parallel Q) =
\frac{(\mu_P - \mu_Q)^2}{2\sigma_Q^2} +
\frac{1}{2}\bigg( \frac{\sigma_P}{\sigma_Q} - \ln\big(\frac{\sigma_P}{\sigma_Q}\big) - 1 \bigg) ~,
\end{equation}
where \(\mu\) and \(\sigma\) represent the respective standard deviations of the Normal distributions.
- Parameters
-
[in] | meanDiffSq | : The input scalar or array of the same shape as other array-like arguments of the same type kind as the output kld , representing the difference-squared of the mean parameters of the two Normal distributions ( \((P, Q)\)).
(optional, default = 0. , that is, the two distributions have the same mean.) |
[in] | varP | : The input scalar or array of the same shape as other array-like arguments, of the same type and kind as the output kld representing the variance of the reference Normal distribution \(P\).
(optional, default = 1. , it must be present if and only if varQ is also present.) |
[in] | varQ | : The input scalar or array of the same shape as other array-like arguments, of the same type and kind as meanDiffSq representing the variance of the target Normal distribution \(Q\).
(optional, default = 1. , it must be present if and only if varP is also present.) |
- Returns
kld
: The input scalar or array of the same shape as input array-like arguments, of,
-
type
real
of kind any supported by the processor (e.g., RK, RK32, RK64, or RK128),
representing the Kullback-Leibler divergence of the target distribution with respect to the reference.
Possible calling interfaces ⛓
Generate and return the Kullback-Leibler Divergence (KLD) of a given univariate Normal distribution ...
This module contains classes and procedures for computing various statistical quantities related to t...
- Warning
- The conditions
0 <= meanDiffSq
must hold for the corresponding input arguments.
The conditions 0 < varP
must hold for the corresponding input arguments.
The conditions 0 < varQ
must hold for the corresponding input arguments.
These conditions are verified only if the library is built with the preprocessor macro CHECK_ENABLED=1
.
-
The
pure
procedure(s) documented herein become impure
when the ParaMonte library is compiled with preprocessor macro CHECK_ENABLED=1
.
By default, these procedures are pure
in release
build and impure
in debug
and testing
builds.
Example usage ⛓
10 type(display_type) :: disp
14 call disp%show(
"!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%")
15 call disp%show(
"!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%")
16 call disp%show(–
"! Compute the KullbackLeibler Divergence (KLD) distance the Normal distribution Q from P.")
17 call disp%show(
"!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%")
18 call disp%show(
"!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%")
22 call disp%show(
"getNormKLD(meanDiffSq = (0.5_RKS - 1.5_RKS)**2) ! assuming standard deviations of 1.")
24 call disp%show(
"getNormKLD(meanDiffSq = (0.5_RKD - 1.5_RKD)**2)")
26 call disp%show(
"getNormKLD(meanDiffSq = (0.5_RKH - 1.5_RKH)**2)")
31 call disp%show(
"getNormKLD(varP = 2._RKS , varQ = 3._RKS) ! assuming similar means.")
33 call disp%show(
"getNormKLD(varP = 2._RKD , varQ = 3._RKD)")
35 call disp%show(
"getNormKLD(varP = 2._RKH, varQ = 3._RKH)")
40 call disp%show(
"getNormKLD(varP = 3._RKS , varQ = 2._RKS ) ! assuming similar means.")
42 call disp%show(
"getNormKLD(varP = 3._RKD , varQ = 2._RKD )")
44 call disp%show(
"getNormKLD(varP = 3._RKH, varQ = 2._RKH)")
49 call disp%show(
"getNormKLD(meanDiffSq = 0.2_RKS , varP = 3._RKS , varQ = 2._RKS ! assuming similar means.")
51 call disp%show(
"getNormKLD(meanDiffSq = 0.2_RKD , varP = 3._RKD , varQ = 2._RKD )")
53 call disp%show(
"getNormKLD(meanDiffSq = 0.2_RKH, varP = 3._RKH, varQ = 2._RKH)")
This is a generic method of the derived type display_type with pass attribute.
This is a generic method of the derived type display_type with pass attribute.
This module contains classes and procedures for input/output (IO) or generic display operations on st...
type(display_type) disp
This is a scalar module variable an object of type display_type for general display.
This module defines the relevant Fortran kind type-parameters frequently used in the ParaMonte librar...
integer, parameter IK
The default integer kind in the ParaMonte library: int32 in Fortran, c_int32_t in C-Fortran Interoper...
integer, parameter RKD
The double precision real kind in Fortran mode. On most platforms, this is an 64-bit real kind.
integer, parameter SK
The default character kind in the ParaMonte library: kind("a") in Fortran, c_char in C-Fortran Intero...
integer, parameter RKH
The scalar integer constant of intrinsic default kind, representing the highest-precision real kind t...
integer, parameter RKS
The single-precision real kind in Fortran mode. On most platforms, this is an 32-bit real kind.
Generate and return an object of type display_type.
Example Unix compile command via Intel ifort
compiler ⛓
3ifort -fpp -standard-semantics -O3 -Wl,-rpath,../../../lib -I../../../inc main.F90 ../../../lib/libparamonte* -o main.exe
Example Windows Batch compile command via Intel ifort
compiler ⛓
2set PATH=..\..\..\lib;%PATH%
3ifort /fpp /standard-semantics /O3 /I:..\..\..\include main.F90 ..\..\..\lib\libparamonte*.lib /exe:main.exe
Example Unix / MinGW compile command via GNU gfortran
compiler ⛓
3gfortran -cpp -ffree-line-length-none -O3 -Wl,-rpath,../../../lib -I../../../inc main.F90 ../../../lib/libparamonte* -o main.exe
Example output ⛓
14+0.500000000000000000000000000000000000
20+0.36065887387415563E-1
22+0.360658873874155243223398910655078833E-1
28+0.47267445945917808E-1
30+0.472674459459178090109934422678254821E-1
33getNormKLD(meanDiffSq
= 0.2_RKS , varP
= 3._RKS , varQ
= 2._RKS
35getNormKLD(meanDiffSq
= 0.2_RKD , varP
= 3._RKD , varQ
= 2._RKD )
36+0.97267445945917810E-1
37getNormKLD(meanDiffSq
= 0.2_RKH, varP
= 3._RKH, varQ
= 2._RKH)
38+0.972674459459178090109934422678254845E-1
- Test:
- test_pm_distNorm
- Todo:
- High Priority: The KLD implementation should be implemented with
log1mexp
for better accuracy.
Final Remarks ⛓
If you believe this algorithm or its documentation can be improved, we appreciate your contribution and help to edit this page's documentation and source file on GitHub.
For details on the naming abbreviations, see this page.
For details on the naming conventions, see this page.
This software is distributed under the MIT license with additional terms outlined below.
-
If you use any parts or concepts from this library to any extent, please acknowledge the usage by citing the relevant publications of the ParaMonte library.
-
If you regenerate any parts/ideas from this library in a programming environment other than those currently supported by this ParaMonte library (i.e., other than C, C++, Fortran, MATLAB, Python, R), please also ask the end users to cite this original ParaMonte library.
This software is available to the public under a highly permissive license.
Help us justify its continued development and maintenance by acknowledging its benefit to society, distributing it, and contributing to it.
- Copyright
- Computational Data Science Lab
- Author:
- Amir Shahmoradi, Oct 16, 2009, 11:14 AM, Michigan
Definition at line 3942 of file pm_distNorm.F90.