ParaMonte Fortran 2.0.0
Parallel Monte Carlo and Machine Learning Library
See the latest version documentation.
pm_distNorm::getNormKLD Interface Reference

Generate and return the Kullback-Leibler Divergence (KLD) \(D_{KL}(P \parallel Q)\) of a given univariate Normal distribution \(Q\) from a reference Normal distribution \(P\). More...

Detailed Description

Generate and return the Kullback-Leibler Divergence (KLD) \(D_{KL}(P \parallel Q)\) of a given univariate Normal distribution \(Q\) from a reference Normal distribution \(P\).

The Kullback-Leibler Divergence, also known as the relative entropy, of a univariate Normal distribution \(Q\) from a reference univariate Normal distribution \(P\) is defined as,

\begin{equation} \large D_{KL}(P \parallel Q) = \frac{(\mu_P - \mu_Q)^2}{2\sigma_Q^2} + \frac{1}{2}\bigg( \frac{\sigma_P}{\sigma_Q} - \ln\big(\frac{\sigma_P}{\sigma_Q}\big) - 1 \bigg) ~, \end{equation}

where \(\mu\) and \(\sigma\) represent the respective standard deviations of the Normal distributions.

Parameters
[in]meanDiffSq: The input scalar or array of the same shape as other array-like arguments of the same type kind as the output kld, representing the difference-squared of the mean parameters of the two Normal distributions ( \((P, Q)\)).
(optional, default = 0., that is, the two distributions have the same mean.)
[in]varP: The input scalar or array of the same shape as other array-like arguments, of the same type and kind as the output kld representing the variance of the reference Normal distribution \(P\).
(optional, default = 1., it must be present if and only if varQ is also present.)
[in]varQ: The input scalar or array of the same shape as other array-like arguments, of the same type and kind as meanDiffSq representing the variance of the target Normal distribution \(Q\).
(optional, default = 1., it must be present if and only if varP is also present.)
Returns
kld : The input scalar or array of the same shape as input array-like arguments, of,
  1. type real of kind any supported by the processor (e.g., RK, RK32, RK64, or RK128),
representing the Kullback-Leibler divergence of the target distribution with respect to the reference.


Possible calling interfaces

KLD = getNormKLD(meanDiffSq)
KLD = getNormKLD(varP, varQ)
KLD = getNormKLD(meanDiffSq, varP, varQ)
Generate and return the Kullback-Leibler Divergence (KLD) of a given univariate Normal distribution ...
This module contains classes and procedures for computing various statistical quantities related to t...
Warning
The conditions 0 <= meanDiffSq must hold for the corresponding input arguments.
The conditions 0 < varP must hold for the corresponding input arguments.
The conditions 0 < varQ must hold for the corresponding input arguments.
These conditions are verified only if the library is built with the preprocessor macro CHECK_ENABLED=1.
The pure procedure(s) documented herein become impure when the ParaMonte library is compiled with preprocessor macro CHECK_ENABLED=1.
By default, these procedures are pure in release build and impure in debug and testing builds.
Remarks
The procedures under discussion are elemental.


Example usage

1program example
2
3 use pm_kind, only: SK
4 use pm_kind, only: IK, RKS, RKD, RKH ! all real kinds are supported.
5 use pm_distNorm, only: getNormKLD
6 use pm_io, only: display_type
7
8 implicit none
9
10 type(display_type) :: disp
11 disp = display_type(file = "main.out.F90")
12
13 call disp%skip()
14 call disp%show("!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%")
15 call disp%show("!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%")
16 call disp%show(–"! Compute the KullbackLeibler Divergence (KLD) distance the Normal distribution Q from P.")
17 call disp%show("!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%")
18 call disp%show("!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%")
19 call disp%skip()
20
21 call disp%skip()
22 call disp%show("getNormKLD(meanDiffSq = (0.5_RKS - 1.5_RKS)**2) ! assuming standard deviations of 1.")
23 call disp%show( getNormKLD(meanDiffSq = (0.5_RKS - 1.5_RKS)**2) )
24 call disp%show("getNormKLD(meanDiffSq = (0.5_RKD - 1.5_RKD)**2)")
25 call disp%show( getNormKLD(meanDiffSq = (0.5_RKD - 1.5_RKD)**2) )
26 call disp%show("getNormKLD(meanDiffSq = (0.5_RKH - 1.5_RKH)**2)")
27 call disp%show( getNormKLD(meanDiffSq = (0.5_RKH - 1.5_RKH)**2) )
28 call disp%skip()
29
30 call disp%skip()
31 call disp%show("getNormKLD(varP = 2._RKS , varQ = 3._RKS) ! assuming similar means.")
32 call disp%show( getNormKLD(varP = 2._RKS , varQ = 3._RKS) )
33 call disp%show("getNormKLD(varP = 2._RKD , varQ = 3._RKD)")
34 call disp%show( getNormKLD(varP = 2._RKD , varQ = 3._RKD) )
35 call disp%show("getNormKLD(varP = 2._RKH, varQ = 3._RKH)")
36 call disp%show( getNormKLD(varP = 2._RKH, varQ = 3._RKH) )
37 call disp%skip()
38
39 call disp%skip()
40 call disp%show("getNormKLD(varP = 3._RKS , varQ = 2._RKS ) ! assuming similar means.")
41 call disp%show( getNormKLD(varP = 3._RKS , varQ = 2._RKS ) )
42 call disp%show("getNormKLD(varP = 3._RKD , varQ = 2._RKD )")
43 call disp%show( getNormKLD(varP = 3._RKD , varQ = 2._RKD ) )
44 call disp%show("getNormKLD(varP = 3._RKH, varQ = 2._RKH)")
45 call disp%show( getNormKLD(varP = 3._RKH, varQ = 2._RKH) )
46 call disp%skip()
47
48 call disp%skip()
49 call disp%show("getNormKLD(meanDiffSq = 0.2_RKS , varP = 3._RKS , varQ = 2._RKS ! assuming similar means.")
50 call disp%show( getNormKLD(meanDiffSq = 0.2_RKS , varP = 3._RKS , varQ = 2._RKS ) )
51 call disp%show("getNormKLD(meanDiffSq = 0.2_RKD , varP = 3._RKD , varQ = 2._RKD )")
52 call disp%show( getNormKLD(meanDiffSq = 0.2_RKD , varP = 3._RKD , varQ = 2._RKD ) )
53 call disp%show("getNormKLD(meanDiffSq = 0.2_RKH, varP = 3._RKH, varQ = 2._RKH)")
54 call disp%show( getNormKLD(meanDiffSq = 0.2_RKH, varP = 3._RKH, varQ = 2._RKH) )
55 call disp%skip()
56
57end program example
This is a generic method of the derived type display_type with pass attribute.
Definition: pm_io.F90:11726
This is a generic method of the derived type display_type with pass attribute.
Definition: pm_io.F90:11508
This module contains classes and procedures for input/output (IO) or generic display operations on st...
Definition: pm_io.F90:252
type(display_type) disp
This is a scalar module variable an object of type display_type for general display.
Definition: pm_io.F90:11393
This module defines the relevant Fortran kind type-parameters frequently used in the ParaMonte librar...
Definition: pm_kind.F90:268
integer, parameter IK
The default integer kind in the ParaMonte library: int32 in Fortran, c_int32_t in C-Fortran Interoper...
Definition: pm_kind.F90:540
integer, parameter RKD
The double precision real kind in Fortran mode. On most platforms, this is an 64-bit real kind.
Definition: pm_kind.F90:568
integer, parameter SK
The default character kind in the ParaMonte library: kind("a") in Fortran, c_char in C-Fortran Intero...
Definition: pm_kind.F90:539
integer, parameter RKH
The scalar integer constant of intrinsic default kind, representing the highest-precision real kind t...
Definition: pm_kind.F90:858
integer, parameter RKS
The single-precision real kind in Fortran mode. On most platforms, this is an 32-bit real kind.
Definition: pm_kind.F90:567
Generate and return an object of type display_type.
Definition: pm_io.F90:10282

Example Unix compile command via Intel ifort compiler
1#!/usr/bin/env sh
2rm main.exe
3ifort -fpp -standard-semantics -O3 -Wl,-rpath,../../../lib -I../../../inc main.F90 ../../../lib/libparamonte* -o main.exe
4./main.exe

Example Windows Batch compile command via Intel ifort compiler
1del main.exe
2set PATH=..\..\..\lib;%PATH%
3ifort /fpp /standard-semantics /O3 /I:..\..\..\include main.F90 ..\..\..\lib\libparamonte*.lib /exe:main.exe
4main.exe

Example Unix / MinGW compile command via GNU gfortran compiler
1#!/usr/bin/env sh
2rm main.exe
3gfortran -cpp -ffree-line-length-none -O3 -Wl,-rpath,../../../lib -I../../../inc main.F90 ../../../lib/libparamonte* -o main.exe
4./main.exe

Example output
1
2!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
3!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
4! Compute the Kullback–Leibler Divergence (KLD) distance the Normal distribution Q from P.
5!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
6!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
7
8
9getNormKLD(meanDiffSq = (0.5_RKS - 1.5_RKS)**2) ! assuming standard deviations of 1.
10+0.500000000
11getNormKLD(meanDiffSq = (0.5_RKD - 1.5_RKD)**2)
12+0.50000000000000000
13getNormKLD(meanDiffSq = (0.5_RKH - 1.5_RKH)**2)
14+0.500000000000000000000000000000000000
15
16
17getNormKLD(varP = 2._RKS , varQ = 3._RKS) ! assuming similar means.
18+0.360658765E-1
19getNormKLD(varP = 2._RKD , varQ = 3._RKD)
20+0.36065887387415563E-1
21getNormKLD(varP = 2._RKH, varQ = 3._RKH)
22+0.360658873874155243223398910655078833E-1
23
24
25getNormKLD(varP = 3._RKS , varQ = 2._RKS ) ! assuming similar means.
26+0.472674370E-1
27getNormKLD(varP = 3._RKD , varQ = 2._RKD )
28+0.47267445945917808E-1
29getNormKLD(varP = 3._RKH, varQ = 2._RKH)
30+0.472674459459178090109934422678254821E-1
31
32
33getNormKLD(meanDiffSq = 0.2_RKS , varP = 3._RKS , varQ = 2._RKS ! assuming similar means.
34+0.972674340E-1
35getNormKLD(meanDiffSq = 0.2_RKD , varP = 3._RKD , varQ = 2._RKD )
36+0.97267445945917810E-1
37getNormKLD(meanDiffSq = 0.2_RKH, varP = 3._RKH, varQ = 2._RKH)
38+0.972674459459178090109934422678254845E-1
39
40
Test:
test_pm_distNorm
Todo:
High Priority: The KLD implementation should be implemented with log1mexp for better accuracy.


Final Remarks


If you believe this algorithm or its documentation can be improved, we appreciate your contribution and help to edit this page's documentation and source file on GitHub.
For details on the naming abbreviations, see this page.
For details on the naming conventions, see this page.
This software is distributed under the MIT license with additional terms outlined below.

  1. If you use any parts or concepts from this library to any extent, please acknowledge the usage by citing the relevant publications of the ParaMonte library.
  2. If you regenerate any parts/ideas from this library in a programming environment other than those currently supported by this ParaMonte library (i.e., other than C, C++, Fortran, MATLAB, Python, R), please also ask the end users to cite this original ParaMonte library.

This software is available to the public under a highly permissive license.
Help us justify its continued development and maintenance by acknowledging its benefit to society, distributing it, and contributing to it.

Author:
Amir Shahmoradi, Oct 16, 2009, 11:14 AM, Michigan

Definition at line 3942 of file pm_distNorm.F90.


The documentation for this interface was generated from the following file: