pm_mathLogSumExp::getLogSumExp Interface Reference

Generate and return the natural logarithm of the sum of the exponential of the input array robustly (without numerical overflow). More...

Detailed Description

Generate and return the natural logarithm of the sum of the exponential of the input array robustly (without numerical overflow).

Parameters

[in]	array	: The input contiguous array of shape (:) of either, type real of kind any supported by the processor (e.g., RK, RK32, RK64, or RK128) or, type complex of kind any supported by the processor (e.g., CK, CK32, CK64, or CK128), whose natural logarithm of sum of the exponential of its elements will be generated and returned.
[in]	maxArray	: The input scalar of the same type and kind as the input array representing the maximum value in array (i.e., maxArray = maxval(array)). If the input array is of complex type, then only real component must be considered in the computation of the maximum value. Specifying this argument, if known a priori, will significantly aid the runtime performance. (optional. It must be present if the input argument control is also present.)
	control	: The input scalar object that can be, the constant selection or equivalently, an object of type selection_type. Specifying this option enables the runtime checks for underflow occurrence via branching and dynamic dispatch. Enabling this option can aid runtime efficiency when the division of a significant number of elements of array (for example, half or more) by maxArray causes underflow. This option avoids the computation of an exponentiation term, leading to better runtime efficiency. Note that exponentiation is highly expensive (on the order of ~200 CPU cycles). See the relevant benchmark here. The presence of this argument is merely for compile-time resolution of the procedures of this generic interface. (optional. If missing, a sequence control flow will be assumed without dynamic dispatch.)

Returns

logSumExp : The output scalar of the same type and kind as the input array containing the natural logarithm of the sum of the exponential of the input array robustly (without numerical overflow).

Possible calling interfaces ⛓

use pm_mathLogSumExp, only: getLogSumExp, selection
 
logSumExp = getLogSumExp(array(1:np), maxArray)
logSumExp = getLogSumExp(array(1:np), maxArray, control)

Warning

The condition real(maxArray) == maxval(real(array)) must hold.
This condition is verified only if the library is built with the preprocessor macro CHECK_ENABLED=1.

The pure procedure(s) documented herein become impure when the ParaMonte library is compiled with preprocessor macro CHECK_ENABLED=1.
By default, these procedures are pure in release build and impure in debug and testing builds.

See also

getLog1p
get1mexp
getMinMax
setMinMax
getLogAddExp
getLogSubExp
getLogSumExp
getCumPropExp
setCumPropExp
getCumSum
setCumSum

Example usage ⛓

program example
 
    use pm_kind, only: SK
    use pm_io, only: display_type
    use pm_kind, only: IK, CKS, CKD, CKH, RKS, RKD, RKH
    use pm_mathLogSumExp, only: getLogSumExp 
 
    implicit none
 
    type(display_type)      :: disp
    disp = display_type(file = "main.out.F90")
 
    call disp%skip()
    call disp%show("!%%%%%%%%%%")
    call disp%show("32-bit real")
    call disp%show("!%%%%%%%%%%")
    call disp%skip()
 
    call disp%skip()
    call disp%show("exp( getLogSumExp( array = log([1._RKS, 2._RKS, 3._RKS, 4._RKS]), maxArray = log(4._RKS) ) )")
    call disp%show( exp( getLogSumExp( array = log([1._RKS, 2._RKS, 3._RKS, 4._RKS]), maxArray = log(4._RKS) ) ) )
    call disp%skip()
 
    call disp%skip()
    call disp%show("!%%%%%%%%%%")
    call disp%show("64-bit real")
    call disp%show("!%%%%%%%%%%")
    call disp%skip()
 
    call disp%skip()
    call disp%show("exp( getLogSumExp( array = log([1._RKD, 2._RKD, 3._RKD, 4._RKD]), maxArray = log(4._RKD) ) )")
    call disp%show( exp( getLogSumExp( array = log([1._RKD, 2._RKD, 3._RKD, 4._RKD]), maxArray = log(4._RKD) ) ) )
    call disp%skip()
 
    call disp%skip()
    call disp%show("!%%%%%%%%%%%")
    call disp%show("128-bit real")
    call disp%show("!%%%%%%%%%%%")
    call disp%skip()
 
    call disp%skip()
    call disp%show("exp( getLogSumExp( array = log([1._RKH, 2._RKH, 3._RKH, 4._RKH]), maxArray = log(4._RKH) ) )")
    call disp%show( exp( getLogSumExp( array = log([1._RKH, 2._RKH, 3._RKH, 4._RKH]), maxArray = log(4._RKH) ) ) )
    call disp%skip()
 
    call disp%skip()
    call disp%show("!%%%%%%%%%%%%%")
    call disp%show("32-bit complex")
    call disp%show("!%%%%%%%%%%%%%")
    call disp%skip()
 
    call disp%skip()
    call disp%show("exp( getLogSumExp( array = cmplx(log([(1._CKS,-1._CKS), (2._CKS,-2._CKS), (3._CKS,-3._CKS)]), kind = CKS), maxArray = log((3._CKS,-3._CKS)) ) )")
    call disp%show( exp( getLogSumExp( array = cmplx(log([(1._CKS,-1._CKS), (2._CKS,-2._CKS), (3._CKS,-3._CKS)]), kind = CKS), maxArray = log((3._CKS,-3._CKS)) ) ) )
    call disp%skip()
 
    call disp%skip()
    call disp%show("!%%%%%%%%%%%%%")
    call disp%show("64-bit complex")
    call disp%show("!%%%%%%%%%%%%%")
    call disp%skip()
 
    call disp%skip()
    call disp%show("exp( getLogSumExp( array = cmplx(log([(1._CKD,-1._CKD), (2._CKD,-2._CKD), (3._CKD,-3._CKD)]), kind = CKD), maxArray = log((3._CKD,-3._CKD)) ) )")
    call disp%show( exp( getLogSumExp( array = cmplx(log([(1._CKD,-1._CKD), (2._CKD,-2._CKD), (3._CKD,-3._CKD)]), kind = CKD), maxArray = log((3._CKD,-3._CKD)) ) ) )
    call disp%skip()
 
    call disp%skip()
    call disp%show("!%%%%%%%%%%%%%%")
    call disp%show("128-bit complex")
    call disp%show("!%%%%%%%%%%%%%%")
    call disp%skip()
 
    call disp%skip()
    call disp%show("exp( getLogSumExp( array = cmplx(log([(1._CKH,-1._CKH), (2._CKH,-2._CKH), (3._CKH,-3._CKH)]), kind = CKH), maxArray = log((3._CKH,-3._CKH)) ) )")
    call disp%show( exp( getLogSumExp( array = cmplx(log([(1._CKH,-1._CKH), (2._CKH,-2._CKH), (3._CKH,-3._CKH)]), kind = CKH), maxArray = log((3._CKH,-3._CKH)) ) ) )
    call disp%skip()
 
end program example

Example Unix compile command via Intel ifort compiler ⛓

#!/usr/bin/env sh
rm main.exe
ifort -fpp -standard-semantics -O3 -Wl,-rpath,../../../lib -I../../../inc main.F90 ../../../lib/libparamonte* -o main.exe
./main.exe

Example Windows Batch compile command via Intel ifort compiler ⛓

del main.exe
set PATH=..\..\..\lib;%PATH%
ifort /fpp /standard-semantics /O3 /I:..\..\..\include main.F90 ..\..\..\lib\libparamonte*.lib /exe:main.exe
main.exe

Example Unix / MinGW compile command via GNU gfortran compiler ⛓

#!/usr/bin/env sh
rm main.exe
gfortran -cpp -ffree-line-length-none -O3 -Wl,-rpath,../../../lib -I../../../inc main.F90 ../../../lib/libparamonte* -o main.exe
./main.exe

Example output ⛓

 
!%%%%%%%%%%
32-bit real
!%%%%%%%%%%
 
 
exp( getLogSumExp( array = log([1._RKS, 2._RKS, 3._RKS, 4._RKS]), maxArray = log(4._RKS) ) )
+10.0000000
 
 
!%%%%%%%%%%
64-bit real
!%%%%%%%%%%
 
 
exp( getLogSumExp( array = log([1._RKD, 2._RKD, 3._RKD, 4._RKD]), maxArray = log(4._RKD) ) )
+10.000000000000002
 
 
!%%%%%%%%%%%
128-bit real
!%%%%%%%%%%%
 
 
exp( getLogSumExp( array = log([1._RKH, 2._RKH, 3._RKH, 4._RKH]), maxArray = log(4._RKH) ) )
+10.0000000000000000000000000000000000
 
 
!%%%%%%%%%%%%%
32-bit complex
!%%%%%%%%%%%%%
 
 
exp( getLogSumExp( array = cmplx(log([(1._CKS,-1._CKS), (2._CKS,-2._CKS), (3._CKS,-3._CKS)]), kind = CKS), maxArray = log((3._CKS,-3._CKS)) ) )
(+6.00000048, -6.00000048)
 
 
!%%%%%%%%%%%%%
64-bit complex
!%%%%%%%%%%%%%
 
 
exp( getLogSumExp( array = cmplx(log([(1._CKD,-1._CKD), (2._CKD,-2._CKD), (3._CKD,-3._CKD)]), kind = CKD), maxArray = log((3._CKD,-3._CKD)) ) )
(+6.0000000000000000, -5.9999999999999991)
 
 
!%%%%%%%%%%%%%%
128-bit complex
!%%%%%%%%%%%%%%
 
 
exp( getLogSumExp( array = cmplx(log([(1._CKH,-1._CKH), (2._CKH,-2._CKH), (3._CKH,-3._CKH)]), kind = CKH), maxArray = log((3._CKH,-3._CKH)) ) )
(+6.00000000000000000000000000000000000, -5.99999999999999999999999999999999923)
 
 

Benchmarks:

Benchmark :: The effects of control on runtime efficiency ⛓

The following program compares the runtime performance of getLogSumExp algorithm with and without checking for underflows.

! Test the performance of `getLogSumExp()` with and without underflow check vs. direct computation of `logSumExp`.
program benchmark
 
    use iso_fortran_env, only: error_unit
    use pm_kind, only: IK, LK, RK, SK
    use pm_bench, only: bench_type
 
    implicit none
 
    integer(IK)                         :: i
    integer(IK)                         :: iarr
    integer(IK)                         :: fileUnitN
    integer(IK)                         :: fileUnitU
    integer(IK) , parameter             :: NARR = 11_IK
    integer(IK) , parameter             :: NBENCH = 3_IK
    integer(IK)                         :: arraySize(NARR)
    real(RK)                            :: dummySum = 0._RK
    real(RK)                            :: maxArray
    real(RK)    , allocatable           :: array(:)
    real(RK)                            :: logSumExp
    type(bench_type)                    :: bench(2,NBENCH)
    logical(LK)                         :: underflowEnabled
 
    bench(1:2,1) = bench_type(name = SK_"getLogSumExpMaxSequence", exec = getLogSumExpMaxSequence , overhead = setOverhead)
    bench(1:2,2) = bench_type(name = SK_"getLogSumExpMaxSelection", exec = getLogSumExpMaxSelection , overhead = setOverhead)
    bench(1:2,3) = bench_type(name = SK_"getLogSumExpDirect", exec = getLogSumExpDirect, overhead = setOverhead)
 
    arraySize = [( 2_IK**iarr, iarr = 1_IK, NARR )]
 
    write(*,"(*(g0,:,' '))")
    write(*,"(*(g0,:,' '))") "getLogSumExp(...) vs. getLogSumExp(..., control = selection) vs. direct method."
    write(*,"(*(g0,:,' '))")
 
    open(newunit = fileUnitN, file = "main.normal.out", status = "replace")
    open(newunit = fileUnitU, file = "main.underflow.out", status = "replace")
 
        write(fileUnitN, "(*(g0,:,','))") "arraySize", (bench(1,i)%name, i = 1, NBENCH)
        write(fileUnitU, "(*(g0,:,','))") "arraySize", (bench(2,i)%name, i = 1, NBENCH)
 
        loopOverArraySize: do iarr = 1, NARR
 
            allocate(array(arraySize(iarr)))
            write(*,"(*(g0,:,' '))") "Benchmarking with array size", arraySize(iarr)
 
            underflowEnabled = .false._LK
            do i = 1, NBENCH
            !do i = NBENCH, 1, -1
                bench(1,i)%timing = bench(1,i)%getTiming(minsec = 0.07_RK)
            end do
            write(fileUnitN,"(*(g0,:,','))") arraySize(iarr), (bench(1,i)%timing%mean, i = 1, NBENCH)
 
            underflowEnabled = .true._LK
            do i = 1, NBENCH
                bench(2,i)%timing = bench(2,i)%getTiming(minsec = 0.07_RK)
            end do
            write(fileUnitU,"(*(g0,:,','))") arraySize(iarr), (bench(2,i)%timing%mean, i = 1, NBENCH)
 
            deallocate(array)
 
        end do loopOverArraySize
        write(*,"(*(g0,:,' '))") dummySum
        write(*,"(*(g0,:,' '))")
 
    close(fileUnitN)
    close(fileUnitU)
 
contains
 
    !%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    ! procedure wrappers.
    !%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
    subroutine setOverhead()
        call setArray()
        call getDummy()
    end subroutine
 
    subroutine setArray()
        call random_number(array)
        if (underflowEnabled) array = array * (maxexponent(0._RK) - minexponent(0._RK)) + minexponent(0._RK)
        maxArray = maxval(array)
    end subroutine
 
    subroutine getDummy()
        dummySum = dummySum + logSumExp
    end subroutine
 
    subroutine getLogSumExpMaxSequence()
        use pm_mathLogSumExp, only: getLogSumExp
        call setArray()
        logSumExp = getLogSumExp(array, maxArray)
        call getDummy()
    end subroutine
 
    subroutine getLogSumExpMaxSelection()
        use pm_mathLogSumExp, only: getLogSumExp, selection
        call setArray()
        logSumExp = getLogSumExp(array, maxArray, selection)
        call getDummy()
    end subroutine
 
    subroutine getLogSumExpDirect()
        call setArray()
        logSumExp = maxArray + log(sum(exp(array - maxArray)))
        call getDummy()
    end subroutine
 
end program benchmark

Example Unix compile command via Intel ifort compiler ⛓

#!/usr/bin/env sh
rm main.exe
ifort -fpp -standard-semantics -O3 -Wl,-rpath,../../../lib -I../../../inc main.F90 ../../../lib/libparamonte* -o main.exe
./main.exe

Example Windows Batch compile command via Intel ifort compiler ⛓

del main.exe
set PATH=..\..\..\lib;%PATH%
ifort /fpp /standard-semantics /O3 /I:..\..\..\include main.F90 ..\..\..\lib\libparamonte*.lib /exe:main.exe
main.exe

Example Unix / MinGW compile command via GNU gfortran compiler ⛓

#!/usr/bin/env sh
rm main.exe
gfortran -cpp -ffree-line-length-none -O3 -Wl,-rpath,../../../lib -I../../../inc main.F90 ../../../lib/libparamonte* -o main.exe
./main.exe

Postprocessing of the benchmark output ⛓

#!/usr/bin/env python
 
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
 
import os
dirname = os.path.basename(os.getcwd()) 
 
fontsize = 14
 
 
 
df = pd.read_csv("main.normal.out")
colnames = list(df.columns.values)
 
ax = plt.figure(figsize = 1.25 * np.array([6.4,4.6]), dpi = 200)
ax = plt.subplot()
 
for colname in colnames[1:]:
    plt.plot( df[colnames[0]].values
           #, np.ones(len(df[colnames[0]].values))
            , df[colname].values / df[colnames[1]].values
            , linewidth = 2
           #, linestyle = "-"
            )
 
plt.xticks(fontsize = fontsize)
plt.yticks(fontsize = fontsize)
ax.set_xlabel("Array Size", fontsize = fontsize)
ax.set_ylabel("Runtime Ratio ( W.R.T. cenabled = .false. )", fontsize = fontsize)
ax.set_title("Performance when the input arrays\ncause no underflow instances.\nLower is better.", fontsize = fontsize)
ax.set_xscale("log")
#ax.set_yscale("log")
plt.minorticks_on()
plt.grid(visible = True, which = "both", axis = "both", color = "0.85", linestyle = "-")
ax.tick_params(axis = "y", which = "minor")
ax.tick_params(axis = "x", which = "minor")
ax.legend   ( colnames[1:]
           #, loc='center left'
           #, bbox_to_anchor=(1, 0.5)
            , fontsize = fontsize
            )
 
plt.tight_layout()
plt.savefig("benchmark." + dirname + ".normal.png")
 
 
 
df = pd.read_csv("main.underflow.out")
colnames = list(df.columns.values)
 
ax = plt.figure(figsize = 1.25 * np.array([6.4,4.6]), dpi = 200)
ax = plt.subplot()
 
for colname in colnames[1:]:
    plt.plot( df[colnames[0]].values
           #, np.ones(len(df[colnames[0]].values))
            , df[colname].values / df[colnames[1]].values
            , linewidth = 2
           #, linestyle = "-"
            )
 
plt.xticks(fontsize = fontsize)
plt.yticks(fontsize = fontsize)
ax.set_xlabel("Array Size", fontsize = fontsize)
ax.set_ylabel("Runtime Ratio W.R.T. {})".format(colnames[1]), fontsize = fontsize)
ax.set_title("Performance when the input arrays\ncause many underflow instances.\nLower is better.", fontsize = fontsize)
ax.set_xscale("log")
#ax.set_yscale("log")
plt.minorticks_on()
plt.grid(visible = True, which = "both", axis = "both", color = "0.85", linestyle = "-")
ax.tick_params(axis = "y", which = "minor")
ax.tick_params(axis = "x", which = "minor")
ax.legend   ( colnames[1:]
           #, loc='center left'
           #, bbox_to_anchor=(1, 0.5)
            , fontsize = fontsize
            )
 
plt.tight_layout()
plt.savefig("benchmark." + dirname + ".underflow.png")

Visualization of the benchmark output ⛓

Benchmark moral ⛓

1. If the input array has many (half the size of array or more) elements whose division by the maxval(array) causes underflow, then setting control = selection to allow branching will likely result in a faster runtime.
   Conversely, if the divisions are not expected to cause any or too many underflows, then removing the input argument control when calling getLogSumExp will lead to a better runtime performance (at the expense of occasional expensive but redundant exponentiation operations).
   
2. Note the performance of manual computation against getLogSumExp with or without setting the optional argument control.
   

Test:

test_pm_mathLogSumExp

Todo:

Normal Priority: This generic interface can be expanded to include input arrays with Weights.

Final Remarks ⛓

---

If you believe this algorithm or its documentation can be improved, we appreciate your contribution and help to edit this page's documentation and source file on GitHub.
For details on the naming abbreviations, see this page.
For details on the naming conventions, see this page.
This software is distributed under the MIT license with additional terms outlined below.

1. If you use any parts or concepts from this library to any extent, please acknowledge the usage by citing the relevant publications of the ParaMonte library.
   
2. If you regenerate any parts/ideas from this library in a programming environment other than those currently supported by this ParaMonte library (i.e., other than C, C++, Fortran, MATLAB, Python, R), please also ask the end users to cite this original ParaMonte library.
   

This software is available to the public under a highly permissive license.
Help us justify its continued development and maintenance by acknowledging its benefit to society, distributing it, and contributing to it.

Copyright

Computational Data Science Lab

Author:

Fatemeh Bagheri, Tuesday 08:49 PM, August 10, 2021, Dallas, TX

Definition at line 147 of file pm_mathLogSumExp.F90.

---

The documentation for this interface was generated from the following file:

• src/fortran/main/pm_mathLogSumExp.F90