# Case Study: Relative Entropy Minimization

Back to main page

Case study background and problem formulations

Instructions for optimization with PSG Run-FilePSG MATLAB Subroutines and PSG R.

PROBLEM: problem_entropyr

Minimize Entropyr (minimizing Relative Entropy)
subject to
Linear = 0 (constraint on sum of decision variables (probabilities))
A * x = b (linear equality constraint)
Box constraints (non-negative lower bound for probabilities)
——————————————————————–
Entropyr = Relative Entropy
A = matrix in the linear equality constraint
b = vector in the right hand side of the linear equality constraint
Box constraints = constraints on individual decision variables
——————————————————————–
Data and solution in Run-File Environment

Problem Datasets # of Variables # of Scenarios Objective Value Solving Time, PC 2.66GHz (sec)
Dataset1 Problem Statement Data Solution 250 3 2.167273 <0.01
Dataset2 Problem Statement Data Solution 100,000 3 1.923068 1.38

Data and solution in MATLAB Environment

Problem Datasets # of Variables # of Scenarios Objective Value Solving Time, PC 2.66GHz (sec)
Dataset1 Matlab code Data Solution 250 3 2.167273 <0.01
Dataset2 Matlab code Data Solution 100,000 3 1.92307 1.1

Data and solution in R Environment

Problem Datasets # of Variables # of Scenarios Objective Value Solving Time, PC 2.66GHz (sec)
Dataset1 R code Data 250 3 2.167273 <0.01
Dataset2 R code Data 100,000 3 1.92307 1.1

CASE STUDY SUMMARY

This case study demonstrates an optimization problem for minimizing Relative Entropy with linear constraints. Relative Entropy is used to find a probability distribution which is the most close to some “prior” probability distribution subject to available information about the distribution. For instance, moments of a distribution can be known and we want to find the “best” distribution accounting for this information.
Problem is solved for 250 probability atoms of a discrete probability distribution (250 decision variables), and for 100,000 probability atoms of a discrete probability distribution (100,000 decision variables).
Problem is solved using 2 datasets:
• Dataset1 for 250 decision variables;
• Dataset2 for 100,000 decision variables.