# Case Study: Support Vector Regression: Risk Quadrangle Framework

Instructions for optimization with PSG Run-File, PSG MATLAB Toolbox, PSG Python, and PSG R.

Problem 1: Nu-SVR as Error Minimization (primal)

Value:
var_risk
——————————————————————————————
cvar_risk = conditional value-at-risk specified by an abs(matrix of scenarios)
var_risk = value-at-risk specified by an abs(matrix of scenarios)
——————————————————————————————
Data and solution in different environments:

Dataset 1 1,000 0.7742 <0.01 # of Variables # of Scenarios Objective Value Solving Time, PC 2.7GHz (sec) Environments Run-File Problem Statement Data Solution Matlab Toolbox Data Python Python Code R R Code

Problem 2: epsilon-SVR as Error Minimization (primal)

Value:
pr_pen
——————————————————————–——————
pm_pen = partial moment function specified by an abs(matrix of scenarios)
pr_pen = probability of exceedance specified by an abs(matrix of scenarios)
——————————————————————–——————
Data and solution in different environments:

Dataset 1 1,000 0.4082 <0.01 # of Variables # of Scenarios Objective Value Solving Time, PC 2.7GHz (sec) Environments Run-File Problem Statement Data Solution Matlab Toolbox Data Python Python Code R R Code

Problem 3: Nu-SVR as Deviation Minimization (primal)

Minimize (1-alpha)/2*cvar_dev + (1+alpha)/2*cvar_dev + quadratic
Value:
var_risk
——————————————————————–——————
cvar_dev = conditional value-at-risk deviation specified by a reduced matrix of scenarios
——————————————————————–——————
Data and solution in different environments:

Dataset 1 1,000 0.7742 <0.01 # of Variables # of Scenarios Objective Value Solving Time, PC 2.7GHz (sec) Environments Run-File Problem Statement Data Solution Matlab Toolbox Data Python Python Code R R Code

Problem 4a: Nu-SVR as Error Minimization (dual, constraints linearized automatically)

Subject to
polynom_abs <= C*(1-alpha)
linear = 0
Box: <= C, >= -C
——————————————————————–——————
linear = linear fumction specified by a vector of dependent values (in objective) and by a unit vector (in constraint)
polynom_abs = l1 norm specified by a unit vector
——————————————————————–——————
Data and solution in different environments:

Dataset 1,000 1,000 774.2196 <0.2 # of Variables # of Scenarios Objective Value Solving Time, PC 2.7GHz (sec) Environments Run-File Problem Statement Data Solution Matlab Toolbox Data Python Python Code R R Code

Problem 4b: Nu-SVR as Error Minimization (dual, constraints linearized manually)

Subject to
linear <= C*(1-alpha)
linear = 0
-1<=linearmulti<=1
limearmulti>=0
limearmulti>=0
——————————————————————–——————
linear = linear fumction specified by a vector of dependent values (in objective) and by a unit vector (in constraint)
polynom_abs = l1 norm specified by a unit vector
linearmulti = linear multiple function specified by a sparse pmatrix in each case
——————————————————————–——————
Data and solution in different environments:

Dataset 1,000 1,000 774.2196 <0.2 # of Variables # of Scenarios Objective Value Solving Time, PC 2.7GHz (sec) Environments Run-File Problem Statement Data Solution Matlab Toolbox Data Python (GUROBI) Python Code R R Code

CASE STUDY SUMMARY

This case study conducts Support Vector Regression (SVR) as regularized error minimization, regularized deviation minimization, and constrained quadratic programming (CQP) problem on simulated data (1000 observations), cf. .
Specifically, four equivalent implementations of SVR are considered (cf. Proposition 4.2 and Corollary 4.5 in  for the proof of equivalence ):
1. The CVaR Norm plus quadratic regularization penalty is minimized. The CVaR Norm is calculated as the superposition of the CVaR Risk and the absolute value of regression residuals (i.e., the absolute value of the standard PSG linear losses).
2. The Partial Moment function plus quadratic regularization penalty is minimized.
3. Convex combination of two CVaR deviations plus quadratic regularization penalty is minimized. The intercept is calculated as the average of two corresponding symmetric quantiles.
4. QP problem (dual to the CVaR Norm plus quadratic regularization penalty minimization) is solved. The solution to the primal problem is constructed.

References

 Malandii, A. and S. Uryasev. Support Vector Regression: Risk Quadrangle Framework, Jan 2023, arXiv:2212.09178