Back to main page** Case study background and formulation of problems**

Problem 1: Nu-SVR as Error Minimization (primal)

Problem 1: Nu-SVR as Error Minimization (primal)

Minimize (1-alpha)*cvar_risk + quadratic

Value:

var_risk

——————————————————————————————

quadratic = quadratic function specified by a matrix_quadratic

cvar_risk = conditional value-at-risk specified by an abs(matrix of scenarios)

var_risk = value-at-risk specified by an abs(matrix of scenarios)

——————————————————————————————

Data and solution in different environments:

# of Variables |
# of Scenarios |
Objective Value |
Solving Time, PC 2.7GHz (sec) |
||||

Dataset | 1 | 1,000 | 0.7742 | <0.01 | |||
---|---|---|---|---|---|---|---|

Environments |
|||||||

Run-File | Problem Statement | Data | Solution | ||||

Matlab Toolbox | Data | ||||||

Python | Python Code | ||||||

R | R Code |

Problem 2: epsilon-SVR as Error Minimization (primal)

Minimize pm_pen + quadratic

Value:

pr_pen

——————————————————————–——————

pm_pen = partial moment function specified by an abs(matrix of scenarios)

pr_pen = probability of exceedance specified by an abs(matrix of scenarios)

quadratic = quadratic function specified by a matrix_quadratic

——————————————————————–——————

Data and solution in different environments:

# of Variables |
# of Scenarios |
Objective Value |
Solving Time, PC 2.7GHz (sec) |
||||

Dataset | 1 | 1,000 | 0.4082 | <0.01 | |||
---|---|---|---|---|---|---|---|

Environments |
|||||||

Run-File | Problem Statement | Data | Solution | ||||

Matlab Toolbox | Data | ||||||

Python | Python Code | ||||||

R | R Code |

Problem 3: Nu-SVR as Deviation Minimization (primal)

Minimize (1-alpha)/2*cvar_dev + (1+alpha)/2*cvar_dev + quadratic

Value:

var_risk

——————————————————————–——————

cvar_dev = conditional value-at-risk deviation specified by a reduced matrix of scenarios

quadratic = quadratic function specified by a reduced matrix_quadratic

——————————————————————–——————

Data and solution in different environments:

# of Variables |
# of Scenarios |
Objective Value |
Solving Time, PC 2.7GHz (sec) |
||||

Dataset | 1 | 1,000 | 0.7742 | <0.01 | |||
---|---|---|---|---|---|---|---|

Environments |
|||||||

Run-File | Problem Statement | Data | Solution | ||||

Matlab Toolbox | Data | ||||||

Python | Python Code | ||||||

R | R Code |

Problem 4a: Nu-SVR as Error Minimization (dual, constraints linearized automatically)

Maximize linear – 0.5*quadratic

Subject to

polynom_abs <= C*(1-alpha)

linear = 0

Box: <= C, >= -C

——————————————————————–——————

linear = linear fumction specified by a vector of dependent values (in objective) and by a unit vector (in constraint)

polynom_abs = l1 norm specified by a unit vector

quadratic = quadratic function specified by a linear kernel matrix

——————————————————————–——————

Data and solution in different environments:

# of Variables |
# of Scenarios |
Objective Value |
Solving Time, PC 2.7GHz (sec) |
||||

Dataset | 1,000 | 1,000 | 774.2196 | <0.2 | |||
---|---|---|---|---|---|---|---|

Environments |
|||||||

Run-File | Problem Statement | Data | Solution | ||||

Matlab Toolbox | Data | ||||||

Python | Python Code | ||||||

R | R Code |

Problem 4b: Nu-SVR as Error Minimization (dual, constraints linearized manually)

Maximize linear – 0.5*quadratic

Subject to

linear <= C*(1-alpha)

linear = 0

-1<=linearmulti<=1

limearmulti>=0

limearmulti>=0

——————————————————————–——————

linear = linear fumction specified by a vector of dependent values (in objective) and by a unit vector (in constraint)

polynom_abs = l1 norm specified by a unit vector

quadratic = quadratic function specified by a linear kernel matrix

linearmulti = linear multiple function specified by a sparse pmatrix in each case

——————————————————————–——————

Data and solution in different environments:

# of Variables |
# of Scenarios |
Objective Value |
Solving Time, PC 2.7GHz (sec) |
||||

Dataset | 1,000 | 1,000 | 774.2196 | <0.2 | |||
---|---|---|---|---|---|---|---|

Environments |
|||||||

Run-File | Problem Statement | Data | Solution | ||||

Matlab Toolbox | Data | ||||||

Python (GUROBI) | Python Code | ||||||

R | R Code |

**CASE STUDY SUMMARY**

This case study conducts Support Vector Regression (SVR) as regularized error minimization, regularized deviation minimization, and constrained quadratic programming (CQP) problem on simulated data (1000 observations), cf. [1].

Specifically, four equivalent implementations of SVR are considered (cf. Proposition 4.2 and Corollary 4.5 in [1] for the proof of equivalence ):

1. The CVaR Norm plus quadratic regularization penalty is minimized. The CVaR Norm is calculated as the superposition of the CVaR Risk and the absolute value of regression residuals (i.e., the absolute value of the standard PSG linear losses).

2. The Partial Moment function plus quadratic regularization penalty is minimized.

3. Convex combination of two CVaR deviations plus quadratic regularization penalty is minimized. The intercept is calculated as the average of two corresponding symmetric quantiles.

4. QP problem (dual to the CVaR Norm plus quadratic regularization penalty minimization) is solved. The solution to the primal problem is constructed.

References

References

[1] Malandii, A. and S. Uryasev. Support Vector Regression: Risk Quadrangle Framework, Jan 2023,

**arXiv:2212.09178**