Revised simplex algorithm¶
Part 4 · Matrix methods, sensitivity and geometry · Chapter 11 · Lecture notes, volume 4 (PDF) · Exercises: Revised simplex algorithm
Example 11.1: an LP problem in canonical form and its standard form
We consider the following LP problem in canonical maximization form with \(n=2\) variables and \(m=3\) constraints:
Its equivalent standard form with \(n+m=5\) variables and \(m=3\) constraints is:
For this LP problem in standard form, we have:
Since \(\begin{pmatrix} 12 \\[1ex] 12 \\[1ex] 8 \end{pmatrix} \ge \begin{pmatrix} 0 \\[1ex] 0 \\[1ex] 0 \end{pmatrix}\), with \(\pazocal{B}=\{3,4,5\}\) and \(\pazocal{N}=\{1,2\}\) we have the following initial basic feasible solution:
11.1 Pivoting operations with two systems of linear equations¶
-
Given a basis \({\boldsymbol A}_{\pazocal{B}}\), all the information necessary for the pivoting operations can be determined by solving two systems of linear equations.
-
1. First system of linear equations
-
The reduced costs and the basic feasible solution value can be determined by computing directly \(\red{{{\boldsymbol c}'_{\pazocal{B}} \; {\boldsymbol A}_{\pazocal{B}}^{-1}}}\) since:
\[ {{ \bar{\boldsymbol c}}'} = {{\boldsymbol c}'_{\pazocal{N}} - \red{{{\boldsymbol c}'_{\pazocal{B}} \; {\boldsymbol A}_{\pazocal{B}}^{-1}}} \;{\boldsymbol A}_{\pazocal{N}}}~~~~~ \text{and}~~~~~ {\bar{z}} = \red{{{\boldsymbol c}'_{\pazocal{B}} \; {\boldsymbol A}_{\pazocal{B}}^{-1}}} \; {\boldsymbol b} \]By setting \({\boldsymbol \pi} = {{{\boldsymbol c}'_{\pazocal{B}} \; {\boldsymbol A}_{\pazocal{B}}^{-1}}}\), the row vector \({\boldsymbol \pi} \in \R^{1 \times m}\) of \(m\) columns can be found by solving the following system of linear equations:
\[ {\boldsymbol \pi} \; {\boldsymbol A}_{\pazocal{B}} = {\boldsymbol c}'_{\pazocal{B}} ~~~\Longrightarrow~~~ {\boldsymbol \pi} = \red{{{\boldsymbol c}'_{\pazocal{B}} \; {\boldsymbol A}_{\pazocal{B}}^{-1}}} \text{ where } {\boldsymbol \pi} = [\pi_{j}] \text{ with } j \in \pazocal{B} \]In the examples the components of \({\boldsymbol \pi}\) are labelled with the indices \(j \in \pazocal{B}\) of the basic variables, but each component of \({\boldsymbol \pi}\) corresponds to a constraint, i.e., to a row of \({\boldsymbol A}_{\pazocal{B}}\): the \(i\)-th component of \({\boldsymbol \pi}\) is associated with the \(i\)-th constraint.
-
-
2. Second system of linear equations
-
Given a pivot column \(x_p\) with \(p \in \pazocal{N}\), the pivot row and the update of the current basic feasible solution can be done by directly computing \(\violet{{\boldsymbol A}_{\pazocal{B}}^{-1} \;{\boldsymbol A}_{p}}\) since:
\[ {\boldsymbol x}_{\pazocal{B}} = {{\bar{\boldsymbol b}}} - \violet{{{\boldsymbol A}_{\pazocal{B}}^{-1} \;{\boldsymbol A}_{p}}} \; x_{p} \]By setting \({\boldsymbol d} = {{ {\boldsymbol A}_{\pazocal{B}}^{-1}} {\boldsymbol A}_{p}}\), the column vector \({\boldsymbol d} \in \R^{m \times 1}\) of \(m\) rows (associated with the basic variables) can be found by solving the following system of linear equations:
\[ {\boldsymbol A}_{\pazocal{B}} \; {\boldsymbol d} = {\boldsymbol A}_{p} ~~~\Longrightarrow~~~{\boldsymbol d} = \violet{{ {\boldsymbol A}_{\pazocal{B}}^{-1}} {\boldsymbol A}_{p}} \text{ where } {\boldsymbol d} = [d_{i}] \text{ with } i \in \pazocal{B} \]
-
Example 11.2: Systems of linear equations
For the LP problem of Example 11.1 with \(\pazocal{B}= \{1,~~3,~~4\}\), \(\pazocal{N}= \{2,~~5\}\) and \(x_2\) as the pivot column, we have:
We solve \({\boldsymbol \pi} \; {\boldsymbol A}_{\pazocal{B}} = {\boldsymbol c}'_{\pazocal{B}}\) to compute \({\boldsymbol \pi} = {{{\boldsymbol c}'_{\pazocal{B}} \; {\boldsymbol A}_{\pazocal{B}}^{-1}}}\), first system of linear equations:
We solve \({\boldsymbol A}_{\pazocal{B}} \; {\boldsymbol d} = {\boldsymbol A}_{2}\) to compute \({\boldsymbol d}={ {\boldsymbol A}_{\pazocal{B}}^{-1}} {\boldsymbol A}_{2}\), second system of linear equations:
The revised simplex algorithm never computes the inverse matrix \({\boldsymbol A}_{\pazocal{B}}^{-1}\): at every iteration it solves two systems of linear equations, one for the row vector \({\boldsymbol \pi}\) and one for the column vector \({\boldsymbol d}\). Solving a system of \(m\) linear equations is computationally cheaper than inverting a matrix of size \(m\), and the matrix \({\boldsymbol A}_{\pazocal{B}}\) of the two systems is the same.
11.2 Pseudo-code of the revised simplex algorithm¶
Algorithm
-
Step 0: Initialization
Find a basic feasible solution \({\boldsymbol x}_{\pazocal{B}}={{\bar{\boldsymbol b}}}\) and \({\boldsymbol x}_{\pazocal{N}}={{{\boldsymbol 0}}}\) of the LP problem in standard maximization form. Go to Step \(1\).
-
Step 1: Optimality check
-
a. Solve \({\boldsymbol \pi} \; {\boldsymbol A}_{\pazocal{B}} = {\boldsymbol c}'_{\pazocal{B}}\) to compute \({\boldsymbol \pi} = {{{\boldsymbol c}'_{\pazocal{B}} \; {\boldsymbol A}_{\pazocal{B}}^{-1}}}\)
-
b. Compute the reduced costs and the basic feasible solution value:
\[ {{ \bar{\boldsymbol c}}}' = {{\boldsymbol c}'_{\pazocal{N}} - \underbrace{{{\boldsymbol c}'_{\pazocal{B}} \; {\boldsymbol A}_{\pazocal{B}}^{-1}}}_{={\boldsymbol \pi}} \;{\boldsymbol A}_{\pazocal{N}}} \qquad \qquad {\bar{z}} = \underbrace{{{\boldsymbol c}'_{\pazocal{B}} \; {\boldsymbol A}_{\pazocal{B}}^{-1}}}_{={\boldsymbol \pi}} \; {\boldsymbol b} \]
If \({{ \bar{\boldsymbol c}}'} \le {\boldsymbol 0}\) then the basic feasible solution \({\boldsymbol x}_{\pazocal{B}}={{\bar{\boldsymbol b}}}\) and \({\boldsymbol x}_{\pazocal{N}}={{{\boldsymbol 0}}}\) is an optimal solution and \({\bar{z}}\) is the optimal value else go to Step \(2\).
-
-
Step 2: Pivoting operations
The pivot column is a non-basic variable \(x_p\) with \(p \in \pazocal{N}\) such that:
\[ {c}_{p} >\underbrace{{{\boldsymbol c}'_{\pazocal{B}} \; {\boldsymbol A}_{\pazocal{B}}^{-1}}}_{={\boldsymbol \pi}} \;{\boldsymbol A}_{p} \]-
a. Solve \({\boldsymbol A}_{\pazocal{B}} \; {\boldsymbol d} = {\boldsymbol A}_{p}\) to compute \({\boldsymbol d}={ {\boldsymbol A}_{\pazocal{B}}^{-1}} {\boldsymbol A}_{p}\)
-
b. If \({\boldsymbol d} \le {\boldsymbol 0}\), then the LP problem is unbounded.
STOP. -
c. Find the largest value \(t\) (called \(t_m\)) such that:
\[ {{\bar{\boldsymbol b}}} - t \; \underbrace{ {\boldsymbol A}_{\pazocal{B}}^{-1} {\boldsymbol A}_{p}}_{={\boldsymbol d}} \ge 0 \qquad \Longrightarrow \qquad t_m \leftarrow \min \left\{~\frac{\bar b_i}{d_i}: i\in\pazocal{B},~d_i>0 ~\right\} \]
The pivot row is a basic variable \(x_r\) with \(r \in \pazocal{B}\) such that:
\[ \bar{b}_r - t_m \; \underbrace{\bar{a}_{rp}}_{=d_r} = 0 \]-
d. Update the basis with the new partition of the indices of the variables:
\[ \pazocal{B} \leftarrow \big(\pazocal{B} \setminus \{ r \} \big) \cup \{ p \} \text{ and } \pazocal{N} \leftarrow \big(\pazocal{N} \setminus \{ p \} \big) \cup \{ r \} \] -
e. Update the basic feasible solution (new vector \({{\bar{\boldsymbol b}}}\)):
\[ x_i = \bar{b}_i - t_m \; \underbrace{\bar{a}_{ip}}_{=d_i},~\forall i \in \pazocal{B} \setminus \{p\},~~ x_p = t_m \text{ and } x_i = 0,~\forall i \in \pazocal{N} \]
Go to Step \(1\).
-
Example 11.3: revised simplex algorithm – first iteration
Initialization
In the LP problem of Example 11.1, \(\begin{pmatrix} x_3 \\ x_4 \\ x_5 \\ \end{pmatrix} \!=\! \begin{pmatrix} 12 \\ 12 \\ 8 \\ \end{pmatrix} ~ \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} \!=\! \begin{pmatrix} 0 \\ 0 \end{pmatrix}\) is a basic feasible solution with \(\bar{\boldsymbol b} \!=\! \begin{pmatrix} 12 \\ 12 \\ 8 \\ \end{pmatrix}\).
Optimality check
With \(\pazocal{B}=\{3,4,5\}\) and \(\pazocal{N}=\{1,2\}\), we solve \({\boldsymbol \pi} \; {\boldsymbol A}_{\pazocal{B}} = {\boldsymbol c}'_{\pazocal{B}}\) to compute \({\boldsymbol \pi} = {{{\boldsymbol c}'_{\pazocal{B}} \; {\boldsymbol A}_{\pazocal{B}}^{-1}}}\).
We compute the basic feasible solution value \({\bar{z} } = {\boldsymbol \pi} \; {\boldsymbol b} = \begin{pmatrix} 0 & 0 & 0 \\ \end{pmatrix} \begin{pmatrix} 12 \\ 12 \\ 8 \\ \end{pmatrix} = 0.\) We compute the reduced costs \({{ \bar{\boldsymbol c} }'} = {{\boldsymbol c}'_{\pazocal{N}} - {\boldsymbol \pi} \; {\boldsymbol A}_{\pazocal{N}}}\):
Pivoting operation
Since \({c}_{1}>{\boldsymbol \pi} \;{\boldsymbol A}_{1}\), \(x_1\) is the pivot column. We solve \({\boldsymbol A}_{\pazocal{B}} \; {\boldsymbol d} = {\boldsymbol A}_{1}\) to compute \({\boldsymbol d}={ {\boldsymbol A}_{\pazocal{B}}^{-1}} {\boldsymbol A}_{1}\).
We have to find the largest \(t\) such that \({{ \bar{\boldsymbol b} }} - t \; {{\boldsymbol d }} \ge 0\):
The value of the pivot column becomes \(t_{m}\), i.e., \(x_1=\frac{8}{3}\). The value of \(x_5\) becomes zero and \(x_5\) is the pivot row. The other basic variables take values \({ \bar{\boldsymbol b} } - t_{m} \; {\boldsymbol d}\), then we have:
The new partition of the indices of the variables is \(\pazocal{B}=\{1,3,4\}\) and \(\pazocal{N}=\{2,5\}\).
Example 11.4: revised simplex algorithm – second iteration
Optimality check
With \(\pazocal{B}=\{1,3,4\}\) and \(\pazocal{N}=\{2,5\}\), we solve \({\boldsymbol \pi} \; {\boldsymbol A}_{\pazocal{B}} = {\boldsymbol c}'_{\pazocal{B}}\) to compute \({\boldsymbol \pi} = {{{\boldsymbol c}'_{\pazocal{B}} \; {\boldsymbol A}_{\pazocal{B}}^{-1}}}\).
We compute the basic feasible solution value \({\bar{z} } = {\boldsymbol \pi} \; {\boldsymbol b} = \begin{pmatrix} 0& 0 & \frac{5}{3} \\ \end{pmatrix} \begin{pmatrix} 12 \\ 12 \\ 8 \\ \end{pmatrix} = \frac{40}{3}.\) We compute the reduced costs \({{ \bar{\boldsymbol c} }'} = {{\boldsymbol c}'_{\pazocal{N}} - {\boldsymbol \pi} \; {\boldsymbol A}_{\pazocal{N}}}\):
Pivoting operation
Since \({c}_{2} > {\boldsymbol \pi} \;{\boldsymbol A}_{2}\), \(x_2\) is the pivot column. We solve \({\boldsymbol A}_{\pazocal{B}} \; {\boldsymbol d} = {\boldsymbol A}_{2}\) to compute \({\boldsymbol d}={ {\boldsymbol A}_{\pazocal{B}}^{-1}} {\boldsymbol A}_{2}\).
We have to find the largest \(t\) such that \({{ \bar{\boldsymbol b} }} - t \; {{\boldsymbol d}} \ge 0\):
The value of the pivot column becomes \(t_{m}\), i.e., \(x_2=\frac{7}{2}\). The value of \(x_4\) becomes zero and \(x_4\) is the pivot row. The other basic variables take values \({ \bar{\boldsymbol b} } - t_{m} \; {\boldsymbol d}\), then we have:
The new partition of the indices of the variables is \(\pazocal{B}=\{1,2,3\}\) and \(\pazocal{N}=\{4,5\}\).
Example 11.5: revised simplex algorithm – third iteration
Optimality check
With \(\pazocal{B}=\{1,2,3\}\) and \(\pazocal{N}=\{4,5\}\), we solve \({\boldsymbol \pi} \; {\boldsymbol A}_{\pazocal{B}} = {\boldsymbol c}'_{\pazocal{B}}\) to compute \({\boldsymbol \pi} = {{{\boldsymbol c}'_{\pazocal{B}} \; {\boldsymbol A}_{\pazocal{B}}^{-1}}}\).
We compute the basic feasible solution value \({\bar{z} } = {\boldsymbol \pi} \; {\boldsymbol b} = \begin{pmatrix} 0 & 2 &1 \\ \end{pmatrix} \begin{pmatrix} 12 \\ 12 \\ 8 \\ \end{pmatrix} = 32\)
We compute the reduced costs \({{ \bar{\boldsymbol c} }'} = {{\boldsymbol c}'_{\pazocal{N}} - {\boldsymbol \pi} \; {\boldsymbol A}_{\pazocal{N}}}\):
All reduced costs are non-positive, then the basic feasible solution
is an optimal solution of the LP problem and \({\bar{z}} = 32\) is the optimal value.
Exercises on this chapter: Revised simplex algorithm