# CS 598CSC: Approximation Algorithms Lecture date: 1/28/11 Instructor: Chandra Chekuri ```CS 598CSC: Approximation Algorithms
Instructor: Chandra Chekuri
Lecture date: 1/28/11
Scribe: Lewis Tseng (2009)
Notes edited by instructor in 2011.
We introduce the use linear programming (LP) in the design and analysis of approximation
algorithms.The topics include Vertex Cover, Set Cover, randomized rounding, dual-fitting. It is
assumed that the students have some background knowledge in basics of linear programming.
1
Vertex Cover via LP
Let G = (V, E) be an undirected graph with arc weights w : V → R+ . Recall the vertex cover
problem from previous lecture. We can formulate it as an integer linear programming problem as
follows. For each vertex v we have a variable xv . We interpret the variable as follows: if xv = 1
if v is chosen to be included in a vertex cover, otherwise xv = 0. With this interprtation we can
easily see that the minimum weight vertex cover can be formulated as the following integer linear
program.
X
min
wv x v
v∈V
subject to
xu + xv ≥ 1
xv
∈ {0, 1}
∀e = (u, v) ∈ E
∀v ∈ V
However, solving integer linear programs is NP-Hard. Therefore we use Linear Programming
(LP) to approximate the optimal solution, OPT(I), for the integer program. First, we can relax
the constraint xv ∈ {0, 1} to xv ∈ [0, 1]. It can be further simplified to xv ≥ 0, ∀v ∈ V .
Thus, a linear programming formulation for Vertex Cover is:
X
min
wv x v
v∈V
subject to
xu + xv ≥ 1
∀e = (u, v) ∈ E
xv ≥ 0
We now use the following algorithm:
Vertex Cover via LP:
Solve LP to obtain an optimal fractional solution x∗
Let S = {v | x∗v ≥ 12 }
Output S
Then the following claims are true:
Claim 1 S is a vertex cover.
Proof: Consider any edge, e = (u, v). By feasibility of x∗ , x∗u + x∗v ≥ 1, and thus either x∗u ≥
x∗v ≥ 21 . Therefore, at least one of u and v will be in S.
Claim 2 w(S) ≤ 2OPTLP (I).
P
Proof: OPTLP (I) = v wv x∗v ≥
Therefore, OPTLP (I) ≥
1
2
OPT(I)
2
P
v∈S
wv = 12 w(S)
1
2
or
2
2
for all instances I.
Note: For minimization problems: OPTLP (I) ≤ OPT(I), where OPTLP (I) is the optimal solution found by LP; for maximization problems, OPTLP (I) ≥ OPT(I).
Integrality Gap
We introduce the notion of integrality gap to show the best approximation guarantee we can acquire
by using the LP optimum as a lower bound.
Definition: For a minimization problem Π, the integrality gap for a linear programming relaxOPT(I)
.
ation/formulation LP for Π is supI∈π OPT
LP (I)
That is, the integrality gap is the worst case ratio, over all instances I of Π, of the integral
optimal value and the fractional optimal value. Note that different linear programming formulations
for the same problem may have different integrality gaps.
Claims 1 and 2 show that the integrality gap of the Vertex Cover LP formulation above is at
most 2.
Question: Is this bound tight for the Vertex Cover LP?
Consider the following example: Take a complete graph, Kn , with n vertices, and each vertex
has wv = 1. It is clear that we have to choose n − 1 vertices to cover all the edges. Thus,
OPT(Kn ) = n − 1. However, xv = 12 for each v is a feasible solution to the LP, which has a total
weight of n2 . So gap is 2 − n1 , which tends to 2 as n → ∞.
Other Results on Vertex Cover
1
1. The current best approximation ratio for Vertex Cover is 2 − Θ( √log
) .
n
2. Open problem: obtain a 2 − ε approximation or to prove that it is NP-hard to obtain 2 − ε
for any fixed ε > 0. Current best hardness of approximation: unless P=NP, there is no 1.36
approximation for Vertex Cover .
3. The vertex cover problem can be solved optimally in polynomial time for bipartite graphs.
This follows from what is known as K´’onig’s theorem.
4. The vertex cover problem admits a polynomial time approximation scheme (PTAS), that is
a (1 + )-approximation for any fixed > 0, for planar graphs. This follows from a general
approach due to Baker [?].
2
Set Cover via LP
The input to the Set Cover problem consists of a finite set U = {1, 2, ..., n}, and m subsets of U ,
S1 , S2 , ..., Sn . Each set Sj has a non-negative weigh wj and the goal is to find the minimum weight
collection of sets which cover all elements in U (in other words their union is U ).
A linear programming relaxation for Set Cover is:
X
min
wj x j
j
subject to
X
xj ≥ 1
∀i ∈ {1, 2, ..., n}
j:i∈Sj
xj
≥ 0
1≤j≤m
And its dual is:
max
n
X
yi
i=1
subject to
X
yi ≤ wj
∀i ∈ {1, 2, ..., n}
i∈Sj
yi ≥ 0
∀i ∈ 1, 2, ..., n
We give several algorithms for Set Cover based on this primal/dual pair LPs.
2.1
Deterministic Rounding
Set Cover via LP:
Solve LP to obtain an optimal solution x∗ , which contains fractional numbers.
Let P = {i | x∗i ≥ f1 }, where f is the maximum number of sets that contain any element
Output {Sj | j ∈ P }
Note that the above algorithm, even when specialized to vertex cover, is different from the one
we saw earlier. It includes all sets which have a strictly positive value in an optimum solution the
LP.
∗ be an optimum solution to the dual, and let
Let x∗ be an optimal solution to the primal LP, yP
P
P = {j | x∗j > 0}. First, note that by strong duality, j wj x∗j = i yi∗ . Second,
P by complementary
slackness if x∗j > 0 then the corresponding dual constraint is tight, that is i∈Sj yi∗ = wj .
Claim 3 The output of the algorithm is a feasible set cover for the given instance.
2
Proof: Exercise.
Claim 4
P
j∈P
wj ≤ f
P
j
wj x∗j = OPTLP .
Proof:

X
j∈P
wj =
X
j:x∗j >0
(wj ) =
X

X

j:x∗j >0
yi∗  =
i∈Sj

X

X
yi∗ 
1 ≤ f
X
j:i∈Sj ,x∗j >0
i
yi∗ ≤ f OPTLP (I).
i
2
.
Notice that the the second equality is due to complementary slackness conditions (if xj > 0, the
corresponding dual constraint is tight), the penultimate inequality uses the definition of f , and the
last inequality follows from weak duality (a feasible solution for the dual problem is a lower bound
on the optimal primal solution).
Therefore we have that the algorithm outputs a cover of weight at most f OPTLP . We note
that f can be as large as n in which case the bound given by the algorithm is quite weak. In fact,
it is not construct examples that demonstrate the tightness of the analysis.
Remark: The analysis cruically uses the fact that x∗ is an optimal solution. On the other hand
the algorithm for vertex cover is more robust.
2.2
Randomized Rounding
Now we describe a different rounding that yields an approximation bound that does not depend on
f.
Solving Set Cover via Randomized Rounding:
A = ∅, and let x∗ be an optimal solution to the LP.
for k = 1 to 2 ln n do
pick each Sj independently with probability x∗j
if Sj is picked, A = A ∪ {j}
end for
Output the sets with indices in A
Claim 5 P r[i is not covered in an iteration] =
Q
j:i∈Sj
(1 − x∗j ) ≤ 1e .
P
Intuition: We know that j:i∈Sj x∗j ≥ 1. Subject to this constraint, if and want to minimize
the probability, we can let x∗j equal to each other, then the probability = (1 − k1 )k , where x∗j = 1/k.
Proof: P r[i is not covered in an iteration] =
Q
∗
j:i∈Sj (1 − xj ) ≤
Q
∗
−xj
≤e
j:i∈Sj e
−
P
j:i∈Sj
x∗j
≤ 1e .
2
We then obtain the following corollaries:
Corollary: P r[i is not covered at the end of the algorithm] ≤ e−2 log n ≤
1
.
n2
Corollary: P r[all elements are covered, after the algorithm stops] ≥ 1 − n1 .
The above follows
from the union bound. The probability that i is not covered is at most 1/n2 , hence the probability
that there is some i that is not covered is at most n · 1/n2 ≤ 1/n.
P
wj x∗j , where E[X] denotes the
Let Ct = cost of sets picked in iteration t, then E[Ct ] = m
P2 ln n j=1
P ln n
expectation of a random variable X. Then, let C = t=1 Ct ; we have E[C] = 2t=1
E[Ct ] ≤
1
2 ln nOPTLP . We know that P r[C > 2E[C]] ≤ 2 by Markov’s inequality, so we have P r[C ≤
4 ln nOPTLP ] ≥ 12 . Therefore, P r[C ≤ 4 ln nOPTLP and all items are covered] ≥ 21 − n1 . Thus,
the randomized rounding algorithm, with probability close to 1/2 succeeds in giving a feasible
solution of cost O(log n)OPTLP . Note that we can check whether the solution satisfies the desired
properties (feasibility and cost) and repeat the algorithm if it does not.
1. We can check if solution after rounding satisfies the desired properties, such as all elements
are covered, or cost at most 2c log nOPTLP . If not, repeat rounding. Expected number of
iterations to succeed is a constant.
2. We can also use Chernoff bounds (large deviation bounds) to show that a single rounding
1
succeeds with high probability (probability at least 1 − poly(n)
).
3. The algorithm can be derandomized. Derandomization is a technique of removing randomness
or using as little randomness as possible. There are many derandomization techniques, such
as the method of conditional expectation, discrepancy theory, and expander graphs. Broder
et al.  use min-wise independent permutations to derandomize the RNC algorithm for
approximate set cover due to S. Rajagopalan and V. Vazirani .
4. After a few rounds, select the cheapest set that covers each uncovered element. This has low
expected cost. This algorithm ensures feasibility but guarantees cost only in the expected
sense.
Other Results related to Set Cover
1. Unless P = NP, there is no c log n approximation for some fixed c .
2. Unless NP ⊆ DT IM E(nO(log log n) ), there is no (1 − o(1)) ln n-approximation .
3. Unless P = NP, there is no (1 −
2.3
1
e
+ ε)-approximation for max-coverage for any fixed ε > 0.
Dual-fitting
In this section, we introduce the technique of dual-fitting for the analysis of approximation algorithms. At a high-level the approach is the following:
1. Construct a feasible solution to the dual LP.
2. Show that the cost of the solution returned by the algorithm can be bounded in terms of the
value of the dual solution.
Note that the algorithm itself need not be LP based. Here, we use Set Cover as an example.
Please refer to the previous section for the primal and dual LP formulations of Set Cover.
We can interpret the dual as follows: Think of yi as how much element i is willing to pay to
be covered; the dual maximizes the total payment, subject to the constraint that for each set, the
total payment of elements in that set is at most the cost of the set.
The greedy algorithm for weighted Set Cover is as follows:
Greedy Set Cover:
Covered = ∅;
A = ∅;
While Covered 6= U do
k
);
j ← arg mink ( |Sk ∩ Uwncovered|
Covered = Covered ∪ Sj ;
A = A ∪ {j}.
end while;
Output sets in A as cover
Theorem 6 Greedy Set Cover picks a solution of cost ≤ Hd ·OPTLP , where d is the maximum
set size, i.e., d = maxj |Sj |.
To prove this, we can augment the algorithm a little bit:
Augmented Greedy Algorithm of weighted Set Cover:
Covered = ∅;
while Covered 6= U do
k
j ← arg mink ( |Sk ∩ Uwncovered|
);
wj
if i is uncovered and i ∈ Sj , set pi = |Sj ∩ U ncovered|
;
Covered = Covered ∪ Sj ;
A = A ∪ {j}.
end while;
Output sets in A as cover
It is easy to see that the algorithm outputs a set cover.
P
P
Claim 7
j∈A wj =
i pi .
Proof: Consider when j is added to A. Let Sj0 ⊆ Sj be the elements
P that are uncovered before j
0
0
is added. For each i ∈ Sj the algorithm sets pi = wj /|Sj |. Hence, i∈S 0 pi = wj . Moreover, it is
j
easy to see that the sets Sj0 , j ∈ A are disjoint and together partition U . Therefore,
X
XX
X
wj =
pi =
pi .
j∈A
For each i, let
yi0
=
1
Hd pi
j∈A i∈Sj0
i∈U
2
.
Claim 8 y 0 is a feasible solution for the dual LP.
P
P
Suppose the claim is true, then the cost of Greedy Set Cover’s solution = i pi = Hd i yi0 ≤
Hd OPTLP . The last step is because any feasible solution for the dual problem is a lower bound
on the value of the primal LP (weak duality).
Now, we prove the claim. Let Sj be an arbitrary set, and let |Sj | = t ≤ d. Let Sj = {i1 , i2 , ..., it },
where we the elements are ordered such that i1 is covered by Greedy no-later than i2 , and i2 is
covered no later than i3 and so on.
Claim 9 For 1 ≤ h ≤ t, pih ≤
wj
t−h+1 .
Proof: Let Sj 0 be the set that covers ih in Greedy. When Greedy picked Sj 0 the elements
ih , ih+1 , . . . , it from Sj were uncovered and hence Greedy could have picked Sj as well. This implies
wj
that the density of Sj 0 when it was picked was no more than t−h+1
. Therefore pih which is set to
wj
the density of Sj 0 is at most t−h+1 .
2
From the above claim, we have
X
X
wj
= wj Ht ≤ wj Hd .
p ih ≤
t−h+1
1≤h≤t
1≤h≤t
Thus, the setting of yi0 to be pi scaled down by a factor of Hd gives a feasible solution.
References
 G. Karakostas. A better approximation ratio for the Vertex Cover problem. ECCC Report
TR04-084, 2004.
 I. Dinur and S. Safra. The importance of being biased. Proceedings of the 34th Annual ACM
Symposium on Theory of Computing, pages 33-42, 2002.
 U. Feige. A Threshold of ln n for Approximating Set Cover. Journal of the ACM , v.45 n.4,
p.634 - 652, 1998.
 R. Raz and M. Safra. A sub-constant error-probability low-degree test, and a sub-constant
error-probability PCP characterization of NP. Proceedings of STOC 1997, pp. 475-484, 1997.
 A. Z. Broder, M. Charikar, and M. Mitzenmacher. A derandomization using min-wise independent permutations Journal of Discrete Algorithms, Volume 1, Issue 1, pages 11-20, 2003.
 S. Rajagopalan and V. Vazirani. Primal-Dual RNC Approximation Algorithms for Set Cover
and Covering Integer Programs SIAM Journal on Computing, Volume 28, Issue 2, p.525 - 540,
1999.
```