Robert C. Schrag
Abstract
Introduction
While CNF propositional satisfiability (SAT) is a specific
kind constraint satisfaction problem (CSP), until recently
there has been little application of popular CSP look-back
techniques in SAT algorithms. In previous work [Bayardo
& Schrag 96] we demonstrated that a look-back-enhanced
version of the Tableau algorithm for 3SAT instances [Crawford and Auton 96] can solve easily many instances which
without look-back are exceptionally hard -- orders of
magnitude harder than other instances with the same surface characteristics. In this work the instances were artificially generated. Here, we demonstrate the practical utility
of CSP look-back techniques by using a look-backenhanced algorithm related to Tableau to solve large SAT
instances derived from real-world problems in planning,
scheduling, and circuit diagnosis and synthesis. Kautz and
Selman [96] had found unenhanced Tableau inadequate to
solve several planning-derived instances and resorted to
using a stochastic algorithm, WSAT (also known as WalkSAT) [Selman et al. 94]; our results show that look-back
enhancements make this recourse unnecessary.
Given the usual framework of backtrack search for systematic solution of the finite-domained constraint satisfaction problem (CSP), techniques intended to improve
efficiency can be divided into two classes: look-ahead techniques, which exploit information about the remaining
search space, and look-back techniques, which exploit
information about search which has already taken place.
The former class includes variable ordering heuristics,
value ordering heuristics, and dynamic consistency enforceCopyright 1997, American Association for Artificial Intelligence
(www.aaai.org). All rights reserved.
Definitions
A propositional logic variable ranges over the domain
{ true, false } . An assignment is a mapping of these values
to variables. A literal is the occurrence of a variable, e.g. x ,
or its negation, e.g. x ; a positive literal x is satisfied
when the variable x is assigned true, and a negative literal
x is satisfied when x is assigned false. A clause is a simple disjunction of literals, e.g. ( x y z ) ; a clause is satisfied when one or more of its literals is satisfied. A unit
clause contains exactly one variable, and a binary clause
contains exactly two. The empty clause ( ) signals a contradiction (seen in the interpretation, choose one or more
literals to be true from among none). A conjunctive normal formula (CNF) is a conjunction of clauses (e.g.
( a b ) ( x y z ) ); a CNF is satisfied if all of its
clauses are satisfied.
For a given CNF, we represent an assignment notationally as a set of literals each of which is satisfied. A nogood
is a partial assignment which will not satisfy a given CNF.
The clause ( a b c ) encodes the nogood { a, b, c } .
We call such a nogood-encoding clause a reason. Resolution is the operation of combining two input clauses mentioning a given literal and its negation, respectively,
deriving an implied clause which mentions all other literals
besides these. For example, ( a b ) resolves with
( b c ) to produce ( a c ) .
vars
2,069
2,809
828
843
1,141
1,729
2,353
3,016
6,325
clauses
29,508
48,920
6,718
7,301
10,719
21,943
37,121
50,457
131,973
sat
Y
Y
Y
Y
Y
N
N
Y
Y
type
planning
planning
planning
planning
planning
planning
planning
planning
planning
vars
986
2,180
718
1931
clauses
2,315
6,778
4,932
14,468
sat
N
N
Y
Y
type
diagnosis
diagnosis
planning
planning
Experimental Methodology
Test Suites
We use three separate test suites to compare the performance of look-back-enhanced DP with other algorithms
whose performance has been reported for the same
instances: SAT-encoded planning instances from Kautz and
Selman1; selected circuit diagnosis and planning instances
from the DIMACS Challenge directory associated with the
1993 SAT competition2; and planning, scheduling, and cir-
vars
19,500
19,500
20,700
20,700
21,800
22,500
283
410
758
590
649
708
125
252
8,432
8,704
clauses
108,887
104,527
111,567
113,729
118,607
123,329
9,690
24,758
47,820
1,422
1,562
1,702
310
766
31,310
32,316
sat
Y
Y
Y
Y
Y
Y
Y
Y
Y
N
Y
Y
Y
Y
Y
Y
type
scheduling
scheduling
scheduling
scheduling
scheduling
scheduling
planning
planning
planning
synthesis
synthesis
synthesis
synthesis
synthesis
synthesis
synthesis
Experimental Results
Table 4 displays performance data for relsat(4), WSAT, and
ntab_back on Kautz and Selmans planning instances. Cutoff time was 10 minutes for each instance except bw_dir.d,
for which it was 30 minutes. The times for WSAT are those
reported by Kautz and Selman [96], normalized to SPARC10 CPU seconds. Relsat(4) outperformed WSAT on most
instances. One exception where WSAT is clearly superior is
on instance log_dir.c which caused relsat(4) to reach cutoff
22 times. Instance bw_dir.d caused relsat(4) to reach cutoff
18 times, but it still outperformed WSAT by several minutes even after averaging in 30 minutes for each relsat cutoff. Though it is difficult to draw solid conclusions about
the performance of ntab_back since the times reported are
only for a single run, we can determine that relsat(4) is
more effective than ntab_back on the instances for which
relsat(4) never reached cutoff, yet ntab_back required substantially more than 10 minutes to solve. This includes all
log_gp and log_un instances.
TABLE 4. Performance of relsat(4) on Kautz and Selmans
planning instances.
instance
log_gp.b
log_gp.c
log_dir.a
log_dir.b
log_dir.c
log_un.b
log_un.c
bw_dir.c
bw_dir.d
relsat(4)
12.9
39.4
4.1
16.6
90.3
66.8
192.5
119
813.3
% fail
0%
0%
0%
0%
22%
0%
0%
0%
18%
WSAT
75.2
419.2
4.3
2.6
3.0
--1072
1499
ntab_back
2,621
11,144
369.7
161.4
> 12 hours
12,225
> 12 hours
16.9
> 12 hours
run-time
-115
2.6
.5
3.6
.6
assgnmnts
-999,555
18,754
3,914
23,107
4,391
% fail
100%
0%
0%
0%
0%
0%
run-time
-415
242
278
71
46
assgnmnts
-9.4 Million
5.0 Million
4.7 Million
1.2 Million
.62 Million
% fail
100%
23%
0%
4%
0%
0%
run-time
-325
214
227
254
183
assgnmnts
-3.5 Million
1.7 Million
1.4 Million
1.6 Million
.89 Million
% fail
100%
94%
92%
79%
13%
1%
6. Freeman also reports that POSIT exhibits high run-time variability on ssa2670-141, though the variance is not quantified.
Discussion
Look-back enhancements clearly make DP a more capable
algorithm. For almost every one of the instances tested here
(selected for their difficulty), learning and CBJ were critical
for good performance. We suspect the dramatic performance improvements resulting from the incorporation of
look-back is in fact due to a synergy between the lookahead and look-back techniques applied. Variable selection
heuristics attempt to seek out the most-constrained areas of
the search space to realize inevitable failures as quickly as
possible. Learning schemes, through the recording of
derived clauses, can create constrained search-sub-spaces
for the variable selection heuristic to exploit.
Size-bounded learning is effective when instances have
relatively many short nogoods which can be derived with-
look-back-enhanced DP should probably replace unenhanced DP; where another SAT algorithm is used, DP
should be given a new evaluation using look-back enhancements. Finally, look-back-enhanced DP should become a
standard algorithm, along with unenhanced DP, against
which other styles of SAT algorithm are compared.
Conclusions
References
Bayardo, R. J. and Miranker, D. P. 1996. A Complexity Analysis
of Space-Bounded Learning Algorithms for the Constraint Satisfaction Problem. In Proc. 13th Natl Conf. on Artificial Intelligence, 558-562.
Bayardo, R. J. and Schrag, R. 1996. Using CSP Look-Back Techniques to Solve Exceptionally Hard SAT Instances. In Proc. Second Intl Conf. on Principles and Practice of Constraint
Programming (Lecture Notes in Computer Science v. 1118),
Springer, 46-60.
Crawford, J. M. and Auton, L. D. 1996. Experimental Results on
the Crossover Point in Random 3SAT. Artificial Intelligence 81(12), 31-57.
Crawford, J. M. and Baker, A. B. 1994. Experimental Results on
the Application of Satisfiability Algorithms to Scheduling Problems. In Proc. Twelfth Natl Conf. on Artificial Intelligence, 10971097.
Davis, M., Logemann, G. and Loveland, D. 1962. A Machine Program for Theorem Proving, CACM 5, 394-397.
Freeman, J. W. 1995. Improvements to Propositional Satisfiability
Search Algorithms. Ph.D. Dissertation, U. Pennsylvania Dept. of
Computer and Information Science.
Frost, D. and Dechter, R. 1994. Dead-End Driven Learning. In
Proc. of the Twelfth Natl Conf. on Artificial Intelligence, 294-300.
Ginsberg, M. and McAllester, D. 1994. GSAT and Dynamic Backtracking, Principles of Knowledge Representation and Reasoning:
Proceedings of the Fourth Intl Conf., 226-237.
Kautz, H. and Selman, B. 1996. Pushing the Envelope: Planning,
Propositional Logic, and Stochastic Search. In Proc. 13th Natl
Conf. on Artificial Intelligence, 558-562.
Mazure, M., Sais, L. and Gregoire, E. 1996. Detecting Logical
Inconsistencies. In Proc. of the Fourth Intl Symposium on Artificial Intelligence and Mathematics, 116-121.
Prosser, P. 1993. Hybrid Algorithms for the Constraint Satisfaction Problem. Computational Intelligence 9(3):268-299.
Selman, B., Kautz, H., and Cohen, B., 1994. Noise Strategies for
Local Search. In Proc. Twelfth Natl Conf. on Artificial Intelligence, 337-343.
Selman, B., Levesque, H. and Mitchell, D. 1992. A New Method
for Solving Hard Satisfiability Problems, In Proc. Tenth Natl
Conf. on Artificial Intelligence, 440-446.
Stallman R. M. and. Sussman G. J., 1977. Forward Reasoning and
Dependency-Directed Backtracking in a System for ComputerAided Circuit Analysis. Artificial Intelligence 9, 135-196.