Constraint Satisfaction Problems
Topic 04 · Reading

Constraint Satisfaction Problems — Reading Notes

These notes follow the same order as the interactive Constraint Satisfaction Problems lecture — how a problem is written as variables, domains and constraints; how inference shrinks domains; how backtracking search uses ordering heuristics and interleaved inference; how local search repairs complete assignments; and how the structure of the constraint graph can make a hard problem easy. Where the lecture page lets you click through live examples (the map of Australia, AC-3, forward checking, min-conflicts), these notes spell the same ideas out in prose with the same notation and worked numbers. Every number on this page was produced by the same engine that runs the lecture and the Playground.

Materials and content adapted from Artificial Intelligence: A Modern Approach (AIMA), 4th edition, by Stuart Russell and Peter Norvig — specifically Chapter 6 (Constraint Satisfaction Problems, §6.1–6.5).

1.Defining Constraint Satisfaction Problems

In Topics 01–03 every state was atomic: a black box that the search algorithm could only generate, test for the goal, and compare by cost or utility. A constraint satisfaction problem (CSP) opens the box. A state is described by a set of variables, each holding a value, and the problem is solved when every variable has a value that satisfies all of the constraints on it. This factored representation lets general-purpose algorithms use the structure of the problem — which variables interact, and how — to prune huge parts of the search space at once.

1.1Variables, Domains & Constraints

A CSP consists of three components:

X — variables
A set {X1, …, Xn}.
D — domains
One domain Di = {v1, …, vk} of allowable values for each variable Xi.
C — constraints
Each constraint Cj = ⟨scope, rel⟩ names the variables it involves (its scope) and a relation saying which combinations of their values are allowed. The relation can be an explicit list of tuples or an abstract test such as X1 ≠ X2.

The running example is AIMA's map colouring of Australia: colour each state and territory red, green or blue so that no two neighbouring regions share a colour.

X = {WA, NT, Q, NSW, V, SA, T} · Di = {red, green, blue} · C = {SA ≠ WA, SA ≠ NT, SA ≠ Q, SA ≠ NSW, SA ≠ V, WA ≠ NT, NT ≠ Q, Q ≠ NSW, NSW ≠ V}
Western Australia (WA)Northern Territory (NT)Queensland (Q)New South Wales (NSW)Victoria (V)South Australia (SA)Tasmania (T)WANTQNSWVSAT
WANTQNSWVSAT
Figure 1. Left: one solution of the Australia map-colouring CSP. Right: its constraint graph — one node per variable, one edge per binary constraint (AIMA Fig 6.1).

1.2Assignments & Solutions

An assignment gives values to some or all variables, e.g. {WA = red, NT = green}. It is consistent (or legal) if it violates no constraint, and complete if every variable has a value. A solution is a complete, consistent assignment. A partial assignment leaves some variables unassigned; a partial solution is a partial assignment that is consistent.

The key observation for search is that a partial assignment can already be inconsistent. As soon as WA and NT are both red, no extension of that assignment can be a solution, so an algorithm can discard every one of the 35 = 243 ways of completing it in a single step. This is exactly what a black-box state representation cannot do.

1.3The Constraint Graph

A binary CSP is naturally drawn as a constraint graph: a node for each variable, and an edge between two variables whenever a constraint involves both. Structure is visible at a glance. Tasmania has degree 0 — it is an independent subproblem and can be coloured any way at all. South Australia has degree 5 — it is involved in more constraints than any other region, which makes it the natural variable to decide first (§3.2) and the natural variable to remove to break every cycle (§4.5).

1.4Kinds of Constraints

  • Unary constraints restrict a single variable, e.g. ⟨(SA), SA ≠ green⟩.
  • Binary constraints relate two variables, e.g. ⟨(SA, NSW), SA ≠ NSW⟩. A binary CSP has only unary and binary constraints and can be drawn as a constraint graph.
  • Global (higher-order) constraints involve an arbitrary number of variables. The most common is Alldiff, which requires all of its variables to take different values — every row, column and box of a Sudoku. In a cryptarithmetic puzzle such as TWO + TWO = FOUR, Alldiff(F, T, U, W, R, O) is combined with column constraints like O + O = R + 10 · C1 that link letters to auxiliary carry variables C1, C2, C3; these are drawn as a constraint hypergraph.

Every finite-domain constraint can be reduced to a set of binary constraints if enough auxiliary variables are introduced, or by building the dual graph in which constraints become variables. This is why most algorithms in this topic are stated for binary CSPs, although specialised propagation for global constraints (§2.5) is often far more efficient in practice.

1.5Hard vs. Soft Constraints: University Timetabling

So far every constraint has been absolute: violating it rules out a solution. Many real problems also contain preference constraints that say which solutions are better. A CSP with preferences is a constraint optimisation problem (COP): find a solution that satisfies all hard constraints and minimises the total penalty of the violated soft ones.

The Playground's Course Scheduling lab uses the following formulation. Each of the 12 course sessions is a variable; its domain is the set of (timeslot, room) pairs — 3 days × 4 periods × 4 rooms = 48 values.

IdHard constraint (feasibility — 0 clashes allowed)Arity
H1No room double-booking: two courses cannot share a room in the same timeslot.binary
H2Room capacity ≥ enrolment, and room type matches (lab courses in the Computer Lab).unary
H3No instructor double-booking.binary
H4Instructor availability: no assignment in a declared unavailable slot.unary
H5No student-cohort clash: a cohort's mandatory courses cannot overlap.binary
H6Complete assignment: every course is scheduled.goal test
IdSoft constraint (quality)Penalty weight
S1Instructor preferences — penalise slots outside an instructor's preferred times.2 per session
S2Room utilisation — penalise rooms with more than twice the needed seats.1 per session
S3Student dead hours — penalise empty periods between a cohort's classes on a day.2 per gap
S4Fatigue — penalise more than two back-to-back classes for a cohort.3 per extra class
S5Campus transit — penalise consecutive classes in different buildings.1 per transition

A standard two-phase strategy is to find a feasible timetable with backtracking search (H2 and H4 are removed up front by node consistency, §2.1), and then improve the soft score with local search that never breaks a hard constraint (§4.1). In the engine's reference run, backtracking with MRV and forward checking finds a feasible timetable with 12 assignments and no backtracks; local search then lowers the soft penalty from 21 to 3.

A CSP can be solved by the search algorithms of Topic 02 using the incremental formulation:

Initial state
The empty assignment { }.
Actions
Assign a value to an unassigned variable, provided it does not conflict with the current assignment.
Goal test
The assignment is complete (and therefore, by construction, consistent).
Path cost
Irrelevant — every solution appears at depth n, so depth-first search is the natural choice.

Done naïvely, the branching factor at the top level is n · d (any variable, any value), at the next level (n − 1) · d, and so on: a tree with n! · dn leaves, even though there are only dn complete assignments. The fix is commutativity: the order in which values are assigned does not change the result, so it is enough to consider a single variable at each node of the tree. With that restriction the tree has exactly dn leaves. For Australia this is the difference between 7! · 37 = 11,022,480 and 37 = 2,187.

2.Constraint Propagation: Inference in CSPs

A search algorithm for atomic states can only search. A CSP algorithm can also infer: use the constraints to reduce the number of legal values for a variable, which in turn reduces the legal values of its neighbours, and so on. This is constraint propagation. It may be done as a preprocessing step, or interleaved with search (§3.4–3.5). The central idea is local consistency: treat each variable as a node and each binary constraint as an arc, and remove values that cannot be part of any solution because they fail some local test.

2.1Node Consistency

A variable is node-consistent if all the values in its domain satisfy its unary constraints; a network is node-consistent if every variable is. Node consistency is enforced once, by simply deleting offending values, after which all unary constraints can be dropped.

In the timetabling CSP, H2 and H4 are unary. For the ML Lab session AI302L (24 students, a lab, taught by Dr. Okafor who is unavailable on Monday), 36 of the 48 (slot, room) values fail H2 (the three lecture rooms) and 4 fail H4 (the Computer Lab on Monday), leaving a domain of just 8 values before search even begins.

2.2Arc Consistency

A variable Xi is arc-consistent with respect to Xj if for every value in Di there is some value in Dj that satisfies the binary constraint on the arc (Xi, Xj). A network is arc-consistent if every variable is arc-consistent with every other variable. The basic operation is REVISE(Xi, Xj), which deletes every value of Xi that has no support in Dj.

AIMA's example: the constraint Y = X² with X, Y ∈ {0, …, 9}. Making X arc-consistent with Y leaves X ∈ {0, 1, 2, 3} (since 4² = 16 is out of range); making Y arc-consistent with X leaves Y ∈ {0, 1, 4, 9}. Note that arcs are directed — each direction is a separate revision.

Arc consistency does not always help. Colouring Australia with no pre-assignments, every value of every region has a different-coloured supporting value in each neighbour, so arc consistency removes nothing.

2.3The AC-3 Algorithm

AC-3 maintains a queue of arcs. Initially the queue holds every arc in the CSP (each binary constraint contributes two arcs). AC-3 pops an arbitrary arc (Xi, Xj) and revises it. If Di is unchanged it moves on; if Di shrank, every arc (Xk, Xi) that points into Xi must be re-checked and is added to the queue; if Di became empty, the whole CSP has no consistent solution and AC-3 returns failure immediately.

The AC-3 Algorithm (AIMA Figure 6.3)
function AC-3(csp) returns false if an inconsistency is found and true otherwise
    queue ← a queue of arcs, initially all the arcs in csp
    while queue is not empty do
        (Xi, Xj) ← POP(queue)
        if REVISE(csp, Xi, Xj) then
            if size of Di = 0 then return false
            for each Xk in Xi.NEIGHBORS − {Xj} do
                add (Xk, Xi) to queue
    return true

function REVISE(csp, Xi, Xj) returns true iff we revise the domain of Xi
    revised ← false
    for each x in Di do
        if no value y in Dj allows (x, y) to satisfy the constraint between Xi and Xj then
            delete x from Di
            revised ← true
    return revised

With c binary constraints and domains of size at most d, each arc can be re-inserted at most d times (once per deleted value of its tail) and each revision costs O(d²), so AC-3 runs in O(c · d³) time.

Run AC-3 on Australia after fixing WA = red and Q = green: revisions remove red and green from NT and from SA, leaving NT = {blue} and SA = {blue}; the arc (SA, NT) then empties D(SA) and AC-3 returns false. The partial assignment is shown to be hopeless with no search at all. With WA = red and NT = green instead, AC-3 forces SA = blue, Q = red, NSW = green and V = red — it colours the entire mainland by itself.

Western Australia (WA)Northern Territory (NT)Queensland (Q)New South Wales (NSW)Victoria (V)South Australia (SA)Tasmania (T)WANTQNSWVSA∅ emptyT
Figure 2. AC-3 after WA = red, Q = green: NT is reduced to {blue} and SA's domain is wiped out (dots show the remaining colours of each unassigned region).

2.4Path Consistency & k-Consistency

Arc consistency can miss inferences. Try to colour the triangle WA–NT–SA with only two colours: every single value has a supporting value across each arc, so the network is arc-consistent — yet it obviously has no solution. Path consistency tightens binary constraints using implicit constraints inferred from triples: a two-variable set {Xi, Xj} is path-consistent with respect to Xm if for every consistent assignment {Xi = a, Xj = b} there is a value of Xm consistent with both. For the two-coloured triangle the only consistent pairs for {WA, NT} are {red, green} and {green, red}, and neither leaves a colour for SA; the PC-2 algorithm deletes both pairs and exposes the inconsistency.

More generally, a CSP is k-consistent if any consistent assignment to any k − 1 variables can be extended to any k-th variable. 1-consistency is node consistency, 2-consistency is arc consistency and 3-consistency is path consistency (for binary constraint networks). A graph that is strongly n-consistent can be solved without any backtracking, but establishing it takes time and space exponential in n in the worst case, so practical solvers stop at arc or path consistency.

2.5Global Constraints

Global constraints come with specialised propagation that is both faster and stronger than decomposing them into binary constraints. A simple test for Alldiff: if m variables are involved and only n distinct values remain in the union of their domains, and m > n, the constraint cannot be satisfied (the pigeonhole principle). Suppose NT, SA and Q are mutually adjacent and all three have been reduced to {green, blue}: an Alldiff check on {NT, SA, Q} sees 3 variables and 2 values and fails at once. The general algorithm repeatedly removes variables with singleton domains (and their value from the others), failing if a domain empties or more variables than values remain.

For resource constraints such as Atmost(10, P1, P2, P3) (at most 10 personnel in total), inconsistency is detected by summing the minimum values of the current domains; for large integer problems, domains are represented by lower and upper bounds and handled by bounds propagation.

2.6Sudoku as a CSP

A 9×9 Sudoku has 81 variables (one per cell) with domain {1, …, 9}; the givens have single-value domains. There are 27 Alldiff constraints — one per row, column and 3×3 box — so every cell has 20 peers. Enforcing arc consistency on the binary ≠ constraints solves the easy newspaper puzzles completely. For AIMA's hardest puzzles AC-3 leaves many candidates and search is required; interleaving AC-3 with backtracking (MAC, §3.5) and the MRV heuristic still solves them in milliseconds. On the Playground's "Hard" puzzle, for example, MAC + MRV needs 291 assignments, while plain backtracking gives up after 1.5 million constraint checks.

3.Backtracking Search for CSPs

Many CSPs cannot be solved by inference alone; at some point we must search. Backtracking search is a depth-first search that chooses values for one variable at a time and backtracks when a variable has no legal values left. It exploits commutativity (§1.6) by fixing one variable per level, and it checks consistency incrementally so that an inconsistent partial assignment is abandoned immediately.

3.1The Algorithm

A simple backtracking algorithm for CSPs (AIMA Figure 6.5)
function BACKTRACKING-SEARCH(csp) returns a solution or failure
    return BACKTRACK(csp, { })

function BACKTRACK(csp, assignment) returns a solution or failure
    if assignment is complete then return assignment
    var ← SELECT-UNASSIGNED-VARIABLE(csp, assignment)
    for each value in ORDER-DOMAIN-VALUES(csp, var, assignment) do
        if value is consistent with assignment then
            add {var = value} to assignment
            inferences ← INFERENCE(csp, var, assignment)
            if inferences ≠ failure then
                add inferences to csp
                result ← BACKTRACK(csp, assignment)
                if result ≠ failure then return result
                remove inferences from csp
            remove {var = value} from assignment
    return failure

The algorithm has three hooks that do not depend on the particular problem: SELECT-UNASSIGNED-VARIABLE (which variable next?), ORDER-DOMAIN-VALUES (in what order to try its values?) and INFERENCE (what can be deduced after each assignment?). Choosing them well is what separates a slow solver from a fast one. With a fixed variable order WA, NSW, NT, Q, SA, V, T and no inference, the engine needs 12 assignments and 5 backtracks to colour Australia: after WA = red, NSW = red and NT = green, Q must be blue and SA then has no colour; the algorithm must undo Q, NT and eventually NSW.

3.2Variable Ordering: MRV and the Degree Heuristic

The minimum-remaining-values (MRV) heuristic chooses the variable with the fewest legal values — also called the "most constrained variable" or "fail-first" heuristic. After WA = red and NT = green, SA has only blue left, so MRV assigns SA next rather than wandering off to Q, NSW or V. If some variable has zero legal values, MRV selects it and the failure is detected immediately, avoiding pointless searches through other variables. MRV typically performs orders of magnitude better than a static or random ordering.

MRV does not help with the first choice in Australia: every region starts with three values. The degree heuristic breaks the tie by choosing the variable involved in the largest number of constraints on other unassigned variables. SA has degree 5, T has degree 0. With SA chosen first, the map can be coloured with no backtracking at all. The degree heuristic is mainly used as a tie-breaker for MRV (MRV + Degree in the lecture and Playground).

3.3Value Ordering: Least-Constraining Value

Once a variable is chosen, the least-constraining-value (LCV) heuristic prefers the value that rules out the fewest choices for the neighbouring variables. After WA = red, NT = green, suppose Q is next. Q = green is illegal. Q = blue would remove SA's last value; Q = red leaves blue for SA. LCV therefore tries red first. The asymmetry with variable ordering is deliberate: a variable must eventually be assigned, so we pick the one most likely to fail early (fail-first), but we only need one solution, so we try the value most likely to succeed (fail-last).

3.4Forward Checking

Inference is most powerful when it is interleaved with search. The simplest form is forward checking: whenever a variable X is assigned, for each unassigned neighbour Y delete from DY any value inconsistent with the value chosen for X. If some DY becomes empty, the current branch fails immediately.

WANTQNSWVSAT
Initial domains
After WA = red
After Q = green
After V = blue∅

The table reproduces AIMA Figure 6.7. After WA = red, red is removed from NT and SA. After Q = green, green is removed from NT, SA and NSW — NT and SA are now both {blue}. After V = blue, blue is removed from NSW and SA, SA's domain becomes empty, and forward checking backtracks. Notice what forward checking missed: at the second row, NT and SA are adjacent and both forced to blue, which is already a contradiction. Forward checking only looks at the neighbours of the variable just assigned, so it cannot see it.

3.5Maintaining Arc Consistency (MAC)

The MAC algorithm fixes this. After a variable Xi is assigned, INFERENCE calls AC-3, but instead of a queue of all arcs it starts with only the arcs (Xj, Xi) for the unassigned neighbours Xj of Xi. From there AC-3 propagates in the usual way, and fails if any domain is emptied. In the example above, once NT is reduced to {blue} the arc (SA, NT) is examined and SA's last value disappears — one level earlier than forward checking. MAC is strictly more powerful than forward checking, because forward checking does the same work as MAC on the initial arcs but, unlike MAC, does not propagate further when a domain shrinks.

3.6Intelligent Backtracking: Looking Backward

When a branch fails, BACKTRACK returns to the most recently assigned variable — chronological backtracking. That is often the wrong place. Take the fixed order Q, NSW, V, T, SA with the partial assignment {Q = red, NSW = green, V = blue, T = red}. SA has no consistent value: red conflicts with Q, green with NSW, blue with V. Chronological backtracking tries T = green and T = blue, which cannot possibly help because Tasmania is not adjacent to SA.

A smarter approach keeps, for each variable, its conflict set: the assigned variables that ruled out one of its values. Here conf(SA) = {Q, NSW, V}. Backjumping jumps back to the most recent variable in the conflict set — V. When forward checking is used, the conflict set comes for free: when forward checking from an assignment X = x deletes a value from DY, it adds X to conf(Y). If the jump target also fails, conflict-directed backjumping merges the conflict sets, conf(Xj) ← conf(Xj) ∪ conf(Xi) − {Xj}, and keeps jumping. Constraint learning goes one step further and records the minimal set of variables responsible for a failure as a no-good, so the same contradiction is never rediscovered in another branch.

4.Local Search & the Structure of Problems

4.1Local Search: Min-Conflicts

Local search algorithms (Topic 02's hill climbing) turn out to be very effective for many CSPs. They use a complete-state formulation: every variable has a value, and each step changes the value of one variable. The min-conflicts heuristic picks a conflicted variable at random and gives it the value that results in the minimum number of conflicts with other variables, breaking ties randomly.

The MIN-CONFLICTS local search algorithm (AIMA Figure 6.8)
function MIN-CONFLICTS(csp, max_steps) returns a solution or failure
    current ← an initial complete assignment for csp
    for i = 1 to max_steps do
        if current is a solution for csp then return current
        var ← a randomly chosen conflicted variable from csp.VARIABLES
        value ← the value v for var that minimizes CONFLICTS(csp, var, v, current)
        set var = value in current
    return failure

Min-conflicts is surprisingly effective. On the n-queens problem (one queen per column; each queen's row is its value), its run-time is roughly independent of problem size: after a greedy initial assignment it solves even the million-queens problem in about 50 steps on average. In the engine's 8-queens demonstrations a random start is typically solved in a few dozen steps; a 60-queens board takes on the order of a hundred.

4.2Plateaus, Tabu Search & Constraint Weighting

The landscape of a CSP under min-conflicts often has plateaus: millions of assignments that are only one conflict away from a solution. Allowing sideways moves (to a value with the same number of conflicts) helps the search wander across them. Tabu search keeps a short list of recently visited states and forbids returning to them. Constraint weighting gives each constraint a numeric weight, initially 1; at each step the algorithm minimises the total weight of violated constraints, and then increases the weight of every constraint still violated. Constraints that are hard to satisfy therefore grow heavier, focusing the search on them.

A second advantage of local search is that it works online: when the problem changes — a flight is cancelled, a lecturer falls ill — the existing schedule can be repaired with a few moves instead of being rebuilt from scratch. This is how the Course Scheduling lab optimises its soft constraints.

4.3Independent Subproblems

The structure of the constraint graph can make a problem dramatically easier. The simplest case: if the graph has several connected components, each is an independent subproblem, and a solution is the union of the component solutions. Suppose n variables fall into components of c variables each, with domain size d. The total work is O(dc · n/c) — linear in n — instead of O(dn). With n = 80 Boolean variables split into four components of 20, this is the difference between 280 (longer than the lifetime of the universe at 10 million nodes per second) and 4 · 220 (a fraction of a second).

4.4Tree-Structured CSPs

A constraint graph is a tree when any two variables are connected by exactly one path. Any tree-structured CSP can be solved in time linear in the number of variables. The key is directional arc consistency (DAC): under an ordering X1, …, Xn, a CSP is DAC if every Xi is arc-consistent with each Xj for j > i.

The TREE-CSP-SOLVER algorithm (AIMA Figure 6.11)
function TREE-CSP-SOLVER(csp) returns a solution, or failure
    n ← number of variables in X
    assignment ← an empty assignment
    root ← any variable in X
    X ← TOPOLOGICALSORT(X, root)
    for j = n down to 2 do
        MAKE-ARC-CONSISTENT(PARENT(Xj), Xj)
        if it cannot be made consistent then return failure
    for i = 1 to n do
        assignment[Xi] ← any consistent value from Di
        if there is no consistent value then return failure
    return assignment

After the backward pass, every parent value has a compatible value in each child, so the forward pass can assign variables from the root down with no backtracking. There are n − 1 arcs and each is revised once at cost O(d²), giving O(n · d²) overall. Exercise 6.6 traces the algorithm on the tree of AIMA Figure 6.10.

A{R, G}B{R, G}C{R}D{G, B}E{G}F{B, G}
Figure 3. The tree-structured CSP of AIMA Fig 6.10 (edges are ≠ constraints), with the initial domains used in the lecture's stepper and in Exercise 6.6.

4.5Cutset Conditioning

Most graphs are not trees, but many are nearly trees. Cutset conditioning chooses a subset S of variables such that the constraint graph becomes a tree after S is removed — a cycle cutset. For each possible consistent assignment to S, it removes from the domains of the remaining variables any values inconsistent with that assignment, and solves the remaining tree-structured CSP. If the cutset has size c, the total run-time is O(dc · (n − c) · d²): very fast when c is small. In Australia, S = {SA}: after assigning SA, the rest of the map is the chain WA – NT – Q – NSW – V plus the isolated T.

WA{G, B}NT{G, B}Q{G, B}NSW{G, B}V{G, B}SAT{R, G, B}
Figure 4. Conditioning on SA = red: SA's neighbours lose red, and the remaining constraint graph is a tree.

4.6Tree Decomposition

A tree decomposition transforms the constraint graph into a tree of connected subproblems. Each subproblem is solved independently, and the resulting solutions are combined. A decomposition must satisfy three requirements: every variable appears in at least one subproblem; if two variables are connected by a constraint, they (and the constraint) appear together in at least one subproblem; and if a variable appears in two subproblems, it appears in every subproblem along the path connecting them. Australia decomposes into {WA, NT, SA}, {NT, SA, Q}, {SA, Q, NSW}, {SA, NSW, V} and {T} (AIMA Fig 6.13).

Each subproblem is solved and its solutions become the domain of a single "mega-variable"; constraints between mega-variables require shared variables to agree; and the resulting tree is solved with the tree algorithm. The tree width w of a decomposition is one less than the size of its largest subproblem (here w = 2); a CSP with tree width w can be solved in O(n · dw+1) time. Finding the decomposition with minimal tree width is NP-hard, but good heuristics exist.

Finally, value symmetry can be exploited: in map colouring, any permutation of the d colours of a solution is also a solution, so imposing an arbitrary symmetry-breaking constraint such as NT < SA < WA (alphabetical colour order) reduces the search space by a factor of d!.

5.Evaluating CSP Solvers

5.1Measuring Work

Wall-clock time depends on the machine and the implementation, so CSP solvers are usually compared by counting the work they do: the number of assignments (search nodes) tried, the number of backtracks, and the number of constraint checks. Inference trades checks for assignments — MAC performs more checks per node than forward checking, which performs more than plain backtracking — but on tightly constrained problems it explores so many fewer nodes that it wins overall. The lecture's live benchmark reproduces this on a 20-region random map: plain backtracking needs 1,113 assignments and 1,093 backtracks, whereas MRV + MAC finds a solution with just 20 assignments and no backtracking at all.

5.2Complexity

General finite-domain CSPs are NP-complete (they include 3-SAT as a special case), so no known complete algorithm is better than exponential in the worst case. Heuristics and inference do not change the worst case; they change the typical case, often by many orders of magnitude. Structure gives genuine polynomial special cases.

AlgorithmTimeSpaceNotes
Backtracking (any ordering)O(dn)O(n)Depth-first; worst case unchanged by heuristics
Forward checking / MACO(dn)O(n · d)Far fewer nodes on tight problems
AC-3 (once)O(c · d³)O(c)c binary constraints
Tree-structured CSPO(n · d²)O(n · d)No backtracking
Cutset conditioningO(dc · (n − c) · d²)O(n · d)c = cycle-cutset size
Tree decompositionO(n · dw+1)O(n · dw)w = tree width
Min-conflictsno bound (incomplete)O(n)≈ constant steps on n-queens

5.3Comparison Matrix

PropertyBacktracking+ MRV / Degree / LCV+ Forward CheckingMACMin-ConflictsTree-CSP Solver
CompleteYesYesYesYesNoYes (trees)
Proves "no solution"YesYesYesYesNoYes
Detects failure earlyNoPartlyEmpty neighbour domainAny arc inconsistency—Yes
Soft constraintsBranch & boundBranch & boundBranch & boundBranch & boundNaturally (cost)Dynamic programming
Best forTiny problemsGeneral CSPsGeneral CSPsTight puzzles (Sudoku)Huge / online repairTree-like graphs

6.Practice Exercises

Eleven worked exercises covering formulation, AC-3, forward checking, ordering heuristics, min-conflicts and structure. Exercises 7–11 are adapted from the AIMA Chapter 6 exercise set (Exercises 6.5, 6.6, 6.8, 6.11 and 6.15), reworded and illustrated for clarity. Try each one before opening the answer; every answer was checked with the lecture's engine or a short solver script.

Exercise 1

Formulating Australia

For the Australia map-colouring CSP with three colours (Figure 1):

  1. How many variables and how many binary constraints are there?
  2. How many complete assignments exist, and how many leaves does the naïve search tree (any variable, any value at each level) have?
  3. How many solutions does the CSP have?
1. Size

7 variables (WA, NT, Q, NSW, V, SA, T) and 9 binary constraints: the 9 edges of the constraint graph — five around SA (WA, NT, Q, NSW, V) plus WA–NT, NT–Q, Q–NSW and NSW–V.

2. Search space
complete assignments = dn      = 37        = 2,187
naïve tree leaves    = n! · dn = 5,040 · 2,187 = 11,022,480
3. Number of solutions

Colour SA first: 3 choices. The five mainland regions around SA form the path WA – NT – Q – NSW – V, and each must avoid SA's colour, so the path is coloured with the remaining 2 colours, alternating: 2 choices once WA is fixed. Tasmania is independent: 3 choices.

solutions = 3 (SA) × 2 (path) × 3 (T) = 18
Exercise 2

AC-3 Trace

Variables X, Y, Z each have domain {1, 2, 3, 4}. The constraints are X < Y and Y < Z. Run AC-3 starting with the queue [(X, Y), (Y, X), (Y, Z), (Z, Y)], processing arcs first-in first-out. Give every revision and the final domains.

Trace
1. REVISE(X, Y): remove 4 from D(X)
     X {1,2,3}  Y {1,2,3,4}  Z {1,2,3,4}
2. REVISE(Y, X): remove 1 from D(Y)
     X {1,2,3}  Y {2,3,4}  Z {1,2,3,4}
3. REVISE(Y, Z): remove 4 from D(Y); re-queue (X, Y)
     X {1,2,3}  Y {2,3}  Z {1,2,3,4}
4. REVISE(Z, Y): remove 1, 2 from D(Z)
     X {1,2,3}  Y {2,3}  Z {3,4}
5. REVISE(X, Y): remove 3 from D(X)
     X {1,2}  Y {2,3}  Z {3,4}
queue empty → arc-consistent.    X {1,2}  Y {2,3}  Z {3,4}
Why (X, Y) is revised twice

When REVISE(Y, Z) removes 4 from D(Y), the arc (X, Y) — which points into Y — is added back to the queue. Its second revision removes 3 from D(X), because X = 3 needed Y = 4 as support.

X ∈ {1, 2} · Y ∈ {2, 3} · Z ∈ {3, 4} — arc-consistent, 6 values removed
Exercise 3

Forward Checking Trace

Starting from full domains {R, G, B}, assign WA = red, then Q = green, then V = blue, applying forward checking after each assignment. Give the domains of all variables after each step. At which step does forward checking detect failure? Could it have been detected earlier, and by which method?

WANTQNSWVSAT
Initial domains{R, G, B}{R, G, B}{R, G, B}{R, G, B}{R, G, B}{R, G, B}{R, G, B}
After WA = red{R}{G, B}{R, G, B}{R, G, B}{R, G, B}{G, B}{R, G, B}
After Q = green{R}{B}{G}{R, B}{R, G, B}{B}{R, G, B}
After V = blue{R}{B}{G}{R}{B}∅{R, G, B}
Failure

After V = blue, D(SA) = ∅, so forward checking fails and backtracks. But the contradiction already existed after Q = green: NT and SA are adjacent and both reduced to {B}. Forward checking does not examine the arc (NT, SA) because neither was just assigned; MAC would have re-queued (SA, NT) when D(NT) shrank and failed one level earlier.

Exercise 4

Ordering Heuristics

  1. With the empty assignment, which variable does MRV choose? Which does MRV with the degree tie-breaker choose?
  2. With {WA = red, NT = green}, give the number of legal values of every unassigned variable and the MRV choice.
  3. In the same state, suppose Q is the variable being assigned. Order its values with LCV.
1. First variable

Every variable has 3 legal values, so MRV alone is a 7-way tie (a static tie-break would pick WA). Degrees to unassigned variables: SA 5, NT 3, Q 3, NSW 3, WA 2, V 2, T 0 — the degree heuristic picks SA.

2. MRV after WA = red, NT = green
SA : {blue}              → 1   (neighbours WA = red, NT = green)
Q  : {red, blue}         → 2   (neighbour NT = green)
NSW: {red, green, blue}  → 3
V  : {red, green, blue}  → 3
T  : {red, green, blue}  → 3
MRV chooses SA.
3. LCV for Q

Q = green is illegal (NT is green). For the unassigned neighbours SA and NSW:

Q = red : SA {blue} (1 value), NSW {green, blue} (2)  → 3 values remain
Q = blue: SA {}     (0 value), NSW {red, green}  (2)  → SA wiped out
LCV order: red, then blue.
Exercise 5

A Min-Conflicts Step on 4-Queens

Columns c1…c4 hold one queen each; rows are numbered 1 (top) to 4. The current complete assignment is (c1, c2, c3, c4) = (2, 3, 1, 3).

♛♛♛♛
The 4-queens state (2, 3, 1, 3). Red queens are attacked.
  1. How many pairs of queens attack each other, and how many conflicts does each queen have?
  2. Min-conflicts selects the queen in c2. Compute CONFLICTS(c2, r) for r = 1…4 and give the new state.
1. Conflicts

Two attacking pairs: (c1, c2) on a diagonal (rows 2 and 3, adjacent columns) and (c2, c4) on row 3. Per queen: c1 = 1, c2 = 2, c3 = 0, c4 = 1.

2. Min-conflicts value for c2
r = 1 : row with c3, diagonals with c1 and c4   → 3 conflicts
r = 2 : row with c1, diagonal with c3               → 2 conflicts
r = 3 : diagonal with c1, row with c4               → 2 conflicts
r = 4 : no row or diagonal shared                    → 0 conflicts

Move the queen to row 4. The new state (2, 4, 1, 3) has no attacking pairs — it is a solution, reached in a single min-conflicts step.

Exercise 6

Exploiting Structure

  1. Run TREE-CSP-SOLVER on the tree of Figure 3 (edges A–B, B–C, B–D, D–E, D–F, all ≠) with domains A {R, G}, B {R, G}, C {R}, D {G, B}, E {G}, F {B, G}, using the order A, B, C, D, E, F.
  2. Using the cycle cutset S = {SA}, how many elementary operations does cutset conditioning need for Australia with d = 3, compared with brute-force enumeration of complete assignments?
1. Backward pass (j = n down to 2)
MAKE-ARC-CONSISTENT(D, F) : no change
MAKE-ARC-CONSISTENT(D, E) : remove G from D(D) → D(D) = {B}
MAKE-ARC-CONSISTENT(B, D) : no change
MAKE-ARC-CONSISTENT(B, C) : remove R from D(B) → D(B) = {G}
MAKE-ARC-CONSISTENT(A, B) : remove G from D(A) → D(A) = {R}
Forward pass (i = 1 to n)
A = R   (root)
B = G   (≠ parent A = R)
C = R   (≠ parent B = G)
D = B   (≠ parent B = G)
E = G   (≠ parent D = B)
F = G   (≠ parent D = B)

Solution {A = R, B = G, C = R, D = B, E = G, F = G}, found without a single backtrack.

2. Cutset conditioning
c = 1, n = 7, d = 3
cutset conditioning : dc · (n − c) · d² = 3 · 6 · 9 = 162
brute force         : dn              = 37       = 2,187

For each of the 3 values of SA, the remaining 6 variables form a tree solved in (n − c) · d² steps.

Exercise 7

Cryptarithmetic by Hand: TWO + TWO = FOUR

Adapted from AIMA Exercise 6.5. In the puzzle below every letter stands for a different digit, and the leading letters T and F cannot be 0. The small variables C1, C2, C3 are the carries passed from one column to the next (each is 0 or 1).

C3C2C1TWOTWO+FOUR
The puzzle as a column addition. Each arrow is a carry: C1 goes from the units column to the tens, C2 from the tens to the hundreds, C3 from the hundreds into the thousands.
The CSP (one constraint per column)
X = {F, T, U, W, R, O, C1, C2, C3}
D(letters) = {0,…,9}  ·  D(T), D(F) = {1,…,9}  ·  D(Ci) = {0, 1}
Alldiff(F, T, U, W, R, O)
units:     O + O        = R + 10·C1
tens:      C1 + W + W = U + 10·C2
hundreds: C2 + T + T = O + 10·C3
thousands: C3 = F

Task. Solve the puzzle by hand with backtracking search + forward checking, choosing variables with MRV and values with LCV. Draw the search tree you explore and give the final digits.

Rules for this trace
Variable: MRV (fewest legal values); break ties by the degree heuristic (most constraints on unassigned variables), then alphabetically.
Value: LCV (the value that leaves neighbours the most options); break ties with the smaller digit.
Forward checking on a column constraint fires only when exactly one of its variables is still unassigned.
1. The first variable: MRV ties, degree decides

The three carries each have 2 values, so MRV is a 3-way tie. Count how many other variables each carry shares a constraint with:

C1 : O, R, W, U, C2            → degree 5
C2 : C1, W, U, T, O, C3        → degree 6   ← chosen
C3 : C2, T, O, F               → degree 4

C2 = 0 and C2 = 1 leave the same domain sizes, so LCV ties and picks 0. Then C1 (MRV, 2 values) = 0 by the same argument.

2. C3 and F: the "thousands" column decides

C3 is next (2 values). LCV reverses the usual order here: C3 = 0 would force F = 0, but F ≠ 0, so forward checking would wipe out D(F). LCV therefore tries C3 = 1, and forward checking leaves D(F) = 1. MRV now picks F (1 value): F = 1.

3. Search tree
{ }startC₂ = 0MRV+degree: C₂C₁ = 0MRV: C₁C₃ = 1MRV: C₃ (LCV: 1)F = 1MRV: F (forced)T (LCV order)T = 5T = 6T = 7O = 0FC: R must be 0 = OD(R) = ∅ → wipe-outO = 2R = 4U ∈ {0,3,5,7,8,9}every U wipes out D(W)(W = U/2 is used or not a digit)O = 4R = 8U = 6W = 3solution ✓
The search tree explored. Teal = the path to the solution; red dashed = a forward-checking wipe-out; grey = a branch that is explored and then abandoned. The hundreds column with C2 = 0, C3 = 1 reads 2T = O + 10, so T ≥ 5: values 2–4 would wipe out D(O), which is why LCV puts them last and tries T = 5 first.
T = 5 : O = 2·5 − 10 = 0   → units: R = O+O = 0 = O   ✗ D(R) = ∅
T = 6 : O = 2, R = 4        → tens: U = 2W (C1 = C2 = 0)
        W ∈ {0,…,4} \ {1,2,4} = {0,3} → U = 0 (= W) or 6 (= T)  ✗ every U fails
T = 7 : O = 4, R = 8        → U = 6, W = 3                         ✓
734 + 734 = 1468  ·  T = 7, W = 3, O = 4, F = 1, U = 6, R = 8  (C1 = 0, C2 = 0, C3 = 1)
Notice where the work went: the carries and F were settled with no backtracking at all, because MRV pushed the most-constrained variables to the top of the tree. All of the failures happen at the T level, and forward checking catches each one within one or two levels instead of at the bottom of the tree.
Exercise 8

From a Ternary Constraint to Binary Constraints

Adapted from AIMA Exercise 6.6. Algorithms such as AC-3 work on binary constraints (two variables each). The constraint A + B = C is ternary: it ties three variables together at once. Show how to replace it by three binary constraints by adding one new (auxiliary) variable.

Use the domains D(A) = {1, 2}, D(B) = {1, 2, 3}, D(C) = {3, 4} to make your answer concrete:

  1. Define the auxiliary variable X and its domain.
  2. Write the three binary constraints that link X to A, B and C.
  3. Make the new CSP arc-consistent. Which values of X survive, and do they match the solutions of the original constraint?
Key idea
A value of the new variable should store everything the old constraint needs to know about A and B at once. Then each original variable only has to "agree" with one part of it — a two-variable check.
BEFORE: ONE TERNARY CONSTRAINTAFTER: AUXILIARY VARIABLE XA + B = CABC{1,2}{1,2,3}{3,4}one constraint, scope of size 3A = X₁B = X₂C = X₁ + X₂XD(X) = pairs ⟨a, b⟩A{1,2}B{1,2,3}C{3,4}
Left: the ternary constraint drawn as a box joined to all three variables. Right: the auxiliary variable X becomes the hub of a star of three binary constraints.
1. The auxiliary variable

Let X range over pairs ⟨a, b⟩, one entry for A and one for B: D(X) = D(A) × D(B). Write X1 and X2 for the first and second entries.

D(X) = {⟨1,1⟩, ⟨1,2⟩, ⟨1,3⟩, ⟨2,1⟩, ⟨2,2⟩, ⟨2,3⟩}     (2 × 3 = 6 pairs)
2. Three binary constraints
⟨X, A⟩ :  A = X1
⟨X, B⟩ :  B = X2
⟨X, C⟩ :  C = X1 + X2

Each constraint mentions only two variables, yet together they say exactly "A + B = C": X fixes A and B, and the third constraint checks their sum against C.

3. Arc consistency
REVISE(X, C): keep pairs whose sum is in D(C) = {3, 4}
  ⟨1,1⟩ sum 2 ✗   ⟨1,2⟩ sum 3 ✓   ⟨1,3⟩ sum 4 ✓
  ⟨2,1⟩ sum 3 ✓   ⟨2,2⟩ sum 4 ✓   ⟨2,3⟩ sum 5 ✗
  → D(X) = {⟨1,2⟩, ⟨1,3⟩, ⟨2,1⟩, ⟨2,2⟩}
REVISE(A, X): A = 1 ✓ (⟨1,2⟩), A = 2 ✓ (⟨2,1⟩)   → no change
REVISE(B, X): B = 1 ✓, 2 ✓, 3 ✓ (⟨1,3⟩)           → no change
REVISE(C, X): C = 3 ✓, C = 4 ✓                    → no change

The 4 surviving pairs are exactly the 4 solutions of A + B = C with these domains: (1, 2, 3), (1, 3, 4), (2, 1, 3), (2, 2, 4).

A + B = C  ⟶  X ∈ D(A) × D(B),   A = X1,   B = X2,   C = X1 + X2
The same trick works for any constraint with k variables: create one auxiliary variable whose values are the allowed k-tuples, and link it to each original variable with a "projection" constraint. The price is the size of D(X), which can grow as dk.
Exercise 9

3-Colouring with Conflict-Directed Backjumping

Adapted from AIMA Exercise 6.8. The graph below has 8 nodes. The chain A1 – A2 – A3 – A4 is joined node-by-node; every Ai is also joined to the hub H; H is joined to T; and T is joined to both F1 and F2. Colour the graph with {R, G, B} so that joined nodes differ.

A1#1A2#5A3#7A4#3H#2T#8F1#4F2#6
The constraint graph. The small #k tag beside each node is its position in the fixed variable order.

Task. Use backtracking with conflict-directed backjumping (no forward checking), with

variable order: A1, H, A4, F1, A2, F2, A3, T  ·  value order: R, G, B

Show every assignment, the conflict set of each variable that fails, and where the search jumps back to. Then compare with plain (chronological) backtracking.

Reminder: how backjumping works
The conflict set of X is the set of earlier-assigned variables that ruled out some value of X. When X runs out of values, jump back to the most recent variable in its conflict set (call it Y), and pass the rest along: conf(Y) ← conf(Y) ∪ conf(X) − {Y}.
1. Forward until the first dead end
#1 A1 = R
#2 H  = G      (R taken by A1)
#3 A4 = R      (only H is assigned among its neighbours)
#4 F1 = R      (T not assigned yet)
#5 A2 = B      (R by A1, G by H)                 conf(A2) = {A1, H}
#6 F2 = R
#7 A3 : R ✗ by A4,  G ✗ by H,  B ✗ by A2        conf(A3) = {A4, H, A2}
2. Two backjumps
Jump ① from A3: most recent in {A4, H, A2} is A2   (F2 is skipped)
        conf(A2) = {A1, H} ∪ {A4, H, A2} − {A2} = {A1, H, A4}
        A2 has no values left (B was its only legal value)
Jump ② from A2: most recent in {A1, H, A4} is A4   (F1 is skipped)
        A4 : G ✗ (H),  B ✓  → A4 = B
A1RHGA4RF1RA2BF2RA3✗Tjump ①jump ②
Backjumps along the variable order. F2 and F1 (grey) are jumped over: they are not connected to the failure, so changing them could never help.
3. Forward again to a solution
#4 F1 = R   #5 A2 = B   #6 F2 = R
#7 A3 = R   (A2 = B, A4 = B, H = G → R is free)
#8 T  = B   (H = G, F1 = F2 = R)
A1#1A2#5A3#7A4#3H#2T#8F1#4F2#6
The colouring found: A1 = R, H = G, A4 = B, F1 = R, A2 = B, F2 = R, A3 = R, T = B.
4. Comparison with chronological backtracking

Plain backtracking returns to the previous variable, F2, and tries G and B — but F2 has nothing to do with A3's problem, so A3 fails again each time. It then backs up through A2 to F1 and repeats the same useless work for F1 = G and F1 = B before finally reaching A4.

MethodAssignments madeSame solution?
Chronological backtracking24Yes
Conflict-directed backjumping12Yes
A1 = R, H = G, A4 = B, F1 = R, A2 = B, F2 = R, A3 = R, T = B  ·  12 assignments vs. 24
Exercise 10

AC-3 Detects a Hidden Dead End

Adapted from AIMA Exercise 6.11. In the Australia map-colouring CSP (Figure 1), we have made only two assignments: WA = green and V = red. Every other region still has {R, G, B}. Nothing is violated yet — but the assignment cannot be extended to a full solution.

WANT{R,G,B}Q{R,G,B}NSW{R,G,B}VSA{R,G,B}T{R,G,B}
The partial assignment WA = green, V = red. Edges are ≠ constraints.

Task. Run AC-3 on this state (treat each assigned variable as having a one-value domain) and show that it empties some domain — proving the partial assignment is inconsistent. Start with all 18 arcs in the queue, in the order the constraints are listed in Section 1.1 (SA ≠ WA, SA ≠ NT, …, NSW ≠ V), each constraint X ≠ Y giving the arc (X, Y) and then (Y, X).

Recall
REVISE(X, Y) removes from D(X) every value that has no compatible value left in D(Y). If D(X) shrinks, every arc (Z, X) pointing into X goes back on the queue. If any domain becomes empty, AC-3 returns false.
1. Trace (only the revisions that remove something)

AC-3 pops 27 arcs before it stops; the 9 that change a domain are:

 1. REVISE(SA,  WA) : remove G  → SA  = {R, B}
 2. REVISE(SA,  V ) : remove R  → SA  = {B}        arcs into SA re-queued
 3. REVISE(NT,  WA) : remove G  → NT  = {R, B}
 4. REVISE(NSW, V ) : remove R  → NSW = {G, B}
 5. REVISE(NT,  SA) : remove B  → NT  = {R}
 6. REVISE(Q,   SA) : remove B  → Q   = {R, G}
 7. REVISE(NSW, SA) : remove B  → NSW = {G}
 8. REVISE(Q,  NSW) : remove G  → Q   = {R}
 9. REVISE(Q,   NT) : remove R  → Q   = { }         ✗ empty → return false
2. Why it fails
WANT{R}Q∅NSW{G}VSA{B}T{R,G,B}
The state when AC-3 stops. Q's three neighbours have been forced to R, B and G — one of each colour — so nothing is left for Q.

The two assignments force a chain of consequences: SA must avoid both green (WA) and red (V), so SA = blue. Then NT (next to WA and SA) must be red and NSW (next to V and SA) must be green. Q touches NT, SA and NSW, which now use all three colours.

WA = G, V = R  ⟹  SA = B  ⟹  NT = R, NSW = G  ⟹  D(Q) = ∅  ·  inconsistent
Forward checking would not catch this right away: after WA = green and V = red it only prunes the direct neighbours of WA and V, and every domain is still non-empty. Arc consistency follows the chain through SA, NT and NSW to Q, so a solver using MAC would reject V = red immediately instead of searching beneath it.
Exercise 11

Why TREE-CSP-SOLVER Works from the Leaves Up

Adapted from AIMA Exercise 6.15. In its first pass, TREE-CSP-SOLVER (Section 4.4) makes each arc (PARENT(Xj), Xj) consistent for j = n down to 2 — that is, starting at the leaves and moving towards the root. Why does it go in this direction? What would go wrong if it went from the root towards the leaves instead?

Use this small tree (all edges are ≠ constraints, topological order A, B, D, C) to support your answer:

ROOT A · ORDER A, B, D, CA{G, R}B{R, G}D{G, B}C{R}
Root A with children B and D; C is a child of B. Values are tried in the order listed in each domain.
Hint
Making the arc (P, X) consistent can only remove values from the parent P. Ask: after a domain shrinks, which arcs that were already processed could become inconsistent again?
LEAVES → ROOT (correct)123A{R}B{G}D{G, B}C{R}A = R, B = G, D = G, C = Rno backtrackingROOT → LEAVES (wrong)123A{G, R}B{G}D{G, B}C{R}arc (A, B) broken by step 3A = G → B has no value → backtrack
Numbered circles show the order in which arcs are revised. Left: leaves first — every domain is final before its parent is revised. Right: root first — step 3 shrinks D(B) after (A, B) was already checked, leaving A = G without support.
1. Leaves → root (the algorithm's order)
1. (B, C): B = R has no support (C = R)  → D(B) = {G}
2. (A, D): every A value has support    → no change
3. (A, B): A = G has no support (B = G)  → D(A) = {R}
Forward pass: A = R, B = G, D = G, C = R   ✓ no backtracking

When (A, B) is revised, D(B) is already final: all of B's own children were handled earlier and nothing later will touch D(B). So once the pass ends, every parent value has a supporting value in each child, and the forward pass from the root can always pick a consistent value.

2. Root → leaves (the wrong order)
1. (A, B): both A values have support (B still {R, G}) → no change
2. (A, D): no change
3. (B, C): B = R has no support                         → D(B) = {G}
Forward pass: A = G (first value) → B must be ≠ G but D(B) = {G}   ✗ backtrack

Step 3 removed a value from B after the arc (A, B) had been declared consistent, so A = G lost its only support. The pass ends with an arc that is not consistent, and the forward pass can hit a dead end. To fix it we would have to re-check (A, B) — i.e. go back up the tree, which is exactly the leaves-first order.

Revising (P, X) shrinks only D(P)  ⟹  process children before parents so no finished arc is ever broken  ⟹  O(n·d²), no backtracking