University of Massachusetts  Artificial Intelligence  Computer Science Department 
CMPSCI 683  Fall 2010 
INTRODUCTION  
Lecture 1: Course Introduction [Wed 9/08]
Course introduction. What is AI? Goals of AI. Importance and Practicality of AI. Issues with AI. AI and Uncertainty. Agents. Combinatorial Auctions. Search and AI. Reading: Russel and Norvig, Chapters 1, 2.  
SEARCH  
Lecture 2: Introduction to Search Strategies [Mon 9/13]
Abstraction. Problem Solving by Search. Knowledge and Problem Types. Search Trees. Reading: Russel and Norvig, Chapter 3.1  3.7, 4.14.4.  
Lecture 3: Search Tree Represenations [Wed 9/15]
Problem formalization issues. Abstraction. Homework 1 assigned  
Lecture 4: Blind and Informed Search Strategies. [Mon 9/20]
Search Algorithms  Breath First Search, Depth First Search, Iterative Search and Bidirectional Search. Heuristic search. Greedy Search. A*. Admissible evaluation functions. Monotone evaluation functions. Reading: Russel and Norvig, 2nd Edition: Chapter 4.14.4. 3rd Edition: Chapter 3.53.6  
Lecture 5: Space Variations in A* [Wed 9/22]
Iterative Deepening A*. Recursive Best First Search (RBFS).  
Lecture 6: Time and Hierarchical Variations of A* [Mon 9/27]
Guest Lescturer: Hala Mostafa Memorybounded heuristic search. Simplified Memory Bounded A* (SMA*). Realtime problem solving. Satisficing and Optimizing. Anytime A*. RealTime A*. Required Reading : Richard E. Korf, RealTime Heuristic Search, Artificial Intellligence 42 (1990), pp 189211. Homework 2 assigned  
Lecture 7: Hierarchial & Local Search [Wed 9/29]
RTA*. Hierarchical A*. Beginning of Local Search Required Readings: Eric A. Hansen, Shlomo Zilberstein, Victor A. Danilchenko, Anytime Heuristic Search: First Results, CS Technical Report, 9750, UMASS Hierarchical A*: R.C. Holte, M.B. Perez, R.M.Zimmer, A.J. Macdonald, Hierarchical A*: Searching Abstraction Hierarchies Efficiently, {AAAI}/{IAAI}, Vol. 1, pp 530535, 1996 Optional Readings: Other Examples of Hierarchical Problem Solving: Craig A. Knoblock, Abstracting the Tower of Hanoi, In Proceedings of the Workshop on Automatic Generation of Approximations and Abstractions, pages 1323, Boston, MA, 1990  
Lecture 8: Local Search [Mon 10/04]
HillClimbing, Simulated Annealing, Genetic Search .  
Lecture 9: CSP Algorithms [Wed 10/06]
Systematic Search. Backtracking Search. Informed Backtracking. Constraint Propagation. Arc Consistency.  
NO CLASS. Colombus Day. [Mon 10/11]  
Lecture 10: Constraint Satisfaction Problem [Tues 10/12]
CSP Heuristics. Informed Backtracking. Advanced Backtracking. Tree Structured CSPs. Subgoal/Subproblem Interactions. Nearly Decomposable Problems.  
Lecture 11: Blackboard Systems [Wed 10/13]
Introduction to Blackboard Architectures. Blackboard Problem Solving. Blackboard and Search. Cooperating Experts. Blackboard Applications. Required Reading: Erman, L.D., HayesRoth, F., Lesser, V.R., and Reddy, D.R. (1980). The HEARSAYII Speech Understanding System: Integrating Knowledge to Resolve Uncertainty. Computing Surveys 12, (2), 213253, 1980. Optional reading: Carver, N. and Lesser, V. The Evolution of Blackboard Control Architectures. Computer Science Technical Report 9271, University of Massachusetts, Amherst. (This is a revised and extended version of paper with same title in Expert Systems with Applications: Special Issue on the Blackboard Paradigm and Its Applications.)  
MIDTERM [Mon 10/18]  
 
MARKOV PROCESSES  
Lecture 12: Markov Decision Processes [Wed 10/20]
Search with Uncertainty. Introduction to Markov Decision Processes. Goals and Rewards. Performance Criteria. Bellman Equation. Value Iteration. Policy Iteration. Value Determination. Reading: Russel and Norvig, Chapter 17.117.4 Homework 2 DUE  
Lecture 13: Partial Observable Markov Decision Processes [Mon 10/25]
Greedy Policy v/s Optimal Policy. Policy Iteration. Value Determination. Introduction to SMDP. Introduction to POMDP. Bayesan Policy Representation. FiniteMemory Policies.  
Lecture 14: Decision Making As An Optimization Problem [Wed 10/27]
Guest Lecturer: Hala Mostafa Formulating the problem of computing a policy as an optimization program, e.g. Linear, Mixed Integer, Nonlinear and Bilinear Programs. Single and multiagent decision making. Optional Reading: Formal Models and Algorithms for Decentralized Decision Making under Uncertainty by Sven Seuken and Shlomo Zilberstein. An Investigation into Mathematical Programming for Finite Horizon Decentralized POMDPs by Raghav Aras, Alain Dutech.  
Lecture 15: Hidden Markov Models [Mon 11/01]
Introduction to Hidden Markov Models. Probabilistic Inference in HMM. Representation for Paths. Viterbi Algorithm. Optional Reading: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition, Sections IIII. Russel and Norvig, Chapter 15.3.  
REASONING UNDER UNCERTAINTY  
Lecture 16: Uncertainty in Intelligent Systems [Wed 11/03]
Ubiquity of Uncertainty. Sources of Uncertainty. Reasoning under Uncertainty. Acting under Uncertainty. Uncertainty in FirstOrder Logic. Nonmonotonicity. Belief and Evidence. Probability v/s Causality. MYCIN's Certainty Factors. Probability Theory. Bayesian reasoning. Abductive Uncertainty. Reading: Russel and Norvig, Chapter 13.1 to 13.6  
Lecture 17: Introduction to Probabilistic Reasoning with Belief Networks [Mon 11/08]
Review of Lecture 16. Introduction to Belief (or Bayesial) Networks. Conditional Independence in BNs. Semantics of BN. Inference in BNs. Reasoning in BNs. Representation of Conditional Probability Tables. Benefits of BNs. Constructing BNs. dSeperation. Inference in BNs. Variable Elimination. Reading: Russel and Norvig, Chapter 14.1 to 14.7  
NO CLASS. Thursday's schedule. [Wed 11/10]  
Lecture 18: Approximate inference for BNs. [Mon 11/15]
Variable Elimination in Chains. Elimination in Chains with Evidence. Incremental Updating of BNs. Belief Propagation in Trees. Inference in Multiply Connected BNs. Clustering methods. Cutset Conditioning. Reading: Russel and Norvig, Chapter 14.4  14.7.  
Lecture 19: Decision Theory [Wed 11/17]
False Positive and Negatives. Inference in Multiply Connected BNs. Stochastic Simulation. Likelihood Weighting. Markov Chain Monte Carlo. Markov Blanket. Introduction to Utility Theory. Axioms of Utility Theory. Utility scales and utility assessment. Value of Information. Reading: Russel and Norvig, Chapter 16  
Lecture 20: Simple Decision Networks (Trees) [Mon 11/22]
Guest Lecturer: Hala Mostafa Value of Information examples. Value of Perfect Information. Properties of the Value of Information. Decision trees. . Reading: Russel and Norvig, Chapter 18.3  
Lecture 21: More on Decision Networks [Wed 11/24]
Introduction to Decision Networks. Nodes in a Decision Network. Knowledge in a Decision Network. Topology of Decision Networks. Evaluating Decision Networks. Evaluation by Graph Reduction. Shachter's Algorithm. DempsterShafer Theory. Fuzzy Set Theory/Logic. Truth Maintainance Systems. Reading: Russel and Norvig, Chapter 18.1,2,4,5.  
LEARNING  
Lecture 22: Introduction to Learning [Mon 11/29]
Definition of Learning. Types of Learned Knowledge. Characterizing Learning Systems. Model of Learning Agents. Dimensions of Learning. Supervised Learning. Ockham's Razor. Decision Trees and Learning.  
Lecture 23: Decision Tree Learning [Wed 12/01]
Decision Tree Algorithm for Learning. Choosing the Best Attribute Based on Information Theory. Splitting Examples. Decision Tree Learning. Performance Measurements. Inductive Bias. Overfitting. Missing Data. MultiValued Attributes. Continuous Valued Attributes. Intro to Neural Networks. Connectionist Computation. Takeover Midterm Exam Review. Reading: Russel and Norvig, Chapters 19.119.5, 20.8.  
Lecture 24: Neural Networks [Mon 12/06]
Artificial Neural Networks. Neural Network Learning. MultiLayer Networks. Perceptron. Perceptron Learning. Gradient Decent. Delta Rule. Approximation to Gradient Decent. Back Propagation. Overfitting. Convergence. Reading: Russel and Norvig, Chapter 20.120.6.  
Lecture 25: Reinforcement Learning [Wed 12/08]
Problem with Supervised Learning. Intro to Reinforcement Learning. Markov Decision Processes. Key Features of RL. Utility Function. ActionValue Function. Passive versus Active Learning. Learning Utility Functions. Direct Utility Estimation. Adaptive Dynamic Programming. Temporal Difference Learning. TicTacToe. Simple Monte Carlo. Limitations. Q Learning for Deterministic Worlds. NonDeterministic Q Learning.  
Questions And Answers  
FINAL EXAM ON Th 12/16 AT 4:00pm in Lederle Grad Res Ctr ROOM A301  
