INTRODUCTION
	Lecture 1: Course Introduction [Wed 9/08] Course introduction. What is AI? Goals of AI. Importance and Practicality of AI. Issues with AI. AI and Uncertainty. Agents. Combinatorial Auctions. Search and AI. Reading: Russel and Norvig, Chapters 1, 2.
SEARCH
	Lecture 2: Introduction to Search Strategies [Mon 9/13] Abstraction. Problem Solving by Search. Knowledge and Problem Types. Search Trees. Reading: Russel and Norvig, Chapter 3.1 - 3.7, 4.1-4.4.
	Lecture 3: Search Tree Represenations [Wed 9/15] Problem formalization issues. Abstraction. Homework 1 assigned
	Lecture 4: Blind and Informed Search Strategies. [Mon 9/20] Search Algorithms - Breath First Search, Depth First Search, Iterative Search and Bi-directional Search. Heuristic search. Greedy Search. A*. Admissible evaluation functions. Monotone evaluation functions. Reading: Russel and Norvig, 2nd Edition: Chapter 4.1-4.4. 3rd Edition: Chapter 3.5-3.6
	Lecture 5: Space Variations in A* [Wed 9/22] Iterative Deepening A*. Recursive Best First Search (RBFS).
	Lecture 6: Time and Hierarchical Variations of A* [Mon 9/27] Guest Lescturer: Hala Mostafa Memory-bounded heuristic search. Simplified Memory Bounded A* (SMA*). Real-time problem solving. Satisficing and Optimizing. Anytime A*. Real-Time A*. Required Reading : Richard E. Korf, Real-Time Heuristic Search, Artificial Intellligence 42 (1990), pp 189-211. Homework 2 assigned
	Lecture 7: Hierarchial & Local Search [Wed 9/29] RTA*. Hierarchical A*. Beginning of Local Search Required Readings: Eric A. Hansen, Shlomo Zilberstein, Victor A. Danilchenko, Anytime Heuristic Search: First Results, CS Technical Report, 97-50, UMASS Hierarchical A*: R.C. Holte, M.B. Perez, R.M.Zimmer, A.J. Macdonald, Hierarchical A*: Searching Abstraction Hierarchies Efficiently, {AAAI}/{IAAI}, Vol. 1, pp 530-535, 1996 Optional Readings: Other Examples of Hierarchical Problem Solving: Craig A. Knoblock, Abstracting the Tower of Hanoi, In Proceedings of the Workshop on Automatic Generation of Approximations and Abstractions, pages 13--23, Boston, MA, 1990
	Lecture 8: Local Search [Mon 10/04] Hill-Climbing, Simulated Annealing, Genetic Search .
	Lecture 9: CSP Algorithms [Wed 10/06] Systematic Search. Backtracking Search. Informed Backtracking. Constraint Propagation. Arc Consistency.
	NO CLASS. Colombus Day. [Mon 10/11]
	Lecture 10: Constraint Satisfaction Problem [Tues 10/12] CSP Heuristics. Informed Backtracking. Advanced Backtracking. Tree Structured CSPs. Subgoal/Subproblem Interactions. Nearly Decomposable Problems.
	Lecture 11: Blackboard Systems [Wed 10/13] Introduction to Blackboard Architectures. Blackboard Problem Solving. Blackboard and Search. Cooperating Experts. Blackboard Applications. Required Reading: Erman, L.D., Hayes-Roth, F., Lesser, V.R., and Reddy, D.R. (1980). The HEARSAY-II Speech Understanding System: Integrating Knowledge to Resolve Uncertainty. Computing Surveys 12, (2), 213-253, 1980. Optional reading: Carver, N. and Lesser, V. The Evolution of Blackboard Control Architectures. Computer Science Technical Report 92-71, University of Massachusetts, Amherst. (This is a revised and extended version of paper with same title in Expert Systems with Applications: Special Issue on the Blackboard Paradigm and Its Applications.)
MIDTERM [Mon 10/18]
midterm 1996 midterm 1998 midterm 2000 solutions 2000 midterm 2002 solutions 2006 Midterm 2008
MARKOV PROCESSES
	Lecture 12: Markov Decision Processes [Wed 10/20] Search with Uncertainty. Introduction to Markov Decision Processes. Goals and Rewards. Performance Criteria. Bellman Equation. Value Iteration. Policy Iteration. Value Determination. Reading: Russel and Norvig, Chapter 17.1-17.4 Homework 2 DUE
	Lecture 13: Partial Observable Markov Decision Processes [Mon 10/25] Greedy Policy v/s Optimal Policy. Policy Iteration. Value Determination. Introduction to SMDP. Introduction to POMDP. Bayesan Policy Representation. Finite-Memory Policies.
	Lecture 14: Decision Making As An Optimization Problem [Wed 10/27] Guest Lecturer: Hala Mostafa Formulating the problem of computing a policy as an optimization program, e.g. Linear, Mixed Integer, Non-linear and Bilinear Programs. Single and multi-agent decision making. Optional Reading: Formal Models and Algorithms for Decentralized Decision Making under Uncertainty by Sven Seuken and Shlomo Zilberstein. An Investigation into Mathematical Programming for Finite Horizon Decentralized POMDPs by Raghav Aras, Alain Dutech.
	Lecture 15: Hidden Markov Models [Mon 11/01] Introduction to Hidden Markov Models. Probabilistic Inference in HMM. Representation for Paths. Viterbi Algorithm. Optional Reading: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition, Sections I-III. Russel and Norvig, Chapter 15.3.
REASONING UNDER UNCERTAINTY
	Lecture 16: Uncertainty in Intelligent Systems [Wed 11/03] Ubiquity of Uncertainty. Sources of Uncertainty. Reasoning under Uncertainty. Acting under Uncertainty. Uncertainty in First-Order Logic. Nonmonotonicity. Belief and Evidence. Probability v/s Causality. MYCIN's Certainty Factors. Probability Theory. Bayesian reasoning. Abductive Uncertainty. Reading: Russel and Norvig, Chapter 13.1 to 13.6
	Lecture 17: Introduction to Probabilistic Reasoning with Belief Networks [Mon 11/08] Review of Lecture 16. Introduction to Belief (or Bayesial) Networks. Conditional Independence in BNs. Semantics of BN. Inference in BNs. Reasoning in BNs. Representation of Conditional Probability Tables. Benefits of BNs. Constructing BNs. d-Seperation. Inference in BNs. Variable Elimination. Reading: Russel and Norvig, Chapter 14.1 to 14.7
	NO CLASS. Thursday's schedule. [Wed 11/10]
	Lecture 18: Approximate inference for BNs. [Mon 11/15] Variable Elimination in Chains. Elimination in Chains with Evidence. Incremental Updating of BNs. Belief Propagation in Trees. Inference in Multiply Connected BNs. Clustering methods. Cutset Conditioning. Reading: Russel and Norvig, Chapter 14.4 - 14.7.
	Lecture 19: Decision Theory [Wed 11/17] False Positive and Negatives. Inference in Multiply Connected BNs. Stochastic Simulation. Likelihood Weighting. Markov Chain Monte Carlo. Markov Blanket. Introduction to Utility Theory. Axioms of Utility Theory. Utility scales and utility assessment. Value of Information. Reading: Russel and Norvig, Chapter 16
	Lecture 20: Simple Decision Networks (Trees) [Mon 11/22] Guest Lecturer: Hala Mostafa Value of Information examples. Value of Perfect Information. Properties of the Value of Information. Decision trees. . Reading: Russel and Norvig, Chapter 18.3
	Lecture 21: More on Decision Networks [Wed 11/24] Introduction to Decision Networks. Nodes in a Decision Network. Knowledge in a Decision Network. Topology of Decision Networks. Evaluating Decision Networks. Evaluation by Graph Reduction. Shachter's Algorithm. Dempster-Shafer Theory. Fuzzy Set Theory/Logic. Truth Maintainance Systems. Reading: Russel and Norvig, Chapter 18.1,2,4,5.
LEARNING
	Lecture 22: Introduction to Learning [Mon 11/29] Definition of Learning. Types of Learned Knowledge. Characterizing Learning Systems. Model of Learning Agents. Dimensions of Learning. Supervised Learning. Ockham's Razor. Decision Trees and Learning.
	Lecture 23: Decision Tree Learning [Wed 12/01] Decision Tree Algorithm for Learning. Choosing the Best Attribute Based on Information Theory. Splitting Examples. Decision Tree Learning. Performance Measurements. Inductive Bias. Overfitting. Missing Data. Multi-Valued Attributes. Continuous Valued Attributes. Intro to Neural Networks. Connectionist Computation. Takeover Midterm Exam Review. Reading: Russel and Norvig, Chapters 19.1-19.5, 20.8.
	Lecture 24: Neural Networks [Mon 12/06] Artificial Neural Networks. Neural Network Learning. Multi-Layer Networks. Perceptron. Perceptron Learning. Gradient Decent. Delta Rule. Approximation to Gradient Decent. Back Propagation. Overfitting. Convergence. Reading: Russel and Norvig, Chapter 20.1-20.6.
	Lecture 25: Reinforcement Learning [Wed 12/08] Problem with Supervised Learning. Intro to Reinforcement Learning. Markov Decision Processes. Key Features of RL. Utility Function. Action-Value Function. Passive versus Active Learning. Learning Utility Functions. Direct Utility Estimation. Adaptive Dynamic Programming. Temporal Difference Learning. Tic-Tac-Toe. Simple Monte Carlo. Limitations. Q Learning for Deterministic Worlds. Non-Deterministic Q Learning.
Questions And Answers
FINAL EXAM ON Th 12/16 AT 4:00pm in Lederle Grad Res Ctr ROOM A301
final 1998 final 2000 final 2002 final 2004 final 2006 final 2008