³éÏó¶¯Ì¬¹æ»®(µÚ2°æ)

1 Introduction
1£®1 Structure ofDynamic Programming Problems
1£®2 Abstract Dynamic Programming Models
1£®2£®1 Problem Formulation
1£®2£®2 Monotonicity and Contraction Properties
1£®2£®3 Some Examples
1£®2£®4 Approximation Models-Projected and Aggregation Bellman Equations
1£®2£®5 Multistep Models-Temporal Difference and ProximalAlgorithms
1£®3 Organizationofthe Book
1£®4 Notes£¬ Sources£¬ and Exercises

2 Contractive Models
2£®1 Bellman's Equation and Optimality Conditions
2£®2 Limited Lookahead Policies
2£®3 Value Iteration
2£®4 Policylteration
2£®4£®1 Approximate Policylteration
2£®4£®2 Approximate Policy Iteration Where Policies Converge
2£®5 Optimistic Policylteration and A-Policylteration
2£®5£®1 Convergence ofOptimistic Policylteration
2£®5£®2 Approximate Optimistic Policylteration
2£®5£®3 Randomized Optimistic Policylteration
2£®6 Asynchronous Algorithms
2£®6£®1 AsynchronousValuelteration
2£®6£®2 AsynchronousPolicylteration
2£®6£®3 Optimistic Asynchronous Policy Iteration with a Uniform Fixed Point
2£®7 Notes£¬ Sources£¬ and Exercises

3 Semicontractive Models
3£®1 Pathologies of Noncontractive DP Models
3£®1£®1 Deterministic Shortest Path Problems
3£®1£®2 Stochastic Shortest Path Problems
3£®1£®3 The Blackmailer's Dilemma
3£®1£®4 Linear-QuadraticProblems
3£®1£®5 An Intuitive View of Semicontractive Analysis
3£®2 Semicontractive Models and Regular Policies
3£®2£®1 S-Regular Policies
3£®2£®2 Restricted Optimization over S-Regular Policies
3£®2£®3 Policy Iteration Analysis of Bellman's Equation
3£®2£®4 Optimistic Policy Iteration and A-Policy Iteration
3£®2£®5 A MathematicalProgrammingApproach
3£®3 Irregular Policies/lnfinite Cost Case
3£®4 Irregular Policies/Finite Cost Case-A Perturbation Approach
3£®5 Applications in Shortest Path and Other Contexts
3£®5£®1 Stochastic Shortest Path Problems
3£®5£®2 Affine Monotonic Problems
3£®5£®3 Robust Shortest Path Planning
3£®5£®4 Linear-QuadraticOptimalControl
3£®5£®5 Continuous-State Deterministic Optimal Control
3£®6 Algorithms
3£®6£®1 AsynchronousValuelteration
3£®6£®2 Asynchronous Policylteration
3£®7 Notes£¬ Sources£¬ and Exercises

4 Noncontractive Models
4£®1 Noncontractive Models-Problem Formulation
4£®2 Finite Horizon Problems
4£®3 Infinite Horizon Problems
4£®3£®1 Fixed Point Properties and Optimality Conditions
4£®3£®2 Value Iteration
4£®3£®3 Exact and Optimistic Policy Iteration-A-Policylteration
4£®4 Regularity and Nonstationary Policies
4£®4£®1 Regularity and Monotonelncreasing Models
4£®4£®2 Nonnegative Cost Stochastic Optimal Control
4£®4£®3 Discounted Stochastic OptimalControl
4£®4£®4 Convergent Models
4£®5 Stable Policies for Deterministic Optimal Control
¡­¡­
Appendix A£º Notation and Mathematical Conventions
Appendix B£º Contraction Mappings£®
References
Index£®£®