Can sequential decision analytics be taught at optimal dynamics?
I then briefly illustrate how these ideas are used at Optimal Dynamics.
Click here for slides to Part II.
I believe that sequential decision analytics is a field that should be taught alongside courses in optimization and machine learning.
This can be done in several ways:.
,
Designing Policies
We are going to search over policies by standing on the shoulders of the entire body of research in the jungle.
However, rather than say something useless like “try everything in all those books,” we are going to observe that every approach can be divided into two broad classes (the policy search class, and the lookahead class), each of which can b.
,
How do you solve a sequential decision problem?
There are four phases to addressing real-world sequential decision problems:
- Phase I:
- Identifying the core elements of the problem (without mathematics):
- Performance metrics – Costs
- profits
- yield
- productivity
- customer service
- employee performance
- … Decisions – What types of decisions are being made
- who makes them
,
Introduction
Sequential decision analytics is an umbrella for a vast range of problems that consist of the sequence: decision, information, decision, information, decision, information, … Our standard model will extend over a finite horizon, but other settings might have just one decision (decision, information, stop), two decisions (decision, information, deci.
,
Introductory Tutorial
For an introduction to the unified framework: I revised and re-recorded a tutorial on the unified framework that I gave at the Kellogg School of Business at Northwestern (February 2020).
It is now on youtube in two parts: 1. 1.1.
Part I – Describes applications, the universal modeling framework, and introduces the four classes of policies for makin.
,
Modeling Sequential Decision Problems
To understand how we are going to model sequential decision problems, it helps to take a brief tour through two other major problem classes: deterministic optimization and machine learning.
We are then going to argue that our approach to stochastic optimization closely parallels the approaches these fields use.
,
Teaching Sequential Decision Analytics
I believe that sequential decision analytics is a field that should be taught alongside courses in optimization and machine learning.
This can be done in several ways: An undergraduate course – I taught a course at Princeton spanning a wide range of applications, using a teach by example style.
Lecture notes and an online book are available here.
H.
,
The Classes of Policies and The Jungle
Each of the communities in stochastic optimization started with one algorithmic strategy, often motivated by a small class of applications that fit the method.
However, as time has passed and the communities have grown, so have the range of problem classes they are working on.
Then, as the characteristics of the problems have evolved, the communiti.
,
Tuning Policies
While the academic research community has focused most of their attention on policies in the lookahead class, the real world has primarily been using policies in the policy search class (with some use of deterministic DLAs, but these can be parameterized as well).
Policy search policies are simpler, which suggests they are somehow less interesting,.
,
What is a stopping rule in sequential decision making?
The procedure to decide when to stop taking observations and when to continue is called the ‘stopping rule.’ The objective in sequential decision making is to find a stopping rule that optimizes the decision in terms of minimizing losses or maximizing gains, including:
- observation costs
,
What is sequential decision making?
A.
Diederich, in International Encyclopedia of the Social & Behavioral Sciences, 2001 Sequential decision making describes a situation where the decision maker (DM) makes successive observations of a process before a final decision is made.
In most sequential decision problems there is an implicit or explicit cost associated with each observation.
