jmarkov.jmdp
Class FiniteMDP<S extends State,A extends Action>

java.lang.Object
  extended by jmarkov.jmdp.MDP<S,A>
      extended by jmarkov.jmdp.FiniteMDP<S,A>
Type Parameters:
S - States class
A - Actions class
Direct Known Subclasses:
FiniteDP, FiniteMDPEv

public abstract class FiniteMDP<S extends State,A extends Action>
extends MDP<S,A>

This class should ONLY be used in FINITE horizon problems. It must be extended in order to represent the appropriate structure for each FINITE Dynamic Programming problem. The user must implement at least the functions that have been declared abstract. It´s also necessary to create one of the extensions of the class Solver. By default, the program includes the FiniteSolver class to solve finite horizon problems. PolicyIterationSolver and ValueIterationSolver are only for infinite horizon problems. To solve the problem follow the instructions in each of the solvers´ instructions.

Author:
Andres Sarmiento, Germán Riaño - Universidad de Los Andes
See Also:
FiniteSolver

Field Summary
protected  int horizon
          Time horizon.
 
Fields inherited from class jmarkov.jmdp.MDP
finite, initial, reporter
 
Constructor Summary
FiniteMDP(S initial, int horizon)
          Creates a finite horizon MDP.
FiniteMDP(States<S> initial, int horizon)
          Creates a new FINITE horizon (MDP) Problem.
 
Method Summary
abstract  Actions<A> feasibleActions(S i, int t)
          Returns the actions available at this state i and at this stage t .
 double finalCost(S i)
          This function returns the cost incurred if the last stage ends with the system at state i.
protected  Solver<S,A> getDefaultSolver()
          The class that extends MDP must define the default solver to use.
 int getHorizon()
          Returns the time lastStage
 States<S> getStates(int t)
          All the states that are available at stage t.
abstract  double immediateCost(S i, A a, int t)
          This funtion must return the Immediate cost incurred when taking action a from state i
abstract  double prob(S i, S j, A a, int t)
          This is the probability of going from state i to state j by taking the action a at stage t.
abstract  States<S> reachable(S i, A a, int t)
          Set of States that can be reached from this state i, at this stage t, after taking the acton a.
protected  void setHorizon(int T)
          Sets the time lastStage at which decisions can be taken
 
Methods inherited from class jmarkov.jmdp.MDP
debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, getSolver, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

horizon

protected int horizon
Time horizon. The last stage in the problem.

Constructor Detail

FiniteMDP

public FiniteMDP(States<S> initial,
                 int horizon)
Creates a new FINITE horizon (MDP) Problem.

Parameters:
initial - set of initial states
horizon - last stage at which actions can be taken

FiniteMDP

public FiniteMDP(S initial,
                 int horizon)
Creates a finite horizon MDP.

Parameters:
initial - a initial state
horizon - horizon.
Method Detail

getDefaultSolver

protected Solver<S,A> getDefaultSolver()
Description copied from class: MDP
The class that extends MDP must define the default solver to use.

Specified by:
getDefaultSolver in class MDP<S extends State,A extends Action>
Returns:
the solver to use for this problem.

immediateCost

public abstract double immediateCost(S i,
                                     A a,
                                     int t)
This funtion must return the Immediate cost incurred when taking action a from state i

Parameters:
i - Current state
a - Action
t - Current time stage
Returns:
Cost value

prob

public abstract double prob(S i,
                            S j,
                            A a,
                            int t)
This is the probability of going from state i to state j by taking the action a at stage t.

Parameters:
i - Current state
j - Destination state
a - Action taken
t - Current time stage
Returns:
Probability

reachable

public abstract States<S> reachable(S i,
                                    A a,
                                    int t)
Set of States that can be reached from this state i, at this stage t, after taking the acton a. The user must implement this method.

Parameters:
i - Current state
a - Action taken
t - Time stage
Returns:
Set of reachable states.

getStates

public States<S> getStates(int t)
All the states that are available at stage t.

Parameters:
t - time stage
Returns:
States available at stage t.

finalCost

public double finalCost(S i)
This function returns the cost incurred if the last stage ends with the system at state i. The user may extend this method.

Parameters:
i - Ending state
Returns:
Cost.

feasibleActions

public abstract Actions<A> feasibleActions(S i,
                                           int t)
Returns the actions available at this state i and at this stage t . The user must implement this method.

Parameters:
i - Current State
t - Time stage
Returns:
Set of feasible actions.

setHorizon

protected void setHorizon(int T)
Sets the time lastStage at which decisions can be taken


getHorizon

public int getHorizon()
Returns the time lastStage

Returns:
Time horizon