jmarkov.jmdp
Class FiniteDP<S extends State,A extends Action>

java.lang.Object
  extended by jmarkov.jmdp.MDP<S,A>
      extended by jmarkov.jmdp.FiniteMDP<S,A>
          extended by jmarkov.jmdp.FiniteDP<S,A>
Type Parameters:
S - States class
A - Actions class

public abstract class FiniteDP<S extends State,A extends Action>
extends FiniteMDP<S,A>

This class should ONLY be used in FINITE horizondeterministic problems. It must be extended in order to represent the appropriate structure for each FINITE Dynamic Programming problem. The user must implement at least the functions that have been declared abstract. It´s also necessary to create one of the extensions of the class Solver. By default, the program includes the FiniteSolver class to solve finite horizon problems. PolicyIterationSolver and ValueIterationSolver are only for infinite horizon problems. To solve the problem follow the instructions in each of the solvers´ instructions.

Author:
Andres Sarmiento, Germán Riaño - Universidad de Los Andes
See Also:
FiniteSolver

Field Summary
 
Fields inherited from class jmarkov.jmdp.FiniteMDP
horizon
 
Fields inherited from class jmarkov.jmdp.MDP
finite, initial, reporter
 
Constructor Summary
FiniteDP(States<S> initial, int lastStage)
          Creates a new FINITE Dynamic Programming (DP) Problem.
 
Method Summary
abstract  S destination(S i, A a, int t)
          State where the system will end up if action a is taken from state i at time t.
 double prob(S i, S j, A a, int t)
          Final function must not be extended or implementes by any user.
 States<S> reachable(S i, A a, int t)
          Final function must not be extended by any user.
 
Methods inherited from class jmarkov.jmdp.FiniteMDP
feasibleActions, finalCost, getDefaultSolver, getHorizon, getStates, immediateCost, setHorizon
 
Methods inherited from class jmarkov.jmdp.MDP
debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, getSolver, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

FiniteDP

public FiniteDP(States<S> initial,
                int lastStage)
Creates a new FINITE Dynamic Programming (DP) Problem.

Parameters:
initial - initial set of known states.
lastStage - number of the last stage.
Method Detail

prob

public final double prob(S i,
                         S j,
                         A a,
                         int t)
Final function must not be extended or implementes by any user.

Specified by:
prob in class FiniteMDP<S extends State,A extends Action>
Parameters:
i - Current state
j - Destination state
a - Action taken
t - Current time stage
Returns:
Probability

reachable

public final States<S> reachable(S i,
                                 A a,
                                 int t)
Final function must not be extended by any user.

Specified by:
reachable in class FiniteMDP<S extends State,A extends Action>
Parameters:
i - Current state
a - Action taken
t - Time stage
Returns:
Set of reachable states.

destination

public abstract S destination(S i,
                              A a,
                              int t)
State where the system will end up if action a is taken from state i at time t. The user must implement this method.

Parameters:
i - Current state
a - Current action
t - Time stage.
Returns:
Destination states