jmdp
Class DTMDP<S extends State,A extends Action>

java.lang.Object
  extended by jmdp.MDP<S,A>
      extended by jmdp.InfiniteMDP<S,A>
          extended by jmdp.DTMDP<S,A>
Direct Known Subclasses:
CT2DTConverter, DTMDPEv, DTMDPEvA, StochasticShortestPath

public abstract class DTMDP<S extends State,A extends Action>
extends InfiniteMDP<S,A>

This class represents a discrete time INFINITE horizon MDP Problems. It must be extended in order to represent the appropriate structure for each INFINITE horizon MDP problem. The user must implement at least the functions that have been declared abstract. It�s also necessary to create one of the extensions of the class Solver. By default, the program includes PolicyIterationSolver and ValueIterationSolver classes to solve infinite horizon problems. The FiniteSolver class is only for finite horizon problems. To solve the problem follow the instructions in each of the solvers� instructions.

Author:
Andres Sarmiento, German Riano - Universidad de Los Andes
See Also:
PolicyIterationSolver, ValueIterationSolver

Field Summary
 
Fields inherited from class jmdp.InfiniteMDP
absorbingStates, explorationTime, hasAbsorbingState, probability, probabilitySolver, states
 
Fields inherited from class jmdp.MDP
finite, initial, reporter
 
Constructor Summary
DTMDP(States<S> initial)
          Creates a new infinite horizon discrete time (MDP) Problem.
 
Method Summary
protected  States<S> explore(States<S> initSet)
           
 ValueFunction<S> getSteadyStateProbabilities()
           
abstract  double immediateCost(S i, A a)
          Cost incurred when taking action a from state i
protected  States<S> oneStageReachable(States<S> initSet)
           
abstract  double prob(S i, S j, A a)
          Probability of going from state i to state j by taking the action a
abstract  States<S> reachable(S i, A a)
          Set of states that can be reached from this state i, after taking the action a.
 void setProbabilitySolver(ProbabilitySolver<S,A> solv)
           
 Solution<S,A> solve(double interestRate)
          Solves the problem with the given interest rate
 
Methods inherited from class jmdp.InfiniteMDP
feasibleActions, getAllStates, getDefaultAverageSolver, getDefaultDiscountedSolver, getDefaultSolver, getSolver, setInterestRate
 
Methods inherited from class jmdp.MDP
debug, getOptimalPolicy, getOptimalValueFunction, getReporter, isFinite, isSolved, operation, printSolution, printSolution, setReporter, setSolver, solve
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

DTMDP

public DTMDP(States<S> initial)
Creates a new infinite horizon discrete time (MDP) Problem.

Parameters:
initial - set of initial states for the exploration algorithm
Method Detail

immediateCost

public abstract double immediateCost(S i,
                                     A a)
Cost incurred when taking action a from state i


prob

public abstract double prob(S i,
                            S j,
                            A a)
Probability of going from state i to state j by taking the action a


reachable

public abstract States<S> reachable(S i,
                                    A a)
Set of states that can be reached from this state i, after taking the action a.


oneStageReachable

protected States<S> oneStageReachable(States<S> initSet)

explore

protected States<S> explore(States<S> initSet)
Specified by:
explore in class InfiniteMDP<S extends State,A extends Action>

getSteadyStateProbabilities

public ValueFunction<S> getSteadyStateProbabilities()
Returns:
a map with the probability for each state.

setProbabilitySolver

public void setProbabilitySolver(ProbabilitySolver<S,A> solv)
Parameters:
solv - Sets the solver that solves the steady state probabilities.

solve

public Solution<S,A> solve(double interestRate)
Solves the problem with the given interest rate

Parameters:
interestRate - the interest rate parameter to solve the problem.