jmarkov.jmdp
Class DTMDP<S extends State,A extends Action>

java.lang.Object
  extended by jmarkov.jmdp.MDP<S,A>
      extended by jmarkov.jmdp.InfiniteMDP<S,A>
          extended by jmarkov.jmdp.DTMDP<S,A>
Type Parameters:
S - State class
A - Action class
Direct Known Subclasses:
CT2DTConverter, DTMDPEv, DTMDPEvA, StochasticShortestPath

public abstract class DTMDP<S extends State,A extends Action>
extends InfiniteMDP<S,A>

This class represents a discrete time infnite horizon MDP problem. It must be extended in order to represent the appropriate structure for each problem. The user must implement at least the functions that have been declared abstract.

Author:
Andres Sarmiento, Germán Riaño - Universidad de Los Andes
See Also:
PolicyIterationSolver, ValueIterationSolver

Field Summary
 
Fields inherited from class jmarkov.jmdp.InfiniteMDP
absorbingStates, explorationTime, hasAbsorbingState, numStates, probability, probabilitySolver, states
 
Fields inherited from class jmarkov.jmdp.MDP
finite, initial, reporter
 
Constructor Summary
DTMDP(S initial)
          Creates a new infinite horizon discrete time (MDP) Problem.
DTMDP(States<S> initial)
          Creates a new infinite horizon discrete time (MDP) Problem.
 
Method Summary
protected  StatesSet<S> generate()
           
 ValueFunction<S> getSteadyStateProbabilities()
           
abstract  double immediateCost(S i, A a)
          Cost incurred when taking action a from state i
protected  StatesSet<S> oneStageReachable(States<S> initSet)
          TFinds the states reached in one step.
abstract  double prob(S i, S j, A a)
          Probability of going from state i to state j by taking the action a
abstract  States<S> reachable(S i, A a)
          Set of states that can be reached from this state i, after taking the action a.
 void setProbabilitySolver(ProbabilitySolver<S,A> solv)
           
 Solution<S,A> solve(double interestRate)
          Solves the problem with the given interest rate
 
Methods inherited from class jmarkov.jmdp.InfiniteMDP
feasibleActions, getAllStates, getDefaultAverageSolver, getDefaultDiscountedSolver, getDefaultSolver, getNumStates, getSolver, setInterestRate
 
Methods inherited from class jmarkov.jmdp.MDP
debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

DTMDP

public DTMDP(States<S> initial)
Creates a new infinite horizon discrete time (MDP) Problem.

Parameters:
initial - set of initial states for the exploration algorithm

DTMDP

public DTMDP(S initial)
Creates a new infinite horizon discrete time (MDP) Problem.

Parameters:
initial - An initial state for the exploration algorithm
Method Detail

immediateCost

public abstract double immediateCost(S i,
                                     A a)
Cost incurred when taking action a from state i

Parameters:
i - Current State
a - Current Action
Returns:
The cost incurred per transition

prob

public abstract double prob(S i,
                            S j,
                            A a)
Probability of going from state i to state j by taking the action a

Parameters:
i - Current state.
j - Destination State
a - Action
Returns:
The probability.

reachable

public abstract States<S> reachable(S i,
                                    A a)
Set of states that can be reached from this state i, after taking the action a.

Parameters:
i - Current State
a - Action taken
Returns:
The reachable states.

oneStageReachable

protected StatesSet<S> oneStageReachable(States<S> initSet)
TFinds the states reached in one step.

Parameters:
initSet -
Returns:
Set of states reached in one step.

generate

protected StatesSet<S> generate()
Specified by:
generate in class InfiniteMDP<S extends State,A extends Action>
Returns:
The set of states found.

getSteadyStateProbabilities

public ValueFunction<S> getSteadyStateProbabilities()
                                                           throws SolverException
Returns:
a map with the probability for each state.
Throws:
SolverException

setProbabilitySolver

public void setProbabilitySolver(ProbabilitySolver<S,A> solv)
Parameters:
solv - Sets the solver that solves the steady state probabilities.

solve

public final Solution<S,A> solve(double interestRate)
                                                       throws SolverException
Solves the problem with the given interest rate

Parameters:
interestRate - the interest rate parameter to solve the problem.
Returns:
The soultion to the problem.
Throws:
SolverException