jmarkov.jmdp
Class InfiniteMDP<S extends State,A extends Action>

java.lang.Object
  extended by jmarkov.jmdp.MDP<S,A>
      extended by jmarkov.jmdp.InfiniteMDP<S,A>
Type Parameters:
S - States class.
A - Actions class.
Direct Known Subclasses:
CTMDP, DTMDP

public abstract class InfiniteMDP<S extends State,A extends Action>
extends MDP<S,A>

This class is a structural class and is. It represents a general Infinite horizon MDP problem. It is extended for discrte and continuous problems.

Author:
Andres Sarmiento, Germán Riaño - Universidad de Los Andes
See Also:
PolicyIterationSolver, ValueIterationSolver

Field Summary
protected  States<S> absorbingStates
          Set of absorving states.
protected  long explorationTime
          Time used to explore the system.
protected  boolean hasAbsorbingState
          Whether an absorving state was found
protected  int numStates
          Number of states.
protected  ValueFunction<S> probability
          The value function
protected  ProbabilitySolver<S,A> probabilitySolver
          Idiotic solver that was of course not needed since JMarkov can handle the job
protected  StatesSet<S> states
          set of states
 
Fields inherited from class jmarkov.jmdp.MDP
finite, initial, reporter
 
Constructor Summary
InfiniteMDP(States<S> initial)
          Creates a new INFINITE Dynamic Programming (DP) Problem.
 
Method Summary
abstract  Actions<A> feasibleActions(S i)
          Returns the set of actions available at this state.
protected abstract  StatesSet<S> generate()
           
 StatesSet<S> getAllStates()
          Complete set of states explored
protected  AbstractAverageSolver<S,A> getDefaultAverageSolver()
           
protected  AbstractDiscountedSolver<S,A> getDefaultDiscountedSolver(double interestRate)
           
protected  AbstractInfiniteSolver<S,A> getDefaultSolver()
          The class that extends MDP must define the default solver to use.
 int getNumStates()
          Returns the number of states in the model.
 AbstractInfiniteSolver<S,A> getSolver()
           
protected  void setInterestRate(double interestRate)
          Sets the interest rate to be used in the problem solving if the objective is to minimze the discounted cost.
 
Methods inherited from class jmarkov.jmdp.MDP
debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

states

protected StatesSet<S extends State> states
set of states


probability

protected ValueFunction<S extends State> probability
The value function


probabilitySolver

protected ProbabilitySolver<S extends State,A extends Action> probabilitySolver
Idiotic solver that was of course not needed since JMarkov can handle the job


hasAbsorbingState

protected boolean hasAbsorbingState
Whether an absorving state was found


absorbingStates

protected States<S extends State> absorbingStates
Set of absorving states.


explorationTime

protected long explorationTime
Time used to explore the system.


numStates

protected int numStates
Number of states. Set when calling generate.

Constructor Detail

InfiniteMDP

public InfiniteMDP(States<S> initial)
Creates a new INFINITE Dynamic Programming (DP) Problem.

Parameters:
initial - set of initial states for the exploration algorithm
Method Detail

feasibleActions

public abstract Actions<A> feasibleActions(S i)
Returns the set of actions available at this state.

Parameters:
i - Current State
Returns:
set of Actions that can be taken at this state.

getNumStates

public final int getNumStates()
Returns the number of states in the model. It causes the model to be generated.

Returns:
The number of ststes in the system.

generate

protected abstract StatesSet<S> generate()
Returns:
The set of states found.

setInterestRate

protected void setInterestRate(double interestRate)
Sets the interest rate to be used in the problem solving if the objective is to minimze the discounted cost.

Parameters:
interestRate - effective interest rate

getDefaultDiscountedSolver

protected AbstractDiscountedSolver<S,A> getDefaultDiscountedSolver(double interestRate)
See Also:
MDP.getDefaultSolver()

getDefaultAverageSolver

protected AbstractAverageSolver<S,A> getDefaultAverageSolver()
See Also:
MDP.getDefaultSolver()

getDefaultSolver

protected AbstractInfiniteSolver<S,A> getDefaultSolver()
Description copied from class: MDP
The class that extends MDP must define the default solver to use.

Specified by:
getDefaultSolver in class MDP<S extends State,A extends Action>
Returns:
the solver to use for this problem.

getAllStates

public StatesSet<S> getAllStates()
Complete set of states explored

Returns:
set of states explored

getSolver

public AbstractInfiniteSolver<S,A> getSolver()
Overrides:
getSolver in class MDP<S extends State,A extends Action>
Returns:
solver of the current problem