jmdp
Class DTMDPEv<S extends State,A extends Action,E extends Event>

java.lang.Object
  extended by jmdp.MDP<S,A>
      extended by jmdp.InfiniteMDP<S,A>
          extended by jmdp.DTMDP<S,A>
              extended by jmdp.DTMDPEv<S,A,E>

public abstract class DTMDPEv<S extends State,A extends Action,E extends Event>
extends DTMDP<S,A>

This class represents an infinite horizon, discrete time, Markov Decision Process with events. It allows the definition of events that can occur in a given state and this makes the cost and probability definition easier to define than in the cases where no events are defined.

Author:
Andres Sarmiento and Germ�n Ria�o. Universidad de los Andes.

Field Summary
 
Fields inherited from class jmdp.InfiniteMDP
absorbingStates, explorationTime, hasAbsorbingState, probability, probabilitySolver, states
 
Fields inherited from class jmdp.MDP
finite, initial, reporter
 
Constructor Summary
DTMDPEv(States<S> initial)
          Creates a new infinite horizon discrete time (MDP) Problem with events
 
Method Summary
abstract  Events<E> activeEvents(S i, A a)
          Set of events that are active from state i given that action a is taken.
 double immediateCost(S i, A a)
          Cost incurred when taking action a from state i
abstract  double immediateCost(S i, A a, E e)
          Reward received when the current state is i, the action taken is a and event e occurs.
abstract  double prob(S i, E e)
          Conditional probability.
 double prob(S i, S j, A a)
          Probability of going from state i to state j by taking the action a
abstract  double prob(S i, S j, A a, E e)
          Conditional probability.
 States<S> reachable(S i, A a)
          Set of states that can be reached from this state i, after taking the action a.
abstract  States<S> reachable(S i, A a, E e)
          Set of reachable states from state i given that action a is taken and event e occurs.
 
Methods inherited from class jmdp.DTMDP
explore, getSteadyStateProbabilities, oneStageReachable, setProbabilitySolver, solve
 
Methods inherited from class jmdp.InfiniteMDP
feasibleActions, getAllStates, getDefaultAverageSolver, getDefaultDiscountedSolver, getDefaultSolver, getSolver, setInterestRate
 
Methods inherited from class jmdp.MDP
debug, getOptimalPolicy, getOptimalValueFunction, getReporter, isFinite, isSolved, operation, printSolution, printSolution, setReporter, setSolver, solve
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

DTMDPEv

public DTMDPEv(States<S> initial)
Creates a new infinite horizon discrete time (MDP) Problem with events

Parameters:
initial - set of initial states for the exploration algorithm
Method Detail

immediateCost

public double immediateCost(S i,
                            A a)
Description copied from class: DTMDP
Cost incurred when taking action a from state i

Specified by:
immediateCost in class DTMDP<S extends State,A extends Action>

prob

public double prob(S i,
                   S j,
                   A a)
Description copied from class: DTMDP
Probability of going from state i to state j by taking the action a

Specified by:
prob in class DTMDP<S extends State,A extends Action>

reachable

public States<S> reachable(S i,
                           A a)
Description copied from class: DTMDP
Set of states that can be reached from this state i, after taking the action a.

Specified by:
reachable in class DTMDP<S extends State,A extends Action>

immediateCost

public abstract double immediateCost(S i,
                                     A a,
                                     E e)
Reward received when the current state is i, the action taken is a and event e occurs.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
reward

prob

public abstract double prob(S i,
                            S j,
                            A a,
                            E e)
Conditional probability. Probability of reaching state j given that the current state is i, the action taken is a and the event that occurs is e.

Parameters:
i - current state
j - state to reach
a - action taken (given)
e - event that occurs (given)
Returns:
Conditional probability

prob

public abstract double prob(S i,
                            E e)
Conditional probability. Probability that event e occurs given that the current state is i.

Parameters:
i - current state
e - event that occurs
Returns:
Conditional probability

reachable

public abstract States<S> reachable(S i,
                                    A a,
                                    E e)
Set of reachable states from state i given that action a is taken and event e occurs.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
set of reachable states.

activeEvents

public abstract Events<E> activeEvents(S i,
                                       A a)
Set of events that are active from state i given that action a is taken.

Parameters:
i - current state
a - action taken
Returns:
set of events that can occur