jmdp
Class CTMDPEv<S extends State,A extends Action,E extends Event>

java.lang.Object
  extended by jmdp.MDP<S,A>
      extended by jmdp.InfiniteMDP<S,A>
          extended by jmdp.CTMDP<S,A>
              extended by jmdp.CTMDPEv<S,A,E>

public abstract class CTMDPEv<S extends State,A extends Action,E extends Event>
extends CTMDP<S,A>

This class represents an Infinite horizon, continuous time Markov Decision Process with events. It allows the definition of events that can occur in a given state and this makes the reward and probability definition easier than in the cases where no events are defined.

Author:
Andres Sarmiento and Germ�n Ria�o. Universidad de los Andes.

Field Summary
 
Fields inherited from class jmdp.CTMDP
activeState, exitRates, maxRate
 
Fields inherited from class jmdp.InfiniteMDP
absorbingStates, explorationTime, hasAbsorbingState, probability, probabilitySolver, states
 
Fields inherited from class jmdp.MDP
finite, initial, reporter
 
Constructor Summary
CTMDPEv(States<S> initial)
          This constructor builds a continuous time MDP with events.
 
Method Summary
abstract  Events<E> activeEvents(S i, A a)
          Set of events that are active from state i given that action a is taken.
 double continuousCost(S i, A a)
          Reward obtained continuously in time until the next transition from state i given that action a is taken.
abstract  double continuousCost(S i, A a, E e)
          Reward obtained continuously in time during the sojourn time in state i until an action a is taken and a transition is triggered.
 double lumpCost(S i, A a)
          Reward instantaneously in the moment when action a is taken from state i.
abstract  double lumpCost(S i, A a, E e)
          Reward instantaneously gained in the moment when action a is taken from state i.
 double rate(S i, S j, A a)
          Rate of going from state i to state j by taking the action a
abstract  double rate(S i, S j, A a, E e)
          Rate.
 States<S> reachable(S i, A a)
           
abstract  States<S> reached(S i, A a, E e)
          Set of reachable states from state i given that action a is taken and event e occurs.
 
Methods inherited from class jmdp.CTMDP
exitRate, explore, getAllStates, getMaxRate, getSteadyStateProbabilities, oneStageReachable, reached, setConverter, solve
 
Methods inherited from class jmdp.InfiniteMDP
feasibleActions, getDefaultAverageSolver, getDefaultDiscountedSolver, getDefaultSolver, getSolver, setInterestRate
 
Methods inherited from class jmdp.MDP
debug, getOptimalPolicy, getOptimalValueFunction, getReporter, isFinite, isSolved, operation, printSolution, printSolution, setReporter, setSolver, solve
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

CTMDPEv

public CTMDPEv(States<S> initial)
This constructor builds a continuous time MDP with events.

Parameters:
initial - set of initial states for the exploration algorithm
Method Detail

lumpCost

public double lumpCost(S i,
                       A a)
Description copied from class: CTMDP
Reward instantaneously in the moment when action a is taken from state i.

Specified by:
lumpCost in class CTMDP<S extends State,A extends Action>

continuousCost

public double continuousCost(S i,
                             A a)
Description copied from class: CTMDP
Reward obtained continuously in time until the next transition from state i given that action a is taken.

Specified by:
continuousCost in class CTMDP<S extends State,A extends Action>

rate

public double rate(S i,
                   S j,
                   A a)
Description copied from class: CTMDP
Rate of going from state i to state j by taking the action a

Specified by:
rate in class CTMDP<S extends State,A extends Action>

reachable

public States<S> reachable(S i,
                           A a)

rate

public abstract double rate(S i,
                            S j,
                            A a,
                            E e)
Rate. Rate of going of reaching state j given that the current state is i, the action taken is a and the event that occurs is e.

Parameters:
i - current state
j - state to reach
a - action taken (given)
e - event that occurs (given)
Returns:
Rate

reached

public abstract States<S> reached(S i,
                                  A a,
                                  E e)
Set of reachable states from state i given that action a is taken and event e occurs.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
set of reachable states.

activeEvents

public abstract Events<E> activeEvents(S i,
                                       A a)
Set of events that are active from state i given that action a is taken.

Parameters:
i - current state
a - action taken
Returns:
set of events that can occur

lumpCost

public abstract double lumpCost(S i,
                                A a,
                                E e)
Reward instantaneously gained in the moment when action a is taken from state i.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
instanteneous reward.

continuousCost

public abstract double continuousCost(S i,
                                      A a,
                                      E e)
Reward obtained continuously in time during the sojourn time in state i until an action a is taken and a transition is triggered.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
instanteneous reward.