jmarkov.jmdp
Class CTMDPEv<S extends State,A extends Action,E extends Event>

java.lang.Object
  extended by jmarkov.jmdp.MDP<S,A>
      extended by jmarkov.jmdp.InfiniteMDP<S,A>
          extended by jmarkov.jmdp.CTMDP<S,A>
              extended by jmarkov.jmdp.CTMDPEv<S,A,E>
Type Parameters:
S - The States class
A - Tha Action class
E - the Events class

public abstract class CTMDPEv<S extends State,A extends Action,E extends Event>
extends CTMDP<S,A>

This class represents an Infinite horizon, continuous time Markov Decision Process with events. It allows the definition of events that can occur in a given state and this makes the reward and probability definition easier than in the cases where no events are defined.

Author:
Andres Sarmiento and Germ�n Ria�o. Universidad de los Andes.

Field Summary
 
Fields inherited from class jmarkov.jmdp.CTMDP
activeState, converter, maxRate
 
Fields inherited from class jmarkov.jmdp.InfiniteMDP
absorbingStates, explorationTime, hasAbsorbingState, numStates, probability, probabilitySolver, states
 
Fields inherited from class jmarkov.jmdp.MDP
finite, initial, reporter
 
Constructor Summary
CTMDPEv(States<S> initial)
          This constructor builds a continuous time MDP with events.
 
Method Summary
abstract  Events<E> activeEvents(S i, A a)
          Set of events that are active from state i given that action a is taken.
 double continuousCost(S i, A a)
          Cost incurred continuously in time until the next transition from state i given that action a is taken.
abstract  double continuousCost(S i, A a, E e)
          Reward obtained continuously in time during the sojourn time in state i until an action a is taken and a transition is triggered.
 double lumpCost(S i, A a)
          Cost incurred instantaneously in the moment when action a is taken from state i.
abstract  double lumpCost(S i, A a, E e)
          Reward instantaneously gained in the moment when action a is taken from state i.
 double rate(S i, S j, A a)
          Rate of going from state i to state j by taking the action a
abstract  double rate(S i, S j, A a, E e)
          Rate.
 States<S> reachable(S i, A a)
          Set of States that can be reached from this state i, after taking the action a.
abstract  States<S> reachable(S i, A a, E e)
          Set of reachable states from state i given that action a is taken and event e occurs.
 
Methods inherited from class jmarkov.jmdp.CTMDP
generate, getAllStates, getMaxRate, getSteadyStateProbabilities, oneStageReachable, setConverter, solve
 
Methods inherited from class jmarkov.jmdp.InfiniteMDP
feasibleActions, getDefaultAverageSolver, getDefaultDiscountedSolver, getDefaultSolver, getNumStates, getSolver, setInterestRate
 
Methods inherited from class jmarkov.jmdp.MDP
debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

CTMDPEv

public CTMDPEv(States<S> initial)
This constructor builds a continuous time MDP with events.

Parameters:
initial - set of initial states for the exploration algorithm
Method Detail

lumpCost

public final double lumpCost(S i,
                             A a)
Description copied from class: CTMDP
Cost incurred instantaneously in the moment when action a is taken from state i.

Specified by:
lumpCost in class CTMDP<S extends State,A extends Action>
Parameters:
i - State
a - Action
Returns:
Lump cost received.

continuousCost

public final double continuousCost(S i,
                                   A a)
Description copied from class: CTMDP
Cost incurred continuously in time until the next transition from state i given that action a is taken.

Specified by:
continuousCost in class CTMDP<S extends State,A extends Action>
Parameters:
i - State
a - Action
Returns:
Rate at which cost is incurred when action a is taken.

rate

public final double rate(S i,
                         S j,
                         A a)
Description copied from class: CTMDP
Rate of going from state i to state j by taking the action a

Specified by:
rate in class CTMDP<S extends State,A extends Action>
Parameters:
i - current state
j - Destination state.
a - Action taken
Returns:
The rate

reachable

public final States<S> reachable(S i,
                                 A a)
Description copied from class: CTMDP
Set of States that can be reached from this state i, after taking the action a.

Specified by:
reachable in class CTMDP<S extends State,A extends Action>
Parameters:
i - current State
a - action taken
Returns:
the reachable states.

rate

public abstract double rate(S i,
                            S j,
                            A a,
                            E e)
Rate. Rate of going of reaching state j given that the current state is i, the action taken is a and the event that occurs is e.

Parameters:
i - current state
j - state to reach
a - action taken (given)
e - event that occurs (given)
Returns:
Rate

reachable

public abstract States<S> reachable(S i,
                                    A a,
                                    E e)
Set of reachable states from state i given that action a is taken and event e occurs.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
set of reachable states.

activeEvents

public abstract Events<E> activeEvents(S i,
                                       A a)
Set of events that are active from state i given that action a is taken.

Parameters:
i - current state
a - action taken
Returns:
set of events that can occur

lumpCost

public abstract double lumpCost(S i,
                                A a,
                                E e)
Reward instantaneously gained in the moment when action a is taken from state i.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
instanteneous reward.

continuousCost

public abstract double continuousCost(S i,
                                      A a,
                                      E e)
Reward obtained continuously in time during the sojourn time in state i until an action a is taken and a transition is triggered.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
instanteneous reward.