jmarkov.jmdp
Class CTMDPEv<S extends State,A extends Action,E extends Event>

java.lang.Object
  jmarkov.jmdp.MDP<S,A>
      jmarkov.jmdp.InfiniteMDP<S,A>
          jmarkov.jmdp.CTMDP<S,A>
              jmarkov.jmdp.CTMDPEv<S,A,E>

Type Parameters:

S - The States class

A - Tha Action class

E - the Events class

public abstract class CTMDPEv<S extends State,A extends Action,E extends Event>
extends CTMDP<S,A>

This class represents an Infinite horizon, continuous time Markov Decision Process with events. It allows the definition of events that can occur in a given state and this makes the reward and probability definition easier than in the cases where no events are defined.

Author:

Andres Sarmiento and Germ�n Ria�o. Universidad de los Andes.

Field Summary

Fields inherited from class jmarkov.jmdp.CTMDP

activeState, converter, maxRate

Fields inherited from class jmarkov.jmdp.InfiniteMDP

absorbingStates, explorationTime, hasAbsorbingState, numStates, probability, probabilitySolver, states

Fields inherited from class jmarkov.jmdp.MDP

finite, initial, reporter

Constructor Summary

CTMDPEv(States<S> initial)
This constructor builds a continuous time MDP with events.

Method Summary

abstract Events<E>	activeEvents(S i, A a) Set of events that are active from state i given that action a is taken.
double	continuousCost(S i, A a) Cost incurred continuously in time until the next transition from state i given that action a is taken.
abstract double	continuousCost(S i, A a, E e) Reward obtained continuously in time during the sojourn time in state i until an action a is taken and a transition is triggered.
double	lumpCost(S i, A a) Cost incurred instantaneously in the moment when action a is taken from state i.
abstract double	lumpCost(S i, A a, E e) Reward instantaneously gained in the moment when action a is taken from state i.
double	rate(S i, S j, A a) Rate of going from state i to state j by taking the action a
abstract double	rate(S i, S j, A a, E e) Rate.
States<S>	reachable(S i, A a) Set of States that can be reached from this state i, after taking the action a.
abstract States<S>	reachable(S i, A a, E e) Set of reachable states from state i given that action a is taken and event e occurs.

Methods inherited from class jmarkov.jmdp.CTMDP

generate, getAllStates, getMaxRate, getSteadyStateProbabilities, oneStageReachable, setConverter, solve

Methods inherited from class jmarkov.jmdp.InfiniteMDP

feasibleActions, getDefaultAverageSolver, getDefaultDiscountedSolver, getDefaultSolver, getNumStates, getSolver, setInterestRate

Methods inherited from class jmarkov.jmdp.MDP

debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CTMDPEv

public CTMDPEv(States<S> initial)

This constructor builds a continuous time MDP with events.

Parameters:

initial - set of initial states for the exploration algorithm

Method Detail

lumpCost

public final double lumpCost(S i,
                             A a)

Description copied from class: CTMDP

Cost incurred instantaneously in the moment when action a is taken from state i.

Specified by:

lumpCost in class CTMDP<S extends State,A extends Action>

Parameters:

i - State

a - Action

Returns:

Lump cost received.

continuousCost

public final double continuousCost(S i,
                                   A a)

Description copied from class: CTMDP

Cost incurred continuously in time until the next transition from state i given that action a is taken.

Specified by:

continuousCost in class CTMDP<S extends State,A extends Action>

Parameters:

i - State

a - Action

Returns:

Rate at which cost is incurred when action a is taken.

rate

public final double rate(S i,
                         S j,
                         A a)

Description copied from class: CTMDP

Rate of going from state i to state j by taking the action a

Specified by:

rate in class CTMDP<S extends State,A extends Action>

Parameters:

i - current state

j - Destination state.

a - Action taken

Returns:

The rate

reachable

public final States<S> reachable(S i,
                                 A a)

Description copied from class: CTMDP

Set of States that can be reached from this state i, after taking the action a.

Specified by:

reachable in class CTMDP<S extends State,A extends Action>

Parameters:

i - current State

a - action taken

Returns:

the reachable states.

rate

public abstract double rate(S i,
                            S j,
                            A a,
                            E e)

Rate. Rate of going of reaching state j given that the current state is i, the action taken is a and the event that occurs is e.

Parameters:

i - current state

j - state to reach

a - action taken (given)

e - event that occurs (given)

Returns:

Rate

reachable

public abstract States<S> reachable(S i,
                                    A a,
                                    E e)

Set of reachable states from state i given that action a is taken and event e occurs.

Parameters:

i - current state

a - action taken

e - event that occurs

Returns:

set of reachable states.

activeEvents

public abstract Events<E> activeEvents(S i,
                                       A a)

Set of events that are active from state i given that action a is taken.

Parameters:

i - current state

a - action taken

Returns:

set of events that can occur

lumpCost

public abstract double lumpCost(S i,
                                A a,
                                E e)

Reward instantaneously gained in the moment when action a is taken from state i.

Parameters:

i - current state

a - action taken

e - event that occurs

Returns:

instanteneous reward.

continuousCost

public abstract double continuousCost(S i,
                                      A a,
                                      E e)

Reward obtained continuously in time during the sojourn time in state i until an action a is taken and a transition is triggered.

Parameters:

i - current state

a - action taken

e - event that occurs

Returns:

instanteneous reward.