jmdp
Class CTMDPEvA<S extends State,A extends Action,E extends Event>

java.lang.Object
  extended by jmdp.MDP<S,A>
      extended by jmdp.InfiniteMDP<S,A>
          extended by jmdp.CTMDP<StateEvent<S,E>,A>
              extended by jmdp.CTMDPEvA<S,A,E>

public abstract class CTMDPEvA<S extends State,A extends Action,E extends Event>
extends CTMDP<StateEvent<S,E>,A>

This class represents an Infinite horizon, continuous time Markov Decision Process with events where actions depend on events. It allows the definition of events that can occur in a given state and this makes the cost and probability definition easier than in the cases where no events are defined.

Author:
Andres Sarmiento and Germ�n Ria�o. Universidad de los Andes.

Field Summary
protected  StateEvent<S,E> activeState
           
protected  States<S> initSet
           
 
Fields inherited from class jmdp.CTMDP
exitRates, maxRate
 
Fields inherited from class jmdp.InfiniteMDP
absorbingStates, explorationTime, hasAbsorbingState, probability, probabilitySolver, states
 
Fields inherited from class jmdp.MDP
finite, initial, reporter
 
Constructor Summary
CTMDPEvA(States<S> initial)
          Creates a new continuous time infinite horizon MDP Problem with events
 
Method Summary
abstract  Events<E> activeEvents(S i)
          Set of events that are active from state i given that action a is taken.
abstract  double continuousCost(S i, A a, E e)
          Reward obtained continuously in time during the sojourn time in state i until an action a is taken and a transition is triggered.
 double continuousCost(StateEvent<S,E> i, A a)
          Reward obtained continuously in time until the next transition from state i given that action a is taken.
protected  double exitRate(StateEvent<S,E> i, A a)
          Exit rate from state i given action a is taken
protected  States<StateEvent<S,E>> explore(States<StateEvent<S,E>> initSet)
           
abstract  Actions<A> feasibleAct(S i)
          Returns the set of actions available at this state.
 Actions<A> feasibleActions(StateEvent<S,E> s)
          Returns the set of actions available at this state.
abstract  double lumpCost(S i, A a, E e)
          Reward instantaneously gained in the moment when action a is taken from state i.
 double lumpCost(StateEvent<S,E> i, A a)
          Reward instantaneously in the moment when action a is taken from state i.
abstract  double rate(S i, S j, A a, E e)
          Rate.
 double rate(StateEvent<S,E> i, StateEvent<S,E> j, A a)
          Rate of going from state i to state j by taking the action a
abstract  States<S> reached(S i, A a, E e)
          Set of reachable states from state i given that action a is taken and event e occurs.
 States<StateEvent<S,E>> reached(StateEvent<S,E> i, A a)
          Set of states that can be reached from this state i, after taking the action a.
 
Methods inherited from class jmdp.CTMDP
getAllStates, getMaxRate, getSteadyStateProbabilities, oneStageReachable, setConverter, solve
 
Methods inherited from class jmdp.InfiniteMDP
getDefaultAverageSolver, getDefaultDiscountedSolver, getDefaultSolver, getSolver, setInterestRate
 
Methods inherited from class jmdp.MDP
debug, getOptimalPolicy, getOptimalValueFunction, getReporter, isFinite, isSolved, operation, printSolution, printSolution, setReporter, setSolver, solve
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

activeState

protected StateEvent<S extends State,E extends Event> activeState

initSet

protected States<S extends State> initSet
Constructor Detail

CTMDPEvA

public CTMDPEvA(States<S> initial)
Creates a new continuous time infinite horizon MDP Problem with events

Parameters:
initial - set of initial states for the exploration algorithm
Method Detail

explore

protected States<StateEvent<S,E>> explore(States<StateEvent<S,E>> initSet)
Overrides:
explore in class CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>

exitRate

protected double exitRate(StateEvent<S,E> i,
                          A a)
Description copied from class: CTMDP
Exit rate from state i given action a is taken

Overrides:
exitRate in class CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>
Parameters:
i - current state
a - action taken
Returns:
exit rate

lumpCost

public double lumpCost(StateEvent<S,E> i,
                       A a)
Description copied from class: CTMDP
Reward instantaneously in the moment when action a is taken from state i.

Specified by:
lumpCost in class CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>

continuousCost

public double continuousCost(StateEvent<S,E> i,
                             A a)
Description copied from class: CTMDP
Reward obtained continuously in time until the next transition from state i given that action a is taken.

Specified by:
continuousCost in class CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>

rate

public double rate(StateEvent<S,E> i,
                   StateEvent<S,E> j,
                   A a)
Description copied from class: CTMDP
Rate of going from state i to state j by taking the action a

Specified by:
rate in class CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>

reached

public States<StateEvent<S,E>> reached(StateEvent<S,E> i,
                                       A a)
Description copied from class: CTMDP
Set of states that can be reached from this state i, after taking the action a.

Specified by:
reached in class CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>

feasibleActions

public Actions<A> feasibleActions(StateEvent<S,E> s)
Description copied from class: InfiniteMDP
Returns the set of actions available at this state.

Specified by:
feasibleActions in class InfiniteMDP<StateEvent<S extends State,E extends Event>,A extends Action>

rate

public abstract double rate(S i,
                            S j,
                            A a,
                            E e)
Rate. Rate of going of reaching state j given that the current state is i, the action taken is a and the event that occurs is e.

Parameters:
i - current state
j - state to reach
a - action taken (given)
e - event that occurs (given)
Returns:
Rate

reached

public abstract States<S> reached(S i,
                                  A a,
                                  E e)
Set of reachable states from state i given that action a is taken and event e occurs.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
set of reachable states.

activeEvents

public abstract Events<E> activeEvents(S i)
Set of events that are active from state i given that action a is taken.

Parameters:
i - current state
Returns:
set of events that can occur

lumpCost

public abstract double lumpCost(S i,
                                A a,
                                E e)
Reward instantaneously gained in the moment when action a is taken from state i.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
instanteneous reward.

continuousCost

public abstract double continuousCost(S i,
                                      A a,
                                      E e)
Reward obtained continuously in time during the sojourn time in state i until an action a is taken and a transition is triggered.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
instanteneous reward.

feasibleAct

public abstract Actions<A> feasibleAct(S i)
Returns the set of actions available at this state. The user must implement this method.

Parameters:
i - current state
Returns:
set of feasible actions