jmarkov.jmdp
Class CTMDPEvA<S extends State,A extends Action,E extends Event>

java.lang.Object
  extended by jmarkov.jmdp.MDP<S,A>
      extended by jmarkov.jmdp.InfiniteMDP<S,A>
          extended by jmarkov.jmdp.CTMDP<StateEvent<S,E>,A>
              extended by jmarkov.jmdp.CTMDPEvA<S,A,E>
Type Parameters:
S - The staes class
A - Actions class
E - Events class

public abstract class CTMDPEvA<S extends State,A extends Action,E extends Event>
extends CTMDP<StateEvent<S,E>,A>

This class represents an Infinite horizon, continuous time Markov Decision Process with events where actions depend on events. It allows the definition of events that can occur in a given state and this makes the cost and probability definition easier than in the cases where no events are defined.

Author:
Andres Sarmiento and Germán Riaño. Universidad de los Andes.

Field Summary
protected  States<S> initSet
          Initail set od States.
 
Fields inherited from class jmarkov.jmdp.CTMDP
activeState, converter, maxRate
 
Fields inherited from class jmarkov.jmdp.InfiniteMDP
absorbingStates, explorationTime, hasAbsorbingState, numStates, probability, probabilitySolver, states
 
Fields inherited from class jmarkov.jmdp.MDP
finite, initial, reporter
 
Constructor Summary
CTMDPEvA(States<S> initial)
          Creates a new continuous time infinite horizon MDP Problem with events
 
Method Summary
abstract  Events<E> activeEvents(S i)
          Set of events that are active from state i given that action a is taken.
abstract  double continuousCost(S i, A a, E e)
          Reward obtained continuously in time during the sojourn time in state i until an action a is taken and a transition is triggered.
 double continuousCost(StateEvent<S,E> i, A a)
          Cost incurred continuously in time until the next transition from state i given that action a is taken.
abstract  Actions<A> feasibleAct(S i)
          Returns the set of actions available at this state.
 Actions<A> feasibleActions(StateEvent<S,E> s)
          Returns the set of actions available at this state.
protected  StatesSet<StateEvent<S,E>> generate()
           
abstract  double lumpCost(S i, A a, E e)
          Reward instantaneously gained in the moment when action a is taken from state i.
 double lumpCost(StateEvent<S,E> i, A a)
          Cost incurred instantaneously in the moment when action a is taken from state i.
abstract  double rate(S i, S j, A a, E e)
          Rate.
 double rate(StateEvent<S,E> i, StateEvent<S,E> j, A a)
          Rate of going from state i to state j by taking the action a
 States<StateEvent<S,E>> reachable(StateEvent<S,E> i, A a)
          Set of States that can be reached from this state i, after taking the action a.
abstract  States<S> reached(S i, A a, E e)
          Set of reachable states from state i given that action a is taken and event e occurs.
 
Methods inherited from class jmarkov.jmdp.CTMDP
getAllStates, getMaxRate, getSteadyStateProbabilities, oneStageReachable, setConverter, solve
 
Methods inherited from class jmarkov.jmdp.InfiniteMDP
getDefaultAverageSolver, getDefaultDiscountedSolver, getDefaultSolver, getNumStates, getSolver, setInterestRate
 
Methods inherited from class jmarkov.jmdp.MDP
debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

initSet

protected States<S extends State> initSet
Initail set od States.

Constructor Detail

CTMDPEvA

public CTMDPEvA(States<S> initial)
Creates a new continuous time infinite horizon MDP Problem with events

Parameters:
initial - set of initial states for the exploration algorithm
Method Detail

generate

protected StatesSet<StateEvent<S,E>> generate()
Overrides:
generate in class CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>
Returns:
The set of states found.

lumpCost

public double lumpCost(StateEvent<S,E> i,
                       A a)
Description copied from class: CTMDP
Cost incurred instantaneously in the moment when action a is taken from state i.

Specified by:
lumpCost in class CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>
Parameters:
i - State
a - Action
Returns:
Lump cost received.

continuousCost

public double continuousCost(StateEvent<S,E> i,
                             A a)
Description copied from class: CTMDP
Cost incurred continuously in time until the next transition from state i given that action a is taken.

Specified by:
continuousCost in class CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>
Parameters:
i - State
a - Action
Returns:
Rate at which cost is incurred when action a is taken.

rate

public double rate(StateEvent<S,E> i,
                   StateEvent<S,E> j,
                   A a)
Description copied from class: CTMDP
Rate of going from state i to state j by taking the action a

Specified by:
rate in class CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>
Parameters:
i - current state
j - Destination state.
a - Action taken
Returns:
The rate

reachable

public States<StateEvent<S,E>> reachable(StateEvent<S,E> i,
                                         A a)
Description copied from class: CTMDP
Set of States that can be reached from this state i, after taking the action a.

Specified by:
reachable in class CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>
Parameters:
i - current State
a - action taken
Returns:
the reachable states.

feasibleActions

public Actions<A> feasibleActions(StateEvent<S,E> s)
Description copied from class: InfiniteMDP
Returns the set of actions available at this state.

Specified by:
feasibleActions in class InfiniteMDP<StateEvent<S extends State,E extends Event>,A extends Action>
Parameters:
s - Current State
Returns:
set of Actions that can be taken at this state.

rate

public abstract double rate(S i,
                            S j,
                            A a,
                            E e)
Rate. Rate of going of reaching state j given that the current state is i, the action taken is a and the event that occurs is e.

Parameters:
i - current state
j - state to reach
a - action taken (given)
e - event that occurs (given)
Returns:
Rate

reached

public abstract States<S> reached(S i,
                                  A a,
                                  E e)
Set of reachable states from state i given that action a is taken and event e occurs.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
set of reachable states.

activeEvents

public abstract Events<E> activeEvents(S i)
Set of events that are active from state i given that action a is taken.

Parameters:
i - current state
Returns:
set of events that can occur

lumpCost

public abstract double lumpCost(S i,
                                A a,
                                E e)
Reward instantaneously gained in the moment when action a is taken from state i.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
instanteneous reward.

continuousCost

public abstract double continuousCost(S i,
                                      A a,
                                      E e)
Reward obtained continuously in time during the sojourn time in state i until an action a is taken and a transition is triggered.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
instanteneous reward.

feasibleAct

public abstract Actions<A> feasibleAct(S i)
Returns the set of actions available at this state. The user must implement this method.

Parameters:
i - current state
Returns:
set of feasible actions