jmarkov.jmdp
Class DTMDPEvA<S extends State,A extends Action,E extends Event>

java.lang.Object
  extended by jmarkov.jmdp.MDP<S,A>
      extended by jmarkov.jmdp.InfiniteMDP<S,A>
          extended by jmarkov.jmdp.DTMDP<StateEvent<S,E>,A>
              extended by jmarkov.jmdp.DTMDPEvA<S,A,E>
Type Parameters:
S - States class
A - Action Class
E - Events class

public abstract class DTMDPEvA<S extends State,A extends Action,E extends Event>
extends DTMDP<StateEvent<S,E>,A>

This class represents an infinite horizon, discrete time, Markov Decision Process with events, where actions depend on events. It allows the definition of events that can occur in a given state and this makes the cost and probability definition easier to define than in the cases where no events are defined.

Author:
Andres Sarmiento and Germán Riaño. Universidad de los Andes.

Field Summary
 
Fields inherited from class jmarkov.jmdp.InfiniteMDP
absorbingStates, explorationTime, hasAbsorbingState, numStates, probability, probabilitySolver, states
 
Fields inherited from class jmarkov.jmdp.MDP
finite, initial, reporter
 
Constructor Summary
DTMDPEvA(States<S> initial)
          Creates a new infinite horizon discrete time (MDP) Problem with events
 
Method Summary
abstract  Events<E> activeEvents(S i, A a)
          Set of events that are active from state i given that action a is taken.
abstract  Actions<A> feasibleAct(S i)
          Returns the set of actions available at this state.
 Actions<A> feasibleActions(StateEvent<S,E> i)
          Returns the set of actions available at this state.
abstract  double immediateCost(S i, A a, E e)
          Reward received when the current state is i, the action taken is a and event e occurs.
 double immediateCost(StateEvent<S,E> i, A a)
          Cost incurred when taking action a from state i
abstract  double prob(S i, E e)
          Conditional event probability.
abstract  double prob(S i, S j, A a, E e)
          Conditional destination probability.
 double prob(StateEvent<S,E> i, StateEvent<S,E> j, A a)
          Probability of going from state i to state j by taking the action a
abstract  States<S> reachable(S i, A a, E e)
          Set of reachable states from state i given that action a is taken and event e occurs.
 States<StateEvent<S,E>> reachable(StateEvent<S,E> i, A a)
          Set of states that can be reached from this state i, after taking the action a.
 
Methods inherited from class jmarkov.jmdp.DTMDP
generate, getSteadyStateProbabilities, oneStageReachable, setProbabilitySolver, solve
 
Methods inherited from class jmarkov.jmdp.InfiniteMDP
getAllStates, getDefaultAverageSolver, getDefaultDiscountedSolver, getDefaultSolver, getNumStates, getSolver, setInterestRate
 
Methods inherited from class jmarkov.jmdp.MDP
debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

DTMDPEvA

public DTMDPEvA(States<S> initial)
Creates a new infinite horizon discrete time (MDP) Problem with events

Parameters:
initial - set of initial states for the exploration algorithm
Method Detail

reachable

public final States<StateEvent<S,E>> reachable(StateEvent<S,E> i,
                                               A a)
Description copied from class: DTMDP
Set of states that can be reached from this state i, after taking the action a.

Specified by:
reachable in class DTMDP<StateEvent<S extends State,E extends Event>,A extends Action>
Parameters:
i - Current State
a - Action taken
Returns:
The reachable states.

immediateCost

public final double immediateCost(StateEvent<S,E> i,
                                  A a)
Description copied from class: DTMDP
Cost incurred when taking action a from state i

Specified by:
immediateCost in class DTMDP<StateEvent<S extends State,E extends Event>,A extends Action>
Parameters:
i - Current State
a - Current Action
Returns:
The cost incurred per transition

prob

public final double prob(StateEvent<S,E> i,
                         StateEvent<S,E> j,
                         A a)
Description copied from class: DTMDP
Probability of going from state i to state j by taking the action a

Specified by:
prob in class DTMDP<StateEvent<S extends State,E extends Event>,A extends Action>
Parameters:
i - Current state.
j - Destination State
a - Action
Returns:
The probability.

feasibleActions

public final Actions<A> feasibleActions(StateEvent<S,E> i)
Description copied from class: InfiniteMDP
Returns the set of actions available at this state.

Specified by:
feasibleActions in class InfiniteMDP<StateEvent<S extends State,E extends Event>,A extends Action>
Parameters:
i - Current State
Returns:
set of Actions that can be taken at this state.

immediateCost

public abstract double immediateCost(S i,
                                     A a,
                                     E e)
Reward received when the current state is i, the action taken is a and event e occurs.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
reward

prob

public abstract double prob(S i,
                            S j,
                            A a,
                            E e)
Conditional destination probability. Probability of reaching state j given that the current state is i, the action taken is a and the event that occurs is e.

Parameters:
i - current state
j - state to reach
a - action taken (given)
e - event that occurs (given)
Returns:
Conditional probability

prob

public abstract double prob(S i,
                            E e)
Conditional event probability. Probability that event e occurs given that the current state is i.

Parameters:
i - current state
e - event that occurs
Returns:
Conditional probability

reachable

public abstract States<S> reachable(S i,
                                    A a,
                                    E e)
Set of reachable states from state i given that action a is taken and event e occurs.

Parameters:
i - current state
a - action taken
e - event that occurs
Returns:
set of reachable states.

activeEvents

public abstract Events<E> activeEvents(S i,
                                       A a)
Set of events that are active from state i given that action a is taken.

Parameters:
i - current state
a - action taken
Returns:
set of events that can occur

feasibleAct

public abstract Actions<A> feasibleAct(S i)
Returns the set of actions available at this state. The user must implement this method.

Parameters:
i - current state
Returns:
set of feasible actions