jmarkov.jmdp
Class DTMDPEv<S extends State,A extends Action,E extends Event>

java.lang.Object
  jmarkov.jmdp.MDP<S,A>
      jmarkov.jmdp.InfiniteMDP<S,A>
          jmarkov.jmdp.DTMDP<S,A>
              jmarkov.jmdp.DTMDPEv<S,A,E>

Type Parameters:

S - The States class

A - Actions class

E - Events class

public abstract class DTMDPEv<S extends State,A extends Action,E extends Event>
extends DTMDP<S,A>

This class represents an infinite horizon, discrete time, Markov Decision Process with events. It allows the definition of events that can occur in a given state and this makes the cost and probability definition easier to define than in the cases where no events are defined.

Author:

Andres Sarmiento and German Ria�o. Universidad de los Andes.

Field Summary

Fields inherited from class jmarkov.jmdp.InfiniteMDP

absorbingStates, explorationTime, hasAbsorbingState, numStates, probability, probabilitySolver, states

Fields inherited from class jmarkov.jmdp.MDP

finite, initial, reporter

Constructor Summary

DTMDPEv(States<S> initial)
Creates a new infinite horizon discrete time (MDP) Problem with events

Method Summary

abstract Events<E>	activeEvents(S i, A a) Set of events that are active from state i given that action a is taken.
double	immediateCost(S i, A a) Cost incurred when taking action a from state i
abstract double	immediateCost(S i, A a, E e) Cost incurred received when the current state is i, the action taken is a and event e occurs.
abstract double	prob(S i, E e) Conditional Event probability.
double	prob(S i, S j, A a) Probability of going from state i to state j by taking the action a
abstract double	prob(S i, S j, A a, E e) Conditional destination probability.
States<S>	reachable(S i, A a) Set of states that can be reached from this state i, after taking the action a.
abstract States<S>	reachable(S i, A a, E e) Set of reachable states from state i given that action a is taken and event e occurs.

Methods inherited from class jmarkov.jmdp.DTMDP

generate, getSteadyStateProbabilities, oneStageReachable, setProbabilitySolver, solve

Methods inherited from class jmarkov.jmdp.InfiniteMDP

feasibleActions, getAllStates, getDefaultAverageSolver, getDefaultDiscountedSolver, getDefaultSolver, getNumStates, getSolver, setInterestRate

Methods inherited from class jmarkov.jmdp.MDP

debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DTMDPEv

public DTMDPEv(States<S> initial)

Creates a new infinite horizon discrete time (MDP) Problem with events

Parameters:

initial - set of initial states for the exploration algorithm

Method Detail

immediateCost

public final double immediateCost(S i,
                                  A a)

Description copied from class: DTMDP

Cost incurred when taking action a from state i

Specified by:

immediateCost in class DTMDP<S extends State,A extends Action>

Parameters:

i - Current State

a - Current Action

Returns:

The cost incurred per transition

prob

public final double prob(S i,
                         S j,
                         A a)

Description copied from class: DTMDP

Probability of going from state i to state j by taking the action a

Specified by:

prob in class DTMDP<S extends State,A extends Action>

Parameters:

i - Current state.

j - Destination State

a - Action

Returns:

The probability.

reachable

public final States<S> reachable(S i,
                                 A a)

Description copied from class: DTMDP

Set of states that can be reached from this state i, after taking the action a.

Specified by:

reachable in class DTMDP<S extends State,A extends Action>

Parameters:

i - Current State

a - Action taken

Returns:

The reachable states.

immediateCost

public abstract double immediateCost(S i,
                                     A a,
                                     E e)

Cost incurred received when the current state is i, the action taken is a and event e occurs.

Parameters:

i - current state

a - action taken

e - event that occurs

Returns:

reward

prob

public abstract double prob(S i,
                            S j,
                            A a,
                            E e)

Conditional destination probability. Probability of reaching state j given that the current state is i, the action taken is a and the event that occurs is e.

Parameters:

i - current state

j - state to reach

a - action taken (given)

e - event that occurs (given)

Returns:

Conditional probability

prob

public abstract double prob(S i,
                            E e)

Conditional Event probability. Probability that event e occurs given that the current state is i.

Parameters:

i - current state

e - event that occurs

Returns:

Conditional probability

reachable

public abstract States<S> reachable(S i,
                                    A a,
                                    E e)

Set of reachable states from state i given that action a is taken and event e occurs.

Parameters:

i - current state

a - action taken

e - event that occurs

Returns:

set of reachable states.

activeEvents

public abstract Events<E> activeEvents(S i,
                                       A a)

Set of events that are active from state i given that action a is taken.

Parameters:

i - current state

a - action taken

Returns:

set of events that can occur