jmarkov.jmdp
Class FiniteMDPEv<S extends State,A extends Action,E extends Event>

java.lang.Object
  extended by jmarkov.jmdp.MDP<S,A>
      extended by jmarkov.jmdp.FiniteMDP<S,A>
          extended by jmarkov.jmdp.FiniteMDPEv<S,A,E>
Type Parameters:
S - States class
A - Actions class
E - Events class

public abstract class FiniteMDPEv<S extends State,A extends Action,E extends Event>
extends FiniteMDP<S,A>

This class represents a finite horizon discrete time MDP with events.

Author:
Andres Sarmiento and Germán Riaño - Universidad de Los Andes

Field Summary
 
Fields inherited from class jmarkov.jmdp.FiniteMDP
horizon
 
Fields inherited from class jmarkov.jmdp.MDP
finite, initial, reporter
 
Constructor Summary
FiniteMDPEv(States<S> initial, int horizon)
           
 
Method Summary
abstract  Events<E> activeEvents(S i, A a, int t)
          Set of events that are active from state i given that action a is taken.
abstract  double immediateCost(S i, A a, E e, int t)
          Reward received when the current state is i, the action taken is a and event e occurs.
 double immediateCost(S i, A a, int t)
          This funtion must return the Immediate cost incurred when taking action a from state i
abstract  double prob(S i, E e, int t)
          Conditional probability.
abstract  double prob(S i, S j, A a, E e, int t)
          Conditional probability.
 double prob(S i, S j, A a, int t)
          This is the probability of going from state i to state j by taking the action a at stage t.
abstract  States<S> reachable(S i, A a, E e, int t)
          Set of reachable states from state i given that action a is taken and event e occurs.
 States<S> reachable(S i, A a, int t)
          Set of States that can be reached from this state i, at this stage t, after taking the acton a.
 
Methods inherited from class jmarkov.jmdp.FiniteMDP
feasibleActions, finalCost, getDefaultSolver, getHorizon, getStates, setHorizon
 
Methods inherited from class jmarkov.jmdp.MDP
debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, getSolver, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

FiniteMDPEv

public FiniteMDPEv(States<S> initial,
                   int horizon)
Parameters:
initial -
horizon -
Method Detail

immediateCost

public double immediateCost(S i,
                            A a,
                            int t)
Description copied from class: FiniteMDP
This funtion must return the Immediate cost incurred when taking action a from state i

Specified by:
immediateCost in class FiniteMDP<S extends State,A extends Action>
Parameters:
i - Current state
a - Action
t - Current time stage
Returns:
Cost value

immediateCost

public abstract double immediateCost(S i,
                                     A a,
                                     E e,
                                     int t)
Reward received when the current state is i, the action taken is a and event e occurs.

Parameters:
i - current state
a - action taken
e - event that occurs
t - current stage
Returns:
reward

prob

public double prob(S i,
                   S j,
                   A a,
                   int t)
Description copied from class: FiniteMDP
This is the probability of going from state i to state j by taking the action a at stage t.

Specified by:
prob in class FiniteMDP<S extends State,A extends Action>
Parameters:
i - Current state
j - Destination state
a - Action taken
t - Current time stage
Returns:
Probability

prob

public abstract double prob(S i,
                            S j,
                            A a,
                            E e,
                            int t)
Conditional probability. Probability of reaching state j given that the current state is i, the action taken is a and the event that occurs is e.

Parameters:
i - current state
j - state to reach
a - action taken (given)
e - event that occurs (given)
t - current stage
Returns:
conditional probability

prob

public abstract double prob(S i,
                            E e,
                            int t)
Conditional probability. Probability that event e occurs given that the current state is i.

Parameters:
i - current state
e - event that occurs
t - current stage
Returns:
Conditional probability

reachable

public States<S> reachable(S i,
                           A a,
                           int t)
Description copied from class: FiniteMDP
Set of States that can be reached from this state i, at this stage t, after taking the acton a. The user must implement this method.

Specified by:
reachable in class FiniteMDP<S extends State,A extends Action>
Parameters:
i - Current state
a - Action taken
t - Time stage
Returns:
Set of reachable states.

reachable

public abstract States<S> reachable(S i,
                                    A a,
                                    E e,
                                    int t)
Set of reachable states from state i given that action a is taken and event e occurs.

Parameters:
i - current state
a - action taken
e - event that occurs
t - current stage
Returns:
set of reachable states.

activeEvents

public abstract Events<E> activeEvents(S i,
                                       A a,
                                       int t)
Set of events that are active from state i given that action a is taken.

Parameters:
i - current state
a - action taken
t - current stage
Returns:
set of events that can occur