jmarkov.jmdp
Class FiniteMDPEv<S extends State,A extends Action,E extends Event>

java.lang.Object
  jmarkov.jmdp.MDP<S,A>
      jmarkov.jmdp.FiniteMDP<S,A>
          jmarkov.jmdp.FiniteMDPEv<S,A,E>

Type Parameters:

S - States class

A - Actions class

E - Events class

public abstract class FiniteMDPEv<S extends State,A extends Action,E extends Event>
extends FiniteMDP<S,A>

This class represents a finite horizon discrete time MDP with events.

Author:

Andres Sarmiento and Germ�n Ria�o - Universidad de Los Andes

Field Summary

Fields inherited from class jmarkov.jmdp.FiniteMDP

horizon

Fields inherited from class jmarkov.jmdp.MDP

finite, initial, reporter

Constructor Summary

FiniteMDPEv(States<S> initial, int horizon)

Method Summary

abstract Events<E>	activeEvents(S i, A a, int t) Set of events that are active from state i given that action a is taken.
abstract double	immediateCost(S i, A a, E e, int t) Reward received when the current state is i, the action taken is a and event e occurs.
double	immediateCost(S i, A a, int t) This funtion must return the Immediate cost incurred when taking action a from state i
abstract double	prob(S i, E e, int t) Conditional probability.
abstract double	prob(S i, S j, A a, E e, int t) Conditional probability.
double	prob(S i, S j, A a, int t) This is the probability of going from state i to state j by taking the action a at stage t.
abstract States<S>	reachable(S i, A a, E e, int t) Set of reachable states from state i given that action a is taken and event e occurs.
States<S>	reachable(S i, A a, int t) Set of States that can be reached from this state i, at this stage t, after taking the acton a.

Methods inherited from class jmarkov.jmdp.FiniteMDP

feasibleActions, finalCost, getDefaultSolver, getHorizon, getStates, setHorizon

Methods inherited from class jmarkov.jmdp.MDP

debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, getSolver, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FiniteMDPEv

public FiniteMDPEv(States<S> initial,
                   int horizon)

Parameters:

initial -

horizon -

Method Detail

immediateCost

public double immediateCost(S i,
                            A a,
                            int t)

Description copied from class: FiniteMDP

This funtion must return the Immediate cost incurred when taking action a from state i

Specified by:

immediateCost in class FiniteMDP<S extends State,A extends Action>

Parameters:

i - Current state

a - Action

t - Current time stage

Returns:

Cost value

immediateCost

public abstract double immediateCost(S i,
                                     A a,
                                     E e,
                                     int t)

Reward received when the current state is i, the action taken is a and event e occurs.

Parameters:

i - current state

a - action taken

e - event that occurs

t - current stage

Returns:

reward

prob

public double prob(S i,
                   S j,
                   A a,
                   int t)

Description copied from class: FiniteMDP

This is the probability of going from state i to state j by taking the action a at stage t.

Specified by:

prob in class FiniteMDP<S extends State,A extends Action>

Parameters:

i - Current state

j - Destination state

a - Action taken

t - Current time stage

Returns:

Probability

prob

public abstract double prob(S i,
                            S j,
                            A a,
                            E e,
                            int t)

Conditional probability. Probability of reaching state j given that the current state is i, the action taken is a and the event that occurs is e.

Parameters:

i - current state

j - state to reach

a - action taken (given)

e - event that occurs (given)

t - current stage

Returns:

conditional probability

prob

public abstract double prob(S i,
                            E e,
                            int t)

Conditional probability. Probability that event e occurs given that the current state is i.

Parameters:

i - current state

e - event that occurs

t - current stage

Returns:

Conditional probability

reachable

public States<S> reachable(S i,
                           A a,
                           int t)

Description copied from class: FiniteMDP

Set of States that can be reached from this state i, at this stage t, after taking the acton a. The user must implement this method.

Specified by:

reachable in class FiniteMDP<S extends State,A extends Action>

Parameters:

i - Current state

a - Action taken

t - Time stage

Returns:

Set of reachable states.

reachable

public abstract States<S> reachable(S i,
                                    A a,
                                    E e,
                                    int t)

Set of reachable states from state i given that action a is taken and event e occurs.

Parameters:

i - current state

a - action taken

e - event that occurs

t - current stage

Returns:

set of reachable states.

activeEvents

public abstract Events<E> activeEvents(S i,
                                       A a,
                                       int t)

Set of events that are active from state i given that action a is taken.

Parameters:

i - current state

a - action taken

t - current stage

Returns:

set of events that can occur