jmarkov.jmdp.solvers
Class StochasticShortestPathSolver<S extends StateC,A extends Action>

java.lang.Object
  jmarkov.jmdp.solvers.Solver<S,A>
      jmarkov.jmdp.solvers.AbstractInfiniteSolver<S,A>
          jmarkov.jmdp.solvers.AbstractTotalSolver<S,A>
              jmarkov.jmdp.solvers.StochasticShortestPathSolver<S,A>

Type Parameters:

S - States class

A - Action class.

All Implemented Interfaces:

JMarkovElement

public class StochasticShortestPathSolver<S extends StateC,A extends Action>
extends AbstractTotalSolver<S,A>

This solver gives a solution for the minimization of the total cost criterion for an infinite horizon MDP.

Author:

Juan F. Redondo, Andr�s Sarmiento , German Ria�o - Universidad de los Andes

Field Summary

Fields inherited from class jmarkov.jmdp.solvers.Solver

policy, printProcessTime, printValueFunction, problem, solved, valueFunction

Constructor Summary

StochasticShortestPathSolver(StochasticShortestPath<S,A> problem)
Default contructor.

Method Summary

protected double	bestAction(S i) Sets the best action to take in state i, in the variable bestAction.
java.lang.String	description() This method return a complete verbal describtion of this element.
double	future(S i, A a) This method calculates the expected value of valueFunction for the current state and a specified action.
long	getIterations()
StochasticShortestPath<S,A>	getProblem() Returns the problem associated with this solver.
long	getProcessTime()
java.lang.String	label() The sub classes must return the Solver name.
Solution<S,A>	solve() Called to solve the problem.

Methods inherited from class jmarkov.jmdp.solvers.AbstractInfiniteSolver

getDiscreteProblem, printSolution

Methods inherited from class jmarkov.jmdp.solvers.Solver

getOptimalPolicy, getOptimalValueFunction, getValueFunction, isSolved, printSolution, setPrintProcessTime, setPrintValueFunction, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface jmarkov.basic.JMarkovElement

equals

Constructor Detail

StochasticShortestPathSolver

public StochasticShortestPathSolver(StochasticShortestPath<S,A> problem)

Default contructor.

Parameters:

problem - the structure of the problem of type StochasticShortestPath

Method Detail

getProblem

public StochasticShortestPath<S,A> getProblem()

Description copied from class: AbstractInfiniteSolver

Returns the problem associated with this solver.

Overrides:

getProblem in class AbstractInfiniteSolver<S extends StateC,A extends Action>

Returns:

the problem associated with this solver.

See Also:

Solver.getProblem()

future

public final double future(S i,
                           A a)
                    throws StructureException

This method calculates the expected value of valueFunction for the current state and a specified action.

Parameters:

i - Current State

a - Action taken

Returns:

Future value from this state.

Throws:

StructureException

solve

public Solution<S,A> solve()
                                                  throws StructureException

Description copied from class: Solver

Called to solve the problem. This method MUST write the local variable policy and valueFunction.

Specified by:

solve in class Solver<S extends StateC,A extends Action>

Returns:

The solution Object taht contains the plicy and value fuenction.

Throws:

StructureException

bestAction

protected double bestAction(S i)
                     throws StructureException

Sets the best action to take in state i, in the variable bestAction. Note that in this case StochasticShortestPathProblem Bertsekas expose a transformation for the graph which modify the inmediate reward function and the transition probability, only to make a graph without self-transition states. This will increase the finite termination probability for the algorithm.

Parameters:

i - state for which the best action is being determined

Returns:

the new ValueFunction for this state.

Throws:

StructureException

description

public java.lang.String description()

Description copied from interface: JMarkovElement

This method return a complete verbal describtion of this element. This description may contain multiple text rows.

Specified by:

description in interface JMarkovElement

Overrides:

description in class Solver<S extends StateC,A extends Action>

Returns:

A String describing this element.

See Also:

Object.toString()

label

public java.lang.String label()

Description copied from class: Solver

The sub classes must return the Solver name.

Specified by:

label in interface JMarkovElement

Specified by:

label in class Solver<S extends StateC,A extends Action>

Returns:

A String label.

See Also:

Solver.label()

getProcessTime

public final long getProcessTime()

Specified by:

getProcessTime in class Solver<S extends StateC,A extends Action>

Returns:

Returns the processTime.

getIterations

public final long getIterations()

Specified by:

getIterations in class AbstractInfiniteSolver<S extends StateC,A extends Action>

Returns:

Returns the iterations.