jmarkov.jmdp.solvers
Class StochasticShortestPathSolver<S extends StateC,A extends Action>

java.lang.Object
  extended by jmarkov.jmdp.solvers.Solver<S,A>
      extended by jmarkov.jmdp.solvers.AbstractInfiniteSolver<S,A>
          extended by jmarkov.jmdp.solvers.AbstractTotalSolver<S,A>
              extended by jmarkov.jmdp.solvers.StochasticShortestPathSolver<S,A>
Type Parameters:
S - States class
A - Action class.
All Implemented Interfaces:
JMarkovElement

public class StochasticShortestPathSolver<S extends StateC,A extends Action>
extends AbstractTotalSolver<S,A>

This solver gives a solution for the minimization of the total cost criterion for an infinite horizon MDP.

Author:
Juan F. Redondo, Andrés Sarmiento , German Riaño - Universidad de los Andes

Field Summary
 
Fields inherited from class jmarkov.jmdp.solvers.Solver
policy, printProcessTime, printValueFunction, problem, solved, valueFunction
 
Constructor Summary
StochasticShortestPathSolver(StochasticShortestPath<S,A> problem)
          Default contructor.
 
Method Summary
protected  double bestAction(S i)
          Sets the best action to take in state i, in the variable bestAction.
 java.lang.String description()
          This method return a complete verbal describtion of this element.
 double future(S i, A a)
          This method calculates the expected value of valueFunction for the current state and a specified action.
 long getIterations()
           
 StochasticShortestPath<S,A> getProblem()
          Returns the problem associated with this solver.
 long getProcessTime()
           
 java.lang.String label()
          The sub classes must return the Solver name.
 Solution<S,A> solve()
          Called to solve the problem.
 
Methods inherited from class jmarkov.jmdp.solvers.AbstractInfiniteSolver
getDiscreteProblem, printSolution
 
Methods inherited from class jmarkov.jmdp.solvers.Solver
getOptimalPolicy, getOptimalValueFunction, getValueFunction, isSolved, printSolution, setPrintProcessTime, setPrintValueFunction, toString
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 
Methods inherited from interface jmarkov.basic.JMarkovElement
equals
 

Constructor Detail

StochasticShortestPathSolver

public StochasticShortestPathSolver(StochasticShortestPath<S,A> problem)
Default contructor.

Parameters:
problem - the structure of the problem of type StochasticShortestPath
Method Detail

getProblem

public StochasticShortestPath<S,A> getProblem()
Description copied from class: AbstractInfiniteSolver
Returns the problem associated with this solver.

Overrides:
getProblem in class AbstractInfiniteSolver<S extends StateC,A extends Action>
Returns:
the problem associated with this solver.
See Also:
Solver.getProblem()

future

public final double future(S i,
                           A a)
                    throws StructureException
This method calculates the expected value of valueFunction for the current state and a specified action.

Parameters:
i - Current State
a - Action taken
Returns:
Future value from this state.
Throws:
StructureException

solve

public Solution<S,A> solve()
                                                  throws StructureException
Description copied from class: Solver
Called to solve the problem. This method MUST write the local variable policy and valueFunction.

Specified by:
solve in class Solver<S extends StateC,A extends Action>
Returns:
The solution Object taht contains the plicy and value fuenction.
Throws:
StructureException

bestAction

protected double bestAction(S i)
                     throws StructureException
Sets the best action to take in state i, in the variable bestAction. Note that in this case StochasticShortestPathProblem Bertsekas expose a transformation for the graph which modify the inmediate reward function and the transition probability, only to make a graph without self-transition states. This will increase the finite termination probability for the algorithm.

Parameters:
i - state for which the best action is being determined
Returns:
the new ValueFunction for this state.
Throws:
StructureException

description

public java.lang.String description()
Description copied from interface: JMarkovElement
This method return a complete verbal describtion of this element. This description may contain multiple text rows.

Specified by:
description in interface JMarkovElement
Overrides:
description in class Solver<S extends StateC,A extends Action>
Returns:
A String describing this element.
See Also:
Object.toString()

label

public java.lang.String label()
Description copied from class: Solver
The sub classes must return the Solver name.

Specified by:
label in interface JMarkovElement
Specified by:
label in class Solver<S extends StateC,A extends Action>
Returns:
A String label.
See Also:
Solver.label()

getProcessTime

public final long getProcessTime()
Specified by:
getProcessTime in class Solver<S extends StateC,A extends Action>
Returns:
Returns the processTime.

getIterations

public final long getIterations()
Specified by:
getIterations in class AbstractInfiniteSolver<S extends StateC,A extends Action>
Returns:
Returns the iterations.