jmdp.solvers
Class StochasticShortestPathSolver<S extends StateC,A extends Action>

java.lang.Object
  extended by jmdp.solvers.Solver<S,A>
      extended by jmdp.solvers.AbstractInfiniteSolver<S,A>
          extended by jmdp.solvers.AbstractTotalSolver<S,A>
              extended by jmdp.solvers.StochasticShortestPathSolver<S,A>

public class StochasticShortestPathSolver<S extends StateC,A extends Action>
extends AbstractTotalSolver<S,A>

This solver gives a solution for the minimization of the total cost criterion for an infinite horizon MDP.

Author:
Juan F. Redondo - Andr�s Sarmiento - Universidad de los Andes

Field Summary
 
Fields inherited from class jmdp.solvers.Solver
policy, printProcessTime, printValueFunction, problem, solved, valueFunction
 
Constructor Summary
StochasticShortestPathSolver(StochasticShortestPath<S,A> problem)
          The constructor method exclusively receives a problem of the type StochasticShortestPath because this solver is only designed to work on infinite horizon problems without discount factor.
 
Method Summary
protected  double bestAction(S i)
          Sets the best action to take in state i, in the static variable bestAction.
 double future(S i, A a)
          This method calculates the expected value of valueFunction for the current state and a specified action.
 int getIterations()
           
 StochasticShortestPath<S,A> getProblem()
          Returns the problem associated with this solver.
 long getProcessTime()
           
 Solution<S,A> solve()
          This is the method which activated the solver, a well-specified problem must be created, then StochasticShortestPathSolver can solve the problem
 java.lang.String toString()
          The sub classes must return the Solver name.
 
Methods inherited from class jmdp.solvers.AbstractInfiniteSolver
getDiscreteProblem, printSolution
 
Methods inherited from class jmdp.solvers.Solver
getOptimalPolicy, getOptimalValueFunction, getValueFunction, isSolved, printSolution, setPrintProcessTime, setPrintValueFunction
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Constructor Detail

StochasticShortestPathSolver

public StochasticShortestPathSolver(StochasticShortestPath<S,A> problem)
The constructor method exclusively receives a problem of the type StochasticShortestPath because this solver is only designed to work on infinite horizon problems without discount factor. Until now, this solver solves the discounted objective function problem. Other objective functions are still under development.

Parameters:
problem - the structure of the problem of type StochasticShortestPath
Method Detail

getProblem

public StochasticShortestPath<S,A> getProblem()
Description copied from class: AbstractInfiniteSolver
Returns the problem associated with this solver.

Overrides:
getProblem in class AbstractInfiniteSolver<S extends StateC,A extends Action>
Returns:
the problem associated with this solver.
See Also:
Solver.getProblem()

future

public final double future(S i,
                           A a)
                    throws StructureException
This method calculates the expected value of valueFunction for the current state and a specified action.

Throws:
StructureException

solve

public Solution<S,A> solve()
This is the method which activated the solver, a well-specified problem must be created, then StochasticShortestPathSolver can solve the problem

Specified by:
solve in class Solver<S extends StateC,A extends Action>

bestAction

protected double bestAction(S i)
Sets the best action to take in state i, in the static variable bestAction. Note that in this case StochasticShortestPathProblem Bertsekas expose a transformation for the graph which modify the inmediate reward function and the transition probability, only to make a graph without self-transition states. This will increase the finite termination probability for the algorithm.

Parameters:
i - state for which the best action is being determined
Returns:
the new ValueFunction for this state.

toString

public java.lang.String toString()
Description copied from class: Solver
The sub classes must return the Solver name.

Specified by:
toString in class Solver<S extends StateC,A extends Action>
See Also:
Object.toString()

getProcessTime

public final long getProcessTime()
Specified by:
getProcessTime in class Solver<S extends StateC,A extends Action>
Returns:
Returns the processTime.

getIterations

public final int getIterations()
Specified by:
getIterations in class AbstractInfiniteSolver<S extends StateC,A extends Action>
Returns:
Returns the iterations.