|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectjmarkov.jmdp.solvers.Solver<S,A>
jmarkov.jmdp.solvers.AbstractInfiniteSolver<S,A>
jmarkov.jmdp.solvers.AbstractDiscountedSolver<S,A>
jmarkov.jmdp.solvers.ValueIterationSolver<S,A>
S
- States Class.A
- Actions class.public class ValueIterationSolver<S extends State,A extends Action>
This class belongs to the set of default solvers included in the jmdp package. It extends Solver and should only be used on INFINITE horizon problems. The objective is to be able to return an optimal policy given a problem structure.
Field Summary | |
---|---|
protected long |
iterations
Used to store the number of iterations |
protected long |
processTime
stores the process time |
Fields inherited from class jmarkov.jmdp.solvers.AbstractDiscountedSolver |
---|
discountFactor |
Fields inherited from class jmarkov.jmdp.solvers.Solver |
---|
policy, printProcessTime, printValueFunction, problem, solved, valueFunction |
Constructor Summary | |
---|---|
ValueIterationSolver(CTMDP<S,A> problem,
double interestRate)
Default Constructor for continuous time problems. |
|
ValueIterationSolver(DTMDP<S,A> problem,
double interestRate)
Default Constructor for Discrte time problems. |
Method Summary | |
---|---|
protected double |
bestAction(S i)
Find the minimal value function for this state and sets the best action to take in state i, in the variable bestAction. |
protected double |
computeNoErrorBounds()
Computes an iteration of the Value Iteration Algorithm without the use of error bounds. |
protected double |
computeWithErrorBounds()
Computes an iteration of the Value Iteration Algorithm with the use of error bounds. |
java.lang.String |
description()
This method return a complete verbal describtion of this element. |
double |
getEpsilon()
|
long |
getIterations()
|
long |
getProcessTime()
|
protected void |
init()
Initializes the valueFunction for all the states. |
boolean |
isAverage()
|
java.lang.String |
label()
The sub classes must return the Solver name. |
void |
setEpsilon(double epsilon)
Value Iteration is a solver method this is theoretically convergent only after infinite iterations. |
Solution<S,A> |
solve()
Solves the problem. |
void |
useErrorBounds(boolean val)
The ErrorBounds modification to the ValueIteration method is a change that is garanteed to have a performance at least as good as the methos without the modifications. |
void |
useGaussSeidel(boolean val)
The GaussSeidel modification of the ValueIteration method is a change that is garanteed to have a performance at least as good as the methos without the modifications. |
boolean |
usesErrorBounds()
|
boolean |
usesGaussSeidel()
|
Methods inherited from class jmarkov.jmdp.solvers.AbstractDiscountedSolver |
---|
future, future, getInterestRate, setDiscountFactor, setInterestRate |
Methods inherited from class jmarkov.jmdp.solvers.AbstractInfiniteSolver |
---|
getDiscreteProblem, getProblem, printSolution |
Methods inherited from class jmarkov.jmdp.solvers.Solver |
---|
getOptimalPolicy, getOptimalValueFunction, getValueFunction, isSolved, printSolution, setPrintProcessTime, setPrintValueFunction, toString |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
Methods inherited from interface jmarkov.basic.JMarkovElement |
---|
equals |
Field Detail |
---|
protected long processTime
protected long iterations
Constructor Detail |
---|
public ValueIterationSolver(DTMDP<S,A> problem, double interestRate)
problem
- the structure of the problem of type DTMDPinterestRate
- represents how much less is the reward
received in the next period instead of receiving it in
the present period.public ValueIterationSolver(CTMDP<S,A> problem, double interestRate)
problem
- the structure of the problem of type CTMDPinterestRate
- represents how much less is the reward
received in the next period instead of receiving it in
the present period.Method Detail |
---|
public void setEpsilon(double epsilon)
epsilon
- maximum difference between iterations.public void useGaussSeidel(boolean val)
val
- sets whether or not the GaussSeidel modification
will be used.useErrorBounds(boolean)
public final double getEpsilon()
public final boolean isAverage()
public final boolean usesErrorBounds()
public final boolean usesGaussSeidel()
public void useErrorBounds(boolean val)
val
- sets whether or not to use the ErrorBounds
modification.public Solution<S,A> solve()
solve
in class Solver<S extends State,A extends Action>
protected void init()
protected double computeNoErrorBounds()
protected double computeWithErrorBounds()
protected double bestAction(S i)
i
- state for which the best action is being determined
public final long getProcessTime()
getProcessTime
in class Solver<S extends State,A extends Action>
public final long getIterations()
getIterations
in class AbstractInfiniteSolver<S extends State,A extends Action>
public java.lang.String label()
Solver
label
in interface JMarkovElement
label
in class Solver<S extends State,A extends Action>
Solver.toString()
public java.lang.String description()
JMarkovElement
description
in interface JMarkovElement
description
in class Solver<S extends State,A extends Action>
JMarkovElement.label()
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |