jmarkov.jmdp
Class MDP<S extends State,A extends Action>

java.lang.Object
  extended by jmarkov.jmdp.MDP<S,A>
Type Parameters:
S - Current state.
A - Action taken.
Direct Known Subclasses:
FiniteMDP, InfiniteMDP

public abstract class MDP<S extends State,A extends Action>
extends java.lang.Object

This class is the main framework to build a Dynamic Programming Problem. It�s initially created to work over Markov Decision Problems which imply random probabilities but can easily be worked out for deterministic problems if the probabilities are set to one. This class should not me extended directly on problems. The default package has FiniteMDP and InfiniteMDP classes that are intended to be extended on problems. See the examples for a clearer reference.

Author:
Andres Sarmiento, Germán Riaño - Universidad de Los Andes
See Also:
FiniteMDP, DTMDP

Field Summary
protected  boolean finite
          States whether the problem is a finite horizon problem or not.
protected  States<S> initial
          Set of initial states.
protected  DebugReporter reporter
          Reporter used for debug information.
 
Constructor Summary
MDP()
           
 
Method Summary
 void debug(int level, java.lang.String message)
          Prints a message in the reporter.
 void debug(int level, java.lang.String s, boolean newline)
          Prints debug information in the reporter.
 void debug(int level, java.lang.String s, boolean newline, boolean indent)
          Prints debug information in the reporter.
 int getDebugLevel()
          Gets the current debug level.
protected abstract  Solver<S,A> getDefaultSolver()
          The class that extends MDP must define the default solver to use.
 Policy<S,A> getOptimalPolicy()
          Returns the optimal policy.
 ValueFunction<S> getOptimalValueFunction()
          Returns the optimal ValueFunction.
 DebugReporter getReporter()
           
 Solver<S,A> getSolver()
           
 boolean isFinite()
           
 boolean isSolved()
          Indicates if the problems has been solved
 double operation(double present, double future)
          The Operator between present and future costs.
 void printSolution()
          Prints the solution to Standard output.
 void printSolution(java.io.PrintWriter pw)
          Prints the solution to the given PrintWriter
 void setDebugLevel(int level)
          Sets teh current level
 void setReporter(DebugReporter reporter)
           
 void setSolver(Solver<S,A> solver)
           
 void solve()
          Solves the problem.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

finite

protected boolean finite
States whether the problem is a finite horizon problem or not.


initial

protected States<S extends State> initial
Set of initial states.


reporter

protected DebugReporter reporter
Reporter used for debug information.

Constructor Detail

MDP

public MDP()
Method Detail

getOptimalPolicy

public final Policy<S,A> getOptimalPolicy()
                                                                throws SolverException
Returns the optimal policy. This function should be called only AFTER solve has been called.

Returns:
The optimal policy.
Throws:
SolverException - Exception thrown if a solution cannot be found

isSolved

public final boolean isSolved()
Indicates if the problems has been solved

Returns:
true if solved

isFinite

public final boolean isFinite()
Returns:
Returns true if the problem's horizon is finite.

solve

public final void solve()
                 throws SolverException
Solves the problem. If no solver has been defined, this used the default solver.

Throws:
SolverException - This Exception is thrown if a solution cannot be found.
See Also:
getDefaultSolver()

getSolver

public Solver<S,A> getSolver()
Returns:
Returns the solver.

setSolver

public void setSolver(Solver<S,A> solver)
Parameters:
solver - The solver to set.

getDefaultSolver

protected abstract Solver<S,A> getDefaultSolver()
The class that extends MDP must define the default solver to use.

Returns:
the solver to use for this problem.

getOptimalValueFunction

public ValueFunction<S> getOptimalValueFunction()
                                                       throws SolverException
Returns the optimal ValueFunction. This causes the problem to be solved if it has not been solved.

Returns:
Returns the valueFunction.
Throws:
SolverException - This exception is thrown if a solution cannot be found.

operation

public double operation(double present,
                        double future)
The Operator between present and future costs. By default is sum, but can be changed by the user, by overriding this method.

Parameters:
present - Cost of current transition
future - Cost of future transitions.
Returns:
By default it returns present + future.

getReporter

public DebugReporter getReporter()
Returns:
Returns the reporter.

setReporter

public void setReporter(DebugReporter reporter)
Parameters:
reporter - The reporter to set.

debug

public void debug(int level,
                  java.lang.String message)
Prints a message in the reporter.

Parameters:
level - maximum debug level at which to show message
message - message
See Also:
DebugReporter

debug

public void debug(int level,
                  java.lang.String s,
                  boolean newline,
                  boolean indent)
Prints debug information in the reporter.

Parameters:
level - the level for the info
s - Message
newline - true if a new line is to be inserted
indent - true if the info is indented according to level
See Also:
DebugReporter, DebugReporter.debug(int, java.lang.String, boolean, boolean)

debug

public void debug(int level,
                  java.lang.String s,
                  boolean newline)
Prints debug information in the reporter.

Parameters:
level - the level for the info
s - Message
newline - true if a new line is to be inserted
See Also:
DebugReporter, DebugReporter.debug(int, java.lang.String, boolean)

getDebugLevel

public int getDebugLevel()
Gets the current debug level.

Returns:
The current debug level
See Also:
DebugReporter, DebugReporter.getDebugLevel()

setDebugLevel

public void setDebugLevel(int level)
Sets teh current level

Parameters:
level - The new level to level
See Also:
DebugReporter, DebugReporter.setDebugLevel(int)

printSolution

public void printSolution()
Prints the solution to Standard output.


printSolution

public void printSolution(java.io.PrintWriter pw)
Prints the solution to the given PrintWriter

Parameters:
pw - The PrintWriter where the solution will be printed. It must have been initialized.