MultiAgentDecisionProcess
PerseusNonStationaryQPlanner Class Referenceabstract

PerseusNonStationaryQPlanner is a Perseus planner that uses non-stationary QFunctions. More...

#include <PerseusNonStationaryQPlanner.h>

Inheritance diagram for PerseusNonStationaryQPlanner:
[legend]

Public Member Functions

 PerseusNonStationaryQPlanner (const PlanningUnitDecPOMDPDiscrete *pu)
 
 PerseusNonStationaryQPlanner (const boost::shared_ptr< const PlanningUnitDecPOMDPDiscrete > &pu)
 
void Plan ()
 
virtual ~PerseusNonStationaryQPlanner ()
 Destructor. More...
 
- Public Member Functions inherited from PerseusNonStationary
void ExportValueFunction (const std::string &filename) const
 Exports the computed value function in a simple text format used by Tony Cassandra for .alpha files. More...
 
double GetQ (const JointBeliefInterface &b, Index jaI) const
 
double GetQ (const JointBeliefInterface &b, Index jaI, AlphaVector::BGPolicyIndex &betaMaxI) const
 
double GetQ (const JointBeliefInterface &b, Index t, Index jaI) const
 
double GetQ (const JointBeliefInterface &b, Index t, Index jaI, AlphaVector::BGPolicyIndex &betaMaxI) const
 
void Load (const std::string &filename)
 
 PerseusNonStationary (const PlanningUnitDecPOMDPDiscrete *pu)
 (default) Constructor More...
 
 PerseusNonStationary (const boost::shared_ptr< const PlanningUnitDecPOMDPDiscrete > &pu)
 
void Save (const std::string &filename) const
 
void SetBeliefSet (const BeliefSetNonStationary &S)
 Sets the belief set S that Perseus should use. More...
 
void SetValueFunction (const std::string &filename)
 Sets the value function. More...
 
void StoreValueFunction (const QFunctionsDiscreteNonStationary &Q)
 
virtual ~PerseusNonStationary ()
 Destructor. More...
 
- Public Member Functions inherited from Perseus
bool CheckConvergence (const std::vector< double > &VB, const std::vector< double > &VBnew, int iter) const
 
std::string GetIdentification () const
 
int GetVerbose () const
 
void Initialize ()
 
 Perseus (const PlanningUnitDecPOMDPDiscrete *pu)
 (default) Constructor More...
 
 Perseus (const boost::shared_ptr< const PlanningUnitDecPOMDPDiscrete > &pu)
 
 Perseus (const PlanningUnitFactoredDecPOMDPDiscrete *pu)
 
 Perseus (const boost::shared_ptr< const PlanningUnitFactoredDecPOMDPDiscrete > &pu)
 
int SampleNotImprovedBeliefIndex (std::vector< bool > stillNeedToBeImproved, int nrNotImproved) const
 Sample an index of a belief in the set which has not been improved. More...
 
void SetComputeVectorForEachBelief (bool compute)
 
void SetDryrun (bool dryrun)
 
void SetIdentification (const std::string &identification)
 
void SetInitializeWithImmediateReward (bool initReward)
 
void SetInitializeWithZero (bool initZero)
 
void SetMaximumNumberOfIterations (int nr)
 
void SetMinimumNumberOfIterations (int nr)
 
void SetResultsFilename (const std::string &filename)
 
void SetSaveIntermediateValueFunctions (bool save)
 
void SetSaveTimings (bool save)
 
void SetVerbose (int verbose)
 
virtual ~Perseus ()
 Destructor. More...
 
- Public Member Functions inherited from AlphaVectorPlanning
 AlphaVectorPlanning (const PlanningUnitDecPOMDPDiscrete *pu)
 (default) Constructor More...
 
 AlphaVectorPlanning (const boost::shared_ptr< const PlanningUnitDecPOMDPDiscrete > &pu)
 
 AlphaVectorPlanning (const PlanningUnitFactoredDecPOMDPDiscrete *pu)
 
 AlphaVectorPlanning (const boost::shared_ptr< const PlanningUnitFactoredDecPOMDPDiscrete > &pu)
 
GaoVectorSet BackProject (const ValueFunctionPOMDPDiscrete &v) const
 Back projects a value function. More...
 
GaoVectorSet BackProject (const VectorSet &v) const
 Back projects a value function, represented as a VectorSet. More...
 
VectorSet CrossSum (const VectorSet &A, const VectorSet &B) const
 Compute the cross-sum of two vector sets. More...
 
void ExportPOMDPFile (const std::string &filename) const
 Exports the POMDP to file named filename. More...
 
size_t GetAcceleratedPruningThreshold () const
 
ValueFunctionPOMDPDiscrete GetImmediateRewardValueFunction () const
 Returns the value function induced by the reward model. More...
 
const
PlanningUnitDecPOMDPDiscrete
GetPU () const
 Returns a ref to the PlanningUnit. More...
 
void Initialize ()
 
QFunctionsDiscrete Prune (const QFunctionsDiscrete &Q) const
 Prune a Q-valuefunction. More...
 
ValueFunctionPOMDPDiscrete Prune (const ValueFunctionPOMDPDiscrete &V) const
 Prune a POMDP valuefunction. More...
 
VectorSet Prune (const VectorSet &V) const
 Prune a VectorSet. More...
 
BeliefSet SampleBeliefs (const ArgumentHandlers::Arguments &args) const
 Sample a belief set according to the arguments. More...
 
BeliefSetNonStationary SampleBeliefsNonStationary (const ArgumentHandlers::Arguments &args) const
 Sample a non-stationary belief set according to the arguments. More...
 
BeliefSetNonStationary SampleBeliefsNonStationary (int nrBeliefs=10, int uniqueBeliefs=0, int resetAfter=0, int useQMDPforSamplingBeliefs=0, double QMDPexploreProb=0.1) const
 Sample a non-stationary belief set specifying each argument. More...
 
void SetAcceleratedPruningThreshold (size_t acceleratedPruningThreshold)
 
VectorSet Union (const VectorSet &A, const VectorSet &B) const
 Computes the union of two vector sets. More...
 
QFunctionsDiscrete ValueFunctionToQ (const ValueFunctionPOMDPDiscrete &V) const
 
virtual ~AlphaVectorPlanning ()
 Destructor. More...
 
- Public Member Functions inherited from TimedAlgorithm
void AddTimedEvent (const std::string &id, clock_t duration)
 Adds event of certain duration, e.g., an external program call. More...
 
std::vector< double > GetTimedEventDurations (const std::string &id)
 Returns all stored durations (in s) for a particular event. More...
 
void LoadTimers (const std::string &filename)
 Load timing info from file filename. More...
 
void PrintTimers () const
 Print stored timing info. More...
 
void PrintTimersSummary () const
 Sums data and prints out a summary. More...
 
void SaveTimers (const std::string &filename) const
 Save collected timing info to file filename. More...
 
void SaveTimers (std::ofstream &of) const
 Save collected timing info to ofstream of. More...
 
void StartTimer (const std::string &id) const
 Start to time an event identified by id. More...
 
void StopTimer (const std::string &id) const
 Stop to time an event identified by id. More...
 
 TimedAlgorithm ()
 (default) Constructor More...
 
virtual ~TimedAlgorithm ()
 Destructor. More...
 

Private Member Functions

virtual QFunctionsDiscrete BackupStageAll (const BeliefSet &S, const QFunctionsDiscrete &Q) const =0
 
virtual QFunctionsDiscrete BackupStageSampling (const BeliefSet &S, const QFunctionsDiscrete &Q) const =0
 
void PlanAll ()
 
void PlanSampling ()
 

Additional Inherited Members

- Static Public Member Functions inherited from Perseus
static std::string BackupTypeToString (const QAVParameters &params)
 
static QAVParameters ProcessArguments (const ArgumentHandlers::Arguments &args)
 
- Static Public Member Functions inherited from AlphaVectorPlanning
static void CrossSum (const std::vector< AlphaVector > &A, const std::vector< AlphaVector > &B, std::vector< AlphaVector > &output)
 Compute the cross-sum of two vector sets. More...
 
static bool EqualVS (const VectorSet &VS1, const VectorSet &VS2)
 
static void ExportBeliefSet (const BeliefSet &B, const std::string &filename)
 Exports a belief set to a file. More...
 
static void ExportPOMDPFile (const std::string &filename, const DecPOMDPDiscreteInterface *decpomdp)
 Exports the POMDP represented by pu to file named filename. More...
 
static void ExportValueFunction (const std::string &filename, const ValueFunctionPOMDPDiscrete &V, bool includeBGindices=true)
 Exports a value function V to file named filename. More...
 
static void ExportValueFunction (const std::string &filename, const QFunctionsDiscrete &Q, bool includeBGindices=true)
 Exports a Q functions Q to file named filename. More...
 
static void ExportValueFunction (const std::string &filename, const QFunctionsDiscreteNonStationary &Q, bool includeBGindices=true)
 Exports a Q functions Q to file named filename. More...
 
static std::vector< int > GetDuplicateIndices (const VectorSet &V)
 Returns a vector<int> which for each vector k in V specifies whether it is unique, in which case vector<int>[k] is set to -1, or whether it is a duplicate of another vector l in V, in which case vector<int>[k] is set to the index of l, where 0 <= l < V.size()). More...
 
static ValueFunctionPOMDPDiscrete GetImmediateRewardValueFunction (const PlanningUnitDecPOMDPDiscrete *pu)
 Returns the value function induced by the reward model of pu. More...
 
static ValueFunctionPOMDPDiscrete ImportValueFunction (const std::string &filename)
 Imports a value function from a file named filename. More...
 
static
QFunctionsDiscreteNonStationary 
ImportValueFunction (const std::string &filename, size_t nr, size_t nrA, size_t nrS)
 
static ValueFunctionPOMDPDiscrete QFunctionsToValueFunction (const QFunctionsDiscrete &Q)
 Takes all the vectors from the Q-function Q and throws them together as a 'V' function. More...
 
static std::string SoftPrint (const VectorSet &VS)
 
static QFunctionsDiscrete ValueFunctionToQ (const ValueFunctionPOMDPDiscrete &V, size_t nrA, size_t nrS)
 
static VectorSet ValueFunctionToVectorSet (const ValueFunctionPOMDPDiscrete &V)
 
static bool VectorIsDominated (const AlphaVector &alpha, const ValueFunctionPOMDPDiscrete &V)
 
static bool VectorIsDominated (Index i, const VectorSet &V, const std::vector< bool > &vectorsInVtoConsider)
 
static bool VectorIsInValueFunction (const AlphaVector &alpha, const ValueFunctionPOMDPDiscrete &V)
 
static VectorSetVectorOfVectorsToVectorSet (const std::vector< std::vector< double > > &vectors)
 
static ValueFunctionPOMDPDiscrete VectorSetToValueFunction (const VectorSet &VS, Index a=0, AlphaVector::BGPolicyIndex betaI=-1)
 
- Protected Member Functions inherited from PerseusNonStationary
std::vector< double > GetImmediateRewardBeliefSet () const
 
void InitializeBeliefs (int nrB, bool uniquify)
 
- Protected Member Functions inherited from Perseus
GaoVectorSet BackupStageLeadIn (const ValueFunctionPOMDPDiscrete &V) const
 
void BackupStageLeadOut (GaoVectorSet Gao) const
 
QFunctionsDiscreteNonStationary GetInitialNonStationaryQFunctions () const
 
QFunctionsDiscrete GetInitialQFunctions () const
 
ValueFunctionPOMDPDiscrete GetInitialValueFunction () const
 
template<class VF >
void PlanEndOfIteration (const VF &V)
 
void PlanEndOfIteration () const
 
void PlanLeadIn ()
 
void PlanLeadOut ()
 
template<class VF >
void PlanStartOfIteration (int iter, const std::vector< double > &VB, const VF &V) const
 
void PrintMaxRewardInBeliefSet () const
 Prints the maximum immediate reward present in the belief set. More...
 
virtual void StoreValueFunction (const ValueFunctionPOMDPDiscrete &V)
 
virtual void StoreValueFunction (const QFunctionsDiscrete &Q)
 
void UpdateValueFunctionName ()
 
- Protected Attributes inherited from PerseusNonStationary
BeliefSetNonStationary_m_beliefs
 The non-stationary belief set. More...
 
QFunctionsDiscreteNonStationary _m_qFunctions
 The resulting Q functions. More...
 
- Protected Attributes inherited from Perseus
bool _m_beliefsInitialized
 Whether or not the belief set has been set. More...
 
double _m_bestValue
 
bool _m_computeVectorForEachBelief
 
bool _m_dryrun
 
std::string _m_identification
 
std::string _m_resultsFilename
 
bool _m_storeIntermediateValueFunctions
 
bool _m_storeTimings
 
std::string _m_valueFunctionFilename
 

Detailed Description

PerseusNonStationaryQPlanner is a Perseus planner that uses non-stationary QFunctions.

Constructor & Destructor Documentation

PerseusNonStationaryQPlanner::PerseusNonStationaryQPlanner ( const PlanningUnitDecPOMDPDiscrete pu)
PerseusNonStationaryQPlanner::PerseusNonStationaryQPlanner ( const boost::shared_ptr< const PlanningUnitDecPOMDPDiscrete > &  pu)
PerseusNonStationaryQPlanner::~PerseusNonStationaryQPlanner ( )
virtual

Destructor.

Member Function Documentation

virtual QFunctionsDiscrete PerseusNonStationaryQPlanner::BackupStageAll ( const BeliefSet S,
const QFunctionsDiscrete Q 
) const
privatepure virtual

Implemented in PerseusBGNSPlanner.

Referenced by PlanAll().

virtual QFunctionsDiscrete PerseusNonStationaryQPlanner::BackupStageSampling ( const BeliefSet S,
const QFunctionsDiscrete Q 
) const
privatepure virtual

Implemented in PerseusBGNSPlanner.

Referenced by PlanSampling().

void PerseusNonStationaryQPlanner::Plan ( )
virtual