MultiAgentDecisionProcess
|
PerseusBGPOMDPPlanner implements the Perseus planning algorithm with for mixed BG/POMDP backups. More...
#include <PerseusBGPOMDPPlanner.h>
Public Member Functions | |
virtual AlphaVector | BeliefBackup (const JointBeliefInterface &b, Index a, const GaoVectorSet &G, const QFunctionsDiscrete &Q) const =0 |
PerseusBGPOMDPPlanner (const PlanningUnitDecPOMDPDiscrete *pu) | |
(default) Constructor More... | |
PerseusBGPOMDPPlanner (const boost::shared_ptr< const PlanningUnitDecPOMDPDiscrete > &pu) | |
virtual | ~PerseusBGPOMDPPlanner () |
Destructor. More... | |
Public Member Functions inherited from PerseusQFunctionPlanner | |
PerseusQFunctionPlanner (const PlanningUnitDecPOMDPDiscrete *pu) | |
PerseusQFunctionPlanner (const boost::shared_ptr< const PlanningUnitDecPOMDPDiscrete > &pu) | |
void | Plan () |
virtual | ~PerseusQFunctionPlanner () |
Destructor. More... | |
Public Member Functions inherited from PerseusStationary | |
void | ExportBeliefSet (const std::string &filename) const |
Exports the belief set in a simple matrix format. More... | |
void | ExportValueFunction (const std::string &filename) const |
Calls AlphaVectorPlanning::ExportValueFunction. More... | |
double | GetQ (const JointBeliefInterface &b, Index jaI) const |
double | GetQ (const JointBeliefInterface &b, Index jaI, AlphaVector::BGPolicyIndex &betaMaxI) const |
double | GetQ (const JointBeliefInterface &b, Index t, Index jaI) const |
double | GetQ (const JointBeliefInterface &b, Index t, Index jaI, AlphaVector::BGPolicyIndex &betaMaxI) const |
QFunctionsDiscrete | GetQFunctions () const |
ValueFunctionPOMDPDiscrete | GetValueFunction () const |
void | Load (const std::string &filename) |
PerseusStationary (const PlanningUnitDecPOMDPDiscrete *pu) | |
(default) Constructor More... | |
PerseusStationary (const boost::shared_ptr< const PlanningUnitDecPOMDPDiscrete > &pu) | |
PerseusStationary (const PlanningUnitFactoredDecPOMDPDiscrete *pu) | |
PerseusStationary (const boost::shared_ptr< const PlanningUnitFactoredDecPOMDPDiscrete > &pu) | |
void | Save (const std::string &filename) const |
void | SetBeliefSet (const BeliefSet &S) |
Sets the belief set S that Perseus should use. More... | |
void | SetValueFunction (const std::string &filename) |
Sets the value function. More... | |
virtual | ~PerseusStationary () |
Destructor. More... | |
Public Member Functions inherited from Perseus | |
bool | CheckConvergence (const std::vector< double > &VB, const std::vector< double > &VBnew, int iter) const |
std::string | GetIdentification () const |
int | GetVerbose () const |
void | Initialize () |
Perseus (const PlanningUnitDecPOMDPDiscrete *pu) | |
(default) Constructor More... | |
Perseus (const boost::shared_ptr< const PlanningUnitDecPOMDPDiscrete > &pu) | |
Perseus (const PlanningUnitFactoredDecPOMDPDiscrete *pu) | |
Perseus (const boost::shared_ptr< const PlanningUnitFactoredDecPOMDPDiscrete > &pu) | |
int | SampleNotImprovedBeliefIndex (std::vector< bool > stillNeedToBeImproved, int nrNotImproved) const |
Sample an index of a belief in the set which has not been improved. More... | |
void | SetComputeVectorForEachBelief (bool compute) |
void | SetDryrun (bool dryrun) |
void | SetIdentification (const std::string &identification) |
void | SetInitializeWithImmediateReward (bool initReward) |
void | SetInitializeWithZero (bool initZero) |
void | SetMaximumNumberOfIterations (int nr) |
void | SetMinimumNumberOfIterations (int nr) |
void | SetResultsFilename (const std::string &filename) |
void | SetSaveIntermediateValueFunctions (bool save) |
void | SetSaveTimings (bool save) |
void | SetVerbose (int verbose) |
virtual | ~Perseus () |
Destructor. More... | |
Public Member Functions inherited from AlphaVectorPlanning | |
AlphaVectorPlanning (const PlanningUnitDecPOMDPDiscrete *pu) | |
(default) Constructor More... | |
AlphaVectorPlanning (const boost::shared_ptr< const PlanningUnitDecPOMDPDiscrete > &pu) | |
AlphaVectorPlanning (const PlanningUnitFactoredDecPOMDPDiscrete *pu) | |
AlphaVectorPlanning (const boost::shared_ptr< const PlanningUnitFactoredDecPOMDPDiscrete > &pu) | |
GaoVectorSet | BackProject (const ValueFunctionPOMDPDiscrete &v) const |
Back projects a value function. More... | |
GaoVectorSet | BackProject (const VectorSet &v) const |
Back projects a value function, represented as a VectorSet. More... | |
VectorSet | CrossSum (const VectorSet &A, const VectorSet &B) const |
Compute the cross-sum of two vector sets. More... | |
void | ExportPOMDPFile (const std::string &filename) const |
Exports the POMDP to file named filename. More... | |
size_t | GetAcceleratedPruningThreshold () const |
ValueFunctionPOMDPDiscrete | GetImmediateRewardValueFunction () const |
Returns the value function induced by the reward model. More... | |
const PlanningUnitDecPOMDPDiscrete * | GetPU () const |
Returns a ref to the PlanningUnit. More... | |
void | Initialize () |
QFunctionsDiscrete | Prune (const QFunctionsDiscrete &Q) const |
Prune a Q-valuefunction. More... | |
ValueFunctionPOMDPDiscrete | Prune (const ValueFunctionPOMDPDiscrete &V) const |
Prune a POMDP valuefunction. More... | |
VectorSet | Prune (const VectorSet &V) const |
Prune a VectorSet. More... | |
BeliefSet | SampleBeliefs (const ArgumentHandlers::Arguments &args) const |
Sample a belief set according to the arguments. More... | |
BeliefSetNonStationary | SampleBeliefsNonStationary (const ArgumentHandlers::Arguments &args) const |
Sample a non-stationary belief set according to the arguments. More... | |
BeliefSetNonStationary | SampleBeliefsNonStationary (int nrBeliefs=10, int uniqueBeliefs=0, int resetAfter=0, int useQMDPforSamplingBeliefs=0, double QMDPexploreProb=0.1) const |
Sample a non-stationary belief set specifying each argument. More... | |
void | SetAcceleratedPruningThreshold (size_t acceleratedPruningThreshold) |
VectorSet | Union (const VectorSet &A, const VectorSet &B) const |
Computes the union of two vector sets. More... | |
QFunctionsDiscrete | ValueFunctionToQ (const ValueFunctionPOMDPDiscrete &V) const |
virtual | ~AlphaVectorPlanning () |
Destructor. More... | |
Public Member Functions inherited from TimedAlgorithm | |
void | AddTimedEvent (const std::string &id, clock_t duration) |
Adds event of certain duration, e.g., an external program call. More... | |
std::vector< double > | GetTimedEventDurations (const std::string &id) |
Returns all stored durations (in s) for a particular event. More... | |
void | LoadTimers (const std::string &filename) |
Load timing info from file filename. More... | |
void | PrintTimers () const |
Print stored timing info. More... | |
void | PrintTimersSummary () const |
Sums data and prints out a summary. More... | |
void | SaveTimers (const std::string &filename) const |
Save collected timing info to file filename. More... | |
void | SaveTimers (std::ofstream &of) const |
Save collected timing info to ofstream of. More... | |
void | StartTimer (const std::string &id) const |
Start to time an event identified by id. More... | |
void | StopTimer (const std::string &id) const |
Stop to time an event identified by id. More... | |
TimedAlgorithm () | |
(default) Constructor More... | |
virtual | ~TimedAlgorithm () |
Destructor. More... | |
Private Member Functions | |
QFunctionsDiscrete | BackupStage (const BeliefSet &S, const QFunctionsDiscrete &V) const |
Compute a Perseus backup stage. More... | |
QFunctionsDiscrete | BackupStageAll (const BeliefSet &S, const QFunctionsDiscrete &V) const |
QFunctionsDiscrete | BackupStageSamplingAlt (const BeliefSet &S, const QFunctionsDiscrete &V) const |
Additional Inherited Members | |
Static Public Member Functions inherited from Perseus | |
static std::string | BackupTypeToString (const QAVParameters ¶ms) |
static QAVParameters | ProcessArguments (const ArgumentHandlers::Arguments &args) |
Static Public Member Functions inherited from AlphaVectorPlanning | |
static void | CrossSum (const std::vector< AlphaVector > &A, const std::vector< AlphaVector > &B, std::vector< AlphaVector > &output) |
Compute the cross-sum of two vector sets. More... | |
static bool | EqualVS (const VectorSet &VS1, const VectorSet &VS2) |
static void | ExportBeliefSet (const BeliefSet &B, const std::string &filename) |
Exports a belief set to a file. More... | |
static void | ExportPOMDPFile (const std::string &filename, const DecPOMDPDiscreteInterface *decpomdp) |
Exports the POMDP represented by pu to file named filename. More... | |
static void | ExportValueFunction (const std::string &filename, const ValueFunctionPOMDPDiscrete &V, bool includeBGindices=true) |
Exports a value function V to file named filename. More... | |
static void | ExportValueFunction (const std::string &filename, const QFunctionsDiscrete &Q, bool includeBGindices=true) |
Exports a Q functions Q to file named filename. More... | |
static void | ExportValueFunction (const std::string &filename, const QFunctionsDiscreteNonStationary &Q, bool includeBGindices=true) |
Exports a Q functions Q to file named filename. More... | |
static std::vector< int > | GetDuplicateIndices (const VectorSet &V) |
Returns a vector<int> which for each vector k in V specifies whether it is unique, in which case vector<int>[k] is set to -1, or whether it is a duplicate of another vector l in V, in which case vector<int>[k] is set to the index of l, where 0 <= l < V.size()). More... | |
static ValueFunctionPOMDPDiscrete | GetImmediateRewardValueFunction (const PlanningUnitDecPOMDPDiscrete *pu) |
Returns the value function induced by the reward model of pu. More... | |
static ValueFunctionPOMDPDiscrete | ImportValueFunction (const std::string &filename) |
Imports a value function from a file named filename. More... | |
static QFunctionsDiscreteNonStationary | ImportValueFunction (const std::string &filename, size_t nr, size_t nrA, size_t nrS) |
static ValueFunctionPOMDPDiscrete | QFunctionsToValueFunction (const QFunctionsDiscrete &Q) |
Takes all the vectors from the Q-function Q and throws them together as a 'V' function. More... | |
static std::string | SoftPrint (const VectorSet &VS) |
static QFunctionsDiscrete | ValueFunctionToQ (const ValueFunctionPOMDPDiscrete &V, size_t nrA, size_t nrS) |
static VectorSet | ValueFunctionToVectorSet (const ValueFunctionPOMDPDiscrete &V) |
static bool | VectorIsDominated (const AlphaVector &alpha, const ValueFunctionPOMDPDiscrete &V) |
static bool | VectorIsDominated (Index i, const VectorSet &V, const std::vector< bool > &vectorsInVtoConsider) |
static bool | VectorIsInValueFunction (const AlphaVector &alpha, const ValueFunctionPOMDPDiscrete &V) |
static VectorSet * | VectorOfVectorsToVectorSet (const std::vector< std::vector< double > > &vectors) |
static ValueFunctionPOMDPDiscrete | VectorSetToValueFunction (const VectorSet &VS, Index a=0, AlphaVector::BGPolicyIndex betaI=-1) |
Protected Member Functions inherited from PerseusStationary | |
std::vector< double > | GetImmediateRewardBeliefSet () const |
void | InitializeBeliefs (int nrB, bool uniquify) |
void | StoreValueFunction (const ValueFunctionPOMDPDiscrete &V) |
void | StoreValueFunction (const QFunctionsDiscrete &Q) |
Protected Member Functions inherited from Perseus | |
GaoVectorSet | BackupStageLeadIn (const ValueFunctionPOMDPDiscrete &V) const |
void | BackupStageLeadOut (GaoVectorSet Gao) const |
QFunctionsDiscreteNonStationary | GetInitialNonStationaryQFunctions () const |
QFunctionsDiscrete | GetInitialQFunctions () const |
ValueFunctionPOMDPDiscrete | GetInitialValueFunction () const |
template<class VF > | |
void | PlanEndOfIteration (const VF &V) |
void | PlanEndOfIteration () const |
void | PlanLeadIn () |
void | PlanLeadOut () |
template<class VF > | |
void | PlanStartOfIteration (int iter, const std::vector< double > &VB, const VF &V) const |
void | PrintMaxRewardInBeliefSet () const |
Prints the maximum immediate reward present in the belief set. More... | |
virtual void | StoreValueFunction (const QFunctionsDiscreteNonStationary &Q) |
void | UpdateValueFunctionName () |
Protected Attributes inherited from PerseusStationary | |
BeliefSet * | _m_beliefs |
The belief set. More... | |
QFunctionsDiscrete | _m_qFunction |
The resulting Q functions, derived from _m_valueFunction. More... | |
ValueFunctionPOMDPDiscrete | _m_valueFunction |
The resulting value function. More... | |
Protected Attributes inherited from Perseus | |
bool | _m_beliefsInitialized |
Whether or not the belief set has been set. More... | |
double | _m_bestValue |
bool | _m_computeVectorForEachBelief |
bool | _m_dryrun |
std::string | _m_identification |
std::string | _m_resultsFilename |
bool | _m_storeIntermediateValueFunctions |
bool | _m_storeTimings |
std::string | _m_valueFunctionFilename |
PerseusBGPOMDPPlanner implements the Perseus planning algorithm with for mixed BG/POMDP backups.
PerseusBGPOMDPPlanner::PerseusBGPOMDPPlanner | ( | const PlanningUnitDecPOMDPDiscrete * | pu | ) |
(default) Constructor
PerseusBGPOMDPPlanner::PerseusBGPOMDPPlanner | ( | const boost::shared_ptr< const PlanningUnitDecPOMDPDiscrete > & | pu | ) |
|
virtual |
Destructor.
|
privatevirtual |
Compute a Perseus backup stage.
Implements PerseusQFunctionPlanner.
References Perseus::_m_computeVectorForEachBelief, BackupStageAll(), and BackupStageSamplingAlt().
|
private |
References Perseus::BackupStageLeadIn(), Perseus::BackupStageLeadOut(), BeliefBackup(), PlanningUnitMADPDiscrete::GetNrJointActions(), PlanningUnitMADPDiscrete::GetNrStates(), AlphaVectorPlanning::GetPU(), BeliefValue::GetValues(), Perseus::GetVerbose(), AlphaVectorPlanning::QFunctionsToValueFunction(), and AlphaVectorPlanning::VectorIsInValueFunction().
Referenced by BackupStage().
|
private |
References Perseus::BackupStageLeadIn(), Perseus::BackupStageLeadOut(), BeliefBackup(), BeliefValue::GetMaximizingVector(), PlanningUnitMADPDiscrete::GetNrJointActions(), PlanningUnitMADPDiscrete::GetNrStates(), AlphaVectorPlanning::GetPU(), BeliefValue::GetValues(), Perseus::GetVerbose(), AlphaVectorPlanning::QFunctionsToValueFunction(), Perseus::SampleNotImprovedBeliefIndex(), and AlphaVectorPlanning::VectorIsInValueFunction().
Referenced by BackupStage().
|
pure virtual |
Implemented in PerseusWeightedPlanner.
Referenced by BackupStageAll(), and BackupStageSamplingAlt().