MultiAgentDecisionProcess
|
GMAA_MAA_ELSI Generalized MAA* Exploiting Last-Stage Independence. More...
#include <GMAA_MAA_ELSI.h>
Public Member Functions | |
bool | ConstructAndValuateNextPoliciesExactBG (const boost::shared_ptr< PartialPolicyPoolItemInterface > &ppi, const boost::shared_ptr< PartialPolicyPoolInterface > &poolOfNextPolicies) |
BayesianGameIdenticalPayoff * | ConstructBayesianGame (const boost::shared_ptr< const PartialJointPolicyDiscretePure > &jpolPrevTs, std::vector< size_t > &nrOHts, size_t &nrJOHts, std::vector< Index > &firstOHtsI, double &ExpR_0_prevTS) |
construct a BG for stage t given ^t. More... | |
void | Fill_FirstOHtsI (Index ts, std::vector< Index > &firstOHtsI) |
Fills the (empty) vector firstOHtsI, with the indices (for each agent) of the first observation history of time step ts. More... | |
void | Fill_jaI_Array (Index ts, const JointObservationHistoryTree *joht, const boost::shared_ptr< const PartialJointPolicyDiscretePure > &jpolPrevTs, Index *jaI_arr) |
Fills the array jaI_arr with the joint actions taken for joht according to jpolPrevTs. More... | |
void | Fill_joI_Array (const Index ts, const std::vector< Index > &indTypes, const std::vector< Index > &firstOHtsI, Index *joI_arr) |
Fills the array of joint observation given the individual types and offsets (firstOHtsI). More... | |
const JointObservationHistoryTree * | Get_joht (const Index ts, const Index *joI_arr) |
Gets the joint observation history from joI_Array. More... | |
GMAA_MAA_ELSI (size_t horizon=3, FactoredDecPOMDPDiscreteInterface *p=0, const PlanningUnitMADPDiscreteParameters *params=0) | |
Constructor. More... | |
JointBeliefInterface * | ProbRewardForjoahI (Index ts, Index jtI, Index *jaI_arr, Index *joI_arr, Index &jaohI, double &PjaohI, double &ExpR_0_prevTS_thisJAOH) |
Calculates the jaohI corresponding to jaI_arr and joI_arr and also returnes the P(jaohI) and the expected obtained reward for previous time steps GIVEN this joint action history. More... | |
Public Member Functions inherited from GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete | |
GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete (size_t horizon=3, FactoredDecPOMDPDiscreteInterface *p=0, const PlanningUnitMADPDiscreteParameters *params=0, int verbose_level=0, double slack=0.0) | |
(default) Constructor More... | |
double | GetExpectedReward () const |
Returns the expected reward of the best found joint policy. More... | |
boost::shared_ptr< JointPolicy > | GetJointPolicy () |
Returns the found joint policy. More... | |
virtual GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete * | GetThisFromMostDerivedPU () |
every derived class must implement this function as follows: More... | |
GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete & | operator= (const GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete &o) |
Copy assignment operator. More... | |
void | Plan () |
The function that performs the actual planning. More... | |
void | SetFactoredQHeuristic (FactoredQFunctionStateJAOHInterface &q) |
void | SetFactoredQHeuristic (FactoredQFunctionStateJAOHInterface *q) |
Public Member Functions inherited from PlanningUnitFactoredDecPOMDPDiscrete | |
void | ComputeFSAOHDist (FactoredStateAOHDistribution *fsaoh, const PartialJointPolicyDiscretePure &pJPol) const |
const FactoredDecPOMDPDiscreteInterface * | GetFDPOMDPD () const |
Returns the FactoredDecPOMDPDiscreteInterface pointer. More... | |
virtual FactoredStateAOHDistribution * | GetNewFactoredStateAOHDistribution () const |
PlanningUnitFactoredDecPOMDPDiscrete (size_t horizon=3, FactoredDecPOMDPDiscreteInterface *p=0, const PlanningUnitMADPDiscreteParameters *params=0) | |
the (default) Constructor. More... | |
void | SetProblem (FactoredDecPOMDPDiscreteInterface *p) |
Set the problem (FactoredDecPOMDPDiscreteInterface) using a pointer. More... | |
std::vector< Index > | StateIndexToFactorValueIndices (Index stateI) const |
Public Member Functions inherited from PlanningUnitDecPOMDPDiscrete | |
void | ExportDecPOMDPFile (const std::string &filename) const |
Exports the Dec-POMDP to file named filename. More... | |
double | GetDiscount () const |
Returns the discount parameter. More... | |
DecPOMDPDiscreteInterface * | GetDPOMDPD () const |
Returns the DecPOMDPDiscreteInterface pointer. More... | |
virtual boost::shared_ptr < JointPolicyPureVector > | GetJointPolicyPureVector () |
Returns the found joint policy. More... | |
double | GetReward (Index sI, Index jaI) const |
Return the reward for state, joint action indices. More... | |
PlanningUnitDecPOMDPDiscrete (size_t horizon=3, DecPOMDPDiscreteInterface *p=0, const PlanningUnitMADPDiscreteParameters *params=0) | |
the (default) Constructor. More... | |
void | SetProblem (DecPOMDPDiscreteInterface *p) |
Tell which SetReferred to use by default. More... | |
Public Member Functions inherited from PlanningUnitMADPDiscrete | |
virtual bool | AreCachedJointToIndivIndices (PolicyGlobals::PolicyDomainCategory pdc) const |
Check whether certain index conversions are cached. More... | |
void | ComputeHistoryArrays (Index hI, Index t, Index t_offset, Index Indices[], size_t indexDomainSize) const |
This function computes the indices of the sequence corr. More... | |
Index | ComputeHistoryIndex (Index t, Index t_offset, const Index indices[], size_t indexDomainSize) const |
This function computes the index of a history. More... | |
void | ExportDotGraph (const std::string &filename, const PolicyDiscretePure &policy, Index agentI, bool labelEdges=true) const |
Export a policy in dot format (from the GraphViz tools). More... | |
const Action * | GetAction (Index agentI, Index a) const |
Returns a ref to the a-th action of agent agentI. More... | |
void | GetActionHistoryArrays (Index agentI, Index ahI, Index t, Index aIs[]) const |
Computes the joint actions for jahI. More... | |
ActionHistoryTree * | GetActionHistoryTree (Index agentI, Index ohI) const |
Returns a pointer to action history# ohI of agent# agentI. More... | |
void | GetActionObservationHistoryArrays (Index agentI, Index aohI, Index t, Index aIs[], Index oIs[]) const |
Computes the joint actions and observations for aohI. More... | |
Index | GetActionObservationHistoryIndex (Index agentI, Index t, const std::vector< Index > &actions, const std::vector< Index > &observations) const |
converts the vectors of actions and observations of length t to a (individual) ObservationHistory Index for agentI. More... | |
ActionObservationHistoryTree * | GetActionObservationHistoryTree (Index agentI, Index aohI) const |
Returns a pointer to observation history# ohI of agent# agentI. More... | |
virtual PolicyGlobals::PolicyDomainCategory | GetDefaultIndexDomCat () const |
Return the default PolicyDomainCategory for the problem. More... | |
LIndex | GetFirstJointActionObservationHistoryIndex (Index ts) const |
Returns the index of the first joint action observation history of time step ts. More... | |
LIndex | GetFirstJointObservationHistoryIndex (Index ts) const |
Returns the index of the first joint observation history of time step ts. More... | |
LIndex | GetFirstObservationHistoryIndex (Index agI, Index ts) const |
Returns the index of the first ts observation history of agent agI. More... | |
double | GetInitialStateProbability (Index sI) const |
Returns the probability of a state in the initial state distribution. More... | |
double | GetJAOHProb (LIndex jaohI, Index p_jaohI=0, const JointBeliefInterface *p_jb=NULL, const JointPolicyDiscrete *jpol=NULL) const |
returns the probability of jaohI. More... | |
double | GetJAOHProbGivenPred (LIndex jaohI) const |
Gives the conditional probability of the realization of the joint action-observation history jaohI (and thus of the joint belief corresponding to JointActionObservationHistory jaohI). More... | |
double | GetJAOHProbs (JointBeliefInterface *jb, LIndex jaohI, LIndex p_jaohI=0, const JointBeliefInterface *p_jb=NULL, const JointPolicyDiscrete *jpol=NULL) const |
returns the probability of jaohI AND the corresponding joint belief given the predecessor p_jaohI (and its corresponding belief) More... | |
double | GetJAOHProbsRecursively (JointBeliefInterface *jb, const Index jaIs[], const Index joIs[], Index t_p, Index t, LIndex p_jaohI=0, const JointPolicyDiscrete *jpol=NULL) const |
the function that perfoms most of the work, called by GetJAOHProbs. More... | |
const JointAction * | GetJointAction (Index jaI) const |
Returns a ref to the i-th joint action. More... | |
Index | GetJointActionHistoryIndex (Index t, const std::vector< Index > &jointActions) const |
converts the vector joint observations of length t to a JointObservationHistory Index. More... | |
Index | GetJointActionHistoryIndex (Index t, const Index jointActions[]) const |
converts the vector joint observations of length t to a JointObservationHistory Index. More... | |
Index | GetJointActionHistoryIndex (JointActionHistoryTree *joh) const |
Returns the index of a JointActionHistoryTree pointer. More... | |
JointActionHistoryTree * | GetJointActionHistoryTree (Index jahI) const |
Returns a pointer to joint action history#. More... | |
void | GetJointActionObservationHistoryArrays (LIndex jaohI, Index t, Index jaIs[], Index joIs[]) const |
Computes the joint actions and observations for jaohI. More... | |
LIndex | GetJointActionObservationHistoryIndex (JointActionObservationHistoryTree *jaoh) const |
Returns the index of a JointActionObservationHistoryTree pointer. More... | |
Index | GetJointActionObservationHistoryIndex (Index t, const std::vector< Index > &Jactions, const std::vector< Index > &Jobservations) const |
converts the vectors of actions and observations of length t to a joint ActionObservationHistory Index. More... | |
Index | GetJointActionObservationHistoryIndex (Index t, const std::vector< Index > &Jactions, const std::vector< Index > &Jobservations, const Scope &agentSope) const |
like GetJointActionObservationHistoryIndex, but works on a subset of agents. More... | |
JointActionObservationHistoryTree * | GetJointActionObservationHistoryTree (LIndex jaohI) const |
Returns a pointer to JointActionObservation history#. More... | |
void | GetJointActionObservationHistoryVectors (LIndex jaohI, std::vector< Index > &jaIs, std::vector< Index > &joIs) const |
returns the vectors with joint actions and observations for JointActionObservationHistory jaohi. More... | |
JointBeliefInterface * | GetJointBeliefInterface (LIndex jaohI) const |
brief Returns a pointer to a new joint belief. More... | |
const JointObservation * | GetJointObservation (Index joI) const |
Returns a ref to the joI-th joint observation. More... | |
void | GetJointObservationHistoryArrays (Index johI, Index t, Index joIs[]) const |
Computes the joint observations for johI. More... | |
Index | GetJointObservationHistoryIndex (JointObservationHistoryTree *joh) const |
Returns the index of a JointObservationHistoryTree pointer. More... | |
Index | GetJointObservationHistoryIndex (Index t, const std::vector< Index > &jointObservations) const |
converts the vector joint observations of length t to a JointObservationHistory Index. More... | |
Index | GetJointObservationHistoryIndex (Index t, const Index jointObservations[]) const |
converts the vector joint observations of length t to a JointObservationHistory Index. More... | |
JointObservationHistoryTree * | GetJointObservationHistoryTree (Index johI) const |
Returns a pointer to joint observation history#. More... | |
MultiAgentDecisionProcessDiscreteInterface * | GetMADPDI () |
const MultiAgentDecisionProcessDiscreteInterface * | GetMADPDI () const |
JointBeliefInterface * | GetNewJointBeliefFromISD () const |
Returns a new joint belief with the value of the initial state distribution. More... | |
virtual JointBeliefInterface * | GetNewJointBeliefInterface () const |
a function that forces derives classes to specify which types of joint beliefs are used. More... | |
virtual JointBeliefInterface * | GetNewJointBeliefInterface (size_t size) const |
size_t | GetNrActionHistories (Index agentI) const |
Returns the number of action histories for agentI. More... | |
size_t | GetNrActionHistories (Index agentI, Index ts) const |
Returns the number of action histories for agentI for time step ts. More... | |
size_t | GetNrActionObservationHistories (Index agentI) const |
Returns the number of action observation histories for agentI. More... | |
const std::vector< size_t > & | GetNrActions () const |
Returns the number of actions vector. More... | |
size_t | GetNrActions (Index agentI) const |
Returns the number of actions of agent agentI. More... | |
size_t | GetNrAgents () const |
Gets the number of agents. More... | |
size_t | GetNrJointActionHistories () const |
Returns the number of joint action histories. More... | |
size_t | GetNrJointActionObservationHistories () const |
Returns the number of jointActionObservation histories. More... | |
size_t | GetNrJointActions () const |
return the number of joint actions. More... | |
size_t | GetNrJointObservationHistories () const |
Returns the number of joint observation histories. More... | |
size_t | GetNrJointObservationHistories (Index ts) const |
Returns the number of joint observation histories for time step ts. More... | |
size_t | GetNrJointObservations () const |
Returns the number of joint observations. More... | |
LIndex | GetNrJointPolicies () const |
Returns the number of joint policies. More... | |
size_t | GetNrObservationHistories (Index agentI) const |
Returns the number of observation histories for agentI. More... | |
size_t | GetNrObservationHistories (Index agentI, Index ts) const |
Returns the number of observation histories for agentI for time step ts. More... | |
const std::vector< size_t > | GetNrObservationHistoriesVector () const |
Returns a vector with the number of OHs for each agent. More... | |
const std::vector< size_t > | GetNrObservationHistoriesVector (Index ts) const |
Returns a vector with the number of OHs in stage ts for each agent. More... | |
const std::vector< size_t > & | GetNrObservations () const |
Returns the number of observations vector. More... | |
size_t | GetNrObservations (Index agentI) const |
Returns the number of observations of agent agentI. More... | |
LIndex | GetNrPolicies (Index agentI) const |
Returns the number of policies for agentI. More... | |
size_t | GetNrPolicyDomainElements (Index agentI, PolicyGlobals::PolicyDomainCategory cat, size_t depth=MAXHORIZON) const |
Get the number of elements in the domain of an agent's policy. More... | |
size_t | GetNrStates () const |
Returns the number of states. More... | |
const Observation * | GetObservation (Index agentI, Index o) const |
Returns a ref to the o-th observation of agent agentI. More... | |
void | GetObservationHistoryArrays (Index agentI, Index ohI, Index t, Index oIs[]) const |
Computes the observations for ohI. More... | |
Index | GetObservationHistoryIndex (Index agentI, Index t, const std::vector< Index > &observations) const |
converts the vector observations of length t to a (individual) ObservationHistory Index for agentI. More... | |
ObservationHistoryTree * | GetObservationHistoryTree (Index agentI, Index ohI) const |
Returns a pointer to observation history# ohI of agent# agentI. More... | |
const ObservationModelDiscrete * | GetObservationModelDiscretePtr () const |
double | GetObservationProbability (Index jaI, Index sucSI, Index joI) const |
Returns P(joI | jaI, sucSI ). Arguments are time-ordered. More... | |
const PlanningUnitMADPDiscreteParameters & | GetParams () const |
Get the parameters for this planning unit. More... | |
const MultiAgentDecisionProcessDiscreteInterface * | GetProblem () const |
Returns a reference to the problem of the PlanningUnitMADPDiscrete. More... | |
MultiAgentDecisionProcessDiscreteInterface * | GetProblem () |
const State * | GetState (Index i) const |
Get a pointer to a State by index. More... | |
Index | GetSuccessorAHI (Index agentI, Index ohI, Index oI) const |
Index | GetSuccessorAOHI (Index agI, Index aohI, Index aI, Index oI) const |
Returns the index of the successor of joint action-observation history jaohI via joint action jaI and joint observation joI. More... | |
Index | GetSuccessorJAHI (Index johI, Index joI) const |
LIndex | GetSuccessorJAOHI (LIndex jaohI, Index jaI, Index joI) const |
Returns the index of the successor of agent agI's action-observation history aohI via action aI and observation oI. More... | |
Index | GetSuccessorJOHI (Index johI, Index joI) const |
Returns the index of the successor of observation history johI via joint observation joI. More... | |
Index | GetSuccessorOHI (Index agentI, Index ohI, Index oI) const |
Returns the index of the successor of observation history ohI of agentI via observation joI. More... | |
Index | GetTimeStepForAHI (Index agentI, Index ohI) const |
Returns the time step of observation history ohI. More... | |
Index | GetTimeStepForAOHI (Index agentI, Index aohI) const |
Returns the time step of joint action-observation historyaohI. More... | |
Index | GetTimeStepForJAHI (Index johI) const |
Returns the time step of joint observation history johI. More... | |
Index | GetTimeStepForJAOHI (LIndex jaohI) const |
Returns the time step of joint action-observation historyjaohI. More... | |
Index | GetTimeStepForJOHI (Index johI) const |
Returns the time step of joint observation history johI. More... | |
Index | GetTimeStepForOHI (Index agentI, Index ohI) const |
Returns the time step of observation history ohI. More... | |
const TransitionModelDiscrete * | GetTransitionModelDiscretePtr () const |
double | GetTransitionProbability (Index sI, Index jaI, Index sucSI) const |
Returns the trans. prob for state, joint action, suc state indices. More... | |
Index | IndividualToJointActionHistoryIndex (Index t, const std::vector< Index > &indivIs) const |
converts individual history indices to a joint index More... | |
Index | IndividualToJointActionIndices (const Index *indivActionIndices) const |
Returns the joint action index that corresponds to the array of specified individual action indices. More... | |
Index | IndividualToJointActionIndices (const std::vector< Index > &indivActionIndices) const |
Returns the joint action index that corresponds to the vector of specified individual action indices. More... | |
LIndex | IndividualToJointActionObservationHistoryIndex (Index t, const std::vector< Index > &indivIs) const |
converts individual history indices to a joint index More... | |
Index | IndividualToJointObservationHistoryIndex (Index t, const std::vector< Index > &indivIs) const |
converts individual history indices to a joint index More... | |
Index | IndividualToJointObservationIndices (const std::vector< Index > &inObs) const |
Returns the joint observation index that corresponds to the vector of specified individual observation indices. More... | |
void | JointAOHIndexToIndividualActionObservationVectors (LIndex jaohI, std::vector< std::vector< Index > > &indivO_vec, std::vector< std::vector< Index > > &indivA_vec) const |
computes the vectors of actions and obs. More... | |
void | JointAOHIndexToIndividualActionObservationVectors (LIndex jaohI, std::vector< std::vector< Index > > &indivAO_vec) const |
computes the vector of action-observations corresponding to jaohI indivAO_vec[agentI][t] = aoI More... | |
std::vector< Index > | JointToIndividualActionHistoryIndices (Index JAHistI) const |
Returns a vector containing the indices of the individual ObservationHistory s corresponding to the JointActionHistory index JAHistI. More... | |
const std::vector< Index > & | JointToIndividualActionHistoryIndicesRef (Index JAHistI) const |
Returns a reference to a cached vector containing the indices of the indiv. More... | |
std::vector< Index > | JointToIndividualActionIndices (Index jaI) const |
Returns a vector containing the indices of the indiv. More... | |
std::vector< Index > | JointToIndividualActionObservationHistoryIndices (LIndex jaohI) const |
Returns a vector containing the indices of the indiv. More... | |
const std::vector< Index > & | JointToIndividualActionObservationHistoryIndicesRef (LIndex jaohI) const |
Returns a vector containing the indices of the indiv. More... | |
std::vector< Index > | JointToIndividualObservationHistoryIndices (Index johI) const |
Returns a vector containing the indices of the indiv. More... | |
const std::vector< Index > & | JointToIndividualObservationHistoryIndicesRef (Index johI) const |
Returns a vector containing the indices of the indiv. More... | |
std::vector< Index > | JointToIndividualObservationIndices (Index joI) const |
Returns a vector containing the indices of the indiv. More... | |
std::vector< Index > | JointToIndividualPolicyDomainIndices (Index jdI, PolicyGlobals::PolicyDomainCategory cat) const |
Converts joint indices to individual policy domain element indices. More... | |
const std::vector< Index > & | JointToIndividualPolicyDomainIndicesRef (Index jdI, PolicyGlobals::PolicyDomainCategory cat) const |
Converts individual policy domain element indices to joint indices. More... | |
PlanningUnitMADPDiscrete (size_t horizon=3, MultiAgentDecisionProcessDiscreteInterface *p=0, const PlanningUnitMADPDiscreteParameters *params=0) | |
Constructor with specified parameters. More... | |
PlanningUnitMADPDiscrete (size_t horizon=3, MultiAgentDecisionProcessDiscreteInterface *p=0) | |
Constructor with default parameters. DEPRECATED?: More... | |
std::string | PolicyToDotGraph (const PolicyDiscretePure &policy, Index agentI, bool labelEdges=true) const |
Convert a policy to dot format (from the GraphViz tools). More... | |
void | Print () |
Prints info regarding the planning unit. More... | |
void | PrintActionHistories () |
Prints the action histories for all agents. More... | |
void | PrintActionObservationHistories () |
Prints the actionObservation histories for all agents. More... | |
void | PrintObservationHistories () |
Prints the observation histories for all agents. More... | |
void | RegisterJointActionObservationHistoryTree (JointActionObservationHistoryTree *jaoht) |
Register a new jaoht in the vector of indices. More... | |
void | SetHorizon (size_t h) |
Sets the horizon for the planning problem. More... | |
void | SetParams (const PlanningUnitMADPDiscreteParameters ¶ms) |
Sets the parameters for this planning unit. More... | |
void | SetProblem (MultiAgentDecisionProcessDiscreteInterface *madp) |
Sets the problem for which to plan, using a pointer. More... | |
std::string | SoftPrintAction (Index agentI, Index actionI) const |
soft prints action actionI of agent agentI. More... | |
std::string | SoftPrintObservationHistory (Index agentI, Index ohIndex) const |
soft prints ObservationHistory ohIndex of agent agentI. More... | |
std::string | SoftPrintPolicyDomainElement (Index agentI, Index dI, PolicyGlobals::PolicyDomainCategory cat) const |
Virtual function that has to be implemented by derived class. More... | |
~PlanningUnitMADPDiscrete () | |
Destructor. More... | |
Public Member Functions inherited from PlanningUnit | |
size_t | GetHorizon () const |
Returns the planning horizon. More... | |
Index | GetNextAgentIndex () |
Maintains a agent index and returns the next one on calling */. More... | |
size_t | GetNrAgents () const |
Return the number of agents. More... | |
const MultiAgentDecisionProcessInterface * | GetProblem () const |
Get the problem pointer. More... | |
int | GetSeed () const |
Returns the random seed stored. More... | |
void | InitSeed () const |
Initializes the random number generator (srand) to the stored seed. More... | |
PlanningUnit (size_t horizon, MultiAgentDecisionProcessInterface *p) | |
(default) Constructor More... | |
void | SetProblem (MultiAgentDecisionProcessInterface *p) |
Updates the problem pointer. More... | |
void | SetSeed (int s) |
Stores the random seed and calls InitSeed(). More... | |
virtual | ~PlanningUnit () |
Destructor. More... | |
Public Member Functions inherited from Interface_ProblemToPolicyDiscretePure | |
LIndex | GetNrJointPolicies (PolicyGlobals::PolicyDomainCategory cat, size_t depth=MAXHORIZON) const |
Get the number of joint policies, given the policy's domain. More... | |
LIndex | GetNrPolicies (Index ag, PolicyGlobals::PolicyDomainCategory cat, size_t depth=MAXHORIZON) const |
Get the number of policies for an agent, given the policy's domain. More... | |
virtual | ~Interface_ProblemToPolicyDiscretePure () |
Destructor. More... | |
Public Member Functions inherited from Interface_ProblemToPolicyDiscrete | |
size_t | GetNrJointActions () const |
Get the number of joint actions. More... | |
Interface_ProblemToPolicyDiscrete () | |
(default) Constructor More... | |
virtual | ~Interface_ProblemToPolicyDiscrete () |
Destructor. More... | |
Public Member Functions inherited from GeneralizedMAAStarPlanner | |
GeneralizedMAAStarPlanner (int verbose_level=0, double slack=0.0) | |
(default) Constructor More... | |
double | GetExpectedReward () const |
boost::shared_ptr< JointPolicy > | GetJointPolicy () |
boost::shared_ptr < JointPolicyDiscrete > | GetJointPolicyDiscrete () |
boost::shared_ptr < JointPolicyDiscretePure > | GetJointPolicyDiscretePure () |
LIndex | GetMaxJPolPoolSize () const |
double | GetMaxLowerBound () const |
LIndex | GetNrEvaluatedJPolBGs () const |
GeneralizedMAAStarPlanner & | operator= (const GeneralizedMAAStarPlanner &o) |
Copy assignment operator. More... | |
void | Plan () |
void | SetDeadline (size_t deadlineInS) |
void | SetIntermediateResultFile (std::ofstream &of) |
void | SetIntermediateTimingFilename (const std::string &filename) |
void | SetSaveAllBGs (const std::string &filename) |
void | SetVerbose (int verbose) |
~GeneralizedMAAStarPlanner () | |
Destructor. More... | |
Public Member Functions inherited from TimedAlgorithm | |
void | AddTimedEvent (const std::string &id, clock_t duration) |
Adds event of certain duration, e.g., an external program call. More... | |
std::vector< double > | GetTimedEventDurations (const std::string &id) |
Returns all stored durations (in s) for a particular event. More... | |
void | LoadTimers (const std::string &filename) |
Load timing info from file filename. More... | |
void | PrintTimers () const |
Print stored timing info. More... | |
void | PrintTimersSummary () const |
Sums data and prints out a summary. More... | |
void | SaveTimers (const std::string &filename) const |
Save collected timing info to file filename. More... | |
void | SaveTimers (std::ofstream &of) const |
Save collected timing info to ofstream of. More... | |
void | StartTimer (const std::string &id) const |
Start to time an event identified by id. More... | |
void | StopTimer (const std::string &id) const |
Stop to time an event identified by id. More... | |
TimedAlgorithm () | |
(default) Constructor More... | |
virtual | ~TimedAlgorithm () |
Destructor. More... | |
Protected Member Functions | |
bool | CAVNP_quick_n_dirty2 (const boost::shared_ptr< PartialPolicyPoolItemInterface > &ppi, const boost::shared_ptr< PartialPolicyPoolInterface > &poolOfNextPolicies) |
bool | ConstructAndValuateNextPolicies (const boost::shared_ptr< PartialPolicyPoolItemInterface > &ppi, const boost::shared_ptr< PartialPolicyPoolInterface > &poolOfNextPolicies, bool &cleanUpPPI) |
the (main part of the) 'NEXT' function from refGMAA. More... | |
void | ResetPlanner () |
This should reset the planner, so it can be started from the beginning. More... | |
void | SelectPoliciesToProcessFurther (const boost::shared_ptr< PartialPolicyPoolInterface > &poolOfNextPolicies, bool are_LBs, double bestLB) |
Limits the policies to be further examined. More... | |
Protected Member Functions inherited from GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete | |
virtual boost::shared_ptr < PartialJointPolicyDiscretePure > | ConstructExtendedJointPolicy (const PartialJointPolicyDiscretePure &jpolPrevTs, const JointPolicyDiscretePure &jpolBG, const std::vector< size_t > &nrOHts, const std::vector< Index > &firstOHtsI) |
Extends a previous policy jpolPrevTs to the next stage. More... | |
Scope | GetAgentScopeForLQF (Index e, Index stage) const |
double | GetHeuristicLocalQValue (Index lqf, Index stage, std::vector< Index > sfacValues, std::vector< Index > aoHistIs, Index agSc_jaI) const |
double | GetHeuristicQ (Index joahI, Index jaI) const |
size_t | GetNrHeuristicLQFs (Index stage) const |
Return the number of local Q value functions for stage. More... | |
Scope | GetStateFactorScopeForLQF (Index e, Index stage) const |
virtual boost::shared_ptr < PartialJointPolicyDiscretePure > | NewJPol () const |
Returns a new policy of the type used by the GMAA implementation. More... | |
virtual boost::shared_ptr < PartialPolicyPoolInterface > | NewPP () const |
Return a new PartialPolicyPoolInterface*. More... | |
virtual boost::shared_ptr < PartialPolicyPoolItemInterface > | NewPPI (const boost::shared_ptr< PartialJointPolicyDiscretePure > &jp, double v) const |
Return a new PartialPolicyPoolItemInterface*. More... | |
void | SetCBGbounds (const boost::shared_ptr< PartialPolicyPoolItemInterface > &ppi, const boost::shared_ptr< BayesianGameIdenticalPayoffSolver > &bgips) |
Protected Member Functions inherited from PlanningUnitDecPOMDPDiscrete | |
bool | SanityCheck () const |
Runs some consistency tests. More... | |
Protected Member Functions inherited from GeneralizedMAAStarPlanner | |
void | Prune (PartialPolicyPoolInterface &JPVs, size_t k) |
void | SelectKBestPoliciesToProcessFurther (const boost::shared_ptr< PartialPolicyPoolInterface > &poolOfNextPolicies, bool are_LBs, double bestLB, size_t k) |
Returns the k best-ranked (partial) joint policies. More... | |
void | SetCBGbounds (const boost::shared_ptr< PartialPolicyPoolItemInterface > &ppi, const boost::shared_ptr< BayesianGameIdenticalPayoffSolver > &bgips, bool is_last_ts, double discount) |
Additional Inherited Members | |
Static Public Member Functions inherited from PlanningUnitDecPOMDPDiscrete | |
static void | ExportDecPOMDPFile (const std::string &filename, const DecPOMDPDiscreteInterface *decpomdp) |
Exports the Dec-POMDP represented by pu to file named filename. More... | |
Protected Attributes inherited from GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete | |
FactoredQFunctionStateJAOHInterface * | _m_qHeuristic |
A pointer to the heuristic used by this. More... | |
Protected Attributes inherited from PlanningUnitMADPDiscrete | |
std::vector< ActionHistoryTree * > | _m_actionHistoryTreeRootPointers |
A vector that stores pointers to the roots of the action history trees of each agent. More... | |
std::vector< std::vector < ActionHistoryTree * > > | _m_actionHistoryTreeVectors |
A vector which, for each agents, stores a vector with all ActionHistoryTree pointers. More... | |
std::vector < ActionObservationHistoryTree * > | _m_actionObservationHistoryTreeRootPointers |
A vector that stores pointers to the roots of the action-observation history trees of each agent. More... | |
std::vector< std::vector < ActionObservationHistoryTree * > > | _m_actionObservationHistoryTreeVectors |
A vector which, for each agents, stores a vector with all ActionObservationHistoryTree pointers. More... | |
std::vector< std::vector < LIndex > > | _m_firstAHIforT |
The _m_firstAHIforT[aI][t] contains the first action history for time-step t of agent aI. More... | |
std::vector< std::vector < LIndex > > | _m_firstAOHIforT |
The _m_firstAOHIforT[aI][t] contains the first actionObservation history for time-step t of agent aI. More... | |
std::vector< LIndex > | _m_firstJAHIforT |
The _m_firstJAHIforT[t] contains the first joint action history for time-step t. More... | |
std::vector< LIndex > | _m_firstJAOHIforT |
_m_firstJAOHIforT[t] contains the first joint actionObservation history for time-step t. More... | |
std::vector< LIndex > | _m_firstJOHIforT |
The _m_firstJOHIforT[t] contains the first joint observation history for time-step t. More... | |
std::vector< std::vector < LIndex > > | _m_firstOHIforT |
The _m_firstOHIforT[aI][t] contains the first observation history for time-step t of agent aI. More... | |
std::vector< double > | _m_jaohConditionalProbs |
Stores the conditional probability of this joint belief. More... | |
std::vector< double > | _m_jaohProbs |
Caches the probabilities of JointActionObservationHistory's (assuming b^0 is as specified by the problem and that a pure joint policy consistent with the i-th JointActionObservationHistory is followed). More... | |
std::vector< const JointBeliefInterface * > | _m_jBeliefCache |
_m_jBeliefCache[i] stores a pointer to the joint belief corresponding to the i-th JointActionObservationHistory (assuming b^0 is as specified by the problem and that a pure joint policy consistent with the i-th JointActionObservationHistory is followed) More... | |
JointActionHistoryTree * | _m_jointActionHistoryTreeRoot |
The root node of the joint action histories tree. More... | |
std::vector < JointActionHistoryTree * > | _m_jointActionHistoryTreeVector |
A vector which stores a JointActionHistoryTree pointer. More... | |
std::map< LIndex, JointActionObservationHistoryTree * > | _m_jointActionObservationHistoryTreeMap |
A map which is used instead of _m_jointActionObservationHistoryTreeVector when we don't cache all JointActionObservationHistoryTree's. More... | |
JointActionObservationHistoryTree * | _m_jointActionObservationHistoryTreeRoot |
The root node of the joint actionObservation histories tree. More... | |
std::vector < JointActionObservationHistoryTree * > | _m_jointActionObservationHistoryTreeVector |
A vector which stores JointActionObservationHistoryTree pointer. More... | |
JointObservationHistoryTree * | _m_jointObservationHistoryTreeRoot |
The root node of the joint observation histories tree. More... | |
std::vector < JointObservationHistoryTree * > | _m_jointObservationHistoryTreeVector |
A vector which stores a JointObservationHistoryTree pointer. More... | |
std::vector< size_t > | _m_nrActionHistories |
A vector that keeps track of the number of action histories per agent. More... | |
std::vector< std::vector < size_t > > | _m_nrActionHistoriesT |
A vector that keeps track of the number of action histories per agent per time step. More... | |
std::vector< size_t > | _m_nrActionObservationHistories |
A vector that keeps track of the number of action-obs. More... | |
std::vector< std::vector < size_t > > | _m_nrActionObservationHistoriesT |
Keeps track of the number of action-obs. More... | |
size_t | _m_nrJointActionHistories |
The number of joint action histories. More... | |
std::vector< size_t > | _m_nrJointActionHistoriesT |
The number of joint action histories per time-step. More... | |
size_t | _m_nrJointActionObservationHistories |
The number of joint actionAction histories. More... | |
std::vector< size_t > | _m_nrJointActionObservationHistoriesT |
The number of joint actionObservation histories per time-step. More... | |
size_t | _m_nrJointObservationHistories |
The number of joint observation histories. More... | |
std::vector< size_t > | _m_nrJointObservationHistoriesT |
The number of joint observation histories per time-step. More... | |
std::vector< size_t > | _m_nrObservationHistories |
A vector that keeps track of the number of observation histories per agent. More... | |
std::vector< std::vector < size_t > > | _m_nrObservationHistoriesT |
Keeps track of the number of observation histories per agent per time step. More... | |
std::vector < ObservationHistoryTree * > | _m_observationHistoryTreeRootPointers |
A vector that stores pointers to the roots of the observation history trees of each agent. More... | |
std::vector< std::vector < ObservationHistoryTree * > > | _m_observationHistoryTreeVectors |
A vector which, for each agents, stores a vector with all ObservationHistoryTree pointers. More... | |
Protected Attributes inherited from GeneralizedMAAStarPlanner | |
std::string | _m_bgBaseFilename |
size_t | _m_bgCounter |
std::vector< size_t > | _m_expanded_childs |
_m_expanded_childs[t] contains the number of child nodes that were expanded 'at stage t'. More... | |
std::ofstream * | _m_intermediateResultFile |
Pointer to an file stream to store the intermediate (timing) results. More... | |
std::string | _m_intermediateTimingFilename |
std::vector< LIndex > | _m_max_expanded_childs |
LIndex | _m_nrJPolBGsEvaluated |
size_t | _m_nrPoliciesToProcess |
std::vector < BayesianGameForDecPOMDPStage * > | _m_pointersToAllBGTS |
bool | _m_saveIntermediateTiming |
double | _m_slack |
when the heuristic is not admissible, or the past reward is an approximation, we may add some slack such that good policies are not pruned More... | |
bool | _m_useSparseBeliefs |
int | _m_verboseness |
the level of verboseness, default=0, >0 verbose, <0 silent More... | |
GMAA_MAA_ELSI Generalized MAA* Exploiting Last-Stage Independence.
is a class that represents a planner that performs MAA* as described by Szer et al. (see refMAA in DOC-References.h) for factored DecPOMDPs (FactoredDecPOMDPDiscreteInterface)
MAA* performs a heuristic search for the optimal policy. When used with an admissible heuristic it is guaranteed to find the optimal policy.
GMAA_MAA_ELSI exploits locality of interaction in the last stage: rather than constructing a BG and enumerating all joint BG policies, it constructs a CGBG, and solves it with non-serial dynamic programming (NDP)
GMAA_MAA_ELSI::GMAA_MAA_ELSI | ( | size_t | horizon = 3 , |
FactoredDecPOMDPDiscreteInterface * | p = 0 , |
||
const PlanningUnitMADPDiscreteParameters * | params = 0 |
||
) |
Constructor.
Takes the planning horizon as argument and a pointer to the FactoredDecPOMDPDiscreteInterface for which planning takes place. the booleans arguments are passed down to PlanningUnitMADPDiscrete. .
References DEBUG_PU_CONSTRUCTORS.
|
protected |
References GeneralizedMAAStarPlanner::_m_bgBaseFilename, GeneralizedMAAStarPlanner::_m_bgCounter, GeneralizedMAAStarPlanner::_m_verboseness, ConstructAndValuateNextPoliciesExactBG(), GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete::ConstructExtendedJointPolicy(), Fill_FirstOHtsI(), Fill_jaI_Array(), Fill_joI_Array(), Get_joht(), GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete::GetAgentScopeForLQF(), GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete::GetHeuristicLocalQValue(), PlanningUnit::GetHorizon(), BayesianGameIdenticalPayoffSolver::GetJointPolicyPureVector(), PlanningUnitMADPDiscrete::GetNrActions(), PlanningUnitMADPDiscrete::GetNrAgents(), GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete::GetNrHeuristicLQFs(), PlanningUnitMADPDiscrete::GetNrJointObservationHistories(), PlanningUnitMADPDiscrete::GetNrObservationHistories(), PlanningUnitMADPDiscrete::GetNrStates(), GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete::GetStateFactorScopeForLQF(), PlanningUnitMADPDiscrete::JointToIndividualActionObservationHistoryIndices(), GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete::NewPPI(), ProbRewardForjoahI(), IndexTools::RestrictIndividualIndicesToScope(), BGCG_SolverNonserialDynamicProgramming::Solve(), TimedAlgorithm::StartTimer(), PlanningUnitFactoredDecPOMDPDiscrete::StateIndexToFactorValueIndices(), and TimedAlgorithm::StopTimer().
Referenced by ConstructAndValuateNextPolicies().
|
protectedvirtual |
the (main part of the) 'NEXT' function from refGMAA.
The function that from a given partial policy from the policy pool constructs a new set of (partial) joint policies.
This function can be overrides the one in the base class GeneralizedMAAStarPlannerForDecPOMDPDiscrete
Implements GeneralizedMAAStarPlanner.
References CAVNP_quick_n_dirty2().
bool GMAA_MAA_ELSI::ConstructAndValuateNextPoliciesExactBG | ( | const boost::shared_ptr< PartialPolicyPoolItemInterface > & | ppi, |
const boost::shared_ptr< PartialPolicyPoolInterface > & | poolOfNextPolicies | ||
) |
References GeneralizedMAAStarPlanner::_m_nrJPolBGsEvaluated, GeneralizedMAAStarPlanner::_m_verboseness, ConstructBayesianGame(), GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete::ConstructExtendedJointPolicy(), PlanningUnit::GetHorizon(), GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete::NewPPI(), PrintTools::PrintProgress(), TimedAlgorithm::StartTimer(), and TimedAlgorithm::StopTimer().
Referenced by CAVNP_quick_n_dirty2().
BayesianGameIdenticalPayoff * GMAA_MAA_ELSI::ConstructBayesianGame | ( | const boost::shared_ptr< const PartialJointPolicyDiscretePure > & | jpolPrevTs, |
std::vector< size_t > & | nrOHts, | ||
size_t & | nrJOHts, | ||
std::vector< Index > & | firstOHtsI, | ||
double & | ExpR_0_prevTS | ||
) |
construct a BG for stage t given ^t.
jpolPrevTs is a partial pure joint policy that specifies actions up to stage t. Given this `past' policy, this function constructs a BG for stage t.
Additionally returned are nrOHts - a vector containing the number of obs-histories for of stage t for each agent. nrJOHts - the number of joint observation histories for stage t firstOHtsI - the indices of the first OH of stage t for each agent ExpR_0_prevTS - the expected reward for stages 0...(t-1) given jpolPrevTs.
References GeneralizedMAAStarPlanner::_m_bgBaseFilename, GeneralizedMAAStarPlanner::_m_bgCounter, GeneralizedMAAStarPlanner::_m_verboseness, Fill_FirstOHtsI(), Fill_jaI_Array(), Fill_joI_Array(), Get_joht(), GeneralizedMAAStarPlannerForFactoredDecPOMDPDiscrete::GetHeuristicQ(), PlanningUnitMADPDiscrete::GetNrActions(), PlanningUnitMADPDiscrete::GetNrAgents(), PlanningUnitMADPDiscrete::GetNrJointActions(), PlanningUnitMADPDiscrete::GetNrJointObservationHistories(), BayesianGameBase::GetNrJointPolicies(), PlanningUnitMADPDiscrete::GetNrObservationHistories(), BayesianGameBase::JointToIndividualTypeIndices(), PrintTools::PrintProgress(), ProbRewardForjoahI(), BayesianGameIdenticalPayoff::Save(), BayesianGameBase::SetProbability(), BayesianGameIdenticalPayoff::SetUtility(), TimedAlgorithm::StartTimer(), and TimedAlgorithm::StopTimer().
Referenced by ConstructAndValuateNextPoliciesExactBG().
Fills the (empty) vector firstOHtsI, with the indices (for each agent) of the first observation history of time step ts.
References Globals::CastLIndexToIndex(), PlanningUnitMADPDiscrete::GetFirstObservationHistoryIndex(), and PlanningUnitMADPDiscrete::GetNrAgents().
Referenced by CAVNP_quick_n_dirty2(), and ConstructBayesianGame().
void GMAA_MAA_ELSI::Fill_jaI_Array | ( | Index | ts, |
const JointObservationHistoryTree * | joht, | ||
const boost::shared_ptr< const PartialJointPolicyDiscretePure > & | jpolPrevTs, | ||
Index * | jaI_arr | ||
) |
Fills the array jaI_arr with the joint actions taken for joht according to jpolPrevTs.
References Globals::CastLIndexToIndex(), TreeNode< Tcontained >::GetIndex(), and JointObservationHistoryTree::GetPredecessor().
Referenced by CAVNP_quick_n_dirty2(), and ConstructBayesianGame().
void GMAA_MAA_ELSI::Fill_joI_Array | ( | const Index | ts, |
const std::vector< Index > & | indTypes, | ||
const std::vector< Index > & | firstOHtsI, | ||
Index * | joI_arr | ||
) |
Fills the array of joint observation given the individual types and offsets (firstOHtsI).
References PlanningUnitMADPDiscrete::GetNrAgents(), ObservationHistoryTree::GetObservationHistory(), PlanningUnitMADPDiscrete::GetObservationHistoryTree(), and PlanningUnitMADPDiscrete::IndividualToJointObservationIndices().
Referenced by CAVNP_quick_n_dirty2(), and ConstructBayesianGame().
const JointObservationHistoryTree * GMAA_MAA_ELSI::Get_joht | ( | const Index | ts, |
const Index * | joI_arr | ||
) |
Gets the joint observation history from joI_Array.
References PlanningUnitMADPDiscrete::GetJointObservationHistoryTree(), and JointObservationHistoryTree::GetSuccessor().
Referenced by CAVNP_quick_n_dirty2(), and ConstructBayesianGame().
JointBeliefInterface * GMAA_MAA_ELSI::ProbRewardForjoahI | ( | Index | ts, |
Index | jtI, | ||
Index * | jaI_arr, | ||
Index * | joI_arr, | ||
Index & | jaohI, | ||
double & | PjaohI, | ||
double & | ExpR_0_prevTS_thisJAOH | ||
) |
Calculates the jaohI corresponding to jaI_arr and joI_arr and also returnes the P(jaohI) and the expected obtained reward for previous time steps GIVEN this joint action history.
input args Index ts, Index jtI, Index* jaI_arr,Index* joI_arr, output args Index& jaohI, double& PjaohI, double& ExpR_0_prevTS_thisJAOH
basically this function is a form of PlanningUnitMADPDiscrete::GetJAOHProbs(Recursively) that also computes the reward.
References Globals::CastLIndexToIndex(), DEBUG_GMAA4, BeliefInterface::Get(), PlanningUnitDecPOMDPDiscrete::GetDPOMDPD(), TreeNode< Tcontained >::GetIndex(), JointActionObservationHistoryTree::GetJointActionObservationHistory(), PlanningUnitMADPDiscrete::GetJointActionObservationHistoryTree(), PlanningUnitMADPDiscrete::GetNewJointBeliefFromISD(), PlanningUnitMADPDiscrete::GetNrStates(), PlanningUnitDecPOMDPDiscrete::GetReward(), JointActionObservationHistoryTree::GetSuccessor(), JointActionObservationHistory::Print(), and JointBeliefInterface::Update().
Referenced by CAVNP_quick_n_dirty2(), and ConstructBayesianGame().
|
protectedvirtual |
This should reset the planner, so it can be started from the beginning.
Implements GeneralizedMAAStarPlanner.
|
protectedvirtual |
Limits the policies to be further examined.
Of the <jpol,val> pairs found by ConstructAndValuateNextPolicies, we may not want to process all of them further. This function performs a selection. This function should be overriden in derived classes to get different planning behavior.
Implements GeneralizedMAAStarPlanner.