MultiAgentDecisionProcess
BayesianGameForDecPOMDPStage Class Reference

BayesianGameForDecPOMDPStage represents a BG for a single stage. More...

#include <BayesianGameForDecPOMDPStage.h>

Inheritance diagram for BayesianGameForDecPOMDPStage:
[legend]

Public Member Functions

 BayesianGameForDecPOMDPStage (const PlanningUnitDecPOMDPDiscrete *pu, const QFunctionJAOHInterface *q, const boost::shared_ptr< const PartialJointPolicyDiscretePure > &pastJPol)
 Constructor that creates and initializes a BG from scratch. More...
 
 BayesianGameForDecPOMDPStage (const PlanningUnitDecPOMDPDiscrete *pu)
 Constructor that creates an empty BG. More...
 
 BayesianGameForDecPOMDPStage (const BayesianGameForDecPOMDPStage &a)
 Copy constructor. More...
 
void ClearAllImmediateRewards ()
 We can also clear this cache. More...
 
void ComputeAllImmediateRewards (const FactoredDecPOMDPDiscreteInterface *fd=0)
 When performing a lot of GetImmediateReward calls we can first compute a cache of immediate rewards, to speed things up. More...
 
double ComputeDiscountedImmediateRewardForJPol (const boost::shared_ptr< JointPolicyDiscretePure > &jpolBG, const PlanningUnitFactoredDecPOMDPDiscrete *pu=0) const
 Compute the discounted expected imm reward for jpolBG. More...
 
double GetImmediateReward (Index jtI, Index jaI, const FactoredDecPOMDPDiscreteInterface *fd=0) const
 Returns the (expected) immediate reward for jtI, jaI. More...
 
const
PlanningUnitDecPOMDPDiscrete
GetPUDecPOMDPDiscrete () const
 
const QFunctionJAOHInterfaceGetQHeur () const
 
BayesianGameForDecPOMDPStageoperator= (const BayesianGameForDecPOMDPStage &o)
 Copy assignment operator. More...
 
void Print () const
 Print this BayesianGameIdenticalPayoff to cout. More...
 
std::string SoftPrint () const
 Prints a description of this entire BayesianGameIdenticalPayoff to a string. More...
 
 ~BayesianGameForDecPOMDPStage ()
 Destructor. More...
 
- Public Member Functions inherited from BayesianGameForDecPOMDPStageInterface
 BayesianGameForDecPOMDPStageInterface ()
 Constructor without arguments, needed for serialization. More...
 
 BayesianGameForDecPOMDPStageInterface (const boost::shared_ptr< const PartialJointPolicyDiscretePure > &pastJPol)
 
 BayesianGameForDecPOMDPStageInterface (Index t)
 
boost::shared_ptr< const
PartialJointPolicyDiscretePure
GetPastJointPolicy () const
 
Index GetStage () const
 
virtual ~BayesianGameForDecPOMDPStageInterface ()
 Destructor. More...
 
- Public Member Functions inherited from BayesianGameIdenticalPayoff
 BayesianGameIdenticalPayoff ()
 
 BayesianGameIdenticalPayoff (const BayesianGameIdenticalPayoff &o)
 
 BayesianGameIdenticalPayoff (size_t nrAgents, const std::vector< size_t > &nrActions, const std::vector< size_t > &nrTypes, bool useSparseRewardModel=false)
 
virtual double ComputeValueJPol (const JointPolicyDiscretePure &jpolBG) const
 evaluates the value of a joint policy. More...
 
double GetUtility (const Index jtype, const Index ja) const
 Gets the utility for (for all agents) jtype, ja. More...
 
double GetUtility (const std::vector< Index > &indTypeIndices, const std::vector< Index > &indActionIndices) const
 Gets the utility for (for all agents) joint type corresponding to the individual type indices (indTypeIndices) and joint action corresponding to individual action indices (indActionIndices). More...
 
BayesianGameIdenticalPayoffoperator= (const BayesianGameIdenticalPayoff &o)
 Copy assignment operator. More...
 
void Print () const
 Print this BayesianGameIdenticalPayoff to cout. More...
 
void PrintUtilForJointType (Index jtype) const
 Prints the utilities for jtype. More...
 
bool SetInitialized (bool b)
 Sets the initialized status to b. More...
 
void SetUtility (const Index jtype, const Index ja, const double u)
 Sets the utility for (for all agents) jtype, ja to u. More...
 
void SetUtility (const std::vector< Index > &indTypeIndices, const std::vector< Index > &indActionIndices, const double u)
 Sets the utility for (for all agents) joint type corresponding to the individual type indices (indTypeIndices) and joint action corresponding to individual action indices (indActionIndices). More...
 
std::string SoftPrint () const
 Prints a description of this entire BayesianGameIdenticalPayoff to a string. More...
 
std::string SoftPrintUtilForJointType (Index jtype) const
 Prints the utilities for jtype. More...
 
virtual ~BayesianGameIdenticalPayoff ()
 Destructor. More...
 
- Public Member Functions inherited from BayesianGameIdenticalPayoffInterface
 BayesianGameIdenticalPayoffInterface ()
 (default) Constructor More...
 
 BayesianGameIdenticalPayoffInterface (size_t nrAgents, const std::vector< size_t > &nrActions, const std::vector< size_t > &nrTypes)
 
double ComputeValueJPol (const JointPolicyDiscretePure &jpolBG) const
 
BayesianGameIdenticalPayoffInterfaceoperator= (const BayesianGameIdenticalPayoffInterface &o)
 
- Public Member Functions inherited from BayesianGameBase
void AddProbability (Index i, double p)
 Adds p to the probability of joint type i. More...
 
void AddProbability (const std::vector< Index > &indIndices, double p)
 Adds p to the probability of joint type corresponding to the individual type indices (indIndices). More...
 
virtual bool AreCachedJointToIndivIndices (const PolicyGlobals::PolicyDomainCategory pdc) const
 Check whether certain index conversions are cached. More...
 
 BayesianGameBase ()
 
 BayesianGameBase (size_t nrAgents, const std::vector< size_t > &nrActions, const std::vector< size_t > &nrTypes, int verboseness=0)
 
 BayesianGameBase (const BayesianGameBase &a)
 Copy constructor. More...
 
bool CacheJointToIndivAOH_Indices () const
 
bool CacheJointToIndivOH_Indices () const
 
bool CacheJointToIndivType_Indices () const
 
virtual
PolicyGlobals::PolicyDomainCategory 
GetDefaultIndexDomCat () const
 Return the default PolicyDomainCategory for the problem. More...
 
size_t GetNrActions (Index agentI) const
 Get the number of invididual actions of a particular agent. More...
 
const std::vector< size_t > & GetNrActions () const
 
size_t GetNrAgents () const
 implement the Interface_ProblemToPolicyDiscrete interface: More...
 
size_t GetNrJointActions () const
 
LIndex GetNrJointPolicies () const
 
size_t GetNrJointTypes () const
 
LIndex GetNrPolicies (Index ag) const
 
size_t GetNrPolicyDomainElements (Index agentI, PolicyGlobals::PolicyDomainCategory cat, size_t depth=MAXHORIZON) const
 Get the number of elements in the domain of an agent's policy. More...
 
const std::vector< size_t > & GetNrTypes () const
 
size_t GetNrTypes (Index agI) const
 
virtual double GetProbability (Index i) const
 Gets the probability of joint type i. More...
 
virtual double GetProbability (const std::vector< Index > &indIndices) const
 Gets the probability of joint type corresponding to the individual type indices (indIndices) More...
 
Index IndividualToJointActionIndices (const Index *IndArr) const
 Converts individual action indices to a joint action index. More...
 
Index IndividualToJointActionIndices (const std::vector< Index > &indices) const
 Converts individual action indices to a joint action index. More...
 
Index IndividualToJointTypeIndices (const std::vector< Index > &indices) const
 
std::vector< IndexJointToIndividualActionIndices (Index jaI) const
 
std::vector< IndexJointToIndividualPolicyDomainIndices (Index jdI, PolicyGlobals::PolicyDomainCategory cat) const
 implementation of JointToIndividualPolicyDomainIndices More...
 
const std::vector< Index > & JointToIndividualPolicyDomainIndicesRef (Index jdI, PolicyGlobals::PolicyDomainCategory cat) const
 implementation of JointToIndividualPolicyDomainIndicesRef More...
 
const std::vector< Index > & JointToIndividualTypeIndices (Index jTypeI) const
 
BayesianGameBaseoperator= (const BayesianGameBase &o)
 
void Print () const
 Print this BayesianGameBase to cout. More...
 
virtual void SanityCheck ()
 Sanity check should be overriden by classes that do not use the implementation provided by this class (e.g. More...
 
void SanityCheckBGBase ()
 
bool SetInitialized (bool b)
 Sets the initialized status to b. More...
 
void SetProbability (Index i, double p)
 Sets the probability of joint type i to p. More...
 
void SetProbability (const std::vector< Index > &indIndices, double p)
 Sets the probability of joint type corresponding to the individual type indices (indIndices) to p. More...
 
std::string SoftPrint () const
 Prints a description of this BayesianGameBase to a string. More...
 
std::string SoftPrintAction (Index agentI, Index actionI) const
 Virtual function that has to be implemented by derived class. More...
 
std::string SoftPrintPolicyDomainElement (Index agentI, Index typeIndex, PolicyGlobals::PolicyDomainCategory cat) const
 Virtual function that has to be implemented by derived class. More...
 
std::string SoftPrintSummary () const
 
std::string SoftPrintType (Index agentI, Index typeIndex) const
 
 ~BayesianGameBase ()
 Destructor. More...
 
- Public Member Functions inherited from Interface_ProblemToPolicyDiscretePure
LIndex GetNrJointPolicies (PolicyGlobals::PolicyDomainCategory cat, size_t depth=MAXHORIZON) const
 Get the number of joint policies, given the policy's domain. More...
 
LIndex GetNrPolicies (Index ag, PolicyGlobals::PolicyDomainCategory cat, size_t depth=MAXHORIZON) const
 Get the number of policies for an agent, given the policy's domain. More...
 
virtual ~Interface_ProblemToPolicyDiscretePure ()
 Destructor. More...
 
- Public Member Functions inherited from Interface_ProblemToPolicyDiscrete
size_t GetNrJointActions () const
 Get the number of joint actions. More...
 
 Interface_ProblemToPolicyDiscrete ()
 (default) Constructor More...
 
virtual ~Interface_ProblemToPolicyDiscrete ()
 Destructor. More...
 

Protected Member Functions

 BayesianGameForDecPOMDPStage (const PlanningUnitDecPOMDPDiscrete *pu, const QFunctionJAOHInterface *q, Index t, size_t nrAgents, const std::vector< size_t > &nrActions, const std::vector< size_t > &nrTypes)
 Constructor that only creates a BG of specified dimensions. More...
 
double ComputeImmediateReward (Index jtI, Index jaI) const
 Compute the immediate reward for an action and joint type. More...
 
PartialJointPolicyDiscretePureConstructExtendedPolicy (PartialJointPolicyDiscretePure &jpolPrevTs, JointPolicyDiscretePure &jpolBG, std::vector< size_t > &nrOHts, std::vector< Index > &firstOHtsI)
 Extends a previous policy jpolPrevTs to the next stage. More...
 
void Fill_FirstOHtsI (Index ts, std::vector< Index > &firstOHtsI)
 Fills the (empty) vector firstOHtsI, with the indices (for each agent) of the first observation history of time step ts. More...
 
void Fill_jaI_Array (Index ts, Index joIs[], boost::shared_ptr< const JointPolicyDiscretePure > jpolPrevTs, Index *jaI_arr)
 Fills the array jaI_arr with the joint actions taken for the JOHs as specified by the array of joint observations joIs according to jpolPrevTs. More...
 
void Fill_joI_Array (const Index ts, const std::vector< Index > &indTypes, const std::vector< Index > &firstOHtsI, Index *joI_arr)
 Fills the array of joint observation given the individual types and offsets (firstOHtsI). More...
 
void Initialize ()
 Initialized the BG - called from constructor. More...
 
void ProbRewardForjoahI (Index ts, Index jtI, Index *jaI_arr, Index *joI_arr, Index &jaohI, double &PjaohI, double &ExpR_0_prevTS_thisJAOH)
 Calculates the jaohI corresponding to jaI_arr and joI_arr and also returnes the P(jaohI) and the expected obtained reward for previous time steps GIVEN this joint action history. More...
 
- Protected Member Functions inherited from BayesianGameBase
void ChangeNrActions (Index agI, size_t new_nr)
 
void ChangeNrTypes (Index agI, size_t new_nr)
 
void Initialize ()
 

Protected Attributes

bool _m_areCachedImmediateRewards
 are the immediate rewards cached? More...
 
std::vector< std::vector
< double > > 
_m_immR
 the cache for the immediate rewards: immR[jt][ja] More...
 
std::vector
< JointBeliefInterface * > 
_m_JBs
 The joint beliefs induced by the joint types. More...
 
- Protected Attributes inherited from BayesianGameBase
bool _m_initialized
 private bool to indicate whether this BG is initialized. More...
 
bool _m_JAoverflow
 Boolean that indicates whether the number of joint actions can be represented by size_t (or that overflowing occurs) More...
 
bool _m_JToverflow
 Boolean that indicates whether the number of joint types can be represented by size_t (or that overflowing occurs) More...
 
std::vector< size_t > _m_nrActions
 the number of actions for each agent More...
 
size_t _m_nrAgents
 the number of players (or agents) More...
 
size_t _m_nrJA
 the number of joint types: More...
 
size_t _m_nrJTypes
 the number of joint actions: More...
 
std::vector< size_t > _m_nrTypes
 the number of types for each agent More...
 
size_t * _m_stepSizeActions
 an size_t array that caches the stepsize array for actions: More...
 
size_t * _m_stepSizeTypes
 an size_t array that caches the stepsize array for types: More...
 
int _m_verboseness
 _m_verboseness >0 verbose, <0 is quiet More...
 

Private Attributes

const
PlanningUnitDecPOMDPDiscrete
_m_pu
 Stores pointer to the PU. More...
 
const QFunctionJAOHInterface_m_qHeuristic
 A pointer to the heuristic used by this Bayesian game —nec.? More...
 

Additional Inherited Members

- Static Public Member Functions inherited from BayesianGameIdenticalPayoff
static BayesianGameIdenticalPayoff GenerateRandomBG (size_t nrAgents, std::vector< size_t > acs, std::vector< size_t > obs)
 Generates a random BG with identical payoffs. More...
 
static BayesianGameIdenticalPayoff Load (const std::string &filename)
 Loads a BG from file. More...
 
static BayesianGameIdenticalPayoff LoadTextFormat (const std::string &filename)
 
static void Save (const BayesianGameIdenticalPayoff &bg, const std::string &filename)
 
static void SaveTextFormat (const BayesianGameIdenticalPayoff &bg, const std::string &filename)
 

Detailed Description

BayesianGameForDecPOMDPStage represents a BG for a single stage.

Constructor & Destructor Documentation

BayesianGameForDecPOMDPStage::BayesianGameForDecPOMDPStage ( const PlanningUnitDecPOMDPDiscrete pu,
const QFunctionJAOHInterface q,
Index  t,
size_t  nrAgents,
const std::vector< size_t > &  nrActions,
const std::vector< size_t > &  nrTypes 
)
protected

Constructor that only creates a BG of specified dimensions.

This constructor does not initialize the BG. This is useful when there are additional computations that have to be done. E.g., when this is a clusterable BG (BayesianGameWithClusterInfo), the types do not correspond in a straigtforward manner to action-observ. histories and therefore the probs. and payoff function have to be computed at this higher level.

BayesianGameForDecPOMDPStage::BayesianGameForDecPOMDPStage ( const PlanningUnitDecPOMDPDiscrete pu,
const QFunctionJAOHInterface q,
const boost::shared_ptr< const PartialJointPolicyDiscretePure > &  pastJPol 
)

Constructor that creates and initializes a BG from scratch.

This constructor creates and initializes a BG for the next stage given the past policy and q function.

References Initialize().

BayesianGameForDecPOMDPStage::BayesianGameForDecPOMDPStage ( const PlanningUnitDecPOMDPDiscrete pu)

Constructor that creates an empty BG.

BayesianGameForDecPOMDPStage::BayesianGameForDecPOMDPStage ( const BayesianGameForDecPOMDPStage a)

Copy constructor.

References _m_JBs.

BayesianGameForDecPOMDPStage::~BayesianGameForDecPOMDPStage ( )

Destructor.

References _m_JBs.

Member Function Documentation

void BayesianGameForDecPOMDPStage::ClearAllImmediateRewards ( )
inlinevirtual
void BayesianGameForDecPOMDPStage::ComputeAllImmediateRewards ( const FactoredDecPOMDPDiscreteInterface fd = 0)
virtual

When performing a lot of GetImmediateReward calls we can first compute a cache of immediate rewards, to speed things up.

Implements BayesianGameForDecPOMDPStageInterface.

References _m_areCachedImmediateRewards, _m_immR, ComputeImmediateReward(), BayesianGameBase::GetNrJointActions(), and BayesianGameBase::GetNrJointTypes().

Referenced by GMAA_MAAstarClassic::ConstructAndValuateNextPolicies().

double BayesianGameForDecPOMDPStage::ComputeDiscountedImmediateRewardForJPol ( const boost::shared_ptr< JointPolicyDiscretePure > &  jpolBG,
const PlanningUnitFactoredDecPOMDPDiscrete pu = 0 
) const
virtual
double BayesianGameForDecPOMDPStage::ComputeImmediateReward ( Index  jtI,
Index  jaI 
) const
protected
PartialJointPolicyDiscretePure * BayesianGameForDecPOMDPStage::ConstructExtendedPolicy ( PartialJointPolicyDiscretePure jpolPrevTs,
JointPolicyDiscretePure jpolBG,
std::vector< size_t > &  nrOHts,
std::vector< Index > &  firstOHtsI 
)
protected

Extends a previous policy jpolPrevTs to the next stage.

This function extends a previous policy jpolPrevTs for ts-1 with the behavior specified by the policy of the BayesianGame for time step ts (jpolBG). jpolPrevTs - a joint policy for the DecPOMDP up to time step ts-1 (i.e. with depth=ts-2) jpolBG - a joint policy for the BayesianGame for time step ts. nrOHts - a vector that specifies the number of observation histories for eac agents at time step ts. firstOHtsI - a vector that specifies the index of the first time step ts observation history for each agent (this functions as the offset in the conversion BG->DecPOMDP index conversion).

returns a new JointPolicyDiscretePure (so it must be explicitly deleted)

References JointPolicyDiscretePure::GetActionIndex(), JointPolicy::GetDepth(), BayesianGameBase::GetNrAgents(), JointPolicyDiscrete::GetPolicyDomainCategory(), PolicyGlobals::OHIST_INDEX, JointPolicyDiscretePure::SetAction(), JointPolicy::SetDepth(), and PolicyGlobals::TYPE_INDEX.

void BayesianGameForDecPOMDPStage::Fill_FirstOHtsI ( Index  ts,
std::vector< Index > &  firstOHtsI 
)
protected

Fills the (empty) vector firstOHtsI, with the indices (for each agent) of the first observation history of time step ts.

References _m_pu, Globals::CastLIndexToIndex(), PlanningUnitMADPDiscrete::GetFirstObservationHistoryIndex(), and BayesianGameBase::GetNrAgents().

Referenced by Initialize().

void BayesianGameForDecPOMDPStage::Fill_jaI_Array ( Index  ts,
Index  joIs[],
boost::shared_ptr< const JointPolicyDiscretePure jpolPrevTs,
Index jaI_arr 
)
protected

Fills the array jaI_arr with the joint actions taken for the JOHs as specified by the array of joint observations joIs according to jpolPrevTs.

References _m_pu, and PlanningUnitMADPDiscrete::GetSuccessorJOHI().

Referenced by Initialize().

void BayesianGameForDecPOMDPStage::Fill_joI_Array ( const Index  ts,
const std::vector< Index > &  indTypes,
const std::vector< Index > &  firstOHtsI,
Index joI_arr 
)
protected

Fills the array of joint observation given the individual types and offsets (firstOHtsI).

References _m_pu, BayesianGameBase::GetNrAgents(), PlanningUnitMADPDiscrete::GetObservationHistoryArrays(), and PlanningUnitMADPDiscrete::IndividualToJointObservationIndices().

Referenced by Initialize().

double BayesianGameForDecPOMDPStage::GetImmediateReward ( Index  jtI,
Index  jaI,
const FactoredDecPOMDPDiscreteInterface fd = 0 
) const
inlinevirtual
const PlanningUnitDecPOMDPDiscrete* BayesianGameForDecPOMDPStage::GetPUDecPOMDPDiscrete ( ) const
inline
const QFunctionJAOHInterface* BayesianGameForDecPOMDPStage::GetQHeur ( ) const
inline
BayesianGameForDecPOMDPStage & BayesianGameForDecPOMDPStage::operator= ( const BayesianGameForDecPOMDPStage o)
void BayesianGameForDecPOMDPStage::Print ( ) const
inlinevirtual
void BayesianGameForDecPOMDPStage::ProbRewardForjoahI ( Index  ts,
Index  jtI,
Index jaI_arr,
Index joI_arr,
Index jaohI,
double &  PjaohI,
double &  ExpR_0_prevTS_thisJAOH 
)
protected

Calculates the jaohI corresponding to jaI_arr and joI_arr and also returnes the P(jaohI) and the expected obtained reward for previous time steps GIVEN this joint action history.

input args Index ts, Index jtI, Index* jaI_arr,Index* joI_arr, output args Index& jaohI, double& PjaohI, double& ExpR_0_prevTS_thisJAOH

basically this function is a form of PlanningUnitMADPDiscrete::GetJAOHProbs(Recursively) that also computes the reward.

basically this function is a form of PlanningUnitMADPDiscrete::GetJAOHProbs(Recursively) that also computes the reward.

References _m_pu, Globals::CastLIndexToIndex(), DEBUG_BG4DECPOMDP4, BeliefInterface::Get(), PlanningUnitDecPOMDPDiscrete::GetDPOMDPD(), TreeNode< Tcontained >::GetIndex(), BeliefInterface::GetIterator(), JointActionObservationHistoryTree::GetJointActionObservationHistory(), PlanningUnitMADPDiscrete::GetJointActionObservationHistoryTree(), PlanningUnitMADPDiscrete::GetNewJointBeliefFromISD(), PlanningUnitMADPDiscrete::GetNrStates(), BeliefIteratorGeneric::GetProbability(), PlanningUnitDecPOMDPDiscrete::GetReward(), BeliefIteratorGeneric::GetStateIndex(), JointActionObservationHistoryTree::GetSuccessor(), BeliefIteratorGeneric::Next(), JointActionObservationHistory::Print(), and JointBeliefInterface::Update().

string BayesianGameForDecPOMDPStage::SoftPrint ( void  ) const
virtual

Member Data Documentation

bool BayesianGameForDecPOMDPStage::_m_areCachedImmediateRewards
protected

are the immediate rewards cached?

Referenced by ComputeAllImmediateRewards(), and operator=().

std::vector< std::vector<double> > BayesianGameForDecPOMDPStage::_m_immR
protected

the cache for the immediate rewards: immR[jt][ja]

Referenced by ComputeAllImmediateRewards(), and operator=().

const QFunctionJAOHInterface* BayesianGameForDecPOMDPStage::_m_qHeuristic
private

A pointer to the heuristic used by this Bayesian game —nec.?

Referenced by GetQHeur(), Initialize(), and operator=().