MultiAgentDecisionProcess
FactoredQLastTimeStepOrElse Class Reference

FactoredQLastTimeStepOrElse is a class that represents a Q-function that is factored at the last stage, and non factored for earlier stages. More...

#include <FactoredQLastTimeStepOrElse.h>

Inheritance diagram for FactoredQLastTimeStepOrElse:
[legend]

Public Member Functions

void Compute ()
 Compute the heuristic. More...
 
void DeInitialize ()
 
 FactoredQLastTimeStepOrElse (const PlanningUnitFactoredDecPOMDPDiscrete *puf)
 (default) Constructor More...
 
 FactoredQLastTimeStepOrElse (const boost::shared_ptr< const PlanningUnitFactoredDecPOMDPDiscrete > &puf)
 
const ScopeGetAgentScopeForLQF (Index e, Index stage) const
 Return the AgentScope for the lqf-th LQF at time-step stage. More...
 
double GetLocalQValue (Index LRF, Index stage, const std::vector< Index > &sfacValues, const std::vector< Index > &aoHistIs, Index agSc_jaI) const
 Get local Q-value using individual and group indices. More...
 
double GetLocalQValue (Index LRF, Index stage, Index sfSC_sI, Index agSC_aoHistI, Index agSc_jaI) const
 Get the Q for LRF, for local state, local aoh and local joint action only then with only 'joint' (i.e., group) indices. More...
 
double GetLocalQValue (Index LRF, Index aoHistI, Index jaI) const
 
size_t GetNrLQFs (Index stage) const
 Return the number of local Q value functions for stage. More...
 
const ScopeGetStateFactorScopeForLQF (Index lqf, Index stage) const
 Return the State factor Scope for the lqf-th LQF at time-step stage. More...
 
void Initialize ()
 
virtual ~FactoredQLastTimeStepOrElse ()
 Destructor. More...
 
- Public Member Functions inherited from FactoredQFunctionStateJAOHInterface
virtual double GetLocalQValue (Index e, Index stage, const std::vector< Index > &sfacValues_e, const std::vector< Index > &aoHistIs_e, const std::vector< Index > &acs_e) const
 Get local Q-value using individual indices. More...
 
virtual double GetLocalQValue (Index e, Index stage, Index sfSC_sI, const std::vector< Index > &aoHistIs_e, Index jaI_e) const
 Get local Q-value using individual and group indices. More...
 
virtual ~FactoredQFunctionStateJAOHInterface ()
 Destructor. More...
 
- Public Member Functions inherited from QFunctionJAOHInterface
virtual double GetQ (Index jaohI, Index jaI) const =0
 Returns Q(joint A-O history jaohI, jaI). More...
 
 QFunctionJAOHInterface ()
 
virtual ~QFunctionJAOHInterface ()
 Destructor. More...
 
- Public Member Functions inherited from QFunctionForDecPOMDPInterface
 QFunctionForDecPOMDPInterface ()
 (default) Constructor More...
 
virtual ~QFunctionForDecPOMDPInterface ()
 Destructor. More...
 
- Public Member Functions inherited from QFunctionInterface
virtual void Load ()
 Load the Qvalues from disk. More...
 
virtual void Save () const
 Stores the Qvalues to disk. More...
 
virtual std::string SoftPrintBrief () const =0
 Returns a short description of the heuristic, can be used for constructing filenames. More...
 
virtual ~QFunctionInterface ()
 
- Public Member Functions inherited from QFunctionForFactoredDecPOMDP
virtual void ComputeWithCachedQValues (bool computeIfNotCached)
 Compute Qvalue function, while caching the Qvalues to disk. More...
 
virtual std::string GetCacheFilename () const
 Gets the filename where the Qvalues should be stored. More...
 
const
PlanningUnitFactoredDecPOMDPDiscrete
GetPU () const
 
const
PlanningUnitFactoredDecPOMDPDiscrete
GetPUF () const
 
virtual void Load (const std::string &filename)
 Load the Qvalues from disk from a file named filename. More...
 
 QFunctionForFactoredDecPOMDP (const PlanningUnitFactoredDecPOMDPDiscrete *puf)
 (default) Constructor More...
 
 QFunctionForFactoredDecPOMDP (const boost::shared_ptr< const PlanningUnitFactoredDecPOMDPDiscrete > &puf)
 
virtual void Save (const std::string &filename) const
 Stores the Qvalues to disk in file named filename. More...
 
void SetPU (const PlanningUnitDecPOMDPDiscrete *pu)
 
void SetPU (const boost::shared_ptr< const PlanningUnitDecPOMDPDiscrete > &pu)
 
void SetPUF (const PlanningUnitFactoredDecPOMDPDiscrete *pu)
 Changes the Planning unit pointed to. More...
 
void SetPUF (const boost::shared_ptr< const PlanningUnitFactoredDecPOMDPDiscrete > &pu)
 
- Public Member Functions inherited from QFunctionForFactoredDecPOMDPInterface
virtual ~QFunctionForFactoredDecPOMDPInterface ()
 (default) Constructor More...
 

Private Attributes

std::vector< Scope_m_agentScopes
 
bool _m_initialized
 
size_t _m_nrLRFs
 The number of local reward functions. More...
 

Detailed Description

FactoredQLastTimeStepOrElse is a class that represents a Q-function that is factored at the last stage, and non factored for earlier stages.

This is the type of Q-function used by #refGMAAELSI.

Constructor & Destructor Documentation

FactoredQLastTimeStepOrElse::FactoredQLastTimeStepOrElse ( const PlanningUnitFactoredDecPOMDPDiscrete puf)

(default) Constructor

References _m_initialized.

FactoredQLastTimeStepOrElse::FactoredQLastTimeStepOrElse ( const boost::shared_ptr< const PlanningUnitFactoredDecPOMDPDiscrete > &  puf)

References _m_initialized.

FactoredQLastTimeStepOrElse::~FactoredQLastTimeStepOrElse ( )
virtual

Destructor.

Member Function Documentation

void FactoredQLastTimeStepOrElse::Compute ( )
virtual
const Scope & FactoredQLastTimeStepOrElse::GetAgentScopeForLQF ( Index  lqf,
Index  stage 
) const
virtual

Return the AgentScope for the lqf-th LQF at time-step stage.

Implements FactoredQFunctionStateJAOHInterface.

References _m_agentScopes, PlanningUnit::GetHorizon(), and QFunctionForFactoredDecPOMDP::GetPUF().

double FactoredQLastTimeStepOrElse::GetLocalQValue ( Index  e,
Index  stage,
const std::vector< Index > &  sfacValues_e,
const std::vector< Index > &  aoHistIs_e,
Index  jaI_e 
) const
virtual

Get local Q-value using individual and group indices.

Get the local Q-value for

  • the e-th local Q-value function
  • for stage (time-step) stage
  • the vector of state factor value indices sfacValues, for the state factors in the scope of the e-th local Q-function (i.e. the state factors corresponding to GetStateFactorScopeForLQF(e, stage) )
  • the vector of individual action-observation history indices for the agents in the agent scope of the e-th local Q-function
  • group action jaI_e.

jaI_e is the group index of the agents in GetAgentScopeForLQF( lqf, stage). Is is easily computed by

IndexTools::IndividualToJointIndices(indices, nrActions)

where the former argument is are the individual action indices of the agents in the scope, and the latter are the number of individual actions they have.

Implements FactoredQFunctionStateJAOHInterface.

References PlanningUnitFactoredDecPOMDPDiscrete::GetFDPOMDPD(), FactoredDecPOMDPDiscreteInterface::GetLRFReward(), QFunctionForFactoredDecPOMDP::GetPUF(), and FactoredDecPOMDPDiscreteInterface::RestrictedStateVectorToJointIndex().

double FactoredQLastTimeStepOrElse::GetLocalQValue ( Index  LRF,
Index  stage,
Index  sfSC_sI,
Index  agSC_aoHistI,
Index  agSc_jaI 
) const
inlinevirtual

Get the Q for LRF, for local state, local aoh and local joint action only then with only 'joint' (i.e., group) indices.

Implements FactoredQFunctionStateJAOHInterface.

double FactoredQLastTimeStepOrElse::GetLocalQValue ( Index  LRF,
Index  aoHistI,
Index  jaI 
) const
inline
size_t FactoredQLastTimeStepOrElse::GetNrLQFs ( Index  stage) const
virtual

Return the number of local Q value functions for stage.

Implements FactoredQFunctionStateJAOHInterface.

References _m_nrLRFs, PlanningUnit::GetHorizon(), and QFunctionForFactoredDecPOMDP::GetPUF().

const Scope & FactoredQLastTimeStepOrElse::GetStateFactorScopeForLQF ( Index  lqf,
Index  stage 
) const
virtual

Member Data Documentation

std::vector<Scope> FactoredQLastTimeStepOrElse::_m_agentScopes
private

Referenced by GetAgentScopeForLQF(), and Initialize().

bool FactoredQLastTimeStepOrElse::_m_initialized
private
size_t FactoredQLastTimeStepOrElse::_m_nrLRFs
private

The number of local reward functions.

(this is equal the the nr. of local Q-functions, as we only represent a factored Q-function for the last stage.)

Referenced by GetNrLQFs(), and Initialize().