We have some first results on using influence-based abstractions in the context of deep reinforcement learning, which will be presented at the ALA workshop in Montreal.
Reinforcement learning is tough. POMDPs are hard. And doing RL in partially observable problems is a huge challenge. With Sammie and Chris Amato, I have been making some progress to get a principled method (based on Monte Carlo tree search) too scale for structured problems. We can learn both how to act, as well as the structure of the problem at the same time. See the paper and bib.
Can deep Q-networks etc. brute force their way through tough coordination problems…? Perhaps not. Jacopo’s work, accepted as an extended abstract at AAMAS’19, takes a first step in exploring this in the one-shot setting.
Not so surprising: “joint Q-learner” can be too large/slow and “individual Q-learners” can fail to find good representations.
But good to know: “factored Q-value functions” which represent the Q-function as a random mixture of components involving 2 or 3 agents, can do quite well, even for hard coordination tasks!
Check out the video:
A popular piece on my ERC research:
We are looking for faculty in AI!
Join the INFLUENCE research team!
I am looking for a 3-year Postdoc in Influence-based Abstraction, Learning and Coordination. Find me at #IJCAI2018 to informally discuss.
More info: vacancies
My invited IJCAI paper giving an overview of (some of, apologies to some coauthors, could not fit everything there….) my research is now available from my publications page.
I’m going to IJCAI: I have been invited to give a talk in the IJCAI-ECAI ’18 Early Career Spotlight track – I feel very honored.
See you in Stockholm!