June 16 - Coupled Learning/ Interdependencies Models

Coupled Learning and Interdependencies - Phanish Puranam & Marlo Raveendran

Modeling Adaptive Rationality - Video and Reading

Video

Modeling Adaptive Rationality (Phanish Puranam, 10 minutes)A short lecture on how one might think about modeling agents that are bounded but "adaptively" rational.  

Additional Reading

Puranam, P., Stieglitz, N., Osman, M., and Pillutla M.M. (2015) Modelling bounded rationality in organizations: Progress and Prospects. Academy of Management Annals 9(1):337-92

Reinforcement Learning by Adaptive Agents - Videos and Readings

Learning is at the heart of adaptive rationality- in which agents learn about, rather than come pre-equipped with knowledge about their task environment. Reinforcement learning is a process through which agents learn about their task environment, by updating their beliefs (representations) based on prediction error (i.e. feedback). It is a very useful framework for modeling adaptive rationality, and has deep foundations in the psychology of learning.

Somewhat confusingly, there is also something called a “reinforcement learning task” in which the feedback to the learning agent is conditioned on their own actions (e.g. learning by doing). It is distinct from vicarious learning (learning from the actions and outcomes of others) or regret based learning (where the rewards for foregone action are also revealed to the learner). One can have a non-RL learning process in a RL task environment, and vice versa.

In our modeling of adaptive rationality, we will usually consider RL processes occurring in RL task environments. Specifically, we will use RL in a multi-armed bandit task (an RL task with a single state, in which the agent must learn from feedback to select the best paying action among many) as a basic building block.

Reading

Park,S. and Puranam P. Self-confirming biased beliefs in organizational learning by doing, Complexity, 2021 https://www.hindawi.com/journals/complexity/2021/8865872/

Sutton and Barto (2018) Reinforcement Learning. This book is written for computer science applications. However, you will see that the RL model we develop is a form of “Temporal Differencing for a non-associative RL task” in their schema- which turns out to be the basis for some of the most successful Artificial Intelligence applications in RL tasks.

Video 1

Video 2