To Explore or To Exploit

Decoding Human Decision Making with R and Python

Erin Bugbee

Shopping for a Flight to Seattle

Optimal Stopping Problem

• Mathematically optimal solution for calculating the optimal stopping point
• Question is when to stop searching

Explore-Exploit Tradeoff

• Finding the optimal balance of:

  • Gathering information through search – exploration

  • Leveraging known information – exploitation

Humans Do Not Decide Optimally

Psychologist Daniel Kahneman

People use heuristics, or mental shortcuts

Can lead to biases, which are systematic deviations from what is economically rational

Cognitive Decision Science

• Cognitive process

• Systematic deviations from rationality

• according to economic theory of rationality

• Study of:
  • how people should make decisions
  • how they actually make decisions
  • how this process can be modeled to both predict and explain behavior
  • how they can make better decisions

About Me

• PhD Candidate at Carnegie Mellon University in Cognitive Decision Science

• ScB in Statistics and Behavioral Decision Sciences at Brown University

• Interned at Apple, Amazon, Disney, Microsoft

• On the job market for 2025

Decoding Human Decision Making

1. Running human experiments
2. Simulating behavior using Python
3. Analyzing data in R

Running human experiments

Simulating behavior

Analyzing data

Running Human Experiments

Design Experiment

• Decision making task:

  • Optimal Stopping Task

• Factors to manipulate:

  • Feedback type

  • Knowledge of distribution of options

Develop Experiment

• Build task for human participants to complete

• Integrate into Qualtrics survey

• Tools:

  • Javascript, HTML, CSS, Qualtrics

Survey

Task

Let’s Decide When to Stop

3, 22

Optimal Stopping Task

Collect Data

Simulating Behavior Using Python

Build Decision Making Environment

• Create task that replicates real-world decision environment
• Represent task through Python code

Task

Code

Agents

Optimal Agents

• Decide according to the optimal process

• Calculate according to mathematical theory

• Serve as baseline

Cognitive Agents

• Decide according to a cognitive algorithm – Instance-Based Learning (IBL) Model

• Use PyIBL Python library

• Serve as prediction of human behavior

PyIBL

Define Agent

import pyibl 
a = pyibl.Agent(["action", "value", "dist_from_end"])
a.similarity(["value", "dist_from_end"], pyibl.positive_linear_similarity)

Give Prior Experience

import pyibl 
a = pyibl.Agent(["action", "value", "dist_from_end"])
a.similarity(["value", "dist_from_end"], pyibl.positive_linear_similarity)

a.populate([{"action": "Select", "value": 50, "dist_from_end": 9}],1)
a.populate([{"action": "Pass", "value": 50, "dist_from_end": 9}],1)

Make Choice

import pyibl 
a = pyibl.Agent(["action", "value", "dist_from_end"])
a.similarity(["value", "dist_from_end"], pyibl.positive_linear_similarity)

a.populate([{"action": "Select", "value": 50, "dist_from_end": 9}],1)
a.populate([{"action": "Pass", "value": 50, "dist_from_end": 9}],1)

value = 50
select_action =  {"action": "Select", "value": value, "dist_from_end": 9}
pass_action =  {"action": "Pass", "value": value, "dist_from_end": 9}
choice = a.choose([select_action, pass_action])
choice["action"]

Receive Feedback

import pyibl 
a = pyibl.Agent(["action", "value", "dist_from_end"])
a.similarity(["value", "dist_from_end"], pyibl.positive_linear_similarity)

a.populate([{"action": "Select", "value": 50, "dist_from_end": 9}],1)
a.populate([{"action": "Pass", "value": 50, "dist_from_end": 9}],1)

value = 50
select_action =  {"action": "Select", "value": value, "dist_from_end": 9}
pass_action =  {"action": "Pass", "value": value, "dist_from_end": 9}
choice = a.choose([select_action, pass_action])
choice["action"]

# Feedback depends on condition
a.respond(feedback)

Why Python?

• PyIBL and other modeling libraries

• General-purpose and flexible

Analyzing Behavior Using

Analyze Data and Report

Qualtrics –> CSV

Javascript Experiment –> CSV

Join and analyze with R in reproducible Quarto document

Compare to optimal and IBL agents

Figures using ggplot2

Why R?

• Libraries for exploring and visualizing data
  • tidyverse
  • ggplot2
• Excels in statistical analysis

Results

People stop searching too early

IBL Model accurately predicts human behavior

People learn from feedback

Having knowledge of the distribution hinders necessary exploration

Conclusion

Tools

• Decoding the human mind is difficult and requires many tools

Exploration in Choosing a Tool

• Use what can help you accomplish the task
• Best when used together
• Don’t be afraid to explore and learn a new tool

When you have a , everything looks like a nail.

Image created by DALL-E

Exploration at posit::conf

• If this is your first posit::conf…

  • There are lots of new people, technologies, topics to learn

• If you have attended many times before…

  • You likely have made connections, know which sessions are of interest

Keep Exploring

• Be careful of over-exploiting your knowledge

• Meet new people, learn new things, see a new place

Thank you!

github.com/erinbugbee/2024positconf-decoding-decisions

erinbugbee.quarto.pub/2024positconf-decoding-decisions

erin_bugbee@icloud.com

erinbugbee.com