Social affordances, although key in human-robot interaction processes, have received little attention in robotics. Hence, it remains unclear whether the prevailing mechanisms to exploit and learn affordances in the absence of human interaction can be extended to affordances in social contexts. This study provides a review of the concept of affordance in psychology and robotics and proposes a new view on social affordances in robotics and their differences from physical affordances. We moreover show how the model-based reinforcement learning theory provides a useful framework to study and compare social and physical affordances. To further study their differences, we present a new benchmark task mixing navigation and social interaction, in which a robot has to make a human follow and reach different goal positions in a row. This new task is solved in simulation using a modular architecture and reinforcement learning.