Abstract: Pedestrian models typically represent interactions between agents in a symmetric fashion. In general, these symmetric relationships are valid for a large number of crowd simulation scenarios. However, there are many cases in which symmetric responses between agents are inappropriate, leading to unrealistic behavior or undesirable simulation artifacts. We present a novel formulation, called right of way, which provides a well-disciplined mechanism for modeling asymmetric relationships between pedestrians. Right of way is a general principle, which can be applied to different types of pedestrian models. We illustrate this by applying right of way to three different pedestrian models (two based on social forces and one based on velocity obstacles) and show its impact in multiple scenarios. Particularly, we show how it enables simulation of the complex relationships exhibited by pilgrims performing the Islamic religious ritual, the Tawaf.