Abstract: There is an ongoing desire to make virtual humans a more accessible tool for use in entertainment, training, and evaluations. From the graphical level to the animation level to the intelligence level, complexities abound. As research progresses some of these complexities become hidden from the end user. Ultimately, we would like to treat agents as real humans and instruct them as you might another person. Here we present a framework, inspired by natural language constructs, that aims to obfuscate the complexities and allow users to control virtual humans through structured English input. Our focus is on object and environment interactions, particularly spatial relationships.