Abstract: We present a new Material Point Method (MPM) for simulating viscoelastic fluids, foams and sponges. We design our discretization from the upper convected derivative terms in the evolution of the left Cauchy-Green elastic strain tensor. We combine this with an Oldroyd-B model for plastic flow in a complex viscoelastic fluid. While the Oldroyd-B model is traditionally used for viscoelastic fluids, we show that its interpretation as a plastic flow naturally allows us to simulate a wide range of complex material behaviors. In order to do this, we provide a modification to the traditional Oldroyd-B model that guarantees volume preserving plastic flows. Our plasticity model is remarkably simple (foregoing the need for the singular value decomposition (SVD) of stresses or strains). Lastly, we show that implicit time stepping can be achieved in a manner similar to [Stomakhin et al. 2013] and that this allows for high resolution simulations at practical simulation times.