jats:p In a paper published in Management Science in 2015, Stewart, Reimers, and Harris (SRH) demonstrated that shapes of utility and probability weighting functions could be manipulated by adjusting the distributions of outcomes and probabilities on offer as predicted by the theory of decision by sampling. So marked were these effects that, at face value, they profoundly challenge standard interpretations of preference theoretic models in which such functions are supposed to reflect stable properties of individual risk preferences. Motivated by this challenge, we report an extensive replication exercise based on a series of experiments conducted as a quasi-adversarial collaboration across different labs and involving researchers from both economics and psychology. We replicate the SRH effect across multiple experiments involving changes in many design features; importantly, however, we find that the effect is also present in designs modified so that decision by sampling predicts no effect. Although those results depend on model-based inferences, an alternative analysis using a model-free comparison approach finds no evidence of patterns akin to the SRH effect. On the basis of simulation exercises, we demonstrate that the SRH effect may be a consequence of misspecification biases arising in parameter recovery exercises that fit imperfectly specified choice models to experimental data. Overall, our analysis casts the SRH effect in an entirely new light. </jats:p>jats:p This paper was accepted by Yuval Rottenstreich, judgment and decision making </jats:p>