In optimization under uncertainty for aerospace design, statistical moments of the quantity of interest are often treated as separate objectives and are traded off in a multi-objective optimization formulation. However, in many design problems the trade-off between statistical moments can be large and the Pareto front representing this trade-off can include designs with undesirable behavior, such as being robust but being guaranteed to give a worse performance than another design. When a simulation of a system is computationally expensive, obtaining the full Pareto front is infeasible and so spending optimization time obtaining such undesirable designs wastes time that could be spent obtaining more desirable alternatives. As a remedy, we propose an optimization formulation that can use multiple dominance criteria to avoid generating potentially inferior designs. We consider various orders of stochastic dominance as criteria to use alongside statistical moment based Pareto dominance, and illustrate how this gives rise to improved designs using a limited computational budget in an acoustic horn design problem and a transonic airfoil design problem.