We continue the investigation, that began in \cite{bianchi} and \cite{glasby}, into finite groups whose set of nontrivial conjugacy class sizes form an arithmetic progression. Let $G$ be a finite group and denote the set of conjugacy class sizes of $G$ by $cs(G)$. Finite groups satisfying $cs(G)=\{1,2,4,6\}$ and $\{1,2,4,6,8\}$ are classified in \cite{glasby} and \cite{bianchi}, respectively, we demonstrate these examples are rather special by proving the following. There exists a finite group $G$ such that $cs(G)=\{1,2\alpha ,2\alpha +1,2\alpha 3\}$ if and only if $\alpha =1$. Furthermore, there exists a finite group $G$ such that $cs(G)=\{1,2\alpha ,2\alpha +1,2\alpha 3,2\alpha +2\}$ and $\alpha$ is odd if and only if $\alpha =1$.\[1ex]