Working memory deficits are present in many neuropsychiatric diseases with diagnosis-related severity. However, it is unknown whether this common behavioral abnormality is a continuum explained by a neural mechanism shared across diseases or a set of discrete dysfunctions. Here, we performed predictive modeling to examine working memory ability (WMA) as a function of normative whole-brain connectivity across psychiatric diseases. We built a quantitative model for letter three-back task performance in healthy participants, using resting state functional magnetic resonance imaging (rs-fMRI). This normative model was applied to independent participants (N = 965) including four psychiatric diagnoses. Individual's predicted WMA significantly correlated with a measured WMA in both healthy population and schizophrenia. Our predicted effect size estimates on WMA impairment were comparable to previous meta-analysis results. These results suggest a general association between brain connectivity and working memory ability applicable commonly to health and psychiatric diseases.