The risk of developing breast cancer is increased in women with family history of breast cancer and particularly in families with multiple cases of breast or ovarian cancer. Nevertheless, many women with a positive family history never develop the disease. Polygenic risk scores (PRSs) based on the risk effects of multiple common genetic variants have been proposed for individual risk assessment on a population level. We investigate the applicability of the PRS for risk prediction within breast cancer families. We studied the association between breast cancer risk and a PRS based on 75 common genetic variants in 52 Finnish breast cancer families including 427 genotyped women and pedigree information on ~4000 additional individuals by comparing the affected to healthy family members, as well as in a case-control dataset comprising 1272 healthy population controls and 1681 breast cancer cases with information on family history. Family structure was summarized using the BOADICEA risk prediction model. The PRS was associated with increased disease risk in women with family history of breast cancer as well as in women within the breast cancer families. The odds ratio (OR) for breast cancer within the family dataset was 1.55 [95 % CI 1.26-1.91] per unit increase in the PRS, similar to OR in unselected breast cancer cases of the case-control dataset (1.49 [1.38-1.62]). High PRS-values were informative for risk prediction in breast cancer families, whereas for the low PRS-categories the results were inconclusive. The PRS is informative in women with family history of breast cancer and should be incorporated within pedigree-based clinical risk assessment.