OBJECTIVES: (1) To assess the methodological quality of radiomics studies investigating histological subtypes, therapy response, and survival in patients with renal cell carcinoma (RCC) and (2) to determine the risk of bias in these radiomics studies. METHODS: In this systematic review, literature published since 2000 on radiomics in RCC was included and assessed for methodological quality using the Radiomics Quality Score. The risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies tool and a meta-analysis of radiomics studies focusing on differentiating between angiomyolipoma without visible fat and RCC was performed. RESULTS: Fifty-seven studies investigating the use of radiomics in renal cancer were identified, including 4590 patients in total. The average Radiomics Quality Score was 3.41 (9.4% of total) with good inter-rater agreement (ICC 0.96, 95% CI 0.93-0.98). Three studies validated results with an independent dataset, one used a publically available validation dataset. None of the studies shared the code, images, or regions of interest. The meta-analysis showed moderate heterogeneity among the included studies and an odds ratio of 6.24 (95% CI 4.27-9.12; p < 0.001) for the differentiation of angiomyolipoma without visible fat from RCC. CONCLUSIONS: Radiomics algorithms show promise for answering clinical questions where subjective interpretation is challenging or not established. However, the generalizability of findings to prospective cohorts needs to be demonstrated in future trials for progression towards clinical translation. Improved sharing of methods including code and images could facilitate independent validation of radiomics signatures. KEY POINTS: • Studies achieved an average Radiomics Quality Score of 10.8%. Common reasons for low Radiomics Quality Scores were unvalidated results, retrospective study design, absence of open science, and insufficient control for multiple comparisons. • A previous training phase allowed reaching almost perfect inter-rater agreement in the application of the Radiomics Quality Score. • Meta-analysis of radiomics studies distinguishing angiomyolipoma without visible fat from renal cell carcinoma show moderate diagnostic odds ratios of 6.24 and moderate methodological diversity.