Cell Painting is a high-content image-based assay which can reveal rich cellular morphology and is applied in drug discovery to predict bioactivity, assess toxicity and understand diverse mechanisms of action of chemical and genetic perturbations. In this study, we investigate label-free Cell Painting by predicting the five fluorescent Cell Painting channels from paired brightfield z-stacks using deep learning models. We train and validate the models with a dataset representing 1000s of pan-assay interference compounds sampled from 17 unique batches. The model predictions are evaluated using a test set from two additional batches, treated with compounds comprised from a publicly available phenotypic set. In addition to pixel-level evaluation, we process the label-free Cell Painting images with a segmentation-based feature-extraction pipeline to understand whether the generated images are useful in downstream analysis. The mean Pearson correlation coefficient (PCC) of the images across all five channels is 0.84. Without actually incorporating these features into the model training we achieved a mean correlation of 0.45 from the features extracted from the images. Additionally we identified 30 features which correlated greater than 0.8 to the ground truth. Toxicity analysis on the label-free Cell Painting resulted a sensitivity of 62.5% and specificity of 99.3% on images from unseen batches. Additionally, we provide a breakdown of the feature profiles by channel and feature type to understand the potential and limitation of the approach in morphological profiling. Our findings demonstrate that label-free Cell Painting has potential above the improved visualization of cellular components, and it can be used for downstream analysis. The findings also suggest that label-free Cell Painting could allow for repurposing the imaging channels for other non-generic fluorescent stains of more targeted biological interest, thus increasing the information content of the assay.