The mechanical properties of a sensor strongly affect its tactile sensing capabilities. By exploiting tactile filters, mechanical structures between the sensing unit and the environment, it is possible to tune the interaction dynamics with the surrounding environment. But how can we design a good tactile filter? Previously, the role of filters' geometry and stiffness on the quality of the tactile data has been the subject of several studies, both implementing static filters and adaptable filters. State-of-the-art works on online adaptive stiffness highlight a crucial role of the filters' mechanical behavior in the structure of the recorded tactile data. However, the relationship between the filter's and the environment's characteristics is still largely unknown. We want to show the effect of the environment's mechanical properties on the structure of the acquired tactile data and the performance of a classification task while testing a wide range of static tactile filters. Moreover, we fabricated the filters using four materials commonly exploited in soft robotics, to merge the gap between tactile sensing and robotic applications. We collected data from the interaction with a standard set of twelve objects of different materials, shapes, and textures, and we analyzed the effect of the filter's material on the structure of such data and the performance of nine common machine learning classifiers, both considering the overall test set and the three individual subsets made by all objects of the same material. We showed that depending on the material of the test objects, there is a drastic change in the performance of the four tested filters, and that the filter that matches the mechanical properties of the environment always outperforms the others.