This note discusses a class of models for panel data that accommodate between‐group heterogeneity that is allowed to exhibit positive within‐group variance. Such a set‐up generalizes the traditional fixed‐effect paradigm in which between‐group heterogeneity is limited to univariate factors that act like constants within groups. Notable members of the class of models considered are non‐linear regression models with additive heterogeneity and multiplicative‐error models suitable for non‐negative limited dependent variables. The heterogeneity is modelled as a non‐parametric nuisance function of covariates whose functional form is fixed within groups but is allowed to vary freely across groups. A simple approach to perform inference in such situations is based on local first‐differencing of observations within a given group. This leads to moment conditions that, asymptotically, are free of nuisance functions. Conventional generalized method of moments procedures can then be readily applied. In particular, under suitable regularity conditions, such estimators are consistent and asymptotically normal, and asymptotically valid inference can be performed using a plug‐in estimator of the asymptotic variance.