Current approaches for fitting stationary (dynamic) factor models to multivariate time series are based on principal components analysis of the covariance (spectral) matrix. These approaches are based on the assumption that the underlying process is temporally stationary which appears to be restrictive because, over long time periods, the parameters are highly unlikely to remain constant. Our alternative approach is to model the time-varying covariances (auto-covariances) via nonparametric estimation, which imposes very little structure on the moments of the underlying process. Because of identification issues, only parts of the model parameters are allowed to be time-varying. More precisely, we consider two specifications: First, the latent factors are stationary while the loadings are time-varying. Second, the latent factors admit a dynamic representation with time-varying autoregressive coefficients while the loadings are constant over time. Estimation of the model parameters is accomplished by application of evolutionary principal components and local polynomials. We illustrate our approach through applications to multichannel EEG data.