Multiplexity and temporal evolution are both known to be structural features that critically affect dynamical behaviors on complex networks. By generalizing the theory of Masuda et al. to multiplex networks, we investigate how the combination of such features impacts on a diffusive dynamics. As in the case of single layer networks, diffusive dynamics is slower in a temporal multiplex network than in its temporal-aggregate version. However, because of a non-linear rescaling of the inter-layer diffusion constant appearing in the solution, we observe a richer phenomenology. In particular, for a range of values of the inter-layer diffusion constant, it exists a critical value of the ratio between the temporal scale of the topological variation and the temporal scale of intra-layer diffusion that separates two different regimes. In addiction, it also exists a critical point of that ratio after which the typical optimum displayed by multiplex networks for the stability of a synchronous state for a system of oscillators disappears.