Multiplexity and temporal evolution are both known to be structural features that critically affect dynamical behaviors on complex networks. By generalizing the theory of Masuda et al.[1] to multiplex networks, we investigate how the combination of such features impacts on a diffusive dynamics. As in the case of single layer networks, diffusive dynamics is slower in a temporal multiplex network than in its temporal-aggregate version. However, because of a non-linear rescaling of the inter-layer diffusion constant appearing in the solution, we observe a richer phenomenology. In particular, for a range of values of the inter-layer diffusion constant, it exists a critical value of the ratio between the temporal scale of the topological variation and the temporal scale of intra-layer diffusion that separates two different regimes. In addiction, it also exists a critical point of that ratio after which the typical optimum displayed by multiplex networks for the stability of a synchronous state for a system of oscillators disappears.