In this talk, I will review classical backward error analysis for symplectic Runge--Kutta methods for Hamiltonian ODEs and explain the difficulties when applying similar ideas in the context of PDEs. I will then explain two stragegies for making backward error analysis work on infinite dimensional Hilbert spaces as well. The first approach is based on exploiting the regularity of the original PDE system and yields, under sufficiently strong assumptions, results which are almost as strong as those available for ODEs. The second approach involves a new construction of modified equations within the framework of variational integrators. This approach is still work-in-progress, but initial numerical tests support the validity and point to possible analytic advantages of this approach. (Joint work with C. Wulff and S. Vasylkevych.)