Fourier Transforms tells you where the repeated patterns are but doesn't tell you how to transform them to make the patterns intelligible.

In order to back trace rays that are not normal to the plane of observation you'll have to know the z ... the depth from the image surface, inward to the object of interest. If I'm right (!) then you have to know the ratio between the actual distance and the image distance, not unlike the classical lens computation, except it's probably an integral of some sort or other since you have to get the sum of rays over the domain of the image, for each point.