How often do you change your fuel filters?

I looking for a kind of rule of thumb here on changing the fuel filters.

Like when do you inspect the strainer? When do you change the primary, and then the secondary...

I did a search but didn't find anything other than just change them if you see a loss in fuel efficiency....Is that it?