Some games have the camera directly in the center of rotation, then it doesn't matter how you measure. But as soon as it is a little bit in front or behind you'll need to measure against object further away. The extreme case is obviously third person where the camera is far behind the center, and to measure the FOV you need to do it against object ideally kilometers away in-game.
The match with 4:3 is actually a coincidence yes