It looks like a NAN error. Extract from the Software FAQ :
I'm using 3 point cap tracking and sometimes tracking fails for some axes and 'NAN' appears in the data readout, FreeTrack may also crash.
The model's front points cannot be allowed to move behind the middle point, relative to the camera plane. This could happen for downward pitching and extreme yawing. To avoid this, make your model deeper, with the middle point further back than the distance between the two front points.
Also make sure that only the tracking points are being tracked at all times and that there is only one tracking cross per point.
Rémy