Thermal anemometers with heated velocity sensors are mostly used for low-velocity measurements in rooms. The heated velocity sensor generates an upward, free convection flow that interacts with the airflow where measurements are to be performed and, thus, has an impact on the accuracy of the velocity measurements. Tests were performed with four anemometers available on the market to identify this impact in an airflow with a constant velocity and in an airflow with a periodically fluctuating velocity. The free convection flow had a significant impact on the accuracy of the velocity measurements, especially at flow velocities below 0.15 m/s and for sensors of a large size and with high overheating temperature. The mechanism of the interaction between the free convection flow and the downward flow was revealed. The impact of free convection has to be considered carefully when measurements are performed to assess room air movement or to validate CFD calculations. The results show that it is possible to model and predict the impact of free convection on the accuracy of the velocity measurements under non-steady-state conditions by simple tests performed under steady-state conditions.