Different heating system controllers for passive solar buildings are compared on two different buildings. The performance criterion combines energy performance and thermal comfort using the "cost function" paradigm. The experimental facilities did not allow a direct experimental comparison by using two identical buildings. The controllers were implemented alternatively in one building and a performance comparison was obtained in two ways: first by identifying short periods that have similar driving variables (weather conditions and building occupancy) and comparing the experimental results obtained in both cases. The second method mixes experiments and simulation using a well-tuned model of the building and its occupants. This paper discusses the results obtained using the above methods and shows that both methods give consistent estimates of the difference between controllers, while the second method allows to extrapolate useful information from the limited data available.