Over the three months ending on Nov 3 I have travelled to the US three times, totalling a stay of 41 days. The dates were 6th of August to the 23rd, the 8th of Sept to the 16th, and the 11th of Oct to the 21st. I am now thinking of going over to spend Thanksgiving there, I would stay for 4 days, but I am concerned about whether I have been going too often. I have travelled on the ESTA all the above-mentioned times.
Obligatory note: The program governing whether you can enter the US without a visa is the Visa Waiver Program (VWP). ESTA is just a form you need to fill in to fly to the US and use the VWP.
There are no hard and fast rules regarding how many times you can visit the US on the VWP. However if the border officer believes that you are attempting to live in the US through repeated VWP visits he or she can refuse you entry at any time (something which will have serious consequences). Admission is at the discretion of the border officer.
You have spent less than half your time in the US over the last three months, and even after your Thanksgiving visit your total time in the US will be less than 90 days, which is the length of a single visit permitted under the VWP. To my mind your visits would qualify for the 'short and occasional' visits that are permitted under the VWP, and would expect that you should be fine. Of course if you have had a long history of spending a lot of time in the US before this the situation may be different.
The above assumes that when you left the US between visits you went on each occasion to a country other than Canada, Mexico and the Caribbean; or that you reside in Canada, Mexico or the Caribbean and went home to your country of residence on those occasions. If you didn't do that then you may be in trouble.
TLDR: You are fine unless you are doing something unusual