I rather doubt the granite for the stone altar-thingy is 4000 years old. I suspect it's a bit older.

But the evidence, such as it is, is tricky to interpret. The area has been inhabited for a long time. Nothing prevents people from using structures they find and adding to them. You're a colonist. In the woods near your cabin you find a stone structure that's partially collapsed. If you rebuild it you have a nice, secure cellar/storage area/small barn. What would you do? Utilize it? Leave it alone so someone 400 years in the future would have a pristine archeological site?