One of the biggest rationale for colony spam is the ability to get LOTS AND LOTS of otherwise difficult to get ideology points.
Consequently, if there's a way to get ideology points efficiently WITHOUT having to colony spam it would be a soft nerf on colony spamming.
As of now, there are only two ways to get lots of ideology points quickly:
Zealot (ideology upon conquest) or
Colony Spam.
There should be other methods. Building ideological buildings is HORRIBLY inefficient compared to those two methods.