This may be hotfixed before anyone has to deal with it, but Ignis is currently bugged. Killing a Brittle Golem will NOT reduce his buff stack. However, if you destroy the golem as a regular construct (the way it spawns), it will decrease the buff. This is not how it was intended, but it does work for now. Otherwise, the encounter is unbeatable. Good luck!