Hot cross buns are sweet. But without the sugar in the dough a sugar sub is needed and those are worse. I would go easy on the glaze but leave the dough recipe in tact. The recipe I use for brunch is below. Ali uses a sugar mix of white and brown in the dough and her preference is a heavy glaze but I make the glaze on the side since I prefer them buttered out of the oven without glaze and my family likes glaze.
AK does a lot of trial and error recipe tweaking which is why I trust her recipes and instruction.