Some pictures might help understand what part of the rack is hitting what part of the exhaust.
Do the factory pannier racks mount to the passenger foot peg bracket? That is how the Givi rack attaches down low. My bike had tipped over and bent the peg bracket in making it difficult to fit that side of the rack because it was smashed against the stock exhaust heat shield. I bent the bracket back as much as I felt comfortable so as not to damage the subframe to which the bracket mounts.
I would first start with loosening all the bolts holding your rack on and see if you can shift it over any.
Second, you might try adding a couple washers behind each mounting point of the foot peg bracket as it helped me push that lower rack mount away from the exhaust. Be sure not to put too many washers to prevent the mounting bolts from having enough threads to bite into.
Last, you might try moving the lower rack mount from behind the foot peg bracket to in front if it isn't already that way.
I would be careful making any adjustments to the exhaust system, but loosening some clamps and gently pushing in while retightening those clamps might help you gain a tiny bit of clearance. You don't want to put any odd stress on exhaust connections and cause a leak, so I caution you on doing that.
That being said, the exhaust system on the 2012+ bikes is a bit disappointing in how they follow the form of the bike. I feel like they stick out more than the -2011 models, particularly in the forward part of the muffler.
2017 Honda Africa Twin DCT
2013 Honda CB1100
2015 Honda Grom