I drilled off the heads of the bolts after the nut started turning. Did not want to mess anything up and bolts can be replaced.
Mounted the square nuts in a vise and used a big crescant wrench to turn them off the remainder of the bolts.
I epoxied the nuts back into place and when I say epoxied, I mean completely filled in the space the nut sits in. After the epoxy set up, I drilled into it and even ran a suitable tap in, although I don't know if I left any threads.
I had two bolts of the right size in my "extras" can and now I can mount or unmount the fender easily, without the nut turning inside.