I accessed the rear reservoir without taking the rack and side panel off, but it was a little awkward and really limited working space. I needed to use a funnel to add fluid and had to be really careful putting the reservoir cover back on. I didn’t break/bend/crease any of the panels, but I guess that would be a risk.
I recently flushed my rear brakes and took the rear rack off, etc. Made access pretty easy.
I think you "could" loosen the panel and provide some access, but it's a lot easier just to take them off IMO.