Is it legal to collect shed antlers on a national wildlife refuge?

No, it is not, unless you have a special permit. Code of Federal Regulations Title 50 Part 27 Section 61 (50 CFR 27.61) prohibits the unauthorized removal of any public property, including natural objects from any national wildlife refuge national wildlife refuge
A national wildlife refuge is typically a contiguous area of land and water managed by the U.S. Fish and Wildlife Service  for the conservation and, where appropriate, restoration of fish, wildlife and plant resources and their habitats for the benefit of present and future generations of Americans.

Learn more about national wildlife refuge
. Because shed antlers are an important source of nutrients to various other animals, the Fish and Wildlife Service authorizes their collection only by special permit.