Even regular light will take out lily pollen stains. Just don't rub it at all, or try normal methods of cleaning. Chenille or sticky tape will take off a lot of it, but then leave the garment in light. I find that if I have lily pollen on my apron, that by the end of the day it's gone.
We use the chenille pipe cleaners to clean lily pollen off the petals, as well, when we missed getting the anthers (or is it stamens?) before they mature. We've also noticed that lilies last longer if the pollen is removed. This goes for amaryllis, too.