Are you going to use the infused oil in a body powder? It might be easier to add the powdered calendula to your other dry ingredients. Infusing the powdered botanical would be more difficult as it tends to sink to the bottom of the oil - unlike the dry herbs which tend to float in the oil.
I made 100% shea soap some years ago and added ground loofa to help with the lather. I must admit that I loved the soap as it was very creamy, non stripping and highly moisturising. Very expensive to make though.