First, the power that draws down the powers of heaven is faith. People can possess the priesthood but still have no power to draw down these powers if they lack faith. The Priesthood without this kind of faith is powerless. On the other hand, by faith the worlds were created, and miracles happen by this power. When the priesthood is the government of God, and the worlds were created by faith it proves that one can not exist without the other. Lack of faith was actually the reason because Moses and the higher priesthood were taken away from the Israelites D&C 84:23-25
You can loose the priesthood by the lack of faith, you can, on the other hand, draw these powers of heaven down by exercising faith.
Secondly, it’s interesting to ponder about how the priesthood gets conferred. D&C 84:42
And wo unto all those who come not unto this priesthood which ye have received, which I now confirm upon you who are present this day, by mine own voice out of the heavens; and even I have given the heavenly hosts and mine angels charge concerning you.
The same pattern can be found in JST Gen.14:26-29
Now Melchizedek was a man OF FAITH, who wrought righteousness; and when a child he feared God, and stopped the mouths of lions, and quenched the violence of fire.
And thus, HAVING BEEN APPROVED of GOD, he was ordained an high priest after the order of the covenant which God made with Enoch,
It being after the order of the Son of God; which order came, not by man, nor the will of man; neither by father nor mother; neither by beginning of days nor end of years; but of God;
And it was delivered unto men by the calling of his own voice, according to his own will, unto as many as believed on his name.
Isn’t it interesting what Melchizedek did by the power of faith, and that he received the Priesthood after he proved himself that way?
All that matters is our faith in Christ. Only our faith sets a limit to what we can receive.