The Story
Fanny Crosby wrote the words to a melody by William J. Kirkpatrick; the hymn was first published in Finest of the Wheat, No. 1 (1890). Its "cleft of the rock" image comes from Exodus 33, where God shelters Moses in a cleft of the rock.
Scripture
And it shall come to pass, while my glory passeth by, that I will put thee in a clift of the rock, and will cover thee with my hand while I pass by:Exodus 33:22 (KJV)