1. Is that Tom dancing with all the girls? He used to be so shy and look at him now! He certainly has come out of his shell.
2. Patty has been sitting on the couch by herself since she arrived. Why don’t you go over and start a conversation with her? See if you can get her to come out of her shell.

The expression suggests that a person who is shy or quiet is like a turtle that retreated into its shell.