The sides of the cave are rough and have small ledges everywhere, they could probably be climbed. But there's no larger ledge in sight - their torches only reach so far, and the cave is, well, caved - the walls vanish behind the expanse of dormouse no matter how strong their torches are.