ladder: (noun) an often portable device that assists people to physically climb up and down; made of two straight or slightly angled sides with steps or rungs between them
latter: (adjective, noun) the second of two choices
Some people accept the use of “latter” with more than two choices, but purists don’t. The terms “last” or “last-mentioned” are preferred.
If someone asks if you want to take the stairs or the elevator, saying you’ll take the ladder will probably confuse them.
The term “ladder” can also be used metaphorically–to climb the corporate ladder or the ladder to success, for example. Whereas a physical ladder allows one to climb up and down, most people don’t want to go down a metaphorical ladder.