It's enough to pause the game (press esc) to save progress. It's supposed to also save periodically but that hasn't been implemented yet.
Losing progress to bugs and crashes is a legitimate concern, but a checkpoint system wouldn't really work past the portal. The game is no longer linear, and the paths and decisions you make will alter its course. The first 3 levels serve as a sort of introduction and character building phase to the main game. Any random elements are rerandomised each time you start at the portal, and you might want to approach it in different ways, which would become impossible if the starting conditions can't be reset.
Btw, if you do experience bugs and crashes, please report the conditions in detail so we can try to fix the issue!