My thought is, that if
Hell existed, it would be a psychological condition, not a literal place where souls are 'physically' tortured. It would be a state of being in which one were immersed in a continuous state of utter terror, panic, helplessness and despair, as if one were in the process of being killed -- except, of course, it never ends.
Being made to feel as if one were tightly confined and suffocating -- a common scenario in nightmares -- as if one were buried alive in a coffin, or welded into a steel drum, as I read once happened to a mafioso -- would certainly constitute one such mental scenario.
Feeling endlessly pursued by something that inspires fear is another. Knowing that the thing that horrifies and repels you the most -- be it a zombie, space monster, or some nameless amorphous mass of wet putrefaction -- is just moments away from catching up with you, and that you must stay on the run without ever being able to rest, forever -- that would certainly qualify as hellish.
The horror wouldn't have to be embodied in an actual monster; it could be something more abstract, like the fear of having one's secret shame exposed, or of impending bankruptcy, depending on one's disposition.
It has occurred to me, though, that in some cases a worse fate would be the realization that one has become not a victim, but a victimizer, oneself; that instead of being chased by a nightmare monster, one has become the monster that haunts someone else's dreams.
Especially if that someone were the object of one's most ardent longing.
To know that the more anxiously you pursued the one you love, the more he/she was frightened and disgusted by you and the more they would reject you and try to escape -- or conversely, that they might find your impertinent desire annoying and pathetic, and might be moved to go through the motions of fulfilling it, only to reveal it was all a charade. That would truly be a hell of despair and degradation.