My thought is, that if
Hell existed, it would be a psychological condition, not a literal place where souls are 'physically' tortured. It would be a state of being in which one were immersed in a continuous state of utter terror, panic, helplessness and despair, as if one were in the process of being killed -- except, of course, it never ends.
Being made to feel as if one were tightly confined and suffocating -- a common scenario in nightmares -- as if one were buried alive in a coffin, or welded into a steel drum, as I heard once happened to a mafioso -- would certainly constitute one such mental scenario.
Feeling endlessly pursued by something that inspires fear is another. Knowing that the thing that horrifies and repels you the most -- like a demon or zombie, maybe -- is just moments away from catching up with you, and that you must stay on the run without ever being able to rest, forever -- that would certainly qualify as hellish.
The horror wouldn't have to be embodied in an actual monster; it could be something more abstract, like the fear of having all one's secret shame exposed -- or even an impending sense of financial ruin, depending on one's disposition.
It has occurred to me, though, that in some cases a worse fate would be the realization that one has become not a victim, but a victimizer, oneself; that instead of being chased by a nightmare monster, one has become the monster that haunts someone else's dreams.
Especially if that someone were the object of one's most ardent longing.
To know that the more anxiously you pursued the one you love, the more he/she was frightened and disgusted by you and the more they would reject you and try to escape -- or conversely, that they might find your impertinent desire annoying and pathetic, and might be moved to go through the motions of fulfilling it, just to humor you and make you stop bothering them. That would truly be a hell of despair and degradation.