It means that a person has little or no chance of success in what he/she is about do. It is an abbreviation of the phrase ‘no more chance than a cat in hell without claws’.

The two key parts of this earlier version of the phrase are ‘hell’ and ‘without claws’.
Hell signifies a very difficult situation and without claws is a reference to being without the means to deal with that situation. Under these circumstances the person cannot succeed.
It was originally a reference to being in a fight without adequate weapons.
That is about all we know about the phrase ‘cat-in-hell’s chance’.
Note that there is an apostrophe in ‘hell’s’.