To answer the question, you have to break it down into categories which I’ve done. There’s a difference between the behaviour of the wildcat ancestor which is the same species as the domestic cat and from which we can learn about domestic cat behaviour, unsterilised male cats, sterilised male cats and kittens parented by the male cat in question and kittens that are not parented by the male cat in question. It’s quite complicated.
Unsterilised male cats
These are normally feral cats. They have a territory which contains a number of females or female groups. They want to repel other males. They may destroy kittens fathered by other males because they carry the genes of rivals.
Sarah Hartwell (messybeast.com), states that unsterilised male cats i.e., tomcats can distinguish between kittens that their own and kittens fathered by another male by their body scent. The secondary benefit is that the queen (the unspayed mother) will come into oestrus quickly and he can then mate with her to create his own kittens.
Tomcats desire to create their own generations of kittens. Hartwell states that “itinerant tomcats” by which she means tomcats without a home range, may “kill all kittens below a certain age”.
And some rogue tomcats may kill the kittens that they have sired which is described as an aberration because it goes against the survival of the species and the family. There is no biological advantage in doing this.
Dr. Desmond Morris (Cat World) describes a tomcat killing his own kittens as “lust-inspired infanticide”. He disagrees that it takes place because the tomcat kills his own kittens in order to get the mother on heat so that he can then mate with her. That is lust inspired infanticide, and it doesn’t make sense biologically.
The ancestor of the domestic cat, the wildcat
We can learn about the domestic cat from wild ancestor behaviour. Desmond Morris states that observations of European wildcats, which belong to the same species as domestic cats, indicate that males sometimes participate in the rearing of the young.
One male wildcat was seen to carry his own food to the entrance of the den where the female had given birth. And in another observation a male supplied the female with food while she was unable to leave the nest.
And there have been observations of male wildcats being defensive towards humans which had not occurred before the kittens were born.
These observations were made in zoos (captive wild cats) which forced the male into close proximity with the female.
In the wild, Morris states that there is little opportunity for this kind of paternal care because the male cat is unlikely to encounter his kittens as they occupy large territories.
If a wildcat in the wild encounters a kitten they’ll probably ignore them or behave paternally towards them as is the case in zoos, or the female may attack the male as he approaches and drives him away or the male kills the kittens. This last option is extremely rare. Nearly all the encounters are described in the first three options.
Accidental kitten deaths
There may be some accidental killings of kittens by males when for example mounting a kitten. In delivering the scruff of the neck bite to subdue the “female” he may kill the kitten. The dead kitten may trigger another reaction because often dead kittens are devoured by their parents as a way to keep the nest clean. This kind of accident can lead to misleading stories about tomcat killing and eating their own kittens.
Sarah Hartwell mentions that sometimes kittens can seem like prey animals to wild cats, and indeed unsterilised domestic cats, prompting them to attack the kitten. The male’s instincts are confused.
Sterilised males
Sterilised (neutered male cats) lose these excessive, testosterone fuelled, male behavioural traits. As a consequence, they tend to be more benevolent. These male cats tolerate kittens which aren’t their own, but they may drive them away as they become adults.
With respect to their own kittens, it would seem to be highly unlikely that they would kill them except by accident as mentioned. Most domestic cats as you are aware are neutered nowadays so this is the default behaviour.
Mothers
Queens are unspayed female cats and they protect their kittens against tomcats. It depends upon the size, her age, physical condition et cetera as to whether she has the ability to defend them. Most defend their kittens against attacks from large animals such as dogs with great courage so it is likely that they will defend them against tomcats.
Tomcats tend to attack kittens when the mother is away. This indicates that the mother does attack tomcats and the male cat is aware of this and avoids her.