The advantages to the public of using a private contractor are several, beginning with the key point that its contract can be non-renewed. When pensions and longterm care are added in, the costs of public sector alternatives are sometimes higher and the quality of prisoner care and security may not be better. Both public and private options are potentially vulnerable to corruption.
So in general it’s impossible to say in the abstract which is better. Instead the elected executives need to get the best deal they can for the public to keep the criminals safely incarcerated and to keep our taxes down. If you think a private company is making excessive profit, either run for office or set up your own company and underbid them and take over the contract.
Finally it is wrong to confuse legal confinement of those convicted of violations of law with the involuntary servitude of people who have committed no wrongs.