Are these finance companies ripping us off?

VAT is a set rate and as you said is set by government. Interest rates are set by finance companies. If the Bank of England lower or increase the base lending rate the majority of High Street lenders normally follow the lead. however they are not legally obliged to. If you have a poor credit record or you shop on store cards the interest rates on these tends to be higher and there is less chance of the interest rate being reduced.