Child Visitor Visas have been replaced by the Standard Visitor Visa as of 2016 and therefore all visitors are now under the effect of Appendix V of the Immigration Rules. Specifically, section V4.13 states that:
A child who holds a visit visa must either:
(a) hold a valid visit visa that states they are accompanied and will be travelling with an adult identified on that visit visa; or
(b) hold a visit visa which
states they are unaccompanied; if neither applies, the child may be
refused entry unless they meet the requirements of V 4.12.
Therefore it's a violation of the rules to leave the child in the UK after entering the country as it's required that the child is accompanied by the individual named in the visa's endorsement. You agreed to this condition when the visa was issued.