Not as hard as you might expect; at least mathematically, if you can handle Classical Mechanics you will be able to cope with the maths in Quantum Mechanics. The big step up is on the conceptual level; whereas Classical Mechanics tends to be intuitive (which is not to say it is easy, just that the answers generally "make sense"), Quantum Mechanics is physically deeply unintuitive and filled with conclusions which would seem nonsensical were it not for the fact that they have been rigourously proven experimentally. However, I would argue it is well worth the struggle, it's probably the area of physics I most enjoyed during my undergraduate degree, and I find those unintuitive implications both profound and endlessly fascinating. Plus of course it means you can feel smug whenever people argue about determinism in the backroom

QM has a fearsome reputation but so long as you have a good grasp of Classical Mechanics and calculus it's not especially harder than any other advanced topic in physics (electromagnetism, fluid mechanics, relativity etc). If you really want you can take a purely mathematical approach to it and not worry about the physical implications, although I would argue this would be missing out on the best bits.