Actually IIRC the numbers Braudel crunched in The Mediterranean in the 1500s much of the Spanish cash flow went to pay for the bloody morass that was the Eighty Years' War, naval pissing contests both on the Atlantic and the Med, courtly high living, and the pockets of the Genoese bankers who pretty much had a lock on the Spanish eonomy...
May also have contributed to the wage-spiral and silver inflation of the period.

Somewhere down the road much of the precious metal probably did end migrating to Asia either at the Levant terminals or at the source, though.