I've tried both leaving it on and rinsing it off, and wasn't too impressed with how my skin felt in either case. If my skin was slightly irritated, alum just made it very irritated.
Like others, I originally left it on and got a very tight face. Then I saw a video where the user left it a minute or so then washed it off before applying their aftershave balm.
Tried that and it works a treat for my skin, but then everyone's skin is different so run with what works for you.
I've been trying out witchazel recently, but I think I prefer the alum block
I left the alum on my face for a few seconds and then rinsed with warm water. That last action appears to be the key, as I've only previously rinsed alum of with cold water, and as with mos things, the warm water dissolves the alum better, and removes it from my fizza.
Chemically there is no difference between the Indian shop ones and the expensive Trumpers or AOS ones. My missus bought me a Trumpers one at Christmas, is is huge block compared to the OSMA or other ones but I do notice the residue when dry is not quite so powdery. It makes no difference to the end result but is is a lot more pleasurable to use.