I bought a block with my first shaving kit and at first I thought it was making the irritation and bumps on my neck worse so stopped using it. I used witch hazel instead because I like the "finger feel" given by an astringent on the face after shaving. A couple of months down the line and I went back to trying alum and while the slight sting of application is still there, and I rinse it off with cold water after a minute or so, it doesn't appear to create any irritation at all. I'd say my neck has never looked better but whether or not the alum is contributing to this is hard to say; my technique could just have improved and I'm no longer going fully ATG on the neck. I'll keep on using it as part of the routine for now.
If you're just starting out and find the alum isn't working out, perhaps give it a few months and re-visit it before writing it off completely.