One of the biggest reasons people like the Fuji's is because of the sensor. The Xpro1, XE-1 (and XE-2 I think) and XM-1 use the same APS-C sized Fuji sensor, and it's a real gem apparently. The lenses are also excellent quality (even by pro standards).
Most of the pros who made the switch did so based on IQ alone and the pro photographer I mentioned earlier uses his Fuji for shooting everything except sports (where he uses a Nikon DSLR).
The cable release is handy because the batteries last forever, but you can always use the Fuji app on a smartphone to control the camera remotely (on some models at least).
Other than size, another advantage of mirrorless is they're much quieter than DSLRs because there's no mirrorbox to flap around, and all of this can help the subject feel more at ease (we've all been at a family get together when uncle Bob terrorises everyone with his hulking new DSLR, right? And everyone in the pictures has that "omg
" look.
). The quietness may also mean that you can take pictures you wouldn't be able to with a noisy DSLR. For example, I took a picture completely silently with my little X10 during my cousin's classical guitar recital (where you could hear a pin drop). The official photographer with a DSLR had to wait until the encore and didn't get a very good shot of him playing.
Many people find the controls are more intuitive too - I had no idea how useful I'd find the dedicated exposure comp wheel. It's just so much easier and quicker to use than my Canon DSLR, it's almost invisible in my hands - like it isn't even there. I find I take more pictures and spend less time fiddling, and concentrate more on what's happening around me rather than the camera.
I agree the looks are subjective. Fuji's X range of cameras all have the retro look about them and having used plasticy, rounded DSLRs for quite a few years, personally I quite like the return to solid, angular metal. I always liked the classic rangefinder look.