I'm also a fan of oak, except in any application where it will come into contact with iron or water. The former because of the tannin which reacts with iron, corroding it and causing a black stain - brass screws are
de rigeur for quality oak furniture - and the latter on account of the unusually high rate of movement with variations in moisture content. Both of these apply double to English oak, but are less problematic with American species, whilst Japanese red oak is actually rather well behaved stuff. For these reasons I'd never consider it for tool handles and razor scales, and be very wary of using it in a brush handle unless it was absolutely blathered with AC lacquer.
Walnut, padauk and sapele: all fine, (except that I think a brush deserves better than sapele), but all three are comparatively open grained. Padauk comes in two completely different varieties, African (Pterocarpus soyauxii) and Andaman (Pterocarpus dalbergioides, P. macrocarpus, P. indicus), the latter being generally regarded as the better timber especially in the burr form which is sold as amboyna. The striking red colouration of the African variety darkens considerably with exposure to air, and there is a chance that this colour may run (it's a dye wood).
Sale of lignum vitae is very tightly restricted in the UK; I'm not even sure if it is obtainable nowadays, although there's plenty to be had in the form of old bowls (as in crown green, that is!). I wouldn't worry about its stability with regard to water: it was used in ship's propeller bearings on account of its self lubricating properties and durability when exposed to salt water.
The ideal stuff to use for a brush handle is, of course, oosik.
