Hypolithic microbial communities (hypolithons) are complex assemblages of phototrophic and heterotrophic organisms associated with the ventral surfaces of translucent minerals embedded in soil surfaces. Past studies on the assembly, structure and function of hypolithic communities have tended to use composite samples (i.e. bulked hypolithic biomass) with the underlying assumption that samples collected from within a ‘homogeneous’ locality are phylogenetically homogeneous. In this study, we question this assumption by analysing the prokaryote phylogenetic diversity of multiple individual hypolithons: i.e. asking the seemingly simple question of ‘Are all hypolithons the same’? Using 16S rRNA gene-based phylogenetic analysis of hypolithons recovered for a localized moraine region in the Taylor Valley, McMurdo Dry Valleys, Antarctica, we demonstrate that these communities are heterogeneous at very small spatial scales (<5 m). Using null models of phylogenetic turnover, we showed that this heterogeneity between hypolithons is probably due to stochastic effects such as dispersal limitations, which is entirely consistent with the physically isolated nature of the hypolithic communities (‘islands in the sand’) and the almost complete absence of a liquid continuum as a mode of microbial transport between communities.