Abundant and rare microbiota both play crucial roles in sedimentary biogeochemical processes and have a profound influence on sediment functions. Uncovering the diversity patterns and assemblies of abundant and rare microorganisms is critical for understanding ecosystem diversity and function. We investigated the assemblies and diversity patterns of abundant and rare microbes in mangrove and intertidal mudflat sediments. Our results showed that the abundant and rare bacteria and fungi had distinct distribution patterns in diversity. The dispersal of abundant microbial taxa was less limited than that of rare taxa, and different variables drove their assemblies. The abundant taxa had significant responses to edaphic variables, but edaphic variables explained only a small proportion of the variation within the rare taxa, suggesting that there may be more complex unknown variables influencing the structure of rare taxa than abundant taxa. Cooccurrence network analysis revealed closer relationships between bacterial species and their enhanced effect in the community compared with fungal taxa and patterns wherein microorganisms in the mangrove regions had more interactions than those in the intertidal mudflat regions. Because rare microorganisms may be as important to sediment functions as abundant species, this patterned shift in microbial taxa is essential to take full advantage of their contributions to mangrove ecosystem services.