Recent studies have begun to elucidate the remarkable diversity of oceanic prokaryotes with estimates of more than 1000 phylotypes in a single habitat. Oceanic prokaryotic communities exhibit large variations in composition at various scales in both time and space. Here, we applied the metacommunity theory to communities of heterotrophic prokaryotes in the ocean. They have crucial roles in multiple biogeochemical processes such as carbon mineralization, link between dissolved organic carbon (DOC) and higher trophic levels, hydrolyzing particulate organic carbon (POC) and vertical sinking of POC (biological pump). By a metacommunity model, we evaluated effects of immigration from a metacommunity on the carbon cycle in the oceans. The model assumed a local community that receives immigrants from the metacommunity. In the local community, both local interactions among various types of prokaryotes via competition for carbon resources (POC and DOC) and immigration from the metacommunity determine the prokaryotic community composition and the prokaryotes-mediated carbon cycles. This model predicts that flexible shifts in prokaryotic community composition, which are realized by the immigration from the metacommunity, lead to the higher efficiency in mineralizing organic carbon resources during the changes in the supply rate of organic carbon (i.e., primary production by phytoplankton). This shift results in the decrease in the efficiency of vertical sinking of POC. Without the immigration, the larger part of fixed CO2 will be exported to the deep oceans. These results suggest that the diversity, spatiotemporal distributions, and physical dispersion of prokaryotes affect oceanic fluxes of organic matter.