We present evidence that infant mortality of stellar clusters is likely to be a major and very efficient process for the dissolution of young clusters in the spiral galaxy NGC 1313. Performing stellar PSF photometry on archival HST/ACS images of the galaxy, we find that a large fraction of early B-type stars are seen outside of star clusters and well spread within the galactic disk, consistent with the scenario of infant mortality. We also calculate the UV flux produced by the stars in and out the clusters and find that 75 to 90% of the UV flux in NGC 1313 is produced by stars outside the clusters. These results suggest that the infant mortality of star clusters is probably the underlying cause of the diffuse UV emission in starburst galaxies. Infant mortality would also explain the numerous B-type stars observed in the background field of our Galaxy as well. We exclude the possibility that unresolved low-mass star clusters and scaled OB associations might be the main source for the diffuse UV emission.