When divergent boundaries occur under continental plates, they form a rift valley. Initially, the valley is fairly narrow, but it still allows rivers or streams to flow through. At one point, it will dip down below sea level, allowing a sea to form. Given the right conditions, and enough time, new ocean basins may even form.
The East Africa Rift Valley is an example of a continental divergent boundary.