Throughout world history, when territories are originally settled and/or developed, there has been a natural attraction for humans to waterways. Remember that back in the day when these settlements first came into being there were no cars or planes. Water was the chief source of transportation, and as most if not all of the settlements were near waterways, it was easier to travel and trade with nearby settlements over water routes.
Oceans also produce food year around, unlike agriculture which has seasons and dormant periods. So people living along the coast often were fisherman for their livelihoods.
I would assume that Australians living near the coast do so for much the same reason that anyone would like to live near the water. Being a native and long-time Floridian and currently living 80 miles from the nearest beach, I find the distance too far for my own desires. The lure of the water is a great one. It provides recreation--particularly swimming, boating and water sports--as well a beach for other similar pleasures. The coastal waters create cool winds in the summer months. For most people, the vista is intensely pleasurable. Living near the coast creates easy access to seafood, whether purchased in a store or caught by the individual.
As for Australia, the island is a vast and arid one, with deserts making up a great deal of the land mass. All of the nation's major cities are found on the various coasts, and these cities and their inhabitants are there for many of the same reasons as listed above. Inland towns and communities are remote, and most of the island's population live along the coast. Historically, most early towns and cities originated on the coast or on rivers due to the easy access by ship. Even in the era of the plane and automobile, shipping is an important form of economic transportation, another major reason for Australia's large coastal populations.
People live near the coast, which means in a 50 kilometre distance from the coast because of the infrastructure built there. First settlement landed from the coast and thus development occurred there. Approximately 85% of the population lives in 1% of Australia. This is because of the jobs, clean water, schools, infrastructure that is in the city.
Basically, Australia is mostly desert/outback. Our entire interior is fairly hot, dry and uninviting for living in (both human and plant). There are few constant water sources in the Australia Interior, and due to Australias water/drought cycle rain cannot be relied upon for agricultural/domestic use.
In coastal regions, there is more non-human life (plants, animals etc.) which means that it is more habitable for humans as there are available resources. Also, there is more chance of rain and there are more permantent water sources. Living near coastal regions also means more access in terms of international trade, which Australia relies upon heavily in terms of export of raw materials and import of manufactured goods.