In 1973, Yamashita formed the band Sugar Babe with musicians including Taeko Ohnuki and Kunio Muramatsu, and released their sole album Songs two years later in 1975. After the group disbanded in 1976, Yamashita signed to RCA and launched his own career, with release of the solo album Circus Town the same year. Also the same year he teamed up with Eiichi Ohtaki, who was the producer of Sugar Babe, and brief Sugar Babe member Ginji Ito to release an album titled Niagara Triangle Vol. 1 that was cited by MTV as one of the finest collaborative efforts from that period. His 1979 award-winning album Moonglow gained moderate success, but his 1980 song “Ride on Time” peaked at No. 3 on the Japanese Oricon and the same-titled album topped the chart subsequently, propelling him to stardom.

As a solo artist, Yamashita has released 17 studio albums, 2 live albums, multiple compilations and over 40 singles. He has been one of the most commercially successful Japanese male solo recording artists in the history of the Japanese album chart, selling approximately 9 million albums in total. He has also composed for films and television commercials, and worked on records by other artists. He is particularly successful as a musical collaborator for Mariya Takeuchi, whom he married in 1982. They have one daughter.