Klein was born on November 19, 1942 to a Jewish family in the Bronx, the son of Flore (née Stern) (1909–2006) and Leo Klein. Leo had immigrated to New York from Hungary, while Flore was born in the United States to immigrants from Galicia (Eastern Europe) and Buchenland, Austrian Empire (modern day-Ukraine).

Klein married Jayne Centre, a textile designer, in 1964. They have a daughter, Marci, who currently works as Talent Producer for NBC’s Saturday Night Live. They divorced in 1974. In September 1986 Klein married his assistant, Kelly Rector, in Rome while they were on a buying trip in Italy. She later became a well-known socialite photographer. After separating in 1996, they divorced in April 2006 after 20 years of marriage.

In 1974, Klein designed the tight-fitting signature jeans that would go on to gross $200,000 in their first week of sales. In that same year he also became the first designer to receive outstanding design in men’s and women’s wear from the CFDA award show. In 1983, he was placed on the International Best Dressed List. Also in 1981, 1983 and 1993, he received an award from Councils of Fashion Designs of America.

Klein went to Isobel Rooney Middle School 80 (M.S.80) as a child. He attended the High School of Art and Design in Manhattan and matriculated at, but never graduated from, New York’s Fashion Institute of Technology, receiving an honorary doctorate in 2003. He did his apprenticeship in 1962 at an old line cloak-and-suit manufacturer, Dan Millstein, and spent five years designing at other New York City shops. In 1968, he launched his first company with a childhood friend, Barry K. Schwartz.

In 2003, Klein bought an ocean-front estate in Southampton, New York on Long Island and demolished it to build a $75 million glass-and-concrete mansion. In 2015, he put his Miami Beach, Florida mansion on the market for $16 million. The Florida home sold in February 2017 for $12,850,000. In June 2015, Klein bought a mansion in Los Angeles, California for $25 million.