Richardson was born in Lubbock, Texas, the son of Billy Jack Richardson and Luddie Hicks. Richardson was a 1974 graduate of Manual High School in Denver, Colorado. He averaged 10 points on a talented team and did not start for the varsity until he was a senior. Richardson played on the 1972 state championship team.

The New York Knicks drafted Richardson with the fourth overall pick in the 1978 NBA draft, and he was billed as “the next Walt Frazier.” Two picks later, the Boston Celtics drafted future Hall of Famer Larry Bird. In his second year, Richardson became the third player in NBA history (Slick Watts – 1976, Don Buse – 1977) to lead the league in both assists (10.1) and steals (3.2), setting the Knicks’ franchise records in both categories. He also recorded 18 triple-doubles, the second-most in franchise history.

At the beginning of the 1982–83 season, on October 22, 1982, Richardson was traded to the Golden State Warriors (along with a fifth-round draft choice) in exchange for Bernard King. After playing only 33 games for the Warriors, Richardson was traded to the New Jersey Nets in exchange for Sleepy Floyd and Mickey Johnson on February 6, 1983.

Richardson was named an All-Star in 1985. In the 1984 playoffs, Richardson led the Nets to a shocking upset of the defending champion Philadelphia 76ers. In the fifth and deciding game, he scored 24 points and had six steals. Richardson wore Leather Converse All Stars briefly with the New Jersey Nets, making him the last to wear the shoe in any form in the NBA.

On February 25, 1986, Richardson was banned for life by NBA commissioner David Stern for three violations of the league’s drug policy. He regained the right to play in the NBA in 1988 if he remained clean, but decided to continue his career in Europe. He never played in the NBA again, despite being reinstated.

Richardson won the European-wide second-tier level FIBA Cup Winners’ Cup, in the 1989–90 season with Virtus Bologna. He won the French League championship with Olympique Antibes in 1995.

On December 14, 2004, he was named head coach of the Albany Patroons in the Continental Basketball Association. Richardson had previously played with Albany in 1987-1988, when it won its second CBA championship under Coach Bill Musselman.

On March 28, 2007, Richardson was suspended for the remainder of the CBA championship series for comments in an interview with the Albany Times Union , in which he stated that Jews were “crafty (because) they are hated worldwide.”The paper also reported that Richardson directed expletives at a heckler, using profanity and an anti-gay slur, at Game 1 of the championship series.

On May 24, 2007, Richardson was named head coach of the reincarnated Oklahoma Cavalry of the Continental Basketball Association. On December 16, 2007 he was fired by the Cavalry, for sticking up for his players when their paychecks bounced, but rehired the next season.

Richardson coached for the relocated Lawton-Ft Sill Cavalry located in Lawton, Oklahoma, winning three consecutive championships in 2008-2010. Richardson led the Cavalry to victory to the Continental Basketball Association Finals in 2008 and 2009 and in the Premiere Basketball League Finals in 2010.

On August 17, 2011, Richardson was hired as the first head coach of the National Basketball League of Canada’s London Lightning. Richardson was named the NBL Canada’s first ever Coach of the Month for November 2011, an award he would win again in January 2012. London finished the regular season at 28-8. On March 25, 2012, Richardson led the Lightning to a 116-92 victory over the Halifax Rainmen in the deciding Game Five of the NBL Canada Finals to win the NBL Canada’s inaugural championship. After the game, Richardson was named the NBL Canada Coach of the Year for 2011–12.

On April 12, 2013, Richardson led the London to an 87-80 victory over the Summerside Storm and TLondon became back to back NBL champions.