Toussaint was born in Pembroke Pines, Florida, to Dany Toussaint and Kahaso Kiti. He is of Haitian and Kenyan descent. Toussaint’s nickname, Touki, is a portmanteau of his parents’ surnames. The family moved to Haiti when he was three months old. Toussaint and his mother moved back to Florida when he was six. He stopped playing baseball at age 10 due to a strikeout-filled season as a batter, and focused on soccer and hockey only to return to baseball two years later. Toussaint attended Coral Springs Charter School for his freshman year before transferring to Coral Springs Christian Academy also in Coral Springs, Florida. As a junior in 2013, Toussaint was a MaxPreps All-American after he was 6–2 with a 0.78 earned run average (ERA) and 83 strikeouts. He also hit .458 with six home runs and 32 runs batted in. He was named Broward County’s Class 5A-4A-3A-2A Player of the Year. Toussaint committed to play college baseball at Vanderbilt University.

Toussaint was considered one of the top prospects for the 2014 MLB draft. The Arizona Diamondbacks selected Toussaint in the first round, 16th overall, of the draft. He signed on June 20, agreeing to a $2.7 million signing bonus, and made his professional debut with the Arizona League Diamondbacks on July 2. On August 3, he was promoted to the Missoula Osprey. Toussaint pitched his first Pioneer League game on August 8, 2014.

On June 20, 2015 he was traded to the Atlanta Braves along with Bronson Arroyo for Phil Gosselin. He finished the 2015 season with a 4.83 ERA in 17 total starts, including 5.73 ERA in ten starts as a member of the Rome Braves. The rocky season was attributed to the Diamondbacks’ request that Toussaint cut down on the use of his curveball, shortly after his debut for Kane County, as the team felt that becoming excessively reliant on the pitch early in his career would lead to injury. Not throwing his curveball gave Toussaint time to improve his changeup, which became a viable third pitch over the course of the season. Toussaint spent 2016 with Rome. He struggled to begin the season, but improved after making adjustments to his pitching delivery.

Toussaint began 2017 with the Florida Fire Frogs and was promoted to the Mississippi Braves in late July 2017, making his Southern League debut on July 31 against the Mobile BayBears. In 145 total innings pitched between Florida and Mississippi, he pitched to a 6–13 record with a 4.53 ERA along with 167 strikeouts. He appeared in the offseason Arizona Fall League with the Peoria Javelinas. Toussaint started the 2018 season in Mississippi, recording a 2.93 ERA with 107 strikeouts until a midseason promotion to the Gwinnett Stripers. Toussaint faced the Norfolk Tides in his first International League game on July 5, 2018, pitching 6 ⁄3 innings while yielding one run on five hits. He was named to the All-Star Futures Game, and appeared in relief to pitch the eighth inning.

On August 13, 2018, the Braves purchased Toussaint’s contract and added him to the roster. He made his MLB debut the same day with a start against the Miami Marlins, yielding one run on two hits in six innings pitched. Toussaint spent time at spring training prior to the start of the 2019 season, and began the year at Gwinnett.

On July 4, 2020, it was announced that Toussaint had tested positive for COVID-19.

In 2020, he was 0-2 with an 8.88 ERA over 24.1 innings.