Brown was born with his twin brother Wes, in a low-income section of  Miami, Florida. He was adopted by Mamie Brown, a 38-year-old single woman who worked as a cafeteria attendant and domestic assistant. Brown said that he was labeled “educable mentally retarded” in elementary school, which damaged his self-esteem and confidence.

When Brown decided to get his life together and get his career going, he was hired by a radio station where he was repeatedly unsuccessful with his speeches. It was only after the on-air failures of the previous afternoon DJ that he was hired full-time. Upon his termination from the radio station, he ran for election in the Ohio House of Representatives and won. After leaving the State Legislature he shifted his career to television and became the host of his own show on PBS and had a syndicated radio show on KFWB in California. Les went on to marry R&B soul singer Gladys Knight, became a motivational speaker, and wrote Live Your Dreams, a motivational self-help book encouraging personal achievement and goal setting.