He was born in Oakland, California, USA on September 18, 1905, into a family of performers. Anderson began his show business career at age 14 in a song-and-dance act with his brother Cornelius and another performer. They billed themselves as the Three Black Aces. At a young age, Anderson permanently damaged his vocal cords (he had to yell loudly for his job selling newspapers), leading to his trademark "raspy" voice. Benny's ordering of his "valet" and Anderson's responses (sometimes a resigned "Yes, Boss", but just as often a snappy joke at Benny's expense) were among the weekly highlights of the long-running show.
Anderson's role as a servant was common for black leads in the popular media of that era, such as Ethel Waters in Beulah. The stereotyping of blacks (or any ethnic group) had been standard practice in the entertainment business for generations. The relationship between Anderson and Benny became more complex and intimate as the years went by, with Rochester's role becoming both less stereotypical (in early episodes he carried a switchblade and shot craps) and less subservient (though he remained a valet), reflecting changing social attitudes toward blacks.
In June 1939, Anderson married Mamie Wiggins Nelson, but sadly after 15 years of marriage, Mamie died August 5, 1954, after a 2-year battle with cancer. Mamie was 42. At the time of her death, her son Billy (whom Eddie had adopted) was playing professional football for the Chicago Bears. Eddie Anderson remarried in 1955, to beauty Evangela 'Eva', the couple had two daughters Stephanie and Evangela Jr. ("Eva"), and son Edmund Jr.