He was married twice, first to actress Emma Littlefield from 1902 until her death on June 25, 1934 at aged 51, then to Shirley Paige in 1942, but the marriage was not announced until a year and a half after they were married, at the time of the announcement Victor Moore was 67 years old and Shirley Paige was just 22 years old, they were married until Victor Moore's death 20 years later. He had 3 children with his first wife Victor Jnr (born 1910), Ora (born 1919) and Robert (born 1921).