Ivor Brown started his career playing with Derby County, but didn't make it to the first team. He began the 1909-10 season playing for Ripley Town in Derbyshire, but was spotted by Tottenham, who brought him to White Hart Lane. Joined initially as a trialist in October 1909 and was drafted straight into the senior side, playing twice before signing professional forms with the club for a fee of £80 plus £6.75 signing on fee.
As a forward who failed to hit the net in his 12 games for the club, he was not perhaps the best attacking signing the club have made, but Ivor dropped down to the reserve side and scored 11 goals in eight games in the South Eastern league in 1909-10 and then another 11 in 25 games in the South East league, of which the Spurs team finished top in 1910-1911.
Unable to regain a first team place, Brown move on to Coventry City for a fee of £75 in July 1911, where he scored regularly in the Southern League, before moving on to play at Reading for one season in the same league and then Port Vale for the 1914-15 Central League season.
In 1919, after World War I, Ivor returned to Southern League to turn out for Swansea Town and got two goals for the Welsh side and finished his career playing for Porth Athletic in Wales.
Remaining in Wales after his playing days were over, Ivor Brown died in Swansea in 1966.