Having spent time as understudy to Ian Walker as third choice keeper, Kelly wanted first team football, so Spurs let him go on an extended loan period at Kingstonian. He did well for the non-League club, helping them to a successful FA Cup run, although in the League, they did not do so well. Had spells on loan at Chelmsford City and Kettering Town too.
Never had a first team opportunity, although he got as far as being on the bench in 2002 and was let go by the club at the end of the 2001-02 season, but he stayed on until he could find a new club. Trailed for Southend United in 2002, but Gavin signed for Welling United before moving on to Cambridge United in 2003, when he also had a trial at Mansfield Town and signed for Scarborough in 2005.