Brian started his LoI career at Bray Wanderers in 2005, spending two years at the seaside club before moving across Dublin to sign for Shelbourne at the start of 2007. Brian spent just one year at Shels before joining Monaghan United in 2008 where he won the 2009 Supporters Club Player of the Year. In January 2011 he moved north of the border to sign for Portadown in the Irish League.

On 8 July 2013 Brian returned to the League of Ireland, signing for Stephen Kenny at Dundalk after two and a half years at Portadown. He made his debut for the club in a 3-0 win over Bohemians at Oriel Park on 12 July 2013 and became an immediate fixture in the back four. He scored twice in 13 league appearances over the latter half of the season as the Lilywhites finished in second place just three points behind champions St Patrick's Athletic.

2014 proved to be an even better season for Brian and Dundalk, with him making 30 league appearances and scoring eight times, as the Lilywhites won the league title on the last day of the season. 2015 brought another Premier Division title, with the centreback scoring six times in 27 league appearances. He also appeared in both of the Lilywhites' Champions League ties and in the FAI Cup Final, in which Dundalk beat Cork City 1-0 after extra-time. A third consecutive league title arrived in 2016 with Brian scoring seven times in 29 league appearances while also playing in seven of the club's 12 European matches. There was also another FAI Cup final appearance for Brian, although this time Dundalk were beaten 1-0 by Cork City.

In October 2017 Brian re-signed with the Lilywhites for the 2018 season as well as the 2019 campaign. In January 2021, Brian re-signed for another year with Dundalk having first joined in 2013.