I will of course defer to the expertise of Col. Drydock, but I think that a No.1 Mk III Lee-Enfield would be ideal for Expansion Era shooting.
The initial "Rifle, Short, Magazine Lee-Enfield" (SMLE) was introduced into British military service in 1903, and its definitive Mk III version came along in 1907. (Although now commonly referred to as the No. 1 rifle, that re-naming of the SMLE did not actually happen until 1926, so until well after WWI the older terminology is most appropriate, historically.)
I have used a SMLE rifle in Expansion Era competition ...
Of course, the most appropriate companion handgun would be a Mk I to Mk V Webley revolver, although if I am not mistaken the Mk VI version (not introduced until 1915) is accepted ...