But Peterhead's Nicky Clark levelled five minutes later when he received Peter Bruce's cross and fired home from inside the penalty area.
Albion almost regained the lead in the final minute as Chris Aitken's free-kick cannoned off the crossbar.
Peterhead: Jarvie, Strachan, Stuart Smith, Mann, MacDonald, McVitie (Gethans 79), Barry Wilson (Bruce 67), Emslie, Bavidge (Young 85), Clark, Sharp. Subs Not Used: Bateman, Bowden.
Booked: McVitie.
Goals: Clark 86.
Stirling: Hogarth, Graham, Forsyth, Allison, Page, Robertson, Taggart (Gibson 63), Aitken, Elliott (Corr 89), McKenna (Byrne 58), O'Brien. Subs Not Used: Christie, O'Neill.
Booked: Byrne.
Goals: O'Brien 81.
Att: 497
Ref: F McDermott
Source: BBC Sport
Source: BBC Sport