Central defender Tom Lees settled the match with a firm header past Elliott Parish from a Peter Sweeney corner 10 minutes after the break.
He said: "It's always going to be a tough game away at Bury. Obviously, they are going for promotion and at home looking to win the game.
"It was almost a perfect away performance, just a lapse of concentration on one set play has cost us.
"With our system, it tends to keep you in the game. When we played Bury at home we played 4-4-2 and we got so over-stretched that they beat us 5-0. There was no way we were going to come here and play 4-4-2 and get battered again.
"It worked for long periods but just one corner has killed us. We've got to concentrate on Monday now and be positive."
Source: PA
Source: PA