I quite often ride with a DSLR, a great solution I have found is buying a small DSLR camera bag just big enough for the camera and one smaller lens attached. I usually take my kit lens biking instead of something nice just in case... But then I put this bag in the helmet holder of my small backpack and put the strap through the top of the camera bag.
Hmmm I might be tired as that didn't make the most amount of sense reading that back, so I'll take a pic.
So far this has held up for 2 years of single track riding, a multi day ride, heaps of technical tight track and even a Super D race. I have had some big offs too but luckily not landed on the camera - although I did smash some ribs. Camera stayed put!