I duct tape a cell phone to my hairy man chest, had better results.
I actually have a little camera that I mount inside the cockpit of my little RC planes that works great in lieu of a gigantic gopro.
It's cheap too:


You can find it on Amazon too, video quality isn't HD or anything, but it's so small you can literally stick it to your forehead with superglue.
Comes with a clip if you'd rather put it on your shirt or something gay like that, also has a few mounts and a silicone case.

If you want something small and basic that captures HD video for only $37, you can buy one of these:

You really don't need any sort of hard case or waterproofing unless you're going to ride through mud and rain.


Something super cheap, just for the purpose of recording any kind of video?
How about a camera the size of a key fob selling for $7?