I'd highly recommend starting with Arduino, there's a whole host of official and 3rd party Chinese knock offs of the boards, but I'd go for an Arduino UNO to start with.
All you need is the board itself, a USB lead, the Arduino IDE software (Free of charge from the official website) and a bit of an idea what you want to try.
Have a look at some of the education and beginner kits on eBay, you often get the board, components, projects in pdf format, and the lead.
I use these things for all sorts of DIY projects ranging from interfacing ANPR cameras with gate controllers to full climate control systems for my reptiles, the limit is your imagination.
There's also hundreds, if not thousands, of forums dedicated to the world of Arduino programming.