Time Machine is the built-in backup that works with your Mac and an external drive or AirPort Time Capsule. Simply "connect the drive, tell Time Machine to use it, and relax" (Apple's Marketing copy). That's all true. In fact, if you're looking for a system for local data backup, I absolutely recommend using Time Machine. It really is that easy to use (and very effective). Time Machine keeps hourly backups for the past 24 hours, daily backups for the past month, and weekly backups until your backup drive is full.
So to start using Time Machine, you first need to have an available external hard drive (I recommend using a drive that won't be used for anything else). If you're buying a new drive, make sure that it's at least twice as large (in capacity, of course) as your internal hard drive or SSD. For example, if you have a 500 GB hard drive, make sure the drive you use for Time Machine is at least 1 TB (and in many cases, there isn't much of a price difference between a 1 TB drive and a 2 TB drive, so you may as well go big).
When you connect your new drive for the first time, you will be prompted to use this drive as a Time Machine backup drive. As tempted as you will be to say "Yes" (again, because it really is that easy), you want to say "No". Why? Because new hard drives come pre-formatted using a format called FAT-32. The upside to this format is it works on both PCs and Macs. The downside to this format is it's not very space efficient if you plan to only use it on a Mac (or a PC for that matter). It will appear to be full sooner than it should be because of the inefficiencies.
Before using the drive as a Time Machine drive, let's reformat it. By doing this, you will gain the maximum storage possible. To do this, connect the drive (if it's not already) and open Disk Utility (Applications > Utilities > Disk Utility). On the left side of the window, highlight your external drive. Now on the right side of the window, click on the "Erase" tab. Change the format to "Mac OS Extended (Journaled)" and give the drive a name (for example "Backup Drive" or "Time Machine").
When you're ready, click "Erase...". Note: this will erase all data on the drive, so be sure there's nothing on the drive that you want to keep. The formatting process itself will take less than a minute. When it's done, you can Quit Disk Utility.
At this point, OS X should have prompted you to use the new drive (now called "Backup Drive") as a Time Machine backup drive. If it did, choose "Yes" and you're all done. If it didn't prompt you, we'll need to set up Time Machine manually. To do this, open System Preferences, then click "Time Machine". Now, click on the "Select Disk..." button. You will see a list of available drives to use. Select the new drive, then click "Use Disk".