There is a simple answer to this and a very complex answer. I will try to go with simple, since I am not the total expert, and it's easier to assume you aren't either (or you wouldn't be asking).
Hardware- This is the stuff you can touch. It's hard and heavy.
Software- This is the stuff you can't touch. Technically, it's really just a bunch of magnetized/demagnetized spots on a medium, or a set of very tiny open or closed switches in a memory chip.
Lets put this into a very simple example. Riding a bike.
The bike and your body are the hardware. The knowledge of how to ride the bike and where you want to go is the software. The chain and the pedals move the wheels, the handlebars steer and the body provides the motive force as well as the fine tuning for navigation. But the head has the information that instructs the body how to pedal and to maneuver, while also providing the map navigating to your destination. Without your brain, the bike is just a piece of metal and rubber that doesn't do anything.
Now we'll move to a little more detail.
Your computer is really just a bunch of machinery. Motors, switches and storage devices. Some of these are VERY small, with millions of switches and microcircuits built in.
When you turn on your computer, it is like a baby that has absolutely no knowledge whatsoever. It needs to be fed instruction every single time you start it up.
The first set of instructions is in the BIOS (Basic Input/Output System). These are the instructions that tell the computer how to access things like Memory, Processors and Harddrive. The device that holds these instructions is a small device with two types of memory, called ROM (Read Only Memory) and RAM (Random Access Memory). Rom holds hardcoded information that can't change, while RAM is dynamic, and can be rewritten. However, RAM requires electricity to maintain that information, so a small battery is placed on the computer to support it.
Once the BIOS tells the processor how to access various components, the hard drive is started to take over the boot up tasks. This is where you have your operating system, like Windows, as well as every other program you have loaded.
When starting up (and also when adding things like USB devices), drivers, another instruction type of software, are loaded for every piece of hardware you have. Drivers have the instructions that tell the CPU (the actual brain) how to access and operate devices at a mechanical level.
At its very basic, software is a series of 1's and 0's. Or you can think of them as On/Off switches. Every piece of software is binary (using only 1's or 0's). The operating system of the computer is what takes any software code and translates it into binary. You can think of these 1's and 0's as positions on a switch.
If your computer was a light switch, your software would be exactly 2 digits long. If the software wants the light on, it sends a 1, which closes the switch and turns on the light. Sending a 0 opens the switch, and turns off the light.
A computer program does the exact same thing. Except there are tens of millions of tiny switches, and they determine how the computer is going to act. Some basic actions the computer performs require the combined activity of thousands of these switches.
When you look at a color on your monitor, the software is telling the CPU what color to display in a specific pixel, or single dot on your monitor. The color black is notated as binary 000000000000000000000000, while white is 1111111111111111111111111, and gray is 011001000110010001100100. The software also has to tell the CPU which pixel to make that color, so another binary number is issued for that location.
Many of these switches are on the CPU itself, but there are processors in the video graphics card, the sound card and the network card that help out the CPU and speed the process up.
When you look at the CPU, every one of those dots on the underside are a pathway, either in or out. The software activates or deactivates the microswitches and determines which path the instructions travel.
The above diagram is the underside of a very simple microprocessor. Each of the dots are the end of a switch. The software tells which switch to open or close, determining where the instructions are going to end up.