There are other nasties that build up in your tank besides nitrates that need to be removed with water changes. Some fish also emit what are called pheronomes (not sure of the spelling) which can restrict growth in other fish in the tank if the water isn't changed regularly. Just think of it this way, if your fish were out there in the lake or stream, they would always be in fresh water, but since they are confined, we have to try and keep that water as fresh as possible. As your water builds up a concentration of these nasties that are not normally tested for, your fish will adjust to the conditions, and you will not notice any change in their behavior. However, 6 months down the road, you could run into problems, and by the time you realize there is a problem your fish cannot be saved. It's called old tank syndrome, and it's pretty much a silent killer:
http://www.bestfish.com/oldtank.html
The reason it is usually recommended to do weekly water changes is so your water conditions generally stay the same all the time. Longer time between water changes can result in the conditions in the tank changing quite drastically, and making the fish sick in the process as they try to adjust to the new conditions.