Interaction Elasticity by Jakob Nielsen contains some useful data for those of us writing usability requirements:
Yes, we know that response times must be less than 1 second for navigation to feel seamless and less than 10 seconds to prevent a user’s attention from wandering. These time limits are caused by the human brain’s structure and are thus firm and stable decade by decade (indeed, century by century).
The entire article is interesting–give it a quick read for some nice reference material on determining response time requirements.