What do the Conditions and AudienceRestriction elements of an assertion do, and why must the SP check them?

Question

Accepted Answer

Conditions set the validity window (NotBefore to NotOnOrAfter), and AudienceRestriction names the service provider(s) the assertion is meant for; an SP must reject an assertion that is outside the window or addressed to someone else.
Imagine an IdP issues an assertion for a harmless partner site. If that partner were malicious and the assertion carried no audience, it could forward the assertion to your bank's SP and log in as the user. The <Audience> value (the SP's entity ID, https://rp.gov.ch in the example) prevents exactly this: the bank sees it is not the named audience and refuses.
The time window does the same in the time dimension: an assertion is a snapshot of an authentication, not a permanent credential, so it is valid only briefly. Because the window is checked against the SP's clock, IdP and SP need synchronised clocks (in practice a small allowed clock skew is configured).
Other optional conditions include OneTimeUse (the assertion may be used only once) and ProxyRestriction (limits on passing it on).
Go deeper:

OASIS: SAML V2.0 Core specification (PDF) — section 2.5 defines conditions, audience restriction and one-time use.
 OWASP SAML Security Cheat Sheet: validate protocol processing rules — the checks an SP must perform on every response.