Measuring risk means walking a thin line. Balancing what is highly unlikely from what it totally impossible. Financial institutions need to be prepared for the highly unlikely but must avoid getting sucked into wasting time worrying about the totally impossible.
Here are some sins that are sometimes committed by risk measurers:
1. Downplaying uncertainty - Uncertainty increases when loss size increases. Do your users know that?
2. Comparing incomparables - Do you add together numbers from different models? Are they really consistent calculations?
3. Validate to Confirmation - Do you stop validation when you get a check that shows that you are right?
4. Selective Parameterization - How different would your parameters be with different data used to set the parameters? How different would the outcome be?
5. Hiding behind Math - Usually the selection of mathematical model is the most important assumption. Do you users understand the implications of that assumption?
6. Ignoring consequences - do you consider what will happen because of your calculations of risk?
7. Crying Wolf - Do you feel that it is your job to only pay attention to the downside?
But there is a sin that may be much more important than any of these ... SeeRISKVIEWS.