Forum

Replies

• 

  Roland Speklé February 7, 2012 14:45

  If anyone is interested, the final version of this paper is now on SSRN and will be published in the European Accounting Review (expected on-line availability end of February, 2012).
• 

  Beulah Misrole June 25, 2011 11:24

  Maybe this is a little tongue in cheek but my practical experience in enterprise risk management taught me two valuable lessons:

  1. I achieved the best result through co-operation and honesty from the key role-players.  This meant that I needed to take time out to engage with employees and leaders.  Only once I was comfortable in understanding the culture of the organisation and where the pockets of institutional knowledge lies; only then did I map out an organisation specific ERM maturity model using elements of models as COSO and ISO.
  2. Don’t expect 100% commitment – we are not in an ideal world; reach maximum consensus with the most influential institutional stalwarts’ behind you , that is enough to get reliable data with high enough integrity to support informed decision making.

  In my view is that risk management is  much like religion – don’t get stuck and fanatical on any one – learnt the pros and cons of each and take the best bit of them all; then exercise good judgement on when to apply what – don’t forget that the human factor is bigger and more powerful than any system, methodology or model; Yet is often over seen as the number one risk factor to overcome to achieve the rest.
• 

  Beulah Misrole June 25, 2011 10:55

  
  
  Patrick McConnell said:
  

  @Patrick, Thank you for your thoughtful analysis. It would be interesting to follow your debate with the authors if you could re-post your analysis on GlobalRisk Community.

   

  Pleased to copy from Linkedin Behavioural Finance Theory and Practice and to engage in debate
  

   

  "This study is pretty lightweight, and would only normally be worth placing in the 'another brick in the wall' pile, except that some of its conclusions feel like they might become an 'urban myth'.
  
  First, the authors point out that the data they analyze is not theirs, and are in fact quite scathing about its data collection - "no explicit attention to scholarly scale construction, validation and measurement considerations ... [and] naive". Nonetheless, this does not stop the authors drawing a very long bow from some very inappropriate data.
  
  The basic conclusion of the study that risk management is not well advanced in the Netherlands is interesting, but no different to elsewhere (nevertheless a useful finding). They also find that when ERM is widely implemented (as opposed to lip service) it is perceived to be effective and in particular, "the frequency of risk assessment, the frequency of risk reporting, and the richness of risk reporting contribute to perceived risk management effectiveness". [Remembering always that this is based on subjective assessment to a general questionnaire].
  
  The authors also raise a very valid question: is ERM the same for private and public institutions? Good question, but not answered here.
  
  If left at those observations, the paper would have been interesting but hardly controversial. But bereft of new 'positive' findings, the authors move into the murkier world of 'null result or 'negative' implications, i.e. conclusions based on questions we didn’t ask!
  
  The original questionnaire asked whether the firm used COSO; some 21% of the sample claimed to use COSO, at least in part, or some 2% of the questionnaires sent out. This is not an unreasonable number however to do some basic statistics on but hardly enough to base robust 'null' conclusions.
  
  Failing to find any interesting answers in the data itself, the authors asked the question somewhat sarcastically, if COSO is so good why don’t more firms use it? They then conclude, without supporting data, that COSO has somehow failed.
  
  First, the original questionnaire asked does the firm use COSO or not (or at least that is how the data is represented), but not whether other standards were used, such as the FERMA standard? So the authors are comparing apples with potatoes and concluding that apples don’t make good potato chips.
  
  COSO is an American not a European standard and the same result probably would have applied, for example, to Australia which standardizes on its local AS/NZS 4360. A pretty dumb conclusion is made that COSO (not the other 80% of non-COSO firms) isn’t working [by the way it might not be but this study doesn’t demonstrate one way or the other]. There is no comparison of COSO versus something else, internal or external. The question should be whether one standard is better than another and this study does not have the data to address this critical question. An even better question is whether any standard is better than none!
  
  Last the conclusion on risk appetite/tolerance is somewhat juvenile; i.e. since most firms don’t have a risk appetite or tolerance ergo they are not needed! This is akin to saying most firms don’t do risk management well therefore it is not needed – the GFC certainly proved that observation wrong!
  
  Research into how firms actually do risk management is very important, but, unfortunately, general questionnaires, such as this data set, give numbers but few insights. To their credit the authors make this point and argue for further research but unfortunately would be better placed if they had left their musings to the construction of their next hypotheses."
• 

  Patrick McConnell June 22, 2011 3:45

  @Patrick, Thank you for your thoughtful analysis. It would be interesting to follow your debate with the authors if you could re-post your analysis on GlobalRisk Community.

   

  Pleased to copy from Linkedin Behavioural Finance Theory and Practice and to engage in debate
  

   

  "This study is pretty lightweight, and would only normally be worth placing in the 'another brick in the wall' pile, except that some of its conclusions feel like they might become an 'urban myth'.
  
  First, the authors point out that the data they analyze is not theirs, and are in fact quite scathing about its data collection - "no explicit attention to scholarly scale construction, validation and measurement considerations ... [and] naive". Nonetheless, this does not stop the authors drawing a very long bow from some very inappropriate data.
  
  The basic conclusion of the study that risk management is not well advanced in the Netherlands is interesting, but no different to elsewhere (nevertheless a useful finding). They also find that when ERM is widely implemented (as opposed to lip service) it is perceived to be effective and in particular, "the frequency of risk assessment, the frequency of risk reporting, and the richness of risk reporting contribute to perceived risk management effectiveness". [Remembering always that this is based on subjective assessment to a general questionnaire].
  
  The authors also raise a very valid question: is ERM the same for private and public institutions? Good question, but not answered here.
  
  If left at those observations, the paper would have been interesting but hardly controversial. But bereft of new 'positive' findings, the authors move into the murkier world of 'null result or 'negative' implications, i.e. conclusions based on questions we didn’t ask!
  
  The original questionnaire asked whether the firm used COSO; some 21% of the sample claimed to use COSO, at least in part, or some 2% of the questionnaires sent out. This is not an unreasonable number however to do some basic statistics on but hardly enough to base robust 'null' conclusions.
  
  Failing to find any interesting answers in the data itself, the authors asked the question somewhat sarcastically, if COSO is so good why don’t more firms use it? They then conclude, without supporting data, that COSO has somehow failed.
  
  First, the original questionnaire asked does the firm use COSO or not (or at least that is how the data is represented), but not whether other standards were used, such as the FERMA standard? So the authors are comparing apples with potatoes and concluding that apples don’t make good potato chips.
  
  COSO is an American not a European standard and the same result probably would have applied, for example, to Australia which standardizes on its local AS/NZS 4360. A pretty dumb conclusion is made that COSO (not the other 80% of non-COSO firms) isn’t working [by the way it might not be but this study doesn’t demonstrate one way or the other]. There is no comparison of COSO versus something else, internal or external. The question should be whether one standard is better than another and this study does not have the data to address this critical question. An even better question is whether any standard is better than none!
  
  Last the conclusion on risk appetite/tolerance is somewhat juvenile; i.e. since most firms don’t have a risk appetite or tolerance ergo they are not needed! This is akin to saying most firms don’t do risk management well therefore it is not needed – the GFC certainly proved that observation wrong!
  
  Research into how firms actually do risk management is very important, but, unfortunately, general questionnaires, such as this data set, give numbers but few insights. To their credit the authors make this point and argue for further research but unfortunately would be better placed if they had left their musings to the construction of their next hypotheses."
• 

  Norman David Marks June 21, 2011 16:37

  Happy to chat - offline
• 

  Leen Paape June 21, 2011 16:35

  Norman, any ideas to further our research are very much welcomed
• 

  Norman David Marks June 21, 2011 16:25

  Len, I am disappointed - but by the work done, not by the task itself. COSO commissioned a study on the effectiveness of its framework, but it was not independent. I believe an independent study that includes the right questions and interviews of executives and practitioners on the effectiveness of the framework can be done and can show supportable results.
• 

  Leen Paape June 21, 2011 16:20

  thanks Norman, alas for ISO the same is true (no support that it helps), the problem is that there will never be proof of the kind you are asking for because the model is too complicated and we can't do the same as the farma industry is doing (testing three groups, one using a pill, one using a placebo and one using nothing at all). Sorry to have disappointed you but the good news is that if we are not able to support or decline the claims of models like COSO we have to just do our utmost. There is some clue though where researchers said that those who had an ERM system available (using the Standard&Poor's review system) are more succesfull than those who don't.
• 

  Norman David Marks June 21, 2011 16:11

  Len, I prefer ISO to COSO (preferred the ANZ standard more) - but that's not relevant. The issue is whether the opinion of the authors is supported by the results as presented. What I saw is evidence that ERM is not practiced as often or as well as we would prefer. But that is not demonstrated as a failing of COSO. If the authors had shown that those who attempted to follow COSO failed, or at least reported difficulties, that might be persuasive. But all I read was an unsupported opinion.
  
  The authors are trying to support a contention that relates to practice - the effectiveness of the COSO ERM Framework. That contention has to be supported by facts that relate to practice - don't you agree?
• 

  Leen Paape June 21, 2011 15:59

  I've been a practitioner for about 27 years and an academic for the last 4 years. The problem is that the gap between 'theory'  and 'practice'  is big and it ain't easy to bring the two fo them together. The issue is that many practitioners tend to use 'best practices' without any proper 'proof' whether they are valid. COSO as a model has no theory to build on let alone that there is any proof that it's a useful model that really works. Any research that might shed light on this issue should be welcomed. It's not my statement that the use of COSO doesn't help to improve risk management at all, it is the statement made by the respondents to our survey. I'm very curious to hear your views regarding the applicability of COSO as a model.