Is COSO helpful? Research shows there is doubt

The much acclaimed COSO model is mostly referred to when discussing and implementing risk management. Research shows there is at least some doubt whether COSO is really helpful. Please, take a look at the paper attached and let me know your comments.

 

 

ERM paper (revised -june 2011).pdf

You need to be a member of Global Risk Community to add comments!

Join Global Risk Community

Votes: 0
Email me when people reply –

Replies

  • If anyone is interested, the final version of this paper is now on SSRN and will be published in the European Accounting Review (expected on-line availability end of February, 2012).

  • Maybe this is a little tongue in cheek but my practical experience in enterprise risk management taught me two valuable lessons:

    1. I achieved the best result through co-operation and honesty from the key role-players.  This meant that I needed to take time out to engage with employees and leaders.  Only once I was comfortable in understanding the culture of the organisation and where the pockets of institutional knowledge lies; only then did I map out an organisation specific ERM maturity model using elements of models as COSO and ISO.
    2. Don’t expect 100% commitment – we are not in an ideal world; reach maximum consensus with the most influential institutional stalwarts’ behind you , that is enough to get reliable data with high enough integrity to support informed decision making.

    In my view is that risk management is  much like religion – don’t get stuck and fanatical on any one – learnt the pros and cons of each and take the best bit of them all; then exercise good judgement on when to apply what – don’t forget that the human factor is bigger and more powerful than any system, methodology or model; Yet is often over seen as the number one risk factor to overcome to achieve the rest.



  • Patrick McConnell said:

    @Patrick, Thank you for your thoughtful analysis. It would be interesting to follow your debate with the authors if you could re-post your analysis on GlobalRisk Community.

     

    Pleased to copy from Linkedin Behavioural Finance Theory and Practice and to engage in debate

     

    "This study is pretty lightweight, and would only normally be worth placing in the 'another brick in the wall' pile, except that some of its conclusions feel like they might become an 'urban myth'.

    First, the authors point out that the data they analyze is not theirs, and are in fact quite scathing about its data collection - "no explicit attention to scholarly scale construction, validation and measurement considerations ... [and] naive". Nonetheless, this does not stop the authors drawing a very long bow from some very inappropriate data.

    The basic conclusion of the study that risk management is not well advanced in the Netherlands is interesting, but no different to elsewhere (nevertheless a useful finding). They also find that when ERM is widely implemented (as opposed to lip service) it is perceived to be effective and in particular, "the frequency of risk assessment, the frequency of risk reporting, and the richness of risk reporting contribute to perceived risk management effectiveness". [Remembering always that this is based on subjective assessment to a general questionnaire].

    The authors also raise a very valid question: is ERM the same for private and public institutions? Good question, but not answered here.

    If left at those observations, the paper would have been interesting but hardly controversial. But bereft of new 'positive' findings, the authors move into the murkier world of 'null result or 'negative' implications, i.e. conclusions based on questions we didn’t ask!

    The original questionnaire asked whether the firm used COSO; some 21% of the sample claimed to use COSO, at least in part, or some 2% of the questionnaires sent out. This is not an unreasonable number however to do some basic statistics on but hardly enough to base robust 'null' conclusions.

    Failing to find any interesting answers in the data itself, the authors asked the question somewhat sarcastically, if COSO is so good why don’t more firms use it? They then conclude, without supporting data, that COSO has somehow failed.

    First, the original questionnaire asked does the firm use COSO or not (or at least that is how the data is represented), but not whether other standards were used, such as the FERMA standard? So the authors are comparing apples with potatoes and concluding that apples don’t make good potato chips.

    COSO is an American not a European standard and the same result probably would have applied, for example, to Australia which standardizes on its local AS/NZS 4360. A pretty dumb conclusion is made that COSO (not the other 80% of non-COSO firms) isn’t working [by the way it might not be but this study doesn’t demonstrate one way or the other]. There is no comparison of COSO versus something else, internal or external. The question should be whether one standard is better than another and this study does not have the data to address this critical question. An even better question is whether any standard is better than none!

    Last the conclusion on risk appetite/tolerance is somewhat juvenile; i.e. since most firms don’t have a risk appetite or tolerance ergo they are not needed! This is akin to saying most firms don’t do risk management well therefore it is not needed – the GFC certainly proved that observation wrong!

    Research into how firms actually do risk management is very important, but, unfortunately, general questionnaires, such as this data set, give numbers but few insights. To their credit the authors make this point and argue for further research but unfortunately would be better placed if they had left their musings to the construction of their next hypotheses."

  • @Patrick, Thank you for your thoughtful analysis. It would be interesting to follow your debate with the authors if you could re-post your analysis on GlobalRisk Community.

     

    Pleased to copy from Linkedin Behavioural Finance Theory and Practice and to engage in debate

     

    "This study is pretty lightweight, and would only normally be worth placing in the 'another brick in the wall' pile, except that some of its conclusions feel like they might become an 'urban myth'.

    First, the authors point out that the data they analyze is not theirs, and are in fact quite scathing about its data collection - "no explicit attention to scholarly scale construction, validation and measurement considerations ... [and] naive". Nonetheless, this does not stop the authors drawing a very long bow from some very inappropriate data.

    The basic conclusion of the study that risk management is not well advanced in the Netherlands is interesting, but no different to elsewhere (nevertheless a useful finding). They also find that when ERM is widely implemented (as opposed to lip service) it is perceived to be effective and in particular, "the frequency of risk assessment, the frequency of risk reporting, and the richness of risk reporting contribute to perceived risk management effectiveness". [Remembering always that this is based on subjective assessment to a general questionnaire].

    The authors also raise a very valid question: is ERM the same for private and public institutions? Good question, but not answered here.

    If left at those observations, the paper would have been interesting but hardly controversial. But bereft of new 'positive' findings, the authors move into the murkier world of 'null result or 'negative' implications, i.e. conclusions based on questions we didn’t ask!

    The original questionnaire asked whether the firm used COSO; some 21% of the sample claimed to use COSO, at least in part, or some 2% of the questionnaires sent out. This is not an unreasonable number however to do some basic statistics on but hardly enough to base robust 'null' conclusions.

    Failing to find any interesting answers in the data itself, the authors asked the question somewhat sarcastically, if COSO is so good why don’t more firms use it? They then conclude, without supporting data, that COSO has somehow failed.

    First, the original questionnaire asked does the firm use COSO or not (or at least that is how the data is represented), but not whether other standards were used, such as the FERMA standard? So the authors are comparing apples with potatoes and concluding that apples don’t make good potato chips.

    COSO is an American not a European standard and the same result probably would have applied, for example, to Australia which standardizes on its local AS/NZS 4360. A pretty dumb conclusion is made that COSO (not the other 80% of non-COSO firms) isn’t working [by the way it might not be but this study doesn’t demonstrate one way or the other]. There is no comparison of COSO versus something else, internal or external. The question should be whether one standard is better than another and this study does not have the data to address this critical question. An even better question is whether any standard is better than none!

    Last the conclusion on risk appetite/tolerance is somewhat juvenile; i.e. since most firms don’t have a risk appetite or tolerance ergo they are not needed! This is akin to saying most firms don’t do risk management well therefore it is not needed – the GFC certainly proved that observation wrong!

    Research into how firms actually do risk management is very important, but, unfortunately, general questionnaires, such as this data set, give numbers but few insights. To their credit the authors make this point and argue for further research but unfortunately would be better placed if they had left their musings to the construction of their next hypotheses."

  • Happy to chat - offline
  • Norman, any ideas to further our research are very much welcomed

  • Len, I am disappointed - but by the work done, not by the task itself. COSO commissioned a study on the effectiveness of its framework, but it was not independent. I believe an independent study that includes the right questions and interviews of executives and practitioners on the effectiveness of the framework can be done and can show supportable results.
  • thanks Norman, alas for ISO the same is true (no support that it helps), the problem is that there will never be proof of the kind you are asking for because the model is too complicated and we can't do the same as the farma industry is doing (testing three groups, one using a pill, one using a placebo and one using nothing at all). Sorry to have disappointed you but the good news is that if we are not able to support or decline the claims of models like COSO we have to just do our utmost. There is some clue though where researchers said that those who had an ERM system available (using the Standard&Poor's review system) are more succesfull than those who don't.
  • Len, I prefer ISO to COSO (preferred the ANZ standard more) - but that's not relevant. The issue is whether the opinion of the authors is supported by the results as presented. What I saw is evidence that ERM is not practiced as often or as well as we would prefer. But that is not demonstrated as a failing of COSO. If the authors had shown that those who attempted to follow COSO failed, or at least reported difficulties, that might be persuasive. But all I read was an unsupported opinion.

    The authors are trying to support a contention that relates to practice - the effectiveness of the COSO ERM Framework. That contention has to be supported by facts that relate to practice - don't you agree?
  • I've been a practitioner for about 27 years and an academic for the last 4 years. The problem is that the gap between 'theory'  and 'practice'  is big and it ain't easy to bring the two fo them together. The issue is that many practitioners tend to use 'best practices' without any proper 'proof' whether they are valid. COSO as a model has no theory to build on let alone that there is any proof that it's a useful model that really works. Any research that might shed light on this issue should be welcomed. It's not my statement that the use of COSO doesn't help to improve risk management at all, it is the statement made by the respondents to our survey. I'm very curious to hear your views regarding the applicability of COSO as a model.
This reply was deleted.

[Free COVID-19 Framework] What's the path to recovery look like?

We created a free presentation (attached), which discusses both global and organizational impacts of the COVID-19 pandemic, along with critical actions organizations should take immediately. This presentation introduces a framework that helps regions and organizations navigate a path to recovery via 9 potential scenarios. These scenarios capture outcomes related to GDP impact, public health response, and economic policies. The presentation also breaks down 6 immediate and critical actions…

Read more…
4 Replies · Reply by Steve Diaz Jul 8, 2023
Views: 204

If risk management is about decision making, are current risk management solutions irrelevant?

Now that the updated COSO and ISO risk management standards emphasize a connection to enterprise objectives and decision making, does this mean ERM and GRC solutions focused on risk registers and regulatory compliance are missing the true value of risk management?Will current risk management solutions evolve to integrate more decision support functionality or will standalone prescriptive analytics and other technology solutions take a more prominent role in enabling risk-informed…

Read more…
3 Replies
Views: 129

A question related to classification of instruments between trading and banking book.

We have an interesting question from one of our members.       "We usually perform OTC FX transactions with clients backed-to-back on the market (with Banks). Now we are going to perform a FX swap (i.e. Spot + forward) JPY/EUR for the Bank account for 1 week at the longest. The purpose is to get EUR place @ CB for LCR compliance purpose (no trading purposes). Bank's Management think that this should be considered as a trading position and therefore be classified within the Bank's trading book.…

Read more…
5 Replies · Reply by Prisha Singh Dec 26, 2023
Views: 309

Plunging oil prices: curse or blessing in disguise?

The recent sudden crash of oil prices has had a major impact on the world economy, leading to many troubled faces in the international arena. The Russians fear the effects of yet another powerful hit on their economy, Venezuela seems to be considering default and the Americans are weary of the consequences for its young and emerging shale oil industry. And then you have the Middle East, where the smallest match is enough to ignite the largest fire. But are these worries really justified or…

Read more…
1 Reply
Views: 90

Introducing the Global Risk Series - Book 1 Risk Management How Tos

Dear GlobalRisk Community member, Our community’s mission is to foster business, networking and educational explorations among members. Learn from some of the top experts in the industry as they clearly explain how to approach the most important Risk management concepts. Check out their expert tips and use the link at the end of each article to navigate back to the website to leave your comment or ask a question.   Some of the topics include: How do you Explain Risk Appetite?  How to Prepare a…

Read more…
16 Replies · Reply by GlobalRiskCommunity Mar 21
Views: 888

    About Us

    The GlobalRisk Community is a thriving community of risk managers and associated service providers. Our purpose is to foster business, networking and educational explorations among members. Our goal is to be the worlds premier Risk forum and contribute to better understanding of the complex world of risk.

    Business Partners

    For companies wanting to create a greater visibility for their products and services among their prospects in the Risk market: Send your business partnership request by filling in the form here!

lead