Home  // . //  //  How Risk Management In Commodity Trading Is Transforming

23:35

Double Quotes
A client called us, concerned after making $80 million from a hedge program. They figured if they could accidentally earn $80 million, they could just as easily lose $80 or $100 million

In recent years, weather-related and geopolitical events have resulted in an emergence of certain risk types that trading organizations may not have been expecting to see at the scale. As businesses ramp up their trading operations, greater activity brings a greater need to understand risk. In this episode of “The Trading Desk,” our experts explore the spectrum of risk, whether it’s being managed or monetized, and misconceptions corporates might have around trading.

Interested in a particular topic? Skip to the following sections:

1:10 — Black swan events and risk management

3:54 — Understanding liquidity risk management

8:04 — Risk as a supervisory entitity

10:00 — Automoted risk portfolios and frameworks

13:47 — Optimizing data infrastructure and architecture

15:16 — Understanding systems, process, and people to work together

17:02 — Trading organizations versus corporate functions

20:17 — Talent sourcing and retention in trading

 

Joanne Salih 

Hi, I'm Joanne Salih, a partner in our Energy and Natural Resources Practice, here at Oliver Wyman, and head of America's Risk and Trading. Today I'm joined by Alex Franke, global head of our Risk and Trading Practice, and a partner in our Energy and Natural Resources Practice. Mike Burger, who is a VP and co-founder of our Veritas business, a specialized risk and trading team within Oliver Wyman. And Marc Zimmerman, a partner in our risk and trading team in Europe, and also a partner in our  Energy Natural Resources Practice.

Today we'll be talking about risk management: manage or monetize? That's the question. Part of our four-part series on risk and trading. The last few years have been certainly kind of extra normal profits, but a lot of volatility, a lot of risk that I think most of our clients in the wider trading environment we're not expecting as with any black swan event and two by nature, which were quite different. What I think was quite interesting though is it highlighted risk that people really haven't been watching out for or quantifying to the extent that they would now be thinking about and certainly some of their limits requirements around liquidity became fundamentally different. So very different environment that we've been playing in and I think hopefully is a bit of a leap forward when it comes to risk management.

Alex Franke

Yeah, I totally agree. Absolutely great opportunity on the commercial side, but also a much higher risk environment and we didn't see the interplay, especially between market and credit risk that we haven't really seen before, not to that degree. So it was really important to focus on the quality of risk management, the understanding the trade-offs between the different risk types and then also steering, the risk taking towards the areas that you would get compensated for to actually capture the opportunity that was available in the market.

Joanne

Maybe on your first point Alex, you're talking about transfer of risk between different risk types.

Alex

The trade off between market and credit risk and to a certain degree as well, credit risk that we see emerge more lately and I think that understanding on how risk materializes between these different risk types, I think that is really important.

Mike Burger

Yeah, I agree. We see clients ignore one or two of these sometimes and they don't realize the interdependence between them, and so that's why it's important to have a comprehensive approach to this and model them in a consistent way and maybe augment that with scenario analysis so that you really can see how they interact.

Alex

The modeling is important and then also how you steer on it. I think the holistic steering is important. If I remember in more than one case there was a focus on market and liquidity risk and also charging for the risks that are being taken. If you disregard the credit dimension of it, what basically happens is that your traders will start to seek out the worst counterparties that are available because this is where you get the higher prices. If you don't price for the credit risk, there is potentially unintended consequences in the back of this.

Marc Zimmerlin

And liquidity risk being actually one of the other aspects that people did not really care about enough, I would say before 2022. So a lot of players actually got hit more or less on surprise not being able to properly respond and also comprehensively what was going on in the exchange positions. I mean in the past they had in some cases try to predominantly OTC and bilateral, they really done the exchange based trading or clearing on the side and now the exchanges and the clearers asking for excessive margin, excessive initial end variation margin in 2022 and some of that carried into 2023 has really changed the way that our clients look at liquidity management, but also liquidity risk management as an important aspect. And I think this has brought a lot of the, let's say more natural finance domain topics as well into trading related to treasury related to capital and capital return on capital even.

Marc

And for a lot of players, and Alex mentioned it, I mean there was a lot of commercial opportunity, but in 22 for a lot of players actually they couldn't capture it because they got stopped out on their risk controls and the risk systems because they weren't designed essentially to make those trade-offs visible and to really give the traders or whoever was taking the commercial decisions, a bit of a framework, they allow to actually weigh some of the decisions against each other. So actually a lot of market participants actually got stopped out in the market in 22.

As Alex mentioned it, there was a lot of commercial opportunity in the market, but if you would be executing on a risk control framework, which was very prohibitive in taking some credit risk maybe on parameters which didn't really hold in 2022 market circumstances or liquidity side, I mean people got scared quite frankly by some of the large numbers they've been seeing. And then the commercial side of the business, may that be the traders or maybe some of the management, they were actually stopping out capturing some of that commercial opportunity and some others in the market. They were more or less blindly executing mechanically on some of the hedging strategies and you got the worst of both worlds I would say, in not applying a risk management mindset, which was more of an maybe educated or advanced mindset to the setting.

Alex  

I think this is where we really saw a divide in the performance that players were able to capture as well. There were the ones that had established risk management frameworks, they were confident in the way that they were quantifying and controlling the risks and also managing the trade-offs on a quantitative basis. And I think this is where then the additional capability needed to come in on connecting the dots to the finance and the treasury topics that were emerging and those were the players that were able to stay in the market for longer and capture more of the opportunity that was available.

I think this is also where you have the basic requirements of risk control that need to be met, so that gives you the confidence to actually stay in the market if it gets rough. And this is where the risk advisory dimension comes in where you have a challenging discussion between risk and the front office on whether you're taking the right risks, whether you're adequately being compensated for the risks that are being taken and how the trade-off between market credit and liquidity risk actually works both from a steering perspective but also from an advisory, from a sparing discussion perspective.

Marc

I would actually even add that that risk advisory even expands beyond the commercial operations, let's say in trading also to management. You need to take a step back and I think what some of those maybe frameworks can deliver on as the unknowns or known unknowns and really being able to then flexibly respond. I mean there was a client, for instance with us when we worked with them in 2022, they really had to rethink credit risk in a way that you would be allowing maybe in terms of absolute numbers, a lot more credit risk towards counterparties. You typically would not, I mean example being an upstream producer essentially making a lot of money out of the high price environment.

Does the credit risk and the default risk, which is associated with that really represent what historically has been seen for that particular rating class and that circumstance> I mean that was a question essentially being asked, and if you stuck to your risk controls as they were defined pre-crisis and just applied, let's say a rating, agency rating to that counterpart and the corresponding limit, you'll be stopped out. And if you have the advisory mindset, you can reflect on the situation, you can reassess the PD in this instance, or you can bend let's say the control framework a little bit to make it work.

Joanne

I mean, I think that's the key point and what we still see playing out. Now, as you mentioned Marc earlier, I mean there are those that kind of learn their lessons and I think a lot of those European players who maybe saw the most extreme version of this all playing out. But then there are others who I think are still grappling with this question and fundamentally risk at best is a supervisory entity, and having real advisory taking place like capabilities, and frankly the systems underlying it becomes pretty fundamental, but it's not something you can get to quickly. I think.

Mike

Alex and Marc have been talking about have been broken all over, not just in Europe, and it feels like we've been having a hundred-year weather or financial event every year. We had a client who called us, concerned after making $80 million from a hedge program. They figured if they could accidentally earn $80 million, they could just as easily lose $80 or $100 million. So, we helped them kind of reassess their hedging program, but it's an example where a firm had very good risk controls in place, I think decent ones, but no real concept of risk management or monetizing their assets.

Joanne

Yeah, I think moving towards monetization is a completely different ball game, and I personally see a lot of control, but control from the perspective of we have limits, we have compliance activities, we are managing within a corporate view of what we believe trading should do and shouldn't do, not really assessing that against value. So what is the potential from the perspective using, you mentioned earlier in such as you Alex, from a credit risk perspective, how do we optimize credit risk? What do we do to create value from mark out to policies?

Alex

So I think the core of the control process is running a daily routine and discipline around creating a view of where the risks are in the portfolio. I think this is something that should be largely automated, so that you create the capacity for risk team to also think about the risks that aren't covered in the current framework and to look beyond, to look into what can happen in terms of stress tests, where are unexpected events going to come from to challenge traders. All of that takes share of mind and you need a bit of extra capacity to deal with all of that. And I think having efficient processes and system support is one way to create the capacity.

Marc

And I mean there's comfort, right? In just executing policy and following the rules and having a clear frame on controls. But as Alex mentioned, I mean you need to free up that time. Also, there's a talent component to that because if you don't have that freed up time, a lot of people would not like to fill their workday or the entire workday at least with very repetitive tasks, essentially just producing the same set of numbers every day. I mean, I did the job myself and having that spare capacity and I had it luckily at an investment bank gives you the time to really think through stuff, right? It is also making the job itself much more attractive because not only the engagement with a front around certain topics, the engagement may be with management on risk management topics, but just having the intellectual, let's say challenge, is quite a distinctly different thing and just walking up to the job every day and doing the same thing over and over again.

Mike

Right? There's a very real difference between risk control and real risk management or risk advisory. And so control is about limiting stopping or making sure that repetitive activities, as you say, are performed every day. And risk management or risk advisory and optimization is all about getting the most commercial value out of every unit of risk taken. When we do diagnostics for firms on risk management practices will look for constructive conversations between the front and middle office. Discussion of risk efficiency and risk-based performance and things like this, if it's just controls, they're really not getting everything out of their team that they can.

Joanne

I think in addition to that, there's also the element of the level of administrative activity that's actually taking place. So in some of what we would consider to be very mature energy companies, you still have systems that basically are not fit for purpose and make sure that risk folks who are highly confident and can provide advisory are actually figuring out how to do an end of day report and piecing together information, right? So it is kind of like let's get at least the knitting of this thing set up properly so that you do have a lot more free time to go out and provide the advisory that's required, fundamentally required by the front office.

Mike

It's interesting about risk systems in many of these firms because they've grown grownup piecemeal, they're all separate, they're not really consistent with one another. And so, we think that it's best to have a common framework and model something like a Monte Carlo that allows you to look at market risk, credit risk liquidity risk together scenarios, augmented by scenarios that allow you to look at some of these black swan events that we've been experiencing and really understand the effect of a five or 10 standard deviation move. This is just not really in place for most firms. So one thing that's made this a lot easier is the current emphasis on data architecture and data strategy. And so the widespread adoption of the cloud has allowed firms to focus more on what we call data-centric architecture as opposed to the ETRM system or systems being the center of the universe. And this allows these risk systems to be developed as point solutions around the edge of a good data repository and the models to be managed in a much more standardized way.

Marc

I mean, I have a client that actually holds for a lot of clients that are currently facing a lot of, let's say, paradigm shifts in the way that their portfolios functions and the way that they need to think about risk. So two of the key things I guess maybe is there's a lot more heterogeneity in what they're trading, may that be geographically or maybe commodity classes. Second thing is there's also expanding risk and trading activity into long data illiquid risks. Is that something that also links to your point around the data and maybe something that can enable it or some of the approaches that you mentioned?

Mike

Yes, because instead of doing that all in one solution, you now have the ability to develop specialized solutions for markets and have them integrated with the same data architecture that you're using for your primary base products.

Joanne

So is this a question of systems, process, people? What is it or is it everything?

Alex  

I think you need both to come together. On the one hand, having the right talent in place and the capabilities, the experience that is important because you want people, like we said before, to look past the framework as well, but then you also need the lean processes to generate the efficiency in the day-to-day and to create the space to also think about what you don't cover in your daily routine. And all of that obviously needs to be underpinned by efficient systems.

Mike

I agree. Technology is the tail wagging the dog. You need to make sure that it directly supports commercial strategy and risk management objectives.

Marc

And I think what we're also seeing a lot what doesn't work is essentially go to the banking space and try to straight export into physical commodity trading. So there always needs to be a bit of a, let's say adaption or at least some sort of link to the fundamentals around the physical side of the business, or I'm giving one example. I mean, I guess a lot of our clients struggling, especially with liquidity risk management and credit risk management. If you go for a bank, I mean there's a credit book that allows you to diversify some of those risks and hedge them, they can hedge the interest rate curves, they can hedge the credit risk to some extent against their own credit books and diversify it. That fundamental is not given in at least a regular more regular commodity trading environment, let's say at an independent trader or captive trader. So that level of adaptation and also being able to essentially work with a risk framework, still capturing some of these risks and so looking after them and monetizing them even, but not necessarily going straight to the banking world to look for let's say best in class or however you want to label types of approaches.

Joanne

I think on the other scale, the other challenge is it's good and great to understand the role of risk and risk management in general within trading. It's often within integrated companies, it's very difficult to have this conversation between the trading organization and the corporate function. So with finance and treasury, because in some ways, whilst they shouldn't be competing objectives, it does feel at times that these two different sets of stakeholders feel they are. So on the one side you have traders talking about optimizing margin and the positions they need to take versus organizational view, which is, well, we want stability in our free cash flow, we don't want to get into markets that seem too risky for us. What does this mean in terms of our ability to access that asset or origination activities? So I think there is a bit of a gap between those two sets of stakeholders that does also need to be managed.

Marc

It's a gap, but I would also argue it's different clock speeds. So group functions, they typically work on yearly cycles. So there's a budget cycle, there's a strategic exercise that's typical be done once a year. The group treasurer would want to go out to the market, maybe to place a bond or something that takes a lot of lead time. And then if you hit that system, something that is as dynamic as trading can be, or even more supercharged like a 2022, where then even the clock speed of trading is increasing to intraday, let's say volatility we've not seen before. Then quite naturally these connects break. And then you need to rethink what is the data, what is the transparency you need to give on your trading business and the risk within? And also to the advisory point, how much do you maybe need to educate management or group functions in order to comprehend what the risks are that the business is carrying and should be carrying as part of their core mandate?

Mike

Another dimension of this is that physical shops don't you know, by history or legacy, really have the same view of credit risk that financial shops do. And so, in big physical trading centers like Houston, many are just managing AR, that's credit risk. And so if you sell index cargos, all you're really worried about are two or three invoices that aren't collected or product that's been delivered and not invoiced. And so what's interesting is as these firms transform into more active trading shops, they'll have a larger and larger portion of their counterparty risk be composed to mark to market. And so they'll be using VAR to manage price risk or sophisticated price risk models, but on the credit side, they're really just using AR and they don't understand how important mark to market has become in their business. And they're not stochastically testing changes in mark to market using PFE or C-bar, tools like that.

Joanne

I mean on that, if you were in the position then of a CRO whose dealing with all of this complexity, market complexity, the growth or evolution of your own business, systems questions, engagement with the corporate, if I could ask you a very tough question, what is the most important thing that you think they need to be thinking about? What would that be?

Alex

On risk management? I think it's about the proposition for the talent. You need to be able to attract and retain high-caliber people and you need to be able to show them how they can progress in their careers. Only then do you get people that also front office will be able to invest in because they will see the value in a discussion and into sparing and advisory. And that's really what makes that discussion a two-way street.

Joanne

Yeah, so a trading career track, a true one, not hopping in and out trading every couple of years.

Marc

And I mean we have the beauty of having being asked this type of question on a more or less weekly basis. And I guess what we are seeing at the moment is to Alex's point around talent, but also around the topic itself, elevating it to the top and giving it a purpose on the top is also quite powerful. So one example being, for instance, we're doing a lot of work currently around bringing risk into the fund financial planning and portfolio planning domains.

So, essentially allowing companies who become more and more heterogeneous to take decisions on investment and performance management, which are risk adjusted. What's the value there? I mean, the value clearly is you can see correlation, you can see how you can potentially also hedge out some of those very high uncertainties that we're currently seeing out of the regulatory environment, out of the price environment, out of maybe some of the more geopolitical environment that could then affect your investments, that particularly when it comes to green investments, can have a long lead time, can have a lot of risk around future cash flows. And elevating that role of risk and elevating that conversation into some of the core management steering processes and even in some cases, bringing that conversation to depth and shareholders as well, is quite powerful. And that in return then empowers as well the risk organization to take a stand and to have a seat essentially at the table.

Mike

So my suggestion might sound a little inconsistent with something we discussed earlier that technology shouldn't be the tail wagging the dog, but I think CROs should throw all their weight behind data strategy and data architecture projects now because we see so many places around the globe, where companies are in the middle of big ETRM programs or other transformation projects, and their data strategy is way behind the times. So they're going to reproduce dozens or hundreds of point-to-point interfaces with old technology. They're not getting advantage of this data-centric architecture that we talked about earlier, and it's a missed opportunity. So I think CRO is a really important stakeholder in these big IT programs, and that's what I think they should really emphasize. It promotes better data analytics, better risk measurement. It's very important.

Joanne

You can't manage where you can't measure, you can't optimize, where you can't measure. So, rings true. Well guys, thank you so much for your time. Fascinating discussion that I hope we continue as we move forward.

This transcript was edited for clarity

    In recent years, weather-related and geopolitical events have resulted in an emergence of certain risk types that trading organizations may not have been expecting to see at the scale. As businesses ramp up their trading operations, greater activity brings a greater need to understand risk. In this episode of “The Trading Desk,” our experts explore the spectrum of risk, whether it’s being managed or monetized, and misconceptions corporates might have around trading.

    Interested in a particular topic? Skip to the following sections:

    1:10 — Black swan events and risk management

    3:54 — Understanding liquidity risk management

    8:04 — Risk as a supervisory entitity

    10:00 — Automoted risk portfolios and frameworks

    13:47 — Optimizing data infrastructure and architecture

    15:16 — Understanding systems, process, and people to work together

    17:02 — Trading organizations versus corporate functions

    20:17 — Talent sourcing and retention in trading

     

    Joanne Salih 

    Hi, I'm Joanne Salih, a partner in our Energy and Natural Resources Practice, here at Oliver Wyman, and head of America's Risk and Trading. Today I'm joined by Alex Franke, global head of our Risk and Trading Practice, and a partner in our Energy and Natural Resources Practice. Mike Burger, who is a VP and co-founder of our Veritas business, a specialized risk and trading team within Oliver Wyman. And Marc Zimmerman, a partner in our risk and trading team in Europe, and also a partner in our  Energy Natural Resources Practice.

    Today we'll be talking about risk management: manage or monetize? That's the question. Part of our four-part series on risk and trading. The last few years have been certainly kind of extra normal profits, but a lot of volatility, a lot of risk that I think most of our clients in the wider trading environment we're not expecting as with any black swan event and two by nature, which were quite different. What I think was quite interesting though is it highlighted risk that people really haven't been watching out for or quantifying to the extent that they would now be thinking about and certainly some of their limits requirements around liquidity became fundamentally different. So very different environment that we've been playing in and I think hopefully is a bit of a leap forward when it comes to risk management.

    Alex Franke

    Yeah, I totally agree. Absolutely great opportunity on the commercial side, but also a much higher risk environment and we didn't see the interplay, especially between market and credit risk that we haven't really seen before, not to that degree. So it was really important to focus on the quality of risk management, the understanding the trade-offs between the different risk types and then also steering, the risk taking towards the areas that you would get compensated for to actually capture the opportunity that was available in the market.

    Joanne

    Maybe on your first point Alex, you're talking about transfer of risk between different risk types.

    Alex

    The trade off between market and credit risk and to a certain degree as well, credit risk that we see emerge more lately and I think that understanding on how risk materializes between these different risk types, I think that is really important.

    Mike Burger

    Yeah, I agree. We see clients ignore one or two of these sometimes and they don't realize the interdependence between them, and so that's why it's important to have a comprehensive approach to this and model them in a consistent way and maybe augment that with scenario analysis so that you really can see how they interact.

    Alex

    The modeling is important and then also how you steer on it. I think the holistic steering is important. If I remember in more than one case there was a focus on market and liquidity risk and also charging for the risks that are being taken. If you disregard the credit dimension of it, what basically happens is that your traders will start to seek out the worst counterparties that are available because this is where you get the higher prices. If you don't price for the credit risk, there is potentially unintended consequences in the back of this.

    Marc Zimmerlin

    And liquidity risk being actually one of the other aspects that people did not really care about enough, I would say before 2022. So a lot of players actually got hit more or less on surprise not being able to properly respond and also comprehensively what was going on in the exchange positions. I mean in the past they had in some cases try to predominantly OTC and bilateral, they really done the exchange based trading or clearing on the side and now the exchanges and the clearers asking for excessive margin, excessive initial end variation margin in 2022 and some of that carried into 2023 has really changed the way that our clients look at liquidity management, but also liquidity risk management as an important aspect. And I think this has brought a lot of the, let's say more natural finance domain topics as well into trading related to treasury related to capital and capital return on capital even.

    Marc

    And for a lot of players, and Alex mentioned it, I mean there was a lot of commercial opportunity, but in 22 for a lot of players actually they couldn't capture it because they got stopped out on their risk controls and the risk systems because they weren't designed essentially to make those trade-offs visible and to really give the traders or whoever was taking the commercial decisions, a bit of a framework, they allow to actually weigh some of the decisions against each other. So actually a lot of market participants actually got stopped out in the market in 22.

    As Alex mentioned it, there was a lot of commercial opportunity in the market, but if you would be executing on a risk control framework, which was very prohibitive in taking some credit risk maybe on parameters which didn't really hold in 2022 market circumstances or liquidity side, I mean people got scared quite frankly by some of the large numbers they've been seeing. And then the commercial side of the business, may that be the traders or maybe some of the management, they were actually stopping out capturing some of that commercial opportunity and some others in the market. They were more or less blindly executing mechanically on some of the hedging strategies and you got the worst of both worlds I would say, in not applying a risk management mindset, which was more of an maybe educated or advanced mindset to the setting.

    Alex  

    I think this is where we really saw a divide in the performance that players were able to capture as well. There were the ones that had established risk management frameworks, they were confident in the way that they were quantifying and controlling the risks and also managing the trade-offs on a quantitative basis. And I think this is where then the additional capability needed to come in on connecting the dots to the finance and the treasury topics that were emerging and those were the players that were able to stay in the market for longer and capture more of the opportunity that was available.

    I think this is also where you have the basic requirements of risk control that need to be met, so that gives you the confidence to actually stay in the market if it gets rough. And this is where the risk advisory dimension comes in where you have a challenging discussion between risk and the front office on whether you're taking the right risks, whether you're adequately being compensated for the risks that are being taken and how the trade-off between market credit and liquidity risk actually works both from a steering perspective but also from an advisory, from a sparing discussion perspective.

    Marc

    I would actually even add that that risk advisory even expands beyond the commercial operations, let's say in trading also to management. You need to take a step back and I think what some of those maybe frameworks can deliver on as the unknowns or known unknowns and really being able to then flexibly respond. I mean there was a client, for instance with us when we worked with them in 2022, they really had to rethink credit risk in a way that you would be allowing maybe in terms of absolute numbers, a lot more credit risk towards counterparties. You typically would not, I mean example being an upstream producer essentially making a lot of money out of the high price environment.

    Does the credit risk and the default risk, which is associated with that really represent what historically has been seen for that particular rating class and that circumstance> I mean that was a question essentially being asked, and if you stuck to your risk controls as they were defined pre-crisis and just applied, let's say a rating, agency rating to that counterpart and the corresponding limit, you'll be stopped out. And if you have the advisory mindset, you can reflect on the situation, you can reassess the PD in this instance, or you can bend let's say the control framework a little bit to make it work.

    Joanne

    I mean, I think that's the key point and what we still see playing out. Now, as you mentioned Marc earlier, I mean there are those that kind of learn their lessons and I think a lot of those European players who maybe saw the most extreme version of this all playing out. But then there are others who I think are still grappling with this question and fundamentally risk at best is a supervisory entity, and having real advisory taking place like capabilities, and frankly the systems underlying it becomes pretty fundamental, but it's not something you can get to quickly. I think.

    Mike

    Alex and Marc have been talking about have been broken all over, not just in Europe, and it feels like we've been having a hundred-year weather or financial event every year. We had a client who called us, concerned after making $80 million from a hedge program. They figured if they could accidentally earn $80 million, they could just as easily lose $80 or $100 million. So, we helped them kind of reassess their hedging program, but it's an example where a firm had very good risk controls in place, I think decent ones, but no real concept of risk management or monetizing their assets.

    Joanne

    Yeah, I think moving towards monetization is a completely different ball game, and I personally see a lot of control, but control from the perspective of we have limits, we have compliance activities, we are managing within a corporate view of what we believe trading should do and shouldn't do, not really assessing that against value. So what is the potential from the perspective using, you mentioned earlier in such as you Alex, from a credit risk perspective, how do we optimize credit risk? What do we do to create value from mark out to policies?

    Alex

    So I think the core of the control process is running a daily routine and discipline around creating a view of where the risks are in the portfolio. I think this is something that should be largely automated, so that you create the capacity for risk team to also think about the risks that aren't covered in the current framework and to look beyond, to look into what can happen in terms of stress tests, where are unexpected events going to come from to challenge traders. All of that takes share of mind and you need a bit of extra capacity to deal with all of that. And I think having efficient processes and system support is one way to create the capacity.

    Marc

    And I mean there's comfort, right? In just executing policy and following the rules and having a clear frame on controls. But as Alex mentioned, I mean you need to free up that time. Also, there's a talent component to that because if you don't have that freed up time, a lot of people would not like to fill their workday or the entire workday at least with very repetitive tasks, essentially just producing the same set of numbers every day. I mean, I did the job myself and having that spare capacity and I had it luckily at an investment bank gives you the time to really think through stuff, right? It is also making the job itself much more attractive because not only the engagement with a front around certain topics, the engagement may be with management on risk management topics, but just having the intellectual, let's say challenge, is quite a distinctly different thing and just walking up to the job every day and doing the same thing over and over again.

    Mike

    Right? There's a very real difference between risk control and real risk management or risk advisory. And so control is about limiting stopping or making sure that repetitive activities, as you say, are performed every day. And risk management or risk advisory and optimization is all about getting the most commercial value out of every unit of risk taken. When we do diagnostics for firms on risk management practices will look for constructive conversations between the front and middle office. Discussion of risk efficiency and risk-based performance and things like this, if it's just controls, they're really not getting everything out of their team that they can.

    Joanne

    I think in addition to that, there's also the element of the level of administrative activity that's actually taking place. So in some of what we would consider to be very mature energy companies, you still have systems that basically are not fit for purpose and make sure that risk folks who are highly confident and can provide advisory are actually figuring out how to do an end of day report and piecing together information, right? So it is kind of like let's get at least the knitting of this thing set up properly so that you do have a lot more free time to go out and provide the advisory that's required, fundamentally required by the front office.

    Mike

    It's interesting about risk systems in many of these firms because they've grown grownup piecemeal, they're all separate, they're not really consistent with one another. And so, we think that it's best to have a common framework and model something like a Monte Carlo that allows you to look at market risk, credit risk liquidity risk together scenarios, augmented by scenarios that allow you to look at some of these black swan events that we've been experiencing and really understand the effect of a five or 10 standard deviation move. This is just not really in place for most firms. So one thing that's made this a lot easier is the current emphasis on data architecture and data strategy. And so the widespread adoption of the cloud has allowed firms to focus more on what we call data-centric architecture as opposed to the ETRM system or systems being the center of the universe. And this allows these risk systems to be developed as point solutions around the edge of a good data repository and the models to be managed in a much more standardized way.

    Marc

    I mean, I have a client that actually holds for a lot of clients that are currently facing a lot of, let's say, paradigm shifts in the way that their portfolios functions and the way that they need to think about risk. So two of the key things I guess maybe is there's a lot more heterogeneity in what they're trading, may that be geographically or maybe commodity classes. Second thing is there's also expanding risk and trading activity into long data illiquid risks. Is that something that also links to your point around the data and maybe something that can enable it or some of the approaches that you mentioned?

    Mike

    Yes, because instead of doing that all in one solution, you now have the ability to develop specialized solutions for markets and have them integrated with the same data architecture that you're using for your primary base products.

    Joanne

    So is this a question of systems, process, people? What is it or is it everything?

    Alex  

    I think you need both to come together. On the one hand, having the right talent in place and the capabilities, the experience that is important because you want people, like we said before, to look past the framework as well, but then you also need the lean processes to generate the efficiency in the day-to-day and to create the space to also think about what you don't cover in your daily routine. And all of that obviously needs to be underpinned by efficient systems.

    Mike

    I agree. Technology is the tail wagging the dog. You need to make sure that it directly supports commercial strategy and risk management objectives.

    Marc

    And I think what we're also seeing a lot what doesn't work is essentially go to the banking space and try to straight export into physical commodity trading. So there always needs to be a bit of a, let's say adaption or at least some sort of link to the fundamentals around the physical side of the business, or I'm giving one example. I mean, I guess a lot of our clients struggling, especially with liquidity risk management and credit risk management. If you go for a bank, I mean there's a credit book that allows you to diversify some of those risks and hedge them, they can hedge the interest rate curves, they can hedge the credit risk to some extent against their own credit books and diversify it. That fundamental is not given in at least a regular more regular commodity trading environment, let's say at an independent trader or captive trader. So that level of adaptation and also being able to essentially work with a risk framework, still capturing some of these risks and so looking after them and monetizing them even, but not necessarily going straight to the banking world to look for let's say best in class or however you want to label types of approaches.

    Joanne

    I think on the other scale, the other challenge is it's good and great to understand the role of risk and risk management in general within trading. It's often within integrated companies, it's very difficult to have this conversation between the trading organization and the corporate function. So with finance and treasury, because in some ways, whilst they shouldn't be competing objectives, it does feel at times that these two different sets of stakeholders feel they are. So on the one side you have traders talking about optimizing margin and the positions they need to take versus organizational view, which is, well, we want stability in our free cash flow, we don't want to get into markets that seem too risky for us. What does this mean in terms of our ability to access that asset or origination activities? So I think there is a bit of a gap between those two sets of stakeholders that does also need to be managed.

    Marc

    It's a gap, but I would also argue it's different clock speeds. So group functions, they typically work on yearly cycles. So there's a budget cycle, there's a strategic exercise that's typical be done once a year. The group treasurer would want to go out to the market, maybe to place a bond or something that takes a lot of lead time. And then if you hit that system, something that is as dynamic as trading can be, or even more supercharged like a 2022, where then even the clock speed of trading is increasing to intraday, let's say volatility we've not seen before. Then quite naturally these connects break. And then you need to rethink what is the data, what is the transparency you need to give on your trading business and the risk within? And also to the advisory point, how much do you maybe need to educate management or group functions in order to comprehend what the risks are that the business is carrying and should be carrying as part of their core mandate?

    Mike

    Another dimension of this is that physical shops don't you know, by history or legacy, really have the same view of credit risk that financial shops do. And so, in big physical trading centers like Houston, many are just managing AR, that's credit risk. And so if you sell index cargos, all you're really worried about are two or three invoices that aren't collected or product that's been delivered and not invoiced. And so what's interesting is as these firms transform into more active trading shops, they'll have a larger and larger portion of their counterparty risk be composed to mark to market. And so they'll be using VAR to manage price risk or sophisticated price risk models, but on the credit side, they're really just using AR and they don't understand how important mark to market has become in their business. And they're not stochastically testing changes in mark to market using PFE or C-bar, tools like that.

    Joanne

    I mean on that, if you were in the position then of a CRO whose dealing with all of this complexity, market complexity, the growth or evolution of your own business, systems questions, engagement with the corporate, if I could ask you a very tough question, what is the most important thing that you think they need to be thinking about? What would that be?

    Alex

    On risk management? I think it's about the proposition for the talent. You need to be able to attract and retain high-caliber people and you need to be able to show them how they can progress in their careers. Only then do you get people that also front office will be able to invest in because they will see the value in a discussion and into sparing and advisory. And that's really what makes that discussion a two-way street.

    Joanne

    Yeah, so a trading career track, a true one, not hopping in and out trading every couple of years.

    Marc

    And I mean we have the beauty of having being asked this type of question on a more or less weekly basis. And I guess what we are seeing at the moment is to Alex's point around talent, but also around the topic itself, elevating it to the top and giving it a purpose on the top is also quite powerful. So one example being, for instance, we're doing a lot of work currently around bringing risk into the fund financial planning and portfolio planning domains.

    So, essentially allowing companies who become more and more heterogeneous to take decisions on investment and performance management, which are risk adjusted. What's the value there? I mean, the value clearly is you can see correlation, you can see how you can potentially also hedge out some of those very high uncertainties that we're currently seeing out of the regulatory environment, out of the price environment, out of maybe some of the more geopolitical environment that could then affect your investments, that particularly when it comes to green investments, can have a long lead time, can have a lot of risk around future cash flows. And elevating that role of risk and elevating that conversation into some of the core management steering processes and even in some cases, bringing that conversation to depth and shareholders as well, is quite powerful. And that in return then empowers as well the risk organization to take a stand and to have a seat essentially at the table.

    Mike

    So my suggestion might sound a little inconsistent with something we discussed earlier that technology shouldn't be the tail wagging the dog, but I think CROs should throw all their weight behind data strategy and data architecture projects now because we see so many places around the globe, where companies are in the middle of big ETRM programs or other transformation projects, and their data strategy is way behind the times. So they're going to reproduce dozens or hundreds of point-to-point interfaces with old technology. They're not getting advantage of this data-centric architecture that we talked about earlier, and it's a missed opportunity. So I think CRO is a really important stakeholder in these big IT programs, and that's what I think they should really emphasize. It promotes better data analytics, better risk measurement. It's very important.

    Joanne

    You can't manage where you can't measure, you can't optimize, where you can't measure. So, rings true. Well guys, thank you so much for your time. Fascinating discussion that I hope we continue as we move forward.

    This transcript was edited for clarity

    In recent years, weather-related and geopolitical events have resulted in an emergence of certain risk types that trading organizations may not have been expecting to see at the scale. As businesses ramp up their trading operations, greater activity brings a greater need to understand risk. In this episode of “The Trading Desk,” our experts explore the spectrum of risk, whether it’s being managed or monetized, and misconceptions corporates might have around trading.

    Interested in a particular topic? Skip to the following sections:

    1:10 — Black swan events and risk management

    3:54 — Understanding liquidity risk management

    8:04 — Risk as a supervisory entitity

    10:00 — Automoted risk portfolios and frameworks

    13:47 — Optimizing data infrastructure and architecture

    15:16 — Understanding systems, process, and people to work together

    17:02 — Trading organizations versus corporate functions

    20:17 — Talent sourcing and retention in trading

     

    Joanne Salih 

    Hi, I'm Joanne Salih, a partner in our Energy and Natural Resources Practice, here at Oliver Wyman, and head of America's Risk and Trading. Today I'm joined by Alex Franke, global head of our Risk and Trading Practice, and a partner in our Energy and Natural Resources Practice. Mike Burger, who is a VP and co-founder of our Veritas business, a specialized risk and trading team within Oliver Wyman. And Marc Zimmerman, a partner in our risk and trading team in Europe, and also a partner in our  Energy Natural Resources Practice.

    Today we'll be talking about risk management: manage or monetize? That's the question. Part of our four-part series on risk and trading. The last few years have been certainly kind of extra normal profits, but a lot of volatility, a lot of risk that I think most of our clients in the wider trading environment we're not expecting as with any black swan event and two by nature, which were quite different. What I think was quite interesting though is it highlighted risk that people really haven't been watching out for or quantifying to the extent that they would now be thinking about and certainly some of their limits requirements around liquidity became fundamentally different. So very different environment that we've been playing in and I think hopefully is a bit of a leap forward when it comes to risk management.

    Alex Franke

    Yeah, I totally agree. Absolutely great opportunity on the commercial side, but also a much higher risk environment and we didn't see the interplay, especially between market and credit risk that we haven't really seen before, not to that degree. So it was really important to focus on the quality of risk management, the understanding the trade-offs between the different risk types and then also steering, the risk taking towards the areas that you would get compensated for to actually capture the opportunity that was available in the market.

    Joanne

    Maybe on your first point Alex, you're talking about transfer of risk between different risk types.

    Alex

    The trade off between market and credit risk and to a certain degree as well, credit risk that we see emerge more lately and I think that understanding on how risk materializes between these different risk types, I think that is really important.

    Mike Burger

    Yeah, I agree. We see clients ignore one or two of these sometimes and they don't realize the interdependence between them, and so that's why it's important to have a comprehensive approach to this and model them in a consistent way and maybe augment that with scenario analysis so that you really can see how they interact.

    Alex

    The modeling is important and then also how you steer on it. I think the holistic steering is important. If I remember in more than one case there was a focus on market and liquidity risk and also charging for the risks that are being taken. If you disregard the credit dimension of it, what basically happens is that your traders will start to seek out the worst counterparties that are available because this is where you get the higher prices. If you don't price for the credit risk, there is potentially unintended consequences in the back of this.

    Marc Zimmerlin

    And liquidity risk being actually one of the other aspects that people did not really care about enough, I would say before 2022. So a lot of players actually got hit more or less on surprise not being able to properly respond and also comprehensively what was going on in the exchange positions. I mean in the past they had in some cases try to predominantly OTC and bilateral, they really done the exchange based trading or clearing on the side and now the exchanges and the clearers asking for excessive margin, excessive initial end variation margin in 2022 and some of that carried into 2023 has really changed the way that our clients look at liquidity management, but also liquidity risk management as an important aspect. And I think this has brought a lot of the, let's say more natural finance domain topics as well into trading related to treasury related to capital and capital return on capital even.

    Marc

    And for a lot of players, and Alex mentioned it, I mean there was a lot of commercial opportunity, but in 22 for a lot of players actually they couldn't capture it because they got stopped out on their risk controls and the risk systems because they weren't designed essentially to make those trade-offs visible and to really give the traders or whoever was taking the commercial decisions, a bit of a framework, they allow to actually weigh some of the decisions against each other. So actually a lot of market participants actually got stopped out in the market in 22.

    As Alex mentioned it, there was a lot of commercial opportunity in the market, but if you would be executing on a risk control framework, which was very prohibitive in taking some credit risk maybe on parameters which didn't really hold in 2022 market circumstances or liquidity side, I mean people got scared quite frankly by some of the large numbers they've been seeing. And then the commercial side of the business, may that be the traders or maybe some of the management, they were actually stopping out capturing some of that commercial opportunity and some others in the market. They were more or less blindly executing mechanically on some of the hedging strategies and you got the worst of both worlds I would say, in not applying a risk management mindset, which was more of an maybe educated or advanced mindset to the setting.

    Alex  

    I think this is where we really saw a divide in the performance that players were able to capture as well. There were the ones that had established risk management frameworks, they were confident in the way that they were quantifying and controlling the risks and also managing the trade-offs on a quantitative basis. And I think this is where then the additional capability needed to come in on connecting the dots to the finance and the treasury topics that were emerging and those were the players that were able to stay in the market for longer and capture more of the opportunity that was available.

    I think this is also where you have the basic requirements of risk control that need to be met, so that gives you the confidence to actually stay in the market if it gets rough. And this is where the risk advisory dimension comes in where you have a challenging discussion between risk and the front office on whether you're taking the right risks, whether you're adequately being compensated for the risks that are being taken and how the trade-off between market credit and liquidity risk actually works both from a steering perspective but also from an advisory, from a sparing discussion perspective.

    Marc

    I would actually even add that that risk advisory even expands beyond the commercial operations, let's say in trading also to management. You need to take a step back and I think what some of those maybe frameworks can deliver on as the unknowns or known unknowns and really being able to then flexibly respond. I mean there was a client, for instance with us when we worked with them in 2022, they really had to rethink credit risk in a way that you would be allowing maybe in terms of absolute numbers, a lot more credit risk towards counterparties. You typically would not, I mean example being an upstream producer essentially making a lot of money out of the high price environment.

    Does the credit risk and the default risk, which is associated with that really represent what historically has been seen for that particular rating class and that circumstance> I mean that was a question essentially being asked, and if you stuck to your risk controls as they were defined pre-crisis and just applied, let's say a rating, agency rating to that counterpart and the corresponding limit, you'll be stopped out. And if you have the advisory mindset, you can reflect on the situation, you can reassess the PD in this instance, or you can bend let's say the control framework a little bit to make it work.

    Joanne

    I mean, I think that's the key point and what we still see playing out. Now, as you mentioned Marc earlier, I mean there are those that kind of learn their lessons and I think a lot of those European players who maybe saw the most extreme version of this all playing out. But then there are others who I think are still grappling with this question and fundamentally risk at best is a supervisory entity, and having real advisory taking place like capabilities, and frankly the systems underlying it becomes pretty fundamental, but it's not something you can get to quickly. I think.

    Mike

    Alex and Marc have been talking about have been broken all over, not just in Europe, and it feels like we've been having a hundred-year weather or financial event every year. We had a client who called us, concerned after making $80 million from a hedge program. They figured if they could accidentally earn $80 million, they could just as easily lose $80 or $100 million. So, we helped them kind of reassess their hedging program, but it's an example where a firm had very good risk controls in place, I think decent ones, but no real concept of risk management or monetizing their assets.

    Joanne

    Yeah, I think moving towards monetization is a completely different ball game, and I personally see a lot of control, but control from the perspective of we have limits, we have compliance activities, we are managing within a corporate view of what we believe trading should do and shouldn't do, not really assessing that against value. So what is the potential from the perspective using, you mentioned earlier in such as you Alex, from a credit risk perspective, how do we optimize credit risk? What do we do to create value from mark out to policies?

    Alex

    So I think the core of the control process is running a daily routine and discipline around creating a view of where the risks are in the portfolio. I think this is something that should be largely automated, so that you create the capacity for risk team to also think about the risks that aren't covered in the current framework and to look beyond, to look into what can happen in terms of stress tests, where are unexpected events going to come from to challenge traders. All of that takes share of mind and you need a bit of extra capacity to deal with all of that. And I think having efficient processes and system support is one way to create the capacity.

    Marc

    And I mean there's comfort, right? In just executing policy and following the rules and having a clear frame on controls. But as Alex mentioned, I mean you need to free up that time. Also, there's a talent component to that because if you don't have that freed up time, a lot of people would not like to fill their workday or the entire workday at least with very repetitive tasks, essentially just producing the same set of numbers every day. I mean, I did the job myself and having that spare capacity and I had it luckily at an investment bank gives you the time to really think through stuff, right? It is also making the job itself much more attractive because not only the engagement with a front around certain topics, the engagement may be with management on risk management topics, but just having the intellectual, let's say challenge, is quite a distinctly different thing and just walking up to the job every day and doing the same thing over and over again.

    Mike

    Right? There's a very real difference between risk control and real risk management or risk advisory. And so control is about limiting stopping or making sure that repetitive activities, as you say, are performed every day. And risk management or risk advisory and optimization is all about getting the most commercial value out of every unit of risk taken. When we do diagnostics for firms on risk management practices will look for constructive conversations between the front and middle office. Discussion of risk efficiency and risk-based performance and things like this, if it's just controls, they're really not getting everything out of their team that they can.

    Joanne

    I think in addition to that, there's also the element of the level of administrative activity that's actually taking place. So in some of what we would consider to be very mature energy companies, you still have systems that basically are not fit for purpose and make sure that risk folks who are highly confident and can provide advisory are actually figuring out how to do an end of day report and piecing together information, right? So it is kind of like let's get at least the knitting of this thing set up properly so that you do have a lot more free time to go out and provide the advisory that's required, fundamentally required by the front office.

    Mike

    It's interesting about risk systems in many of these firms because they've grown grownup piecemeal, they're all separate, they're not really consistent with one another. And so, we think that it's best to have a common framework and model something like a Monte Carlo that allows you to look at market risk, credit risk liquidity risk together scenarios, augmented by scenarios that allow you to look at some of these black swan events that we've been experiencing and really understand the effect of a five or 10 standard deviation move. This is just not really in place for most firms. So one thing that's made this a lot easier is the current emphasis on data architecture and data strategy. And so the widespread adoption of the cloud has allowed firms to focus more on what we call data-centric architecture as opposed to the ETRM system or systems being the center of the universe. And this allows these risk systems to be developed as point solutions around the edge of a good data repository and the models to be managed in a much more standardized way.

    Marc

    I mean, I have a client that actually holds for a lot of clients that are currently facing a lot of, let's say, paradigm shifts in the way that their portfolios functions and the way that they need to think about risk. So two of the key things I guess maybe is there's a lot more heterogeneity in what they're trading, may that be geographically or maybe commodity classes. Second thing is there's also expanding risk and trading activity into long data illiquid risks. Is that something that also links to your point around the data and maybe something that can enable it or some of the approaches that you mentioned?

    Mike

    Yes, because instead of doing that all in one solution, you now have the ability to develop specialized solutions for markets and have them integrated with the same data architecture that you're using for your primary base products.

    Joanne

    So is this a question of systems, process, people? What is it or is it everything?

    Alex  

    I think you need both to come together. On the one hand, having the right talent in place and the capabilities, the experience that is important because you want people, like we said before, to look past the framework as well, but then you also need the lean processes to generate the efficiency in the day-to-day and to create the space to also think about what you don't cover in your daily routine. And all of that obviously needs to be underpinned by efficient systems.

    Mike

    I agree. Technology is the tail wagging the dog. You need to make sure that it directly supports commercial strategy and risk management objectives.

    Marc

    And I think what we're also seeing a lot what doesn't work is essentially go to the banking space and try to straight export into physical commodity trading. So there always needs to be a bit of a, let's say adaption or at least some sort of link to the fundamentals around the physical side of the business, or I'm giving one example. I mean, I guess a lot of our clients struggling, especially with liquidity risk management and credit risk management. If you go for a bank, I mean there's a credit book that allows you to diversify some of those risks and hedge them, they can hedge the interest rate curves, they can hedge the credit risk to some extent against their own credit books and diversify it. That fundamental is not given in at least a regular more regular commodity trading environment, let's say at an independent trader or captive trader. So that level of adaptation and also being able to essentially work with a risk framework, still capturing some of these risks and so looking after them and monetizing them even, but not necessarily going straight to the banking world to look for let's say best in class or however you want to label types of approaches.

    Joanne

    I think on the other scale, the other challenge is it's good and great to understand the role of risk and risk management in general within trading. It's often within integrated companies, it's very difficult to have this conversation between the trading organization and the corporate function. So with finance and treasury, because in some ways, whilst they shouldn't be competing objectives, it does feel at times that these two different sets of stakeholders feel they are. So on the one side you have traders talking about optimizing margin and the positions they need to take versus organizational view, which is, well, we want stability in our free cash flow, we don't want to get into markets that seem too risky for us. What does this mean in terms of our ability to access that asset or origination activities? So I think there is a bit of a gap between those two sets of stakeholders that does also need to be managed.

    Marc

    It's a gap, but I would also argue it's different clock speeds. So group functions, they typically work on yearly cycles. So there's a budget cycle, there's a strategic exercise that's typical be done once a year. The group treasurer would want to go out to the market, maybe to place a bond or something that takes a lot of lead time. And then if you hit that system, something that is as dynamic as trading can be, or even more supercharged like a 2022, where then even the clock speed of trading is increasing to intraday, let's say volatility we've not seen before. Then quite naturally these connects break. And then you need to rethink what is the data, what is the transparency you need to give on your trading business and the risk within? And also to the advisory point, how much do you maybe need to educate management or group functions in order to comprehend what the risks are that the business is carrying and should be carrying as part of their core mandate?

    Mike

    Another dimension of this is that physical shops don't you know, by history or legacy, really have the same view of credit risk that financial shops do. And so, in big physical trading centers like Houston, many are just managing AR, that's credit risk. And so if you sell index cargos, all you're really worried about are two or three invoices that aren't collected or product that's been delivered and not invoiced. And so what's interesting is as these firms transform into more active trading shops, they'll have a larger and larger portion of their counterparty risk be composed to mark to market. And so they'll be using VAR to manage price risk or sophisticated price risk models, but on the credit side, they're really just using AR and they don't understand how important mark to market has become in their business. And they're not stochastically testing changes in mark to market using PFE or C-bar, tools like that.

    Joanne

    I mean on that, if you were in the position then of a CRO whose dealing with all of this complexity, market complexity, the growth or evolution of your own business, systems questions, engagement with the corporate, if I could ask you a very tough question, what is the most important thing that you think they need to be thinking about? What would that be?

    Alex

    On risk management? I think it's about the proposition for the talent. You need to be able to attract and retain high-caliber people and you need to be able to show them how they can progress in their careers. Only then do you get people that also front office will be able to invest in because they will see the value in a discussion and into sparing and advisory. And that's really what makes that discussion a two-way street.

    Joanne

    Yeah, so a trading career track, a true one, not hopping in and out trading every couple of years.

    Marc

    And I mean we have the beauty of having being asked this type of question on a more or less weekly basis. And I guess what we are seeing at the moment is to Alex's point around talent, but also around the topic itself, elevating it to the top and giving it a purpose on the top is also quite powerful. So one example being, for instance, we're doing a lot of work currently around bringing risk into the fund financial planning and portfolio planning domains.

    So, essentially allowing companies who become more and more heterogeneous to take decisions on investment and performance management, which are risk adjusted. What's the value there? I mean, the value clearly is you can see correlation, you can see how you can potentially also hedge out some of those very high uncertainties that we're currently seeing out of the regulatory environment, out of the price environment, out of maybe some of the more geopolitical environment that could then affect your investments, that particularly when it comes to green investments, can have a long lead time, can have a lot of risk around future cash flows. And elevating that role of risk and elevating that conversation into some of the core management steering processes and even in some cases, bringing that conversation to depth and shareholders as well, is quite powerful. And that in return then empowers as well the risk organization to take a stand and to have a seat essentially at the table.

    Mike

    So my suggestion might sound a little inconsistent with something we discussed earlier that technology shouldn't be the tail wagging the dog, but I think CROs should throw all their weight behind data strategy and data architecture projects now because we see so many places around the globe, where companies are in the middle of big ETRM programs or other transformation projects, and their data strategy is way behind the times. So they're going to reproduce dozens or hundreds of point-to-point interfaces with old technology. They're not getting advantage of this data-centric architecture that we talked about earlier, and it's a missed opportunity. So I think CRO is a really important stakeholder in these big IT programs, and that's what I think they should really emphasize. It promotes better data analytics, better risk measurement. It's very important.

    Joanne

    You can't manage where you can't measure, you can't optimize, where you can't measure. So, rings true. Well guys, thank you so much for your time. Fascinating discussion that I hope we continue as we move forward.

    This transcript was edited for clarity