Databricks: Navigating the economic downturn with AI

It’s easy, when talking about the current economic downturn and its impact, to give in to doom and gloom. It has meant a forced reshuffling of priorities for many businesses, shifting from growth to adapting in the face of the challenges that are forecast to last at least until this summer. However, the outlook need not be so bleak. There’s much to be excited about coming from the UK’s tech sector, in particular, with the Chancellor recently announcing ambitions to make the nation the next Silicon Valley. The Government’s National Data Strategy is also working to enhance data and knowledge sharing across the UK. The point is, there are technologies – such as AI – which have the power to help businesses cope, and even thrive, in these challenging times. Furthermore, there is an opportunity for the taking here, to capitalise on the nationwide ambition to make the UK a powerhouse of data and AI. 

When it comes to AI, great inroads have already been made. A recent MIT report, titled “CIO Vision 2025: Bridging the gap between BI and AI” – which surveyed CIOs, CTOs, Chief Data and Analytics Officers, and other senior and data technology executives – found that 94% of respondents were already using AI in their line of business. Adding to this, well over half the surveyed executives said they expected AI use to be widespread or critical in business functions by 2025. 

However, just because AI is already being widely used within businesses does not necessarily mean it’s reaching its full potential. In fact, the same MIT report revealed that many businesses are encountering challenges in rolling out and scaling up their AI strategies. So, how can businesses change this and see the true extent of the benefits that AI can bring?

The value of AI now and in the future 

The MIT report identified the most tangible areas of benefit from AI use today, as well as exploring what respondents expected them to be by 2025. Increased revenues, for instance, whilst holding fourth position today, is expected to become the area where AI’s value will be most felt in 2025. Cost reduction will also be increasingly improved by AI, moving from the sixth spot today to third in two years’ time. As such, it’s clear that respondents feel there is value left to unlock from AI as pertaining to costs and revenue. We can therefore deduce that organisations that focus on, and invest in, facilitating AI will experience a positive impact on their bottom lines – a hugely attractive prospect today, amid an economic downturn. 

Faster product development and ‘time to market, as well as improved efficiency, were among the other top areas expected to benefit from AI. Here, Rolls-Royce, can act as a strong example. The manufacturer used cloud-based models to reduce costs for its customers, as well as create new revenue streams. The manufacturer has been able to collect real-time data generated through the creation of digital twins of its engines – the data is then analysed with AI and ML and used to predict engine failure before it occurs. This has allowed Rolls-Royce to improve efficiency, avoid unplanned grounded planes, and save millions of pounds from the reduced cost of inventory parts.

Time is money, and this old saying has even more truth during an economic downturn. If AI can cut costs, boost revenues, and accelerate how quickly an organisation can deliver a product – or simply operate in general – the value is clear. All of this helps businesses remain competitive. However, to get true value from AI, organisations need the right modern data architecture to facilitate it.

Supporting AI with data architectures 

Without good-quality data, AI simply cannot be implemented to its full potential. The challenge is, many organisations currently operate on complex legacy data architectures. Within these legacy structures, such as data warehouses, information silos can form, preventing data from being easily distributed. Inaccurate datasets, that contain duplicated or outdated information, may also be shared - adding to the problem even further and potentially causing larger future issues. 

To solve these challenges, organisations should start with their foundations and seek a modern data architecture from the outset. The chosen architecture should reduce the number of different platforms needed, removing the complexity of alternative legacy architectures. Modern architectures, such as a data lakehouse, should also ensure the timely flow of accurate data, easily storing data for analysis, as well as AI and ML use cases. When deciding on an architecture to facilitate AI, organisations should look out for the following criteria: 

• Open up: Data technology that favours open standards and open data formats is well placed to facilitate collaboration. AI is a major beneficiary of open, multi-partner collaboration – this is due to the fact that much of AI projects’ future success relies upon data, insights and tools gathered from external sources. As such, openness and AI success go hand in hand. 
• Data is for everyone: Democratising the ability to work with data, and even to configure and improve AI algorithms, will be key to the success of AI. This is because AI specialists and data scientists are in such limited supply – companies simply need more hands-on deck. “Citizen data scientists” – data-literate employees without specialist data science training – can be created via this democratisation. As such, any infrastructure modernisation should work to widen employee access to data to help meet this demand.
• Take a multi-cloud approach: The benefit here is twofold. Firstly, a multi-cloud approach allows organisations to get the best price for their workloads. Why? Because, although cloud vendors typically charge egress fees to move data in and out of their clouds, they are increasingly willing to negotiate lower costs. In fact, many of the largest companies in the world do not pay egress fees. This is a trend which will likely continue, meaning multi-could will become increasingly cost-effective over time. Secondly, multi-cloud is simply integral to scaling AI in a business. This is due to the multitude of complexities involved in monitoring and optimising AI projects across several vendor environments.  

This is not the first, nor will it be the last, time we are faced with economic uncertainty. The natural response may be to cut spending and pause investments, but leaders must remain future-gazing and not panic themselves out of long-term thinking. This means finding a solution that helps businesses ride out both the peaks and the troughs of the economic climate. Whilst investing in a modern architecture may seem daunting, it is truly a long-term investment for organisations looking to future-proof themselves. Modernise now, and enjoy streamlined processes and activity, and greater stability for many years to come.