Conference Season Wrap-Up: Democratising Data & AI

Heard this cliché before? 

“Imagine a world where AI is not just a buzzword, but a powerful tool transforming businesses that all can use.” 

Well, the reality might just be a little closer than you think. The end of the summer marks the start of the technology conference season and our team at The Dot Collective attended a few like Big Data LDN , Google Cloud Summit Databricks Data+AI World Tour and Snowflake Summit over the autumn to explore the latest advancements in data, AI and the cloud. 

Unsurprisingly the top themes were ‘democratisation’ of data and ML, or in plain in English, lowering the barriers to entry and getting more tech into the hands of more people. The second trend, while maybe not the most talked about, is pretty interesting. Data governance. The fact this topic has risen up the agenda is an admission from the mega vendors that poor quality data and security is impeding the value their customers are getting from AI tools, and subsequently slowing AI tool adoption (…and therefore slowing the software vendors’ revenue growth!). This is great news for everyone. Data Governance is hard, so any investment into accelerating the governance journey is always going to be welcome! 

A Focus on Accessibility and Democratisation

One consistent message resonated through all events – the drive to make advanced technologies more accessible to a wider range of users (health warning #1; this also means more people using the software vendors’ products…. but in the name of productivity, so it’s legit…right?). 

While most BI platforms have been trying to weave natural language (NL) queries into their services for some years, the recent advancements in LLMs is really starting to bear fruit and move this beyond simple summary statements that describe trends. We’re seeing genuine “agentic” capabilities emerge. That is inputs that are multi model; (e.g. speech, image or video) reasoning and planning capabilities that leverage LLMs to solve complex problems; and the persistence of perspective over longer durations to answer your question contextually. The weaving together of these capabilities into something that’s “agentic” is genuinely democratising for a business user interacting with data! Platforms like Sightfull. are a great example of this, where they are making waves enabling non tech users to launch complex sql queries with a GenAI assistant using natural language! 

Building on this theme, Google highlighted at their Summit how they were democratisating AI with tools like BigQuery ML – and this doesn’t mean making ML tools available to non-technical people, it means making the tools available to people outside the data science community!  For example, like most data warehouses Google BigQuery ML now brings ML inside the data warehouse with a few sql scripts, but excitingly is integrating its generative AI assistant Gemini to help manage these complex data tasks. Similarly for the developer, capabilities that summarise large documents with GenAI, or analyse images and process complex outputs are presented behind an API – in theory enabling the developer to build agentic applications. These innovations empower users to create and deploy machine learning models without requiring extensive programming expertise, like Python, R or Scarla, traditionally the preserve of the data scientist. 

[Editor’s note!] While these tools lower the barrier to entry for AI, the quality of the underlying data remains critical to achieving accurate, meaningful insights and actions. To truly democratise data, organisations must invest in robust data management practices…… but more on that in a moment.  

To really drive home the developer ML democratisation point, Google presented some super cool customer stories. One of our favourite was VEED.io who are using AI to simplify video content creation and offering that as a service – without a single data scientist in sight! Long live GenAI!! So effective is their service it’s already being used by the likes of Disney, Netflix and the BBC. (Health warning #2 it’s generally accepted in the world of social media instant gratification production quality is not quite that of a Hollywood blockbuster! Take from that what you will, but we’d recommend checking it out, it’s impressive!). 

These diverse applications showcase the broader theme of lowering the bar to entry and getting more tech into the hands of more people, whether that’s data insight or ML tools. 

Solid Foundations for Future Success

And so on to Data Governance, everyone’s favourite topic just after the weather and commuting. How do we know this topic is becoming important? When Satya Nadella, Microsoft's CEO says so. And so it became important, when within minutes of opening his key note at Ignite he stated;

“…in the age of AI, data governance takes on an even more critical, central, important role.” 

The use of AI brings with it issues such as oversharing and presents new attack vectors, like prompt injection. So Satya is right to bring our focus back to governance and ensuring it takes a central role in AI advancement. For Microsoft that means investing more into tools like Purview and accelerating its journey to help us secure, govern and monitor our data. For Databricks and Snowflake seeing them converge their governance tools into a single governance pane, will undoubtedly help accelerate the pace to apply policies, administer and audit data and AI assets. 

The development in these tools is very welcome, they will allow us to accelerate the implementation and the fine tuning of the two main levers of data governance, access and control. Finding the balance and then being able to evolve that point quickly is incredibly valuable for organisations. It will allow them minimize the creation of data silos while promoting the pace of innovation and maintaining the appropriate security posture for their industry. 

Conclusion 

In today's rapidly evolving tech landscape, the democratisation of data and machine learning is no longer just a buzzword but a tangible reality. As we attended various conferences this autumn, it became clear that lowering the barriers to entry for advanced technologies is a top priority. This shift is empowering more users to leverage powerful AI tools, driving productivity and innovation. However, the rise of data governance as a critical focus underscores the need for robust data management practices to ensure the quality and security of data, ultimately enhancing the value derived from AI solutions.