Businesses of every size have been unable to escape the incredible impact that AI has had on the ways in which we do business of late.

From conglomerate to SME, organisations are becoming faster, more agile, and more robust as we outsource administrative and repetitive tasks to our AI co-workers.

One of the newest AI trends is the establishment of Large Language Models (LLMs) in the public domain: machine learning algorithms trained on colossal volumes of data to recognise the structures and patterns of natural language. They are capable of Natural Language Processing (NLP), which allows us to explore huge datasets through everyday questions or commands.

As such, LLMs are the most common way of making AI intelligible – to cite the most well-known example, LLMs are the means by which ChatGPT can answer your questions. But there’s one conventional drawback to that intelligence: it’s stuck in something of a time capsule.

LLMs are intensively trained, with millions upon millions of data points fired at them in a constant feedback loop to teach each model how to make sense of certain datapoints or patterns. But ‘operationalising’ an LLM – taking it off the training circuit and bringing it online as part of your infrastructure – obviously prevents it from learning anything new. Even some of the first versions of ChatGPT, if you ask a question about very recent events, will politely explain its own temporal limitations to you.

That means you’ve got to be sure that the LLM can rely on the systems they’ll be exploring, and the data available to them. And while the corporate giant might have the funding and the tech stack to make that happen, that’s a brave assumption to make of an SME.

Move it or lose it

Historically, we’ve tended to think of data as static. When the layman downloads a file on their PC, the file isn’t ‘there’ until it pops up in your documents, even as millions of individual data bytes quietly stitch themselves into something infinitely more sophisticated.

With that mindset, you can understand why businesses have often opted to capture as much data as they can, and only then set about establishing what they’ve actually collected. Convention would have us pour data into a huge data warehouse or lake, spending an age clearing and preparing that data, and then dig up different cuts for analysis – a method widely known as batch processing.

This is about as efficient as it sounds. Wrestling an entire dataset duplicates work, camouflages insights, and makes huge demands of hardware and power consumption – all while delaying key business decisions. For the SME trying to find ways to compensate for limited funds and personnel, this method undermines the agility and speed that should be their natural advantage.

Given information until now was not required to be consumed in real time, or even collected in real time this has not been a problem until now. But given how many of the new companies’ end customer value proposition relies in real time data (i.e. think of calling a taxi with Uber or a similar application and imagine not seeing the “live” map with the location of your driver) this is now a “must-have” not a “nice-to-have”.

Fortunately, LLMs don’t only function on a batch processing basis. They can interact with data in different ways – and some of those ways don’t demand that data stands still.

Ask and ye shall receive

Just as disruptive SMEs seek to overturn older and more established companies, data streaming is replacing batch processing.

Data streaming platforms use real-time data ‘pipelines’ to collect, store, and use data – continuously, and in real time. The processing, storage, and analysis that batch processing keeps you waiting on can suddenly now be achieved immediately.

Streaming manages this through what we call event-driven principles, which is essentially treating each change in a dataset as an ‘event’ in itself. Each event includes a trigger to receive more data, creating a constant cascade of new information. Instead of having to go and fetch data (usually stored in a table somewhere in a database), data sources “publish” their data in real time, at all times, to anyone who wishes to consume that data simply by “subscribing” to that data.

All of this can free LLMs from the distinction between training and operating. Furthermore, if every data point can be actioned, it’s possible for the LLM to train itself; to use the correctness of its actions to constantly refine the underlying algorithms that define its purpose.

That means the LLM can draw on a constantly updated and curated dataset, while constantly improving the mechanisms that deliver and contextualise that data. Data isn’t at risk of redundancy or abandoned in some forgotten silo – all you have to do is ask for it!

Cut from the SME cloth

So: what does that mean for the SME?

For one, it takes off the proverbial handbrake. The sheer speed at which LLMs can deliver information through a stream-driven infrastructure empowers decision-makers to drive the business forward at their desired pace, with no batch processing to keep them in second gear. The agility that empowers SMEs to outmanoeuvre larger players is back in abundance.

Those decisions are made with less doubt, and more relevant context, than before. It’s so simple to access a specific insight, thanks to the natural language that LLMs recognise, that data streaming can foster a genuine enthusiasm for business transparency right across the board.

Not only is the output faster and more accurate, but SMEs can free themselves from legacy technology, too. Data streaming can take place entirely on premise, entirely in the cloud, or in a mixture of the two. The heavy-duty hardware often required for batch processing is simply no longer necessary if you can ask an LLM for the same result in record time. Also, there are several providers that provide fully managed (turn key) solutions that require zero capital investment from the SME’s.

For SMEs to make the most of LLMs, then, they need to think about the way in which they approach company data. If a company is ready to commit to treating data as a constant stream of information, they’ll be much better placed to maximise the potential that data in motion has to help them evolve.

Read more:
How can SMEs make the most of LLMs?

According to the latest data, 91% of small and 88% of medium-sized businesses use at least one open-source technology. And it’s no surprise.

For those who don’t want to shell out thousands of pounds for enterprise-grade software and licences, open-source lowers barriers to entry and provides high-quality business solutions without the upfront cost.

In fact, the average business would spend 3.5x more on software costs if it wasn’t for open-source alternatives. That’s a serious saving for any business, especially SMEs.

Of course, if it sounds too good to be true, it usually is. While open-source may be cheaper as an upfront cost, and often more flexible, it comes with a catch… complexity.

Cost through complexity

When building your business around open-source technologies there are inevitably complexities around compatibility, management, and development. Whereas enterprise-grade and off-the-shelf solutions come pre-packaged and ready to use, the flexible nature of open-source can be both a blessing and a curse.

Think of it like cooking a meal. Off the shelf software is your microwaved ready meal. They’re pre-made, easy to cook, and ready to go. On the flip side, they’re often overpriced, lack variety, and come packed full of ingredients you don’t really want.

Then there’s open-source, which is like home cooking. You get complete control over what you want to eat, what ingredients you use, and the end result is almost always nicer than a ready meal.

But here’s the catch: home cooking takes time and effort. You have to think about what you want to eat, find a recipe, and pay for all the separate ingredients – some of which you might not be able to source in your local market.

You need to invest your time and energy in cooking them, even though you might not be the best cook yourself – you could over- or undercook it, or simply never be quite sure if the meal’s as good as it could be. And then, at the end of it all… you’re left with the washing up!

While at first open-source may seem like the low-cost option, in the face of growing complexity the costs start to spiral. The need to customise and integrate a variety of different technologies takes serious time and effort, two things that ultimately push up the total cost of ownership (TCO) for your business.

After all, you’re not just paying for your chosen solution, but for all of the integrations, infrastructure and operational costs that come with it. The highly skilled engineers that have mastered the technology you’ve chosen will cost as well – and as they’re hard to come by, it’s likely you’ll have to pay very well. Complexity quickly becomes costly.

So, what’s the solution? Give up on open-source and go back to expensive enterprise grade solutions? Keep eating ready meals for the rest of your life? Thankfully there’s a better way.

Cheaper than free

Open-source software offers overwhelming benefits, from cost, to control, to flexibility. Businesses of all sizes can embrace these advantages, if only they can find a way to overcome the complexities involved. Thankfully there’s a simple solution — outsource the complexity.

Given that open-source technology is collaboration-driven, the solution to its growing complexity must also come from collaborating. In short — make it someone else’s problem!

Rather than attempting to personally manage all aspects of an open-source system, businesses should instead outsource that process to a third-party partner. This partner can act as a central curator, helping to improve, connect, and support their open-source technologies. And because they specialise in dealing with complexity, they can also teach, establishing best practice and economies of scale – as well as sharing those cost savings themselves!

The result is a system that retains all the benefits of open-source, while feeling more like an off-the-shelf solution. Returning to our home cooking example, third-party management becomes your own personal chef: all the benefits of home cooking, without the effort, ensuring the best ingredients and the perfect cooking.

To bring this analogy into the real world, let’s look at a technical example — Apache Kafka.

Payment when it’s due

Apache Kafka is a popular open-source ‘event streaming’ solution, letting businesses access data in real time. The benefits of such a system can’t be overstated, but it’s a hugely complex technology, especially for an SME to manage.

That’s where outsourcing is proving so essential. Through a ‘fully managed’ Apache Kafka service, businesses of all sizes can benefit from real-time open-source event streaming, without the need to manage the technology themselves. Even some of the largest companies are now adopting this approach, switching to a fully managed service for their open-source tech.

As well as saving time and cost, SMEs can also use this approach to drastically simplify all the moving parts that they’d otherwise have to manage. In outsourcing to professionals who have the skills required to implement best practice and to manage and upgrade open-source technologies continuously, downtime is reliably minimised. Less disruption means more peace of mind for everyone.

As a third party is taking on responsibility for infrastructure, SMEs can minimise their own business’s own infrastructural footprint and/or cloud spend, as that third party is able to provide solutions that the SME doesn’t need to own, or duplicate. That third party should be able to scale to match any concerns.

That, in turn, means lower development and operational costs. There’s no need to invest in low-level infrastructure tooling if you’ve got a platform to do it for you, and especially if you’re in control of the scaling of that platform.

Such partners will also be able to provide a much wider support network for technical challenges, from microservices to expert staff. And as there’s so little infrastructure to manage, that support can be much more focused on performance and efficiency than troubleshooting and problem solving.

Having all the expertise and best practices in one fully managed service provider also ensures that they can continuously improve the performance of the environment. In the case of Confluent’s Kafka fully managed service, our cloud native solution has 16X higher performance than open-source Kafka

All these things will drastically reduce the TCO of open-source software, without losing the numerous benefits. It also allows the company acquiring the technology to focus in differentiating their value proposition, not just having to run Kafka.

At its heart, open-source software is about collaborating within a community of like-minded experts – but that collaboration doesn’t have to stop when you take the technology on. In outsourcing its management, SMEs can simplify, cheapen, and accelerate the journey to their goals.

Read more:
Out in the open: the hidden costs of open-source software for SMEs