Workhorse is a cloud-native software tool which makes order and stock management easy. They have a deep understanding of how their clients interact with their software, and are always keen to make improvements which benefit their growing client base.
So when Alastair approached ricky from Beaty Consultancy to talk about implementing an AI document ingestion feature to their platform, the Beaty team jumped at the chance!
The challenge
AI tooling can become really expensive if you’re not careful, and it can also give inconsistent results if not given strong guardrails. So the simple idea of sending email attachments straight from a client environment, into the AI, and straight into Workhorse wouldn’t be sustainable. We needed to add automation and old-fashioned predictable code paths around this flow.
The team set about building a serverless architecture, using a mix of AWS tooling, and the OpenAI APIs. AWS services, like SES, SQS, S3, DynamoDB and Lambda allowed us to build a serverless, stateless execution path from dedicated email inboxes, all the way to OpenAI. But what about those guardrails we spoke about?
The Right Tool for the Right Job
Human oversight is vital in all automation pipelines. If things go wrong, someone needs to be able to easily step into the information flow to be able to ascertain what went wrong, where, and what the effects of that are.
To that end, sensitive information is first copied to S3 buckets which implement strong data resilience features over and above what comes by default with the Simple Storage Service. Bucket versioning and tight least-privileged access controls were layered on top to ensure that we keep an accurate record of what came into the pipeline, regardless of any processing that happens along the way.
Because Workhorse is a Software as a Service (SaaS) application, it needs to be able to scale quickly, and remain performant even when their users demand higher and higher throughput. To that end, the Beaty Consultancy team deployed DynamoDB and SQS in order to record and robustly pipeline the incoming requests to the AI processor. All of this is serverless, and instantly expands to the needs of the userbase. But when things are less busy, costs come right down, allowing Workhorse to remain competitive.
Preprocessing is key
Within the custom Lambda functions which the Beaty Consultancy team developed, is an Optical Character Recognition (OCR) module. This module’s job is to, as you might imagine, turn a document into formatted text, ready to pass to the AI engine.
Of course you can send a whole document, a picture, or anything you like to AI, and it’ll do its best, but it’ll cost you! AI systems work on a system of tokens, and the more complex the work you ask it to do, the more tokens it consumes. So by stripping out formatting from a document, we can use up fewer tokens per request. That gives a faster response from the AI, but importantly saves money per-request.
Business as Usual
Workhorse have been a client of Beaty Consultancy for many years, and so we were the natural choice to help bolster their internal development team when it came to this exciting new feature being developed. Keeping it within our teams also means we’re in the best possible position to support the environment into the future – but don’t worry, there’s a beautiful ReadMe file in the code repository if you’re ever struggling to sleep at night.
