Lossless compression tailored for AI
Research
Kim Martineau
At Think, IBM showed how generative AI is set to take automation to another level
IBM is working on a range of generative AI technologies meant to make work simpler and more efficient. At Think 2024, some the latest breakthroughs were on display.
Consider the telephone. At one time, to call someone, you had to talk to an operator to place your call; then the rotary phone connected you directly. Eventually push-button phones automated the dialing, and now, we can even ask a virtual assistant to call someone without even looking at the phone.
Throughout history, humans have created machines to find simpler, faster, and safer ways to do things. As computers have become more powerful over the decades, we’ve been able to use them to automate increasingly complex tasks. It’s led to the birth of entire new fields marrying automation and business needs. But with the advent of generative AI, we're on the cusp of another revolution that has the potential to radically alter the way we work.
We’ve already started to see the promise of generative AI and large language models (LLMs), and IBM has been working on a myriad of methods, models, and projects to automate every aspect of business, from front-office processes (like customer service and sales engagement), to back-office process (such as HR or procurement) and physical infrastructure, like manufacturing plants.
Each of these areas can be improved by a range of generative AI language and code technologies, spanning a breadth of technical domains. This includes breakthroughs in content extraction, code generation, retrieval-augmented generation (or RAG), summarization, multi-turn conversational AI, APIs, planning, and more. These features are all powered by IBM’s Granite language and code models, which were unveiled in detail at this year’s Think conference.
Teams across IBM are using these emerging tools to build systems that save businesses time, money, and heaps of stress.
Today, developers are only as good as the time they have — and the information at their fingertips. They spend countless hours hunting around the web for bug fix solutions, or snippets of code that solve the problem they’re working on.
This month, IBM also open-sourced its state-of-the-art Granite code models. Available to anyone, they stand up to any comparable model for tasks like fixing code, explanations, or translating code between languages. For the team at IBM Research, these models are a step towards ensuring that anyone’s legacy systems can be supported, as well as tomorrow’s — after all, today’s modern applications will be legacy software in the future.
At Think this year, IBM announced watsonx Code Assistant for Enterprise Java Applications. This new assistant can summarize existing Java code, make recommendations and execute code upgrades, detail code changes, and even generate unit tests. IBM automated test generator framework increases testing coverage by up to about 50%, spanning a range of benchmarks (including code class, method, branch, and line coverage) for IBM Granite and other models. The generated tests are in comprehensible natural language and are easy for developers to maintain.
Most of the modern web is built on languages like Java and Python, incompatible with languages like COBOL, which most developers no longer learn. Web apps on the cloud are more flexible and scalable, but traditional structures like mainframes help companies keep data secure on their premises. While many automated systems that allow mainframes to work with cloud services exist, setting them up and maintaining them can be arduous.
This is why IBM released new capabilities for watsonx Code Assistant for Z at Think. These include new abilities to explain COBOL code and translate it into Java on clients’ premises with code that is 95% parseable by z/OS COBOL benchmark test. Similarly, new capabilities were released for watsonx Code Assistant for Red Hat Ansible Lightspeed entailing interactive playbook generation and explanation, achieving a 20% quality improvement in generated playbooks. Citibank is using watsonx Code Assistant for Red Hat Ansible Lightspeed and Broadridge Solutions is using watsonx Code Assistant for Z.
It’s not just code that’s being automated — it’s the way the entire web runs. As the cloud has matured into the always-on, infinitely capable computing system that underpins the way the world runs, it has required legions of IT professionals to ensure that sites and servers are as reliable as possible.
Site reliability engineers (or SREs) are the people responsible for the health of those systems. They are often overwhelmed as they try to manage the issues with massively complex systems that companies expect to be working at least 99.9999% of the time. Everything from servers to end-user computers, and cloud networking infrastructure — it’s a Herculean task to keep it all running, let alone figure out when small discrepancies are about to turn into major problems.
To even log all the information on the health of every part of a system creates a deluge of information that’s nearly impossible for anyone to sift through. Part of the focus of IBM Research’s work in IT operations (or ITOps, as it’s often called) has been creating tools to automate the collection and analyzing of IT health data for engineers to be able to find problems and work on ways to fix them. Whether that’s a cluster of users experiencing latency issues, or a server with less-than-ideal uptime, the engineer needs as much information on the situation as possible before offering a diagnosis.
IBM is also bringing AI and generative updates to its IT automation portfolio, including intelligent remediation for Instana, GPU optimization for Turbonomic, and cloud cost management tools (often called FinOps) for Turbonomic and Apptio. The Instana updates include generative AI technologies that can summarize IT issues, figure out a probable cause, and recommend actions to take — along with the actual code changes to implement.
The trace-based reinforcement learning technique applied for probable cause analysis increases the true positive rate by a factor of 1.6 and reduces the false positive rate by a factor of 200 when compared to conventional application performance monitoring tools. Some of these technologies have been implemented in Red Hat’s support business and to date, have resulted in more than 2,500 deflected calls per year.
IBM Research is also working with several companies to turn these concepts into working solutions for their systems. Beyond major incidents, there’s also the potential for AI systems to help monitor resource allocations using Turbonomic — IBM has started testing how GPUs are apportioned on its own systems, and for the cluster that works to train our own AI models, the amount of spare GPU capacity at any given moment shot up by a factor of five.
Beyond the world of IT, AI is upending how we build and supply everything. Humanity moved from the steam engine to the factory line, through to programmable machines, and now AI. With each revolution, production improved dramatically, as businesses were able to be more cost effective, scale more easily, and produce their goods more efficiently. AI has the potential to automate even more.
The fundamental underpinning of plant operations (often referred to as Asset Ops) is how work orders are managed. It’s a key part of IBM’s Maximo Application Suite. At Think this year, IBM introduced automated work order intelligence for Maximo, underpinned by IBM’s Granite model. This includes the ability to predict work order failure codes, using models to generate synthetic data. IBM is also using this software on its own sites and expects to save some 10,000 hours of productivity each year.
When it comes to managing plants, businesses today are limited by the information they have access to. But with generative AI, the amount of information that can be taken into consideration, and the sorts of insights that can be gleaned from it, is set to explode.
IBM’s open source Granite time-series models are a major step in this direction. The tiny time mixer (TTM) model in particular, based on an encoder-decoder architecture, significantly outperforms other models for forecasting tasks, with a 3% to 40% accuracy improvement (as measured by mean square error) over other state of the art models — even though the models are considerably smaller than others (1 million parameters, compared to 2.4 million to 311 million). Additionally, TTM can enable better continuous monitoring of assets and processes for anomalies — with significantly fewer false positives. Models of this nature, in tandem with automated work order intelligence, will allow enterprises to take a bold step towards closed loop automation for physical systems.
Saving time and making processes simpler isn’t just for industrial applications. The struggles of a modern office worker are often bound up in simple, but necessary, tasks that just waste a lot of time. Everyone has filled their time with things like filling out forms, copying information, moving files from one place to another — things that over time, build up to be a sizable chunk of one’s day.
Over the years, tools that make common business processes easier through automation started to spring up. IBM’s own Orchestrate product came out of years of research into automated software and can now make tasks of varying complexity extremely easy — from repeatedly adding rows to tables, to things like generating an expense report or creating a job requisition. But even with all these capabilities, generative AI offers the potential to make these sorts of business process automations more flexible and catered to specific users incorporating pertinent technology in the automation build and conversational AI spaces.
At Think, IBM unveiled watsonx Orchestrate Assistant Builder, which helps businesses build assistants composed of underlying skills that fit the exact tasks they need completed, whether that’s in human resources, sales, procurement, or anywhere else. These skills and underpinning open API specifications can be quickly built, enhanced, and validated, transforming the build experience.
Many of the automation examples above rely on predefined tasks and workflows, but IBM Research is also building generative interfaces for automation tasks that users want to create on the fly. They could write in plain English what they need done — rather than clicking away on pages looking for the specific tasks they’re after.
These systems can invoke APIs, sequence multiple APIs, query databases with natural language, and summarize retrieved content, just using IBM’s Granite LLM. We expect technology like this to build on some of the early success already experienced with watsonx Orchestrate by the likes of Sports Clips who have witnessed several orders of magnitude reduction in time to execute HR tasks — from hours to minutes.
This is where the industry is heading. The closer we can get to having simple conversations with the software and machines that we use to complete our work every day, the faster we can get more done.
