Several steps can be taken to alleviate high levels of risk, and of these, the ones that stand out for consideration include agent identity, comprehensive logs, policy checks, human oversight, rapid revocation, the availability of documentation from vendors, and the formulation of evidence for presentation to regulators.

There are several options decision makers can consider that will help create the record of activities undertaken by agentic systems. For example, a Python SDK (software development kit), Asqav, can sign each agent’s action cryptographically and link all records to an immutable hash chain – the type of technique that’s more associated with blockchain technology. If someone or something changes or removes a record, verification of the chain fails.

For governance teams, using a verbose, centralised, possibly-encrypted system of record for all agentic AIs is a measure that provides data well beyond the scattered text logs produced by individual software platforms. Regardless of the technical details of how records are made and kept, IT leaders need to see exactly where, when, and how agentic instances are acting throughout the enterprise.

Many organisations fail at this first step in any recording of automated, AI-driven activity. It’s necessary to keep a registry of every agent in operation, with each uniquely identified, plus records of its capabilities and granted permissions. This ‘agentic asset list’ ties neatly into the requirements of the EU AI Act’s article 9, which states:

• Article 9: For high-risk areas, AI risk management has to be an ongoing, evidence-based process built into every stage of deployment (development, preparation, production), and be under constant review.

Furthermore, decision-makers need to be aware of the Act’s Article 13:

• High-risk AI systems have to be designed in such a way that those deploying them can understand a system’s output. Thus, an AI system from a third-party must be interpretable by its users (not an opaque code blob), and should be supplied with enough documentation to ensure its safe and lawful use.

This requirement means the choice of model and its methods of deployment are both technical and regulatory considerations.

It’s important for any agentic deployment to offer a facility for the revocation of an AI’s operating role, preferably within a matter of seconds. The ability to revoke quickly should be part of emergency response processes. Revocation options should include the immediate removal of privileges, immediate ceasing of API access, and the flushing of queued tasks.

The presence of human oversight, combined with the presentation of enough context for humans to make informed decisions, means that human operators must be able to reject any proposed action. It’s not considered adequate for the person reviewing a decision to see only a prompt or a confidence score. Effective oversight needs information around context, every agent’s authority, and time enough to intervene to prevent mis-steps.

While every agent’s action should be recorded automatically and retained, multi-agent processes are particularly complex to track, as failures can take place among chains of agents. It’s therefore important for security policies to be tested during the development of any system that intends to utilise multiple agents.

Finally, governing authorities may require logs and technical documentation at any time, and will certainly need them after any incident they have been made aware of.

The question to be considered by IT leaders considering using AI on sensitive data or in high-risk environments is whether every aspect of the technology can be identified, constrained by policy, audited, interrupted, and explained. If the answer is unclear, governance is not yet in place.

(Image source: “Last Judgement” by Lawrence OP is licensed under CC BY-NC-ND 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/2.0)

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and co-located with other leading technology events. Click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

Financial services and technology show the most movement from pilot to production, with many implementations in core business functions. The sector can be considered as having the most clear line of sight from automation to measurable returns in terms of income. The practical inference from the report’s findings would be for slower-moving sectors to copy the implementation workflows employed by the fintech industry: Start with narrow, high-volume workflows where performance can be measured and failures can be contained, and focus on the IT function.

According to the survey, generative AI-assisted development is now common in nine of the ten countries surveyed, alongside traditional coding, outsourced development, and SaaS customisation. It undercuts the notion that enterprises are moving into an AI-native or all-AI stack. In fact, most organisations add agents and AI-generated code on top of the processes already proven effective in their development environments.

OutSystems finds that 48% of respondents see integration with legacy systems as the most important ability needed to expand agentic AI, and 38% say legacy systems are the main reason projects stall between pilot and production. Of the potential barriers to AI development that were offered as choices to the survey’s participants, more than 40% cited integration difficulties and legacy fragmentation the most problematic.

Organisations considering large data clean-up programmes (which many AI vendors advocate as a reason why deployments fail to reach production) may want to rethink, the report implies. The authors state agents can be built that can work well in complex data environments, as long as governance and integration are strengthened at the same time as AI implementation. Across the board, most sectors express “moderate trust” levels of agentic AI at around 50%, although responses from different business functions were not broken out in the survey results’ figures.

The financial returns are manifest mostly in IT functions themselves. The report says the most explored use cases are IT operations, at 55%, and data analysis, at 52%. Workflow automation follows at 36%, then customer experience at 33%. On realised return on investment, IT development and productivity lead by a margin, at 40%, ahead of operational efficiency at 22%. That distribution suggests that the first durable value from agentic AI is internal at developers’ desks rather than in customer-facing environments. Customer-facing deployments may still make sense, but the report indicates they require more trust in system performance, stronger controls, better orchestration, and an ability to create watertight oversight mechanisms.

Trust in agentic AI, however, is improving. OutSystems reports that 73% of respondents express either high or moderate trust in letting agents to act autonomously, a rise of around 10% compared to a similar survey the company undertook last year. Trust in code or workflows generated by third-party AI tools is slightly lower, at 67%, a substantial increase from the prior year’s figure, when only 40% ‘mostly trusted’ generative AI to write code without human help.

Only 36% of respondents say they have a centralised approach to AI governance, while 64% say they lack such a facility, and 41% rely on rules implemented on a per-project basis. Two-thirds say building human-in-the-loop checkpoints is technically difficult because it requires orchestration that can pause agents – in effect inserting manual braking on operations that might be fully autonomous.

Many organisations appear to be deploying looser oversight models, although it is not clear if that is a result of greater trust in models or whether business functions are under pressure to deploy AI regardless of security or reliability concerns. If the trend to loosen oversight continues, the report’s authors note that agentic AI adoption may advance faster than the methods of accountability that many consider important.

Firms that want to scale agents in regulated or mission-critical settings should treat orchestration and auditability as part of the product, the survey’s findings state. When compliance checks consider a business’s operations, breadcrumb trails in the form of logfiles and defined responsibilities are considered important elements of any agentic AI rollout.

The report says 94% of leaders are concerned about “AI sprawl”, which is not defined, but could be inferred to be a lack of a centralised management platform that oversees all AI deployments in the enterprise. 39% are very or extremely concerned about the issue, and only 12% currently use a centralised platform to keep that sprawl under control.

The full survey can be accessed here.

(Image source: “Relax” by Koijots is licensed under CC BY-SA 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by-sa/2.0)

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and co-located with other leading technology events. Click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.