Industry Insights
Blog
Explainable AI:
If Your AI Can’t Explain Itself, You’re the One Liable
Using AI in workplace decisions? Learn why explainable AI is becoming essential for WHS compliance, transparency, and reducing organisational risk.
Share this Article:
Artificial intelligence is moving fast inside Australian workplaces. Businesses are using it to screen job applicants, monitor productivity, flag safety risks, allocate work, and inform return-to-work decisions. In many cases, the AI is making recommendations that materially affect people’s livelihoods and wellbeing.
There’s just one problem. In a significant number of those cases, nobody — not the vendor, not the employer, not the HR manager clicking “accept” — can clearly explain why the AI recommended what it did.
That’s not just a technology problem. Under Australian law, it’s increasingly a liability problem. And the regulatory environment is closing in fast.
What "Black Box" AI Actually Means
Most AI tools operate as what researchers call a “black box.” You put data in one end, a recommendation comes out the other, and the process in between — the features the model weighted, the patterns it identified, the factors it prioritised — is invisible.
This works fine when the stakes are low. It becomes a serious problem when the AI is recommending whether someone is fit to return to work, whether a safety risk warrants intervention, or whether an employee should be performance-managed.
The insurance industry has grappled with this directly. Without being able to interpret how AI algorithms work, companies have no way to justify AI decisions — the system provides a view of the input and output, but reveals nothing of the process in between. What’s true for insurance claims is equally true for workplace health decisions.
The Regulatory Environment Is Catching Up — Quickly
For Australian businesses, this is no longer a theoretical concern. The legal and regulatory landscape has shifted materially in the past eighteen months.
The Australian Senate — November 2024
The Senate Select Committee on Adopting Artificial Intelligence tabled its final report in November 2024, recommending a fundamental shift away from the current voluntary, principles-based approach toward mandatory obligations for high-risk AI. Two recommendations are directly relevant to employers.
Recommendation 5 called on the Government to ensure the definition of high-risk AI clearly includes any AI that impacts the rights of people at work. Recommendation 6 called on the Government to extend and apply the existing WHS legislative framework to the workplace risks posed by AI adoption.
Employers have been specifically advised to assess procedures currently in place to maintain transparency and explainability of decisions — with the clear implication that “the AI recommended it” is not a sufficient answer.
NSW — World-First Legislation, February 2026
New South Wales became the first Australian jurisdiction to introduce specific workplace health and safety legislation governing digital work systems, including artificial intelligence, algorithms, automation and online platforms.
The Work Health and Safety Amendment (Digital Work Systems) Act 2026 was passed by the NSW Parliament on 12 February 2026 and assented to on 18 February 2026, amending the Work Health and Safety Act 2011 (NSW) to create explicit duties for businesses using digital work systems.
Under the amended legislation, businesses must identify, assess and manage health and safety risks arising from digital work systems used to allocate, monitor or manage work. This includes considering risks associated with excessive workloads, performance monitoring, workplace surveillance and discriminatory decision-making.
It is worth noting that many of the operational provisions are subject to proclamation dates, with guidance material still being developed by the NSW Government. Other states are expected to follow the NSW lead.
The EU AI Act — Global Benchmark
Internationally, the EU AI Act — adopted in 2024 and applying fully by 2026 — classifies AI systems used in employment contexts as high-risk, requiring them to be transparent and explainable.
Penalties for non-compliance reach €30 million or 6% of global turnover.
While this applies to EU operations, it sets the global standard that Australian regulators are actively watching.
What Has Gone Wrong Elsewhere
The concern isn’t hypothetical. Real-world cases illustrate precisely what can happen when AI makes consequential decisions that nobody can explain or defend.
In the United States, Mobley v. Workday Inc. — currently before the federal courts — involves allegations that an AI applicant screening tool trained on existing workforce data replicated the employer’s existing biases around age, race and disability.
In Australia, a case involving an AI security screening unit illustrated a different failure mode: the system flagged individuals as requiring further investigation, but the secondary human review process was inadequate.
In the insurance sector — one of the most advanced adopters of AI for claims and risk assessment — a fraud-detection AI flagged loyal, long-term customers as fraudsters. The error wasn’t caught until significant customer damage had been done, because nobody could interrogate the model’s reasoning in real time.
In each case, the organisation deploying the AI carried the liability. The vendor being sued doesn’t change that.
The Core Legal Principle: AI Doesn't Carry Liability. You Do.
This is the point that many businesses using AI tools are yet to fully absorb.
Under Australian WHS law, the primary duty of care sits with the Person Conducting a Business or Undertaking (PCBU). That duty is not transferable to a software vendor.
If an AI tool you are using makes a recommendation that contributes to harm, and you cannot explain why that recommendation was made or demonstrate that you applied appropriate scrutiny before acting on it, you are exposed.
The Australian Institute of Health & Safety has also highlighted the importance of ensuring AI is implemented in a way that protects worker health and wellbeing.
The direction of travel is clear: using AI does not reduce your duty of care. In some respects, it increases it.
What "Explainable AI" Looks Like in Practice
Not all AI is a black box. Explainable AI — sometimes called XAI — is designed to show its working. Rather than simply outputting a recommendation, it surfaces the features that drove that recommendation, the relative weight of each factor, and the confidence level of the output.
In a workplace health context, this means a return-to-work recommendation doesn’t just say “cleared for modified duties.” It tells you that the recommendation was influenced by recovery trajectory over the past three weeks, functional capacity assessment scores, the nature of the role’s physical demands, and comparable cases with similar injury profiles. You can see why. You can interrogate it. You can override it with clinical judgement when the circumstances warrant.
A 2024 McKinsey survey found that 40% of AI leaders cited explainability as their top concern in deploying AI. In regulated industries, the principle is increasingly settled: AI should assist expert decision-making, not replace it — and when the AI is uncertain, it should escalate to a human, not proceed.
How Employ Health Approaches This
At Employ Health, we use AI to support workplace health decisions — and we are deliberate about how.
Every AI-assisted recommendation in our practice comes with an explanation of how it was arrived at and which features contributed to the output. Our consultants can see what the model weighted, why, and to what degree. That transparency serves two purposes: it makes the recommendation genuinely useful to the clinician or consultant reviewing it, and it means that if a decision is ever scrutinised — by a regulator, a court, or a client — we can explain every step.
We don’t use AI to replace clinical or professional judgement. We use it to inform and accelerate it. The human remains in the loop, with full visibility of the reasoning behind the tool’s output.
This isn’t just our ethical preference. It’s the standard we believe the law is moving toward — and the standard we think your organisation should be asking of every vendor who uses AI to inform decisions about your people.
Questions to Ask Your Workplace Health Provider
If you’re working with a workplace health provider — or any vendor using AI to inform decisions about your workforce — these are the questions worth asking:
Can you explain how your AI arrived at this recommendation?
If the answer is a variation of “the algorithm determined it,” that’s not an explanation. Push for specifics.
What features or data points drive the output?
A credible explainable AI system can tell you exactly which inputs moved the needle and by how much.
What is the human review process?
AI recommendations should be reviewed by a qualified professional before being acted upon. Ask what that process looks like and where the override mechanism sits.
Has your AI been tested for bias?
Models trained on historical data can replicate historical patterns — including discriminatory ones. Ask whether bias testing is part of the vendor’s development and ongoing review process.
What happens when your AI is uncertain?
Well-designed systems escalate low-confidence outputs for human review. Ask what the confidence threshold is and what happens when it isn’t met.
The Bottom Line
The era of deploying AI in the workplace and treating it as a neutral, unaccountable tool is ending. Australian legislation — especially in NSW — is now explicit that algorithmic and AI-driven decisions about workers carry WHS obligations. The Senate has recommended extending that framework nationally. The courts are beginning to hear cases.
If you’re using AI to inform decisions about your people’s health, safety, fitness for work, or employment conditions, you need to be able to explain those decisions. Not in principle. In practice. Decision by decision.
At Employ Health, that’s the standard we hold ourselves to — and the standard we think every employer deserves from their partners.
EXPLAINABLE AI FOR WORKPLACE HEALTH
Build Confidence in Every AI-Assisted Decision
Employ Health is a workplace health organisation committed to building healthier, safer, and higher-performing workplaces across Australia. Our team are full-time Employ Health employees — not contractors — deeply invested in the outcomes of the organisations we work with for the long term.
If you’d like to understand how we approach AI in workplace health decision-making, or to review your current WHS obligations in this area, we’d welcome the conversation.
This article is intended for general informational and educational purposes only. It does not constitute legal advice and should not be relied upon as such. The legal and regulatory landscape around AI and workplace health and safety is evolving rapidly. We recommend seeking independent legal advice regarding your organisation’s specific obligations and circumstances.
Table of Contents
Read the latest articles
Workplace Health & Wellbeing, Industry Insights, Workplace Safety
Is Your Remote Work Policy a WHS Risk? The Evidence Says It Might Be
Remote work offers flexibility, but new research suggests it may also increase isolation and psychological
Workplace Health & Wellbeing
From Descriptive to Prescriptive: The Evolution of Pre-Employment Risk Intelligence
Financial stress is no longer just a personal issue — it’s impacting workplace wellbeing, productivity,
Data and Technology, Injury Prevention, Workplace Safety
How AI and Machine Learning Are Transforming Onsite Physiotherapy and Workplace Health
Financial stress is no longer just a personal issue — it’s impacting workplace wellbeing, productivity,