Legislative proposal to criminalise unsafe AI chatbots

The question

Under what circumstances would an AI chatbot be deemed sufficiently unsafe to trigger criminal sanctions under the proposed amendment to the Crime and Policing Act 2026 (the Amendments)?

The key takeaway

Companies involved with the development and supply of chatbots may be held responsible for the private use of their products in the conduct of illegal and harmful activity. Though there is no immediate risk for AI developers, it may be prudent for companies to regularly assess the source material used to train their models and vigorously stress test the safeguards implemented to prevent their chatbots being used for illegal purposes.

The background

On 18 March 2026, the House of Lords approved a set of amendments to the Crime and Policing Bill that would make the development or supply of a chatbot that produces certain content a criminal offence.  The Bill will proceed to the House of Commons, where the Amendments are expected to encounter Government opposition. This is because the Government favours addressing the problem by extending the regulatory regime set out in the Online Safety Act 2023 to artificial intelligence services, rather than through criminalisation.

The development

The Amendments draw a distinction between content shown to all users and content shown to users under the age of 18. Where a user is of adult age, the scope of the offence is limited to content that promotes terrorism, violence or threats to national security or public safety.

However, where a user is under the age of 18, the scope is extended to include "content and activity that is harmful to children" as guided by the Online Safety Act 2023. This includes pornographic content, and content which encourages, promotes or provides instructions for suicide, deliberate self-injury, eating disorders and behaviours associated with such disorders, serious violence and bullying.

The Amendments place the onus on companies to carry out risk assessment and address identifiable risks prior to a chatbot being made publicly available in the UK. The consequences proposed for a breach of the Amendments could see offenders face financial penalties, and potentially, a prison sentence of up to five years. 

Why is this important?

The proposed Amendments will provide a telling litmus test on the Government's stance on the extent to which it is prepared to hold companies responsible for the use and impact of their products.

In the U.S., there is clear appetite to explore the possibility of placing responsibility at the feet of companies. Following a recent mass shooting, prosecutors in Florida have issued criminal subpoenas to OpenAI and opened investigations into whether the company could be held criminally liable for the information provided through the shooter's use of the platform in planning and carrying out an attack at Florida State University. The Florida Attorney General James Uthmeier (R) has weighed in, stating that "if that bot were a person, they'd be charged with a principal in first degree murder."

If the Amendments pass the House of Commons without the force of the wording being diluted, it would send a clear signal to industry that, in the UK, technology companies developing and distributing products with widespread societal impact will be expected to rigorously assess and address potential negative consequences; raising the broader question – to what extent can we reasonably expect companies to foresee the impact of their technologies?

Any practical tips?