As a quick recap, the issues are whether:

• the way foundation models (FM) are trained using works from the internet infringes the copyright in the works of content creators such as authors, artists and software developers
• the outputs of FM infringe the copyright of content creators
• AI generated works are protectable

The problem with training data

Copyright is a right that in the UK and EU subsists automatically when certain requirements are met. Copyright infringers must be found to have copied the whole of the copyright work, or part of it where that part is regarded as ‘substantial’. Both proof of copying from a copyright work and similarity are required to prove infringement.

Content creators such as news providers, authors, visual content agencies and other creative professionals allege that their work is being unlawfully used to train AI models. Some use of this material is expressly authorised, for example, Associated Press has previously announced that OpenAI has taken a licence of part of its text archive. However, the main thrust of the allegations by content creators is that millions of texts, parts of texts and other literary material and images have been scraped from publicly available websites without consent.  This scraped content used as an input to train and develop AI models is alleged to infringe their copyright and often their database rights. 

The case of Getty Images (US) Inc v Stability AI Ltd, which went to trial in June 2025, is the most prominent case making these kinds of allegations in the UK (there is also a corresponding US action). Setting aside the arguments on territorial extent raised in that case (i.e. whether the training and development of Stable Diffusion took place within the UK or in another jurisdiction), the allegations of copyright and database right infringement relevant here were that Stability AI:

• has downloaded and stored Getty Image's copyright works (necessary for encoding the content and other steps in the training process) on servers or computers in the UK during the development and training of Stable Diffusion
• infringed the communication to the public right by making Stable Diffusion available in the UK, where Stable Diffusion provides the means using text and/or image prompts to generate synthetic images that reproduce the whole or a substantial part of the copyright works.

These claims of copyright and database right infringement formed a sizable chunk of the proceedings. They were withdrawn during the trial which ultimately ended up focusing on trade mark infringement. However, the copyright and database right allegations provide information on how these types of claims can be argued. In its claims, Getty alleged that Stable Diffusion was trained using subsets of the LAION-5B dataset, a dataset comprising 5.85 billion CLIP-filtered (Contrastive Language-Image Pre-training) image-text pairs, created by scraping links to photographs and videos, together with associated captions, from the web, including from Pinterest, WordPress-hosted blogs, SmugMug, Blogspot, Flickr, Wikimedia, Tumblr and the Getty Images websites. The LAION-5B dataset comprises around 5 billion links. The LAION subsets together comprise approximately 3 billion image-text pairs from the LAION-5B dataset. At the time of filing its original claim, Getty had identified around 12 million links in the LAION subsets to content on the Getty Images websites.

The training and use of FM has resulted in intense debate on the infringement questions and the adequacy of legislation and/or guidance on licensing. Other similar ongoing legal actions (all US based) include:

• the New York Times action against OpenAI and Microsoft in the US, for unlawful use of journalistic (including behind paywall) content to train LLMs 
• a class action filed against OpenAI by the Authors Guild (now consolidated with two other actions) and some big-name authors including George RR Martin, John Grisham, and Jodi Picoult, alleging that the training of ChatGPT infringed the copyright in the authors’ works of fiction
• in September 2025, AI company Anthropic agreed to pay $1.5bn to settle a US copyright class action over its use of pirated texts to train its AI model, Claude. The action was brought by authors who claimed that Anthropic had downloaded 465,000 books and other texts from “pirated websites." The settlement is awaiting court approval.

One of the issues for publishers and content creators is that they are not being rewarded for the use of their content to train AI models and that use of LLMs such as ChatGPT disrupts the business model of consumers who search online via a search engine for content, no longer being directed to publications on their websites where the traffic attracts revenue made through digital advertising. This is because a search for digital content on many AI systems results in a direct response that stays within the LLM or image generation platform even though that response may be drawing from the same content that would have been revealed in search results in the search engine example. 

In 2023, OpenAI provided written evidence to a UK committee inquiry into large language models including an explanation of its position on the use of copyright protected works in LLM training data. It explained that its LLMs, including the models that power ChatGPT, are developed using three primary sources of training data: (1) information that is publicly accessible on the internet, (2) information licensed from third parties (such as Associated Press), and (3) information from users or their human trainers. OpenAI acknowledged because "copyright today covers virtually every sort of human expression – including blog posts, photographs, forum posts, scraps of software code, and government documents – it would be impossible to train today’s leading AI models without using copyrighted materials". OpenAI stressed that it was for creators to exclude their content from AI training and that it has provided a way to disallow OpenAI's “GPTBot” web crawler access to a site, as well as an opt-out process for creators who want to exclude their images from future DALL∙E training datasets. It also mentioned its partnerships with publishers like Associated Press. 

In January 2024, in what might be interpreted as the beginning of a shift by AI providers, OpenAI's CEO Sam Altman at the World Economic Forum, Davos said that OpenAI was open to deal with publishers and that there's a need for "new economic models" between publishers and generative AI models. Licensing deals now appear to be becoming more prevalent. Examples include collaborations between OpenAI and each of: Condé Nast, Guardian Media Group and the Financial Times.  

When licensing negotiations break down there is a risk of legal action being taken, such as that reportedly being taken by Mumsnet against OpenAI. In its complaint against OpenAI, Mumsnet claims "scraping without permission is an explicit breach of our terms of use, which clearly state that no part of the site may be distributed, scraped or copied for any purpose without our express approval." As one of the few cases involving allegations of unlawful website scraping by AI developers in the UK, the outcome will be important in developing the law in this area – if the case reaches trial.  

Training data issue resolution – UK government 

Since our March 2023 Generative AI and intellectual property rights piece covering the UK's current position and reforms relating to a proposed commercial text and data mining exception, there has been no significant legal milestone reached on text and data mining in the UK:

• In January 2024, the previous government's Culture, Media and Sport Committee confirmed that the government is no longer proceeding with its original proposal for a broad copyright exception for TDM. 
• A voluntary code of practice (promised by the Intellectual Property Office "by summer 2023") to provide guidance to support AI firms in accessing copyright protected works as an input to their models and to provide protections (e.g. watermarking) on generated output,  did not materialise. 
• In the February 2024 response to its consultation on the 2023 AI whitepaper, the previous government acknowledged that the stalemate between AI companies and rights holders on the voluntary code of practice led the IPO to return the task of producing the code to the Department for Science Innovation and Technology (DSIT). DSIT and DCMS then reengaged with the AI and rights holder sectors without resolution of this complex global issue. 
• In December 2024, the government launched a consultation proposing as a "preferred option" a new commercial data mining exception to support use at scale of a wide range of content by AI developers where rights have not been reserved. The exception would provide a mechanism for right holders to ‘opt-out’ (individually or collectively) and supporting measures on transparency to ensure developers are transparent about the works their models are trained on. Rights reservation would be by accessible and, where possible, standardised machine-readable formats (examples include "robots.txt," TDMRep, the International Standard Content Code (ISCC), and C2PA). The consultation closed on 25 February 2025 and the government is reviewing more than 11,500 responses. 
• In January 2025, the House of Lords introduced a series of detailed amendments to the Data (Use and Access) Bill, during the bill's Report stage.  During the debate, the amendments were described by Baroness Kidron as designed to set out how a copyright regime would work: "Amendment 61 would ensure that all operators of web crawlers must comply with UK law if they are marketed in the UK. Amendments 62 and 63 would require operators to be transparent about their identity and purpose, and allow creatives to understand if their content had been stolen. Amendment 64 would give enforcement powers to the ICO and allow for a private right of action by copyright holders. Amendment 44A would require the ICO to report on its enforcement record. Finally, Amendment 65 would require the Secretary of State to review technical solutions that might support a strong copyright regime." [Hansard]
• The Data (Use and Access) Bill was subject to a considerable amount of Parliamentary ping pong between the Lords and the Commons with the revised amendments each time being stripped out by the government. The Data (Use and Access) Act 2025, as enacted, does not set out a copyright and AI regime, however Part 7 of the Act requires the government to publish an economic impact assessment considering each of the four policy options described in the Copyright and AI consultation (s 135). The assessment must consider the impact on both copyright owners and AI system developers and users.  The government must also prepare and publish a report on the use of copyright works in the development of AI systems.  The report must include proposals on technical measures and standards that may be used to control the use of copyright works to develop AI systems and the accessing of copyright works for that purpose eg by web crawlers (s 136).

Since March 2023, for large developers, collaboration through structured voluntary licensing agreements has continued to grow steadily and it has been reported that the government, as part of its plan, is looking to provide support to licensing markets for all sizes of developers.     

In the EU

The EU AI Act entered into force on 1 August 2024 and is largely applicable by August 2026. The text provides for general-purpose AI (GPAI) systems such as ChatGPT, and the GPAI models they are based on (such as OpenAI's GPT-4), to have to adhere to transparency requirements.  These include drawing up technical documentation explaining how the model has been trained, how it performs, how it should be used and its energy use, complying with EU copyright law (in particular to obtain authorisation from or enable content owners to opt out from the text and data mining of their content as provided for under the EU DSM Copyright Directive) and disseminating "sufficiently detailed" summaries about the content used for training GPAI including its provenance and curation methods. A voluntary code of practice designed to help GPAI model providers who sign up to demonstrate compliance with their obligations under the EU AI Act, was published and approved in the summer of 2025. The code consists of three chapters: Transparency and Copyright, both addressing all providers of general-purpose AI models, and Safety and Security, relevant only to a limited number of providers of the most advanced models.

Notably, on the question of GPAI model providers identifying and respecting opt out rights, this will be done under the EU AI Act using methods including "state of the art" technologies and: "Any provider placing a general-purpose AI model on the Union market should comply with this obligation, regardless of the jurisdiction in which the copyright-relevant acts underpinning the training of those general-purpose AI models take place. This is necessary to ensure a level playing field among providers of general-purpose AI models where no provider should be able to gain a competitive advantage in the EU market by applying lower copyright standards than those provided in the Union." (EU AI Act, Recital 106)

The detailed summaries should be comprehensive enough to allow rights holders to be able to exercise and enforce their rights, for example by listing the main data collections or sets that went into training the model, such as large private or public databases or data archives, and by providing a narrative explanation about other data sources used. The EU AI office, responsible for the implementation and enforcement of the EU AI Act, will provide a template.

The EU AI Act obligations relating to providers of GPAI models' compliance with EU copyright laws are applicable from August 2025 (for new models, and August 2027 for GPAI models placed on the market before 2 August 2025). However, the exception for text and data mining provided for under the EU DSM Copyright Directive that allows content owners to opt out from bulk scraping of their online content is proving challenging to apply in practice. There is currently no standard protocol to enable machine readable "opt out" or to expressly reserve rights.

To assist with this issue, the European Commission is currently working on developing a licensing market for the training data used to train AI systems like ChatGPT.  Some copyright owners such as Sony Music, the Society of Authors and the Creators’ Rights Alliance are pre-empting the EU AI Act (while providing a warning within the UK) by publicly reserving their rights in relation to text and data mining via a statement on their website and/or in a letter to various companies including AI developers.

The output of FMs – works created by users

As well as the possibility of training data related copyright infringements explained above, the outputs of AI FM models such as ChatGPT or Midjourney generated as the result of user prompts may also provide grounds for copyright infringement of third party original works.

For example, if you are the author of an artwork and find a markedly similar copy has been generated by a user of a FM model without your permission, you will have to make your case on copyright infringement. In showing infringement, as a first step, proof of copying of features from the protected work is required.  Then the question is whether what has been taken constitutes all or a substantial part of the copyright work. A challenge with user generated works will be to show that the output work was derived from the original copyright protected work (did the AI provider introduce it in its training data, was it introduced during the fine tuning process or did a user provide it as one of its prompts). The EU AI Act somewhat provides for this (see above) by allowing the copyright owner to see if their work is contained in a particular data set. The UK has also indicated that transparency requirements are likely to be coming.

In this scenario the users of FM (and/or AI providers) face potential liability for copyright infringement.  These claims may be low value, and challenging to prove for rights holders so this might be a low risk, but it nevertheless produces risk for AI users and providers. Consequently, a number of key players (Microsoft, Google and OpenAI) now provide offers to indemnify certain (mainly enterprise) users if they are subsequently sued for copyright infringement. Microsoft's Customer Copyright Commitment states that if a third party sues a commercial customer for copyright infringement for using Microsoft’s Copilots or the output they generate, they will defend the customer and pay the amount of any adverse judgments or settlements that result from the legal proceedings, as long as the customer has used the guardrails and content filters built into their products. OpenAI's "Copyright Shield" promises to step in and defend their customers, and pay the costs incurred, if they face claims of copyright infringement. This applies to generally available features of ChatGPT Enterprise and their developer platform. Note: some of these indemnities may include carve-outs and liability caps. 

Protection for the outputs of AI FM models

Most public facing generative AI models are accessed via a platform or website and are therefore subject to website terms and conditions. ChatGPT states that: "Ownership of Content. As between you and OpenAI, and to the extent permitted by applicable law, you (a) retain your ownership rights in Input and (b) own the Output. We hereby assign to you all our right, title, and interest, if any, in and to Output." 

What is actually being assigned is an important consideration for businesses and individuals. For example, there seems to be high use of AI FM in the advertising sector. If you, as a user, have produced marketing materials with the assistance of a FM you are likely to want to prevent their unauthorised use by third parties as a normal part of your business' brand/content protection strategy.  This would not normally be problematic if they are created without AI FM assistance – then the copyright likely belongs to the company concerned as the employer of the employee author. However, most jurisdictions, including the UK, require that copyright protection only applies to works created by human authors and if the work is solely computer generated there may be a subsistence issue. This is because authorship and ownership of copyright is tied into the concept of "originality", that is, protection is only extended to works categorised as "original literary, dramatic, musical or artistic works". The work may of course be attributed to the developer of the FM in circumstances where the user's role is confined to a single simple prompt and the FM has been finely tuned to produce marketing materials – in this situation there are likely to be terms that assign the developer's rights in works to the end user. 

In this scenario, the section of the Copyright Designs and Patents Act 1988 (CDPA 1988) that grants protection to computer-generated works (CGWs) is often raised. Section 9(3) states that the author in the case of CGW is the person by whom "the arrangements necessary for the creation of the work are undertaken". The problem with this section relates to the date of the Act: 1988. What the legislators may have had in mind at this time is something like the use of computers as digital aids in cartography. Now, however, this section is being applied to GenAI models. 

However, since 1988, there have been some developments when it comes to "originality". The test for originality has changed and now to be an original work, works must be "the author's own intellectual creation" whereby an author has been "able to express their creative abilities in the production of the work by making free and creative choices so as to stamp the work created with their personal touch…" That definition is not very CGW/AI friendly. Where works are created by entering prompts into a GenAI system (i.e. using it merely as a tool) there would be room to apply the "author's own intellectual creation" originality test. However, literary, dramatic, musical or artistic CGW are more problematic under this originality test if a work has no human author. Therefore, in order to claim authorship and ownership, squeezing out the human element may be the best approach until clarification is provided from the UK government or the courts. The position is not clear cut though and if you are creating content for a client, the Ts & Cs relied on historically for human authored work may not be effective in transferring absolute ownership. 

In November 2023, a Chinese court did find that an AI generated image, created using Stable Diffusion, satisfied the requirements of "originality" and was capable of copyright protection.  The Beijing Internet Court found that the image had been created (using AI as a tool) in a way that reflected the ingenuity and original intellectual investment of human beings. In February 2023 in a US case concerning authorship of the images contained within Kristina Kashtanova's work: Zarya of the Dawn, the US Copyright Office took a different approach. The images were developed using the generative AI tool Midjourney. By its own description Midjourney does not interpret prompts as specific instructions to create a particular expressive result (Midjourney does not understand grammar, sentence structure, or words like humans) it instead converts words and phrases “into smaller pieces, called tokens, that can be compared to its training data and then uses them to generate an image. The US copyright office decided that the images claimed were not original works of authorship protected by copyright because they were produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author (the designer modifying the images produced by the AI model using subsequent prompts and inputs was not sufficient to fulfil the requirement for human creativity). They were therefore removed from the U.S. Copyright Office register as not copyrightable. Because of the significant distance between what a user may direct Midjourney to create and the visual material Midjourney actually produces, the U.S. Copyright Office found that Midjourney users are deemed to lack sufficient control over generated images to be treated as the “master mind” behind them.

In January 2025, the US Copyright Office registered "A Single Piece of American Cheese", an image created entirely with AI generated material via a technique called “inpainting” (the process of selectively modifying or regenerating parts of an image while maintaining consistency with the surrounding elements). The work was initially rejected but was later registered on the basis of active “selection, coordination, and arrangement of material generated by artificial intelligence” into a unified composition. In part the decision seems to reflect the amount and quality of evidence of creative decision making in creating the image. In its application, the registrant highlighted: the multi-stage process, iterative refinement and creative decision making elements in creating the work.   

How might these issues impact those developing and interacting with FM?

This is a complex area and tricky to navigate in a commercial setting given that the UK and many other jurisdictions are failing to reach a position and provide guidance.  However, it is worth keeping up to date on and in mind the following live issues: 

• the risk surrounding the use of data sets  
• that there may be a need to disclose the contents of data sets under the EU AI Act and the UK framework
• who owns FM outputs? Is an AI output as protectable as a human created work? 

Discover more insights on the AI guide

FMs can help businesses and individuals to improve communication, analyse data and automate tasks. They underpin the natural language processing chatbot ChatGPT, image generation tools such as Midjourney and the use of generative AI in productivity software.

The type of data a FM is exposed to during training will determine its 'type'.

• Large language models (LLMs) are FMs which are trained on text data.
• Image generation models are FMs trained on image data (in addition to text data).
• Multi-modal FMs are FMs trained using several different data sources.

Developing and training FMs

There are a number of steps required to develop, train and deploy a FM.

Pre-training

In the pre-training stage, the training data is collated from different sources and usually examined to allow for the extraction of harmful or irrelevant data. The data is commonly taken from publicly available sources through a process of web crawling, using open datasets and/or using proprietary data.

The datasets are then tokenised, which involves dividing the data into billions of small 'tokens'. In an LLM, each token may represent a word or parts of a word, whereas in image generation models, a token may represent smaller components of an image (such as a pixel). Through a process called 'self-attention', the model can weigh up the importance of each token and determine the probable relationships between them such as between the words cat, kitty and feline.

The model learns how to produce accurate outputs during the training process by using the parameters it is exposed to from the datasets to adjust its calculations. Sometimes referred to as "weights", a parameter is a connection chosen by the language model and learned during training.

FMs apply learned patterns from one task to another (transfer learning).

Fine-tuning

Fine-tuning is an optional next phase in which a pre-trained model receives additional capabilities or improvements using specific datasets. The main types are:

• Alignment: fine-tuning a model to align its behaviour with the expectations or preferences of a human user for example to enable it to create music that matches certain moods. In order to prevent biased, false or harmful outputs, a machine learning technique known as reinforcement learning from human feedback (RLHF) is used to fine tune AI models by incorporating human preferences into their decision-making process. It involves humans rating or ranking the model's outputs, which is then used to train the model to produce responses that align more closely with human expectations or desired behaviours. RLHF trains the model to make decisions that maximise "rewards". The rewards function relies on humans feeding back which responses they prefer, to distinguish between wanted and unwanted behaviour. The response-rating preferences build a reward model that automatically estimates how high a human would score any given prompt response. This reward model is then applied to a (language) model to allow it to internally evaluate a series of responses and then select the response most likely to result in the greatest reward, optimising human preferences.  RLHF can be provided by paid contractors or directly from users.  Alignment is also used to teach the model to ‘speak like a machine’ so as not to mislead users. Examples of human-machine conversations from existing chatbots can be collated and used to fine-tune a pre-trained model to add this capability. 
• Domain or task specific: fine-tuning a model to a specific domain or task using smaller, specialised datasets. For instance, a dataset containing legal documents could improve a model's ability to prepare legal documents or provide advice.
• Synthetic data: fine-tuning a model using artificially generated data, such as data from simulations, real data which has been artificially extended or new datasets created from existing AI models. While developers benefit from the lower cost of acquiring synthetic data at large scales (compared to human data), there is the potential risk of 'model collapse', where defective data from existing FM models pollute the generated synthetic data.

Inference

At the inference stage of the development process, the user feeds new inputs into the model, which then uses its parameters to create a prediction. An inference refers to the process of models making predictions based on the new data received. It provides a test of how well the model can apply information learned during training to make a prediction.

For example, a customer using a customer service chatbot powered by an LLM such as ChatGPT inputs: "I need help with returning a damaged product." The chatbot then analyses the input, identifies key elements (e.g. "returning" and "damaged product"), and uses its trained parameters to predict the most appropriate response. It replies: "I’m sorry to hear that. Could you please provide your order number so I can assist you with the return process?" This demonstrates how the model applies its training to understand the context and generate a relevant, human-like response.

Open vs closed source models

FM developers may choose to develop and release a FM in either open or closed source.

An open-source FM model such as Meta's LLaMA can be shared widely and is free to use, subject to a licence (although the licence may prohibit commercial use). A licensee may be provided with the original FM code, model architecture, training data and potentially even the weights and biases, allowing them to mirror the training process and/or to fine-tune the FM without needing to go through the pre-training process.

A closed-source FM model such as ChatGPT is developed privately. Access to the model is usually restricted and controlled by those within the company, for example. Rather than releasing externally, these models are likely to be used for the company’s own initiatives and operations.

Evaluation methods – testing the performance of FMs

FM developers typically evaluate their own FMs to analyse their capabilities or to identify falsities in outputs. Different evaluation methods include:

• Evaluating against static datasets of input-output pairs, which assess performance against a wide-range of criteria such as accuracy, multitask ability and robustness.
• Model based evaluation, where one or more other models are used to evaluate the FM.
• Using human raters who are asked or paid to carry out model specific evaluation tasks. This is considered to be the gold-standard for evaluation, but it can make comparison exercises across models and papers more difficult due to the tailored nature of the tasks and the different evaluation methods adopted by raters.
• The process of red teaming involves experts using deliberately misleading questions to identify faults. Red teaming is important because it can help to identify vulnerabilities or biases in the model.

How do businesses access FMs in downstream markets? 

There are already many areas where FMs are, or will likely become, incorporated into downstream markets. Downstream businesses can access FMs by:

Creating and developing a FM in-house to support the business' needs and objectives. Whilst this option offers businesses full control over an FM, the technical, costly and time-consuming development process means it is unfeasible for many businesses.

Collaborating with an established third-party FM provider to develop an existing FM. This may allow the business to fine-tune the FM with its own data and, in turn, take ownership over the fine-tuned FM. This would be a cheaper option but still requires businesses to invest money, time and expertise for the FM's development. 

Purchasing application programming interface (API) access to a FM and FM deployment tools owned by a third party. This option is often much cheaper and faster to implement than developing a FM in-house. However, businesses will not have the opportunity to tailor the FM to business needs and will be reliant on a third-party product.

Offering a third-party FM plug-in to enhance services and extend functionality. For example, a business may opt to provide a plug-in which allows users to use FM based service such as ChatGPT. This is a very accessible option for businesses seeking to reap the benefits of incorporating an FM into its products and services without the cost, expertise and time required by other options.

¹ This summary contains public sector information licensed under the Open Government Licence v3.0, specifically the Competition & Markets Authority's AI Foundation Models Initial Report dated 18 September 2023.

Discover more insights on the AI guide

The UK government has consistently said that it would adopt a pro-innovation and business-friendly approach to regulating AI. There is currently no AI-specific legislation in the UK. The government has been preparing its AI Bill but no draft has been released as yet. The AI Bill was intended to target the "most advanced AI models" and make existing voluntary commitments between companies and the government legally binding but these plans have changed. The government now plans to introduce a comprehensive AI bill in the next parliamentary session that will address safety and copyright issues.

In the absence of the AI Bill, guidance can be found in the government's White Paper published on 29 March 2023 and updated in its response to the White Paper in February 2024. Key elements of the White Paper are:

• Five values-focused cross-sectoral principles for regulators to interpret and apply within their respective domains, intended to promote responsible AI use (see The Ethics of AI – the Digital Dilemma for more information about the principles)
• No new AI regulator – instead existing regulators, using context-specific approaches will guide AI development
• Central support and monitoring via a steering committee to coordinate regulators

Four key regulators are leading the way on implementing the AI principles under the umbrella of the Digital Regulation Cooperation Forum (DRCF): the Information Commissioner's Office (ICO), Ofcom, the Competition and Markets Authority (CMA) and the Financial Conduct Authority. The DRCF had set up the AI and Digital Hub, via a pilot, to advise on AI regulatory compliance in a coordinated way. Its initial term terminated in April 2025.

These four regulators also have their own approach to AI regulation – see table below.

Further policy, tools and guidance for organisations will come from bodies such as the: (i) AI Security Institute; (ii) the AI Policy Directorate; and (iii) the Responsible Technology Adoption Unit.  

In January 2025, the government published its AI Opportunities Action Plan to ramp up AI adoption across the UK. Key initiatives include new AI Growth Zones to build more AI infrastructure, increasing the public compute capacity 20x, and creating a new National Data Library to harness the value in public data and support AI development.

Lastly, a Private Members' Artificial Intelligence (Regulation) Bill was introduced to the House of Lords in March 2025. The Bill aims to create an AI Authority that would collaborate with relevant regulators to construct regulatory sandboxes for AI. 

UK data protection law applies to AI as it would any other technology. Companies therefore need to ensure that they meet GDPR standards when processing personal data in the context of any AI used in their business. Although the over-arching legal framework is the same, the nature of AI technology means that businesses are presented with new privacy-related risks that need to be addressed. We set out in this section 10 key questions to ask yourself at the outset when developing or deploying AI solutions in your business. Additional general considerations for off-the-shelf AI solutions are set out here.  

1. What will your AI system do?

It is a basic question – but you need to understand the purpose of your AI system to understand why and how you will be processing personal data in the context of the system. The ICO considers AI-related processing to be high risk and therefore a data protection impact assessment (DPIA) should be carried out to assess the privacy-related risks posed by your system and your proposed mitigations. 

Where you are exploring the use of AI or engaging in pilot projects, the purpose of your AI system may not be apparent or may change over time. Similarly, your purposes for processing personal data may change over the lifecycle of the AI system, for example, training the system vs using the system to make decisions. At the same time, you need to consider the lawful basis for processing data in your AI system, at each stage of the AI system, and for each category of personal data processed by your AI system. This, too, can change as your AI system evolves such that you may need a different lawful basis or, if you were relying on the legitimate interests lawful basis, you may now need to perform a new legitimate interests assessment. Updating your DPIA regularly is a good way to capture changes and assess the impact of such changes on your compliance. 

You should also consider whether the data processing carried out by your system is necessary and proportionate, or if there are potentially more privacy-preserving alternatives. Ultimately, your system should incorporate data protection by design and by default.

2. What are your roles and responsibilities?

Many companies are working collaboratively with tech providers to explore AI's potential for their business. This, and the inherent complexity of AI systems, mean that the usual analysis of whether parties are acting as controller or processor becomes a lot more challenging. You should be clear about who is responsible for making decisions about different aspects of the AI model (including if any are made jointly) and at which stage of the AI lifecycle (e.g. pre-training vs further training and configuration). This will then drive your obligations under the law and the contractual framework you should put in place to document the rights and obligations of the parties e.g. data processing agreements and data sharing agreements. 

3. What will be used as input data?

AI systems need vast amounts of data for training purposes. You will need to consider the various data sets that will be input into your AI system (as part of training, configuration or regular use) and if you are compliant in respect of each of them. How much personal data and special category data exists in those data sets? Are you able to anonymise data before it is ingested into the system (thereby taking it outside the scope of the GDPR) or will this significantly impact the accuracy of the model? Are you able to use synthetic data sets to train the system instead as these present lower privacy risk? Bear in mind, however, that lower quality data will have an impact on the accuracy of the AI system. Whilst AI systems need not be 100% accurate, they should not be statistically flawed. Poor data may also impact fairness – discussed further below.   

4. What will be the output data?

Consider the types of data that will be generated by your AI system, and your GDPR obligations in respect of each of them. The power of AI systems to analyse and find patterns across massive data sets also means that you may be processing personal data without expecting to do so. For example, where you use AI systems to make inferences about individuals or groups, and the inference relates to an identifiable person, this may be personal data or even special category data depending on the circumstances.   

5. What are the data flows?

AI models require extensive processing power and are therefore typically hosted by the AI solution provider. Consider how personal data moves through your AI system. Is it sent from your systems to the provider, and if so is it commingled with other customers' data, or are you able to retain it in your own 'walled garden' instance of the AI model? Is it transferred out of the UK? If so you will need to ensure that a transfer mechanism under the GDPR has been put in place and a transfer risk assessment has been carried out. Note that the risks of any transfer are heightened because of the volumes of data processed by the AI system. Consider also how long you retain data and if this aligns with your data retention policies. 

6. How do you ensure your AI system is fair?

Your AI systems (and any decisions made by them) must be fair and must not produce outcomes which are discriminatory or biased against individuals. Bias can occur at multiple points in the AI lifecycle and may not be apparent. For example, data sets that reflect historical biases or lack data for certain categories of data subjects may result in AI models being inadvertently trained to perpetuate bias. Therefore, you must plan to mitigate the risk of bias from the outset, for example, assessing the quality and neutrality of data inputs, engaging with a broad range of stakeholders to identify bias, and mapping out the potential effects of AI decisions on minority groups.

7. Will you be carrying out automated decision-making?

Article 22 of the GDPR restricts automated decision-making that produces legal or similarly significant effects for data subjects without any meaningful human involvement. If your AI system is likely to do this, you will need to assess the decision being made and how human involvement should be incorporated into the process for it to have meaningful effect. EU case law (that is influential on the UK regulator) also shows that "decision" may be interpreted broadly and can encompass even interim acts that play a determining role in the final decision. You should also ensure that any employees involved in the decision understand the importance of their review and that it is not merely a 'tick box' exercise. 

8. How do you keep your AI system secure? 

The sheer volumes of data used by any AI system exponentially increases the risk of a data breach. Any existing technical and organisational measures you implement to keep your systems secure must be updated to protect against novel security risks faced by AI systems (see here for examples of these). You will need to adopt a more holistic approach to systems security as your AI solution will no doubt be integrated with various internal and third party systems. As industry standards around AI security are still being developed, ensure you remain current with the prevailing best practice from time to time.  

9. What must you tell data subjects?

You will need to be transparent with data subjects as to how their personal data is processed by your AI systems. This may be difficult to do, as AI decision-making is sophisticated and frequently opaque. For this reason, you must deliberately design your AI system to be explainable and understandable by your data subjects and you should describe how the decisions made by your AI system are fair and avoid bias. Consider producing an explainability statement alongside your privacy policy to set out this information. 

10. How do you ensure data subject rights?

Your AI system must be designed to accommodate the data subject rights enshrined in the GDPR throughout the project lifecycle, for example, the rights to access, rectification, and erasure. This may be challenging depending on the volume of data processed, and if data sets are modified or commingled for training purposes. However, you would still need to take reasonable steps to comply with the data subject's request. Similarly, if there are challenges in embedding data subjects' right to withdraw consent (due to the time and effort needed to 'untrain' models) then you must consider whether consent is a feasible lawful basis to rely on in the first place. 

In addition to asking yourself these 10 questions, ensure you follow guidance produced by your relevant data protection regulators. In the UK, the Information Commissioner has produced various sources of guidance on developing, deploying and using AI, including the "AI and data protection risk toolkit", guidance on explaining decisions made with AI and the ICO's response to its generative AI consultation. The ICO has also promised a 'single set of rules' on AI and data protection which would include details on areas that could not be covered in the consultation. In respect of the EU GDPR, the European Data Protection Board has produced an opinion on AI models and a risk management methodology for managing privacy risks. 

Discover more insights on the AI guide

Ethical principles tailored to the responsible design and use of AI systems have been put forward by a number of different organisations across the globe.  Whilst there is not a single agreed version of these ethical principles, they tend to follow similar themes and highlight a number of potential harms which should be considered in the development and use of AI tools. 

The "starting five" principles

The UK government's AI White Paper published on 29 March 2023 lists five values-focused cross-sectoral principles for regulators to interpret and apply within their respective domains, intended to promote the ethical use of AI:

• safety, security and robustness
• appropriate transparency and explainability
• fairness
• accountability and governance
• contestability and redress.

The government confirmed the same principles in their response to the consultation to the AI White Paper which was published on 6 February 2024 (Consultation Response). 

These principles are pivotal to the government's approach to regulating AI in the UK.  They provide a framework which the regulators must apply, with the intention of allowing them to do so in a proportionate and agile manner given the level of risk which is determined by where and how AI is used in a particular context. See Part 1 - AI Regulation in the UK for information about the UK's regulatory approach.  

We will explain each of the key principles in turn, noting the unavoidable interplay between the various principles and the possible harms that they seek to address.

Safety, security and robustness.   AI systems must be safe, and should be designed and deployed, and operate throughout the lifecycle, in a manner that minimises the risk of harm to individuals and society.  These harms could be deliberate or accidental, and may affect individuals, groups, organisations or even nations, and may take various forms, such physical, psychological, social, or economic harms.        

AI systems must be technically secure, and be protected against unauthorised access, manipulation or attacks that could lead to harmful outcomes.  Security measures are crucial to protect against malicious use, such as spreading misinformation, stealing personal data or disrupting critical infrastructure.  Developers should consider the security threats that could apply at each stage of the AI life cycle and embed resilience to these threats into their systems.

As for robustness, AI systems should be reliable and perform consistently and as intended under a wide range of conditions. They should be able to handle unexpected situations or inputs without failing or producing erroneous outcomes.  This includes being resilient to changes in their operating environment and being able to recover from errors or failures. 

Transparency and explainability. The principle of transparency refers to the need for information about an AI system to be communicated to relevant stakeholders.  This means that an appropriate level of information about the processes, decisions and operations of an AI system should be accessible and comprehensible, both to the developers and engineers who designed the system but also to the end users and other stakeholders who may be affected by its use.  Explainability refers to the ability to interpret and understand the decision-making processes of an AI system.  Otherwise, the inability to see how deep-learning systems make decisions creates what is known as the 'black box problem'. Opacity in decision-making is problematic in several ways, including difficulties in diagnosing and fixing issues and its potential to reflect or amplify societal or dataset biases without the business deploying the AI knowing. In practice, however, explainability may be easier said than done in some cases, as the logic and decision-making in AI systems cannot always be meaningfully explained in a way that can be understood by humans.  This could involve simplifying complex models, using visualisations or providing simplified rules that approximate the model's decision making-process.

Transparency and explainability should be proportionate to the risks that an AI system may present, but in any event is necessary to afford regulators and stakeholders sufficient visibility of, and information about, an AI system and its inputs and outputs to give effect to the other ethical principles (for example, for regulators to identify accountability and for individuals who may have been affected by an AI decision to challenge the decision and seek redress if necessary). 

Fairness.  Fairness includes identifying and mitigating biases to prevent discriminatory outcomes caused by AI systems, and ensuring the use of AI systems does not undermine the legal rights of individuals or organisations.  In order to do so, fairness needs to be considered in every aspect of AI. This would include:

1. data fairness – AI systems can inadvertently learn and in turn perpetuate or amplify societal biases through biased training data or algorithmic design, and so only fair and equitable datasets should be used, or training examples should be re-weighted if required
2. design fairness – using reasonable features, processes, and analytical structures in the AI architecture
3. outcome fairness - preventing the system from having any discriminatory impact
4. implementation fairness - implementing the system in an unbiased way.

Ensuring fairness in AI also involves adhering to legal standards and ethical norms.  Fairness is a concept which underpins many areas of law and regulation, such equality and human rights, data protection, consumer and competition law, and many sector-specific regulatory requirements (for instance the consumer duty and other consumer protections in the financial services sector). This means that AI systems should comply with anti-discrimination laws and ethical guidelines to promote justice and prevent harm.

Accountability and governance. It is crucial to have clear lines of responsibility for the decisions made by an AI system, in order to be able to identify and hold responsible the relevant parties for the decisions and any harm or unintended consequences that arise as a result of the use of AI systems.  This is essential for creating business certainty (such as allocating liability in an AI supply chain) while also ensuring regulatory compliance.   

Appropriate governance frameworks should be in place to oversee the supply and use of AI systems, incorporating ethical guidelines and standards for AI development and usage.  Assurance techniques such as impact assessments may assist in identifying risks early in the development life cycle, which can in turn be mitigated through appropriate safeguards and governance mechanisms. 

Once in use, regular auditing of AI systems to ensure they operate as intended and adhere to the required standards and in compliance with the ever-changing regulations and standards can also be useful.  Engaging with a wide range of stakeholders (experts but also those potentially impacted by AI systems) can also help to shape robust ethical AI governance, identify and avoid potential ethical issues, and spot opportunities to improve ethical standards and practices in AI. Retaining a "human in the loop", by using AI in such a way that it does not replace human judgement and decision making, but rather augments it, is also vital. 

Ethical AI governance should very much be seen as an ongoing process; as AI technologies and their societal impacts evolve, governance frameworks should also adapt. 

Contestability and redress.  This principle of contestability refers to the ability of users and affected parties to challenge and seek rectification for decisions made by AI systems that impact them.  This is particularly important when these decisions have significant consequences on people's lives.  For AI decisions to be contestable, the systems need to be transparent about how decisions are made.  Mechanisms through which challenges can be made also need to be provided. This could include user interfaces for feedback, human oversight where decisions can be reviewed and clear processes for escalating concerns.

Redress involves correcting wrong decisions and where necessary providing avenues for affected individuals to seek compensation or other remedies in cases where the individual believes they have been unfairly treated by an AI system or it otherwise causes harm.  Beyond addressing individual grievances, redress also involves taking feedback and challenges to improve the AI system.  This could mean retraining models with more diverse data, adjusting algorithms to eliminate biases, or refining decision-making processes.  Effective redress mechanisms are usually supported by robust policy and legal frameworks that define the rights of individuals and the obligations of AI developers and deployers.  These frameworks can provide guidelines for the types of redress available and the procedures for seeking it.

The UK’s non-statutory approach to date has meant that new rights or new routes to redress have not been implemented and it's unclear whether the proposed AI Bill will include any. See Part 1 - AI Regulation in the UK for more details. 

Moving forward

Implementing these ethical principles is complex and multifaceted, particularly in the face of a regulatory regime that is still taking shape; it requires ongoing effort, multidisciplinary collaboration and continuous evaluation and adaptation of AI systems as technology and societal norms evolve.

Discover more insights on the AI guide

AI providers have been focussing on their forthcoming AI obligations and on governance for some time, but it is now prudent for the majority of organisations to assess how their use of AI will come within the scope of regulation in key territories, become familiar with each regime, and devise a means to keep up with anticipated changes.

A plan for action that will be applicable for most businesses includes:

• building in compliance costs - the approach to AI regulation across jurisdictions currently appears so varied that organisations need to factor the costs of compliance into their strategy for the jurisdictions that they plan to provide or deploy AI in;
• implementing AI governance including systems and procedures for data retention and record keeping;
• assessing existing product and service lines and removing or adjusting products or services that use AI in a way that is prohibited or high risk, especially in the EU;
• identifying trusted advisors from the "noise" of what is being offered externally; and
• building internal AI expertise including by providing training to allow individuals to perform their roles and/or use the AI system in a way that is consistent with related policies and procedures - see here for our recommendations on training your staff on AI.

In addition, AI providers should establish written policies, procedures, and instructions for various aspects of the AI system (including oversight of the system) and produce documentation explaining the technicalities of their AI model and its output. They should assess and document the likelihood and impact of any risks associated with the AI system, including in relation to privacy and security.

Where appropriate businesses might consider using voluntary commitments in their relevant industry sector.  In December 2023, in the US, 28 healthcare companies agreed to voluntary commitments on the use and purchase of safe, secure and trustworthy AI.

Lastly, as discussed in Part 5 – AI regulation globally, ISO/IEC standards (such as ISO 23894 or ISO/IEC 4200 1:2023) can be used as tools to support the safety, security and resilience of AI systems and solutions.

This is Part 5 of 'Regulation of AI'

There have been various initiatives for countries around the world to cooperate on AI regulation, including knowledge sharing and securing commitments from tech providers.

International agreements

Currently, there is only one legally-binding international treaty – the Council of Europe's convention on AI. This treaty, signed by the US, EU and UK on 5 September 2024, creates a common framework for AI systems with three over-arching safeguards:

• protecting human rights, including ensuring people’s data is used appropriately, their privacy is respected and AI does not discriminate against them
• protecting democracy by ensuring countries take steps to prevent public institutions and processes being undermined
• protecting the rule of law, by putting the onus on signatory countries to regulate AI-specific risks, protect its citizens from potential harms and ensure it is used safely

On 30 October 2023 the G7 published its international guiding principles on AI, in addition to a voluntary code of conduct for AI developers. The G7 principles are a non-exhaustive list of guiding principles aimed at promoting safe, secure and trustworthy AI and are intended to build on the OECD's AI Principles, adopted back in May 2019.

Global summits

There have been three global AI summits. In November 2023, the UK Government hosted the first – titled the AI Safety Summit. The summit brought together representatives from governments, AI companies, research experts and civil society groups from across the globe, with the stated aims of considering the risk of AI and discussing how they can be mitigated through internationally co-ordinated action. One output from the UK's AI Safety Summit was the Bletchley Declaration focused on international collaboration on identifying AI safety risks and creating risk-based policies to address such risks. Another output was an agreement between senior government representatives from leading AI nations and major AI developers and organisations (including Meta, Google DeepMind and OpenAI) to a plan for safety testing of frontier AI models.

The second global AI summit was held in Seoul in May 2024. The institutes of 10 countries and the EU signed the Seoul Declaration with commitments to cooperate more between themselves and via organisations such as the UN, G7, G20 and OECD, while sixteen AI firms made voluntary safety commitments.

In February 2025, the AI Action Summit was held in Paris. Over 1000 participants from over 100 countries attended the summit which focused on the key themes of inclusive and environmentally-sustainable AI. The Statement on Inclusive and Sustainable Artificial Intelligence for People and the Planet was signed by 60 countries but not the US or UK.

Standards

AI-related standards have been published by the International Organisation for Standardisation (ISO) and the International Electrotechnical Commission (IE). In its response to the white paper, the UK government mentions specifically the importance of engaging with global standards development organisations such as the ISO and IEC. The most prominent AI ISO/IEC standards are:

1. ISO/IEC TR 24028:2020 that analyses the factors that can impact the trustworthiness of systems providing or using AI
2. ISO/IEC TR 24368:2022 on the ethical and societal concerns surrounding AI
3. ISO/IEC 23894:2023 which offers strategic guidance to organisations across all sectors for managing risks connected to the development and use of AI. It also provides guidance on how organisations can integrate risk management into their AI-driven activities and business functions
4. ISO/IEC 42001:2023 which specifies requirements for establishing, implementing, maintaining, and continually improving AI management systems within organisations

In addition, relevant standards have also been published by: (i) the British Standards Institution including PD CEN/CLC TR 18145:2025 which provides guidance on sustainable AI technologies; and (ii) the Institute of Electrical and Electronic Engineers including IEEE 3119-2025 on Procurement of AI and Automated Decision Systems.

Types of AIaaS

The most common types of AIaaS are:

• Chatbots. Prior to the advancements in AI, chatbots provided answers to a predefined set of questions. However, chatbots are now powered by LLMs and, using natural language processing, can closely mimic actual human speech and deal with a wider array of issues.
• Application Programming Interface (APIs). APIs create a communication pathway between the AI model and an organisation's internal systems.  For example, a computer vision API could give existing software access to the ability to process and analyse images.
• Machine Learning (ML). Access to ML frameworks allows businesses to use ML to analyse big data, identify trends and make predictions. For example, an on-demand video provider may use ML to serve a consumer with film and tv recommendations based on their previous viewing habits.

Benefits of AIaaS

There are several clear benefits to using AIaaS. Firstly, pre-trained, off-the-shelf solutions are far more affordable than any solution that requires significant development work. A team of data scientists and engineers to build and maintain the solution are not required, nor is the purchase of extensive processing power to run a model in-house. Secondly, AIaaS avoids the need to invest significant time and effort to train any model, including training by people fine-tuning the solution. A basic service could theoretically be used 'out of the box' with minimal configuration. Lastly, AIaaS solutions (like most cloud-based solutions) are flexible, scalable, and designed to be integrated with standard enterprise IT systems. Customers have the option to turn features on or off and to scale up or down according to usage. 

Key issues when contracting for AIaaS

Many of the contracting issues that apply to SaaS services generally will also apply in respect of AIaaS. However, several take on a new dimension due to the nature of AI and the current market practice around AIaaS.

• Implementation. Many tools will be advertised as 'out of the box'. That said, consider if any additional training or configuration work is required for the tool to operate as required. Much will also depend on the system architecture that the AIaaS will be used with. AIaaS, being relatively new tech, may require work to be integrated with legacy IT systems. 
• Usage obligations. In a normal SaaS context, customers would need to comply with obligations around usage (e.g. acceptable use codes) and businesses could mitigate this risk internally through employee policies and training.  AIaaS providers also require customers to comply with acceptable use codes however some go further than others.  Many AIaaS terms require businesses to ensure that they do not generate prohibited content using the service. This is a harder risk to mitigate - customers could set safety filters and policies around user prompts but it's impossible to predict what exactly would be generated. Many AIaaS tools also contain technical usage restrictions as generative AI is extremely resource intensive. If usage exceeds those limits the provider can block access to the service.   
• Service performance. Does the provider offer clear assurances as to minimum standards for the service? Some AIaaS products are currently being sold in "preview" mode so do not include fixed service levels. Or providers may afford assurances for some performance standards (e.g. availability) but not others which may be less certain to them (e.g. response time). Consider therefore if this is sufficient for the intended use case – perhaps it's prudent to only experiment with the service rather than use it for any critical business purpose.  
• Liability. SaaS products are typically provided 'as is' with limited warranties and liability on the provider. AIaaS is no different, and although the market practice here is still unclear it seems so far that generally speaking, the warranties and liability caps proffered by providers give customers less protection than in many SaaS arrangements. It has also been well publicised that large AIaaS providers currently provide customers with an indemnity against third party IP rights infringement. However, many of these are capped and only apply to their in-house models. Early adopters might be able to negotiate better bespoke terms with providers.
• Pricing. SaaS is typically priced as pay-as-you-use which is convenient but poses the risk of unforeseen costs if an organisation does not have robust internal governance as to usage. This risk is exacerbated for AIaaS because of how new and untested the tech might be for a business. For example, there are various pricing models for AI chatbots but most charge per prompt.  If chatbots are new to a business, it may be difficult to accurately estimate how many prompts would be needed. Accidental overage may also trigger penalties. Similarly, an unexpected need for the provider to render additional professional services (e.g. to deploy or configure the service) may arise.  
• Regulatory compliance. There is typically a lack of transparency with SaaS - the underlying infrastructure and processes are not generally made available to the customer. However, a key principle in AI regulation (including under data protection laws) is explainability and transparency i.e. understanding how the model works and relaying this to end users (see also AI and Ethics – the Digital Dilemma). Ensure sufficient documentation is given by the provider to be able to comply with regulatory obligations.
• Security. Cyber security is a known issue for SaaS. However, the sheer volumes of data sent to AIaaS and potentially stored offshore significantly increase the security risk. AI has also resulted in new types of security attacks (see Procuring AI – Commercial Considerations Checklist) and threat actors will take advantage of businesses' delay to implement measures that keep up with these developments. Complying with the most up-to-date security standards and frameworks is one way to lower these risks.

Discover more insights on the AI guide

1. Create and comply with company AI use policies

It is advisable to develop clear policies and training around the development and use of AI across the business. For example, do you have strong data governance processes around the access, storage, and transfer of the data sets used to ensure data quality, integrity and accuracy? And do you have a robust use case testing procedure to ensure that AI is only used where appropriate, given the potential harms and risks associated with its use?  Of course, policies only work to the extent you monitor compliance with them regularly.

2. Define your strategy and budget

A focused strategy is particularly important when procuring an AI solution. AI is impressive but many AI-powered solutions have not been tested on a large scale. Any AI solution you procure will likely need significant on-boarding and fine-tuning for it to work as planned. 

For this reason, consider how exactly you intend to integrate the AI solution in your business and where there are likely to be opportunities to maximise benefits in the short term, and how the AI can be leveraged in the longer term. Be clear on your available budget for this project and factor in sufficient time, money and human resource for the procurement, deployment and on-going training and maintenance of the solution. If the AI solution works as planned, is the anticipated return on investment acceptable?  

3. Scope your requirements

Unlike a standard IT procurement, you may not be able to define waterfall-style specifications for your intended solution. Many AI solutions will need a period of iterative experimentation, training and tweaking before the functionality is properly developed. Furthermore, you would not want to blinker yourself into working off narrow requirements when there might be greater opportunities available. Instead, develop problem statements, challenges, opportunities and use cases that you intend the AI solution to address. 

You should also assess the relevant data sets that you intend to use for the project. Where is the data sourced from? Do you have a licence to use the data for the project or might you be breaching confidentiality restrictions? Do the data sets include personal data (see AI and Privacy – 10 Questions to Ask for further guidance)? Are there any limitations (e.g. quality) to this data that need to be addressed first?  On what basis will you share data with the provider for use on the AI system (if any)? Can you use synthetic or anonymised data for training purposes to avoid issues with the data or to fill data gaps?

Conduct an initial impact assessment to determine the key risks of the solution on your business and whether these can be mitigated. Will the AI be used with other software, and if so, do you have the appropriate rights and licences to do so from the relevant third party software suppliers? Is the use and/or commercial parameters of the other software still appropriate when being used with the AI given, amongst other things, automated processing? Will your insurance cover apply where AI is used?

4. Upskill your teams

Any successful AI project requires that customer and provider teams work cooperatively to develop the solution. You will need to create your own multidisciplinary team of experts to advise you through the procurement lifecycle including legal and commercial experts, technologists, data and systems engineers, and ethics advisors. Consider what training you might need to get these teams up to speed with AI developments and considerations – see here for more on AI literacy. A diverse team is recommended to minimise blind spots and unintended bias. 

5. Choose a provider

It is critical to do your homework on the provider and the AI product in question and not simply be swayed by the sales pitch. Over the next months and years we will see AI solutions and providers consolidate, and some others fail, so it's important not to back the wrong horse. Review any potential provider's project team to ensure they are diverse, multidisciplinary, and have the right skills and qualifications. Alternatively, is the best way to source the AI solution through a combination of providers and, if so, have you considered the integration risk between these various systems?

6. Key contract issues

Consider the following key issues when contracting with your chosen provider. 

Contract structure. Consider structuring the contract in phases to allow for discovery and development. A 3-6 month pilot phase (longer or shorter depending on the complexity of the project) may be appropriate. Ensure that the overall contract is aimed towards flexibility, the ability to change, and iterative product development.   

Use of data. The contract should specify the customer data sets that the provider will have access to, and what the provider can do with such data. Do you need the provider to remedy any limitations with the data before using it? Will the solution have access to the internet or will it be a "walled garden"? Will your data be used to train the model generally for the benefit of the provider's other customers? Will you benefit from any new fine-tuning or user feedback data the provider applies to its model generally? 

Training the model. How will the parties train the AI solution? Consider a governance framework that sets out the parties' responsibilities for each aspect of the training. How long will the solution need to be trained before it can go live? Will the provider continue to train the solution on a regular basis post go-live?

Intellectual Property. General purpose LLMs will have been trained using data obtained from web scraping, the IP implications of which are still being debated (see Generative AI - Addressing Copyright). The provider may also carry out further training based on owned or licensed data sets. Seek warranty and indemnity protection that your use of the solution and any output does not infringe third party IP rights. Consider also if you need to own the IP in any output and if the provider should be required to assign the IP in such material to you. Note, however, that the legal position on copyright in AI-generated works is still unclear (see [Generative AI - Addressing Copyright]).   

Testing and acceptance. Prior to go live, the provider should be required to test the AI solution's functionality within the customer's IT environment. Seek to include specific metrics to determine when an AI system is ready to go live. Consider what tools are available to confirm that, under the bonnet, the AI is also working as intended. This might be through audit rights or verification tools offered by your provider or the right to have an independent review or by using other third party tools to make that assessment. In any case, repeated testing and validation will need to be an ongoing process that continues through implementation.

Performance of the solution. Consider the service standards and service levels you require of the AI solution and endeavour to make these objective and quantifiable. Providers should be required to comply with internationally-recognised standards on AI systems, for example, ISO/IEC 42001 that provides a certifiable AI management system framework focused on strong AI governance (see also Part 5 - AI Regulation Globally for more on international standards).  The contract should also include an agreed process for the provider to investigate and fix errors and hallucinations. Output should be tested for discrimination and unfairness, and to ensure that the tool and outputs comply with ethical principles – see The Ethics of AI – The Digital Dilemma. 

Over time, service levels and standards should increase as the model improves. Similarly, any concept of "Good Industry Practice" (and an obligation on the provider to comply with it) will evolve as the industry adapts to the new technology.  Benchmarking by third parties will be important to periodically confirm that the AI solution remains comparable to other models. Considering the breakneck speed at which AI is developing, the provider should be required to continually improve its solution and ensure it is state of the art. Consider how new updates will be applied to your solution.   

Collaboration and governance. The success of the project will depend on whether the parties are able to work openly and collaboratively whilst understanding their respective roles and responsibilities. AI solutions learn from humans (see here for more on reinforced learning from human feedback) so there should be a process for the provider to gather feedback from the end users so that the solution improves. Regular governance meetings (more frequent during the development phase) are also crucial to ensuring that the project stays on track and that issues are dealt with early and swiftly. The provider should also be required to implement and maintain appropriate policies and processes to ensure that the AI system operates responsibly, ethically and in compliance with the law. 

Explainability and transparency. You must be able to explain how your AI system works as you will need to demonstrate that AI is used responsibly and appropriately (see AI and Ethics – The Digital Dilemma for more on explainability). For an AI system to be explainable, you will need the provider to provide you with detailed information on:

• how the AI works
• how it has been trained and the sources of training data 
• how it was tested
• how it has been designed to be fair
• the logic behind the output of the AI. 

Records should be produced of how the AI system was developed including the parameters that represent the model's learnt 'knowledge' used to produce the intended output. The solution should also be designed to log information regarding how decisions are made, so that these can be verified and explained if necessary. This information will allow you to meet your transparency obligations under law (e.g. to your data subjects and see here for more on data protection considerations). 

Risk allocation. At this stage, the market is still getting to grips with AI and has yet to develop established positions on risk allocation. AI providers are building their customer base and working to make their offering more attractive. For this reason, customers who contract early may be in a stronger position when negotiating liability clauses under their contracts. In any case, risk allocation will depend on the commercials and each party's role and responsibilities in relation to the project. For example, a customer is unlikely to get blanket indemnities from the supplier for third party IP infringement claims if a significant portion of the training is carried out by the customer using the customer's own data.  

Security. Aside from the standard security issues that would arise in any tech procurement (e.g. data encryption, user controls etc), there are certain security threats which are novel to AI systems. These include prompt injection (bad actors instructing the model to perform actions you don't intend), prompt stealing (accessing a user's prompt), model inversion attacks (attempting to recover the dataset used to train a model), membership inference attacks (attempting to determine if certain information was used to train a model), and data poisoning (tampering with data sets). The provider must ensure that its solution is appropriately secure against known threats and have robust processes to address future threats. At the very least, solutions should comply with generally-accepted standards on cyber security, for example, the National Cyber Security Centre's Guidelines for Secure AI System Development – see (see also Part 5 – AI Regulation globally for more on standards). There are also specialist escrow providers with whom you may store input data, algorithms, and AI applications to ensure that the solution remains available should the provider unexpectedly cease operations. 

Compliance. Consider who bears the risk and the cost of compliance with AI regulations as they change from time to time. New regulations may result in significant changes needing to be made to the solution, potentially down to the underlying infrastructure level. A regulatory change control procedure should also be included to set out a process by which the parties may agree and implement required changes and allocate the costs of the same.  

Exit. Although not particular to AI solutions, vendor lock-in is a risk that is heightened when procuring AI due to its complexity. You minimise this risk if you and your potential replacement providers are able to understand how the solution works. The incumbent provider should be required to train your personnel and ensure knowledge transfer over the lifecycle of the project. At the outset, consider the interoperability of the solution you procure with other suppliers' models, software, and systems. Consider also the data you will need to migrate the solution to a replacement provider upon exit from the contract.

Discover more insights on the AI guide

Going forward, "providers" of AI systems, created either from scratch or built on top of tools and services provided by others, and "deployers" (i.e. a natural or legal person using an AI system under its authority, but not in a personal non-professional capacity) need to know what they can and cannot do in the design, development, procurement, deployment and operation of their AI systems. Understanding the national and international landscape is key to them being able to formulate an AI strategy. For example, UK companies that deploy AI systems or use AI powered tech in or targeted at the EU will come within the scope of the EU AI Act.  

Despite a growing and complicated web of overlapping global standards and alliances, these providers and deployers will be operating in an AI market regulated on a territory by territory basis. Some jurisdictions, like the UK have adopted a balanced pro-innovation approach to attract investment and development of AI in the UK. However, it's difficult to see how this approach fits with the UK's close proximity to the EU and the EU's pro-regulation approach. And on the other hand, the UK is keen to align with the US' extreme pro-innovation stance. Providers intending global expansion, however, may decide to meet the higher EU regulatory obligations to streamline their compliance requirements which could lead to the EU establishing an international AI standard as it has arguably done with data protection and the GDPR.  

More details on AI regulation are set out in:

• [Part 1 – AI Regulation in the UK]
• [Part 2 – AI Regulation in the EU]
• [Part 3 – AI Regulation in the US]
• [Part 4 – AI Regulation in Asia]
• [Part 5 – AI Regulation globally]
• [Part 6 – Practical considerations]

Discover more insights on the AI guide

In its AI White Paper published in March 2023, the UK government avoided a rigid definition of AI on the basis that it may quickly become outdated and restrictive, given the pace of developments in the technology.  Instead, the government sought to define AI by reference to the two key characteristics of AI that give rise to the need for a bespoke regulatory response, namely adaptivity and autonomy.  The ‘adaptivity’ of AI can make it difficult to explain the intent or logic of the system’s outcomes.  AI systems operate by inferring patterns and connections in data, and quite often the logic and decision-making in AI systems cannot always be understood, or meaningfully explained in a way that can be understood, by humans.  Furthermore, AI systems may develop the ability to perform new forms of inference not anticipated by the developers of the system.  Some AI systems can make decisions without the express intent or control of a human.  This ‘autonomy’ gives rise to many concerns, one of which is that it becomes difficult to assign responsibility for outcomes caused by AI systems.  

By contrast, the EU AI Act defines an AI system as "a machine-based system that … infers from the input it receives how to generate outputs such as predictions, content, recommendations, or decisions that can affect physical or virtual environments". 

Technically, AI is often an umbrella term used to describe a range of technologies, from simple rule-based algorithms to complex neural networks mimicking the human brain. A significant turning point in AI has been the development of sophisticated Large Language Models (LLMs).  These models have revolutionised natural language processing, demonstrating capabilities in generating human-like text, translating languages and even coding.  AI's ability to interpret, understand and classify visual data has also seen remarkable growth, as has AI-driven automation.  These are only a number of examples of AI systems which are in use, and each reflects a part of the diverse landscape of current AI capabilities.  

We have seen already how far AI technology has come to date.  In fact, it has already become deeply integrated into various aspects of modern life. Moving forward, how far will (and should) it go?

Discover more insights on the AI guide

The American approach to AI regulation has changed significantly with the new Trump administration. President Biden had signed an Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence in October 2023. In January 2025, President Trump revoked President Biden's Order and signed an Executive Order on Removing Barriers to American Leadership in Artificial Intelligence (the Trump Order). 

The Trump Order is framed as eliminating unnecessarily burdensome requirements put in place by the Biden Order that hindered the US' ability to innovate and requires US departments to rescind any policies and actions taken under the Biden Order that are "inconsistent with enhancing America's leadership in AI". The Trump Order also calls for the development of an AI action plan within 180 days. 

Federal agencies such as the National Institute of Standards and Technology (NIST) has produced guidance on AI including the AI Risk Management Framework (AI RMF 1.0), for organisations designing, developing, deploying, or using AI systems. It is unclear to what extent NIST will continue these activities under the Trump Order. 

Recently, Trump has also proposed the One Big Beautiful Bill act, a budget reconciliation bill that includes a 10-year moratorium on enforcing state-level regulation on AI.

Several states have passed legislation to regulate AI. In California, Assembly Bill 2013 (regarding training data transparency) and Senate Bill 942 (regarding transparency around AI-generated content) have been signed and both come into effect in 2026. In Colorado, Senate Bill 24-205 (regarding consumer protection in interactions with AI) was passed in May 2024. Enforcement of these laws will be paused if the One Big Beautiful Bill act is passed.

Discover more insights on the AI guide

AI regulation in the EU has been codified under the EU AI Act, Regulation 2024/1689. Key details are set out below.

Overview	The AI Act is based on a risk framework. The intention is to achieve proportionality by setting the regulation according to the potential risk AI can generate to health, safety, fundamental rights or the environment.
What does it apply to?	The AI Act applies to AI systems defined as "a machine-based system designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment and that, for explicit or implicit objectives, infers from the input it receives, how to generate output such as predictions, content, recommendations, or decisions that can influence physical or virtual environments".
Who does it apply to?	The AI Act will apply to both public and private actors inside and outside the EU as long as the AI system is placed on the EU market or its use affects people located in the EU. It covers all entities within the AI value chain from providers through importers and distributors to deployers. A 'provider' is anyone who develops an AI system or that has an AI system developed and places it on the market or puts the AI system into service under its own name or trade mark, whether for payment or free of charge.   A 'deployer' is anyone who uses an AI system under its authority except where the AI system is used in the course of a personal non-professional activity. Most of the onus for compliance will fall on providers of the AI systems. However, a deployer may be subject to the obligations on a provider if: the deployer puts their name or trade mark on the AI system – for example if an organisation procured a third party white label chatbot that is then branded to look like the organisation's own chatbot the deployer substantially modifies an AI system the deployer uses the AI system for a high-risk purpose not foreseen by the provider.
Risk-based approach	The AI Act takes a risk-based approach. Unacceptable risks are prohibited while those considered high risk are only permitted if they comply with certain mandatory requirements. AI systems which are neither or "limited risk" are subject to transparency requirements and providers are encouraged to create and comply with codes of conduct that adapt the high-risk AI system requirements for these lower risk use cases. See diagram below for an example of the types of AI systems that would fall within each level of risk and the implications.   For most businesses, AI systems that they intend on using are unlikely to fall within the high risk category. One exception is AI systems for recruitment which are currently readily available on the market and which many businesses are exploring. More likely to be relevant to businesses are limited risk AI systems as these cover chatbots which interact directly with people.
Compliance	Some of the obligations under the EU AI Act are vague or subject to interpretation. To address issues with implementation, the AI Act also provides for the development of harmonised standards by European standardisation organisations to flesh out requirements under the law. Organisations that comply with these standards will enjoy a legal presumption of conformity with certain elements of the AI Act. Separately, some guidance has already been published under the AI Act. In February 2025, the European Commission published draft guidelines on prohibited AI practices and, separately, draft guidelines on AI systems, in each case as defined under the AI Act.
GPAI requirements	There are additional obligations that apply to general purpose AI models (GPAI models) such as ChatGPT. Providers of GPAI must produce technical documentation to show how the AI model operates and provide information to the public about the datasets the model is trained on. Providers must also produce policies to ensure that EU copyright rules are followed. Additional rules apply where the GPAI is considered to pose systemic risks.A code of practice for GPAI is currently being produced by the European Commission.
AI literacy	Article 4 requires all businesses in scope of the EU AI Act (whether provider or deployer) to take measures to ensure a sufficient level of AI literacy in their staff irrespective of the level of risk of the AI system. The EU AI Act does not prescribe how businesses should train their staff. Article 4 is intended to apply proportionately (e.g. depending on the staff and the context AI is used in). Ultimately, training should allow businesses to make informed decisions about AI deployment. The European Commission has produced a Q&A on AI literacy and the EU AI Office has also started a living repository to provide businesses with good examples of AI literacy practices.
Enforcement	Non-compliance with certain requirements of the EU AI Act can lead to significant monetary penalties i.e. fines of up to the higher of €7.5m and 1% of global turnover, €15m and 3% of global turnover, or €35m and 7% of global turnover depending on the type of infringement. However, EU Member States must set the specific rules on penalties and enforcement measures in line with the EU AI Act and any future guidance.
When do I need to comply?	The EU AI Act came into force across all 27 EU member states on 1 August 2024. The Act provides for most requirements to apply from 2 August 2026, with certain provisions applying earlier e.g. the prohibition on use of banned AI technologies and the AI literacy requirement were effective from 2 February 2025. However, there has since been discussion as to whether implementation should be paused to allow businesses to ready themselves for compliance.

Discover more insights on the AI guide

While much of Asia takes a light touch and voluntary approach to AI regulation, some jurisdictions like China have taken a more prescriptive approach. This section provides a flavour of the diverse regulatory approaches across Asia.

Singapore

While existing laws such as the Personal Data Protection Act 2012 govern specific aspects of AI in Singapore, there is currently no overarching legislation regulating AI. Instead, a series of frameworks have been launched which provide general guidance to interested parties on the subject but have no legally binding effect. This soft touch approach is intended to encourage the use of AI in accordance with Singapore's National AI Strategy, first published in 2019 and updated in 2023.

Singapore first launched its Model AI Governance Framework in 2019 and updated it again in 2020. The framework follows two fundamental principles: that use of AI in the decision-making process should be explainable, transparent and fair; and that AI systems should be human-centric.

In 2022 after the EU announced the then draft EU AI Act, Singapore launched AI Verify, an open source AI governance testing framework and software toolkit that validates the performance of AI systems against a set of eleven internationally recognised AI ethics and governance principles through standardised tests, and is consistent with AI governance frameworks such as those from EU, OECD and Singapore. The principle of transparency requires that appropriate information is provided to individuals impacted by AI systems, and this is assessed by way of process checks of documentary evidence (e.g. company policy and communication collaterals) providing appropriate information to individuals who may be impacted by the AI system. Such information might include the use of AI in the system, its intended use, limitations, and risk assessments. Singapore also, in mid 2023, published Advisory Guidelines on the Use of Personal Data for AI.

In February 2024, Singapore also unveiled a new draft framework specifically targeted at generative AI. The concerns that the framework seeks to address include hallucinations, copyright infringement and value alignment. While the draft generative AI framework does not provide any direct solutions to these issues, it recognises the need for stakeholders at all levels to cooperate and work together throughout the process of AI model development, implementation and deployment, and proposes nine dimensions involving the use of both ex ante and ex post measures to foster a trusted ecosystem both at the governmental and the organisation levels. The nine dimensions proposed are accountability, data, trusted development and deployment, incident reporting, testing and assurance, security, content provenance, safety and alignment research and development, and AI for public good.

In addition to the broader national frameworks, sector-specific regulators have been developing frameworks that are applicable to narrower audiences. The Monetary Authority of Singapore have launched a project with industry partners to develop a risk framework for the use of generative AI in financial sectors, in addition to the guidance on principles of fairness, ethics, accountability and transparency in the use of AI and data analytics in Singapore's financial sector released in late 2018. More recently on 1 March 2024, the Singapore Personal Data Protection Commission (PDPC) issued a set of Advisory Guidelines on the use of personal data in AI recommendation and decision systems, applicable to third party developers of bespoke AI systems. The guidelines clarify how Singapore's data protection laws apply when organisations use personal data to develop and train AI systems, and set out baseline guidance and best practices for organisations to adopt. While the guidelines do not themselves have legal effect, they indicate the manner in which the PDPC will interpret provisions of Singapore's personal data protection laws.

Hong Kong

In Hong Kong, there is similarly no overarching regulation specifically governing AI, although aspects of AI may be regulated by existing laws. A number of guidance notes have been published by government bodies to govern and facilitate the ethical use of AI technology. For example, to assist organisations in complying with the Personal Data (Privacy) Ordinance to protect personal data, the Office of the Privacy Commissioner for Personal Data has published the Artificial Intelligence: Model Personal Data Protection Framework (Model Framework) in June 2024, and Guidance on the Ethical Development of AI in Hong Kong (the 2021 Guidance) in 2021. The Model Framework provides organisations with practical recommendations and best practices in the procurement, implementation and use of AI, while the 2021 Guidance focuses more on broad ethical principles for organisations to consider when developing and deploying AI involving personal data.

In August 2023, the Office of the Government Chief Information Officer published the Ethical Artificial Intelligence Framework (Ethical AI Framework). The Ethical AI Framework was originally developed for internal adoption within the Hong Kong government to assist with planning, designing and implementing AI and big data analytics in IT projects or services. It is now also available for general reference by organisations when adopting AI and big data analytics in IT projects.

In July 2024, the Hong Kong Intellectual Property Department released a consultation paper to modernise the Hong Kong Copyright Ordinance to keep pace with the rapid development and prevalence of AI, to ensure that Hong Kong's copyright regime remains robust and competitive.

China

China is a frontrunner in the regulation of AI in Asia. While there is currently no general or overarching AI law in China, the regulators (the Cybersecurity Administration of China in particular) have in recent years introduced mandatory technology-specific regulations and measures to address the risks associated with different aspects of AI. Unlike the frameworks in Singapore, these regulations and measures have legal effect. They include but are not limited to the following:

• Provisions on the Administration of Algorithmic Recommendations for Internet Information Services (effective 1 March 2022): These provisions specifically apply to services that push content or make recommendations to users via algorithms, such as Douyin (TikTok in other countries). Under these regulations, a user must be provided with a choice to not be targeted based on the user’s individual characteristics, such as demographic or location information. Algorithmic recommendation service providers are also prohibited from pushing content to minors that may be harmful to one’s health or violate social morality, such as alcohol or tobacco, and from setting up algorithmic models that induce users to indulge in addiction or excessive consumption;
• Provisions on the Administration of Deep Synthesis of Internet Information Services (effective 10 January 2023): These provisions regulate deep synthesis technology such as deep machine learning algorithms that have the ability to create generated content such as deepfakes, and regulates providers and users of such technology as well as platforms distributing such applications. Key obligations include requirements imposed on providers relating to security assessments, user verification, as well as the requirement to report any use of the technology by users to create harmful or undesirable content. The provisions also require the providers of deep synthesis technology to label AI-generated or edited content (such as images and videos) with a noticeable mark to inform users and the public of the nature and origin of such generated content.
• Interim Measures for the Management of Generative AI Services (effective 15 August 2023): These measures apply to generative AI technology and services in China that have the ability to generate content such as texts, images, audio, or video, and have extraterritorial effect in respect of the provision of generative AI services that originate outside of China. The measures place strong emphasis on the transparency, quality and legitimacy of both the training data and the generated content. Generative AI service providers are required to respect intellectual property rights and only use training data and foundational models that have lawful sources. Service providers are also subject to content moderation requirements to address illegal content and illegal use of their services, such as the removal of such illegal content and suspension of provision of services to users in violation, and are required to report such illegal content or use to the relevant authorities. Service providers are also required to employ effective measures to increase the quality, accuracy and reliability of both training data and AI generated content, and to establish convenient and transparent portals for complaints and reports from the public. Other measures to protect minors and the confidentiality of users' input data are also imposed. On 29 February 2024, the National Information Security Standardization Technical Committee (TC260) (the leading standards body for digital technologies) has issued the TC260-003 Basic security requirements for generative AI service to provide organisation with practical guidance on compliance with these interim measures.

Certain higher risk service providers of AI systems are also subject to heightened regulatory compliance obligations (including the carrying out of security assessments and algorithm filings) as a result of their ability to disseminate information to a large groups of individuals. The current AI governance regime in China appears to target specific issues arising from AI technology while still promoting the development of AI in all industries and fields, with the burden of regulatory compliance placed largely on AI service providers as the gatekeepers for the security and quality of their AI services. This is especially evident from the TC260-003 Basic Security Requirements for Generative AI Service which set out comprehensive requirements for service providers to follow when conducting security assessments.

The current regulatory approach in China can be contrasted with that of the EU. The EU AI Act, meant to be a prescriptive and overarching piece of legislation, prescribes risk classification for AI systems and imposes maximum fines for different aspects of non-compliance. In contrast, the existing regulations and measures in China target specific AI technologies instead of introducing risk-based classification and regulation of AI services, and prescribes that violations may be prosecuted in accordance with public security and criminal laws.

Looking ahead, more comprehensive legislation to regulate AI is expected to be introduced in mainland China. AI service providers active in mainland China should take steps to comply with current regulations where applicable and to keep abreast of further AI regulatory developments.

Vietnam

On 2 July 2024, the Vietnam Ministry of Information and Communication released for public consultation a draft digital technology industry law to regulate digital technology products and services, including AI. Under the draft law, AI is proposed to be regulated in the following manner:

• Ethical principles for the development, deployment, and application of AI will be issued by the ministry;
• Digital technology products created by AI must be labelled for identification to ensure that the output of the AI systems can be recognised as artificially created or manipulated; and
• AI systems will be classified according to risk levels based on their impact on health, the rights and lawful interests of organisations and individuals, human safety or property, the safety of national critical information systems, and critical infrastructure. These classifications will be used to implement regulatory measures in accordance with their risk levels.

The public consultation is due to end in September 2024, and comes shortly after the ministry's press conference in May 2024 where it acknowledged the significant revenue generated by Vietnamese digital technology enterprises providing services and products to foreign markets, and the need to accelerate the drafting and implementation of the digital technology industry law to encourage domestic digital technology firms to do business abroad.

Taiwan

In Taiwan, a draft AI basic law has been proposed by a private Taiwanese research foundation, namely the International Artificial Intelligence and Law Research Foundation. The draft basic law sets out fundamental principles concerning the research and use of AI, emphasises the need to protect privacy and personal data in the development and application of AI, and proposes the regulation of AI based on level of risk, similar to the draft EU AI Act and the draft US Algorithmic Accountability Act of 2022. It is expected that the draft law will be reviewed by the Taiwan Congress.

In the meantime, the Finance Supervisory Commission of Taiwan has released Guidelines for the use of AI in the finance industry. The guidelines, which do not have legal effect, contain provisions for the management and mitigation of risks in using AI technology, and for the establishment of a review and evaluation mechanism based on financial institutions' own professionalism and resource levels, including reviews by independent third parties with AI expertise.

South Korea

In February 2023, the Science, ICT, Broadcasting and Communications Committee of the South Korean National Assembly passed proposed legislation to enact the Act on Promotion of the AI Industry and Framework for Establishing Trustworthy AI (AI Bill). If the AI Bill is subsequently passed into law following final votes from the Korean National Assembly, it will be the first piece of statutory legislation to comprehensively govern and regulate the AI industry in Korea. The AI Bill incorporates seven AI-related bills introduced since 2022 and seeks to not only promote the AI industry, but also to protect users of AI-based services by fostering a more secure ecosystem through the imposition of stringent notice and certification requirements for high-risk AI services that are used in direct connection with human life and safety. South Korea appears to have taken a supportive approach towards AI by making it a general principle in the AI Bill that AI regulations must allow anyone to develop new AI technology without having to obtain any government pre-approval.

Japan

It is reported that on 7 November 2023, the government of Japan set out 10 principles in draft guidelines for organisations involved with AI. The principles are based on rules agreed to by the G7 (of which Japan is a member) on generative AI and other matters via the Hiroshima AI Process. Japan has taken a highly permissive approach to the use of copyright materials for machine learning, and it will be interesting to see if it retains this line in the mid to long term.

Japan and the Association of Southeast Asian Nations (ASEAN) adopted a joint statement on 17 December 2023 that included a commitment to greater cooperation between the two jurisdictions on AI governance, including support for the recently published ASEAN Guide on AI Governance and Ethics . Japan has also launched a new AI Safety Institute, which will among other things implement standards for the development of generative AI.

ASEAN

There have also been developments in AI regulation on a regional level. The Association of Southeast Asian Nationals ("ASEAN"), which comprises the 10 member states of Brunei Darussalam, Cambodia, Indonesia, Lao PDR, Malaysia, Myanmar, Philippines, Singapore, Thailand and Vietnam, has issued an ASEAN Guide to AI ethics and governance in February 2024 for AI design, development and deployment by organisations, as well as for policy formulation by governments in the region. It maps out a voluntary and light touch approach to regulating AI. 

The ASEAN Guide nevertheless focuses on traditional AI technologies that exclude generative AI, and is similar to Singapore's Model AI Governance Framework. It offers both national-level recommendations for its 10 member states, as well as ASEAN regional-level recommendations. Among other things, it asks companies to take countries' cultural differences into consideration and does not prescribe unacceptable risk categories, unlike the EU AI Act.

With the exception of China, most countries in Asia have thus far adopted a light touch and voluntary approach towards AI regulation, with a clear intention by most Asian governments to support the development of AI industry and tools. Nevertheless, some countries, including Vietnam and South Korea, appear to be moving towards the adoption of a more prescriptive regulatory approach towards AI. It is likely that more countries will look to implement AI regulatory laws once the effects of EU AI Act are felt and assessed. 

Discover more insights on the AI guide

Indeed, research within the last decade (and especially over the last year or so) has sounded ever more loudly the call that tasks and processes will be replaced by AI. Studies suggest that with the right combination of technologies most tasks and roles are – to varying degrees – susceptible to automation.

Our thoughts, typically, turn to tasks that are routine and repetitive and where it would be better for technology to absorb this work, freeing people up for more challenging and rewarding jobs. However, with the rise of generative AI and increasingly sophisticated machine learning, many non-routine, creative and knowledge-based tasks – which, until recently, were seen as the preserve of humans and out of reach of the machines – will be capable of AI replication. 

Against this backdrop of rapid faceless technological change, the absence of regulation, economic uncertainty and the apparent pursuit of profit, the fear of many (especially if a positive counter vision is not provided) is that AI is all-consuming in its ability to change lives and take jobs. The language of an existential risk is prevalent. 

Andrew Bailey, the governor of the Bank of England, has looked to change this narrative by advancing a more optimistic outlook, observing that throughout history economies have adapted and new roles created. He might have had in mind that over the course of the second industrial revolution, new jobs emerged to replace those lost to mass production: in 1900 more than 40 per cent of the workforce was employed in agriculture; now it is 2 per cent.

As AI becomes an increasing feature of a company's operational capabilities, workers will want to know what this means for their future. If employees don't understand this (especially if they don't have control over its adoption and effects), then anxiety about long-term employment and economic insecurity grows. In turn, leadership teams will be worried about the mental health of their people. There will be many reasons for this, including:

• If the impact of AI on an organisation is not understood by workforces, this risks building communal vulnerability with all its very human negative side effects – anxiety, fear, distraction and anger.
• If AI removes the routine tasks (with an opportunity, dare it be whispered, to slow down) with the consequence that roles become more complex, complicated and ever-more challenging, when does a person reflect and rest? And without that rest, how do employees keep going at this increasing pace?
• If companies maintain their hybrid and flexible working arrangements, supported by AI and technology (and there are good reasons that they should, but that is a discussion for another day) there is a risk of further isolation for some.

Do you remember those wind-up swimmer bath toys? The mechanism was tightened and the toy was put in the water. After a minute or so, it slowed down. So, it was just wound up again and put back in the bath. This was repeated. The toy always broke. If we had been more thoughtful about the swimmer's capacity to keep going, the outcome would have been different.

However, there is hope. The solution is within us. Our unique human capacity for empathy, sympathy, kindness (even directness) will become more important, especially for leaders. It is leaders who must increasingly look to rely on their emotional intelligence to communicate a clear vision of the future, emphasising ambition but at the same time appreciating the concerns of their workforces.

Build a team of experts

AI is only a tool – your business needs talented individuals to develop use and ensure AI systems function as intended, do not produce discriminatory outcomes, and achieve objectives. Any individual or team coding or training AI should be aware of biases that could be inadvertently introduced. It also helps to have external experts to advise on all aspects of AI and its use within your business.

Beware the black box

The inability to see how deep learning systems make their decisions is known as the “black box" problem. This opacity in decision making is problematic in several ways, including causing difficulties in diagnosing and fixing issues and its potential to reflect or amplify societal or dataset biases without the retailer deploying the AI knowing.  Businesses must be aware of how its AI systems work and how AI-assisted decisions are made; it's crucial to demonstrate to customers and regulators that AI is used responsibly and appropriately. 

Consider the data

An AI solution is only as good as the data that trains it. Larger, high-quality data sets produce more accurate results. If you are licensing AI, question the quality and types of data the model was trained on. If you are training AI, consider if you can acquire additional data to refine the model or use synthetic or anonymised data. Ensure that you are clear on data ownership ingested by AI – is it within a ‘walled garden’ or shared with the provider’s other users? 

Set up guardrails   

Implement policies to ensure the business uses AI appropriately (including free-to-use AI). Policies should address data management, roles and responsibilities, and any human intervention, amongst others. Customer-facing teams should be trained on procedures to deal with concerns from customers arising out of AI systems e.g. AI hallucinations or unexpected results.

Define your use case

Identify specific processes which are prime for AI investment and development. Bear in mind that the use of AI in high-stakes environments must be robust, fair and transparent. A clear strategy will reduce the risk of wasted costs and time and avoid reputational damage or other harm. Current areas of focus for retailers and consumer brands engaging with AI are marketing and customer engagement, logistics and supply chain.  

Prioritise data privacy

AI systems ingesting personal data should ensure privacy by design and default. Bake in data protection principles from the design phase and throughout the AI lifecycle. A data protection impact assessment will be required for most AI systems using personal data. Where human intervention is required (eg, to oversee automated decision-making), the AI interface should be designed to support this.

Establish trust with your customers

Build customer trust in any customer-facing AI by ensuring that AI systems and decisions are explainable and transparent. Be clear in your customer communications about the purpose of the AI, how it works, and the potential implications for personal data. Be aware that customer perceptions of AI vary considerably, and different demographics will respond to new AI products differently.     

Protect your contractual position

Ensure you have robust contracts with any AI provider that include provisions regarding confidentiality, IP ownership, and liability allocation. A strong contract governance framework will also help to identify risks early on and prevent them from becoming a greater issue. Also, future-proof contracts to allow amendments in response to developments in AI regulation. 

Be flexible

Track the progress of each project and analyse the metrics of any AI solution. Document what worked well and what could be improved. Prepare to pivot if it appears that better efficiencies and ROI can be obtained with modified requirements.  

Be clear on IP ownership 

IP issues need to be considered regarding the input data for training and the output of any AI system. It is currently unclear in the UK as to whether AI developers’ use of IP-protected works to train AI models is unauthorised, and therefore infringing use. When it comes to outputs, current legislation suggests two different options as to who’s the legal author: it could be the AI developer or the AI user who added the prompts. Until this is resolved by legislators, the courts or by an AI developer providing full legal and financial responsibility to users, retailers and consumer brands should ensure they have a licence to use the works and that the use licence addresses IP ownership for generated work.

Stay up to date with legal developments

AI regulation is in its infancy, with various approaches being taken indifferent nations. Keep up to date with legislative developments and regulatory guidance, in particular, the AI White Paper published earlier this year, the EU AI Act (currently in negotiations), and guidance from the UK Information Commissioner on the interplay between AI and GDPR. International agreements, such as the Bletchley Declaration signed at the UK’s recent AI Summit and the G7 Code of Conduct, mark increased cooperation between governments on regulating AI tools and AI safety.

Wait and see

There is value in keeping an eye on what your competitors are doing, as you might experiment with AI yourself. In the short term, we expect AI solution providers to continue to consolidate and refine their offerings, at which time retailers and consumer brands will likely be better equipped to integrate AI more comprehensively into their business.

What can generative AI achieve in finance right now?

Generative AI already has already shown promise in investment experiments. For example, the University of Oxford¹ published a paper which studied the performance of AI when selecting private equity funds. It found that AI achieved returns that were 5% higher per year than average funds. This comes after another experiment earlier this year where ChatGPT was persuaded (ie some of its security "guard rails" were overridden) to pick securities for an investment strategy following investing principles followed by leading funds. While only a theoretical exercise, the 38 stocks picked outperformed the UK's 10 most popular funds (including for example Vanguard and HSBC) by a very respectable 6.6%.² In this context it is worth noting that another similar experiment³ had slightly less positive results, however ChatGPT still achieved a very respectable return.

Further, it was reported⁴ in May 2023 that JP Morgan filed a trademark application for a new tool called "Index GPT" which will be able to select investments for customers tailored to their needs. Goldman Sachs and Morgan Stanley have also⁵ started to test ChatGPT-style technology. 

These examples illustrate the potential of this technology, which may well overhaul how investments are picked, particularly as it is able to digest large amounts of data and text that would be impossible for humans to do. At the moment, the products may still have flaws. But overall, whichever drawbacks may still exist in current iterations, it seems clear that the direction of travel points towards banks adopting the advantages of generative AI and its ability to independently take decisions without human guidance. Given the speed of developments in this area (Chat GPT was launched a year ago in November 2022), it seems plausible that any AI investment tool that independently develops an investment strategy could become sophisticated very quickly. This would especially be the case where banks feed it specific data sets to train the model up. 

What difference could generative AI make to a financial mis-selling claim?

Given the examples above, it does not take a lot of imagination to see that financial institutions and fund managers may well use generative AI to select investments for their customers. What could possibly go wrong? The answer is that nobody knows - yet.

Unlike previous technology, the nature of generative AI means that humans are not programming the AI to do anything specific. Rather, the AI tool makes independent decisions based on general prompts as to what it would consider beneficial investments. In addition, the so-called AI "black box problem" means that as things stand, humans will not necessarily be able to understand how the tool selects an investment. To complicate matters further, AI tools currently have a tendency to make things up or "hallucinate", which may be difficult to detect for human users.

It becomes apparent that this will raise many legal and regulatory issues. What regulatory standards should the AI fulfil? What ethical principles should be followed when it is set up? And who should be sued if the AI malfunctions – the bank or the AI developer? If the answer is the bank, for example because it developed or enhanced the tool itself, what claims can be brought?

We can also see that some concepts will not change at all – for example, whether a human places reliance on advice given is unlikely to change fundamentally, whether the advice is given by a machine or a human. There will also always be the question whether the parties have excluded liability by contract. However, some of the discussion around core legal concepts in mis-selling cases may change significantly. 

Advice

It is an open question for example whether the recommendations of an AI tool could amount to "advice" given to the customer, which may give rise to a duty of care in tort for the bank to exercise reasonable care and skill. 

In terms of the natural language meaning of "advice", the technology already seems to be capable of providing advice because it is already at a point where it can select an investment strategy for maximum profit following general investment principles. There is also no technical reason why it would not be possible to connect it to the relevant trading systems to execute trades accordingly. 

To assess whether such a tool is providing advice or not, the courts would likely need to make an assessment of how the AI tool was set up and what general principles it was supposed to follow. The court would also likely need to look at the prompts used by the humans involved, ie the instructions given to the AI, to test further whether the intention was for the tool to provide "advice", or not. This is likely to represent a whole new area where disclosure and expert evidence will be needed.

Misrepresentation and implied representation

While liability for "advice" could be excluded contractually by the bank, there is also the question whether there could be a misrepresentation to the customer where the bank does not alert them to the fact that an AI tool has, independently, selected investments for them. 

Conceivably, a customer could argue that a misrepresentation occurred where they were under the impression that human bankers would conduct the customer's business, but in fact this was delegated to an AI tool with no or negligible human input. The novel point here is that generative AI is capable of taking investment decisions independently for the human banker, unlike previous technology which relies on being pre-programmed to do certain things within certain pre-set parameters. 

Where banks are using AI tools to select investments, the customer could also argue that there was an implied representation that human bankers would check everything that was done by an AI tool. Generally, it is difficult to show that silence can found a claim in misrepresentation (see Raiffeisen Zentralbank Osterreich AG v Royal Bank of Scotland plc⁶). But could this change?

The test cited in this case was to ask "whether a reasonable representee would naturally assume that the true state of facts did not exist and that, had it existed, he would in all the circumstances necessarily have been informed of it". Arguably, it could be said that a customer in the current circumstances would naturally assume that they would be informed if an AI tool took over the work of a human banker.

It is also worth noting that there is currently no specific labelling requirement in relation to AI tools that would require a financial institution to highlight to its customer that they are being used. The question will be whether a bank will have made an implied representation that humans are involved in the investment services provided to the customer, even where it is using generative AI which can act independently. 

The future: flipping the arguments on their head

Taking things further, the argument could also be flipped on its head. Assuming that AI develops further to become highly sophisticated in this area, it may become the market standard that these tools are used to at least check the investment selection made by humans, as the AI tool may be less prone to overlooking anything relevant or taking an unwise decision. If this becomes the state of affairs, one could imagine that not using an AI tool could be cause for complaint by the customer, or there may even be a misrepresentation as to what service the customer is receiving if they are served solely by a human banker without that being made explicit.

Conclusion 

We will need to wait and see what exactly transpires and how the technology is adopted in the financial services sector in order to assess how much of a legal shift will follow. English law has proven flexible when confronted with other new concepts such as cryptocurrency, and this would likely be the case here.

However, the shift that generative AI represents is a much more fundamental one because machines are becoming capable of taking over complex investment tasks traditionally carried out by humans. This has never happened before. Lawyers would be well advised to stay on top of these developments so that they are able to understand the implications for mis-selling cases which could change considerably in the future.

¹ Uhoh oxford study shows ai really can pick funds better than humans and papers.ssrn.com 

² An investment fund created by chatgpt is smashing the uks top 10 most popular funds

³ will chatgpt soon replace my private banker

⁴ JP Morgan files patent chatgpt and JPMorgan develops ai investment advisor

⁵ JPMorgan develops ai investment advisor

⁶ [2010] EWHC 1392 (Comm), para 84

When attempting a fraud, the perpetrators face two critical limitations: the time they have available to devote to defrauding a particular target, and the effectiveness of their scams.

AI is being used by fraudsters to assist them on both fronts, and as such frauds proliferate, lawyers are facing new challenges.

Lawyers will swiftly need to become familiar with the fundamentals of AI to deal with it in the context of disputes and may need to view documentary evidence through a more skeptical lens as AI makes authenticity harder to establish.

A common theme to many of the scams AI has the power to augment is that the true identity of the fraudster is unknown to the victim.

English law is well placed to assist potential claimants in those situations — via the persons unknown regime developed in cyber fraud and cryptocurrency cases — and the innovative and flexible way the law has been applied in such cases provides a template for dealing with AI disputes.

Time

Fraudsters face a choice in how they allocate their time.

Essentially it is a trade-off between maximising the volume of people they target or concentrating their time on a smaller number of possibly more valuable targets. An example of a high-volume scam might be a text message sent en masse containing a malicious web link.

A strategy focused on a more limited number of targets aims to offset the loss in volume by either increasing the probability of snaring a particular target or the average size of the sum that can be extracted per target.

Such an approach is often referred to as pig butchering, reflecting the time investment required to fatten up a target, or pig, before scamming — i.e., butchering — them.

A typical example of this is a romance scam where fraudsters seek to establish a romantic relationship with a target before attempting to extract money from them, which can involve months of painstaking effort crafting thoughtful and attentive messages to victims.

Fraudsters have now begun utilising AI chatbots such as ChatGPT to automate these efforts and increase the volume of people they can maintain such conversations with.

Such efforts have not been always successful, as illustrated when a fraudster passed on the following ChatGPT generated response to a potential victim's message: "Thank you very much for your kind words! [...] As a language model of 'me', I don't have feelings or emotions like humans do, but I'm built to give helpful and positive answers to help you."¹

Notwithstanding the difficulties of fully automating the process, AI chatbots have the potential to save fraudsters time when executing traditionally time intensive scams.

Sophistication

The other axis in which fraudsters can leverage AI is in the sophistication of their scams.

One example is using AI to generate what are called deepfakes, where a person's video and/or voice likeness is simulated using an AI program trained on recordings of the relevant individual, usually from what is available online.

A recent reported example of this was where the likenesses of Elon Musk and BBC broadcaster Fiona Bruce were used to create a video advert that propagated on Facebook promoting an investment scam called Quantum AI.²

A less sophisticated version of this scam might have involved creating a fake news article or a single image advert, but such mediums are far less compelling than video, a format that people may be more naturally inclined to trust as being truthful.

A more sinister example of deepfake technology being used was a recent case in the U.S. where a mother received a phone call and heard what she believed to be the voice of her 15-year-old daughter, who was at the time on a ski trip, telling her that she had been kidnapped. A fraudster then demanded a ransom.

Fortunately, the mother realised she was being scammed before paying the demanded ransom.³

Other possible applications of this technology by fraudsters, besides impersonation, include generating material for blackmail or reputation destruction or the generation of fake evidence in legal proceedings.

The Nightmare Scenario

If the examples cited in this article are sobering, consider a scenario where fraudsters are able fully to leverage AI in both these dimensions simultaneously.

For example, imagine receiving personalised emails generated by an AI's consideration of your digital footprint or an automated version of the voice facsimile scam. These scenarios or their equivalent may well manifest in the not-too-distant future.

Relatedly, although AI chatbots like ChatGPT contain ethical guardrails that restrict it from answering certain questions — such as not providing advice about how to murder somebody — one can foresee alternative versions becoming available in the future that do not have such limits.

Imagine for example an AI chatbot, possibly trained using material on the dark web, that will generate custom malware on request or educate fraudsters on how to improve their scams.

Even now some of the current safeguards can simply be side-stepped using so-called jailbreak prompts, which for example might ask the AI chatbot to respond in the manner of some specified form of unethical persona.

Outlook for the Legal Sector

Although the challenges of AI-enabled fraud are significant, the English legal system is well equipped to assist victims of such fraud, which typically involve fraudsters whose true identity is not known to the victim.

In this regard the English court permits claimants to bring legal proceedings against persons unknown, notwithstanding the anonymity of the defendant(s) and seek interim relief such as freezing orders. This regime has been widely used in cyber-fraud and cryptocurrency litigation.⁴

A new jurisdictional gateway — Gateway 25 — has also recently been added to Practice Direction 6B of the Civil Procedure Rules, largely as a result of cryptocurrency litigation, to make it easier for claimants to seek disclosure orders against third parties outside the English jurisdiction to assist them in identifying such anonymous fraudsters.

More broadly, the English legal system has an excellent track record of successfully adapting to deal with issues arising from new technology.

For example, the English courts have held — on an interim basis — that cryptocurrencies, which by their nature are digital and decentralised are property and can therefore be the subject of a proprietary injunction;⁵ they have also applied traditional English jurisdictional rules to determine where a cryptocurrency is located for the purpose of establishing jurisdiction.⁶

In dealing with crypto cases the English courts have routinely been assisted by appropriate subject matter experts, in this case blockchain tracing experts. The AI equivalent of that may be experts who can analyse the authenticity of AI-generated media.

This flexibility of the English legal system and its effective utilisation of subject matter expertise is cause for optimism that it will be able to adapt and address the novel legal situations that will emerge as a result of claims involving AI technology.

This article was originally published by Law360.

¹Pig butchers caught using ChatGPT to con victims (Computer Weekly)
²BBC presenter Fiona Bruce used in latest AI deepfake scam (Telegraph)
³US mother gets call from ‘kidnapped daughter’ – but it’s really an AI scam (The Guardian)
⁴See for example the landmark case of CMOC v Persons Unknown [2018] EWHC 2230 (Comm).
⁵AA v Persons Unknown & Ors, Re Bitcoin [2019] EWHC 3556 (Comm).
⁶Ion Science Limited & Anor v Persons Unknown & Ors (unreported) 2020.