EU AI Act Brief

AI Policy & Governance, European Policy EU AI Act Brief – Pt. 5, General-Purpose AI Models AI ACT SERIES: CDT Europe has been publishing a series of blog posts and briefing papers on the EU AI Act and what it means for human rights. To receive these briefings in your inbox, don’t forget to subscribe to our AI Bulletin here. Below is the fifth post of the series where we examine General Purpose AI models (GPAI) and how they are regulated under the AI Act and additional processes and documents. *** Foundation models are governed under the AI Act as ‘General Purpose AI models’ (GPAI). Defined as models displaying generality, capable of competently performing a wide range of distinct tasks and being integrated into a variety of downstream systems or applications (Article 3(63) AI Act), GPAI models were originally not foreseen to be regulated in the AI Act proposal put forward by the European Commission. Rather, the AI Act as originally proposed focused on the regulation of AI systems considered to be high-risk due to their specific purpose and context of deployment. The emergence of foundation models challenged this approach, given the considerable risk potential of these models without necessarily being tied to any particular use case or purpose. It was therefore only after the surge in popularity of ChatGPT in the midst of the inter–institutional negotiations on the AI Act that the co-legislators drafted a dedicated chapter in the law laying down obligations for GPAI model providers. Notably, legislators left key details subject to clarification through post-adoption processes and documents. These products – which are crucial to a full understanding of GPAI model governance in the AI Act – were finalised in July 2025: - The GPAI Code of Practice (CoP), which outlines and clarifies the obligations of providers of GPAI models under the AI Act, which are two-tiered based on whether a model presents systemic risk or not - The guidelines for providers of GPAI which clarify the meaning of some key concepts and terminology used in Chapter V of the AI Act, and introduce new quantitative thresholds to serve as guidance for when a fine-tuning entity becomes a provider - A template for the public summary of training content for GPAI models which is required under the AI Act In this explainer, we assess these documents together to break down the state of play of the EU’s approach to regulating GPAI models with a specific focus on opportunities and risks for the protection of fundamental rights and the public interest. We note that despite the publication of these documents, the EU’s position on regulating GPAI models is far from final. Both the guidelines (Article 96(2) AI Act) and the Code (Article 56(8) AI Act) can be amended, and in any event the AI Act intends for obligations applicable to GPAI providers to be ultimately covered by harmonised standards (Article 40(2) AI Act). Scope GPAI models are covered by the AI Act if they meet the abovementioned definition and are placed in the EU market. In contrast to the AI Act’s broad definition of GPAI models, the Guidelines apply a narrower interpretation, clarifying that models would be captured by the scope of the law if their training compute is greater than 1023 floating-point operations (FLOP)1, they display significant generality or are capable of competently performing a wide range of distinct tasks, and can generate language, text-to-image or text-to-video output. In practice, this means that several of the multi-modal LLMs (e.g. GPT 4, Gemini 3, Llama 3, Claude Opus 4.5, Grok 4) underpinning the most commonly used AI assistants (ChatGPT, Gemini, Meta AI, Claude, Perplexity, Grok) fall within the definition of GPAI models under the Act. Despite this, the definition of GPAI models remains somewhat arbitrary. Relying on training compute as the sole factor to decide whether a model is covered by the law or not has severe shortcomings, which the guidelines themselves acknowledge. It is therefore possible that due to efficiency gains and changes in training paradigms, highly capable models will be out of scope of the AI Act in the future because they do not reach this threshold of training compute. While the guidelines acknowledge that the Commission’s approach to defining a GPAI model may change, there is no clear timeline for revision of this threshold nor for a review of the guidelines as such. A GPAI model will be in geographical scope of the Act if it is made available or used in the EU single market in the course of a commercial activity, including where it is free (Article 3(9) and (10) AI Act). Models used for research, development and prototyping activities prior to placement in the market do not fall under the AI Act (Recital 97). It is less clear whether uses of GPAI models in the EU single market outside these scenarios fall within the scope of the Act. For example, models used for internal processes “not essential for providing a product or a service to third parties and [where] the rights of natural persons are not affected” are in principle excluded from the AI Act’s scope (Recital 97), but it is unclear what these models may look like in practice. Providers of GPAI models: a moving target In contrast to the AI Act’s provisions on AI systems – which apply to both providers and deployers –, in the case of GPAI models, the Act only places obligations on providers. Due to the variety of possible uses of GPAI models and the range of actors that may integrate a GPAI model into their own infrastructure and adapt it, the question of who ultimately is the provider – namely the liable entity under the AI Act – is crucial, but unclear on the face of the law. The Guidelines clarify the circumstances in which an entity fine-tuning a GPAI model developed by a different entity may be considered to be a provider within the meaning of the Act, noting that if the training compute used to modify a model is greater than one third of the compute used for the original model, the modified model is to be considered a new model in itself and the modifier will be regarded as the provider of this new model. The downstream modifier is consequently required to comply with the obligations for providers, however only in relation to the modification or fine-tuning (Recital 109 AI Act). If a downstream actor modifies a GPAI model with systemic risk with more than one third of the training compute, the new model is also considered to have systemic risk and the corresponding obligations must be met by that actor. The threshold of one third of the training compute is problematic as fine-tuning using significantly less compute can still dramatically change the characteristics of a model and could therefore impact its risk profile, including potentially introducing new risks. Research has shown that it is possible to “undo” the safety training that guides how models respond to hazardous queries with relatively little compute. This could for instance lead to models responding to harmful instructions resulting in discriminatory language or privacy violations. As long as the modification remains below the threshold, no obligations for the new provider, including carrying out risk assessments and mitigations, apply. The approach is slightly different where GPAI models present a systemic risk, a scenario discussed below. In these cases, the Code foresees “lighter-touch model evaluations”, which need not adhere to the risk assessment and mitigation standards set by the Code — i.e. they can for example be automated —, along the entire model lifecycle. Owing to the unspecified nature of such evaluations and the lack of concrete robustness requirements, risks arising out of a modification that does not reach the threshold might however not be sufficiently thoroughly assessed and mitigated with potential negative impacts on fundamental rights. The guidelines also clarify that the lifecycle of a model begins at the start of the large pre-training run. Any developments at a later stage, for example through distillation, quantisation or merging of model weights, are considered part of the same model’s lifecycle. Obligations for GPAI model providers Contrary to the other parts of the AI Act, where the bulk of the obligations lies on a small number of providers and deployers whose AI systems present a high-risk, providers of an AI model classified as a GPAI model under the AI Act will always be subject to obligations irrespective of the risk level the model might present. These obligations broadly concern the publication and availability of technical documentation, copyright compliance, and training data. Documentation The AI Act requires providers to develop and update technical documentation regarding the model (Article 53(1)(a) AI Act) and to share some of this information downstream with providers of AI systems who intend to integrate the general-purpose AI model into their AI systems (Article 53(1)(b) AI Act). The technical information gathered by GPAI model providers must be summarised in a model documentation form, including relevant information related to model properties, methods of distribution and licenses, acceptable use policies, training process, information on the data used for training, computational resources and energy consumption. The information must be updated in line with model updates, and previous versions of the model documentation must be kept for 10 years after the model has been placed on the market. Model providers must only share a fraction of the above mentioned information with downstream providers. As CDT Europe argued in the Code of Practice process, the information categories required to be shared downstream exclude crucial information, including measures taken to detect the unsuitability of data sources or measures to detect bias. The Code further requires providers to provide additional information upon request to downstream providers if this information is necessary for understanding the capabilities and limitations of a model. Any of the documentation produced must be presented to the AI Office upon request if it is necessary for the Office to fulfil its tasks, or if a national competent authority requires it to exercise their supervisory function – especially when assessing high-risk AI systems built upon GPAI models. The CoP encourages providers to consider publication of the model documentation to promote public transparency but stops short of requiring it, resulting in a missed opportunity for comprehensive public oversight. Copyright compliance obligations Training and developing GPAI models may include unauthorised extractions and reproductions of copyrighted material, leaving rightsholders with limited transparency as to the extent their content has been used for training, without credit or compensation for their work. As commentators have addressed, AI presents novel challenges to the copyright framework which put at a disadvantage those sustaining the public knowledge ecosystem, while benefiting those profiting from their exploitation. The AI Act takes a step to address this tension. Under the AI Act, providers are obliged to introduce a policy to comply with EU copyright law (Article 53(1)(c) AI Act), yet the Code only requires a summary of this to be made publicly available. Among other measures, the Code requires providers to exclude websites recognised by EU court and authorities as persistently infringing copyright law from their web-crawling activity and to employ web crawlers that respect robots.txt protocol and other standardised or widely adopted machine-readable rights reservation protocols. Providers must implement technical safeguards to prevent models from generating copyright-infringing outputs and include prohibitions against copyright infringement in their terms of use. Finally, the section requires providers to have an electronic point of contact for affected rightsholders with commitment to handle complaints diligently. Public summary of training data In a rare public transparency requirement, the AI Act requires providers to prepare and publish “a sufficiently detailed summary” about the content used for training the model, based on a template provided by the AI Office (Article 53(1)(d) AI Act). The template published by the European Commission seeks to strike a balance between fundamental rights and trade secrets, with the Explanatory Note to the template stressing the relevance of the template to uphold rightholders’ interests, as well as other fundamental rights such as data protection or non-discrimination, for example in order to enable access to a data subject’s rights or support downstream providers or harmed individuals to assess the representativeness of the data used. The requirements cover all stages of the model development, from pre-training to post-training. The template requires the disclosure of large publicly available, as well as private, datasets obtained from third parties that the model was trained on. Providers need to present a summary of the top 10% of all domain names from which content was scraped as determined by the overall quantity of the content scraped. While this is a positive first step, leaving out the majority of domains provides for limited public transparency regarding the data scraped, which in turn limits the opportunities to use the template as a vector to enforce data protection law. Moreover, the wording could be interpreted as requiring one list across all modalities (text, image, audio, video and other), instead of the top 10% per modality. This could however mean that smaller and specialised, but nevertheless significant, sources remain invisible, curbing meaningful transparency. Another limitation is that the template requires only minimal information about user data collected through user interactions with all services and products of the providers, which is an increasingly important source of data for AI development. The last section of the template requires providers to disclose some data processing aspects, including a general description of the measures taken to remove illegal content under Union law, such as child sexual abuse material or terrorist content. The AI Office refrained however from prescribing a more detailed provision of information on processing activities which could be for example relevant for bias research or to understand to what extent personal data has been anonymised. The summary will need to be made publicly available when the model is placed on the EU market at the latest. Providers are obliged to update the summary — at least at six months intervals — when they further train the model on additional data. They are also encouraged to go beyond the requirements in the template and enable stakeholders, including rightsholders, with a legitimate interest to request information on whether their work has been scraped and used for training content. While the European Commission is required to verify whether the template has been filled in correctly, it will not perform a work-by-work assessment (Recital 108 AI Act) or check whether specific content has been used or not for the training. For this reason, despite its role in providing important information for third-party scrutiny, the ultimate usefulness of the template and the effectiveness of enforcement therefore remains to be seen. Obligations for providers of GPAI models with systemic risks The most robust obligations under the AI Act – ranging from risk assessments to the preparation of safety frameworks – apply to GPAI models with systemic risk, ie models with “high impact capabilities evaluated on the basis of appropriate technical tools and methodologies, including indicators and benchmarks” (Article 51(1)(a) AI Act). As a general rule, models will be presumed to present a systemic risk if the cumulative amount of training compute is greater than 1025 FLOPs (Article 51(2) AI Act); alternatively, a model can be designated as such by the European Commission if the model has equivalent impact or capabilities. In addition to the obligations already mentioned, providers of GPAI models with systemic risk are required to: - Assess and mitigate possible systemic risks - Perform model evaluations, in accordance with standardised protocols and tools reflecting the state of the art, including adversarial testing - Keep track of, document and report information about serious incidents and corrective measures taken - Ensure an adequate level of cybersecurity protection (Article 55(1) AI Act). These obligations are further detailed in the safety and security chapter of the Code which outlines ten commitments to be taken by signatories. The following section provides an overview of the commitments and resulting obligations most relevant from a fundamental rights perspective. The risk assessment layers: identification, analysis, evaluation and mitigation The hierarchy of risk identification As one of the commitments under the CoP, Signatories are required to identify the systemic risks stemming from the model. As problematised by CDT Europe, previous versions of the Code differentiated between selected systemic risks, largely falling under the category of existential risks, which were mandatory to be evaluated and mitigated by providers, and other systemic risks, including risks to fundamental rights, which were optional to be identified. The final version does not follow this two-fold approach but obliges providers to consider five types of systemic risks, namely risks to public health, safety, public security, fundamental rights and the society as a whole. The Code requires providers to compile a list of risks that could stem from their models based on those five overarching types, analyse this preliminary list according to certain criteria, and then based on this assessment,identify the final systemic risks. Considerations for this exercise must include requiring that the risk is specific to high-impact capabilities, has significant impact on the Union single market and can be propagated at scale across the value chain – and contributing characteristics. In parallel to this, they are required to always identify the so-called specified risks, which encompass the selected systemic risks from earlier Code drafts. While the requirement to consider comprehensive types of risks brings the CoP back into alignment with the text of the AI Act, in practice, the final draft still retains a two-tiered approach where some risks always need to be identified whereas the identification of fundamental rights risks will still be up to the discretion of the providers and how they consider their nature. Nevertheless, the recitals require them to interpret the probability and severity of harm in good faith. Further, providers need to justify their risk identification process in their Safety and Security Model Report which is shared with the European Commission. To what extent a comprehensive set of systemic risks will indeed be identified by providers therefore also depends on the scrutiny in enforcement by the Commission, which can also issue additional guidance on the identification of systemic risks. Under this commitment, signatories are also required to develop appropriate risk scenarios for each of the identified systemic risks. Contrary to previous versions of the Code, they are however not anymore asked to take into account reasonable foreseeable uses and misuses of the model, which renders assessments more abstract and less tied to real-world use cases. Systemic risk analysis, including through external actors After identifying the applicable systemic risks, signatories commit to analysing them. This analysis includes, among other things, conducting state-of-the-art model evaluations and post-market monitoring. The Code lists several examples of such evaluation methods, including Q&A sets, task-based evaluations, benchmarks or red-teaming. In a move enabling crucial external and independent scrutiny, the Code requires independent external evaluators to facilitate model evaluations, including the post-market monitoring of the models. A failure to appoint an external evaluator is accepted either if the model is a similarly safe or safer model or if the provider failed to appoint adequately qualified evaluators despite early search efforts (through a public call open for 20 business days). Providers must state whether such external evaluators were engaged in the Model Report, and if not, they must declare whether the conditions for any of the two exemptions are met. This means that providers are still able to exclude external evaluators if they claim that no adequate person could be found. While the Act asserts the particular role and responsibility of GPAI model providers along the AI value chain (Recital 101), the final Code provides for less strict commitments for systemic risk assessments along the entire model lifecycle, including during the planning and development stages. Instead, it requires signatories to merely conduct “lighter-touch model evaluations” which need not adhere to the standards set by the Code. The Code also does not foresee any obligation for cooperation between providers and downstream modifiers to assess and mitigate risks. Indeed, the final version of the CoP removed previous sections requiring providers to cooperate with other actors, such as civil society or downstream providers. This cooperation is now only mentioned as a principle in the recitals. As an illustration, the final version of the Code has removed the section on “models as part of systems” which required signatories to take into account reasonably foreseeable integrations of the models into an AI system when evaluating the model. This section also required signatories to cooperate with licensees and downstream providers in order to consider the integration of their model. This is problematic given the correlation between risks at model and system level. It also risks abuses by providers to shift responsibilities to less resourceful downstream actors. The final Code no longer includes a section requiring signatories to advance the state of the art in systemic risk assessment and mitigation techniques through exploratory research and open-ended red-teaming that engages expert or lay representatives of civil society, academia and other relevant stakeholders as participants. This therefore allows for less external input on fundamental rights-related issues. On an equally problematic note, the previous section called “context, modalities and representative model evaluations”, which obliged providers to ensure that their model evaluations matched the expected use context, including by cooperating with relevant actors along the value chain, such as appropriately remunerated expert or lay representatives from civil society, academia or other relevant stakeholders was removed from the final version. Systemic risk acceptance determination Following the analysis, Signatories need to determine whether the systemic risks are acceptable to proceed with the model development. To establish criteria for this assessment, the Code requires providers to define measurable systemic risk tiers in terms of model capabilities. While previous versions of the Code obliged them to define at least one tier that was unacceptable due to the risks reaching that tier would pose, the final version does not include this requirement anymore, simply asking them to define “at least one systemic risk tier that has not been reached by the model”. This frees providers from their obligation to consider the impact and unacceptable risks their model could pose. Providers are now also given the option to not define systemic risk tiers, if they deem this methodology not suitable, as long as they define other appropriate systemic risk acceptance criteria. The final version continues the obligation of providers to only proceed with the development of a model if all systemic risks are deemed to be acceptable. Similarly, they must not make the model available on the market or restrict its access as well as take risk mitigation measures and conduct another round of risk identification, analysis and acceptance determination if a risk is deemed unacceptable. The acceptability of the systemic risks stemming from the model needs to be justified in the Model Report (referred to below). Serious incident reporting The CoP requires signatories to report serious incidents to the AI Office, including a serious harm to a person’s health, fundamental rights, property or the environment, if a causal relationship between their model and the harm is established or reasonably likely suspected. The incident needs to be reported no later than 15 days after becoming aware. There is however no requirement to publicly communicate about the occurrence of a serious incident. Consequently, other relevant stakeholders, such as civil society or researchers, will have limited or no knowledge about serious incidents which affects their ability to contribute to understanding, preventing and mitigating such risks. The CoP also does not include an obligation to engage with the affected individuals or communities to ensure that a serious incident is effectively addressed and its effects are mitigated. Conducting frequent dialogues with affected stakeholders is however mentioned as an example of a post-market monitoring method. Documentation requirements Signatories commit to adopting a state-of the-art Safety and Security Framework. This Framework is dedicated to describing the systemic risk management process by the providers and measures taken to ensure the acceptability of risks in accordance with providers’ obligations under the AI Act. The framework needs to be completed no later than two weeks before placing a model on the market. The AI Office obtains (unredacted) access to the framework which enables the Commission to scrutinise how signatories are complying with their obligations under the AI Act. Providers are not required to make the framework available to the public. Signatories are also required to create a Safety and Security Model Report to report to the AI Office information about their model and their systemic risk assessment and mitigation measures. Providers can share their model report up to 15 business days after placing the model on the market. This means that the Commission will not be able to vet the rolling out of certain models due to for example shortcomings in the systemic risk evaluation and mitigation process. The final version specifies that Model Reports need to be updated “within a reasonable amount of time after the Signatory becomes aware of the grounds that necessitate an update”. This provides for more specific timing for updates than previous versions. The final version of the CoP also newly introduces the obligation of providers to make available to the AI Office an updated Model Report at least every six months, if the model is amongst their most capable models available on the market, unless specific exceptions apply. The CoP obliges providers to make publicly available a summary of their Safety and Security Framework and Model Report if they deem this necessary for the assessment and mitigation of systemic risks. Such publications shall include high-level descriptions of the systemic risks assessment and the mitigations taken. Removals are however allowed for safety and security reasons or to protect trade secrets. While this provision may enable some limited form of public scrutiny, for example by fundamental rights organisations, its limitation to the narrow case where publication is necessary for risk assessment or mitigation, which remains up to the provider’s discretion to decide, is highly problematic. Downstream integration of GPAI models Apart from the principle of cooperation between model and downstream providers, the Code does little to regulate the downstream integration of GPAI models – for example, a model being incorporated into customer relationship management platforms or on social media networks. These integrations are relevant: embedding a GPAI model within a specific application can diversify and increase risk. The Code recognises this to a limited extent, noting for example that access to or interactions with other AI models or systems is a relevant affordance to be considered as a source of systemic risk. The Act compensates for this omission in its AI system obligations section. For instance, if a GPAI model is used in any of the high-risk areas under Annex III or intended to be used as a safety component of a product or a product itself under Annex I, the obligations for high-risk AI systems apply to the provider of the system (not the model) and the deployer. Documentation drawn up under the different obligations by GPAI model providers can thereby fulfil an important function to support high-risk system providers to fulfil their respective obligations. If a GPAI model is integrated into an AI system that has the capability to serve a variety of purposes, it is considered to be a GPAI system (Article 3(66) AI Act). They can be used as high-risk systems themselves or be components of other high-risk systems (Recital 85). While it would be possible to interpret the AI Act as classifying GPAI systems by default as high-risk because they could potentially be used in a high-risk area under Annex III, it has been argued that the more likely interpretation would understand GPAI systems not to be high-risk as long as the provider has excluded such high-risk use from the system’s intended purpose. It remains however open what such exclusion could look like. Enforcement At the time of writing, the obligations for GPAI providers under the AI Act are applicable since 2 August 2025 for new models and from 2 August 2027 for models placed on the market before this date. The European Commission, acting through the AI Office, is responsible for the enforcement of the rules on GPAI models (Article 88(1) AI Act). This includes the power to request access to documentation produced by a GPAI model provider, conduct evaluations of any given GPAI model in order to assess its compliance with the AI Act, request mitigation measures, and – importantly – restrict the model’s availability on the market, or withdraw it altogether. The information-gathering and evaluation powers by the Commission are supported by a scientific panel, an independent body of 60 experts selected to monitor risks of GPAI models at EU level. The Act sets a low bar for these powers to be exercised. Conversely, stricter enforcement powers including the restriction of a GPAI model’s circulation on the EU market are notoriously more difficult to exercise outside blatant non-compliance scenarios. The instruments supplementing the AI Act – namely the Guidelines and the Code – are ultimately non-binding even if they represent the European Commission’s interpretation of the AI Act and are likely to guide their enforcement. Signatories of the Code of Practice – identified in this updated list – benefit from a presumption of compliance with the AI Act if they meet the requirements set in the Code, but that is not to say that they will be the target of enforcement measures if they fail to do so. Following the Code is a near-guarantee of compliance, but departing from it is by no means an enforcement trigger: after all, it is always open to providers to demonstrate compliance through alternative means. Overall, the exercise of enforcement powers by the Commission beyond requests for information is likely to be informally subject to a high threshold of non-compliance. Conclusion The Code of Practice was developed as a compromise instrument to hold providers of GPAI models accountable for the risks emerging from their models, while at the same time affording them a large margin of discretion as to how these risks were specifically assessed and mitigated. This places a high degree of trust in GPAI model providers’ ability to robustly and honestly map and address risks posed by their models – but as the recent scandal surrounding the generation of non-consensual intimate imagery by model-powered chatbots such as Grok, this trust must be earned. The role of the Code of Practice in enabling quick regulatory action is still to be determined, but several factors undermine this. Aside from external model evaluators, the Code does not require any level of engagement with third parties such as civil society or affected individuals. Against this backdrop, the Code is best understood as a tool to mainstream risk management practices as opposed to an actionable accountability mechanism. *** - FLOP are used to measure the training compute understood by the guidelines as “the amount of computational resources used to train” an AI model. - Alexander Erben et al., Training Compute Thresholds – Key Considerations for the EU AI Act (European Commission Joint Research Centre 2025), p.16. - Assessing and mitigating systemic risks along the entire model lifecycle is now merely a consideration in a recital. - Specific risks mentioned include risks of major accidents; risks to critical sectors or infrastructure, public mental health, freedom of expression and information, non-discrimination, privacy and the protection of personal data, the environment, non-human welfare, economic security, and democratic processes; risks from concentration of power and illegal, violent, hateful, radicalising, or false content, including risks from child sexual abuse material and non-consensual intimate images. - Chemical, biological, radiological and nuclear weapons or attacks, loss of control, cyber offense and harmful manipulation. - These are defined as “a natural or legal person that has no financial, operational, or management dependence on the Signatory or any of its subsidiaries or associates, and is otherwise free from the Signatory’s control in reaching conclusions and/or making recommendations, including through contractual safeguards and suitable conflict of interest policies”. - Models that are considered similarly safe or safer than safe reference models according to different criteria listed in the CoP. Safe reference models are models that have completed the systemic risk assessment and mitigation process under the Code or that have already been placed on the market before the publication of the CoP. - Principle (e) of Cooperation. - Guidelines para 9.