Welcome to the last Algorithm Governance Roundup of 2023. This month’s Community Spotlight is Karine Perset, the Head of the AI Unit of the OECD Digital Economy Policy Division. She leads the Unit’s work convening policymakers and expert groups working on AI governance and supporting evidence-based policymaking through the OECD.AI Policy Observatory.

This issue marks the first anniversary of this newsletter! Thank you to all our readers. I am especially grateful to everyone who spoke to me for our community spotlight. 

It has been a busy year for algorithm governance. Some highlights:

• The release of ChatGPT-3.5 prompted a wave of public attention and action, which we covered in our April newsletter. In September, I also interviewed the Safe AI Companion Collective about their data protection and consumer protection complaints against an LLM chatbot provider.

• It has been a big year for novel platform and AI regulation. In the EU, I spoke with the head of the new European Centre for Algorithmic Transparency and France's PEReN about their role supporting Digital Services Act (DSA) enforcement. I also interviewed AlgorithmWatch and SNV about auditing platforms for systemic risk under the DSA. In the UK, I spoke with Ofcom about its preparatory work on algorithmic assessments for the Online Safety Act. And in this month's issue, I cover the recent White House Executive Order on AI and European legislators' provisional agreement on the AI Act, which concluded last week.

You can find the full archive of the Algorithm Governance Roundup here.

Happy holiday and happy reading!

Esme Harrington, AWO

In the EU, the Parliament and Council have reached a provisional agreement on the AI Act after 36 hours of negotiations. The final text will be published in the coming weeks and must be formally adopted by both the Parliament and Council. 

Whilst the final text has not been published yet, reports detail the list of prohibited applications; obligations for high-risk systems, including fundamental rights impact assessments; transparency requirements for foundation models; and further obligations for high-impact foundation models. The agreed text also creates an AI Office within the EU Commission which will contribute to enforcement and oversight.

The European Commission has opened a consultation concerning its implementing act on transparency reports for platform content moderation, as required by the Digital Services Act. The consultation includes mandatory templates that specify the type and form of content to be reported. The deadline for consultation responses is 24 January.

A group of AI experts have launched the International Algorithmic Auditors Association (IAAA) which is the first non-profit professional organisation for AI auditing. The IAAA aims to promote collaboration and shared knowledge on standards, methodologies, and tools.

In Germany, the Bundeskartellamt competition authority has published a case summary explaining why Microsoft’s involvement in OpenAI did not trigger a notification obligation under German merger control rules.

In the UK, the UK government held the first global AI Safety Summit. The Summit brought together countries, leading AI companies, and researchers to discuss the risks of frontier AI models, which are defined as capable general-purpose AI models.

In preparation for the Summit, the UK’s Frontier AI Taskforce published its Second Progress Report, announcing an expansion of its research team and new partnerships with AI organisations. It also shared details of the establishment of the UK’s AI Research Resource (‘Isambard-AI’) at the University of Bristol, which will be the UK’s largest compute facility.

Attendees of the AI Summit signed the Bletchley Declaration, which recognises the risks and opportunities of AI systems and commits to further research on frontier AI models to support risk-based policymaking. Signatories have committed to support a State of the Science Report to understand the capabilities and risks of frontier AI. Country signatories have also committed to work together to research AI safety testing.

At the Summit, the UK government also announced the creation of the UK’s AI Safety Institute, which will conduct research on frontier AI safety by examining, evaluating, and testing emerging AI systems. The research will inform UK and international policymaking and provide technical tools for governance and regulation. The Institute will have access to the AI Research Resource (‘Isambard-AI’) to conduct this research. It will build on the work of the Frontier AI Taskforce and the AI Safety Summit.

Following attendance at the UK’s AI Safety Summit, the US White House announced the creation of the US’s AI Safety Institute. The Institute will operationalise NIST’s risk management framework by creating guidelines, tools, benchmarks, and best practices for evaluation and mitigation. The White House also announced several other initiatives including a virtual hackathon to combat AI driven fraudulent phone calls, and a call for cooperation on international standards to identify and trace digital and AI-generated content.

Ofcom has published its implementation approach for the Online Safety Act, including an indicative timeline of public consultations. To begin this process, Ofcom has opened a consultation on draft codes and guides related to the illegal harms duties. The deadline for response is 23 February. Ofcom has also opened a consultation on draft guidance on age assurance for online pornography services. The deadline for response is 05 March.

The National Cyber Security Centre has published guidance on how to design, develop, and deploy AI systems in ways that ensure their cybersecurity. The guidelines have been endorsed by 17 other countries, including all G7 members.

In Australia, the government has announced its intention to amend the Basic Online Safety Expectations and statutorily review the Online Safety Act to ensure social media platform recommendation systems do not amplify harmful or extreme content, including racism, sexism and homophobia.

In Canada, the Minister of Innovation, Science and Industry has announced a Voluntary Code of Conduct on the Responsible Development and Management of Advanced Generative AI Systems. The code provides companies with common standards to demonstrate that they are developing and using generative AI systems responsibly. This will apply until more formal AI regulation takes effect. (The Canadian AI and Data Act is currently being debated at the committee stage.)

In South Korea, the Personal Information Protection Committee (PIPC) opened a public consultation on establishing an AI privacy team (original announcement). The consultation is open until 14 January. The head of the PIPC has stated South Korea’s commitment to becoming a leader in setting AI guidelines globally.

In the US, the White House has announced an Executive Order on Safe, Secure and Trustworthy AI. The Executive Order requires developers of the most powerful AI systems to share safety test results with the government. It also requires government agencies, including the National Institute of Standards (NIST), to develop standards, tools, and tests to ensure trustworthiness, and the Department of Commerce to develop guidance for watermarking and authentication of AI-generated content. Other elements focus on cybersecurity, privacy, civil rights, consumers, worker protections, and innovation. 

The Executive Office of the President’s Office of Management and Budget has published a draft policy for federal agency and government development, procurement, and use of AI. The draft guidance requires agencies to set up governance structures, develop an AI strategy, and implement safeguards including impact assessments and independent evaluations.

The California District Court has granted part of Stability AI, DeviantArt, and Midjourney’s motion to dismiss a copyright case, except for the direct copyright infringement allegation against Stability. The claimants are able to refile the complaint.

The Ada Lovelace Institute has published Going Public: Exploring public participation in commercial AI labs. This research interviewed 12 industry practitioners and public engagement experts about the use of public participation approaches in the development of AI research and products.

The Alan Turing Institute’s Professor Marion Oswald MBE has led a team of researchers on Exploring Police Perspectives on Algorithmic Transparency: A Qualitative Analysis of Police Interviews in the UK. This research interviewed police personnel and representatives of commercial providers to analyse and make recommendations on the use of the UK’s algorithmic transparency standard.

AlgorithmWatch and AI Forensics have published ChatGPT and Co: Are AI-driven search engines a threat to democratic elections? This study demonstrates that large language models are an unreliable source of information concerning voting and pose a danger to the formation of public opinion in a democracy.

AI Forensics and Check First have published The Amazing Library: An analysis of Amazon’s Bookstore Algorithms within the DSA Framework. This research analysed over 60,000 book recommendations and search queries across Belgian and French stores. It found that Amazon often steered users towards misleading content and were inadvertently exposing minors to sexually explicit material.

Lighthouse Reports has published France’s Digital Inquisition. This investigation, conducted with AlgorithmWatch, La Quadrature, Changer de Cap and Le Monde, obtained and analysed the source code for the French social security agency’s (CNAF) risk assessment algorithm. The analysis revealed that the model often flags the most vulnerable individuals, as well as arbitrary changes in behaviour.

Panoptykon Foundation has published Prototyping User Empowerment: Towards DSA-compliant recommender systems.This research offers recommendations for UX design features that would empower user choice in relation to social media and other platforms. The features are related to feedback, content curation and well-being.

UNESCO is calling for papers for its Global Forum on the Ethics of AI. It is open to academics, industry practitioners, policymakers, and students.
Deadline for submission is 22 December.

ACM Conference on Fairness, Accountability, and Transparency 2024 is calling for papers. Submissions should focus on audits and evaluation practices, system development and deployment, experiences and interactions, critical studies, law and policy, or philosophy.
Deadline for abstract submission is 11 January.

Responsible AI UK is calling for applications from industry, civil society, and policymakers to join its Partner Network. Partners can provide research and case studies on the design, operation, and application of AI.
Deadline is ongoing.

Ranking Digital Rights is calling for feedback on its Draft Generative AI Accountability Indicators.
Deadline is ongoing.

MLCommons is calling for academic, industry, civil society, and public sector researchers to join its AI Safety Working Group. The Group develops safety benchmarks, supports consumer decision-making, and evidence-based policymaking.
Deadline is ongoing.

Karine Perset is the Head of the AI Unit of the OECD Digital Economy Policy Division. She leads the Unit’s work convening policymakers and expert groups working on AI governance and supporting evidence-based policy making through the OECD.AI Policy Observatory.

Q: Can you introduce the AI Unit of the OECD Digital Economy Policy Division?
Karine: The AI Unit sits within the OECD Digital Economy Policy Division, which is responsible for the breadth of digital issues. The Division has several units including AI, privacy and data governance, telecommunications, cybersecurity, and platform governance. Since the governance of AI impacts a range of sectors, the AI Unit plays a coordination function across these and other sectoral working parties within the OECD.

Q: Can you tell us about the OECD Working Party on AI Governance and the OECD.AI Network of Experts?
Karine: The Working Party convenes policymakers working on AI strategy and regulation in different Member and Associate countries. The Working Party meets physically twice a year and occasionally virtually to enable international cooperation on the identification of common priorities and completion of work. The Working Party is supported by advisory committees and an informal network of experts. This network of experts focuses on specific areas of public policy. There are six expert groups that include stakeholders from policy and civil society. Each one has a different focus:

The first expert group focuses on AI futures. This group works on the medium to long term risks of AI, and has a major membership of academic and industry researchers. I attended the UK’s AI Safety Summit at Bletchley Park, which successfully raised awareness of this more niche area with the heads of national states and other organisations in attendance. The Summit was impressive for helping to encourage a more proactive rather than reactive approach to these risks.

The second expert group focuses on AI compute. For example, it examines the industry provision of cloud service providers and microchip manufacturing. One of the aims of this working group is to measure both current compute usage and required compute.

The third expert group is building an AI index, with membership drawing heavily from academia and national statistical offices.

The fourth expert group focuses on risk and accountability. It has a varied membership including the standardisation organisations and bodies that are developing risk management frameworks such as NIST in the US, CEN/CENELEC in the EU, and the IEEE and ISO. This group is focused on making sense of the range of frameworks emerging across the globe. Originally, it intended to develop a novel OECD AI risk management framework. However, given the number of existing and similar frameworks that have been developed, the group decided it would be more useful to bring these approaches together and link them with the OECD's Responsible Business Conduct Guidance. We hope that this will enable international companies to implement a standard approach to due diligence management across jurisdictions and ensure they benefit from a presumption of compliance. This streamlined approach should prevent due diligence from becoming overly onerous which wastes resources and risks reducing it to mere box-ticking.

The fifth expert group focuses on AI incidents. Currently, reporting frameworks for consumer protection and product recall are specific to each national state. This means that reporting is siloed to each state and other countries are not aware of, or benefit from, the incident report. We want to avoid this siloed approach for reporting AI Incidents. This is necessary because industry will be required to report any serious AI incidents under various pieces of regulation across the globe. For example, the proposed EU AI Act requires reporting after the Act has been in force for two years, and similar reporting obligations are found in Australian health regulation and British financial regulation.

For the past two years, this expert group has been developing a common reporting framework that will be applicable across jurisdictions and regulation. This framework is being developed with the European Commission. Currently, the expert group and the Commission are developing definitions (e.g. what constitutes an incident or hazard) that are applicable across sectors and incident type.
 
We are also building a full taxonomy of AI incidents that will be made public. This taxonomy catalogues both historic and real-time AI incidents that are reported by the press. To process these incidents in real-time, we have developed our own automated system that provides a classification which we review. To train the system we first created a dataset containing manually identified AI incidents from reputable press articles. The next step will be to create and train the system on a dataset containing caselaw related to AI incidents and scientific publications. We will also begin to enable industry to submit voluntary reports, using our reporting framework, and create a mechanism for industry to contest reports. Eventually, we will incorporate regulatory requirements.

This taxonomy will be an important resource for a more evidence-based approach to regulation. For example, The European Commission are interested in using the taxonomy findings to validate and adapt the AI Act’s high-risk list to ensure that it accurately covers the most high-risk types of AI systems and industry use cases. Whilst the current taxonomy relies on press interest to report incidents, it is providing a useful baseline to enable comparison over time at a country level. Since November 2022 we have already seen interesting trends related to sector and severity level.

The final expert group focuses on data governance and AI. It aims to convene and enable perspective sharing between the data protection and AI governance communities. These communities have a difference in paradigms, for example around data minimisation to preserve privacy and data maximisation for AI development.

Q: Can you tell us about the OECD.AI Policy Observatory, including the recently launched catalogue of Tools and Metrics for Trustworthy AI?
Karine: The OECD.AI Policy Observatory provides resources for policymakers as well as expert contributions.

For example, the observatory shares trends and data at national level on venture capital investment in AI, types of research conducted, what people are searching for online, skills and jobs posting by AI companies, and patent registrations. The observatory is also collecting and cataloguing national and sectoral policies on AI governance. This will enable policymakers to research and analyse approaches in similar national states. To facilitate this research, we have built a downstream GPT-based search tool.

Q: Can you tell us about your recent research, such as the policy briefing on generative AI?
Karine: We have conducted research on generative AI, particularly thinking about the open-source ecosystem and licensing approaches. Whilst open source developed naturally due to the academic approach to research, industry actors have different incentives. We want to ensure we help industry adopt the right model to foster innovation. We also considered issues around dating scraping and intellectual property rights.