I. Introduction

The Internet and social media have triggered a tectonic shift in our digital “global village”.¹ Discourse production has moved onto a new online medium with a radically different structure and dynamic.² Besides creating a revolutionary “participatory” communications model (i.e. shifting from a few-to-many to many-to-many dynamic),³ a key feature of our digital free speech infrastructure has been the emergence of a small group of powerful privately-owned digital intermediaries—the so-called “Big Five” (Google, Meta (formerly, Facebook), Amazon, Microsoft, and Apple)⁴—who not only effectively “own” and operate the Internet, but function as increasingly decisive arbiters of what information users access online, and what content ultimately reaches the public sphere.⁵ Generating unprecedented regulatory challenges, a combination of these influential “new governors”,⁶ an increasingly complex digital media infrastructure, and continuing technological advances not only creates tension with existing legal rules and principles,⁷ but gives rise to increasing lower-salience structural threats to democracy, manifesting in unprecedented global surveillance, manipulation, and control.⁸ Regardless of which of the two leading regulatory approaches is championed—viz., the European Union’s predominant “notice-and-action” model or America’s contrasting system of “market self-regulation”—conventional online regulations exhibit near-singular focus on restricting “problematic” online content (e.g. hate speech and misinformation), leaving the accelerating and more disquieting phenomena of mass surveillance and privatised government censorship unaddressed.⁹ As we have previously cautioned, without prioritising these structural threats, regulators—much like physicians—risk treating only the symptoms of our increasingly dysfunctional online public sphere, rather than grasping their aetiology of broader tensions, patterns, and interrelationships.¹⁰

A promising antidote to these growing regulatory challenges is Canada’s evolving “multi-stakeholder” approach, which has been marked by extensive public and expert consultations. Inspired at first by Germany’s popular “notice-and-takedown” model,¹¹ following widespread criticism of likely encroachments on freedom of expression by Bill C-36 (Canada’s provisional hate speech legislation introduced in 2021), politicians quickly announced plans to go back to the proverbial “drawing board”.¹² Mindful of the need for political and regulatory compromise, Canada’s minority Liberal government proceeded on the sensible expectation that future regulations would not be a straightforward “panacea”, but would comprise only “one piece of a bigger puzzle”.¹³ By avoiding a fixed timeframe for introducing their new and potentially more forward-thinking framework, Canada’s regulators vowed instead to take whatever time necessary to meet the challenge of “getting the legislation right”.¹⁴ On 26 February 2024, following earlier regulatory attempts by Germany, the European Union, and the United Kingdom, Canada exploited its apparent “second mover” status by finally introducing Bill C-63 which, through its Online Harms Act and related amendments, proposes an innovative “systems-based risk assessment” model for regulating harmful content online.

In this article, we argue that despite the Canadian government’s enthusiasm and lofty aspirations, any truly consultative or “systems-based” approach will benefit from regulatory insights and prescriptions informed by the following two interdisciplinary sources. First, the balance of authority of constitutional and media law scholars emphasises the necessity of stepping outside conventional regulatory models by employing more “context-based” and “systems thinking” approaches—a regulatory turn seemingly consistent with Canada’s pivot towards an innovative “systems-based” model. Second, any enhanced framework aimed at “cracking the code” of digital media regulation will benefit from profound insights native to the disciplines of social medicine and diagnostic theory. Besides providing a convincing case for expanding aetiological (and regulatory) inquiry to include the effects of social and environmental signals, established principles of medical diagnosis provide a valuable self-reflexive decision-making protocol for present-day regulators. Taken together, a careful review of “systems-inspired” regulatory scholarship and medico-diagnostic principles suggests that prevailing “systems-based” models—as epitomised by Canada’s proposed Online Harms Act—would appear to function as a “blueprint” for privatised government censorship,¹⁵ providing regulators with the legislative mandate, informational transparency, and compliance authority for regulatory capture that leading scholars have long understood as one of the Internet’s “Big Picture” regulatory dilemmas.¹⁶

In the end, just as earlier medical debates between germ theorists and proponents of social medicine exposed the importance of host responses and environmental cues to our knowledge of health and illness,¹⁷ contemporary tensions in the field of digital media regulation can shed much-needed light on the dangers of untreated structural threats to the discursive health of our global body politic.

II. Global Regulatory Approaches

Despite the original aim of cyber-libertarians to create an unfettered online environment, two predominant models of Internet regulation have emerged worldwide, reflecting fundamentally different schools of thought and approaches to freedom of expression.

1. “Notice-and-action” model (NetzDG/DSA)

Typified by Germany’s Network Enforcement Act (Netzwerkdurchsetzungsgesetz - NetzDG) and Europe’s Digital Services Act (DSA),¹⁸ the “notice-and-action” model is characterised by a relatively strict regulatory approach.¹⁹ This model limits digital platforms’ speech interests by obliging them to delete or block illegal online content within prescribed periods, ranging from 24 hours to seven days. Platforms must also provide an accessible and user-friendly complaints procedure for illegal online content, and are obliged to report potentially criminal content to law enforcement authorities.²⁰ Importantly, systematic non-compliance leads to severe penalties.

Besides prompting extensive public and private co-optation, this regulatory model has suffered from ambiguous definitions of “illegal” online content: NetzDG, for example, references specific infractions in Germany’s Criminal Code, (e.g. insult and disturbances to the public peace), whereas the DSA introduces a significantly broader definition, not enumerating specific criminal provisions. This definitional ambiguity is ultimately left to digital platforms to resolve—a complex legal assessment that can cause broadly divergent results in each of the EU’s 27 Member States²¹—which places platforms in the unenviable role of powerful gatekeepers at the threshold of human rights.

2. “Market self-regulation” (USA)

Canonically associated with the United States of America, the “market self-regulation” model represents a fundamentally different approach to regulating online communications and is characterised by two essential elements. First, platforms are shielded from liability for speech torts committed on their platforms under section 230 of the Communications Decency Act (CDA). Second, the US Constitution provides an enlarged scope of protection for “offensive” speech under the First Amendment, including hate speech.²² In effect, “market self-regulation” allows platforms to determine—with minimal state interference and risk of liability—what content to carry and remove. Compared to the “notice-and-action” model, free-speech restrictions under “market self-regulation” are not imposed by government legislators, but by modifying platforms’ content moderation policies, or Terms of Use.

III. Canada’s “Systems-Based” Regulatory Proposal

Compared to the EU and America, Canada has embraced a novel “multi-stakeholder” approach to resetting its regulatory framework. In its consultative journey, the Canadian government has pivoted from conventional “notice-and-action” models to a more “systems-based” approach. By imposing a “duty to act responsibly” on digital platforms, Canada’s new Bill C-63 seeks to provide Canadian regulators with information and greater transparency about key ex ante and systemic decision-making processes taking place outside and upstream of more conventional models of ex post content review and error correction.

1. Moving from a “notice-and-takedown” to a “systems-based” model

Canada’s “multi-stakeholder” approach is notable for two particularities. Besides moving from a conventional “notice-and-takedown” to a more “systems-inspired” regulatory approach, Canadian legislators have shown a distinct preference for combating harmful online content rather than heeding and prioritising concerns expressed by the public and experts alike with rising censorship and more structural threats to democratic governance.

a) Public consultation – concerns with privatised government censorship

Following its abandonment of Bill C-36, the Canadian government began public consultations soliciting Canadians’ views on regulating harmful online content. From July to September 2021, the government requested written submissions from the public and tech-industry on its original “notice-and-takedown” regulatory model (i.e. Bill C-36), and associated technical and discussion papers.

While public respondents unanimously accepted the necessity of state intervention—as opposed to “market self-regulation”—far fewer supported the proposed legislative framework as a whole. Importantly, from the very beginning of Canada’s extensive regulatory planning, a broad cross-section of stakeholders expressed six main or “prominent” concerns on the dangers of censorship and the over-regulation of online content, relating to: (1) definitional clarity of harmful content; (2) proactive monitoring; (3) expedited takedown requirements (e.g. 24-hour rule); (4) economic drivers of platform content moderation; (5) bureaucratic overreach; and (6) transparency and accountability reporting duties.

First, respondents criticised the lack of definitional detail for online harms, warning that overly broad definitions would invite bias and could have a chilling effect that might “create a broader trend toward over-censorship of lawful expression writ large”.²³ Second, quite aside from its present-day reality, stakeholders expressed concern that a general proactive monitoring obligation on platforms would be extremely problematic as it would “likely […] amount to pre-publication censorship”, and ultimately “operate as a de facto system of prior restraint”.²⁴ Third, many respondents called for removing the 24-hour takedown rule borrowed from Germany’s NetzDG, arguing that “it would incentivize platforms to be over-vigilant and over-remove content […]”.²⁵ Fourth, multiple respondents keenly observed that rather than focus exclusively on content moderation, regulators should target “the economic factors that drive platform design and corporate decision making”,²⁶ including other “[…] structural factors like advertising practices, user surveillance, and algorithmic transparency […]”.²⁷ Fifth, despite the overall enthusiasm for urgent regulatory intervention, stakeholders questioned “the number of regulatory entities, emphasizing potential overlaps in authority and the sheer size of the proposed bureaucratic structure dedicated to ‘censoring’ online expression”.²⁸ Sixth and finally, moderate concern was expressed about transparency and accountability requirements. As one of the most powerful governance tools, respondents hoped that mandated and audited transparency could operate as “important safeguards to mitigating the regime’s potential for over-removal and censorship”.²⁹

b) Expert consultation – pivoting to a “systems-based” regulatory approach

The second phase of Canada’s “multi-stakeholder” approach involved the solicitation of expert advice. In March 2022, an Expert Advisory Group on Online Safety (EAG) was convened composed of Canadian experts in platform governance and content regulation, online harms, civil liberties, informatics, and national security. Its dual mandate was to provide insights and recommendations on how best to design a legislative and regulatory framework to address harmful online content, and to advise on “how to best incorporate the feedback received during the national consultation […]”.³⁰ Like ordinary Canadians, the EAG endorsed state regulation, proclaiming that online safety “cannot be left to the good graces of industry players”.³¹

Remarkably, while two of the five censorship concerns voiced in the public consultation were taken up by the EAG (i.e. definitional clarity and proactive monitoring), the remaining three worries were effectively downplayed or disregarded. While expert comment was anonymised by the government, the issue of generalised or proactive platform monitoring was mentioned repeatedly in two of the ten EAG workshops. When advising on the appropriate types of regulatory content, multiple experts worried that “whatever framework is chosen, it would be critically important that it not incentivize a general system of monitoring”.³² When experts turned their minds to evaluating the new regulatory approach under consideration, some stressed that “there is a risk that a systems-based approach could indirectly promote a system of general monitoring”, advising that “each legislative provision must be scrutinized to ensure no general monitoring obligation exists […]”.³³ Moreover, besides confirming earlier concerns with definitional uncertainties regarding harmful content,³⁴ the EAG expanded these to include the new framework’s proposed “duty to act responsibly”.³⁵ Experts cautioned that “if regulated services are not told how to comply with their duty to act responsibly, the systems they put in place might be rudimentary and result in blunt over-regulation […]”.³⁶

Notwithstanding other minor and less-specific references to freedom of expression and government censorship, the EAG took particular interest in regulating disinformation, with most experts agreeing that “the Government cannot be in the business of deciding what is true or false online, or of determining intent behind creating or spreading false information”.³⁷ In a statement reminding Canadians of the grave dangers of regulatory capture, most EAG members insisted categorically that “the Government [cannot] censor content based on its veracity, no matter how harmful”.³⁸ Finally, unlike the more critical and far-reaching citizen concerns with the economic drivers of online censorship—which was more amenable (at least in theory) to acknowledging the economic foundations of over-filtering and over-blocking—some members of the EAG highlighted the importance of only the financial and economic drivers of disinformation. Apparently unwilling or reluctant to contemplate the relationship between economic motives and online censorship, these experts nonetheless suggested that successful answers to disinformation may lie beyond regulatory reach if advertising law and practices were not altered to effectively “demonetize disinformation”.³⁹

At last, apart from these relatively few and abridged regulatory concerns, previously vetted worries with rising regulatory capture and privatised government censorship did not appear to resonate as strongly with Canada’s expert panel.

c) Citizens’ assemblies on democratic expression and national roundtable discussions

The final phases in Canada’s lengthy consultative process involved important input from the Canadian Commission on Democratic Expression and the Department of Canadian Heritage, which provided vital feedback on the EAG and the state of regulatory input to date. Importantly, as with initial public consultations, significant concerns were again expressed about the dangers of censorship and avoiding over-regulation of speech interests.

aa) “Capstone” assembly on democratic expression – protecting dissenting opinions

Following the EAG’s counsels on how best to design a regulatory framework for addressing harmful online content, Canadian Heritage requested a third and final Citizens’ Assembly on Democratic Expression to review and respond to the EAG’s suggestions and all work that had preceded its input and efforts. At stake in the minds of many members of this “capstone” assembly was nothing less than the future of Canadian democracy.⁴⁰

Although reflecting the emerging consensus on the urgent need for state regulation, this second public consultation again acknowledged the vital importance of avoiding censorship and over-regulation of free expression. First, Assembly members expressed concern that “online users […] be able to share dissenting or unpopular opinions”,⁴¹ and that any risk-based model contain appropriately “strong whistle-blower protections”.⁴² Second, comparable to feedback from the first public consultation in 2021, Assembly members pointed out the detrimental economic implications and overall costs of digital platforms’ business models and over-reliance upon click-through ads in our digital “attention economy”, warning that platforms’ overriding “goal of profit from advertising sales comes at a detrimental cost, and with great disregard, to the well-being of our society”.⁴³

bb) National roundtable discussions – misapprehending economic regulatory motives

Finally, in July 2022—shortly after the EAG completed its work—the Canadian government conducted 19 nationwide roundtables to incorporate victim and platform perspectives on the EAG’s advice and recommendations.⁴⁴

As confirmed throughout the consultative process, consensus was again reached over the urgent need for state regulation of harmful online content. Still, evidencing an overall concomitant fading of concern with censorship and over-regulation, participant feedback was limited to passing references to the dangers of government involvement in regulating disinformation, and the regulatory implications of platforms’ business models. Echoing the EAG’s insistence that the government cannot be deciding what is “true” or “false” online, roundtable participants were greatly uneased “at the notion that the government should be the entity deciding what material constitutes misinformation and disinformation”.⁴⁵ Importantly, this feedback provided yet more evidence of persisting confusion over the scope of effects of economic factors on content moderation. Many participants expressed concern only about their impact on delaying removal of harmful online content, voicing scepticism over “the willingness of social media platforms to self-regulate content […] due to the site traffic and revenue the content can generate”, and “platforms prioritizing profits rather than monitoring content […]”.⁴⁶ Besides implicitly endorsing proactive monitoring, overlooked again was the impact of economic drivers on over-filtering and over-blocking, and the more veiled dangers of privatised government censorship.

2. Bill C-63: Canada’s latest regulatory framework

On 26 February 2024, Canada introduced Bill C-63⁴⁷—its long-awaited regulatory framework for addressing harmful online content. Besides amending (among others) the Criminal Code and the Canadian Human Rights Act (CHRA), Bill C-63 introduced the Online Harms Act, intended to make good on its earlier promise to Canadians of “getting the legislation right”.

Besides imposing sensible duties to protect children and to make non-consensually distributed intimate images and child pornography inaccessible in Canada within 24 hours, the Online Harms Act imposes on digital platforms a “duty to act responsibly” by implementing measures to mitigate the risks that users will be exposed to harmful content. This negligence-based duty requires (above all) that platforms submit regular Digital Safety Plans—containing detailed risk assessments, mitigation strategies, and evaluations of their efficacy—to a newly established Digital Safety Commission of Canada, whose mandate would be administering and enforcing the Act. Besides this governing regulatory body, the proposed Act also establishes a Digital Safety Ombudsperson to support users of regulated services and to advocate for the public respecting systemic online safety issues, and a Digital Safety Office of Canada to provide administrative support to the two newly-created agencies.

Consistent with Canada’s regulatory focus on combating harmful online content, Bill C-63 includes three vital harm-related provisions. First, the Online Harms Act adds two additional categories of harm (i.e. child bullying and self-harm) to the following five categories discussed throughout Canada’s consultative process, namely: (1) content that sexually victimises a child or revictimises a survivor; (2) intimate content communicated without consent; (3) content that foments hatred; (4) content that incites violent extremism or terrorism; and (5) content that incites violence. Second, Bill C-63 amends the Criminal Code by: (1) proposing a long-awaited definition of “hatred”; (2) creating a controversial standalone “hate crime” offence (liable to imprisonment for life) that applies to existing criminal offences and parliamentary acts motivated by hatred;⁴⁸ (3) increasing penalties for existing hate crimes; and (4) instituting a new “peace bond” designed to prevent the commission of hate crimes and offences. Third and finally, Bill C-63 aims to reinstate Section 13 of the CHRA to make it a discriminating practice “[…] to communicate or cause to be communicated hate speech by means of the Internet or any other means of telecommunication […]”,⁴⁹ thereby broadening the scope of remedies for victims of online harm.

In the end, notwithstanding the broad range of public and expert concern voiced over the dangers of censorship and over-regulation of speech interests during its extended consultation process, Canadian legislators appear to have focused disproportionally on harmful content at the expense of addressing lower-salience structural threats to democratic governance.

IV. Differential Diagnosis in Online Governance

After introducing Canada’s new “systems-based” framework, Part IV demonstrates that reliable indications as to its optimal form and content can be discerned from two key interdisciplinary sources: (1) constitutional and media law scholarship emphasising the necessity of employing “context-based” and “systems thinking” approaches to online regulation; and (2) profound regulatory insights native to the fields of social medicine and diagnostic theory. Taken together, these confirm that future regulatory models must openly embrace synthetic enquiry and careful avoidance of overly-reductionist approaches to online dysfunctions.

1. “Systems thinking”: Stepping outside conventional regulatory models

The nature and limitations of Canada’s “systems-based” model can be first gathered from leading constitutional and media law scholars who collectively endorse: a) adopting more structurally sophisticated means of integrating socio-technical-legal elements into regulatory theory and design; b) adopting novel “context-based” approaches to digital platform liability; and c) reframing content moderation in terms of “systems thinking”. Despite developing such insights within relatively narrow fields of reference, these scholarly efforts assist greatly in envisioning an integrative perspective on online regulation.

a) Multi-ordinal mapping of digital information flow

One of the most challenging aspects of ongoing technological advances in cyberspace has been reconciling their disruptive regulatory effects (and failures), and identifying the details and guiding principles for an effective global framework of Internet governance.⁵⁰ Central to this aim has been confronting the “shaky” theoretical grounds underlying current regulatory structures and—given the Internet’s clash with the principle of territoriality—embedding technological advances into an effective global system.⁵¹ Despite a lack of consensus about the conceptual grounds of online regulation, scholars have agreed on an important feature about its structural complexity. Reflecting hard-won lessons of legislators worldwide, commentators insist that “a single concept cannot explain the complex structure of cyberspace” and hence resort to some form of “systems-inspired” or “interrelated thinking seems unavoidable”.⁵²

aa) Murray’s three-dimensional “complexity matrix”

An important early contribution to defining possible future perspectives on Internet governance was provided by Andrew Murray.⁵³ Writing in an earlier online era focused on optimising digital information flow, Murray’s principal insight was that cyberspace is a complex, even chaotic, environment that requires legislators to employ a “[…] more cohesive, measured, prudent and non-interventionist approach”.⁵⁴ Distinguishing his pioneering regulatory theory from earlier “cyber-libertarian” and “cyberpaternalist” models, Murray’s “complexity thesis” rejected their joint assumption of a static regulatory setting by endorsing a more dynamic model capturing the complexities of State and private sector actors. Murray advised that by recognising parties’ dual roles as “regulator” and “regulatee”—and adopting a more dynamic “systems-inspired” view of the regulatory environment—legislators “[…] are offered the opportunity to produce effective complimentary regulation”.⁵⁵ Accordingly, in his bid to minimise disruption and to harmonise regulatory efforts with policy outcomes—both aims resonant with autopoiesis theory—Murray’s contrasting model of “symbiotic regulation” endorsed a distinctive protocol harnessing the complex relationships between the various regulatory actors.⁵⁶

Inspired by these biological and remedial concepts, Murray introduced a novel three-dimensional matrix for structuring and regulating complex, digital media environments.⁵⁷ According to Murray, successful online regulation requires that the complexity of the broader media environment be accurately mapped, including the communications networks already in place.⁵⁸ Recognising that “all actors in the regulatory environment play an active role […]”,⁵⁹ interventions in such complex networked systems are fundamentally indeterminate in that “[…] the complexity of the matrix means that it is impossible to predict the response of any other point […]”.⁶⁰ This however does not mean that cyberspace is fundamentally unregulable. Quite the contrary. Owing to the overall “malleability of its environment”,⁶¹ Murray insisted that our online environment is highly amenable to regulation using a reflexive three-step process.

The first step is to produce a dynamic model of the regulatory environment, being careful to record all relevant parties and to map their primary communication dynamics. The focus is not on capturing actual content, but on mapping the relationships between actors well enough to “anticipate the regulatory tensions that are likely to arise […]”.⁶² Second, based on the accuracy and comprehensiveness of this initial environmental modeling, regulatory interventions can be optimally formulated to anticipate and avoid regulatory tensions between its main actors, thereby offering a positive communication “to the subsystems, or nodes, within the matrix […]”.⁶³ Murray further specified that these regulatory interventions are “intended to harness[] the natural communications flow by offering to the subsystems, or nodes […] a positive communication that encourages them to support the regulatory intervention”.⁶⁴ Third, regulatory interventions must then be tested by monitoring positive and negative nodular feedback. According to Murray, whether aiming to reinforce already successful regulations, or to engender modifications directed at enhancing deficient regulatory outcomes, “[…] regulator[s] should be prepared in light of this feedback to make alterations in their position and to continue to monitor feedback on each change […]”.⁶⁵ By following this three-stage process regulators are, according to Murray, best equipped “to design successful […] interventions in the most complex regulatory environment”.⁶⁶

At last, while criticised as being “difficult to implement” and “[…] impossible to adequately carry out”,⁶⁷ Murray’s “complexity thesis” nonetheless remains a vital early contribution to confronting rising challenges of regulating complex networked environments.

b) “Context-based” approaches to regulating platform liability

A second indication as to the nature and limitations of Canada’s “systems-based” regulatory model can be gathered from examining the underlying bases of platform liability. Several forward-thinking scholars have endorsed a broad array of “context-based” models.

aa) Lavi’s “descriptive social technological” model

A significant early contribution to online regulatory theory and design in the social media era was Michal Lavi’s innovative “context-based” model.⁶⁸ Aiming to reconcile tensions between prevailing legal rules and the attribution of liability for online speech torts, Lavi noted presciently that our modern-day digital media ecology places the right to free expression and its underlying justifications decidedly in “a new light”.⁶⁹ Concerned particularly about the “chilling effect” of holding content providers liable for speech torts committed on their platforms, Lavi cautioned that a single, overarching regulatory approach would be “insensitive to different online contexts and lead to distortions and improper [regulatory] consequences”.⁷⁰

In response, Lavi endorsed an innovative “descriptive social technological” model erected on a three-level conceptual taxonomy for matching liability rules to an overarching sociological criterion that measures the strength of social ties and their potential for causing harm. By dividing digital platforms into three categories with increasingly strong social ties—(1) “freestyle platforms” (e.g. Yahoo! Message board); (2) “peer production platforms” (e.g. Yelp and other user review sites); and (3) “deliberation and structuring communities” (e.g. Meta (formerly Facebook, X (formerly Twitter), and other social networks)—in simplest terms, Lavi proposed a model of “differential liability regimes”,⁷¹ arguing that since platforms’ various technical and functional capabilities influence speech-related harms differently, liability should increase concomitantly with each platform’s potential for doing so. That is to say, whenever the severity of harm is low and there is a substantial likelihood for private ordering, legal regulations are unnecessary. But where the social media context increases harm to external victims and results in a failure of private ordering, content providers should not be granted legal immunity (e.g. under section 230 CDA), and should be subject to some form of “notice-and-takedown” procedure.⁷² Consistent with earlier warnings against the impracticality of Murray’s “complexity thesis”, Lavi advised that her regulatory model—along with “context-based” approaches generally—might provide courts and legislators with a more practical alternative—“[…] a simple rule of thumb for defining content providers’ scope of liability”.⁷³

Importantly, the regulatory implications of Lavi’s “context-based” model extend well beyond issues of doctrinal coherence. Reiterating concerns of lower-salience structural threats to democracy advanced by leading free speech scholars like Jack Balkin,⁷⁴ Lavi stressed that the fundamental motive for platform content moderation is “economic and not driven by legal considerations”.⁷⁵ This point is critically important not only for “optimally balancing” competing policy rationales underlying platform liability, but to identifying the “root causes” of over-filtering, over-blocking, and acknowledging the potential for and dangers of privatised government censorship—structural concerns vital both to the maintenance of a healthy marketplace of ideas, and for effectively holding power to account.⁷⁶

At last, besides the utility of Lavi’s model for ensuring doctrinal coherence and reform, it also attests to the regulatory dangers of ignoring the discomfiting reality that the “economic logic” driving platform content moderation too often conflicts with human rights norms, particularly free speech and its vital “checking function” rationale.⁷⁷

bb) Sander’s “structural” human rights law model

A second valuable contribution to online regulatory theory and design in the social media era was Barrie Sander’s “structural” human rights law model.⁷⁸ Building on many of the “context-based” regulatory insights noted earlier, Sander argued that shifting to a more structural conception of human rights law would—by broadening Lavi’s approach to platform liability even further—require “[…] a more holistic and evidence-based approach to the design of intermediary liability laws that strives to account for the systemic effects of such frameworks on online expression”.⁷⁹ Calling for greater state protection of free speech, Sander’s “structural approach” to regulating online content requires that sufficiently “[…] robust mechanisms of transparency, due process, accountability and oversight are embedded in platform moderation systems […]”,⁸⁰ including government and cross-platform collaborations.

By examining content moderation (and data protection) liability within the wider context of rising accountability deficits pervading our digital media ecology,⁸¹ Sander took aim at the prevailing “marketized” model of human rights law in our “increasingly, privately controlled, neoliberal communication sphere”.⁸² In particular, Sander argued that a marketised conception premised on the laissez-faire notion of “[…] protect[ing] individual choice and agency against state intervention” is problematic for two reasons.⁸³ First, it endorses a form of abstract individualism that “[…] neglects power asymmetries between individual users and other actors that participate in the social media ecosystem […]”.⁸⁴ Second, it pays limited attention “[…] to the systemic effects of state and platform practices on the social media environment as a whole”.⁸⁵

In response, Sander endorsed a “structural” conception of human rights law, one typified by “a greater openness to positive state intervention as a means of safeguarding public and collective values such as media pluralism and diversity”.⁸⁶ By doing so, Sander aimed to not only contest the use of human rights discourse in the realm of social media governance,⁸⁷ but to “[…] begin to close the accountability deficits associated with content moderation […]” that increasingly threaten our democracies.⁸⁸ While leaving the regulatory details unspecified, Sander’s commitment to preserving the “functionality” of our digital public sphere provides important normative grounds for expanding our regulatory toolbox to include “common carrier” doctrine for mitigating platform censorship and increasing the quantity and diversity of democratic discourse.⁸⁹

In the end, when interpreted in light of Murray’s three-dimensional “complexity matrix” and Lavi’s “descriptive social technological” model of platform liability, Sander’s model again attests to the vital importance for online regulators of turning their minds to the broader regulatory environment—including its primary stakeholders’ economic motives and discursive predilections—for clues to calibrating our regulatory interventions to better promote international human rights, domestic policy goals, and the health of our online environment.

c) Content moderation as “systems thinking”

A third indication as to the nature and limitations of Canada’s “systems-based” model can be inferred from scholarship endorsing a “second wave” of more sophisticated regulatory frameworks for online content moderation. Looking to step outside overly reductionist models, legal scholars have continued to incorporate key concepts and insights from systems theory to optimise our understanding and regulation of today’s digital media environment.

aa) Douek’s “monitored self-regulation” model of content moderation

A third notable contribution to online regulatory theory and design in the social media era was Evelyn Douek’s ambitious reframing of content moderation (and its regulatory dynamics) in terms of “systems thinking”.⁹⁰ Arguing that today’s content moderation models (e.g. “notice-and-action” and “market self-regulation”) are equally outdated, misleading, and incomplete,⁹¹ Douek claimed that the “blind spots” and mistaken assumptions of this “standard” regulatory picture—a “first wave” of regulation focused incorrectly on ex post review of individual online posts and error correction—must be updated and replaced with a “second wave” capturing the underlying “patterns and interrelationships” of our modern regulatory landscape. As Murray foresaw a generation earlier, Douek maintained that content moderation is ultimately a complex and dynamic system of “mass speech administration”,⁹² which requires wide-ranging procedural design interventions focused more on “[…] systems rather than individual cases, on wholes and interrelationships rather than parts, and on ‘patterns of change rather than static snapshots’”.⁹³

Starting from the sensible bases that “there will never be agreement on what constitutes ‘good’ content moderation”⁹⁴ and—perhaps most importantly—that “the status quo of private companies determining matters of […] public significance without any form of accountability, transparency, or meaningful public input is inadequate”,⁹⁵ Douek’s main regulatory objective involves achieving “meaningful accountability” by reframing content moderation as a complex and dynamic administrative system.⁹⁶ Endorsing a self-styled “substance-agnostic” approach,⁹⁷ Douek’s regulatory framework draws on familiar “principles and practices of administrative law”,⁹⁸ focused more on “key ex ante and systemic decision-making” taking place outside and upstream of the standard picture’s familiar “assembly line” of ex post individual review and error correction. Rather than providing “substantive” reforms, Douek’s overriding objective of mitigating online “accountability deficits”—a policy aim endorsed earlier by Sander—requires adopting two coordinate sets of structural and procedural reforms.

First, any proper system of “mass speech administration” must begin by restructuring internal platform moderation bureaucracies to avoid unreported bias and to incentivise neutral enforcement of their Terms of Use.⁹⁹ Douek’s “separation of functions” principle hence requires intra-corporate separations of personnel and functions “that aim to ‘eliminate the incentives that would make [biased] conduct possible or likely in the first place’”.¹⁰⁰ Second, rather than relying on “user-initiated complaints in individual cases”,¹⁰¹ a more comprehensive governance framework must authorise a suitable regulatory body—as reflected by Canada’s proposed Digital Safety Commission—to operate an “external channel” for fielding complaints and conducting its own investigations. Third, to best facilitate regulatory oversight of complex content moderation systems, platforms should be required “to disclose the nature and extent of involvement of outside decisionmakers in their content moderation […]”,¹⁰² including external “fact-checkers” and (at least in theory) government agencies. Lastly, as accepted by Canadian legislators, Douek proposed a scheme of regular platform reporting obligations (i.e. Digital Safety Plans) designed to expose “the broader functioning of their [content moderation] systems”,¹⁰³ which purports only to improve accountability and to prevent regulators from “legislating in the dark”.¹⁰⁴

Besides these structural reforms, Douek argued that optimising regulatory accountability requires digital platforms to comply with three procedural fiats. First, while admitting that platform self-reporting “may sound like a feeble form of accountability”,¹⁰⁵ and that the “[e]mpirical effects of speech regulation are deeply contested”,¹⁰⁶ platforms should nonetheless produce “annual content moderation plans and compliance reports”.¹⁰⁷ Besides forcing them “to think proactively and methodically about potential operational risks”,¹⁰⁸ as illustrated by Canada’s proposed Digital Safety Plans, Douek maintained that such disclosures can benefit regulatory efforts by: (1) creating a “paper trail” of platform decision-making that “facilitat[es] future review and accountability”;¹⁰⁹ (2) facilitating policy learning by encouraging “cross-industry reporting” and formulating “general compliance standards” or “best practices”;¹¹⁰ and (3) much like Canada’s own consultative approach, facilitating public involvement through “multi-stakeholder” engagement into proposed regulations.¹¹¹ Regardless of their efficacy, Douek sensibly insisted that as “a necessary first step to more sweeping reform”, we must first admit that “[t]here is […] no way of currently knowing what platforms have been doing, what works, and what doesn’t”.¹¹²

Douek’s second procedural proposal also aimed to improve informational transparency, in this case by requiring platforms both to demonstrate that “they have quality assurance […] measures in place for their decision-making systems”¹¹³—a core internal administrative law requirement—and to subject such self-assurances of “quality” to “independent auditing”.¹¹⁴ As Douek rightly cautioned, without independent verification, such “[…] transparency reports could be as accurate as Enron’s financial statements […]”.¹¹⁵ A third and final procedural recommendation would require platforms to offer an “aggregated review mechanism[]”.¹¹⁶ Instead of mandating appeals and procedural protections for individual online users, Douek insisted that to better identify and address system-wide trends, patterns, and failures, platforms should “review, as a class, all adverse decisions in a certain category of rule violation over a certain period […]”.¹¹⁷ Drawing on analogies to the EU’s data protection regime (i.e. General Data Protection Regulation), Douek professed that these structural and procedural proposals together amounted to a model of “monitored self-regulation”, one that is more dynamic, better at leveraging the particular capacities of private and public actors, and can generate a virtuous cycle of continuous iterative improvements.¹¹⁸

In the end, despite Douek’s worthy aim of prompting a “second wave” of content moderation theory and regulatory design, many important aspects of her framework remain underdefined extensionally (e.g. capturing the extent of regulatory activity in our global public sphere),¹¹⁹ and significantly undertheorised—ironically in the areas of “systems-theory” and accountability scholarship.¹²⁰ Owing to perfunctory engagement with these vital foundational materials—and adopting an unnecessarily narrow view of “digital platforms” as the main unit of regulatory analysis—Douek’s model leaves the following broader regulatory issues unexamined: (1) the rising structural threats to democracy posed by the Internet’s ad-based business model, including its impact on over-filtering and over-blocking, and its overall effects on the quantity and quality of democratic discourse; and, (2) the implications of a “systems-based” model for facilitating regulatory capture and sanctioning (perchance unintentionally) privatised government censorship.

2. A way forward: Regulatory insights from social medicine and diagnostics

Despite these residual scholarly gaps, perhaps the most valuable lesson that has emerged from our review of “systems-inspired” models is harnessing their collective capacity for optimising regulatory “diagnosis and improvement”—an important remedial goal of Douek’s model.¹²¹ Taking up this implicit mantle, further indications as to the nature and limitations of Canada’s proposed regulatory model are afforded by expanding our inquiry into the instructive parallels between the legal and medical sciences.

a) Insights from social medicine and theoretical biology

As we have maintained in the past,¹²² any regulatory framework aimed at “cracking the code” of online communications will benefit from exploring the considerable synergies between law and medicine.¹²³ Recommending this same source of interdisciplinary insight when searching for suitable regulatory interventions in cases of constitutional limitation or infringements on liberty, US Supreme Court Justice Benjamin Cardozo encouraged both courts and legislators alike to increasingly turn to “[…] medicine—to a Jenner or a Pasteur or a Virchow or a Lister as freely and submissively as to a Blackstone or a Coke”.¹²⁴ Poised on the crest of revolutionary twentieth-century advances in theoretical physics, Justice Cardozo’s open-minded views have since only gained in currency in light of powerful insights generated by these new scientific paradigms within the fields of social medicine and theoretical biology.

aa) Importance of social and environmental signals to public health regulation

One specially revealing nineteenth-century German medical anecdote (and pioneering medical figure) bears mention. It concerned a typhus epidemic that broke out in the winter of 1847 in Upper Silesia, an economically depressed Prussian province. The epidemic coincided with a famine, and conditions deteriorated so badly that government intervention became necessary. Following time-honoured practice, an outside expert was appointed to survey the situation and submit a regulatory report. The individual chosen for this seemingly routine task was the physician Rudolf Virchow, then aged 26 years, and a junior lecturer in pathology at the Charité Hospital in Berlin.

The report based on his three weeks’ observation was revolutionary for its time and even now sets a standard for attempting to understand and change the social conditions that produce disease. Conspicuously, Virchow’s ‘medical’ proposals were quite limited. Since he based the origins of ill health in broader social conditions, the most reasonable regulatory approach to addressing the Upper Silesian ‘epidemic’ was to identify and alter the underlying factors that permitted it to occur. Virchow reasoned:

Don’t crowd diseases point everywhere to deficiencies of society? One may adduce atmospheric or cosmic conditions or similar factors. But never do they alone make epidemics. They produce only where due to bad social conditions people have lived for some time in abnormal situations. Typhus would not have spread epidemically in Upper Silesia if there had not been a physically and mentally neglected people […].¹²⁵

Evidencing a growing awareness of the complex interrelationships between medicine, social conditions, and political reform, Virchow later insisted that if medicine was to fulfill its great task, then it must enter the public realm, famously declaring:

Medicine is a social science, and politics is nothing else but medicine on a large scale. Medicine, as a social science, as the science of human beings, has the obligation to point out problems and to attempt their theoretical solution: the politician, the practical anthropologist, must find the means for their actual solution […].¹²⁶

Insisting that “[t]he physicians are the natural attorneys of the poor, and the social problems should largely be solved by them”,¹²⁷ Virchow envisioned a medical profession that obliged physicians to investigate the complex relationships between socio-political stressors and corporeal experience. Virchow’s intriguing reversal of the traditional roles of doctors and lawyers was borne from a deep conviction that medicine’s clinical realities must inform society’s organisation and structure, predominantly through careful design of its laws and regulations. Stressing their importance as society’s dominant prescriptive force, Virchow stated: “If medicine is the science of man both healthy and ill, which after all it should be, what other science could then be more appropriate to deal with law-making, in order to apply the laws that are given in mankind’s nature to the foundations of the organization of society”?¹²⁸

At last, while Virchow’s inquiries into the social origins of illness were to help establish the interdisciplinary scientific field of “social medicine”, these issues quickly fell from sight owing to more reductionist scientific developments that shaped the course of medicine during the late-nineteenth century—particularly the germ theory of disease.¹²⁹

bb) The biopsychosocial response: A “systems-based” paradigm of health and illness

The urgency for developing a new medical paradigm responsive to such diagnostic blind spots was reinforced by George Engel.¹³⁰ In Engel’s view, medicine was in crisis because of its adherence to a disease model that was no longer adequate for its scientific tasks and social responsibilities.¹³¹ Like Virchow before him, Engel hoped for an epistemological shift in medical science focused on greater interaction, with renewed emphasis on defining adaptive genetic and epigenetic limitations as they are set by broader social and environmental signals. Arguing for a revolutionary “systems-inspired” biomedical paradigm—one typified by a transactional, holistic, analogical, and probabilistic approach—Engel effectively confirmed Virchow’s more tentative causal inferences, instructing:

No linear concept of etiology is appropriate; rather, the pathogenesis of disease involves a series of negative and positive feedbacks with multiple simultaneous and sequential changes potentially affecting any system of the body. The central nervous system is so organized that a reciprocal interrelationship between the mental apparatus and the rest of the body in the pathogenesis of disease states and maintenance of health is not only possible but inevitable.¹³²

Among its implications, Engel’s general systems theory-inspired “biopsychosocial” model requires physicians to explore complex relationships between social stressors and bodily experience, to study how the corporealisation of cultural experience occurs, and to explore humanity’s adaptive limits to rising levels of immunological stressors. Reflecting the “systems thinking” that led Rudolf Virchow to designate nineteenth-century physicians “the natural attorneys of the poor”,¹³³ this new model implicated physicians in wider political debates from which modern conceptions of suffering and disease often insulate them, a point shown by containing suffering within the sole rubric of prevailing (and potentially misleading) microbiological and genetic disease models.¹³⁴

In the end, Engel anticipated that as the social bases of health and illness were gradually revealed, new avenues of research could be opened in precisely the way that Thomas Kuhn had in mind—generating a “systems-inspired” paradigm shift in medical science that might through its example advance broader socio-political regulations.¹³⁵ That is, Engel’s “biopsychosocial” paradigm might yet inspire and foster amongst today’s regulators a similar perspectival shift in global online governance—in this case, to a more scientifically probing and less ideologically encumbered and contextually reductionist “systems-inspired” approach.

b) Regulatory insights from medical diagnostics

Besides these structural insights from social medicine and theoretical biology, valuable clues for designing “systems-inspired” regulatory models can also be grasped from the principles and methods of medical diagnostics.

aa) The diagnostic process: “Clinical reasoning” in conditions of uncertainty

Instructive synergies between “systems-based” regulatory approaches and the principles and practices of medical diagnosis can be shown by analysing the latter’s three conceptual pillars.

First, and above all, diagnosis is a process.¹³⁶ As with “systems-based” models committed to optimising “learning and iterative” regulatory outcomes, medical diagnosis consists of a similarly cyclical and “continuous process of information gathering, integration, and interpretation [that] involves hypothesis generation and updating prior probabilities as more information is learned” about hidden dysfunctions.¹³⁷ Moreover, similar to regulatory measures directed at rectifying dysfunctions in complex networked environments, the diagnostic process encompasses a self-reflexive method of “modification and refinement” that operates under conditions of regulatory uncertainty.¹³⁸ As Professor Jerome P Kassirer, MD explained:

Absolute certainty in diagnosis is unattainable, no matter how much information we gather, how many observations we make, or how many tests we perform. A diagnosis is a hypothesis about the nature of a patient’s illness, one that is derived from observations by the use of inference. As the inferential process unfolds, our confidence as [clinicians] in a given diagnosis is enhanced by the gathering of data that either favor it or argue against competing hypotheses. Our task is not to attain certainty, but rather to reduce the level of diagnostic uncertainty enough to make optimal therapeutic decisions.¹³⁹

Of upmost relevance to regulatory interventions, a critical issue through the diagnostic process then is deciding when sufficient information has been obtained to make a reliable diagnosis.

Second, this shared decision-making context of “diagnostic indeterminacy” has inspired a common evaluative approach. Namely, much like the importance of political experience and judgment to formulating useful legislative measures, “[a]ccurate, timely, and patient-centered diagnosis relies on proficiency of clinical reasoning”,¹⁴⁰ an evaluative process that involves the proper exercise of “judgment under uncertainty”.¹⁴¹ Based “[…] within clinicians’ minds (facilitated or impeded [contextually] by the work system)”,¹⁴² and influenced by “dual process theory” (i.e. a combination of analytical and non-analytical models), clinical reasoning has been defined by the National Academy of Sciences “[…] as the clinician’s quintessential competency”—being “the cognitive process that is necessary to evaluate and manage a patient’s medical problems”.¹⁴³

Third, the conceptual model of medical diagnosis also demonstrates—not unlike Murray and Sander’s “systems-inspired” regulatory models—that the diagnostic process takes place within a complex, dynamic, and interrelated context (i.e. “work system”), consisting of: (1) diagnostic team members; (2) tasks; (3) technologies and tools; (4) organisational factors; (5) the physical environment; and (6) the external environment. As with “systems thinking” more generally, it is crucial to recall that—like Murray’s “complexity thesis” and the many levels of abstraction involved in Engel’s “biopsychosocial” model—this diagnostic “work system” provides the inescapable context within which evaluative decision-making occurs, meaning—perhaps, above all—that “[a]ll components of the work system interact, and […] affect the diagnostic process […]”.¹⁴⁴ In short, all is relational.

bb) Regulatory lessons: Indeterminate interventions in multi-ordinal environments

As seen from medical diagnostics’ three conceptual pillars, the parallels between the decision-making processes and requirements of clinical reasoning and “systems-based” online regulatory models are salient, pointing to several key lessons.

First, there exists a striking similarity-of-structure between Murray’s earlier regulatory proposals and the nature of diagnostic science. Despite his settled view of the indeterminacy of the online environment, Murray’s conviction of its malleability and amenability to regulation prompted endorsement of a “three-step” protocol remarkably like the diagnostic process. His self-reflexive stages of environmental mapping, regulatory interventions, and evaluation and incorporation of regulatory feedback essentially restate the three diagnostic stages of information gathering, integration and interpretation, and updating working hypotheses.

Second, as also forecasted by Murray and his “complexity thesis”—much like reframing health and illness within a broader biopsychosocial framework—cyberspace must be similarly understood as a complex networked environment.¹⁴⁵ Besides Murray’s regulatory call for a “non-interventionist” approach,¹⁴⁶ the self-reflexive method driving medical diagnosis speaks (at the very least) to the fundamental procedural necessity of engaging in unbounded probing of potential aetiological (or regulatory) factors well before ending the investigative process. Freed from unnecessary ideological impediments and investigatory blind spots, “systems-inspired” regulatory approaches must take seriously a full panoply of potential causal/aetiological factors. In other words, before regulatory problems can be effectively “overcome”, all relevant factors must first be tabled for consideration.

Lastly, this commitment to minimally encumbered scientific investigation significantly amplifies the structural regulatory concerns of Murray, Lavi, and Sander. By incorporating the broader “work system” into the diagnostic process—and its implicit recognition of the causal influences of the “physical” and “external” environments—scientific inquiry is not only freed from “blind spots” compromising our diagnosis of hidden dysfunctions, but for crafting suitable prescriptions or “treatments”. Importantly, our comprehensive review of leading “systems thinking” models demonstrates that even together they exhibit insufficient attention to confirming the systemic effects and prescriptive implications of state regulations and content moderation practices on the overall health of our digital public sphere. Whether in regulatory or academic contexts, more work needs to be done. When considered in light of Canada’s proposed Online Harms Act, the relevantly overlooked “social and environmental signals” would appear to be the economic drivers of contemporary digital censorship (i.e. over-filtering and over-blocking), and the relationship between its “systems-based” transparency obligations and the rise of privatised government censorship—factors intuited by average Canadians, but not taken up satisfactorily by either of their expert advisors or political representatives.

V. Conclusion

As we have seen, with the possible exception of Murray’s original “complexity thesis”, growing appeals to “systems-based” online regulatory approaches by legal commentators and regulators alike would appear to be at considerable risk of overpromising and underdelivering. It is more than ironic that whilst engaging in a comprehensive review of this burgeoning “second wave” of “systems-inspired” regulatory material, it remains difficult (if not impossible) to acquire a full complement of the “patterns and interrelationships” that Canadian legislators initially seemed so desperate to acquire. Despite their individual contributions, what remains to be done—indeed, very much like acquiring missing pieces of “a bigger puzzle”—is incorporating each scholar’s theoretical contributions and insights into a broader, composite regulatory framework better suited to tracking the systemic effects of state and platform practices on the overall social media environment. A critical and largely ignored component of any genuine “systems-inspired” regulatory approach must be to embrace systemic causation.

This need for adopting a more integrative approach to online phenomena was also shown by profound insights native to social medicine and diagnostic theory. Besides providing a convincing case for expanding aetiological (and regulatory) inquiry to include the effects of social and environmental signals, established principles of medical diagnosis also provided a valuable decision-making protocol for online regulators. Here too, our extensive review of leading “systems-inspired” regulatory models indicates that the nearest we can expect to approximate the scientific neutrality and openness of the diagnostic method is to combine the contributions of leading scholars into a comprehensive system. Rather than supporting current regulatory preoccupations with harmful online content—as shown by Canada’s over-criminalisation of hate offences in its proposed Online Harms Act—early indications point to taking more seriously the underlying infrastructure and economic drivers not only of harmful content and disinformation, but of rising censorship and risks of over-regulating online speech interests.

The key takeaway from our review of “systems-inspired” regulatory scholarship and medico-diagnostic principles consequently is that prevailing “systems-based” regulatory approaches—as epitomised by Canada’s new Online Harms Act—would appear to function as a “blueprint” for privatised government censorship, providing regulators with the legislative mandate, informational transparency, and compliance authority for regulatory capture that leading free speech scholars have appropriately labelled the “moderators’ dilemma”. That is to say, “the more speech-protective the government’s policy, the more hands-on the government’s approach will need to be”.¹⁴⁷ As shown by Canada’s newly proposed “systems-based risk-assessment” model, this unsettling trade-off “sewn into the logic of the Internet”, not only appears to apply to combating increasing online censorship by using “must-carry” legal interventions (i.e. common-carrier laws preventing the exclusion of speakers or restricting content), but to all regulatory “proxy-censor” interventions aimed at tamping down harmful online content.¹⁴⁸ Since Canadian regulators have not engaged in an uncompromising “differential diagnosis” of online phenomena—which, as we have seen, benefits diagnosticians and legislators alike by situating the patient’s or public sphere’s symptoms in their broadest aetiological context—we are tempted, perhaps ironically, to look not to the future, but to the distant past.

After all these years, Virchow’s pioneering view on the diagnosis and regulation of public health remains an invaluable perspective that Canada and other countries would do well to study and apply. In a dynamic, interconnected world increasingly at odds with the principle of territoriality—where “physicians are the natural attorneys of the poor”, and politicians its “natural anthropologists”—it is with some surprise and much regret that it remains a matter of any controversy or dismay that we lawyers and jurists should bear a greater share of the solemn responsibility of being its “natural diagnosticians”.