The Web Was Built to Be Open, Then It Became Centralized

The early days of the internet promised open protocols, permissionless publishing, and equal opportunities for all. However, in practice, the allure of convenience, security, and scalability has led traffic toward a few dominant players. Search engines became gatekeepers, social platforms took charge of identity and distribution, mobile app stores dictated engagement rules, and cloud platforms centralized computing power.

This centralization has delivered clear benefits—speedier experiences, strengthened security, and uniform tools for billions. Yet, it has also created dependencies and gatekeepers. To gain visibility, one needed to optimize for a search algorithm. Reaching your audience required adherence to a platform’s rules. And if global performance was necessary, reliance on a select few cloud services and CDNs became inevitable.

Now, AI is shifting these dynamics once again. Answer engines are taking over from traditional lists of links, and foundation models disproportionately favor those with substantial data and computing resources. Consequently, a new stack is emerging, blending centralized and decentralized elements, creating a hybrid web.

What AI Actually Changes

1) Discovery Shifts from Links to Direct Answers

For the past 25 years, search engines have been the primary organizers of the web. With the advent of generative AI, responses are increasingly shown as direct answers rather than as a list of sources. For instance, Google’s AI Overviews summarize results directly on the results page, reducing the need for users to click through to publishers (Google). Both publishers and regulators worry that this trend could diminish traffic to independent websites while funneling attention into platform interfaces (Reuters).

2) Data, Compute, and Models Concentrate Power

• Data: High-quality, up-to-date data fuels modern AI. Platforms with large, logged-in user bases, proprietary content, and distribution have significant advantages.
• Compute: Training and deploying state-of-the-art models requires substantial capital. A mere three providers dominate cloud infrastructure spending globally, reinforcing reliance on a small number of hyperscalers (Synergy Research).
• Models: A few general-purpose models lead in benchmarks and audience awareness. However, open-weight models like Llama 3 are progressing rapidly, providing local or self-hosted options for numerous tasks (Meta).

3) Interfaces Collapse Layers of the Web

When assistants answer questions, draft text, and interact with tools, they implement a new layer in front of the traditional web stack. This can bypass navigation, search, and even some applications. Interface owners gain considerable influence over visibility and value distribution, reminiscent of mobile app stores but possibly extending much further.

Why Centralization Persists

• Convenience: Users favor familiar interfaces and enjoy the ease of single sign-on. Bundled services reduce friction.
• Safety and Trust: Effective spam filtering, content moderation, and fraud prevention are costly at scale. Centralized platforms can spread these expenses across a broad user base.
• Economies of Scale: Bandwidth, storage, and global content delivery favor large providers, and the demands of AI training and inference amplify this tendency.
• Network Effects: As more consumers join a platform, it becomes more valuable to every other user, resulting in increasing dependency over time.

While these advantages hold water, the real question is whether centralization has gone too far, leading to excessive dependency, opacity, and opportunism.

Risks of Excessive Power Concentration

• Gatekeeping and Fees: A select few platforms can dictate terms for discovery, distribution, and monetization. Antitrust regulators are increasingly focusing on these choke points in search, mobile, and marketplaces (DOJ vs Apple), (DOJ vs Google), (FTC vs Amazon).
• Fragility and Single Points of Failure: Service outages, sudden policy changes, or API discontinuations can disrupt entire ecosystems almost instantly. Risks from dependency increase when a few firms govern critical infrastructure layers.
• Opacity and Bias: Algorithmic rankings and AI-generated answers can solidify narratives, reflect biases in training data, or amplify misinformation when governance lacks transparency.
• Privacy and Surveillance: Centralized systems for identity, advertising, and analytics facilitate widespread tracking without robust safeguards. Balancing data minimization and portability with ad-tech goals is challenging.
• Innovation Bottlenecks: Developers might self-censor or refrain from building features that could be appropriated by platforms, reducing experimentation.

The Working Decentralization Toolbox

Decentralization isn’t merely a single technology, but rather a collection of patterns designed to shift control from companies to protocols and facilitate open ecosystems from proprietary products. Some of these tools have proven their resilience while others are still in the experimental stage.

Protocols Over Platforms

Protocols like email (SMTP/IMAP), the web (HTTP/HTML), and RSS illustrate how open standards can foster durable ecosystems. No single entity owns these protocols, enabling relatively simple provider switching. Newer social protocols are reintroducing this concept:

• ActivityPub: A W3C standard for federated social networking that allows different servers to communicate, similar to email (W3C). Meta has begun experimenting with ActivityPub support in Threads, aiming for integration into the broader fediverse (Meta/Threads).
• AT Protocol: Created by Bluesky, the protocol separates identity, moderation, and algorithmic choices to facilitate portable social graphs and customizable feeds (AT Protocol).
• Nostr: A minimalist protocol designed for social applications, prioritizing simplicity and censorship resistance via a network of relays (Nostr).

Portable Identity and Data

• Data Portability Rights: The GDPR in the EU establishes the right to data portability, and the Digital Markets Act mandates interoperability for designated gatekeepers in specific services (GDPR Art. 20), (DMA).
• Data Transfer Project: A collaborative initiative that enables users to move photos, emails, contacts, and other data seamlessly between services using common adapters (Data Transfer Project).
• Decentralized Identity: The W3C Decentralized Identifiers aim to define identifiers unlinked to any single provider, allowing for cryptographic verification (W3C DID).
• Solid Data Pods: Tim Berners-Lee’s Solid project separates applications from data storage, allowing users to keep their data in individual pods while granting apps permission as needed (Solid).

Decentralized Storage and Content Addressing

Beyond traditional HTTP, content-addressed approaches like IPFS access data by its hash rather than its location, allowing for resilient mirroring across many nodes (IPFS). Filecoin complements this by introducing economic incentives for sustained storage (Filecoin). While these tools may not cover all workloads, they expand the design possibilities.

Content Authenticity Signals

As synthetic media becomes prevalent, provenance standards like C2PA can provide tamper-evident metadata outlining the origins of content and the editing history (C2PA). While they won’t eliminate misinformation alone, they can aid trustworthy actors in affirming reliability.

Open Models and Transparent Evaluation

Open-weight AI models enable organizations to perform inference on their infrastructure and to customize behaviors without relinquishing data to third parties. Community benchmarks and reproducible evaluations enhance accountability. The Open Source Initiative is working on providing a comprehensive definition for open-source AI, clarifying what openness should entail (OSI).

Policy Updates That Shift the Balance

• EU Digital Markets Act (DMA): Sets interoperability and fairness requirements for designated gatekeepers, including app stores, messaging, and advertising platforms (European Commission).
• EU AI Act: Establishes a risk-based framework covering transparency, safety, and governance requirements for AI systems, encompassing rules for general-purpose models (European Parliament).
• US Antitrust Actions: Ongoing cases address supposed monopolization in search, smartphones, and online retail, scrutinizing default agreements and self-preferencing behaviors (DOJ vs Google), (DOJ vs Apple), (FTC vs Amazon).
• Interoperability and Data Portability Rights: Various privacy frameworks now acknowledge user rights to data access and transfer, making it easier to switch and utilize multiple services (GDPR).

While these regulations won’t decentralize the web on their own, they do encourage a shift towards more open interfaces and lower switching barriers.

Practical Guides for Different Roles

If You Build Products

• Design for Portability: Implement one-click exports, provide well-documented APIs, and create easy import processes. Make it simple for users to leave to earn their loyalty.
• Adopt Open Standards: Use protocols like ActivityPub for social networking, WebSub for feeds, and WebAuthn for authentication, along with standardized formats like JSON and CSV for data exchange.
• Combine Centralized UX with Decentralized Backends: Offer managed services for user-friendliness but store user-owned data in repositories they can control or self-host.
• Plan for Multi-Cloud and Locality: Isolate stateful data from stateless compute processes. Utilize infrastructure-as-code and portability layers to prevent deep lock-in.
• Integrate Evaluation and Provenance: Make your AI features accountable by incorporating quality, safety, and traceability metrics. Support C2PA standards when applicable.

If You Are a Publisher or Creator

• Own Your Domain and Distribution List: Publish first on your site. Maintain email and RSS feeds while syndicating out to keep your primary presence.
• Engage with Federated Networks: Share updates on platforms like Mastodon or other ActivityPub-based services. Explore cross-network posting.
• Structure Content for Direct Answers: Utilize clear headings, concise summaries, and schema markup to assist both search engines and AI assistants in correctly routing traffic.
• Negotiate Data Usage: Assess robots.txt files, permissions, and licensing. Explore opt-in partnerships that offer reciprocal value, analytics, or backlinks.

If You Are an Engineer or Researcher

• Evaluate Open-Weight Models: Many workloads can be successfully handled by open models that meet quality standards while enhancing privacy and cost effectiveness.
• Utilize Content-Addressed Storage Where Applicable: For public assets, employing IPFS-style pinning can bolster resilience and cacheability.
• Prioritize Reproducibility: Make evaluation scripts, seeds, and dataset documentation publicly available. Choose benchmarks that reflect real-world tasks rather than synthetic metrics alone.

If You Shape Policy

• Enforce Portability and Interoperability: Develop actionable guidelines and test suites to ensure that rights are practically applicable, not just theoretical.
• Support Open Standards Bodies: Invest in the development and maintenance of essential protocols. Advocate for government procurement to favor open interfaces.
• Target Choke Points, Not Just Size: Focus on default arrangements, self-preferencing behaviors, and access to critical interfaces that could hinder competition.
• Enhance Transparency in AI Systems: Mandate documentation for training data sources, risk assessments, and governance related to high-impact models (EU AI Act).

Realistic Expectations: Decentralization with Guardrails

While it’s easy to view centralization vs. decentralization as a black-and-white choice, the reality is that the web often thrives on hybrid solutions: centralized user experiences combined with decentralized backends. Consider email: federated by nature, yet predominantly accessed through centralized options like Gmail. Although the web is open, CDNs and cloud services often dominate its delivery.

• AI Interfaces at the Edge: AI interactions will occupy a space between users and services. Some will be operated by major platforms, while others remain open-source or local.
• Protocols Under the Hood: Identity, messaging, and content will likely utilize open protocols, fostering switching and interoperability.
• Multiple Model Tiers: Expect general-purpose models for broad applications, domain-specific models for specialized needs, and open-weight models where control and customization are essential.
• Verification Layers: Given the increasing blend of synthetic and authentic content, signals for provenance and authenticity will grow in importance.

The goal isn’t to eradicate centralization; it’s about recalibrating incentives so that users, developers, and publishers can engage under fairer terms.

Open Questions Worth Tracking

• Attribution and Compensation: As answers supersede clicks, how should value be redirected back to the sources that generated those answers?
• Compute Concentration: Can new hardware innovations, on-device models, or collaborative cloud initiatives mitigate dependency on a limited number of providers?
• Interoperable Social Platforms: Will ActivityPub or AT Protocol achieve consumer-scale adoption, and how will moderation approaches withstand real-world challenges?
• Regulatory Convergence: Will mandates for portability and interoperability align across regions, easing compliance burdens for smaller entities?
• Definitions of Open AI: Where will the delineation be drawn between transparent and genuinely open models, and how will that affect their adoption?

Conclusion: The Web We Get Is the One We Build

Centralized platforms didn’t seize control of the web by mere chance—they addressed complex challenges, gained users’ trust, and benefitted from substantial network effects. Decentralization isn’t a retreat to the past but a practical response to overly concentrated power, fragility, and misaligned motivations.

AI raises the stakes, collapsing layers of interfaces and concentrating data and computational power. Nonetheless, protocols, open models, and evolving policies are broadening the horizon of what can be achieved beyond a handful of protected environments.

The future will be shaped by the decisions we make today: embracing open standards, prioritizing portability, supporting provenance, and aligning incentives for creators and users. No single entity owns the web, but collectively, we can choose whether it remains a shared space for everyone to innovate or a marketplace ruled by a few.

FAQs

What do you mean by decentralization on the web?

Decentralization refers to services operating across multiple providers or nodes, governed by open standards that allow users to transfer their identity and data without requiring permission from a single company.

Is Web3 the same as decentralization?

No, Web3 often pertains to blockchain-based applications and tokens. While some aspects are decentralized, others aren’t. Decentralization also encompasses non-blockchain tools like ActivityPub, RSS, and Solid.

Do decentralized systems sacrifice safety or usability?

They can, if not designed effectively. The most viable approaches combine decentralized protocols with user-friendly, managed services. For example, email is federated by design but accessed through polished clients and reliable hosting.

Will AI make the web more centralized or more open?

Both outcomes are possible. Training typically favors concentration, but developments in open-weight models, on-device inference, and protocol-based networks are counteracting that trend. Expect a hybrid environment where interfaces may be centralized while underlying data and identity become more portable.

How can an individual start today?

Create a domain, publish content on your site, enable RSS, maintain an email list, and mirror your content on a federated social account. If you develop software, prioritize export features and standard formats by default.

Sources

1. Google – Generative AI in Search and AI Overviews
2. Reuters – Publishers raise concerns about Google’s AI Overviews
3. Synergy Research – Cloud market growth and provider share
4. Meta – Llama 3 open-weight models
5. W3C – ActivityPub Recommendation
6. Meta – Threads and ActivityPub interoperability
7. AT Protocol – Technical overview
8. Nostr – Protocol overview
9. GDPR – General Data Protection Regulation
10. European Commission – Digital Markets Act
11. Data Transfer Project – Open-source data portability effort
12. W3C – Decentralized Identifiers (DID) Core
13. Solid Project – Personal data pods
14. IPFS – What is IPFS
15. Filecoin – What is Filecoin
16. C2PA – Content provenance standard
17. Open Source Initiative – Open Source AI Definition
18. European Parliament – EU AI Act adoption
19. US DOJ – United States v. Google LLC (Search)
20. US DOJ – United States v. Apple Inc. (Smartphones)
21. US FTC – FTC v. Amazon (Monopolization)
22. GDPR Info – Article 20 Data Portability