Data Privacy in AI: What Business Leaders Should Know (and Do) Right Now

A New Era

When OpenAI released ChatGPT—an artificial intelligence (AI) chatbot capable of holding humanlike conversations—in November 2022, the tech startup couldn’t have predicted its explosion of popularity.

It was a big deal. An Internet-disrupting-level big deal.

It brought algorithm usage into people’s everyday lives. By February, just two months after its launch, an estimated 100 million users had flocked to the app, asking it to do everything from building code to writing essays to giving relationship advice.

But in March 2023, a bug leaked some of ChatGPT users’ chat history.

Although a fairly minor glitch, the incident cast an international spotlight on a much bigger and rapidly growing issue: data privacy.

The Evolution of AI and the Rise of Data Privacy Issues are Inextricably Linked

Machine learning algorithms are nothing new; in fact, they determine the list of results whenever we Google a search phrase. These AI tools use a lot of data, and the more data they have to work with, the more effective they can be.

But as AI continues to grow in ability and application, it opens up opportunities to encroach on privacy rights by elevating the analysis of personal information to previously unimagined levels of power and speed.

The rapid evolution of AI foreshadows an inevitable acceleration in data privacy regulations and concerns, and organizations across all industries will have to decide how to respond.

What is Data Privacy?

Before diving deeper into this complex issue, let’s define some of the foundational concepts.

Data privacy refers to the protection of personal information from unauthorized access, misuse, or disclosure. Essentially, it’s a person’s autonomy to decide when, how, and to what extent personal data about them is shared with others.

Personal data is any information related to an individual which can directly or indirectly identify them, including names, email addresses, ethnicity, gender, biometric data, political opinions…even pseudonymous data can be called “personal data” if it’s relatively easy to identify someone from it.

But beyond that, data privacy refers to the ability of a data system to:

• withstand adversarial attacks or, more generally, unexpected changes in its environment or use
• maintain its functions and structure in the face of internal and external change
• degrade gracefully when this is necessary

The collection of personal data sits at the heart of many modern business models. More and more, companies are tracking user behavior, building individual profiles based on this data, and using this information to build algorithms and automated systems that further track, profile, and impact the public.

Personal data is a commodity. In our digital, algorithm-driven age, it’s one of our most precious—and threatened—resources.

Governments are Doubling Down on Data Privacy Laws

Until now, businesses have had little incentive to prioritize data protection. Data leaks have made for some splashy headlines but have in most cases led to negligible consequences for companies.

That’s changing.

Soon after the news of ChatGPT’s data breach broke, Italy became the first country in the West to ban the question-and-answer AI, citing the lack of legal basis underpinning the massive collection and processing of personal data used to “train” the algorithms as a chief concern.

After OpenAI agreed to the regulators’ demands, Italy reversed its ban.

As mentioned earlier, this speaks to a larger trend: mounting governmental regulations and calls for accountability around data privacy.

As more decisions are based on formulas that depend on personal data collection, governments are passing more laws around the collection, use, and storage of that data.

Right now, data privacy laws in the U.S. consist of an erratic patchwork of state regulations with minimal federal oversight. For instance, Washington state Governor Jay Inslee signed the My Health My Data Act into law in May 2023. This first-of-its-kind act gives Washingtonians the right to request data deletion, restricts geofencing around healthcare facilities, and forbids health data collection without consent.

But there has been a growing interest in developing AI regulations on a federal level, starting in October 2022 with the White House publishing the blueprint for an AI Bill of Rights, which specifically narrows in on data privacy as one of the pillars for guiding the design, use, and deployment of automated systems.

More recently, the Biden administration is seeking public comment on upcoming AI policies, tasking the U.S. Commerce Department’s National Telecommunications and Information Administration (NTIA) with gathering public input on developing AI audits, assessments, certifications, and other tools to engender trust from the public.

All of this heralds a definite shift in the U.S. toward more comprehensive and detailed data governance.

In other areas of the world, like the European Union, more stringent data privacy regulations – notably the General Data Protection Regulation (GDPR) – have already been enacted and are being more fully developed.

As AI evolves, the call for data privacy laws that hold organizations accountable for how they manage data will escalate. It’s unavoidable.

And the consequences to organizations can be material. In fact, ChatGPT risks facing a fine of 20 million euros ($21.8 million) or 4% of its global annual revenue if it doesn’t remedy the concerns cited by the Italian Data Protection Agency, according to CNBC.

Data Privacy is a Market Driver Across Industries

That said, data privacy is more than a matter of legal compliance. It’s about making users feel safe and inspiring confidence in any situation in which they engage.

Before users will engage online, they need to feel confident that their personal data will be handled with care and shielded from malicious eyes.

They need to know that when they give their email address in exchange for a 10% coupon, they won’t be spammed. Or more importantly, when they disclose their insurance information to a healthcare worker, the hospital software will protect it.

Data privacy isn’t just a “nice-to-have”; it’s a necessity for businesses across all industries.

Innovators who prioritize data privacy are the ones who will see their impact grow in a world where privacy is increasingly a market driver.

7 Foundational Principles for Building Data Privacy into Business Processes

Data privacy isn’t a nice feature that can be bolted on at the end of the design process; it needs to be top-of-mind from the start to ensure seamless integration and faultless execution.

Here are seven “privacy by design” principles to guide organizations’ efforts to secure data protection through technology design:

1. Be proactive, not reactive
   Data privacy regulations are a matter of when—not if. The number of countries with some form of data protection and privacy laws has doubled from 68 in 2010 to 137 today and is continuing to grow. Brands that prioritize data privacy now will find themselves ahead of the game as laws protecting consumers’ rights are passed.

   A privacy-first attitude will naturally support a preventative approach to privacy. Instead of reacting to privacy risks or invasions when they happen, companies will actively build processes and procedures to prevent them from occurring in the first place.

2. Make privacy the default
   Users shouldn’t have to worry about their privacy settings when browsing a website, opening an app, or logging into software. Establishing privacy as the default automatically sets users’ privacy to the highest level of protection, whether or not a user interacts with those settings.

   Such default settings include minimizing collection and use of data and limiting retention and disclosure.

3. Embed privacy into the design from the start
   When building a website, mobile app, or software application, privacy should be as high priority as user experience. For embedded privacy to work, it needs to be included in all phases of the design process. It also can’t be obvious or awkwardly included so as to detract from the functionality of the program. Every decision and new process must be filtered through a privacy-first mindset to promote both functionality and privacy protection.
4. Don’t sacrifice functionality
   Those who argue that robust security protocols are inherently at odds with an optimal user experience are missing the point. Integrating privacy seamlessly into every design element is a win-win approach that establishes organizations as industry leaders.
5. Supply end-to-end security
   From the point personal data is received to its deletion after serving its purpose — and everything in between—privacy by design ensures the security of this data through the processing lifecycle.

   This full lifecycle protection is where the interdisciplinary nature of privacy by design shines. It leans heavily on security best practices to provide end-to-end data protection. Security also ensures data remains confidential, true to its original form, and accessible during its time with the company.

6. Maintain visibility and transparency
   Openness about your privacy policies and procedures will build accountability and trust with your users.

   Privacy by design means documenting and communicating actions clearly, consistently, and transparently. It presents a shared attitude of privacy as a duty, and one your team takes seriously. That promise should be supported by an accessible and effective complaint submission and resolution process, as well as independent verification of your policies and promises to users.

7. Keep it user-centric
   Respect for user privacy involves always having the users’ privacy interests in mind and providing the necessary safeguards and features to protect such interests. This respect inspires every design decision.

   The best user experience puts privacy first. This includes putting the power in the hands of the user to manage their own data and actively seeking their engagement in the process.

Adopt a Privacy by Design Mindset

The process of protecting data will be less arduous and expensive for organizations that start with a privacy-first mindset as opposed to having to re-engineer privacy following a failure.

In addition to approaching new projects with the principles outlined above, organizations would be smart to examine the security of existing technologies with a privacy impact assessment (PIA), a systematic assessment that determines a project’s impact on the privacy of individuals and outlines recommendations for managing, minimizing, or eliminating those impacts.

The International Association of Privacy Professionals (IAPP) and U.S. Federal Trade Commission provide PIA resources that can be useful for organizations seeking to implement privacy by design.

While AI has the potential to change lives and the way society functions for the better, it can only achieve this life-altering good if organizations safeguard personal data. With governments doubling down and users becoming distrustful, the early adopters of data privacy practices are the ones who will build the technology that shapes the future.