Public Consultation Paper on Artificial Intelligence and Personal Data Protection Framework

External AI Providers and Cross-Border Transfers

The Proposed Framework addresses the procurement, integration and use of external AI systems, including those supplied or supported by developers, vendors and cloud service providers. It makes clear that engaging an AI service provider does not transfer or diminish the data controller’s responsibilities under the PDPA.

The key proposals include:

1. Vendor due diligence: Before procuring, integrating or using an external AI system, data controllers would need to conduct due diligence proportionate to the nature and sensitivity of the personal data, the intended use of the system and the potential risks to individuals. Relevant considerations include the system’s functionality and limitations, security measures, involvement of other service providers, and whether personal data may be retained or reused for model training or other purposes beyond the intended service.

2. Roles and processing arrangements: Data controllers would need to clearly define the parties’ respective roles and responsibilities based on their actual involvement in processing. For example, where an AI service provider processes personal data on behalf of the data controller, appropriate arrangements would need to address matters such as processing purposes and instructions, security measures, the involvement of other processors, breach handling, and the return or disposal of personal data. Data controllers may request model cards, system cards or technical documentation from the provider where reasonably necessary to understand the AI system’s functionality, operation, personal data processing practices and limitations.

3. Cross-border transfers: Where an external AI system involves transferring personal data outside Malaysia, the transfer would need to comply with the requirements of the PDPA, such as section 129 of the PDPA. Relevant considerations include the countries where data is stored or processed, the parties with access, applicable safeguards and onward transfers. Appropriate records of transfers and safeguards would also need to be maintained.

4. Ongoing oversight: Data controllers would need to maintain appropriate oversight of external AI systems and providers throughout the processing period. Significant changes to the system, terms of service or processing activities would require an assessment of whether additional safeguards, revised arrangements or further review are necessary before continued use of the AI system.

Data Subject Rights and Safeguards

The Proposed Framework affirms that data subjects’ rights under the PDPA continue to apply where personal data is processed through AI systems. Data controllers would need to establish appropriate processes to facilitate the exercise of these rights and implement safeguards proportionate to the nature of the processing and the potential risks to individuals.

The main proposals are summarised below.

1. Access and correction: Data subjects would be able to request access to, and correction of, their personal data processed through AI systems. Where applicable, access may extend to personal data generated or derived through an AI system that relates to the individual, where such personal data can reasonably be identified and retrieved. Additionally, where an AI system causes or contributes to an inaccuracy, data controllers would need to verify and correct the personal data and, where appropriate, address the source of the inaccuracy.

2. Withdrawal of consent and data portability: Where an individual withdraws consent to the processing of their personal data through an AI system, the data controller would need to cease that processing for the relevant purpose. Where the right to data portability applies, data controllers would also need to facilitate transmission in accordance with the applicable requirements and take reasonable steps to protect the security, confidentiality and integrity of the data.

3. Mechanisms for exercising rights: Data controllers would need to provide clear and accessible channels for individuals to exercise their rights. Where AI-related processing may not be apparent to individuals, appropriate explanations should be provided to support the effective exercise of those rights.

4. Fairness and accuracy: AI systems would need to be designed, tested, used and monitored to minimise foreseeable risks of unfair, biased or discriminatory outcomes. Measures may include assessing datasets for quality and potential bias, reviewing AI-generated outputs, implementing corrective measures, and establishing review or escalation mechanisms where outputs may significantly affect individuals. Data controllers would also need processes to identify, review and correct inaccurate personal data used as inputs or contained in AI-generated outputs, and to prevent reliance on such inaccuracies for further processing or decisions that may materially affect individuals.

5. Security and breach response: Data controllers would need to implement appropriate technical and organisational safeguards throughout processing, including access controls, encryption, and logging and incident response procedures, where applicable. Additional measures may be considered to address AI-specific risks, such as controls to prevent unauthorised submission of personal data to external AI models or APIs. AI-related personal data breaches would remain subject to section 12B of the PDPA, with arrangements needed to ensure that processors and AI service providers communicate relevant information promptly. 

Governance, Risk Assessments and Accountability

The Proposed Framework emphasises appropriate organisational oversight and accountability throughout the AI system lifecycle. Governance arrangements would need to be proportionate to the nature, scope, context and purposes of processing, taking into account the personal data involved, the complexity of the AI system and the potential risks to individuals.

The key proposals are outlined below.

1. Roles and responsibilities: Data controllers would need to clearly assign responsibilities for the governance, operation, monitoring and oversight of AI systems. Where a Data Protection Officer (“DPO“) is appointed or required under the PDPA, the DPO would need to be appropriately involved in overseeing AI-related personal data processing. Significant decisions or material issues affecting personal data processing or individuals would also need to undergo appropriate internal review, escalation or approval.

2. Internal policies and training: Data controllers would need to establish policies and procedures governing the development, deployment and use of AI systems involving personal data. These should address responsibilities, approval processes, use restrictions and processing controls, where appropriate. Relevant personnel would need appropriate training and awareness, and policies would need to be reviewed periodically and updated to reflect material changes. Where appropriate, AI systems would also need to be configured to apply privacy-preserving settings by default.

3. Risk assessments: Data controllers would need to conduct appropriate risk and impact assessments to identify, assess and mitigate privacy and personal data protection risks. Where appropriate, these assessments would include, or be complemented by, a Data Protection Impact Assessment (“DPIA“). Significant risks would need to be addressed through appropriate measures before deployment or continued operation. Assessments would also need to be reviewed periodically and following material changes to the system, processing activities, intended use or identified risks.

4. Documentation and evidence of compliance: Data controllers would need to maintain documentation proportionate to the nature and risks of the processing to demonstrate compliance throughout the AI system lifecycle. Relevant records may include policies and procedures, processing purposes and activities, data sources and legal bases, risk assessments, testing and audit records, governance decisions and approvals, records relating to AI service providers, and training and awareness records.

5. Internal reviews and audits: Data controllers would need appropriate processes for periodic reviews and audits of AI systems and related processing activities. Their frequency and scope would depend on the nature and risks of the processing. Findings and corrective actions would need to be documented, with appropriate measures taken to address identified deficiencies, non-compliance or risks and to monitor the implementation of those measures.

Observations

The Proposed Framework signals JPDP’s expectation that personal data protection be addressed throughout the AI system lifecycle. Its proposed scope extends beyond AI developers to organisations using off-the-shelf tools, cloud services or software with embedded AI functionality. It could therefore have implications for a wide range of organisations, including those that may not regard themselves as using AI at all, where AI functionality has been integrated into existing applications.

A key implication is that organisations may need a clearer understanding of how, and for what purposes, their AI systems process personal data. The proposals on transparency and consent raise questions about whether existing privacy notices and consent arrangements adequately cover these activities. The proposed restriction on processing personal data through unauthorised AI systems also brings internal approval processes and controls over staff use of publicly available AI tools into focus.

The Proposed Framework forms part of JPDP’s broader approach to data protection compliance and complements the DPbD, DPIA and Automated Decision-Making and Profiling Guidelines. Together, these instruments address safeguards at the design stage, assessments of processing risks and protections relating to automated decisions affecting individuals. Organisations that have already implemented these guidelines may therefore be able to build on their existing compliance processes to accommodate the proposed requirements.

However, applying the requirements of the Proposed Framework may be challenging where organisations rely on external AI providers. Although the Proposed Framework reaffirms the data controller’s responsibility, the data controller’s understanding of whether personal data is retained or reused for model training or other secondary purposes may depend on information supplied by the provider. Compliance may therefore be complicated where providers offer limited transparency, inadequate technical controls or little flexibility in their standard service arrangements.

These challenges also have implications for organisational oversight of AI-related personal data risks. The Proposed Framework’s emphasis on clear responsibilities and documented risk assessments could give these risks greater prominence within existing risk management and internal control processes. This may involve closer coordination between data protection, technology and procurement departments, both when AI systems are introduced and throughout their use.

As these systems evolve, changes to datasets, functionality or provider practices may affect processing purposes, introduce new risks or alter the adequacy of safeguards. The proposed requirements for periodic reviews, reassessments and record keeping may therefore place ongoing demands on personnel and resources, particularly in organisations using multiple AI systems or relying on frequently updated external services.

As the Proposed Framework remains subject to consultation, the extent of these implications will depend on its final wording. Accordingly, the consultation provides an opportunity for organisations to raise concerns and submit feedback on the proposed requirements, including any practical challenges they may face in implementation.

Should you require assistance in understanding the Proposed Framework’s potential implications for your organisation or addressing any other data protection or technology, media and telecommunications (“TMT“) matter, please feel free to contact us.

For regional data protection and TMT matters, please see Rajah & Tann’s Technology, Media & Telecommunications Practice  for more information.

Contribution Note:

This Legal Update is contributed by Partners Deepak Pillai (Head, TMT & Data Protection) and Yong Shih Han, with the assistance of Associate Leslie Bong.