Is private connectivity enough for LLM APIs? — Four boundaries from Bedrock to self-hosting

An LLM API security review often starts with one question: “Can we send sensitive data if the API uses a private network?” Seeing names such as AWS PrivateLink or Azure Private Endpoint can make the review feel complete. Private connectivity only describes the network path between the client and the API service. Separate documents govern where the model runs, how long inputs and outputs remain, how safety systems use data, and which legal basis supports a cross-border transfer.

That distinction matters for healthcare data. Under Korea's Personal Information Protection Act (PIPA), health data is sensitive information. Even if the API endpoint is in Seoul and traffic stays on the AWS backbone, inference outside Korea can trigger a separate cross-border transfer review. This article focuses on how to classify and approve LLM API access paths, not on ranking models.

Identify the API operator before the model

The same model name can represent different services depending on the access path. Calling Claude through the Anthropic API means using Anthropic's account, contract, and retention policy. Calling it through Amazon Bedrock means using an AWS endpoint, IAM, an inference profile, and Bedrock's data-processing settings. Partner models in Microsoft Foundry or Google Cloud's Gemini Enterprise Agent Platform start with those platforms' deployment types and contractual boundaries.

The request format does not identify the operator. Bedrock Mantle exposes an OpenAI-compatible API and the Anthropic Messages API, but the target is an AWS service rather than the direct OpenAI or Anthropic API.¹ An AI Gateway can combine several providers behind one base_url, but the final processor and contracting party still exist. Record the endpoint, authentication principal, billing party, and data-processing agreement before the model name.

Five ways to reach an LLM API

For review purposes, access paths fall into five types. The useful dividing line is where the operator and level of control change, not an exhaustive list of providers.

An AI Gateway is not a sixth inference destination. It is a control layer for authentication, allowed models, budgets, and logs. If the gateway invokes a Bedrock Global profile, the processing scope remains Global. Applications can also bypass gateway policy if direct provider access stays open. That bypass can affect security and cost controls, as shown by credential theft and LLMjacking on Bedrock. Prove data residency with records from the final model API, not the gateway.

A private endpoint covers one of four boundaries

An approval record needs the API operator and evidence for all four boundaries.

PrivateLink is evidence for the first row. Microsoft Foundry can disable public access with Private Endpoint, while its inference scope still depends on whether the deployment is Global, Data Zone, or Regional.² Gemini Enterprise Agent Platform can connect its generative AI API through Private Service Connect. Retention and feature-specific ZDR conditions remain a separate review.³

Calling from Seoul is not processing in Seoul

According to the AWS model cards checked on August 2, 2026, Claude Sonnet 5 and the latest Opus model, Claude Opus 4.8, show Global ✓ / In-Region ✕ / Geo ✕ for Seoul. The official Bedrock catalog does not list Claude Opus 5.⁴⁵⁶

A Seoul endpoint is not evidence of Seoul processing. When an application invokes global.anthropic.claude-sonnet-5 or global.anthropic.claude-opus-4-8, AWS selects a processing Region within the Global profile. Blocking other processing Regions with SCPs does not leave Seoul as the only route. AWS states that a cross-Region request fails if an SCP blocks any destination Region included in the profile.⁷

To force both invocation and processing into Seoul, first confirm that the model supports In-Region inference there. Place the workload and the bedrock-runtime VPC endpoint in ap-northeast-2, then invoke the Seoul foundation model ID instead of a Global or Geo profile. IAM and SCPs should allow only the approved Seoul model and deny other Regions and inference profiles. Use CloudTrail, the model ID, and VPC Flow Logs to verify that no bypass exists. A single-Region application inference profile must use the Seoul foundation model as its source.

These controls cannot be applied to Claude Sonnet 5 or Opus 4.8 because neither model supports In-Region inference in Seoul. If the latest Claude model is mandatory, lawful cross-border processing and safeguards become prerequisites. If processing in Seoul is absolute, choose a model with Seoul In-Region support or deploy another compatible model on EC2, EKS, or SageMaker in Seoul.

Retention follows the path and feature, not the model name

Treating store=false as zero retention creates a control gap. Bedrock uses retention modes such as none, default, and provider_data_share, together with each model's allowed_modes. store=false prevents customer retrieval but does not always disable retention for safety review. For strict ZDR, set the account or project mode to none and verify that the model permits it.⁸

The retention policy also changes when the access path changes. Anthropic says it deletes commercial API inputs and outputs within 30 days by default. Longer-lived features such as the Files API, a separate ZDR agreement, safety-policy violations, and legal obligations are exceptions.⁹ Gemini Enterprise Agent Platform limits training use across managed models, but some features keep caches or state. Session Resumption, Grounding, Files, Batch, and Agent features each have their own retention conditions. A stateless inference default cannot describe the full data lifecycle.

OpenAI's South Korea data residency option also separates storage from processing. The Korea endpoint supports storage residency but not in-country inference processing. OpenAI Private Link connects Azure workloads after OpenAI onboarding and approval.¹⁰¹¹ Calling the direct OpenAI API from an AWS VPC and using an OpenAI-family model through Bedrock have different network, contract, Region, and retention controls.

Korean healthcare data also needs a cross-border basis

Article 23 of PIPA treats health information as sensitive information. When an external LLM provider processes an input, the legal basis for processing sensitive information is the first review item. If the arrangement is an outsourced processing relationship, review the written scope, purpose limitation, safeguards, and processor disclosures required by Article 26.¹²¹³

When personal information is provided, accessed, outsourced, or stored abroad, review the lawful bases and safeguards under Article 28-8. A cross-border transfer does not always require separate consent. Contract performance, legal obligations, and other bases may apply, so legal and privacy owners must assess the actual data flow and relationships between the parties.¹⁴

The technical team records the data flow and configuration rather than making the legal conclusion. Remove or pseudonymize patient identifiers before transmission and send only the fields needed for the purpose. Disable raw prompt and response logs by default and record exceptional access. Model output should not automatically decide clinical care or an adverse action; a person reviews it. The Personal Information Protection Commission's generative AI guidance and healthcare data guidelines should be read across the same lifecycle.¹⁵¹⁶

Put data classification, access rights, retention and deletion, and audit logs into operating procedures. This article is not legal advice and does not guarantee that a healthcare organization's consent form, privacy notice, or processor agreement is adequate.

Approval depends on an evidence contract, not a product name

“Uses Bedrock” or “uses PrivateLink” is not a complete architecture decision. Bundle the following evidence for every allowed path.

If Claude Sonnet 5 or Opus 4.8 is mandatory, Global invocation from Seoul is available but processing in Korea is not guaranteed. Organizations that permit cross-border processing should compare the contract, retention, and audit differences between direct and cloud-managed APIs. If in-country processing is absolute, only a Seoul In-Region model or a deployment operated in Seoul remains. If all external communication is prohibited, a private VPC deployment is still insufficient; consider on-premises or isolated operation.

Like the six gates for moving an AI proof of concept into production, an LLM API path should not be approved because one control passes. This article did not measure model quality, latency, cost, or whether a system requires medical-device authorization. Those questions need separate evaluation and legal review.

Private networking is an important control. Approval is complete only when the organization can explain where the request is processed, what remains, and who is accountable under which legal basis.

Review VPC, IAM, SCP, audit logging, and resilient access paths with Cloud & Infrastructure, and review model selection, guardrails, evaluation, and healthcare AI operations with Data & ML Engineering.

References