Security — Where the Danger Actually Is

The risks fall into two entirely separate groups: loading a model, where the file itself may execute code on your machine, and running one, where it may be deceived or leak data.

8.1 Load-time risk — model files can run code

Here is the fact most beginners do not know: loading a classic PyTorch model is not like opening an image. That format uses pickle, which is designed to reconstruct Python objects — and in doing so, it can execute code. An attacker can therefore embed commands inside a weights file.

Format	Executes code on load?	Recommendation
safetensors	No — it stores numbers and metadata	Make it the organisation-wide default
.bin / .pt / .pth / .ckpt	Yes, via pickle	Avoid. If unavoidable, load in an isolated environment with no network and no credentials
.h5 / Keras	Possible, via a Lambda layer	Especially careful when loading directly through Keras
GGUF	No, by design	Fairly safe, though reader bugs remain a risk — keep the runtime updated

# ✗ Risky — loading a pickle format with no protection
model = AutoModelForCausalLM.from_pretrained("some-user/mystery-model")

# ✓ Far safer
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-8B",
    use_safetensors=True,     # fail outright rather than quietly load a pickle
    trust_remote_code=False,  # do not run code shipped with the repo
    revision="a1b2c3d4",      # pin the version
)

# If you genuinely must load an old checkpoint
import torch
state = torch.load("old_model.pt", weights_only=True)   # read numbers, build no objects

weights_only=True is the difference between “reading data” and “running someone else’s program”.

8.2 trust_remote_code — the most dangerous parameter in the ecosystem

trust_remote_code=True means: “download the .py files from that repository and run them on my machine, with my user’s privileges.”

It exists for a legitimate reason: new architectures usually arrive before library support does, so developers ship the architecture code inside the repository. The consequence, though, is that a repository anyone can upload to becomes a way to run code on your machine.

The rules worth adopting:

• The default is always False
• Set True only for repositories from verified developers
• If you must enable it for an unfamiliar repository, read every .py file first, and run inside a container with no network and no credentials
• Never use it in an automated system pointed at a repository someone else controls

8.3 Scan before use — and know the limits

uv tool install modelscan
modelscan -p ./models/downloaded-model/

# Or scan the pickle file specifically
uv tool install picklescan
picklescan -p ./models/downloaded-model/pytorch_model.bin

# In CI, before a model enters your internal registry
modelscan -p ./incoming/ --settings-file ./modelscan-settings.toml || exit 1

A scanner is not a guarantee

Scanners read a file sequentially, while the real deserializer executes as it reads. An attacker placing code in the right position can have it run before the scanner reaches the part it understands. This technique has been confirmed in the wild as a way around detection, and multiple scanner vulnerabilities have been reported. Treat scanning as an additional layer, not a certificate — the primary layers are using safetensors and choosing trustworthy sources.

8.4 How to run a model you do not yet trust

docker run --rm -it \
  --gpus all \
  --network none \                 # no network means nothing can be sent out
  --read-only \                    # read-only filesystem
  --tmpfs /tmp:rw,size=2g \        # the only writable place
  -v "$PWD/models:/models:ro" \    # models read-only
  -v "$PWD/out:/out:rw" \
  --user 1000:1000 \               # not root
  --cap-drop ALL \
  --security-opt no-new-privileges \
  --memory 32g --pids-limit 512 \
  pytorch/pytorch:latest \
  python /out/test_model.py

A safe order of operations:

1. Load it in this container and watch whether it tries to reach the network or write unexpected files
2. Scan it with modelscan
3. Test quality against your own eval set
4. Only if all three pass, move it into the internal registry and use it

8.5 Runtime risks

Risk	How it happens	How to reduce it
Prompt injection	A document, web page or email the model reads contains embedded instructions, and the model follows them	Treat everything the model reads as data, never as instructions. Limit the privileges of systems connected to the model, and require human confirmation before anything irreversible
Data leaking through prompts	Customer data sent to a public API with no agreement behind it	Redact personal data before sending, or use a self-hosted model for sensitive material
Secrets in a conversation	Pasting a log or config containing a key so the model can help read it	Redact before pasting, and if it happens, treat the key as disclosed and rotate it immediately
Data leaking through logs	The system records every prompt and answer unredacted	Redact before writing logs, set a retention period, and restrict who can read them
Confidently wrong answers	The model invents facts in a confident voice	Require citations for work that must be accurate, verify with rules or another model, and never give it the final decision on anything important
A silently swapped model	The system falls back to a different model or version and nobody notices	Record the model name and version in every log, and keep a smoke test that catches behavioural change
Single-provider dependence	Prices rise, terms change, or the model is retired	Call through a standard API, keep a tested alternative, and hold an eval set so a replacement can be checked quickly

8.6 Checklist before going live

Choosing and downloading

• From a verified developer account, or otherwise vetted
• Using safetensors or GGUF, not .bin / .pt
• Revision pinned to a commit sha, not main
• trust_remote_code=False, or a recorded reason if it must be on
• Scanned with modelscan
• First load performed in a container with no network
• Licence traced to the base model, not just what the uploader wrote
• sha256 of the files recorded for later comparison

Deploying

• VRAM computed at peak user count, not at one user
• Benchmarked with prompts as long as the real workload
• Passes your own eval set of at least 30 cases
• Called through a standard API so the runtime can be changed
• Model name and version recorded in every log
• A redaction filter runs before anything is logged
• A fallback plan exists for when the model or provider fails
• It is written down which decisions the model may not make alone

What this chapter settles

A model file is not merely data. The pickle formats execute code at load time, and trust_remote_code=True permits a repository to run a program on your machine. Default to safetensors, keep trust_remote_code off, try anything new inside a network-isolated container, and treat everything the model reads as data rather than instruction.

The next chapter is the appendix: vocabulary, everyday commands, and a learning path.