Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Offline EXO Deployment
- Utilize EXO_OFFLINE to prevent runtime internet access.
- Pre-load models into EXO_MODELS_READ_ONLY_DIRS from trusted internal mirrors.
- Verify model weight integrity using SHA-256 checksums and signed model cards.
- Execute EXO within air-gapped networks, removing dependencies on HuggingFace.
Dashboard and API Access Control
- Install and configure reverse proxies (nginx, Caddy) with TLS termination.
- Implement role-based access control for the EXO dashboard and REST API.
- Store API authentication secrets using the macOS keychain or Linux pass.
- Restrict access to administrative endpoints by specific source IP ranges.
Cluster Isolation and Network Security
- Segment EXO clusters using EXO_LIBP2P_NAMESPACE and VLANs.
- Configure host firewalls (macOS application firewall, iptables, nftables) for EXO ports.
- Prevent unauthorized device discovery and the injection of rogue nodes.
- Encrypt libp2p traffic between nodes in scenarios where RDMA is unavailable.
Model Governance and Provenance
- Establish an internal model registry containing approved model lists and metadata.
- Tag and version quantized weights (4-bit, 8-bit) alongside source checkpoints.
- Enforce restrictions so that only specific HuggingFace repositories or internal artifacts can be loaded.
- Document model lineage, license terms, and acceptable use policies.
Audit Logging and Compliance
- Configure EXO log forwarding to immutable audit trails (SIEM, WORM storage).
- Correlate API call logs with user identity and timestamps.
- Capture events related to model instance creation, deletion, and inference requests.
- Generate periodic compliance reports for internal and external auditors.
Threat Modeling and Incident Response
- Identify threats such as data exfiltration via model outputs, prompt injection, and side-channel leaks.
- Implement prompt monitoring and content filtering pipelines.
- Develop incident response runbooks for cluster compromise scenarios.
- Isolate affected nodes, preserve forensic logs, and reconstruct clean environments.
Physical Security and Hardware Boundaries
- Secure Thunderbolt ports to prevent unauthorized RDMA cable connections.
- Leverage secure enclaves and Apple Silicon hardware attestation where applicable.
- Control physical access to clustered Macs and shared storage systems.
- Document hardware lifecycle and decommissioning procedures.
Regulatory Considerations
- Map EXO deployments to GDPR, HIPAA, and SOC 2 requirements.
- Maintain data residency by keeping inference processes on-premise.
- Document vendor supply-chain risks associated with MLX, EXO, and model weights.
- Prepare for AI governance frameworks, including EU AI Act Article 53.
Requirements
- Hands-on experience with EXO or another local LLM runtime
- Comprehensive understanding of Unix filesystem permissions and networking ACLs
- Familiarity with TLS/SSL certificate management and fundamental encryption concepts
Target Audience
- Security engineers
- Compliance officers
- AI infrastructure administrators responsible for sensitive data
Testimonials (1)
The trainer had an excellent knowledge of fortigate and delivered the content very well. Thanks a lot to Soroush.
