Skip to main content

FAQs

General Questions​

Q: What is the difference between Model Repositories and Model Endpoints?

A: Model Repositories are storage systems for model files, while Model Endpoints are running services that serve inference requests. Repositories store models; Endpoints use models from repositories to serve API requests.

Q: Can I use Model Repositories without Model Endpoints?

A: Yes! You can use Model Repositories independently for storage, versioning, and sharing models. They're also useful with Instances.

Q: What file formats are supported?

A: Model Repositories support any file format. There's no restriction - you can store PyTorch models, TensorFlow models, ONNX, TensorRT engines, or any custom format.

Storage Questions​

Q: What storage options are available?

A: Three options:

  1. New EOS Bucket - TIR provisions a new bucket automatically
  2. Existing EOS Bucket - Link to an existing EOS bucket
  3. External EOS Bucket - Connect to external S3-compatible storage

Q: Can I change the storage type after creation?

A: No, storage type cannot be changed after creation. You'll need to create a new repository with the desired storage type.

Q: How do I access files in a Model Repository?

A: Multiple ways:

  • Dashboard: File Browser tab for visual browsing
  • CLI: Minio CLI (mc) or s3cmd
  • SDK: Python SDK or other language SDKs
  • API: S3-compatible REST API

Q: Can I encrypt my models?

A: Yes! EOS encryption is available. Enable encryption when creating a new EOS bucket.

Encryption must be enabled at bucket creation.

Versioning Questions​

Q: How do I version my models?

A: Use folder structures:

  • /v1, /v2 for numbered versions
  • /production, /staging for environments
  • /experiments/exp-001 for experimental versions

Q: Can I delete specific versions?

A: Yes, you can delete specific folders/files using CLI or SDK. Be careful - deletion is permanent.

Q: How many versions should I keep?

A: Best practice: Keep 2-3 previous versions for rollback. Archive or delete older versions to save storage costs.

Integration Questions​

Q: How do I link a Model Repository to a Model Endpoint?

A: You can link a repository in two ways.

  • From Model Repository: Use the Deploy Model option in the repository table; you are taken to the Model Endpoint flow, where you select the framework (e.g., vLLM, SGLang, NVIDIA Triton) and link the repository to the endpoint.

  • From Model Endpoints: Create a Model Endpoint, select Link with Model Repository, choose your repository, and specify the model path (e.g., / or /v1). In both cases, the endpoint automatically downloads model files from the repository during startup.

Troubleshooting​

Q: Upload is slow. How can I speed it up?

A: Try the following:

  • Use parallel uploads: mc cp --parallel
  • Ensure good network connectivity
  • Compress files before upload
  • Use SDK for programmatic uploads (may be faster)

Q: Model Endpoint can't find my model files. What's wrong?

A: Check:

  1. Model path in endpoint configuration matches folder structure
  2. Files are actually uploaded (verify via File Browser)
  3. Bucket name is correct
  4. Check endpoint logs for specific errors
Last updated on May 15, 2026.