Skip to main content

FAQs

General Questions​

Q: What is the difference between Model Repositories and Model Endpoints?

A: Model Repositories are storage systems for model files, while Model Endpoints are running services that serve inference requests. Repositories store models; Endpoints use models from repositories to serve API requests.

Q: Can I use Model Repositories without Model Endpoints?

A: Yes! You can use Model Repositories independently for storage, versioning, and sharing models. They're also useful with Instances.

Q: What file formats are supported?

A: Model Repositories support any file format. There's no restriction - you can store PyTorch models, TensorFlow models, ONNX, TensorRT engines, or any custom format.

Storage Questions​

Q: What storage options are available?

A: Three options:

  1. New EOS Bucket - TIR provisions a new bucket automatically
  2. Existing EOS Bucket - Link to an existing EOS bucket
  3. External EOS Bucket - Connect to external S3-compatible storage

Q: Can I change the storage type after creation?

A: No, storage type cannot be changed after creation. You'll need to create a new repository with the desired storage type.

Q: How do I access files in a Model Repository?

A: Multiple ways:

  • Dashboard: File Browser tab for visual browsing
  • CLI: Minio CLI (mc) or s3cmd
  • SDK: Python SDK or other language SDKs
  • API: S3-compatible REST API

Q: Can I encrypt my models?

A: Yes! EOS encryption is available. Enable encryption when creating a new EOS bucket.

Encryption must be enabled at bucket creation.

Versioning Questions​

Q: How do I version my models?

A: Use folder structures:

  • /v1, /v2 for numbered versions
  • /production, /staging for environments
  • /experiments/exp-001 for experimental versions

Q: Can I delete specific versions?

A: Yes, you can delete specific folders/files using CLI or SDK. Be careful - deletion is permanent.

Q: How many versions should I keep?

A: Best practice: Keep 2-3 previous versions for rollback. Archive or delete older versions to save storage costs.

Integration Questions​

Q: How do I link a Model Repository to a Model Endpoint?

A: You can link a repository in two ways.

  • From Model Repository: Use the Deploy Model option in the repository table; you are taken to the Model Endpoint flow, where you select the framework (e.g., vLLM, SGLang, NVIDIA Triton) and link the repository to the endpoint.

  • From Model Endpoints: Create a Model Endpoint, select Link with Model Repository, choose your repository, and specify the model path (e.g., / or /v1). In both cases, the endpoint automatically downloads model files from the repository during startup.

Troubleshooting​

Q: Upload is slow. How can I speed it up?

A: Try the following:

  • Use parallel uploads: mc cp --parallel
  • Ensure good network connectivity
  • Compress files before upload
  • Use SDK for programmatic uploads (may be faster)

Q: Model Endpoint can't find my model files. What's wrong?

A: Check:

  1. Model path in endpoint configuration matches folder structure
  2. Files are actually uploaded (verify via File Browser)
  3. Bucket name is correct
  4. Check endpoint logs for specific errors