Implement ModelMesh support in ISVC handling and shift to python Black formatting #129
Conversation
Signed-off-by: Harshit Nayan <[email protected]>
Signed-off-by: Harshit Nayan <[email protected]>
backend/apps/common/utils.py
Outdated
| # Try to find ModelMesh deployment (usually named modelmesh-serving) | ||
| try: | ||
| modelmesh_deployment = api.get_custom_rsrc( | ||
| **versions.K8S_DEPLOYMENT, | ||
| namespace=namespace, | ||
| name="modelmesh-serving" | ||
| ) |
There was a problem hiding this comment.
Are you sure it's safe to assume this deployment is always named modelmesh-serving?
There was a problem hiding this comment.
You're correct to point it out. I did some digging and modelmesh-serving is named as per the template at: https://github.com/kserve/modelmesh-serving/blob/05994465603e8baa3d23bae126a085f5f8bb591f/config/internal/base/deployment.yaml.tmpl#L17
name: {{.ServiceName}}-{{.Name}}
labels:
modelmesh-service: {{.ServiceName}}
…sh resources using label selectors Signed-off-by: Harshit Nayan <[email protected]>
Signed-off-by: Harshit Nayan <[email protected]>
…and efficiency Signed-off-by: Harshit Nayan <[email protected]>
backend/apps/common/utils.py
Outdated
| # Fallback: infer from model format (e.g., "sklearn" -> "mlserver-sklearn") | ||
| if ( | ||
| model_format := component_spec.get("model", {}) | ||
| .get("modelFormat", {}) | ||
| .get("name") | ||
| ): | ||
| return f"mlserver-{model_format.lower()}" |
There was a problem hiding this comment.
I'm not sure you really need this. If an inference service doesn't specify the runtime it'll never work 🤷
There was a problem hiding this comment.
Yup removed this part and it works just fine.
backend/apps/common/utils.py
Outdated
| """ | ||
| default_name = "modelmesh-serving" | ||
| # A configurable environment variable is more practical than discovery | ||
| service_name = os.environ.get("MODELMESH_SERVICE_NAME", default_name) |
There was a problem hiding this comment.
so the user is supposed to set this environment variable? Is there no other way to get the modelmesh service? It's probably ok to assume the user didn't change the name of the modelmesh service IMO
There was a problem hiding this comment.
the default namespace for modelmesh is set here: https://github.com/kserve/modelmesh-serving/blob/05994465603e8baa3d23bae126a085f5f8bb591f/docs/quickstart.md?plain=1#L37
There was a problem hiding this comment.
I'll remove the env part then.
…e retrieval Signed-off-by: Harshit Nayan <[email protected]>
LogicalGuy77
changed the title
Fixes: #97
Summary:
The changes allow users to view ModelMesh-specific resources and status for each component of an InferenceService.
Testing