Skip to content

llama-stack-client models list failed #313

@gyliu513

Description

@gyliu513
(llama-stack) (base) gualiu@gualiu-mac llama-stack % llama-stack-client --version
llama-stack-client, version 0.4.2
(llama-stack) (base) gualiu@gualiu-mac llama-stack % llama-stack-client models list
INFO:httpx:HTTP Request: GET http://localhost:8321/v1/models "HTTP/1.1 200 OK"
╭───────────────────────────────────────────────────────╮
│ Failed to list models                                 │
│                                                       │
│ Error Type: AttributeError                            │
│ Details: 'Model' object has no attribute 'model_type' │
╰───────────────────────────────────────────────────────╯
(llama-stack) (base) gualiu@gualiu-mac llama-stack % curl http://localhost:8321/v1/models | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   920  100   920    0     0   486k      0 --:--:-- --:--:-- --:--:--  898k
{
  "data": [
    {
      "id": "ollama/llama3.2-vision:latest",
      "object": "model",
      "created": 1769033235,
      "owned_by": "llama_stack",
      "custom_metadata": {
        "model_type": "llm",
        "provider_id": "ollama",
        "provider_resource_id": "llama3.2-vision:latest"
      }
    },
    {
      "id": "ollama/llama3.2:3b-instruct-fp16",
      "object": "model",
      "created": 1769033235,
      "owned_by": "llama_stack",
      "custom_metadata": {
        "model_type": "llm",
        "provider_id": "ollama",
        "provider_resource_id": "llama3.2:3b-instruct-fp16"
      }
    },
    {
      "id": "ollama/llama3.2:3b",
      "object": "model",
      "created": 1769033235,
      "owned_by": "llama_stack",
      "custom_metadata": {
        "model_type": "llm",
        "provider_id": "ollama",
        "provider_resource_id": "llama3.2:3b"
      }
    },
    {
      "id": "sentence-transformers/nomic-ai/nomic-embed-text-v1.5",
      "object": "model",
      "created": 1769033235,
      "owned_by": "llama_stack",
      "custom_metadata": {
        "model_type": "embedding",
        "provider_id": "sentence-transformers",
        "provider_resource_id": "nomic-ai/nomic-embed-text-v1.5",
        "embedding_dimension": 768
      }
    }
  ]
}

Root Cause: python client is using item.xx https://github.com/llamastack/llama-stack-client-python/blob/main/src/llama_stack_client/lib/cli/models/models.py#L54-L59 to get those values, but actually those values are in item.metadata, this only impact CLI but not models.list() API, so all integration test still works as they are not using llama-stack-client CLI but API call.

@cdoern ^^

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions