Multi-Interface Interaction Tutorial

This guide provides comprehensive instructions for interacting with hosted models via direct API calls, SDKs, and graphical interfaces.

Table of Contents

1. Environment Configuration
2. Service Discovery & Health
3. REST API Interaction (curl)
4. SDK Integrations
   • Anthropic SDK
   • OpenAI SDK
5. Graphical Interfaces
6. IDE & Agent Integrations
7. Advanced Features
8. Service Capability Matrix

1. Environment Configuration

To streamline the following examples, configure your terminal environment with your API base URL and access token.

# Set your API base URL (e.g., http://<ip>:8080/v1)
export LLM_API_BASE="http://localhost:8080/v1"

# Set your personal access token (provided by admin)
export LLM_API_TOKEN="your-token-here"

2. Service Discovery & Health

Before initiating requests, verify the operational status of the gateways and discover available model endpoints.

2.1 Health Verification

Confirm the responsiveness of both the public LLM and private Admin gateways.

# Verify LLM Gateway (Public: 8080)
curl ${LLM_API_BASE%/v1}/health

# Verify Admin Gateway (Private: 8081)
curl $(echo $LLM_API_BASE | sed 's/:8080/:8081/')/health

2.2 Model Catalog

Retrieve a list of active models currently served by the platform.

curl $LLM_API_BASE/models \
  -H "X-API-Token: $LLM_API_TOKEN"

Note: This endpoint is served on the public port (8080) and requires authentication.

3. REST API Interaction (curl)

The gateway provides full OpenAI-compatible REST endpoints.

Chat Completions

curl $LLM_API_BASE/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Token: $LLM_API_TOKEN" \
  -d '{
    "model": "bg-digitalservices/Gemma-4-E4B-it-NVFP4",
    "messages": [{"role": "user", "content": "Explain Blackwell architecture in one sentence."}],
    "temperature": 0.7
  }'

4. SDK Integrations

The platform supports native integration with leading LLM SDKs through its dual-protocol support.

4.1 Anthropic SDK

The gateway proxies Anthropic-compatible requests, allowing for seamless use of Claude-optimized tools.

import os
from anthropic import Anthropic

client = Anthropic(
    base_url=os.environ.get("LLM_API_BASE"),
    api_key=os.environ.get("LLM_API_TOKEN")
)

message = client.messages.create(
    model="bg-digitalservices/Gemma-4-E4B-it-NVFP4",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Hello, how are you?"}
    ]
)
print(message.content[0].text)

4.2 OpenAI SDK

Standard integration for tools built on the OpenAI specification.

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ.get("LLM_API_BASE"),
    api_key=os.environ.get("LLM_API_TOKEN")
)

response = client.chat.completions.create(
    model="bg-digitalservices/Gemma-4-E4B-it-NVFP4",
    messages=[{"role": "user", "content": "How many parameters does Gemma 4 E4B have?"}],
    stream=True
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Observation: Performance and energy metrics are available in the response object under the x_vllm_metrics field.

5. Graphical Interfaces

Open WebUI

Open WebUI is the recommended interface for general interaction and "Model" (Gem) creation.

Configuration:
Set the following environment variables in your Open WebUI deployment:

OPENAI_API_BASE_URL="$LLM_API_BASE"
OPENAI_API_KEY="$LLM_API_TOKEN"

Key Features:

• Automated Discovery: Models are automatically populated from the catalog.
• Streaming Support: Real-time token generation is supported natively.
• Artifact Generation: Use WebUI "Models" to create reproducible, system-prompted environments.

6. IDE & Agent Integrations

The gateway can be integrated into popular development environments.

VSCode (Continue)

Add the following configuration to your config.json:

{
  "title": "Local Gemma 4 E4B",
  "model": "bg-digitalservices/Gemma-4-E4B-it-NVFP4",
  "apiBase": "$LLM_API_BASE",
  "apiKey": "$LLM_API_TOKEN",
  "provider": "openai"
}

VSCode (Cline / Roo Code)

1. Select OpenAI Compatible as the provider.
2. Set Base URL to the value of $LLM_API_BASE.
3. Set API Key to the value of $LLM_API_TOKEN.

CLI Agent (Aider)

export OPENAI_API_BASE="$LLM_API_BASE"
export OPENAI_API_KEY="$LLM_API_TOKEN"
aider --model openai/bg-digitalservices/Gemma-4-E4B-it-NVFP4

7. Advanced Features

Reproducible Artifacts

To replicate "Gemini Gem" behavior (system instructions + context + reproducible output), utilize Open WebUI Models:

1. Navigate to Workspace > Models.
2. Create a new model with bg-digitalservices/Gemma-4-E4B-it-NVFP4 as the base.
3. Define strict system instructions for specialized tasks (e.g., "Always output valid C++17 code").

8. Service Capability Status