Skip to content

webui: add server info to chat message #14065

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
+45 −1
Open

webui: add server info to chat message #14065

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f494884
webui: add server info to chat message
am17an Jun 8, 2025
15fb2cf
create helper function modelNameFromPath
am17an Jun 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions tools/server/webui/src/Config.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ export const CONFIG_DEFAULT = {
apiKey: '',
systemMessage: '',
showTokensPerSecond: false,
showServerInformation: false,
showThoughtInProgress: false,
excludeThoughtOnReq: true,
pasteLongTextToFileLen: 2500,
Expand Down
25 changes: 24 additions & 1 deletion tools/server/webui/src/components/ChatMessage.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { useMemo, useState } from 'react';
import { useAppContext } from '../utils/app.context';
import { Message, PendingMessage } from '../utils/types';
import { classNames } from '../utils/misc';
import { classNames, modelNameFromPath } from '../utils/misc';
import MarkdownDisplay, { CopyButton } from './MarkdownDisplay';
import {
ArrowPathIcon,
Expand Down Expand Up @@ -189,6 +189,29 @@ export default function ChatMessage({
</div>
</div>
)}

{/* render model information if enabled and available */}
{config.showServerInformation && msg.serverProps && (
<>
<br />
<div className="dropdown dropdown-hover dropdown-top">
<div
tabIndex={0}
role="button"
className="cursor-pointer font-semibold text-sm opacity-60"
>
Model:{' '}
{msg.serverProps.model_path && modelNameFromPath(msg.serverProps.model_path)}
</div>
<div className="dropdown-content bg-base-100 z-10 w-80 p-2 shadow mt-4">
<b>Server Information</b>
<br />- Path: {msg.serverProps.model_path}
<br />- Build: {msg.serverProps.build_info}
<br />- Context: {msg.serverProps.n_ctx}
Comment on lines +208 to +210
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use <ul> <li> instead

Copy link
Contributor Author

@am17an am17an Jun 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did it this way to maintain homogeneity with the other dropdown. Let me if we should still use <ul> <li> here

<div className="dropdown-content bg-base-100 z-10 w-64 p-2 shadow mt-4">
                    <b>Prompt</b>
                    <br />- Tokens: {timings.prompt_n}
                    <br />- Time: {timings.prompt_ms} ms
                    <br />- Speed: {timings.prompt_per_second.toFixed(1)} t/s
                    <br />
                    <b>Generation</b>
                    <br />- Tokens: {timings.predicted_n}
                    <br />- Time: {timings.predicted_ms} ms
                    <br />- Speed: {timings.predicted_per_second.toFixed(1)} t/s
                    <br />
                  </div>

</div>
</div>
</>
)}
</>
)}
</div>
Expand Down
5 changes: 5 additions & 0 deletions tools/server/webui/src/components/SettingDialog.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,11 @@ const SETTING_SECTIONS: SettingSection[] = [
label: 'Show tokens per second',
key: 'showTokensPerSecond',
},
{
type: SettingInputType.CHECKBOX,
label: 'Show server information',
key: 'showServerInformation',
},
{
type: SettingInputType.LONG_INPUT,
label: (
Expand Down
7 changes: 7 additions & 0 deletions tools/server/webui/src/utils/app.context.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,13 @@ export const AppContextProvider = ({
content: null,
parent: leafNodeId,
children: [],
serverProps: serverProps
? {
model_path: serverProps.model_path,
build_info: serverProps.build_info,
n_ctx: serverProps.n_ctx,
}
: null,
Comment on lines +191 to +197
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite fragile for 2 reasons:

  • If user changes model but forget to reload the frontend, this serverProps object will be out of sync
  • This design is not future-proof. If we decide to add a new key into serverProps in the future, it will be quite messy. The serverProps object is not supposed to be saved an reuse later

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ngxson thanks for your review!
getServerProps is only called once on mount, so at the moment there is no way to get these in sync in case the backend changes but the frontend has not reloaded. One imprecise solution is to periodically poll the backend /health API and reload once the server comes back online. Let me know if that works
In case of future proofing the design, do you wish to de-couple these things? I think it might still be good to use the serverProps object to read these values but store them as separate types in the message.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One imprecise solution is to periodically poll the backend /health API and reload once the server comes back online

We can simply call /props each N seconds to get the up-to-date server props. But this requires some thinking about the UX

};
setPending(convId, pendingMsg);

Expand Down
7 changes: 7 additions & 0 deletions tools/server/webui/src/utils/misc.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,3 +195,10 @@ export const getServerProps = async (
throw error;
}
};

export const modelNameFromPath = (path: string): string => {
// get the last non-empty part
const trimmed = path.replace(/[\\\/]+$/, '');
const parts = trimmed.split(/[\\\/]/);
return parts[parts.length - 1] || path;
};
1 change: 1 addition & 0 deletions tools/server/webui/src/utils/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ export interface Message {
content: string;
timings?: TimingReport;
extra?: MessageExtra[];
serverProps?: LlamaCppServerProps | null;
// node based system for branching
parent: Message['id'];
children: Message['id'][];
Expand Down