Files
Eric BuehlerandGitHub aef2c9aa5f Audio input support (Phi 4 multimodal) (#1448)
* Deps

* Add conformer

* Nemo loading

* Position embeds

* Load t5 attn bias

* Attn and feed forward

* Add conv module and glu pointwise

* Implement relative attn bias

* Add the forward methods

* Add encoder embedding

* Fix oproj

* Some loading

* Conformer loads!

* Fully loading speech stack

* Merger

* Dont need that

* First pass at audio processing

* Read samples

* Optional

* Small loading fix

* Runs but not correct yet

* Improved audio processing?

* Works with this

* Fix t5 attn bias

* It works!

* Comment

* Use some other crates

* Clippy

* Allow bf16 on metal

* Add prefix_audio

* Remove unused

* Typo

* User specified

* Add audio url parsing

* AudioProjectionMode -> InputMode

* Audio prefix caching

* Fix bug in audio prefix caching

* Support both at the same time!

* Tweak logging

* Support stereo

* Add mistralrs-audio

* Support batching

* Add server and rust api example

* Add python api

* Fix add_multimodal_message

* Fix unfold for conformer

* Streaming example

* Add web chat support

* Add modalities registry
2025-06-09 09:53:41 -04:00

79 lines
2.7 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Mistral.rs Chat</title>
<link rel="stylesheet" href="styles.css">
</head>
<body>
<div id="sidebar">
<h2>Control panel</h2>
<select id="modelSelect"></select>
<button id="newChatBtn">➕ New chat</button>
<button id="clearBtn">🧹 Clear chat</button>
<button id="renameBtn">✏️ Rename chat</button>
<button id="deleteBtn">🗑️ Delete chat</button>
<hr />
<div class="sidebar-card">
<h3>Web Search</h3>
<div class="sidebar-card-content">
<label class="toggle-label"><input type="checkbox" id="enableSearch" /> Enable web search</label>
<div id="webSearchOptions" hidden>
<label for="searchContextSize">Context size:</label>
<select id="searchContextSize">
<option value="low">Low</option>
<option value="medium" selected>Medium</option>
<option value="high">High</option>
</select>
</div>
</div>
</div>
<ul id="chatList"></ul>
</div>
<div id="main">
<h1 style="margin-top:0;">Mistral.rs Chat</h1>
<div id="log"></div>
<form id="form">
<textarea
id="input"
placeholder="Type your message… (Press Ctrl+Enter to send)"
></textarea>
<!-- hidden file inputs -->
<input type="file" id="imageInput" accept="image/*" hidden />
<input type="file" id="textInput" accept=".txt,.md,.markdown,.py,.js,.ts,.rs,.html,.css,.json,.xml,.yaml,.yml,.toml,.c,.cpp,.java,.go,.rb,.php,.sh,.sql,.log,.csv" hidden />
<!-- audio input (wav/mp3/ogg/… ) -->
<input type="file" id="audioInput" accept="audio/*" hidden />
<!-- file upload buttons and send button share the same class -->
<label for="imageInput" class="btn" id="imageLabel">📎 Image</label>
<label for="audioInput" class="btn" id="audioLabel">🎤 Audio</label>
<label for="textInput" class="btn" id="textLabel">📄 File</label>
<button type="submit" class="btn">Send</button>
<button type="button" class="btn" id="stopBtn">⏹️ Stop</button>
<div id="spinner" class="spinner hidden" aria-hidden="true"></div>
</form>
<div id="image-container"></div>
<div id="audio-container"></div>
<div id="text-files-container"></div>
<!-- External dependencies -->
<script src="marked.min.js"></script>
<!-- Application scripts -->
<script src="js/utils.js"></script>
<script src="js/chat.js"></script>
<script src="js/models.js"></script>
<script src="js/websocket.js"></script>
<script src="js/ui.js"></script>
<script src="js/app.js"></script>
</div><!-- /#main -->
</body>
</html>