Fabio
c36510c21e
Some checks failed
Linux Build / Linux cpu (push) Has been cancelled
Linux Build / Linux cpu-clang (push) Has been cancelled
Linux Build / Linux vulkan (push) Has been cancelled
mac-build / macOS CPU (push) Has been cancelled
Nix Build / Nix (cpu) (push) Has been cancelled
Nix Build / Nix (vulkan) (push) Has been cancelled
windows-build / Windows CPU (push) Has been cancelled
24 lines
1 KiB
C++
24 lines
1 KiB
C++
#pragma once
|
|
|
|
#include "config.h"
|
|
|
|
#include <cstddef>
|
|
#include <optional>
|
|
|
|
namespace minitts::server {
|
|
|
|
// Estimated resident bytes a model will occupy once loaded (weights plus a
|
|
// runtime overhead factor for GPU buffers / compute graphs). A family that
|
|
// declares auxiliary resources reports their paths and final resident bytes
|
|
// through its loader. The server excludes those paths from weight scans and adds
|
|
// the reported bytes without interpreting execution plans or residency policies.
|
|
// Returns nullopt
|
|
// when the footprint is indeterminate: a model directory holding several GGUFs
|
|
// and no model.gguf. The guard must not refuse such a load with a 503 -- the
|
|
// loader either rejects the ambiguous directory itself (the spec-driven path
|
|
// fails with its "contains N GGUF files" error) or accepts it under a
|
|
// family-specific layout the estimator cannot resolve, so the guard skips
|
|
// rather than guess.
|
|
std::optional<size_t> estimate_model_memory_bytes(const ServerModelConfig & model);
|
|
|
|
} // namespace minitts::server
|