mirror of
https://github.com/we-promise/sure.git
synced 2026-04-14 09:34:05 +00:00
88952e4714
* Initial implementation * FIX keys * Add langfuse evals support * FIX trace upload * Delete .claude/settings.local.json Signed-off-by: soky srm <sokysrm@gmail.com> * Update client.rb * Small LLMs improvements * Keep batch size normal * Update categorizer * FIX json mode * Add reasonable alternative to matching * FIX thinking blocks for llms * Implement json mode support with AUTO mode * Make auto default for everyone * FIX linter * Address review * Allow export manual categories * FIX user export * FIX oneshot example pollution * Update categorization_golden_v1.yml * Update categorization_golden_v1.yml * Trim to 100 items * Update auto_categorizer.rb * FIX for auto retry in auto mode * Separate the Eval Logic from the Auto-Categorizer The expected_null_count parameter conflates eval-specific logic with production categorization logic. * Force json mode on evals * Introduce a more mixed dataset 150 items, performance from a local model: By Difficulty: easy: 93.22% accuracy (55/59) medium: 93.33% accuracy (42/45) hard: 92.86% accuracy (26/28) edge_case: 100.0% accuracy (18/18) * Improve datasets Remove Data leakage from prompts * Create eval runs as "pending" --------- Signed-off-by: soky srm <sokysrm@gmail.com> Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
67 lines
2.9 KiB
Plaintext
67 lines
2.9 KiB
Plaintext
<div class="space-y-4">
|
|
<div>
|
|
<h2 class="font-medium mb-1"><%= t(".title") %></h2>
|
|
<% if ENV["OPENAI_ACCESS_TOKEN"].present? %>
|
|
<p class="text-sm text-secondary"><%= t(".env_configured_message") %></p>
|
|
<% else %>
|
|
<p class="text-secondary text-sm mb-4"><%= t(".description") %></p>
|
|
<% end %>
|
|
</div>
|
|
|
|
<%= styled_form_with model: Setting.new,
|
|
url: settings_hosting_path,
|
|
method: :patch,
|
|
data: {
|
|
controller: "auto-submit-form",
|
|
"auto-submit-form-trigger-event-value": "blur"
|
|
} do |form| %>
|
|
<%= form.password_field :openai_access_token,
|
|
label: t(".access_token_label"),
|
|
placeholder: t(".access_token_placeholder"),
|
|
value: (Setting.openai_access_token.present? ? "********" : nil),
|
|
autocomplete: "off",
|
|
autocapitalize: "none",
|
|
spellcheck: "false",
|
|
inputmode: "text",
|
|
disabled: ENV["OPENAI_ACCESS_TOKEN"].present?,
|
|
data: { "auto-submit-form-target": "auto" } %>
|
|
|
|
<%= form.text_field :openai_uri_base,
|
|
label: t(".uri_base_label"),
|
|
placeholder: t(".uri_base_placeholder"),
|
|
value: Setting.openai_uri_base,
|
|
autocomplete: "off",
|
|
autocapitalize: "none",
|
|
spellcheck: "false",
|
|
inputmode: "url",
|
|
disabled: ENV["OPENAI_URI_BASE"].present?,
|
|
data: { "auto-submit-form-target": "auto" } %>
|
|
|
|
<%= form.text_field :openai_model,
|
|
label: t(".model_label"),
|
|
placeholder: t(".model_placeholder"),
|
|
value: Setting.openai_model,
|
|
autocomplete: "off",
|
|
autocapitalize: "none",
|
|
spellcheck: "false",
|
|
inputmode: "text",
|
|
disabled: ENV["OPENAI_MODEL"].present?,
|
|
data: { "auto-submit-form-target": "auto" } %>
|
|
|
|
<%= form.select :openai_json_mode,
|
|
options_for_select(
|
|
[
|
|
[t(".json_mode_auto"), ""],
|
|
[t(".json_mode_strict"), "strict"],
|
|
[t(".json_mode_none"), "none"],
|
|
[t(".json_mode_json_object"), "json_object"]
|
|
],
|
|
Setting.openai_json_mode
|
|
),
|
|
{ label: t(".json_mode_label") },
|
|
{ disabled: ENV["LLM_JSON_MODE"].present?,
|
|
data: { "auto-submit-form-target": "auto" } } %>
|
|
<p class="text-xs text-secondary mt-1"><%= t(".json_mode_help") %></p>
|
|
<% end %>
|
|
</div>
|